Here is a COG identified through the Problem BrainStorm needing support
Individuals that provide facts will earn points if a COG is validated by Steering Team.
COG: The current feedback system is inconsistent, subjective, immeasurable, does not provide a mechanism to accurately translate assessments into the OES, and has a poor culture of accountability.
PROVIDE EVIDENCE
With many fortune 500 companies using survey tools to provide members 360 feedback, success is proven in their figures. These tools are organized into a survey like feedback. The metrics and grading criteria is set for the corporation to meet the corporate vision and mission. These tools provide effective feedback. The employees are graded against a set standard. It easily allows managers to know where their employees rank. It fosters the feedback session and is a tool to help managers give and get feedback.
ReplyDelete(https://primalogik.com/) (http://360Fdbk.com) (http://www.assesshub.com)
Interesting COA but we are looking for facts to support this problem statement. Can you offer any?
DeleteThis comment has been removed by the author.
ReplyDeleteOES should contain both quantitative and qualitative data. By utilizing metrics (quantitative) we can highlight the deficits but data alone will not determine why or how to improve; According to the HR Management Consultants Group, “it is the human element that must come first and the quantitative elements are there as back-up. Metrics are a highly useful aid to effective personal judgement and discretion, but they are not a replacement.” According to the group, feedback should be on-going and not annual or semi-annual discussion. The following topics were suggest:
ReplyDelete* discuss and agree on what organizational goals have been accomplished, the future goals and the upcoming priorities.
* discuss and agree on future training.
*discuss and agree on what changes would improve their performance.
* obtain and discuss feedback from subordinates, peers and internal and external customers.
* discuss career options.
Feedback/evaluation should be a supportive approach. It should foster an ongoing learning environment where employees look forward to the feedback as an opportunity to learn, improve and gain the overall competencies. Qualitative or Quantitative data alone will not provide this opportunity. There must be both to give an adequate mentoring session.
HR Group Management Support Group, Http://hrgconsulting.com
While I like your proposal of a hybrid between objective and subjective evaluations... we need to focus on identifying facts that show us that we are suffering from ineffective feedback which reduces the effectiveness of the officer evaluation system. I am looking forward to hearing your COA in the next stage.
DeleteAbout higher education students, but directly applicable:
ReplyDelete"Giving students detailed feedback about the strengths and weaknesses of their work, with suggestions for improvement, is becoming common practice in higher education. However, for many students, feedback seems to have little or no impact, despite the considerable time and effort put into its production. With a view to increasing its effectiveness, extensive theoretical and empirical research has been carried out into its structure, timing and other parameters. For students to be able to apply feedback, they need to understand the meaning of the feedback statements. They also need to identify, with near certainty, the particular aspects of their work that need attention. For these to occur, students must possess critical background knowledge. This article sets out the nature of that knowledge and how students can acquire it. They must appropriate for themselves three fundamental concepts – task compliance, quality and criteria – and also develop a cache of relevant tacit knowledge."
Sadler, D. Royce. "Beyond feedback: Developing student capability in complex appraisal." Assessment & Evaluation in Higher Education 35.5 (2010): 535-550.
Marcus, interesting article. Can you show that AF officers do not under task compliance, quality, and criteria..." i am not sure what "criteria" means but you have the groundwork here for some solid facts that may prove that we do not understand the three fundamental concepts. By that logic, if we do not understand then our feedback system would be considered ineffective...
ReplyDeleteCan you show that we don't understand?
Feedback I do believe is important, it is currently just mishandled and know one knows how to do it. Far too often when people first hear feedback they get tense and feel like oh no here comes a brutal talking too. This should be further from the truth. In an article in the entrepreneur, Scott Halford, says we need to use a 5 Step process.
ReplyDelete1. Create Safety, ensure your Airman, know they are not in trouble and they can feel comfortable by you.
2. Be positive, Negative feedback while important, if there is too much, can have drastic consequences. "Give at least as much positive feedback as you do negative. Positive feedback stimulates the reward centers in the brain, leaving the recipient open to taking new direction."
3. Be Specific, Say something specific and positive pointed at the task you want accomplished, such as, "You're smart. I want to hear at least one opinion from you in every meeting we're in together going forward." To be able to do this however, a Leader must truly know their Airman.
4. Be Immediate, I see this as you can never give enough feedback. initial, midterm, isn't enough. Its hard to be like "you remember X months ago when this happened, well that was not right needed to be like this." This should happen within days of something significant happening. While it is still fresh, and the Airman is still receptive to what they have done. It is okay to fail, we just have to be immediate when it happens, to fix the problem. This will help ensure the Airman, does not think what they did was good or the right way.
5. Be tough, not mean, We need to be stern about what we want them to do or aspire to. But we can not be mean or demeaning, as this can surely lead to less productivity. I like to think of this as the disappointed talk from you dad. That is always harder to hear than the being yelled at. No one wants to know they let someone down. This usually leads to people working harder to prove they are good at what they do.
https://www.entrepreneur.com/article/219437
A look at the private sector will also glean some comparable conclusions. Deloitte recently completely overhauled their performance feedback system after finding that their feedback was cumbersome and ineffective. They were able to identify the enormous amount of man hours wasted (to Deloitte, time LITERALLY is money) and substantially able to increase effectiveness of their metrics and ability to evaluate employees. The big change: "We ask leaders what they’d do with their team members, not what they think of them"
ReplyDeleteThey also found a strong correlation in improved performance with frequent feedback - weekly. This goes to show that initial, midterm, and annual feedback should be absolutely required and regulated.
"Reinventing Performance Management" by Marcus Buckingham and Ashley Goodall, Harvard Business Review
https://hbr.org/2015/04/reinventing-performance-management
I read similar Air Force OES issues in an Army article by Timothy Reese (2002), titled " Transforming the Officer Evaluation System : using a 360-degree feedback model." He argued that the Army, as a COA, should look towards a 360-degree feedback model, similar to what Thomas Meyer had stated in the first comment of the blog. He referenced the fact that, "Army uses peer feedback to train its leaders before commissioning. The Reserve Officer Training Program and the United States Military Academy, using different systems, both use peer feedback to develop prospective officers. Why should we assume that this kind of feedback would not be useful to an officer as he or she develops their leadership ability over time?" (Reese, 2002, pg 7). Similar to the Air Force, peer reviews are used all throughout commissioning sources and PME, like ASBC and here now in SOS, but are never exposed to it in our career fields. Ineffective feedback that is only subjective by a rater, rather than from peers and subordinates, provides a one-sided viewpoint of an officer's actual merit as a skilled leader.
ReplyDelete2002;Timothy R. Reese;Army War College (U.S.)
The OES is subjective in nature, and it is easy to get stuck in task mode and track every little thing to carefully manage performance (i.e. micromanagement). According to Liz Ryan, leader of the Human Workplace movement: “Performance Management is the name of a popular HR hoax and scam that turns any job into a series of tasks and goals that you’ll be held accountable for on a daily, weekly and monthly basis. No job worth doing breaks down into tiny, measurable parts.
ReplyDeleteGood jobs are whole. You know what your mission is and you work toward your mission every day, checking in with your manager as appropriate. Run away from any company that surrounds you with yardsticks and measurements.”
This is what the AF does, and we must change that.
http://www.forbes.com/sites/lizryan/2016/03/07/ten-unmistakable-signs-of-a-bad-place-to-work/2/#1fc237d2675c
Feedback is often delayed since supervisors/commanders wait to put all their observations on the Airmen Comprehensive Assessment form, and timely feedback is rare. It may happen more quickly when the supervisor or commander sees a perceived problem that they feel a need to address or fix. Amanda Augustine of TheLadders gives this advice regarding feedback: “Feedback is best given shortly after you’ve observed the behavior or event. Do not wait a month after a bad incident to broach the subject… properly prepare so you can provide solid, actionable feedback.”
http://www.itbusinessedge.com/slideshows/six-tips-for-giving-feedback-in-the-workplace-06.html
Gen Welsh and Chief Cody rolled out an improved feedback system back in 2014.
ReplyDeletehttp://www.af.mil/News/ArticleDisplay/tabid/223/Article/485347/af-releases-new-feedback-forms.aspx
Quotes:
"We must get this right. Proper feedback is the most important element of a strong evaluation
system. It is the only way we can cultivate a culture that drives performance. Airmen must
know what we expect of them. We owe them direction and guidance so they can reach their
fullest potential and capitalize on opportunities. If we fail at feedback, we fail our Airmen." Gen Welsh
“We all need feedback, and we need it often … it enables us to accomplish great things for our nation, as individuals and as a team,” Cody said. “It’s one of the most powerful influences on learning and achievement … it is essential to winning the fight, strengthening the team and shaping the future.” -Chief Cody
--------
Looks like this train is already in motion. What can we do to improve on what has already been "improved?" Do we not agree with the changes that were implemented? I think we might have a hard time coming up with better solution than what what is already out there. Do we need to focus on accountability?
AFI 36-2406 (http://static.e-publishing.af.mil/production/1/af_a1/publication/afi36-2406/afi36-2406.pdf):
--2.1 ...Raters document the session on the ACA worksheet and
use the Performance Feedback in Section VI to assess or discuss the objectives, standards,
behavior, and performance with the ratee.
COMMENT: This right here captures what Marcus Peduzzi suggested above (task compliance, quality, and criteria) and more.
--2.2.3. The rater’s rater will:
2.2.3.1. Monitor personnel to ensure raters properly conduct ACA sessions.
2.2.3.2. Conduct ACA sessions when:
2.2.3.2.1. A lower-level rater is not available due to unusual circumstances.
2.2.3.2.2. Officially assuming the subordinate rater’s responsibilities.
--2.2.6. Unit will:
2.2.6.1. At the unit commander’s request, develop a tracking mechanism to ensure ACAs
are conducted. It is the responsibility of individual raters to maintain copies of all
completed ACAs and all signed ACA notices (or appropriate statements) on their
assigned ratees (RegAF only).
COMMENT: If these are the regs, are people actually keeping them? Do we need more education/training? I'm not sure where I can find facts on people not completing these. Does the MPS actually track these? Could we get these numbers from our units?
This comment has been removed by the author.
DeleteThe only measurable quantities on the ACA AF724 are the subjective ratings from "Does not meet" to "Clearly Exceeds." This does not easily translate to the OPR AF707. The only ratings on the OPR are "Meets" and "Does not Meet."
Delete(http://static.e-publishing.af.mil/production/1/af_a1/form/af724/af724_locked.pdf)
(http://static.e-publishing.af.mil/production/1/af_a1/form/af707/af707_locked.pdf)
In my flight of 13 only one has received meaningful feedback from their supervisor. All but one wrote their own OPRs removing any opportunity for feedback from the supervisor.
ReplyDeleteTeam,
DeleteMatthew, I just polled my own flight of 14: 0 had meaningful feedback and all wrote their own OPRs.
V/R
I have been an officer for just over 4 years and had 3 OPRs. I have been deployed during each of these rating periods and have not received effective feedback, feedback that could help me improve at my the work that I do, once. Peer feedback could help fill this gap.
ReplyDeleteMarijan Pavisic, a human resources consultant at Zagreb Global, recommends anonymous peer feedback.
"Pavisic says one company defied this recommendation. Instead, it gathered the executives together in a conference room and read their reviews of each other, aloud.
'Joe said about Bob, you know, this and that. And Joanne said about Michael this and that. What happened is actually two of the senior VPs resigned because they took it so personally. The distraction to the business for the next week was horrendous because they kept arguing,' Pavisic says."
http://www.npr.org/2015/08/21/433192324/peer-review-feedback-the-good-the-bad-the-really-ugly
Excerpt from "Toward a Superior Promotion System" by Kyle Byard, Ben Malisow and Col Martin E.B. France, Air and Space Power Journal, July-August 2012
ReplyDeleteAccording to the Officer Evaluation System: Training Guide, the accompanying
sample statements describe four strata of officer strength:
Top:
“My #1 of 12 . . . finest officer I’ve ever known”
“Top 3% of my 35 Majors”
“My #1 choice for [senior developmental education] now . . . big [group
commander] next!”
2nd level:
“Top 10% in wing”
“Top 10% I’ve known in my career”
3rd level:
“One of my best”
Lowest:
“Outstanding Officer”
“Superior Officer”
It is possible that a formal rating system which equates the meanings of
“superior” and “lowest” and interprets the description “one of my best”
as “3rd level” may not provide optimal clarity of meaning to either the
promotion board or the ratee. Even in the favorable levels, significant
ambiguity exists between “top 3% of my 35 majors” (mathematically
“first”) and “top 10% in wing.” This deliberately created, somewhat Orwellian
alternative language (“superior officer” equals “the lowest performing
officer I know”) exists in official guidance although not in the
system’s establishing regulations (e.g., AFI 36-2501, Officer Promotions
and Selective Continuation).
Found Here
http://www.airpower.maxwell.af.mil/digital/pdf/articles/2012-Jul-Aug/F-Byardetal.pdf
Superior is interpreted as lowest. This is an obvious example of confusing and inaccurate feedback. I understand that it may be uncomfortable to wound the ego of others, but it is necessary to promote positive change.
DeleteTeam,
ReplyDeleteSorry for the delay in response. Monitoring and evaluation are powerful evaluation tools that supervisors can use to motivate and achieve results. There has been an evolution away from the traditional implementation approach to a results based approach. The latter helps to answer the "so what" question. Why does our action every day matter to the Air Force as a whole? Outcomes and impacts must be evaluated. The monitoring and evaluation system should provide a tool for management to determine if and when goals are being achieved over time. A handbook was designed in 2004 to stand alone as guide (ten step model) to implement a M&E system.
Kusek, Jody Zall, and Ray C. Rist. Ten steps to a results-based monitoring and evaluation system: a handbook for development practitioners. World Bank Publications, 2004.
V/R
Giselle