COG #2: Ineffective Feedback--The current feedback system is inconsistent, subjective, immeasurable, does not provide a mechanism to accurately translate assessments into the OES, and has a poor culture of accountability.
How do we fix this?
How do we fix this?
Teams that bring completed COAs across the line (validated by ST) will receive 1 point
Submit your ideas here. Ideas can be anything however fully flushed out ideas will contain:
COA Title:
- What is the banner that your team will rally behind
Objectives:
- What is your COA trying to accomplish? Use action verbs.
Desired Effects:
- State how this COA will positively impact the Officer Evaluation System.
Milestones:
- Red: What we need to do right now: 1/3 of the COA has been accomplished
- Yellow: The next major step: 1/3 and 2/3 of COA have been accomplished
- Green: How do we sustain measure success: >2/3 of COA accomplished
Please label your posts to avoid confusion
i.e. The Title of the COA is ... The Objective of this COA is ...
Each team MUST identify at least 3 implications for executing their COA.
  |
| Gold Gear is the COA, the Gears coming off are Implications |
- Achieve 5 validated effects from each implication all those who contributed will receive an extra point.
Username: bet
Password: auciiteam
I will post an idea for the objective. The objective of this COA should be to create a system that fosters measurable and constructive feedback. The OES needs a simple to use tool that fosters feedback between the rater and ratee. It should produce measurable results that can be used to compare progress.
ReplyDeleteTitle: Trending Feedback?
Delete-- Are you taking about creating measurable continuity in the form of a control chart. Basically every time feedback is given there are multiple areas of assessment. If feedback is actually done trends should become apparent as well as outliers. Sort of like the PT test... you are able to "see" trend as 90-95-100-75 etc. I think there is an easy kill here in the area of Desired Effect, if anyone wants to continue this in another thread.
Desired Effect: Create a system that allows graphical and numerical presentations, so the rater and ratee can track progress.
DeleteI definitely think a Likert scale is the way to go. I like your idea about having a system where you can see the trend like the PT test.
Implication 1: Will remove the subjectivity in stratifications.
DeleteEffect 1: Metrics will provide a quantitative means of ranking individuals against their peers at any level of an organization.
Effect 2: Organizations will reduce the amount of time they spend in knife fights over stratifications.
Effect 3: Higher Commands will have an easy way of gauging the overall trends of good/bad performance and can use it to create better training to address shortcomings.
Effect 4: Commanders will lose the ability to fully control how individuals get stratified.
Effect 5: The system may begin to inflate to resemble the problems of the old EPR system. (Note: this can be mitigated by providing similar % restrictions or other guidelines for application)
Implication 2: Will create a cleaner/easier to read report of individuals' work/performance history
DeleteEffect 1: Reworked PRF. Most of the info that has to be collected on PRFs would now be easily discoverable, making the PRF either redundant or in need of evolution, possibly to a narrative-styled recommendation
Effect 2: Less time required for organizing/formatting reports, less work for Group/above execs/admins.
Effect 3: Returning more time/admin bodies back to completing the actual mission
Effect 4: De-humanizing individuals even further...less a person, more a metric.
Effect 5: Loss of "so what" to individual's/org's mission impacts. Less "what you're doing" and more "how well you're doing it"
Implication 3: Will provide better accountability
DeleteEffect 1: Improved performance from motivated individuals.
Effect 2: Easier path to remove/separate poor performing individuals.
Effect 3: Harder to "good ol boy" the system to hide an under-performing friend
Effect 4: Separation of individuals who preferred the "squishy" nature of the old OES
Effect 5: Retention-bonus offers tailored to the performance of the individual.
Title: Ineffective by Omission
ReplyDeleteThe current feedback system is ineffective because there is no effective method for fostering feedback session and how we are directly meeting the unit mission and vision. The 360 degree feedback was proposed on the discussion board by Thomas Meyer and Liezl Starte. This would be a solid COA.
See Burnside thread below.
DeleteSee Burnside thread below.
DeleteTitle: Ineffective by Non Standard Work
ReplyDeleteThe current feedback system is also ineffective because of lack of standard work. The lack of a standard abbreviation system is one example. Additionally, how we monitor and evaluate the work completed by an individual during the course of a year is lacking. If we implemented the 10 steps that have been identified (2004 handbook) as proven methods for development of monitoring and evaluation systems would be a possible COA. This COA is prime for drill down (I think).
V/R
Implication: Highlight significant accomplishments or potential and recommendations for improvements.
DeleteEffect 1 - Improved communication between units
Effect 2 - Clarifies objectives and minimum vs above requirements
Effect 3 - Translates objectives into performance indicators
Effect 4 - Regular updates to management (possible problems updated early)
Effect 5 - Data collected at intervals to ensure quantifiable results
Implication: Highlight significant accomplishments or potential and recommendations for improvements.
DeleteEffect 1 - Improved communication between units
Effect 2 - Clarifies objectives and minimum vs above requirements
Effect 3 - Translates objectives into performance indicators
Effect 4 - Regular updates to management (possible problems updated early)
Effect 5 - Data collected at intervals to ensure quantifiable results
COA Title:
ReplyDelete- Active Feedback Accountability
Objectives:
- Establish effective accountability web-based system for all required feedback
Desired Effects:
- This system would be used to highlight individual feedback status. Supervisors that have not performed the required feedback would be flagged in the system.
Milestones:
- Red: What we need to do right now: Develop web-based system to track feedback
- Yellow: The next major step: Get approval and support from leadership in order to make the use of the system policy
- Green: How do we sustain measure success: Somehow provide incentive to members completing all required feedback. Or the opposite: seek some type of disciplinary action for those who have not completed all required feedback.
Implications: 1) New web-based system will need to be developed and tested, or perhaps a system already in use could be adapted for this purpose (depending on what is available). 2) Regulations/AFIs will need to be updated and adapted to align with the new system. 3) Training on the new system will be needed, especially for individuals not comfortable with using an electronic feedback system. 4) A major culture shift will be required. 5) The entire OES will feel the ripple effects if this method is implemented, since it will fundamentally change how we evaluate and assess people and their performance--OPRs and PRFs will have to be adapted as well.
DeleteWe already just rolled out the vPC application on AF Portal, which documents and highlights all those things anyways. If you were advocating the use of this to keep supervisors/raters accountable for timely initial/midterm feedback, those changes could be made, but since the official policy is that neither needs to be officially recorded, all it would take is an electronic "yes boss, I did that" to CYA without having done anything.
DeleteAnd Clare, your assessment that the OES would feel the ripple changes clearly isn't true since the program has been online for about a year and either of you seem to know about it. All the system does is change the form from a pdf to a website-hosted e-form, which was so cumbersome to add/download at each rung of the rating ladder that my last unit would resolve any issues via emailed pdfs and only input the e-form submission at the final signature point.
Based on personal observations, I don't think finding/making a new system will be very effective at improving accountability. Just my two cents.
I see where you are coming from. I know about vPC and I've used it before. If we don't use a system to keep people accountable, what can we use? How do we ensure that people at a minimum perform the feedback? Yes, people might say they are completing the feedback when they aren't, but that's a bigger problem all together. Currently, my UPC sends us an email when vPC alerts her that we need to accomplish a feedback. But that's it. We don't send it back or anything. What's the point of having a requirement if we don't keep people accountable to it?
DeleteCOA Title: 365 Feedback
ReplyDeleteObjective: Bring feedback considerations into the daily, weekly, and monthly activites of individuals
Desired Effects: To track progress towards goals set during the feedback session. Outlook calendar integrated program (or one that can easily export info to) allows for individual and supervisor to track real time progress towards a particular goal set forth during the feedback session. This can be used not only to track progress but also for the superisor to demonstrate to an individual what they said they were going to do, vs what they acutally did and can direct the conversation for midterm feedback as well as the OPR.
Milestones:
Red: Develop computer based, outlook integrated, feedback form. Program will ideally, take milestones developed during the feedback session along with proposed acountability dates and automatically export them to the individual's and supervisor's outlook calendar.
Yellow: train/demontrate to supervisors how to properly utilize the functionality, with empahsis on providing feedback/goals that can be tracked
Green: Survey completed by individual and supervisor at initial and midterm feedback, as well as at the OPR close out. Incorporate suggested improvements from end users.
The above is a first wag, and will need a bit more fleshing out, but its a start.
While I like this idea on the whole, it looks like it could very easily become another time-sucker that adds more to a both individuals calendars if they have to be regularly updating information on a weekly/monthly basis. if this is just intended to add general accountability for the completion of initial/midterm feedbacks, it has merit. But if it's meant to become a crucial leg in a completely overhauled OES, I don't see it providing any more significant benefits without adding equally sizeable (or bigger) detractors.
DeleteImplication: The 365 degree feedback will provide real-time, accurate feedback from supervisors to subordinates.
DeleteEffect 1 - If a web based program is developed, then there will be a way to capture positive or negative information from different supervisors of the same subordinate. This is useful as a recent poll of my flight revealed that they changed leadership three times during the year (on average) and those that rated them never actually worked with them.
Effect 2 - The military has provided many different computer systems that did not meet the intent they were developed for and sometimes the government just lost millions when programs were scrapped.
Effect 3 - There needs to be proven success with the program so that leadership feels that the time it takes to input data is worth the reward. Also, needs to be user friendly.
Effect 4 - Establish minimum input requirements and repercussions if these milestones are not accomplished. Agree that it would be beneficial that these milestones are flagged to several supervisors outlook calendar.
Effect 5 - Higher leadership will have to endorse the new program. If there is a point system associated with above and beyond bullets in the bank will then affect stratification less subjectively.
Implication: The 365 degree feedback will provide real-time, accurate feedback from supervisors to subordinates.
DeleteEffect 1 - If a web based program is developed, then there will be a way to capture positive or negative information from different supervisors of the same subordinate. This is useful as a recent poll of my flight revealed that they changed leadership three times during the year (on average) and those that rated them never actually worked with them.
Effect 2 - The military has provided many different computer systems that did not meet the intent they were developed for and sometimes the government just lost millions when programs were scrapped.
Effect 3 - There needs to be proven success with the program so that leadership feels that the time it takes to input data is worth the reward. Also, needs to be user friendly.
Effect 4 - Establish minimum input requirements and repercussions if these milestones are not accomplished. Agree that it would be beneficial that these milestones are flagged to several supervisors outlook calendar.
Effect 5 - Higher leadership will have to endorse the new program. If there is a point system associated with above and beyond bullets in the bank will then affect stratification less subjectively.
COA Title: Stratification Style Blocks Included in Standard Feedback Form
ReplyDeleteObjective: Streamline the Feedback, OPR, and PRF system by ensuring that quantifiable data is provided in every form and enable ratees to know where they fall amongst their peers early and often so that they know where they currently stand as well as areas where they must improve in order to earn a better stratification on their future OPRs. Additionally, enable raters and additional raters the ability to document ratee trends and legitimize future stratifications
Desired Effect:
1. Quantify ratee performance based on current standings within their flight/unit they are being rated against
2. Remove subjective feedback or favorable stratifications merely based on good relations with rater or additional rater
3. Create a paper trail of positive and negative performance to legitimize stratifications on OPRs which eventually are included on PRFs
4. Create a system of accountability for raters to show that they are continuously providing useful data to ratees through feedback and that such feedback sessions ultimately line up with how ratees are stratified on OPRs. Additional raters provide oversight to this process as well
Milestones:
Red - Edit current feedback forms to include a 1-10 stratification rating system to go along with each criteria, replacing the "meets standards" "clearly exceeds," etc. boxes. 1 represents the bottom 10% amongst those being rated by the rater and 10 representing the top 10%.
Yellow - Train raters and additional raters as to the execution standards and practices expected by AFI 36-2406. Emphasize stepping stone process from initial to midterm to final feedback and how the OPR builds on feedback and is essentially a reflection of the feedback process.
Green - Begin the feedback process for individuals with reports closing greater than 120 days from 1 October 2016, the start of the new fiscal year for the Air Force. This will provide enough time to train and mentor raters and additional raters as well as prepare the first ratees that will experience the feedback.