Improving feedback to improve airmen.

Editorial Abstract: This article contends that force development in the US Air Force is undermined by lackluster feedback at the tactical level. The authors outline the current use of feedback in the service, review factors related to creating effective developmental feedback at the tactical level, and comment on current initiatives designed to improve feedback and force development within the Air Force.

UNITED STATES AIR Force (USAF) leaders have recently highlighted the importance of deliberate individual development and overall force development. (1) Unfortunately, force development is currently undermined by the lackluster practice of feedback at the tactical level. Indeed, "deliberate development" requires that Airmen receive beneficial feedback as they progress through their Air Force careers. (2) Yet, we believe that feedback is often underutilized in the USAF. To ensure long-term success, the USAF needs to integrate feedback more effectively into a career-long process of development. This article outlines the current use of feedback in the USAF, reviews factors related to creating effective developmental feedback at the tactical level, and provides comments on current USAF initiatives designed to improve feedback and force development.

Current Feedback System

Airmen receive performance feedback in many forms, such as check rides, inspections, and promotion tests. However, for many Airmen, the first thing that comes to mind when they hear the term feedback is the Air Force's performance feedback worksheet (PFW). Feedback in the USAF actually takes on three basic forms: informal feedback, formal feedback (e.g., Air Force [AF] Forms 724A, 724B, 931, and 932), and official performance measures (e.g., check rides, quality assurance evaluator inspections, and performance reports).

Informal Feedback

Informal feedback consists of the information that Airmen provide to one another during their regular workplace communications. It can be as simple as a supervisor or coworker commenting on a uniform flaw or an incorrectly completed procedure. Airmen often dispense positive informal feedback by writing letters of appreciation, praising others publicly, or telling coworkers when they did something well. Some career fields tend to be more conducive to this type of feedback than others. For example, aircrews routinely debrief sorties to discuss strengths and weaknesses, and security-force teams regularly conduct "hotwashes" of exercises. Through daily interactions and informal feedback, leaders establish the key interpersonal-relationship connections that can make formal feedback processes more effective.

Formal Feedback

Formal feedback is generated through structured, organized procedures. In the USAF, formal performance feedback is usually conducted using the grade-specific PFW. According to Air Force Instruction (AFI) 36-2406, Officer and Enlisted Evaluation Systems, "Performance feedback is a private, formal communication a rater uses to tell a ratee what is expected regarding duty performance and how well the ratee is meeting those expectations." (3) Formal feedback is required for all enlisted personnel and all officers through the rank of colonel. Initial feedback occurs within 60 days of the Airman's assignment to the rater, and midterm feedback occurs 180 days after initial feedback. The form is handwritten or typed by the rater and does not become a permanent part of the ratee's record. The rater may keep a copy of the PFW, but there are restrictions on who can access this copy. If done correctly, the PFW gives Airmen specific competencies to develop that can lead to improved performance and behavior change.

Official Performance Measures

Official performance measures exist in many forms such as check rides, promotion tests, and, at least annually, training reports, officer performance reports (OPR), or enlisted performance reports (EPR). Again, referring to AFI 36-2406, the officer and enlisted evaluation systems exist not only to provide meaningful feedback and advice for improvement, but also to "provide a reliable, long-term, cumulative record of performance and potential based on that performance ... [and] to provide officer central selection boards, senior Non-Commissioned Officer evaluation boards, the Weighted Airman Promotion System and other personnel managers sound information to assist in identifying the best qualified officers and enlisted personnel." (4) OPRs and EPRs, along with promotion recommendation forms (PRF), therefore, have two interconnected purposes: performance documentation and selection.

Each type of feedback contributes something essential to the development and advancement of Airmen (see fig.). Unfortunately, Airmen often receive hurried official appraisals, rushed formal feedback, and limited informal feedback. Also, Airmen frequently view OPRs, EPRs, PRFs, and training reports simply as a means for selection or administrative action but overlook their developmental value. Many feedback sessions go something like this: the supervisor is behind on several tasks and receives notification that feedback is due; the feedback is squeezed in between other duties; and both the supervisor and the subordinate are left feeling that not much was accomplished. This type of situation undermines the effectiveness of feedback in the USAF culture, despite USAF policy that deliberate development is critical to healthy, long-term force development.

[FIGURE OMITTED]

The largest impediment to successful feedback is probably a lack of time. Finding time to give or receive effective feedback is understandably difficult. In stressful operational environments, actual opportunities for formal feedback may be few. However, this ought not to be the case in garrison operations or training situations. While a shortened performance feedback session (in accordance with AFI 362406) is occasionally necessary in the field, feedback meetings need to be a priority when Airmen return to their home station or are assigned to a training unit. Still, even when Airmen agree that feedback is a top priority, many will resist the feedback process, particularly when it involves giving or receiving negative feedback. Discomfort with feedback can result in maladaptive behaviors such as procrastination, denial, brooding, and self-sabotage. (5)

The USAF clearly lacks immunity from the consequences associated with ineffective feedback. Feedback failures have led to dreadful outcomes such as the B-52 crash at Fairchild AFB, Washington. (6) The USAF should strive, therefore, to integrate the effective use of feedback into USAF culture and force development. Feedback could serve well as an advantageous and strategic practice: leaders could benefit from a more effective team, followers could benefit through continued professional development, and the USAF could benefit from more-proficient personnel. To improve feedback, Airmen must first be educated about the complexities of the feedback process. Airmen often fail to realize the pitfalls of the feedback process and thereby provide ineffective feedback in their daily interactions. USAF senior leaders must be aware of the same pitfalls as they develop initiatives designed to enhance feedback and force development.

Characteristics of Effective Feedback

Organization members must realize that creating effective developmental feedback is a challenging process that requires time and energy. Many believe that simply increasing the amount of feedback and perhaps instituting formal feedback programs will improve organizational effectiveness. This is simply not true. Research demonstrates that feedback can result in more harm than good. One major review estimated that one-third of feedback recipients became less motivated after receiving feedback. (7) Although this percentage may seem high, it is likely easy for individuals to recall "real world" examples of coworkers who left a feedback session frustrated and angry, rather than motivated to improve.

The challenges associated with feedback are further illuminated by clarifying that "feedback" really refers to two overlapping processes. First, feedback begins with the collection of information that will be provided to the feedback recipient (i.e., the target). This process is particularly clear when information is assembled formally for a structured feedback event, such as an annual supervisor assessment. Yet, information is constantly gathered informally as well, such as when coworkers form opinions about a particular person's strengths and weaknesses. To make feedback effective, these formal and informal assessments should be accurate. This seems obvious, but research demonstrates repeatedly that the agreement between observers can often be remarkably low. (8) Second, even after information has been developed either through formal or informal means, such information must be delivered to feedback recipients effectively. Indeed, communicating "areas for improvement" is a challenging task. For example, a supervisor informs a subordinate that he or she needs to work on communication skills. If this assessment is inaccurate, the subordinate might leave the session unnecessarily concerned about these skills and meanwhile overlook skills that truly need development. Should the assessment be accurate, the supervisor's delivery, if poor, could be detrimental. To elucidate some of the specific factors which can inhibit effective feedback, we will review the feedback process using four main themes: purpose, provider, preparation, and prevalence.

Purpose

When implementing a feedback system, it is important to remain clear about the purpose of the feedback. Feedback can often be construed as developmental or administrative. Developmental feedback is intended primarily to develop the effectiveness of an organization's individual members. It is not connected to any positive or negative administrative action. For example, when an Airman receives feedback about his or her presentation skills, a low rating would not result in an official reprimand or a high rating in any official reward. The commander has simply identified an area where the Airman could improve. This developmental opportunity may be documented to help the commander and the Airman track progress, but such documentation would often be kept confidential, perhaps maintained by the Airman. Alternatively, administrative feedback can influence specific administrative decisions such as adverse actions, bonuses, promotions, or job selection. In these cases, raters and the target know that the assessments will be reflected in the target's personnel records.

I. M. Jawahar and Charles R. Williams reviewed 22 studies that examined feedback programs. (9) They found that ratings were more positive when created for administrative rather than developmental purposes. In the Air Force rating system, OPRs, EPRs, PRFs, and training reports are feedback tools used for administrative purposes. Not surprisingly, feedback generated by these tools tends to be over whelmingly positive, and Airmen generally believe (perhaps accurately) that if an OPR or EPR is good to average, then it is actually bad. This positive bias in administrative feedback largely negates the usefulness of administrative feedback for the purpose of developing organizations and people.

Provider

Organizations must decide who will be raters in the feedback process. Raters normally see their own assessment efforts as objective and accurate, but the organizational positions of raters relative to the target individual can lead to inaccuracies in ratings. Traditionally, superiors have been the primary providers of performance feedback. This "top-down" assessment system makes sense. Superiors usually bring a considerable amount of experience to the assessment and development of subordinates. However, research shows that superior assessments are not infallible. Superiors might fail to recognize the strategies that subordinates may employ to ensure they appear at their best. (10) Airmen can also exhibit characteristics (e.g., accepting the status quo) that appear more acceptable to superiors than to peers and subordinates. (11) In addition, superiors can observe only a selection of an Airman's behavior because, quite obviously, they have their own responsibilities and duties to perform. It follows then that superiors sometimes lack important information about their Airmen and therefore may assess them inaccurately. (12)

Some organizations supplement superiors' ratings with assessments from other raters. The following terms describe this kind of feedback: multirater feedback, multisource feedback, and 360-degree feedback. Some authors use all of these terms interchangeably. For purposes of this review, multirater feedback and multisource feedback will describe any form of feedback coming from more than one person (e.g., a self-assessment and a peer assessment), and 360-degree feedback will describe a special type of feedback that includes self-, superior, peer, and subordinate assessments. (13) This is not to say that a feedback system should include all these raters. Rather, 360-degree feedback is only one form of multirater feedback. Organizations may have compelling reasons for using particular combinations of these four rater groups or perhaps only one rater (e.g., time, availability, and cost). The important point is that multirater feedback can be powerful because each group of raters provides a different perspective in the feedback process.

Peers and subordinates have distinct advantages and disadvantages as raters, resulting from their particular relationships to a target. Peers may provide informative assessments because they understand best the target's work circumstances. At the same time, peer assessments may be more susceptible to friendship bias because peers might inflate particular performance ratings so their friends will not be viewed unfavorably. (14) Subordinates can also contribute useful information to targets because subordinates are in a unique, and often advantageous, position to evaluate supervisor effectiveness. (15) There is concern that subordinates might provide biased feedback due to fear that negative ratings could result in retaliation, but multiple subordinates can enhance the opportunity to create truly anonymous assessments. (16) However, it is worth noting that supervisors can feel that their authority is undermined when subordinates' ratings are the only source of feedback. (17)

Self-ratings are another source of feedback which offers unique advantages and disadvantages. One obvious disadvantage is that individuals tend to view their own performance more positively than deserved. (18) Even when individuals believe that they are evaluating themselves accurately, they might not be aware of how others interpret their performance. (19) For example, an Airman may consider himself or herself reserved, but others might see the same behavior as apathetic. Self-ratings, however, do provide a unique perspective into individual performance. By allowing Airmen to assess their own performances, they may feel more empowered in the feedback process. Comparing self-ratings to other ratings can also help individuals understand and acknowledge others' perceptions and the important influence that such perceptions can have on achieving workplace success. (20) When other ratings verify positive self-ratings, they can serve as reinforcement for good performance.

Preparation

Many Airmen believe that they are intrinsically capable of delivering and receiving feedback effectively. They do not understand that giving and receiving feedback is a skill acquired through training and practice. Certainly nobody would think it wise to pilot an aircraft without proper and thorough flight training. Unfortunately, many Airmen fail to draw a similar connection to the feedback process. This is not the only misconception concerning feedback. While numerous individuals agree that it is difficult to receive feedback, especially negative feedback, they likely overlook the difficulties involved in giving feedback. Indeed, many individuals can experience negative emotions when they provide others with positive or negative feedback. (21) Training can help feedback providers to simply get comfortable with being uncomfortable. After all, providing feedback, particularly in the USAF, is an essential and unavoidable aspect of leadership. Training is also recommended for those who receive feedback. (22) Through training, the targets of feedback can learn how to make the most of their feedback experiences and thereby improve their performances. For example, Airmen probably dislike receiving feedback because they anticipate that it will be negative. However, training can help Airmen understand that they should welcome all kinds of feedback since it can contribute to their long-term growth and success.

Prevalence

Formal feedback should occur at regular intervals so that organization members learn to expect it. Researchers have positively linked the frequency of feedback to superior job performance. (23) However, this finding should not cause leaders to choose quantity over quality. In some organizations it may be impossible due to mission-related, structural, and other reasons to give quality feedback frequently or during certain critical periods of time. Air Force commanders should not offer numerous, superficial feedback sessions. Instead, as research indicates, a commander can add value to single feedback sessions with subsequent follow-up discussions. (24)

Improving Feedback across the USAF

Compared to private organizations, the USAF experiences reduced ability to hire established talent from outside the organization. Instead, the service tends to develop its own people to fill its upper-level positions. The USAF experiences additional personnel-development challenges, considering that military warfare is changing rapidly and that military careers turn over more quickly than do business careers. A typical military "career" often lasts just over 20 years whereas a career in the private sector can last much longer.

Fortunately, the USAF recognizes the importance of effective feedback in terms of organizational effectiveness. Air Force Doctrine Document (AFDD) 1-1, Leadership and Force Development, clearly calls for a focus on continued Airman development and ties that development to seeking constructive feedback from coworkers. (25) Furthermore, deliberate development is emphasized in plans to formally develop a more rigorous and effective system of feedback and Airman development.

According to Lt Col Danny Miller (assigned to the Air Staff AF/A1D-Airman Development and Sustainment), there is no immediate plan to institute a USAF-wide multirater-feedback program. Rather, the USAF is developing a broader plan to (a) streamline overlapping USAF training programs, (b) outline enduring competencies needed in Airmen, (c) outline additional occupation-based competencies for Airmen, (d) provide a central Internet-based resource suite that provides leadership-development information for the entire USAF, and (e) improve informal and formal feedback throughout the USAF. This transformation has begun with the implementation of multirater feedback into various USAF agencies. Some of these implementations include Airman-development programs (e.g., Air War College, the Chief Master Sergeant Leadership Course, and the GS-15 [US Federal Civil Service pay grade] Leadership Course) and similar programs offered within some USAF organizations (e.g., Air Force Personnel Center, Air Force Research Laboratory, and Air Force Materiel Command). The long-term goal is to develop multirater feedback in all professional military education (PME), supervisor, and commander courses and to relate this feedback to each Airman's developmental requirements. (26)

This general plan to improve USAF feedback, while reducing costs by removing redundancies in training programs, is laudable. At the same time, it is important to note that organizations can be resistant to change. Success will require a genuine culture shift at the strategic, operational, and tactical levels. (27)

Strategic

Strategic leadership shapes strategy and policy, ensures integration and proper resourcing, and drives the execution of culture change. (28) Feedback is one part of a complete force-development package that includes providing the right training at the right time, maintaining effective career management with active commander involvement, and focusing on carrying out the wartime mission. Strategic leaders provide a clear, long-term vision regarding feedback and ensure that PME programs include lessons and courses about feedback philosophy, skills, and procedures. In addition, strategic leaders must properly fund and staff the feedback process to make certain that the system is accomplishing its purpose. Senior leaders can guide their inspection teams to confirm that feedback is a command-interest item on inspector-general inspections and staff-assistance visits. Commander's courses should contain a module on the vision and implementation of feedback systems. Without a clearly articulated strategic plan and sustained effort, subordinate commanders will be inconsistent at best when delivering feedback.

Operational

Operational leadership focuses on establishing a vision for the unit, mentoring and coaching for success, and partnering up and down the chain of command to maximize unit effectiveness. (29) This level of leadership is key to the culture change that must take place to establish effective feedback as an integral part of USAF culture. The operational leader should also serve as the example of a feedback provider to leaders at the tactical level. This trickle-down effect allows the operational leader's experience and vision to reach the lowest levels of the unit. Through effective coaching and mentoring, the operational leader can increase efficiency in the unit--a key outcome in today's high-tempo, low-resource environment. Without the support of the operational leader, feedback initiatives are bound to fail.

Tactical

Feedback becomes reality at the tactical level of leadership, which primarily includes personal leadership skills such as the ability to accurately self- assess, inspire trust, and communicate effectively. (30) Tactical leaders are the frontline supervisors responsible for the development of their people. Leaders at this level must take seriously their developmental responsibility. This means that they must take the time to get to know the people in their unit and understand their developmental needs. Tactical leaders must challenge their subordinates to receive and provide developmental feedback on a regular basis and must ensure that subordinate leaders are effectively leading their people as well. In addition, they must set the example by soliciting feedback to improve their leadership skills. Leaders at this level should learn about effective feedback techniques and seek mentors who can help form their feedback skills.

General Recommendations

Even with support from the strategic levels, the USAF should remain attentive to the potential for feedback failure at the operational and tactical levels. Many Airmen at these levels represent the "middle management" of the USAF because they direct the execution of USAF policy. Research in industrial and organizational psychology demonstrates that strategic-level initiatives often fail because they lack "buy-in" at this level. In fact, researchers have referred to middle management as a "concrete layer" due to the likelihood that strategic initiatives will fail there. (31) While the need for effective feedback might seem obvious to senior members of the military who can reflect on careers made up of successes and mistakes, that need might be less obvious to less-experienced and more middle-management Airmen.

Airmen probably tend to see their own behavior as effective and might therefore feel less need to get feedback from others. Additionally, research regarding personal beliefs about leadership demonstrates that some individuals do not believe in leadership development and show reduced motivation for leadership-development-related programs. (32) Furthermore, busy work schedules in many USAF units are unlikely to change. Without authentic buy-in at the tactical level, the high-operations tempo will only exacerbate misgivings about the time required to create an effective formal feedback process. Unless integration and planning are successful, units will conduct formal feedback programs haphazardly, if they do them at all.

Evidence of the potential breakdown is apparent in the PFW, the utility of which appears obvious because it entails nothing more than a formal communication between commanders and Airmen. Yet the PFW seems to be used ineffectively. (33) Furthermore, many can recall the often heated debates over Total Quality Management (TQM) in the mid to late 1990s. Col Charles J. Dunlap Jr., in a 1996 opinion piece about the future of the USAF, was harshly critical of TQM, indicating that it was a faddish program which ultimately undermined military discipline. (34) Negative views of TQM such as Dunlap's were pervasive in the 1990s Air Force, and USAF leaders could face similar "push-back" with any new and mandatory formal feedback system. The ineffectiveness of the PFW and programs like TQM speaks to the difference between instituting change through rules and regulation and creating change by training and encouraging Airmen towards a collective vision. The PFW is required but used poorly, and the USAF must do more than simply require installation of new programs. Through training and sustaining a deep culture shift, Airmen can make the feedback process a major priority.

Despite these cautions, the USAF can and should continue its current efforts to improve the USAF approach to feedback. The continued development of formal feedback programs can assist this process so long as such programs are instituted correctly and given sufficient support. USAF leaders should institute formal feedback programs with attention to the myriad of issues already summarized in this article. Important summary points address successful feedback programs in the context of purpose, provider, preparation, and prevalence (see table).

Leadership should emphasize developmental, rather than administrative, feedback programs. OPRs and EPRs, which are primarily administrative evaluations, receive a great deal of attention. The USAF has a formal developmental-feedback tool in the PFW, but it often has a low priority and is inadequately applied. The development and implementation of new formal-assessment programs could fill the need for developmental feedback to serve Airmen in building their own careers.

Increased emphasis on feedback must be accompanied by effective training regarding the giving and receiving of feedback. Feedback training might prepare Airmen to deal with potentially negative feedback, decrease defensiveness and other ineffective behaviors, bring self-development ideas to the feedback session, and express disagreement constructively. Learning about giving and receiving feedback can begin in the Promotion Fitness Examination (PFE) Study Guide (AFPAM 36-2241V1) or in books such as London's Job Feedback. (35) Both works provide useful tips that are helpful to feedback givers and receivers. The following provides some examples of feedback tips that could be utilized in feedback training:

* Remain professional. If an Airman becomes defensive, do not take it personally and respond with destructive comments. (36)

* Before offering an evaluation, empower Airmen by giving them a chance to describe their own performances and to suggest areas of improvement. (37)

* Provide positive and negative feedback. Review specific accomplishments before launching into improvements. (38)

* Focus on behaviors rather than on general personality characteristics. (39) For example, "I have observed that your production has decreased" rather than "I think you are becoming lazy."

* Listen carefully, and ask questions for clarification. (40)

* Be sincere. Effective feedback givers must be genuinely interested in their personnel. (41)

Most importantly, leadership must not assume that the institution of required feedback programs means that feedback will improve. Extant research indicates that many factors can influence the effectiveness of multirater feedback and that the institution of multirater feedback programs can be ineffective or even deleterious. (42) The USAF must also address the attitudes that Airmen possess about feedback. This process should begin when Airmen enter the USAF and remain consistent throughout their careers. Airmen must be convinced that leadership and professional effectiveness are developable skills and that they must not buy into the idea that leadership is something that they either have or they don't. Airmen should also understand that their own self-assessments are not necessarily accurate and believe that feedback can contribute to their personal success and the overall success of the USAF. A motivated Airman does not need a formal feedback program to receive developmental feedback and can self-generate feedback simply by contacting others, usually fellow Airmen, and asking for assistance. Similarly, a unit commander need not rely only on required formal feedback programs. Commanders have the responsibility to ensure that feedback enhances the development of their people and to supplement such programs when necessary.

Conclusion

The USAF can facilitate force development by increasing Airmen's proficiency with feedback. Airmen will become more engaged in the feedback process as (a) they believe personally that feedback is an important component of their development, and (b) they participate in effective feedback programs. The obvious challenge is that feedback and feedback training require time and energy, but we believe that this is a price worth paying. Giving and receiving feedback effectively is an important leadership competency. As such, feedback training can be viewed as an additional form of force development. Training Airmen to be proficient with feedback might actually reduce costs by ensuring that new formal feedback programs succeed. Finally, change is occurring at an unprecedented rate, and the USAF will experience continued difficulty in predicting future developments. By using effective feedback to develop required profession-based competencies, the USAF can most effectively prepare the next generation of Airmen for the challenges ahead.

Notes

(1.) Air Force Doctrine Document (AFDD) 1-1, Leadership and Force Development, 18 February 2004, iii, http://www.e-publishing.af.mil/pubfiles/af/dd/afdd1-1/afdd1-1.pdf (accessed 8 June 2006).

(2.) The term Airmen often refers to uniformed USAF personnel. It is used because of the specific nature and demands of the military profession. This is not meant to minimize, however, the importance of developing all USAF personnel, civilian or uniformed, as important components of the USAF mission.

(3.) Air Force Instruction (AFI) 36-2406, Officer and Enlisted Evaluation Systems, 15 April 2005, 11, http://www.e-publishing.af.mil/pubfiles/af/36/afi36-2406/afi36-2406.pdf (accessed 8 June 2006).

(4.) Ibid., 6.

(5.) Jay M. Jackman and Myra H. Strober, "Fear of Feedback," Harvard Business Review 81, no. 4 (April 2003): 102.

(6.) For further discussion regarding the devastating crash of Czar 52 at Fairchild AFB and other accidents that may have been prevented through effective feedback, see Anthony T. Kern, Darker Shades of Blue: The Rogue Pilot (New York: McGraw-Hill, 1999).

(7.) Avraham N. Kluger and Angelo DeNisi, "The Effects of Feedback Interventions on Performance: A Historical Review, a Meta-Analysis, and a Preliminary Feedback Intervention Theory," Psychological Bulletin 119, no. 2 (March 1996): 254.

(8.) Gary J. Greguras and Chet Robie, "A New Look at Within-Source Interrater Reliability of 360-Degree Feedback Ratings," Journal of Applied Psychology 83, no. 6 (1998): 962-64.

(9.) I. M. Jawahar and Charles R. Williams, "Where All the Children Are Above Average: The Performance Appraisal Purpose Effect," Personnel Psychology 50, no. 4 (Winter 1997): 916.

(10.) Kayyum A. Bohra and Janak Pandey, "Ingratiation toward Strangers, Friends, and Bosses," Journal of Social Psychology 122, no. 2 (April 1984): 217.

(11.) Sabrina Salam, Johnathan F. Cox, and Henry P. Sims Jr., "In the Eye of the Beholder: How Leadership Relates to 360-Degree Performance Ratings," Group and Organizational Management 22, no. 2 (1997): 185-209.

(12.) Janka I. Stoker and Beate I. J. M. Van der Heijden, "Competence Development and Appraisal in Organizations," Journal of Career Development 28, no. 2 (Winter 2001): 103, http://www.indicator-ict.com/docs/pdf/competence %20development%20paper%201.pdf.

(13.) Dr. Craig A. Foster and Melanie R. F. Law, "How Many Perspectives Provide a Compass? Differentiating 360-Degree and Multi-Source Feedback," International Journal of Selection and Assessment 14, no. 3 (September 2006): 290.

(14.) Glenn M. McEvoy and Paul F. Buller, "User Acceptance of Peer Appraisals in an Industrial Setting," Personnel Psychology 40, no. 4 (Winter 1987): 785-86.

(15.) H. John Bernardin, "Subordinate Appraisal: A Valuable Source of Information about Managers," Human Resource Management 25, no. 3 (Fall 1986): 423-24.

(16.) Ibid., 430.

(17.) H. John Bernardin, Sue A. Dahmus, and Gregory Redmon, "Attitudes of First-Line Supervisors toward Subordinate Appraisals," Human Resource Management 32, no. 2/3 (Summer and Fall 1993): 321-23.

(18.) Raymond F. Zammuto, Manuel London, and Rendrith M. Rowland, "Organization and Rater Differences in Performance Appraisals," Personnel Psychology 35, no. 3 (Autumn 1982): 648-49, 651-52.

(19.) Leanne E. Atwater and Francis J. Yammarino, "Self-Other Rating Agreement: A Review and Model," Research in Personnel and Human Resources Management 15 (1997): 145.

(20.) Ibid., 161-65.

(21.) Thomas P. Sawyer Jr., Lisa A. Hollis-Sawyer, and Amanda Pokryfke, "Personality and Social-Evaluative Anxieties Associated with Rating Discomfort in Anticipated Positive and Negative Feedback Conditions," Social Behavior and Personality 30, no. 4 (June 2002): 359-72 passim.

(22.) David A. Waldman and Leanne E. Atwater, "Confronting Barriers to Successful Implementation of Multisource Feedback," in Handbook of Multisource Feedback, ed. David W. Bracken, Carol W. Timmreck, and Allan H. Church (San Francisco, CA: Jossey-Bass, 2001): 474.

(23.) Anjelo J. Kinicki et al., "A Covariance Structure Analysis of Employees' Response to Performance Feedback," Journal of Applied Psychology 89, no. 6 (December 2004): 1063.

(24.) Manuel London, Job Feedback: Giving, Seeking, and Using Feedback for Performance Improvement (Mahwah, NJ: Lawrence Erlbaum Associates, 1997), 81.

(25.) AFDD 1-1, Leadership and Force Development, 5.

(26.) Lt Col Danny Miller, phone interview by the authors, 17 November 2005.

(27.) AFDD 1-1, Leadership and Force Development, 16-18.

(28.) Ibid., 17-18.

(29.) Ibid., 16-17.

(30.) Ibid., 16.

(31.) Leonard A. Schlesinger and James L. Heskett, "The Service-Driven Service Company," Harvard Business Review 69, no. 5 (September/October 1991): 81.

(32.) Dr. Craig A. Foster and Jeffrey E. Nelson, "The Influence of Implicit Theories of Leadership Ability on Corresponding Goals and Motivation" (working paper, US Air Force Academy, Colorado Springs, CO, 2005).

(33.) Saundra J. Reinke and J. Norman Baldwin, "Is Anybody Listening? Performance Evaluation Feedback in the U. S. Air Force," Journal of Political and Military Sociology 29, no. 1 (Summer 2001): 166, 172.

(34.) Col Charles J. Dunlap Jr., "Melancholy Reunion: A Report from the Future on the Collapse of Civil-Military Relations in the United States," Airpower Journal 10, no. 4 (Winter 1996): 96-97.

(35.) Air Force Pamphlet (AFPAM) 36-2241V1, Promotion Fitness Examination (PFE) Study Guide, vol. 1, 1 July 2005, 170-73, http://www.e-publishing.af.mil/pubfiles/af/36/afpam36-2241v1/ afpam36-2241v1.pdf (accessed 8 June 2006); and London, Job Feedback, 89-109.

(36.) London, Job Feedback, 15-22, 101-2.

(37.) Ibid., 94.

(38.) Ibid., 94, 100.

(39.) AFPAM 36-2241V1, PFE Study Guide, 171.

(40.) Ibid., 172.

(41.) Ibid.

(42.) For an introduction into research about multi-rater feedback, see Bracken, Timmreck, and Church, eds., Handbook of Multisource Feedback.

MELANIE R. F. LAW DR. CRAIG A. FOSTER COL GARY A. PACKARD JR., USAF *

* Correspondence concerning this article may be addressed to Dr. Craig A. Foster, HQ USAFA/DFBL, 2354 Fairchild Drive, US Air Force Academy, CO 80840-6228. Correspondence may also be sent via electronic mail to [e-mail].

This research was supported financially by a United States Air Force Research Laboratory grant awarded to the Department of Behaviorial Sciences and Leadership at the United States Air Force Academy. The authors thank Jennifer Clarke and Rylan Charlton for their valuable inputs.

Table. Summary of feedback themes

Feedback Theme   Summary

Purpose          Distinguish between administrative (OPRs, EPRs, etc.)
                   and developmental feedback.
                 Do not link developmental feedback with incentives,
                   promotions, assignments, or penalties.
                 Do not communicate feedback as developmental and later
                   use feedback administratively.

Provider         Understand that different raters often bring
                   particular strengths and weaknesses to the feedback
                   process.
                 Consider the value of giving and receiving 360-degree
                   feedback (self-assessment as well as assessments by
                   superiors, subordinates, and peers).

Preparation      Practice feedback to improve personal levels of
                   effectiveness.

Prevalence       Balance quantity and quality of feedback.
                 Read the situation, and provide the maximum amount
                   of feedback possible, given mission constraints.
                 Make feedback meaningful, not trivial.