Content, concurrent and construct validity of the leadership skills inventory.

The Leadership Skills Inventory was designed to assist students at the upper-elementary, secondary, and post-secondary levels in analyzing the strength of their leadership skills. This study was a subsidiary analyses of the research of Edmunds & Yewchuk (1996) to examine the content, concurrent and construct validity of the LSI. The results of a critical semantic comparison indicate support for content validity. Concurrent validity was supported statistically significant positive correlations between the LSI and all other leadership indicators. Interestingly, all nine categories of the LSI had statistically significant positive correlations with each other. Factor analysis revealed that the LSI contained only one factor, not nine as the separate categories infer. This supports the construct validity of the LSI, but as having only one construct, not nine as the categories suggest. There appears to be merit in using the LSI as a leadership indicator. However, further research is needed to improve its psychometric qualities.

Since Marland (1972) defined leadership as a component of giftedness and advocated its place in gifted education, researchers have stated the need for instruments to identify potential leaders (Karnes & Meriweather-Bean, 1991), to determine the efficacy of leadership training (Kames, Meriweather & D'Ilio, 1987), and to enhance our understanding of leadership as a construct (Feldhusen, 1994). Despite the endeavors of many researchers (Cattell & Cattell, 1975; Karnes, et al, 1984; Renzulli, Hartman & Callahan, 1971), an adequate comprehensive indicator of leadership ability does not appear to exist (Hollingworth, 1939; Kames, Chauvin & Trant, 1984; Otey, 1978; Plowman, 1981; Stutzman & Jawetz, 1982). The indicators that do exist appear to have been derived from differing research bases and the inter-relationships among these indicators have not been examined (Friedman, Friedman & Van Dyke, 1984; Karnes & Meriweather-Bean, 1991; Sisk, 1985). Thus, a comprehensive and proven measure of leadership ability has long been wanting, whether that be in business (House, 1971; McGregor, 1960; Stutzman & Jawetz, 1982) or education (Chemers & Rice, 1973; Hollingworth, 1939; Otey, 1978; Plowman, 1981). Despite numerous articles and texts about leadership traits, programs, teaching methods and the value of leadership training, a detailed search for empirical research on leadership indicators revealed only nine such studies since 1980 and only three since 1993.

As a result of this search it appears that the four prevalent indicators of leadership are; pencil and paper tests (Kames & Chauvin, 1986); various forms of election, nomination or ranking (Kitano & Kirby, 1986); observed actual leadership behavior (Friedman et al, 1984); and past leadership behavior (Bass, 1981; Stogdill, 1974).

The Leadership Skills Inventory

The Leadership Skills Inventory (LSI) (Kames & Chauvin, 1985) is a 125-item, Likert-type measure in which individuals report their strengths and weaknesses in nine leadership categories: Fundamentals of Leadership; Written Communication Skills; Speech Communication Skills; Values Clarification; Decision Making Skills; Group Dynamic Skills; Problem Solving Skills; Personal Development Skills, and Planning Skills. While some view the LSI as having considerable merit for measuring leadership (Feldhusen, 1994), others have raised concerns about its content, concurrent, and construct validity (Eckart, 1988; Kerr, 1988; Lee, 1988).

Several studies have sought to establish various forms of validity for the LSI. Karnes & Chauvin (1985) reported that content validity was established by having a group of adults working in the development of leadership and a group of youths from leadership organizations review and revise the 125 items contained in the nine categories. No other content validity data is reported. Kames and D'Ilio (1988a) attempted to determine the concurrent validity of the LSI by examining "the extent to which students' ratings of themselves corresponded to their teachers' ratings of them on the same inventory". In this case, determining concurrent validity is asking if the test score can be substituted for the teacher rating. There were no significant differences between student LSI scores and the ratings of two teachers experienced in leadership. However, there were significant differences between student LSI scores and two teachers without leadership experience.

The authors concluded that the LSI is best used by experienced teachers familiar with its use but do not comment on the concurrent validity of the LSI. Given the evidence, it does not appear that the concurrent validity of the LSI was established. In an assessment of criterion-related validity, Karnes and D'Ilio (1988b) compared the LSI scores of students who had recently completed a one-week Leadership Studies Program (Kames, Meriweather & D'Ilio, 1987) with criterion scores of acknowledged community leaders. The students earned scores which suggested they had concepts and skills similar to those of the community leaders. Thus, the authors concluded that the results offered support to the criterion-related validity of the LSI. However, this appears to be a generous conclusion given the immediacy of the leadership training the students had received. This writer also notes that the Leadership Studies Program was designed and developed by the authors from the same literature base as the LSI and that the program explicitly teaches the skills and concepts included in the LSI.

Karnes, Meriweather and D'Ilio (1987) used the LSI as a pre/post instrument to determine if leadership training resulted in improved leadership effectiveness. One hundred and thirteen students in Grades 6 to 11 completed the LSI prior to participating in a Leadership Studies Program (Kames, Meriweather and D'Ilio, 1987) and again at its conclusion. The authors attributed the students' increased in mean LSI scores to the intervention of the Leadership Studies Program. However, this writer contends that this is an optimistic conclusion as both the instrument and the training program were derived from the same literature review and contain the same nine essential components. This suggests they were testing the program, not independently assessing leadership skills as implied by the LSI. In addition, the study had no way of controlling for students who did not learn anything in the program yet decided to purposefully increase their LSI self-assessments. Thus, despite the research to date, it appears that the LSI, as a measurement of leadership abilities, needs to be validated.

This study was designed as a subsidiary analyses of the Edmunds & Yewchuk (1996) data to determine if it provided further information regarding the content, concurrent and construct validity of the LSI, psychometric qualities that has been questioned by several researchers.

Method

The sample consisted of 90 academically gifted Grade 12 students from an urban magnet high school for academically capable students. The average grade scores for the 45 girls and 45 boys was 81% and the average IQ was 126. No students had previously participated in leadership training.

Edmunds & Yewchuk (1996) reported the other prevalent indicators of leadership as being actual leadership behavior, nomination and past leadership behavior. Brief descriptions of these leadership indicators and their respective scoring mechanisms are provided here. However, for full details see Edmunds (1993). The scoring of a task demanding actual leadership behavior has been shown to be an accurate indicator of leadership (Bass, 1981; Friedman et al, 1984; Stogdill, 1974). The actual leadership task required that the students generate creative ideas, demonstrate their leadership ability, and indicate their commitment to participate in a leadership activity. Its unique design and scoring mechanism was a modification of the model used by Friedman, Friedman and Van Dyke (1984) and is consistent with Renzulli's (1979) three-ring conception of giftedness.

The creative component required the students to generate innovative topics for Expanded Learning Curriculum (ELC) (Edmunds, 1989), an enrichment method which uses community experts to enhance the curriculum. Students were given 10 minutes to develop a list of as many uniquely creative and different topics as possible (see Torrance Test of Creative Thinking Norms Technical Manual, 1974a). The creativity score was derived from each student's combination of fluency, flexibility, and originality scores compiled in the following manner: the number of topics generated minus duplications constituted the fluency score; the number of different categories a student's topics fell under constituted the flexibility score; and originality scores were derived by awarding 2 points for each topic suggested by only one student, 1 point for each topic suggested by only one other student, and no points for topics suggested by three or more students. Fluency, flexibility, and originality were converted to `z' scores and summed to produce the creativity score.

The leadership ability component of this task involved each student being randomly assigned to 4 - 6 member groups which had to reach consensus regarding the rank order of the top eight ELC topics. Once consensus was reached, each student was asked to rate each of the other group members on two five-point scales: how well the individual helped get the job done, and how well the individual helped the group feel good about working together. Averages for each category were computed and both averages were summed to constitute the leadership ability score.

The task commitment score was derived from each student's response to the following item: "I would be willing to devote (0 to 10) hours of work to help prepare and implement the 8 topics selected for the ELC program". The number of hours recorded constituted their task commitment score. Leadership ability and task commitment were also converted to `z' scores and summed with creativity to produce the actual leadership behavior score.

Nomination indicators of leadership have consistently shown to be valid indicators of leadership (Friedman et al, 1984; Jarecky, 1959; Kitano & Kirby, 1986; Renzulli et al, 1971). Self, peer, and teacher nomination scores were derived by asking all students (and four teachers) to rank order their alphabetic class list, including him/herself, as to who would be the best leader of a committee to carry out the 8 ELC topic sessions. Each student received summed self, peer and teacher rank scores. All scores were converted to z-scores

Past leadership behavior is also recognized as an accurate predictor of future leadership ability (Bass, 1981; Kitano and Kirby, 1986; Stogdill, 1974). Each student provided a frequency list of currently or previously held, elected or nominated leadership positions. A panel of education professionals differentiated acceptable from unacceptable leadership positions. Each acceptable leadership position was awarded a score of 1 and the total constituted the past leadership behavior score. Also, each student completed the Leadership Skills Inventory (LSI) (Karnes and Chauvin, 1985).

All students responded to the four leadership indicators as indicated above which produced complete scores on all measures for the entire sample. The LSI was scored according to the manual instructions (Karnes & Chauvin, 1985). The data were examined and educational leadership experts were invited to review all 125 test items.

Results

Three independent reviewers, all experts in educational leadership, conducted a critical semantic comparison of the 125 LSI items to the leadership literature. They found that all test items elicited responses about an individual's leadership abilities, thus, supporting the content validity of the instrument. As noted in the Edmunds & Yewchuk (1996) study, significant positive relationships were found between the scores of the LSI and actual leadership behavior (r = .341, p [is less than] .01) and past leadership behavior (r = .389, p [is less than] .01). This supports the concurrent validity of the LSI. The scores on the LSI also had significant positive correlations with nomination indicating further support for concurrent validity. However, this is a tenuous conclusion as nomination scores may have been affected by varying small group sizes.

Interestingly, all nine categories of the LSI had significant correlations with each other (r = .295 to .761, p [is less than] .01). The data in Table 1 clearly indicate that the nine categories have more in common than previously portrayed by the literature.

Pearson Correlation Coefficients of the Nine Components of the
LSI

              L1      L2       L3       L4       L5
L1             -
L2            .53      -
L3            .53    .722      -
L4            .32    .408    .566      -
L5            .44    .449    .587    .599      -
L6            .46    .598    .761    .596    .711
L7            .31    .388    .453    .295    .574
L8            .42    .472    .611    .673    .576
L9            .39    .439    .574    .408    .604

              L6     L7       L8       L9
L1
L2
L3
L4
L5
L6              -
L7            .48     -
L8            .62    .298      -
L9            .65    .550    .602     -

All correlations significant at p <. 01 N = 90

Table 1

Furthermore, factor analysis of the nine categories (StatView 512) revealed that the LSI contained only one factor. The factor loadings and eigen values for each category are listed in Table 2.

Component                     Factor loading

1) Fundamentals of Leadership    .636
2) Written Communication         .733
3) Speech Communication          .856
4) Values Clarification          .718
5) Decision-Making               .817
6) Group Dynamic Skills          .876
7) Problem-Solving               .625
8) Personal Development          .780
9) Planning                      .769

Eigen values of Factors
                               Magnitude

Factor 1                        5.219
Factor 2                         .871
Factor 3                         .85
Factor 4                         .536
Factor 5                         .497

Table 2

This result, in addition to the content and concurrent validity evidence previously noted supports the construct validity of the LSI. However, this result indicates that the LSI examines only one construct of leadership, not the nine suggested by the authors. Despite the pattern of factor loadings, there is no suggestion in the literature that these categories should be related in this or any other manner. The authors indicate that "it was determined that the skills necessary to be a leader fell into nine categories" (p. 5) but neither the LSI administration manual nor the literature provide evidence that the nine categories are separate factors or domains. These results indicate that the differences portrayed by the descriptors of the nine categories are semantic and do not reflect empirically separate domains.

Discussion

The results of this study appear to support the content, concurrent and construct validity of the LSI. The implications of these results are: that the LSI appears to validly measure a component of leadership that is different from other indicators; it appears useful as a measure of leadership but not as a means to identify separate categories of leadership; and the LSI could be used with other measures to comprehensively identify leadership ability, a notion that is widely supported (Sisk, 1985; Karnes & Meriweather-Bean, 1991).

It is evident that further research is needed to refine the composition of the LSI items to improve the instrument's psychometric qualities. A possible method might be to use a much larger sample and factor analyze all 125 items to solidify the universe of the items in the domain of leadership. This would also strengthen its content validity, a psychometric quality founded in the semantic value of the content, because at this point, the categories exhibit merely, semantic, not empirical differences. Further research into assembling a comprehensive leadership assessment battery is also warranted as research in this domain may consolidate the differing research bases from which the various indicators have been derived and examine the inter-relationships among them.

REFERENCES

Bass, B. M. (1981). Stogdill's Handbook of Leadership: A Survey of Theory and Research. New York: The Free Press.

Cattell, R. B. & Cattell, M. D. (1975). Handbook for the Junior-Senior High School Personality Questionnaire. Champaign, IL: IPAT.

Chemers, M. & Rice, M. (1973). Leader LPC training and effectiveness: An experimental examination. Unpublished manuscript. University of Utah, Department of Psychology.

Eckart, J. A. (1988). Review of the Leadership Skills Inventory. In D. J. Keyser and R.C. Sweetland (Eds.). Test Critiques, Vol. VI. Kansas City, MI: Test Corporation of America.

Edmunds, A. L. (1993). Leadership Indicators in Gifted Grade Twelves. Unpublished Doctoral Dissertation. Department of Educational Psychology, University of Alberta. Edmonton, Alberta, Canada.

Edmunds, A. L. (1989). Expanded learning curriculum: The use of community professionals to enrich gifted programs. Paper presented at the Annual Conference of the Gifted and Talented Education Council, Alberta Teachers' Association. (Edmonton, Alberta: April 14-15).

Edmunds, A. L. & Yewchuk, C. R. (1996). Indicators of leadership in gifted grade twelve students. Journal of Secondary Gifted Education, 7(2), 345-355.

Feldhusen, J. F. (1994). Leadership curriculum. In J. VanTassel-Baska (Ed.), Comprehensive Curriculum for Gifted Learners. Boston, MA: Allyn & Bacon. (pp. 347-365)

Friedman, P. G., Friedman, R. J. & Van Dyke, M. (1984). Identifying the leadership gifted: Self, peer, or teacher nominations? Roeper Review, 7(2), 91-94.

Hollingworth, L. S.. (1939). What we know about the early selection and training of leaders. Teachers College Record, 40, 575-592.

House, R. J. (1971). A path-goal theory of leadership effectiveness. Administrative Science Quarterly, 16, 321-338.

Jarecky, R. (1959). Identification of the socially gifted. Exceptional Children, 25, 415-19.

Karnes, F. A. & Chauvin, J. C. (1986). The leadership skills: Fostering the forgotten dimensions of giftedness. Gifted Child Today, 9(3), 22-23.

Karnes, F. A. & Chauvin, J. C. (1985). Leadership Skills Inventory: Administration Manual and Manual of Leadership Activities. East Aurora, NY: D.O.K. Pub.

Karnes, F. A., Chauvin, J. C. & Trant, T. J. (1984). Leadership profiles as determined by the HSPQ of students identified as intellectually gifted. Roeper Review, 7(1), 46-48.

Karnes, F. A., & Meriweather-Bean, S. (1991). Leadership and Gifted Adolescents. In M. Bireley and J. Genshaft (Eds.), Understanding the Gifted Adolescent. New York: Teachers' College Press, Columbia University. (pp. 122-138)

Karnes, F. A. & D'Ilio, V. (1988a). Assessment of the concurrent validity of the Leadership Skills Inventory with gifted students and their teachers. Perceptual and Motor Skills, 66(1), 59-66.

Karnes, F. A. & D'Ilio, V. (1988b). Assessment of the criterion-related validity of the Leadership Skills Inventory. Psychological Reports, 62, 263-267.

Karnes, F. A., & Meriweather, S. & D'Ilio, V. (1987). The effectiveness of the leadership studies program, Roeper Review, 9 (4), 238-240.

Kerr, B. A. (1988). Review of the Leadership Skills Inventory. In BRS (#1012-129) update of J. V. Mitchell Jr. (Ed.). The Ninth Mental Measurements Yearbook., Vol. H. Lincoln, Nebraska: University of Nebraska Press.

Kitano, M. K. & Kirby, D. F. (1986). Gifted Education: A Comprehensive Review. Boston: Little, Brown and Company.

Lee, S. W. (1988). Review of the Leadership Skills Inventory. In BRS (#1012-129) update of J. V. Mitchell Jr. (Ed.). The Ninth Mental Measurements Yearbook. Vol. H. Lincoln, Nebraska: University of Nebraska Press.

Marland, S. P. (1972). Education of the Gifted and Talented, Vols. 1 and H. Report to the Congress of the United States by the U.S. Commissioner of Education, Washington, DC: U.S. Government Printing Office.

McGregor, D. (1960). The Human Side of Enterprise. New York: McGraw-Hill.

Otey, J. (1978). Identification of gifted students. Psychology in the Schools, 15, 16-21.

Plowman, P. D. (1981). Training extraordinary leaders. Roeper Review, 3(3), 13-16.

Renzulli, J. S. (1979). What makes Giftedness? A re-examination of the definition of the gifted and talented. Ventura, CA: National/State Leadership Training Institute on the Gifted and Talented.

Renzulli, J. S., Hartman, R. K. & Callahan, C. M. (1971). Teacher identification of superior students. Exceptional Children, 38(3), 211-214 and 243-248.

Sisk, D. A. (1985). Leadership development: Its importance in programs for gifted youth. NASSP Bulletin, 69, 48-53.

Stogdill, R. M. (1974). Handbook of Leadership: A Survey of Theory and Research. New York: MacMillan Publishing Company.

Stutzman, T. M. & Jawetz, K. A. (1982). Early identification using self-assessment in measuring management potential in young scientists and engineers. Paper presented at the Annual Convention of the American Psychological Association. (90th, Washington, DC: August 23-27).

Torrance, E. P. (1974a). Torrance Tests of Creative Thinking: Norms Technical Manual. Bensonville, IL: Scholastic Testing Service.

Alan Edmunds is an assistant professor of Education at St. Francis Xavier University in Nova Scotia, Canada. His current research interests include leadership gifted education, exemplary teachers and cognitive strategy interventions.

Manuscript submitted May, 1997. Revision accepted October, 1997.