Competency-based training: Evidence of a failed policy in training reform.

The political decision to implement competency-based training widely throughout vocational education systems in Australia, before rigorous evaluation through adequate pilot programs, was made because it was viewed as essential for increasing skill levels and work productivity. Recent data indicate that Australia's relative international competitiveness actually declined during 1994-97, suggesting an urgent need to reassess underpinning policies. Despite the marked reluctance of the Australian National Training Authority to commission studies specifically assessing the effectiveness of competency-based training, several independent studies have been carried out. These and other commissioned studies indicate some major problems with competency-based training which has not achieved stated objectives of increasing skill levels. Research also indicates that competency-based training has not been adopted widely by business and industry. The appropriateness of using public agencies to implement innovations which are untested, and may not be supported by the wider community intended to benefit from their introduction, is queried.

Introduction

The first impact of the technological and economic revolutions became apparent in Australia by the late eighties and brought recognition that more effective work and management practices, increased skill levels and better forms of training were required to counter increased international competitiveness and the effects of the globalisation of trade (Cullen, 1997; Industry Task Force, 1995). This led the Australian federal Labor government and the ministers responsible for vocational education and training in the states and territories to embrace competency-based training in 1990 as part of the Training Reform Agenda.

Competency-based training was viewed as the foundation for reform in vocational and postcompulsory education (Beevers, 1993; Smith & Keating, 1997), and the means of increasing skill levels and productivity. Beevers (1993) considers that the particular form of competency-based training adopted was seen `as a universal truth and cure-all for problematic issues such as economic deterioration and workplace restructuring' in addition to providing `equitable access to vocational education and employment as well as the means of constructing the clever country' (p. 89). There were concerted efforts to ensure that newer competency-based training approaches (Hager & Gonczi, 1993) were implemented widely throughout technical and further education (TAFE) systems in Australia, in workplace training and in high schools where trade subjects are being taught. Competency-based training still remains a major plank in federal Coalition government policy with competency-based standards and assessment as essential elements in training packages, although the concept of a Training Reform Agenda has been supplanted (Hawke & Cornford, 1998).

The controversy generated in academic and vocational education circles by the proposal to introduce competency-based training was considerable since it was seen by many as a simplistic solution based on a flawed ideology (see below). Despite claims by Smith (1997), that the debate has died down since `CBT (competency-based training) implementation has been inexorable ... and those "opposed" to CBT have lost the battle and perhaps lost interest' (p. 69), the debate over the effectiveness of competency-based training in raising the quality of skilling in Australia continues unabated (e.g. Foyster, 1997a, 1997b; Ryan, 1997a, 1997b). If anything, the debate has intensified as the focus has shifted from being largely academic to consideration of what has actually occurred in vocational areas where competency-based training has been introduced. The issues have become more serious since ineffective training will have substantial ramifications in the form of skill shortages and further reduction in international competitiveness (see Cullen, 1997).

Almost a decade after the commencement of efforts for wider implementation of competency-based training, there is now a greater opportunity for gaining some perspective of what has resulted from its introduction. Competency-based training was introduced on the basis of apparent advantage, as perceived by a number of influential groups (see below), in the absence of any substantial empirical evidence as to its effectiveness in attaining desired goals. More recent studies, involving practical implementation, enable assessments about the effectiveness of competency-based training, and `whether the demands of policy makers are actually workable on the ground' (Collins, 1993, p. 12). It would appear important to submit to intense scrutiny again what was regarded as one of the underpinning policy initiatives to increase international competitiveness since the comparative ratings indicate that Australia's relative performance has actually declined between 1994-97 (Cullen, 1997).

Although it has been argued that it is very difficult to measure competency-based training effectiveness because of differing views about what it involves (Smith & Keating, 1997, p. 111), it is argued here that improved skill performance in teaching-learning contexts operating under competency-based training and adoption of competency-based training by industry are the key elements. Both these elements relate directly to improved productivity and work performance. It is considered by the researcher that ultimately the success or failure of competency-based training rests upon pragmatic considerations, particularly whether it has been comprehensively implemented and whether it meets stakeholder needs.

The diverse attempts to define competency-based training, lack of agreement on definition and the problems that this causes for implementation are problematic background issues. Although important, these specific issues are not explored further in this article, as they have already been well analysed previously (e.g. see Smith & Keating, 1997, pp. 101-108). This lack of agreed definition however is central to one of the main arguments advanced in this article, since it may be seen as symptomatic of the ideological and political nature of a paradigm implemented before there was any clear conception of what it involved (Raggatt, 1997). Whether such a system, lacking in conceptual clarity, can work in practice was recognised as a key question relatively early (Collins, 1993; Jackson, 1993). This article examines whether competency-based training is working in practice and producing superior skill performance. In the next section, the ideological nature of the competency-based training policy framework and the nature of related research commissioned by government agencies are briefly explored. Subsequent sections consider the development of competency-based training policies from a historical perspective, what has happened as a result of policy initiatives, empirical evidence that has emerged which indicates degrees of effectiveness achieved in the light of espoused objectives, and the current state of debate over the effectiveness of competency-based training policy.

Politics, ideology and competency-based training

Although Hager (1994, p. 3) argued that personal politics did not seem to indicate the position of individuals in the early debates on the introduction of competency-based training, the introduction of this over other possible approaches to upskilling has since come to be seen widely as political and ideological (Raggatt, 1997; Ryan, 1997a, 1997b; Stevenson, 1995). The decision was political in that there was agreement among a number of key groups with vested interests and little or no consultation with practitioners who had to implement it (Hawke & Cornford, 1998). Even the House of Representatives Standing Committee for Long Term Strategies (1995) accepted that the decision to adopt and implement competency-based training involved political leaders, business leaders and the trade union movement with little consultation with teachers and vocational education experts. Omission of teachers and vocational education experts from the consultation process was a serious miscalculation, since it is the teachers who implement such initiatives and are best able to make pragmatic judgements about the practical issues that are involved. Many concepts which may appear attractive political propositions are unworkable in practice (Cornford, 1997). Furthermore, as McBeath (1995) has pointed out, the history of curriculum innovation and change indicates that no new curricula have been successfully introduced without the support of teachers.

Further evidence of the political, ideological nature of the decision to implement competency-based training can be identified through the absence of research funded by the Australian National Training Authority, and associated bodies clustered around the Department of Employment, Education and Training (now the Department of Employment, Education, Training and Youth Affairs - DETYA), to assess the effectiveness of competency-based training. A major theme of the inaugural Australian Vocational Education and Training Association Conference in February 1998 was the problem of obtaining substantial independent finance for vocational education and training research since the Australian National Training Authority constituted the sole major source with projects advertised closely linked to the Australian National Training Authority agendas (Maglen, 1998). A distinction is drawn here between whether skill levels increase as a result of competency-based training as opposed to research into the extent to which it has been introduced into TAFE settings (Smith, Hill, Smith, Perry, Roberts, & Bush, 1995) or the ways implementation has altered teachers' work (Smith, Lowrie, Hill, Bush, & Lobegeier, 1997).

Failure to conduct research into competency-based training effectiveness in raising skill levels is curious as the Training Reform Agenda was based on achieving effective performance in an era of economic rationalism. As such, lack of such research can be attributed to ideology or blind optimism and is linked to the failure by the Federal Government and the Australian National Training Authority to acknowledge that the paradigm being espoused was an entirely theoretical one. Because of the ideology (see Raggatt, 1997), there were no serious attempts to quantify the effectiveness of this new paradigm or to examine the practical implementation problems of, for example, newer concepts of assessment moving from normative to criterion referenced forms before official attempts at implementation. There were certainly some attempts to pilot programs based upon competency-based training principles, such as with Fabrication and Welding (Heavy) Industries in New South Wales (NSW) TAFE at Orange and Bankstown in 1990. The fact that, in the following year (1991), these were fully introduced statewide (Lidbury, 1995) indicates the relatively poor efforts at substantial evaluation before wider implementation.

Especially curious, and deserving of further research, has been the failure of the Australian National Training Authority to disclose financial data on the investments in competency-based training in an era of economic rationalism purportedly driven by hard-headed realism and cost-benefit analyses. The costs in attempts to develop national curricula were considerable (Beevers, 1993), and relatively few national curricula and agreed national standards emerged, with this approach now abandoned (Hawke & Cornford, 1998). Despite the importance of the issue of the effectiveness of competency-based training, only a handful of studies to date have moved beyond theoretical analyses or the anecdotal in the absence of Australian National Training Authority project funding.

Phases in the debate over competency-based training

There appears to have been a number of different concerns focused upon in the earlier phases of the debate over competency-based training. Studies like that of Watson (1993) engaged in theoretical analyses weighing up the pros and cons and identifying possible sources of problems. Many other articles like that of Jackson (1993) analysed the ideological underpinnings and were not overly impressed by the directions envisioned for workers and society. General disquiet by writers like these appear to have led to attempts by Hager (1994) to attempt a strictly logical defence and justification for the implementation of competency-based training in an article entitled `Is there a cogent philosophical argument against competency standards?'. The form of competency-based training espoused by Hager and Gonczi, which has incorporated cognitive and affective elements (Gonczi, Hager, & Oliver, 1990; Hager & Gonczi, 1993; Heywood, Gonczi, & Hager, 1992), appears to be the form which has received the official Federal Government endorsement as judged by publications by the Australian Government Publishing Service (Cornford, 1997).

In the United Kingdom, whence Australian politicians' and bureaucrats' enthusiasm for competency-based training originated after implementation by Margaret Thatcher's New Right, there is increasing interest in explaining the resurgence of enthusiasm for a previously discarded approach and how a deeply flawed set of policies was conceived and implemented (Raggatt, 1997). In Australia, too, there is growing recognition that competency-based training has been ineffective, although this acknowledgement has not come from official bodies. Foyster (1997a, p. 32) reported that, at the final meeting of the Australian National Training Authority Research Council conference in October-November, 1996 in a straw vote on competency-based training, the balance of votes was `clearly against CBT'. He raised a number of possible reasons for the problems and controversy including: (a) lack of understanding of how competency-based training operates; (b) the experiencing of inadequate implementations of competency-based training; (c) suitability of competency-based training for some areas but not others; and (d) the suitability for some subgroups in the community but not others. Foyster then argued that: `if it proves to be that the third and fourth elements are significant, then the one-size-fits-all strategy which has been adopted is certain to fail ... If, on the other hand, the third and fourth elements are negligible, then it will be necessary to deal with the first two elements ... Effective implementation of the national strategy would then demand appropriate quality control with respect to CBT implementation' (p. 32).

Examination of the effectiveness of implementation of competency-based training for convenience can be divided into two distinct categories. The first of these comprises the issues confronting teachers and teachers' judgements concerning the effectiveness of competency-based training in developing superior skills in specific trades and professions. The second is the degree of implementation of competency-based training in the training community generally and its adoption by business and industry. It is considered that these are essential issues in evaluation of the effectiveness of competency-based training, and lie at the heart of the debate between Ryan (1997a, 1997b) and Van Berkel (1997a, 1997b), a debate largely conducted on the basis of anecdotal rather than empirical evidence.

Effectiveness of competency-based training: Views of vocational teachers

Despite the importance of teachers in curriculum innovation (McBeath, 1995), very few studies have investigated the problems that vocational teachers have faced with competency-based training (Smith, 1997). The two studies by Cornford (1996, 1997) sought the views of teachers in the New South Wales Department of Technical and Further Education from a wide range of trade and professional subjects who were teaching or had taught competency-based subjects. Two distinct groups were surveyed. The first (Cornford, 1997) involved teachers in the first and second years of initial formal teacher education study in the Bachelor of Teaching degree at University of Technology, Sydney. These included country and city teachers from a wide range of TAFE colleges. Because of present NSW TAFE recruitment policies, those surveyed included a large number of teachers with previous part- or full-time teaching experience. Average length of teaching for this group of 72 teachers was 5.2 years with a range from 9 months to 23 years. Average length of industrial experience was 18.6 years with the range from 4 to 35 years.

Since findings are always considered more reliable and valid if similar findings emerge from populations with different characteristics, the same survey questionnaire employed in the first study was used with a different group of teachers. The second study (Cornford, 1996) involved 36 more experienced teachers in the first and second years of the Bachelor of Education degree at the University of Technology, Sydney. All of these teachers had previously completed a Diploma of Teaching (Technical) and had an average of 9.7 years teaching experience with a range from 4.5 to 20 years. This group of teachers had on the average 15.4 years of industrial experience with a range from 4 to 34 years across a wide variety of specialisations. It was considered that the teaching and industrial experiences of these two groups of more and less experienced teachers placed them in a better position to evaluate realistically the skill performance levels achieved by their students under competency-based training, and competency-based training implementation issues, than either academics or bureaucrats. Van Berkel (1997b) has challenged the judgement of educationists, whom he sees as having vested interests and a lack of knowledge or commitment to the needs of business and industry. These teachers can scarcely be considered unaware of either the needs of their students or the demands of industry.

Many similar findings emerged from both the more and less experienced teaching groups (see Cornford, 1996, 1997). Perhaps the most striking finding was that 63.9 per cent of experienced teachers considered that the introduction of competency-based training had hindered or severely hindered students' attainment of skilled performance levels with a further 25 per cent considering that performance levels had not changed. The less experienced teachers held very similar views with 61.7 per cent considering that the introduction of competency-based training had hindered or severely hindered the attainment of skill performance levels by students and 20.6 per cent indicated that levels had not changed. Only 11.1 per cent of experienced teachers and 17.7 per cent of less experienced teachers considered that the introduction of competency-based training had improved skill levels, with written comments indicating that communications, jewellery, and Koori carpentry and joinery programs were areas which had specifically benefited. It was considered significant that none of the respondents in either group of teachers indicated that skill levels had been greatly improved with the introduction of competency-based training. Results from these two studies indicate that, in Foyster's (1997a) terms, competency-based training is not suitable for all occupational or skill areas, but that some respondents considered that some specialised areas had benefited, although this was not verified empirically.

Despite the fact that similar findings have emerged in both studies with subjects with different characteristics (Cornford, 1996, 1997), these two studies have limitations and do not provide a total picture. First, they reflect the views of a relatively small number of teachers who may be more critical on account of being engaged in teacher education courses. It would be desirable to use a larger sample even though, from anecdotal evidence, there are strong reasons to believe that the results do reflect the views of many NSW TAFE teachers. Second, they reflect the opinions of only NSW TAFE teachers. Discussions with those from other states who are engaged in parallel research, however, do suggest findings similar to these for NSW TAFE teachers with only a few specialty areas experiencing general success (see Simons, 1996). Third, these findings only apply to TAFE teachers. It is possible that trainers in business and industry and private providers may have different views, although there is very limited adoption of competency-based training within the private sector (see below). Fourth and last, they do not examine any one subject specialisation in any depth; that is to say a macro approach was adopted in both these studies.

The use of opinions of teachers in Cornford's (1996) study to determine competency-based training effectiveness has been questioned by Smith and Keating (1997) who state: `It is necessary, of course, to treat such opinions with caution, and also bear in mind that skill performance of students could vary depending on which CBT features were being used in the courses' (p. 151). Short of extensive research to establish previous and present levels of student performance, an expensive, intensive and difficult research undertaking likely to stretch over several years (Cornford, 1997), the best way of determining what is occurring is to ask those directly involved, provided that they can make valid judgements. Although different categories of' people, for example politicians, bureaucrats, teachers and academics, may use different bases for judgement, specific focus upon improved learning outcomes would appear to be a logically necessary basis. Without this, subsequent change of behaviour involving increased levels of skill and work performance do not seem possible. What makes the judgements of these teachers particularly important is their length of teaching and industrial experience and the fact that all had had experience in teaching competency-based training subjects. On this basis, and upon expertise in their subject specialisation, they, rather than bureaucrats and politicians, would seem better placed to make valid judgements about whether competency-based training has increased skill levels of students, a major objective of competency-based training.

Another basis for challenging research on the effectiveness of competency-based training is to claim that teachers were hostile to change, especially those in TAFE systems (Smith, 1997; Smith & Keating, 1997). This certainly cannot be levelled at the teachers in Cornford's (1997) study, as all were newly employed TAFE teachers, with the impetus for the study originating in the fact that these teachers were enthusiastic about trying a new approach but were experiencing considerable difficulties in trying to implement competency-based training. The issue of what features may be incorporated into competency-based training which may have influenced judgements is considered in more depth below.

Micro analysis of competency-based training's effectiveness

The Cornford (1996, 1997) studies' findings outlined above involved a macro approach of surveying across a range of professions and trades. A study by Roux-Salembien, McDowell, and Cornford (1996) adopted a micro approach by examining in depth the views of teachers on competency-based training in commercial cookery. Commercial cookery, which has a long history of performance-based assessment, might be considered an ideal occupational area for the introduction of competency-based training. However, this research was undertaken because there was much anecdotal evidence of dissatisfaction with competency-based training among commercial cookery teachers. A total of 82 completed questionnaires were returned from 110 NSW TAFE teachers, giving a return rate of 74.5 per cent. Included in the sample were male and female, city and country, full-time and part-time teachers, with 70.5 per cent of the sample having more than 5 years teaching experience and 53.6 per cent of the group having taught competency-based subjects for a year or more.

Results generally indicated strong dissatisfaction with competency-based training and the way it had been implemented. Issues of student motivation emerged as of very considerable concern. In response to the statement that competency-based training was improving student learning outcomes, 48.7 per cent either disagreed or strongly disagreed and 26.2 per cent were undecided. Some 69.5 per cent of teachers either disagreed or strongly disagreed with the statement that students' motivation was heightened as a result of implementing competency-based training. With the statement that competency-based training allowed students to achieve their full potential, 64.5 per cent either disagreed or strongly disagreed, whereas 69.8 per cent either disagreed or strongly disagreed with the proposition that excellence in students' performance was being encouraged by implementing the competency-based assessment system. Responses to the statement that competency-based assessment of pass/fail causes students' loss of motivation clearly indicated that many thought this was a cause, with 74.2 per cent either agreeing or strongly agreeing.

Foyster's (1997a) proposition that competency-based training may be suitable for some groups, but not others, is put to a severe test with the findings from the Roux-Salembien et al. (1996) study, since commercial cookery is an area which could be considered particularly compatible with competency-based training on account of the fact that outcomes traditionally have been a major focus in teaching and assessment in this specialisation. The reasons for such dissatisfaction appear to reside in difficulties of implementation and a failure to appreciate the importance of teaching history and aesthetic elements which are important in effective skill development in this specialty area (see Cornford, 1997). Additional evidence that competency-based training is not suitable for all groups is to be found in the general lack of efforts to establish competency standards for managers (Australian Mission on Management Skills Report, 1991; Industry Task Force, 1995; Smith & Keating, 1997). Field (1996) has challenged the appropriateness of competency-based training beyond lower levels of skill learning, which indicates that it is largely inappropriate in the learning organisation.

Problems in implementation

Contradictions and confusion abound in competency-based training implementation. This is because there is no agreement on precise definition of competency-based training (Raggatt, 1997; Smith & Keating, 1997, pp. 101-108) or what exactly needs to be implemented to constitute competency-based training in practice (Smith et al., 1995, 1997). This poses major problems for implementation. It also provides a logical escape for competency-based training enthusiasts who can always claim that implementation of alternative combinations of competency-based training elements would have resulted in success when research indicates lack of success with other combinations (e.g. see Smith & Keating, 1997, p. 151).

Smith et al. (1995), examining the degree of implementation of competency-based training courses by TAFE and samples of private training providers in 1994 in different states, proceeded with a working definition and used classification categories involving clusters of increasing stringency. This involved creation of specific categories of competency-based training, as in their Bands SC, SCIR and SCIRA, through inclusion of additional elements at the more restricted, higher levels of categorisation, with Smith et al.'s findings indicating these additional elements were less frequently found in practical implementation. Band SC courses met the criteria of competency-based training with courses based on National Training Board standards and course documentation wholly or mostly in competency-based training format; Band SCIR courses met the working definition of Band SC and additionally involved course monitoring by industry and recognition of prior learning procedures; and Band SCIRA included all the criteria for Band SCIR plus requiring indication that assessment was available on demand and at least partly in the workplace. For the purposes of this section of this article, the requirements of Band SC are considered to constitute competency-based training in accordance with Smith et al.'s judgement.

Smith et al. (1995) found that, on 1994 data, 29 per cent of TAFE courses and 39 per cent of non-TAFE courses were in Band SC. With Band SCIR, 7 per cent of TAFE courses were included in this category, whereas 20 per cent of non-TAFE courses were. Only 1 per cent of TAFE courses qualified as Band SCIRA, with 8 per cent of non-TAFE courses also meeting these criteria. This, as the researchers pointed out, means that the majority of courses could not be described as competency-based and that targets set by ministers for substantial implementation by the end of 1.993 had not been met. Although numbers of courses had become competency-based by 1998 if using a wide interpretation of what competency-based training involves, the exact percentage in TAFE and private providers is difficult to determine exactly. Smith (1997) claims that competency-based training has been widely implemented and, although this appears true, the percentage of courses converted to or developed anew as competency-based may not be great, as indicated by her earlier research (Smith et al., 1995).

In more recent research surveying vocational teachers in different specialisations, Smith et al. (1997) examined the way in which the work of vocational teachers had changed with the :implementation of competency-based training. Changes are to be expected since there have been substantial efforts to ensure that competency-based training, closer links between providers and industry, modularisation and flexible delivery, in combination, have been introduced across Australia through policy, legislative and financial incentives or dictates (Hawke & Cornford, 1998). Although descriptions of how teachers are coping with the changes are of some interest, Smith et al. appear to have undertaken a commissioned project which is seriously flawed. Establishment of standards against which to judge practices researched would appear to be important and logically necessary to make valid and worthwhile research judgements, since different practices and procedures may be effective or ineffective, yet such considerations are missing from the study and did not constitute part of the commissioned brief. Indeed Hager (1995; Hager & Gonczi, 1993) has indicated that one of the major problems is the co-existence and competition between newer forms of competency-based training incorporating cognitive and affective elements and an inappropriate, older form of essentially behavioural competency-based training. Distinguishing between these different forms would appear to be important if Hager's arguments are accepted. Yet absent from Smith et al.'s (1997) study is any serious consideration of the effectiveness of competency-based training and what better forms of teaching practice are necessary to achieve effective competency-based training outcomes.

Given the degrees of confusion that lack of adequate definition creates, quite apart from specialisation differences, and regional and state variations, it is not surprising that Smith et al. (1997) found that many teachers were not very confident that what they were implementing was in fact competency-based training or were appropriate competency-based training processes. But, in arriving at their general judgements, Smith et al. appear to have side-stepped the problem of judgement of effective and ineffective competency-based training processes, by assuming a wide range of practices resulted in beneficial outcomes in the absence of empirical evidence that this was so. Other studies, however, have indicated that a number of the processes adopted for the implementation of competency-based training have created major problems for teachers seeking effective learning outcomes under it.

Stevenson (1997, p.viii) summarised the findings of Cornford's (1997) examination of teachers' experiences in implementing competency-based training involving modularisation, assessment and organisational issues as revealing the disaggregation of knowledge, inadequate development of skill and production of low levels of expertise. Similar results emerged from Cornford's (1996) research with more experienced teachers. A finding of particular concern from these studies was that teachers perceived little agreement on assessment standards and that many considered the standards set as too low. This last finding challenges a number of the assumptions underlying the adoption of industry-based competency standards in Australia (see Smith, 1997, p. 70).

Factors undermining effective implementation

An important assumption underpinning competency-based training is that competency standards can be established through analysis of work carried out in business and industry and that there will be agreement about these standards. In fact it has been an exceptionally difficult process to establish national competency standards in many industries (Beevers, 1993). Further, the fact that so many teachers in Cornford's studies (1996, 1997) perceived a lack of agreement in standards, with these teachers all having extensive industrial experience, leads to the conclusion that there is a major problem with the concept of agreed, industry-based competency standards for the following reasons. In reality, in industry, there are three distinct levels of standards in most fields of specialisation: the first is essentially the cheap and barely sufficient, the second involves a middle level of excellence, and the third the deluxe or very superior job involving high level craftsmanship. It depends upon the circumstances and the business organisation as to which standards will be preferred at any one time. The introduction of enterprise bargaining has further eroded any assumption of uniform standards. Now it has become possible for an enterprise agreement to establish quite unique sets of competency standards for workers within the individual enterprise, irrespective of Industrial Training Advisory Board or national standards (Ewer & Ablett, 1996). The variety of positions that TAFE teachers may adopt on this continuum of standards, when faced with less than explicit statements of standards including cognitive and affective elements (Hager & Gonczi, 1993) and/or forced to interpret them, will be dependent upon their past industrial experiences and their conceptions of their role as teachers.

This possible variety of industrial standards stands as a problem quite distinct from the fact the newer version of competency-based training advocated by Hager and Gonczi (1993) includes holistic assessment, and cognitive and affective as well as performance elements. Such departures from the older behavioural standards result in statements of competency standards of greater generality, and thus concomitant problems with interpretation and establishing reliability and validity in assessment (Ewer & Ablett, 1996). Further, it is possible that the newer conceptualisation of competency-based training advanced by Hager and Gonczi (1993) was seriously and grievously flawed from the outset. More recently Hager (1995) has admitted that competency standards are concerned with summative assessment, that is measuring the effectiveness of overall training. Summative assessment and competency-based training standards thus represent the end product and do not reflect the complex processes of learning which lead to this desirable end state of training (Cornford, 1993, 1999), that is, issues of formative assessment (Hager, 1995). Formative assessment, which involves all assessment of learning prior to summative assessment, provides vital feedback necessary for improvement in the (usually) long skill learning process.

It is also apparent that the competency-based training model advanced by Hager and Gonczi lacks a substantial basis provided by research and theory from skill learning, and cognition and development of expertise areas (Cornford, 1993; Stevenson, 1994, 1995). Hence, because of lack ora substantive base for formative assessment, it cannot provide any substantial guidance for those who need to develop curricula over a number of stages of development of learning in whatever specialist field. Nor can it provide guidance for teachers as they assist students with feedback through the development of various skills at varying levels of expertise (Cornford, 1997). The absence of guidance in curriculum development and implementation in the Hager-Gonczi model almost guarantees counterproductive diversity and confusion (Cornford, 1999). In addition, because the newer competency-based training paradigm, which Gonczi and Hager have introduced, lacks clear conceptualisation of formative skill stages and learning processes, it cannot effectively supplant the earlier, inappropriate, behavioural competency-based training paradigm (Cornford, 1997).

Acceptance and implementation of competency-based training by business and industry

Governments are at liberty to enforce new approaches, strategies and philosophies through the public sector which they control, by making policies and financial allocations contingent upon adoption. It has been argued that governments should use public sector utilities in such ways because this provides a model for private industry and a means of disseminating change through such means (Steedman, 1994). This appears to have been accepted as a valid way of effecting change within Australia. However, the issues surrounding governments' `leaning' upon public utilities, over which they have much more control than private business and industry, is starting to receive more attention (see Berliner, 1996; Stasz, 1996). The fact is that governments are often unable to control business and industry because of this sector's inclination to a more hard-headed approach and requirement of evidence of effectiveness or advantage before it will accept and implement innovations.

The issue of whether business and industry have widely accepted and implemented competency-based training is of considerable importance. These groups are the groups which are supposed to benefit most from the development of superior skills through competency-based training. Whether they have widely embraced competency-based training can also be seen as a measure of its perceived effectiveness by groups which are less moved by ideology and more by objective assessment of costs and benefits. What also is of very considerable importance is the fact that current government policy for training and accreditation has centred assessment in the workplace around use of competency standards (Hawke & Cornford, 1998); thus the amount of training committed to under these circumstances may reflect in part business enthusiasm for competency-based training. Figures reveal that business investment in training has been declining over several years (Fooks, 1998) and this may indicate business disenchantment with competency-based training as reflected in FitzGerald's (1994) earlier findings.

Results of research attempting to quantify the adoption of competency-based training by business and industry paint a picture of little real enthusiasm for it, despite claims of the benefits by those like Van Berkel (1997a, 1997b). Research by the Australian National Training Authority in 1995 indicated that only 44 per cent of employers with recent vocational education and training graduates had even heard of competency-based training (Foyster, 1997b). As Foyster argued, those who did not have recent vocational education and training graduates would not have heard of it and even of the 44 per cent presumably not all approved of it. Additionally Foyster estimates that only 10 per cent of employers regard themselves as having a good or very good knowledge of competency-based training, with no real indication of how many of these approved of it. If restricted to large employers, a minority source of employment in Australia, 90 per cent of these employers consider they have a good or better understanding of competency-based training. In an important report on competency-based training, sampling chiefly larger organisations, FitzGerald (1994) concluded that there was `evident widespread industry acceptance of, and support for, training based on competency'. However, from interviews and discussions with firms, `it was apparent that this support for the concept does not necessarily translate into support for the reforms as implemented' (FitzGerald, 1994, p. 20). FitzGerald identified several key concerns of business:

   * There is conceptual confusion about exactly what constitutes a
   competencybased system leading to a failure to consider adequately
   curriculum, delivery and assessment implications.

   * The process of developing competency standards has become far more
   complicated and resource intensive than initially thought.

   * Industry standards are being developed by bodies unrepresentative of key
   enterprises in those industries.

   * The standards development methodology is too task oriented, detailed and
   prescriptive and consequently reduces enterprise and training provider
   flexibility and promotes uniformity rather than diversity in delivering
   training. (p. 20)

More recent research by Pickersgill (1996), using a macro approach, and Lidbury (1995), involving a micro approach, provides more substantial empirical evidence concerning lack of adoption and use of competency-based training by business and industry.

Use of competency standards

Pickersgill (1996), of the Australian Centre for Industrial Relations Research and Training (ACIRRT) at the University of Sydney, was contracted by the Competency Standards Body -- Assessors and Workplace Trainers to examine the use of competency standards for assessment in the workplace by purchasers of these materials. In this study, a stratified random sample was drawn from invoices held by the Competency Standards Body and telephone interviews were conducted using both closed and open questions. The sample represented approximately 13 per cent of that population and covered 18 industry areas defined by Industry Training Advisory Board coverage, including Building and Construction, Food Processing, Light Manufacturing, Tourism and Hospitality, TAFE, Skillshare, Consultants and others.

Findings were that purchasers of the standards represented all Industry Training Advisory Board defined industry areas. In terms of proportions, the highest numbers from an industry area were Public Administration (17 per cent), Community Health and Services (11 per cent), and Finance, Banking, Insurance and Office Skills (6 per cent). Training providers and consultants represented a significant proportion of purchasers with 24 per cent being `other providers', that is, together with 16 per cent TAFE and 8 per cent of consultants, giving a total of approximately 48 per cent of total purchasers. Inspection of individual responses revealed strong public sector involvement. Further coding revealed that 37 per cent were drawn from the private sector, comprising 20 per cent from private firms and 17 per cent private providers, trainers, individuals and consultants. Of the 64 per cent from the public sector, 25 per cent represented public sector training (e.g. TAFE, tertiary, schools and Skillshare), 20 per cent represented public sector organisations (e.g. government departments, government business enterprises, community sector), and 19 per cent represented representative bodies (Industry Training Advisory Boards, peak bodies, national/statewide organisations). (Percentages are rounded, hence some totals amount to more than 100 per cent.) The public sector rose from 64 per cent to 70 per cent and the private sector fell from 37 per cent to 30 per cent, when those not using or intending to use the standards were excluded (Pickersgill, 1996, pp. 7-8).

Organisational size was found to be important, with approximately 65 per cent of purchasers coming from organisations with more than 50 employees. Approximately 35 per cent of purchasers came from organisations with fewer than 50 employees. Respondents from organisations with fewer than 20 employees were individuals, consultants or representing co-ordinating/administrative bodies such as Industry Training Advisory Boards. Organisations which purchased standards were likely to have policies on assessment and training, with these as formal, written policies. On rating of the standards for usefulness, 62 per cent stated that the standards were `very useful', and 38 per cent stated that they were `somewhat useful'. This finding needs to be placed into perspective because many of the purchasers worked in public organisations and/or were actively deriving income from the implementation of competency-based training. Concerns were expressed by numbers of respondents about the difficulty of understanding aspects of the assessment standards and procedures. Pickersgill indicated that some caution is needed in interpretation of findings because workplace assessment is only in an early stage of development and because of the small sample size.

Micro study: Adoption of competency-based training by small business

Lidbury's (1995) study extends understanding of the penetration of small, private industry by competency-based training. Lidbury attempted to evaluate the levels of awareness of competency-based training by Fabrication and Welding (Heavy) industries in the Hunter region of NSW with 30 or fewer employees. The Metals and Engineering industries are widely regarded as the most aware and proactive industry training groups (Ewer & Ablett, 1996). Small businesses are a major source of employment in Australia and are very important in the performance of the economy (Industry Task Force, 1995). As Lidbury noted, companies with 30 or fewer employees contributed the largest proportion of apprentices taught in the Metal Fabrication and Welding (Heavy) Trade courses in the 6 college campuses comprising the Hunter Institute of Technology. Questionnaires were mailed to 30 companies selected at random from the 158 companies with 30 or fewer employees operating in this area, with geographic balance preserved, that is a stratified random sample. A response rate of 63 per cent was achieved. Companies were engaged in structural work (39 per cent), maintenance (36 per cent), mining (16 per cent), and heavy engineering (9 per cent); 95 per cent of companies had an apprentice engaged in competency-based training.

It was found that, whereas 40 per cent of companies claimed to be aware of competency-based training, 90 per cent of respondents indicated that they did not understand the concept, 67 per cent were unaware of the assessment procedures and 78 per cent were not aware of their responsibilities when providing training under the competency-based training system. It was also found that since 1991, when competency-based training was introduced, management had increased training by 32 per cent, but in this same period there had been a decrease in training in trade areas by 55 per cent. The reason for the lack of understanding appears to lie in failure to disseminate relevant material to employers. Sample literature was sent out by the researcher and only one company stated that it had received similar material previously, but 63 per cent of companies indicated that they wanted more relevant literature.

Although there are obvious gaps in our knowledge concerning business adoption of competency-based training, Pickersgill's (1996) study indicates that the public sector is by far the largest user of competency-based assessment, with relatively little adoption by genuinely private business organisations. Lidbury's (1995) study indicates that there have been major problems in dissemination to small business in one of the key industrial areas, Metal Trades, which is generally viewed as one of the most proactive areas in industrial relations and awards (Ewer & Ablett, 1996), and that basic, key concepts are not well understood. Since FitzGerald's (1994) earlier report highlighted, among other things, the problems in conceptualisation and understanding, the findings by Pickersgill and Lidbury should not be too surprising. Overall the conclusions to be drawn are that there has been little adoption of competency-based training by business and industry, there were few government agency attempts to overcome the barriers identified by FitzGerald (1994) and others, and attempts to use public sector bodies as models to encourage private sector adoption have largely failed.

Discussion and conclusions

Although competency-based training has been implemented widely in Australia, empirical evidence from the range of studies reviewed indicates that generally it has not been successfully or extensively implemented in either the public or private sectors. There are wide differences in degrees of implementation in TAFE in different states (see Smith et al., 1995), and in business and industry, empirical evidence indicates very limited adoption of competency-based training (Foyster, 1997b; Lidbury, 1995; Pickersgill, 1996). Where competency-based training has been implemented in TAFE, many of the teachers who have been surveyed in NSW clearly did not believe that it has improved levels of performance except in a very limited range of specialist areas (Cornford, 1996, 1997). The evidence of success in the specialist areas of communications, jewellery and Koori carpentry and joinery was obtained as additional written comments from individuals at the end of questionnaires and remains an untested assertion. It is possible that the novelty of the approach is operating to produce some changes (the Hawthorne effect) or, alternatively, the introduction of standards where none existed, or were not explicit, may be responsible for changes in performance (Cornford & Athanasou, 1995). Results from the Roux-Salembien et al. (1996) study indicate that, even where competency-based training could be expected to be implemented successfully and easily in commercial cookery, there appear to have been serious concerns about the effects of implementation procedures, strategies and motivation of students.

Problems in implementation are certainly causally related to the findings and the lack of effectiveness of competency-based training. However it would be a mistake to see errors in implementation or faulty combinations of different strategies as largely responsible for the outcomes, with these outcomes being largely rectifiable by introduction of different combinations as has been suggested by Smith and Keating (1997). The conceptual confusion that surrounds competencybased training among teachers and business and industry personnel, as identified by FitzGerald (1994), Lidbury (1995) and Roux-Salembien et al. (1996) among others, is clearly a major stumbling block. The fact that there appears no general agreement on the nature of competency-based training or the elements which need to be implemented to achieve success would appear to indicate that we are dealing with a broad, general concept that cannot be easily or adequately defined enough to operationalise it. In any case, in terms of Foyster's (1997a) propositions of the possible need to maintain quality control if there is ineffective implementation, such a proposal for quality control is almost certainly impossible since there appears no agreement possible on what it is that has to be quality controlled!

Hager's (1995) admissions concerning competency standards and summative assessment, and the failure of the paradigm advocated by him and Gonczi (1993) to provide a theoretical basis to support competency-based training curriculum and implementation in formative teaching and assessment, or a means of supplanting the older behavioural paradigm, indicate a shell of a theory rather than a coherent, integrated and practically implementable one. Although this paradigm advanced has been eclectically added to overcome criticisms (see Stevenson, 1995), this has produced more uncertainty since the competency standards incorporating cognitive and affective elements are so general that they constitute major problems for reliable and valid assessment (Ewer & Ablett, 1996). The flawed ideological and political nature of the decision to adopt and implement competency-based training, before the gathering of any substantial empirical evidence as to its effectiveness, becomes only too evident through this analysis. The unrealistic desire for a `quick fix' blinded decision makers to the entirely theoretical and untested nature of the paradigm which was adopted.

In terms of Collins's (1993) question about the ability to fuse disparate elements of competency-based training together to permit policy makers' demands to be workable, the answer is decidedly in the negative. Many of what have come to be seen as key elements of competency-based training (Smith & Keating, 1997), such as recognition of prior learning, still have not been subjected to empirical scrutiny for effectiveness, and criterion-referenced assessment has had to undergo substantial transformations to involve levels or grades to satisfy teachers and business and industry (see Cornford, 1996). Even self-paced learning, which might offer some solutions to the various problems generated by competency-based training and modularisation (Cornford, 1996, 1997; Watson, 1993), is not considered suitable for younger age groups, which comprise a significant proportion of vocational and postcompulsory education, even by some enthusiastic about competency-based training (see Smith, 1997). Ewer and Ablett (1996, p. 202) have admitted that those involved in the union movement who were a major force in the conceptualisation and adoption of competency-based training `were singularly badly qualified to analyse CBT as a pedagogy'.

Although further empirical research is needed to provide a more complete picture of the limited effectiveness of competency-based training, enough evidence has emerged to challenge any notions that there are only minor implementation problems which easily can be surmounted. Perhaps it is now time for those responsible for the continuing implementation of competency-based training to admit that they have implemented a fatally flawed paradigm and to set about serious replanning. Performance-based assessment, of which competencybased training is an off-shoot, is widely understood and offers practica], manageable approaches (Hayton & Wagner, 1998) divorced from the competency-based training ideology. Wide consultation with teachers and other experts involved in practical implementation with any major innovations is important: in any attempt to salvage Australian vocational education systems from this unsuccessful competency-based training implementation, it will be critical. It is a mistake to believe that teachers and experts in the field of vocational education do not wish to see improvement and thus need to be side-stepped as they will prove obstructive (e.g. see Smith, 1997).

To achieve the goal of increased vocational skill levels in Australia, there is a need to take into consideration the development of expertise and cognitive psychology literature which generally have been ignored in competency-based training policy making and implementation (see Cornford, 1993, 1997; Stevenson, 1994, 1995). Only if there is effective use of human resources and integration of research knowledge, with some of this research knowledge resulting from trials of new courses and paradigms prior to wider implementation, will it be possible to develop substantial frameworks to guide teachers in the complex process of curriculum design, and to assist teachers to ensure that students achieve genuinely superior levels of learning and skilled performance (Cornford, 1999).

If, as Beevers (1993), Raggatt (1997) and others suggest, the competencybased training agenda was part of a wider campaign to improve skill development and productivity, it has been a conspicuous failure in Australia in terms of increasing international competitiveness (Cullen, 1997). However, despite its present failure, there is little doubt that, some time in the future, competency-based training again will rise Lazarus-like: its underlying ideal is too tempting. Politicians, bureaucrats and others who desire the `broad vision' and the `meta-narrative' are not inclined to recognise that there must be immense effort to connect ideals with the many intricate steps of careful analysis and planning to convert ideals into real, tangible outcomes. There are a number of valuable lessons for policy makers which should be learned from this experience with competency-based training. It is reasonable to believe that extensive trials in pilot programs would have revealed serious shortcomings and gaps in translating a broad theory into specific strategies and structures for real-life implementation. Such a program of pilot testing may well have saved the Australian taxpayer from the (as yet) undisclosed multimillion dollars of expenditure for policies which have not succeeded. What also needs to be seriously questioned in policy formulation is the use of public agencies as bell-wethers and putative models for implementation by business and industry.

Keywords

competency based     improvement programs   policy implementation
  education
ideologies           policy formulation     vocational education

Acknowledgements

This article is a revised version of a paper presented at the Australian Teacher Education Association Annual Conference and 7th National Workshop on Vocational Teacher Education, Yeppoon, Queensland, 5-8 July 1997.

References

Australian Mission on Management Skills. (1991). Report (Vol. 1). Canberra: AGPS.

Beevers, B. (1993). Competency-based training in TAFE: Rhetoric and reality. In C.

Collins (Ed.), Competencies. Canberra: Australian College of Education. Berliner, D. C. (1996) Uninvited comments from an uninvited guest. Educational Researcher, 25 (8), 47-50.

Collins, C. (1993). Introduction. In C. Collins (Ed.), Competencies. Canberra: Australian College of Education.

Cornford, I. (1993). Theories of skill learning and research into the development of expertise: Some implications for competency-based training. In After competence: The Future of Post-compulsory Education and Training Conference: Conference papers, Vol. 1. Brisbane: Griffith University, Centre for Skill Formation, Research and Development.

Cornford, I. (1996). Experienced teachers' views of competency-based training in NSW TAFE. In Learning & work: The challenges: Con, fence papers, Vol. 4 (pp. 105-115). Brisbane: Griffith University, Centre for Learning and Work Research.

Cornford, I. (1997). Competency-based training: An assessment of its strengths and weaknesses by NSW vocational teachers. Australian and New Zealand Journal of Vocational Education Research, 5 (1), 53-76.

Cornford, I. (1999). Rediscovering the importance of learning and curriculum in vocational education and training in Australia. Journal of Vocational Education and Training, 51, 93-113.

Cornford, I. & Athanasou, J. (1995). Developing expertise through practical training. Industrial and Commercial Training, 27 (2), 10-18.

Cullen, R. B. (1997). Workskills and national competitiveness: External benchmarks. Report No. 2: Benchmarking Australian qualification profiles. A project conducted by Performance Management Solutions for ANTA. Hawthorn, Vic.: Performance Management Solutions Pty Ltd.

Ewer, P. & Ablett, D. (1996). Competency based training: Has it all gone wrong? In C. Selby-Smith & F. Ferrier (Eds.), The economic impact of vocational education and training. Canberra: AGPS.

Field, L. (1996, March-May). Is there room for CBT in the learning organisation? Australian Training Review, 18, pp. 24-25.

FitzGerald, V. (1994). Successful reform. Report to the Australian National Training Authority from the Allen Consulting Group Pty Ltd. Melbourne: Allen Consulting Group.

Fooks, D. (1998). Industry partnership: Leaning on a broken reed. Training Agenda, 6 (3), 15-16.

Foyster, J. (1997a, March-May). Do we still need CBT? Australian Training Review, 22, p. 32.

Foyster, J. (1997b, June-August). Does Australia need CBT? Australian Training Review, 23 pp. 21-25.

Gonczi, A., Hager, P., & Oliver, L. (1990) Establishing competency standards in the professions (National Office of Overseas Skills Recognition Research Paper No. 1). Canberra: AGPS.

Hager, P. (1994). Is there a cogent philosophical argument against competency standards? Australian Journal of Education, 38, 3-18.

Hager, P. (1995). Competency standards -- a help or a hindrance? An Australian perspective. Journal of Vocational Education and Training, 47, 141-151.

Hager, P. & Gonczi, A. (1993). Attributes and competence. Australian and New Zealand Journal of Vocational Education Research, 1, 36-45.

Hawke, G. & Cornford, I. (1998). Australian vocational education policy change: But will the revolution improve the quality of training? Australian and New Zealand Journal of Vocational Education Research, 6 (2), 104-133.

Hayton, G. & Wagner, Z. (1998). Performance assessment in vocational education and train-ing. Australian and New Zealand Journal of Vocational Education Research, 6 (1), 69-85.

Heywood, L., Gonczi, A., & Hager, P. (1992) A guide to development of competency standards for professions (National Office of Overseas Skills Recognition Research Paper No. 7). Canberra: AGPS.

House of Representatives Standing Committee for Long Term Strategies (B. Jones, Chair). (1995). Report of the inquiry into the workforce of the future. Canberra: AGPS.

Industry Task Force on Leadership and Management Skills (D. Karpin, Chair). (1995). Enterprising nation: Report. Canberra: AGPS.

Jackson, N. (1993). Competence: A game of smoke and mirrors? In C. Collins (Ed.), Competencies. Canberra: Australian College of Education.

Lidbury, R. (1995). Level of awareness of competency-based training in the Fabrication and Welding (Heavy) Industry in the Hunter Institute of Technology Region. Unpublished research project submitted for the Bachelor of Education (Technical), University of Technology, Sydney.

Maglen, L. (1998). Keynote address. Inaugural Conference of the Australian Vocational Education and Training Research Association (AVETRA), Sydney, 16-17 February.

McBeath, C. (1995). Overcoming barriers to effective curriculum change: A case study in dissemination practice. In 3rd International Conference on Post-compulsory Education and Training, 21-23 November: Conference papers, Vol. 2. Brisbane: Griffith University, Centre for Learning and Work Research.

Pickersgill, R. (1996). Who is using the competency standards for assessment? Survey of the purchasers of the `Competency standards for assessment'. Clovelly, NSW: CSB -- Assessors and Workplace Trainers.

Raggatt, P. (1997). The politics of competence: A case study in policy. In 5th International Conference on Post-compulsory Education and Training: Conference papers, Vol. 2. (pp. 69-84). Brisbane: Griffith University.

Roux-Salembien, D., McDowell, D., & Cornford, I. (1996). Teachers' perceptions of the introduction of competency-based training in NSW TAFE Commercial Cookery. In Learning & work: The challenges: Conference papers, Vol. 3. (pp. 95-105). Brisbane: Griffith University, Centre for Learning and Work Research.

Ryan, R. (1997a, June-August). Does Australia need CBT? Australian Training Review, 23, 21-25.

Ryan, R. (1997b, September-November). CBT: The final words. Australian Training Review, 24, p. 35.

Simons, M. (1996). `Something old, something new': TAFE teachers' ways of working with CBT. In Learning & work: The challenges: Conference papers, Vol. 3. (pp. 23-31). Brisbane: Griffith University, Centre for Learning and Work Research.

Smith, E. (1997). Does competency-based training mean the end of teaching as we know it? In 5th International Conference on Post-compulsory Education and Training: Conference papers, Vol. 2 (pp. 69-84). Brisbane: Centre for Learning and Work Research, Griffith University.

Smith, E., Hill, D., Smith, A., Perry, P., Roberts, P., & Bush, T. (1995). The availability of competency-based training in TAFE and non-TAFE settings in 1994. Canberra: AGPS.

Smith, E. & Keating, J. (1997). Making sense of training reform and competency based training. Wentworth Falls, NSW: Social Science Press.

Smith, E., Lowrie, T., Hill, D., Bush, A., & Lobegeir, J. (1997). Making a difference? How competency-based training has changed teaching and learning. Wagga Wagga: Charles Sturt University.

Stasz, C. (1996). Workplace skills in practice: Understanding the new basic skills. Keynote address at the 4th Annual International Conference on Post-compulsory Education and Training, Surfers Paradise, 2-4 December. In Learning & work: The challenges. Brisbane: Griffith University, Centre for Learning and Work Research.

Steedman, H. (1994). Assessment, certification and recognition of occupational skills and competences. European Journal of Vocational Training, 1, 36-42.

Stevenson, J. (Ed.). (1994). Cognition at work: The development of vocational expertise. Adelaide: NCVER.

Stevenson, J. (1995). The political colonisation of the cognitive construction of competence. Journal of Vocational Education and Training, 47, 353-364.

Stevenson, J. (1997). Editorial. Australian and New Zealand Journal of Vocational Education Research, 5 (1), vii-ix.

Van Berkel, C. (1997a, September-November). CBT: The final words. Australian Training Review, 24, pp. 34-35.

Van Berkel, C. (1997b, June-August). Does Australia need CBT? Australian Training Review, 23, pp. 21-25, 35.

Watson, A. (1993). Competency-based vocational education and training in Australia: Some unresolved issues. Australian and New Zealand Journal of Vocational Education Research, 1, 93-125.

Dr Ian Cornford is a Senior Lecturer in the Faculty of Education, Markets Campus, University of Technology, Sydney, PO Box 123, Broadway, New South Wales 2007.