Assessing BSW student direct practice skill using standardized clients and self-efficacy theory.
Recently, the construct of self-efficacy has been used in outcome assessment in social work education (Holden, Meenaghan, & Anastas, 2003, 2005; Holden, Meenaghan, Anastas, & Metrey, 2002). This idea is supported by research from the fields of education and counseling that suggest self-efficacy may be a predictor of performance (Larsen & Daniels, 1998; Multon, Brown, & Lent, 1991). In contrast, however, Kruger and Dunning (1999) suggest that students may be "unskilled and unaware," unable to accurately self-assess, with tendencies to over- or underestimate their abilities. Research is needed in social work education to establish whether self-efficacy is predictive of direct practice skill and if self-efficacy mediates the relationship between education and skill performance. This study seeks to examine the relationship among BSW education, self-efficacy, and direct practice skill performance as assessed using standardized clients. Specifically, the following research questions are addressed: Does participation in an accredited BSW program increase the direct practice skills of students? Does participation in an accredited BSW program increase direct practice self-efficacy? Is self-efficacy predictive of direct practice skill performance? And does self-efficacy mediate the relationship between BSW education and direct practice skill performance?
Bandura (1997) defines self-efficacy as "beliefs in one's capabilities to organize and execute the courses of action required to produce a given attainment" (p. 3). Self-efficacy has been shown to positively affect performance across a variety of tasks, including health behavior, academic outcomes, work-related tasks, and counseling behaviors (Bandura, 1997; Holden, 1991; Multon et al., 1991; Stajkovic & Luthans, 1998). Self-efficacy beliefs are felt to impact performance through their influence on people's choice of goals, on the amount of effort expended on the task, on how long the effort will be sustained, and on the mediation of the emotional or affective response relative to the task (Maddux, 1995; Pajares, 1996; Stajkovic & Luthans, 1998).
Four sources of self-efficacy are noted in the literature (Anderson & Betz, 2001; Bandura, 1997; Stajkovic & Luthans, 1998). The most influential source of self-efficacy is enactive mastery experiences. Enactive mastery experiences provide the most authentic evidence regarding one's skills, with successful experiences enhancing self-efficacy and failure decreasing self-efficacy. Vicarious experiences or modeling also serve as an important source of efficacy beliefs. The ability to observe others engaged in an activity allows the observer to make comparative inferences regarding his or her own abilities. Observing people similar to oneself succeed in a task increases self-efficacy, with the opposite effect occurring when observing like persons fail. Verbal persuasion and one's physiological or affective responses (emotional arousal) also influence the development of self-efficacy. For example, lower counseling self-efficacy has been correlated with higher levels of anxiety (Larson & Daniels, 1998).
In recent years, the construct of self-efficacy has been used at both the undergraduate and graduate levels in social work as an outcome measure. In general, self-efficacy significantly increased following the educational or field experience examined, including multimedia child-welfare training (Cauble & Thurston, 2000), gerontology (Bell, Rawlings, & Johnson, 2005), research (Unrau & Grinnell, 2005), hospital social work (Cuzzi, Holden, Rutter, Rosenberg, & Chernack, 1997; Holden, Cuzzi, Rutter, Rosenberg, & Chernack, 1996; Holden et al., 1997) and foundation social work skills (Holden et al., 2002; Holden et al., 2003, 2005). Only one study loosely examines the relationship between practice behavior and self-efficacy within social work. Fortune, Lee, and Cavazos (2005) examined aspects of achievement motivation, including self-efficacy, as predictors of student satisfaction with field and of skill performance (measured by field instructor final evaluation forms). Interestingly, achievement motivation (including self-efficacy measures) was not predictive of student performance as evaluated by the field instructor.
Other than Fortune et al. (2005), none of these studies to date have tested whether self-efficacy within social work education is predictive of skill performance. Additionally, Bandura (1997) notes that tests of causation are more persuasive when they rely on tested mediation rather than presumed mediation, thus providing evidence of dual linkages in the causal process. Additionally, research suggests that students who lack knowledge of practice behaviors may overestimate their abilities, leading to an inflated sense of self-efficacy that does not correlate with their actual performance (Bandura, 1997; Holden et al., 2002; Urbani et al., 2002). Further, when task demands or circumstances under which they are performed are ambiguous, the risk of overestimation of skill increases (Bandura, 1997). Kruger and Dunning (1999) argue the skills needed for competence in a domain are the same ones necessary for accurate self-appraisal, cautioning about the risk of students being "unskilled and unaware." Given these questions, if social work education continues to use self-efficacy as an outcome measure, the next step is to assess, rather than to assume, the mediating effect of self-efficacy on education and performance.
BSW Education Outcomes
Research using direct measures to assess outcomes of BSW education is limited, particularly in the case of direct practice skills. Primarily, existing research on BSW practice outcomes examines how BSW students compare with students in an MSW program who have completed the foundation curriculum. In general, findings suggest BSW graduates are comparable to or exceed foundation MSW students, including such areas as interviewing skills (Carrillo & Thyer, 1994), critical thinking (Clark, 2002), scores on the basic examination for BSW licensing (Thyer, Vonk, & Tandy, 1995-1996), and field instructor surveys of perceptions of student preparedness for practice (Knight, 1993). This comparison, however, may or may not reflect knowledge or skills gained from the BSW program itself, as no data are reported for entering students. Two studies examine the impact of field education on knowledge acquisition, but do not directly examine skill development (Cavazos, 1996; Cavazos & Galvan-Posada, 1999). Riebschleger and Grettenberger (2006) assessed exiting students' attainment of program objectives as rated by field instructors, but no pre-post data were collected. In one of the few studies specific to BSW education using a pre-post design, Waites (2000), using case study responses, found students exiting the program scored higher than entering students. In spite of these encouraging findings that suggest a positive impact of BSW education, studies assessing direct practice skill are few and primarily quasi-experimental, with small samples from single institutions.
Based on social cognitive theory, the following model is proposed. Completion of BSW coursework and field practicum experience leads to higher direct practice skill and higher direct practice skill self-efficacy, through the provision of enactive mastery experiences and vicarious learning opportunities. Higher direct practice skill self-efficacy will have mediating and direct effects on direct practice skill performance (see Figure 1). Based on this model, the following hypotheses will be tested by comparing the direct practice skill performance of students just completing BSW education with students just entering BSW education, examining the construct of self-efficacy for both direct and mediating effects on performance while controlling for effects of social work experience gained outside of the BSW program and intent to enter direct practice. Outside BSW direct practice experience was selected as a control, as experience could contribute to mastery, which Bandura (1997) notes as a primary causal factor of self-efficacy, thus confounding the effects of BSW education. Intent to work in direct practice was controlled, considering students who want to work in direct practice with clients as opposed to macro-level social work could potentially be more highly motivated to perform.
[FIGURE 1 OMITTED]
1. Students who have completed BSW education will have significantly higher direct practice skill self-efficacy than students entering BSW education.
2. Students who have completed BSW education will have significantly higher direct practice skill than students entering BSW education.
3. Self-efficacy will be predictive of direct practice skill performance.
4. Self-efficacy will play a mediating role between BSW education and direct practice skill performance.
For this model, BSW education is defined as an undergraduate program in social work that has been accredited by CSWE, meeting the requirements for curriculum content on social work practice and for field education. Direct practice skill is defined as abilities and techniques necessary to facilitate face-to-face interchanges with individuals or families for the purpose of alleviating distress or for the promotion of well-being. For this study, to build consistency between the self-efficacy and practice measures direct practice skill was conceptualized further as beginning the relationship; exploring; contracting; case management; and the core conditions of warmth, empathy, and genuineness. Direct practice skill self-efficacy refers to the confidence one has about his or her abilities to facilitate face-to-face interchanges with individuals or families for the purpose of alleviating distress or for the promotion of well-being.
This study is a cohort design comparing two groups of students (entering BSW students and exiting BSW students) from a single institution while controlling for outside experience and intent to work in direct practice. BSW students exiting the curriculum were recruited for the study from the Field Practicum Seminar course 3 weeks prior to the end of the final semester in the program. BSW students just beginning the curriculum in the Introduction to Social Work course were recruited within 4 weeks of entering the course. Students who volunteered to participate in the study first completed a questionnaire in class assessing their direct practice self-efficacy and a demographic form, which included items on outside experience and intent to work in direct practice. As an incentive for participation, students from each group were eligible via lottery to win a $50 gift certificate from the university bookstore. All students were provided full informed consent information and signed a written consent form. The study was approved through the university's Institutional Review Board.
Students were scheduled to complete a videotaped interview with a standardized client within 1 week of completing the questionnaire to assess their direct practice skill. Use of standardized clients, although common in fields of medicine, is only just beginning to emerge in social work literature (Badger & MacNeil, 2002; Miller, 2004). The use of a standardized client (actors trained to enact a practice situation) allows for the direct assessment of skill, while exposing all students to the same problem and level of difficulty, allowing for variation in skill to be evident. The standardized client for this study was developed according to guidelines presented in the literature to increase reliability and validity (Collins & Hardin, 1998; Van der Vleuten & Swansen, 1990; Vu & Barrows, 1994). Ladyshewsky (1999) defines reliability as the consistency of the standardized patient's performance over time, and validity as the degree to which the standardized patient accurately portrays the behaviors associated with a real patient or client.
For this study, one person acted as the standardized client for all the videotapes, and was trained in presenting one client vignette. The vignette was designed to reflect casework roles that a BSW graduate might encounter. The standardized client was blind to the level of student. The training videotapes were reviewed for consistency in case presentation by both the actress and the researcher. The study participants were given 15 minutes to conduct an initial intake interview. Immediately after the interview, students were asked to complete written responses to three questions that assessed case management skill. Two independent raters, clinical social workers with experience in supervising students, blind to the level of student, were then randomly assigned to evaluate the videotaped performance and written response of the participants using the direct practice skill measure.
As previously noted, a convenience sample from a single institution was recruited to establish two groups, entering BSW students and exiting BSW students. There were 32 students total, with 16 in each group. Of the sample, 93% were female and 7% were male. Seventy-eight percent of the participants were Caucasian. No statistical tests were completed on group differences for gender and ethnicity because of sparse cell sizes, but gender and diversity were evenly distributed between groups (see Table 1).
One-way analysis of variance was used to evaluate any between-group differences in age and GPA (see Table 2) to further establish group comparability. Groups were not significantly different in GPA (p<.05), but they were in age. Age had a bimodal distribution, with a predominance of 18-year-old students in the entering group and 21- and 22-year-old students in the exiting group. This bimodal distinction is logical given the traditional student population of the university and that the BSW program is typically completed over a 2- to 3-year period.
To assess the direct practice skill variable, the mean score of an 18-item scale was used. The scale was comprised of a 14-item instrument, developed by Chang and Scott (1999) and tested by Pike, Bennett, and Chang (2003), combined with an additional four items that this author developed. This measure was selected because it specifically addresses the direct practice skill areas included in this study of beginning with clients, exploring skills, contracting, and the core conditions of genuineness, warmth, and empathy. Of the items added to the scale, three assessed case management, which was not reflected in the original scale, and the fourth was an item rating overall skill. Each item on the measure was rated on a 5-point Likert scale ranging from 1=ineffective and/or inappropriate to 5=highly effective and/or appropriate (Pike, Bennett, & Chang, 2003). Again, the mean score of the total scale was used for hypothesis testing.
The complete 18-item scale had a Cronbach's alpha of .96. Internal consistency for the subscales was as follows: Beginning Skills ([alpha]=.75); Exploring Skills ([alpha]=.90); Core Conditions ([alpha]=.90); and Case Management Skills ([alpha]=.68). Contracting Skill was assessed by a single item only. The three-item Case Management subscale and the two-item Beginning Skills subscale showed moderate consistency. Because these subscales are comprised of only two to three items, reliability was deemed adequate. The single item for assessing contracting skill may limit its reliability. Interrater reliability, with five subjects randomly selected to be assessed by both raters, was assessed with Pearson's r, r=.64. Interestingly, on the item rating overall performance the correlation was r=.75. Ninety percent of item scores were within one point of each other.
A 37-item self-report measure was used to assess the variable of direct practice skill self-efficacy. The mean score of the total scale was used to test the hypotheses. In an effort to ensure that the self-efficacy measure was consistent with the practice skills measure, direct practice self-efficacy was assessed by combining several existing subscales into a single scale to fully capture the direct social work practice construct (beginning skills, exploring, contracting, case management, and core conditions) as conceptualized in this study. A 15item subscale from the Counselor Activity Self-Efficacy Scales, called the Helping Skill Self-Efficacy scale (Lent, Hill, & Hoffman, 2003), was used to assess the exploring skills aspect of direct practice. Three subscales from the Social Work Self-Efficacy Scale were used (Holden et al., 2002): Supportive Skills, Treatment Plan/Evaluation, and Case Management. This author developed a 5-item subscale on beginning skills reflecting the Chang and Scott (1999) scale anchors, as none of the existing scales adequately reflected this area. The instructions for the total scale read: "Please indicate how confident you are to use each of the following helping skills effectively, over the next week, in working directly with clients." Each item was scored on a Likert scale of 0-9 with 0=no confidence at all and 9=complete confidence. Again, the mean of the complete 37-item scale score was used for hypothesis testing. Subscale mean scores were calculated for group comparison, but not for hypothesis testing. The complete scale assessing social work direct practice self-efficacy demonstrated strong internal consistency ([alpha]=.98), along with the subscales Exploring Self-Efficacy ([alpha]=.90), Beginning Skills Self-Efficacy ([alpha]=.95), Core Conditions Self-Efficacy ([alpha]=.95), Contracting Self-Efficacy ([alpha]=.95), and Case-Management Self-Efficacy ([alpha]=.97). Five times on single items, students circled two responses, (e.g., 5 and 6). In each case the lower of the two numbers was entered. For both scales exploratory principal components analysis was not conducted due to inadequate sample size.
For regression analysis, the variable BSW education was coded as 1=exiting students and O=entering students. The control variable direct practice experience gained outside of the BSW program was calculated by asking students to list any experiences in the past 5 years they had outside of the BSW program that involved direct practice with clients. Direct practice was defined as interactions with individuals, families, or children for the purpose of alleviating distress for the promotion of well-being. Students described the experience by listing the activity, how many hours per month on average they were involved in face-to-face activity, and for how many months. A total hour score was calculated from student responses. Due to kurtosis in the data with a high number of students reporting no prior experience, data was recoded into categories with 0=no prior experience (n=20), 1=less than 100 hours of experience (n=6), and 2=more than 100 hours of experience (n=6) to meet assumptions for regression analysis.
The other control variable, intent to enter social work direct practice, was assessed using a single item. Students were asked to rate their intent to enter direct practice by answering the following: "On a scale of 1-10, note how likely you are to work in direct practice with clients as part of your career as a social worker with 1=do not plan to work in direct practice at all, 5=plan to work some in direct practice, 10=plan to work primarily in direct practice." GPA was also originally considered as a possible control variable. However, GPA was dropped as a control due to high numbers of missing data for the entry-level group.
Prior to the analysis descriptive information, frequency distributions and correlations of the items and variables were calculated to assess for missing data and potential outliers, to test for skewness or kurtosis, and to examine risk of collinearity. ANOVA was used for group comparison. To control for outside experience and intent to work in direct practice, ordinary least squares hierarchical regression techniques were used to test Hypotheses 1, 2, and 3 of the model. Using guidelines laid out by Baron and Kenny (1986), ordinary least squares hierarchical regression was used to test the proposed model of self-efficacy mediating the effect of BSW education on direct practice skill performance (Hypothesis 4).
Prior to analysis all items were reviewed for variance and distribution. All items were in the normal range for skewness (<2.00) and kurtosis (<7.00), except age and prior hours of experience as previously described. Mean scores and analysis of variance are presented in Tables 3 and 4. Mean scores were higher on all self-efficacy and direct practice scales and subscales for exiting students, and initial one-way analysis of variance showed significant differences between groups (entering and exiting) (p<.05) on each of the scales and subscales. Levene's test of homogeneity of variance was calculated for the total scales and subscales. Two subscales were significant for nonhomogeneity (p<.05), Case Management Self-Efficacy and Contracting Skill. Therefore, these scores were compared using an independent groups t tests with equal variances not assumed, rather than analysis of variance. Mean scores for these scales were also significantly higher for exiting students, with Case Management t(22.22)=-6.13, p<.01, and Contracting Skill t(24.58)=-4.78, p<.01.
To further examine the possible effects of control variables, an independent groups t test showed no significant difference (p<.05) between groups (entering and exiting students) on the control variable intent to enter direct practice. There was significantly (p<.01) more outside BSW experience for exiting BSW students than entering BSW students, supporting the need to control this variable in the model. For both variables, homogeneity of variance was conducted with Levene's test. Experience was significant for nonhomogeneity and differences were assessed using t-test scores for equal variances not assumed. Results are summarized in Table 5.
In preparing for regression analysis, correlations using Pearson's r were calculated to determine potential confounding variables and multicollinearity risks. Correlations are presented in Table 6.
As anticipated, BSW education was highly correlated with both direct practice self-efficacy and direct practice skill. However, direct practice self-efficacy and direct practice skill were not significantly correlated. Diagnostic tests did not reflect high collinearity risk (Variance Inflation Factor < 4; Tolerance > .3). Correlation between BSW education and intent was not statistically significant, and intent was not significantly related to either outcome. Interestingly, hours of prior experience was significantly correlated with direct practice skill, but not direct practice self-efficacy. To summarize, BSW education was significantly correlated with direct practice self-efficacy, direct practice skill, and outside BSW experience. Direct practice self-efficacy was not significantly correlated with direct practice skill or the control variables.
In the findings, Hypothesis 1 was supported, with BSW education found to be a significant positive predictor (see Table 7) of direct practice self-efficacy after controlling for intent and outside experience ([beta]=.75, p<.01). BSW education contributed 44% of the explained variance in self-efficacy, over and above the 10% explained by the control variables themselves, for a total explained variance of 54%.
Hypothesis 2 was also supported (see Table 8). BSW education was a significant positive predictor of direct practice skill ([beta]=.63, p<.01). The total model accounts for 54% of the variance, with BSW education accounting for an additional 31% of the variance after controls. Interestingly, the control variables account for 13% more variance in this model for predicting direct practice skill than they did for the model predicting direct practice self-efficacy. In particular, prior to the inclusion of BSW education, outside BSW experience was a significant predictor of direct practice skill, suggesting that experience is related to higher skill as well.
Hypothesis 3 is partially supported, in that self-efficacy is significant in the model but not in the direction anticipated (see Table 9). After controlling for BSW education, self-efficacy was a significant negative predictor, not a positive one as presumed in the model ([beta]=-.41, p<.05), with the total model accounting for 62% of the variance, with self-efficacy adding 8% of the variance in the final step. This suggests that when experience, education, and intent are controlled, those with higher self-efficacy actually have lower performance. Recall that there was no correlation between self-efficacy and skill in the bivariate analysis.
Further, results did not support self-efficacy as mediating between BSW education and direct practice skill (Hypothesis 4; see Table 9), analyzed using criteria for testing mediation by Baron and Kenney (1987). Beta weights for each path in the model are summarized in Figure 2. In this analysis, the model including both BSW education and direct practice self-efficacy explains a significant amount of variance, with each being a significant predictor (p<.05). However, direct practice self-efficacy acted as a negative predictor as previously articulated. The significant point, however, is adding self-efficacy to the model actually strengthened BSW education as a predictor, increasing the beta to [beta]=.94 in Step 3 from [beta]=.63 in Step 2. According to Baron and Kenny, if direct practice self-efficacy is acting as mediator, the beta weight for BSW education should be lower, not higher, when direct practice self-efficacy is added to the model.
To assist in understanding mediation results of self-efficacy, the raw data were further examined. When direct practice skill mean scores were arranged in ascending order, there was no relationship between direct practice self-efficacy scores (see Figure 3). Data were also examined separately for entering students (see Figure 4) and exiting students (see Figure 5) arranged in ascending order by direct practice score.
The data for entering students show the self-efficacy scores of the least skilled students were highly variable, whereas a negative relationship between self-efficacy and skills is evident among the more skilled entering students. Data for exiting students show a fairly clear negative relationship between self-efficacy and skill. As such, exploratory hierarchical regressions were calculated to determine if exiting students' self-evaluations predicted performance better than those of the entering students, after controlling for outside direct practice experience. In this case self-efficacy was not a significant predictor of skill for entering students (R=.18, p=.26, [beta]=-.16, p=.14); however, self-efficacy was a significant predictor for the exiting students, but in a negative direction (R=.37, p=.05; [beta]=-.41, p=.04) as reflected in the raw data.
Direct Practice Skill
Evidence of direct practice skill is critical to the overall assessment of student learning outcomes within social work education. Findings support that BSW education predicts higher direct practice skill in exiting BSW students. It is safe to suggest that educational outcomes in this area were attained. It is difficult, however, to place the author's findings in the context of the existing literature due to the paucity of studies in this area. As noted in the literature review, although studies are limited, knowledge and skill development are positively reflected in previous research. The findings are consistent with that trend, supporting skill attainment of students. However, this study is unique in that it compares entering and exiting students using standardized clients to directly assess skills, while also controlling critical, potentially confounding variables. The study's strengths are the cohort design, with established group comparability, and the pre- and post-intervention (BSW education) design, which controls for testing effects.
The use of performance-based assessment using a standardized client interview, carefully designed according to established guidelines, also strengthens the validity of these findings and presents an exciting option for future assessment of social work skill as compared to reliance on self-report or field education evaluations. Although the use of standardized clients is beginning to get more attention in social work (Badger & McNeil, 2002; Linsk & Tunney, 1997; Miller, 2004), other disciplines have established the reliability and validity of this method as an outcome assessment tool (Collins & Hardin, 1998; Hodges, Hanson, McNaughton, & Regehr, 2002; Wass, Van der Vlueten, Shatzer, & Jones, 2001). The use of a standardized client allows for independent observation of student skill, not biased by instructor or field supervisor relationship, nor based on self-evaluation, while also allowing for a consistent testing environment.
[FIGURE 2 OMITTED]
The challenge in using standardized clients is that they are more complicated and expensive to implement, and further instrument development is needed for student evaluation. Validity questions also need further testing. For example, do students demonstrating higher skill in standardized testing situations also perform at a higher level in the field, and do they impact client outcomes differently than students demonstrating weaker skill? At what level do students need to perform in order to be considered ready to practice? In spite of these questions, the use of standardized clients for assessment of student skill offers an exciting opportunity for social work education.
For self-efficacy, the findings are consistent with prior research in social work (Bell et al., 2005; Holden et al., 1997, 2003, 2005; Unrau & Grinnell, 2005) and supports BSW education predicting higher direct practice self-efficacy. Similar results have been reported in the counseling field where several studies found self-efficacy to be higher among more advanced students; however, other studies found a nonlinear relationship, with the impact of education on self-efficacy being strongest in the initial stages of education (Larson & Daniels, 1998). There is the potential that entering BSW students may have overrated their skills or may be less accurate in their ability to self-assess; regardless, exiting students were significantly higher in their self-efficacy than entering students. In considering this finding, the relationship of experience gained outside the BSW program to self-efficacy, as with direct practice skill, may need further exploration. Stoltenberg (1998) comments on the role of experience as related to counseling self-efficacy, stating, "We believe there is sufficient empirical support for the importance of this variable that necessitates its inclusion in most investigations. To ignore this potential confound may leave any conclusions drawn from these studies uncertain" (para. 13).
[FIGURE 3 OMITTED]
Performance and Mediation Effects of Self-Efficacy
Because of existing evidence that suggests education generally results in higher self-efficacy, it might be presumed that self-efficacy could be used as a proxy outcome measure for skill in social work education. The data in this study suggests caution in making this assumption as in this case self-efficacy did not positively predict performance. As skill increased, self-efficacy decreased. This finding is comparable to results reported by Sharpley and Ridgway (1993) in a pre/mid/post study of students in a counseling skill class where student confidence was not predictive of counseling skill, as evaluated directly using a role-play situation. In fact, midway through the course, it was predictive in a negative direction (they did not, however, test mediation effects).
[FIGURE 4 OMITTED]
[FIGURE 5 OMITTED]
In their review of counseling self-efficacy and counseling performance, Larson and Daniels (1998) report wide variability in self-efficacy scores and performance across several studies. Three of the six studies they reviewed reported correlations ranging from .19 to .42, whereas the other three studies found no correlation between self-efficacy and counseling performance. They conclude that the research findings are suggestive only and further study is needed. Larson and Daniels (1998) also found no evidence that supervisor perception of student skill was correlated with student self-reported self-efficacy. Within social work, Fortune et al. (2005) found self-efficacy was not a predictor of performance, as assessed by field instructor ratings using regular program field evaluations.
Interestingly, when entering and exiting students were compared separately, there was no significant correlation between performance and self-efficacy scores for entering students, but a significant negative correlation existed for exiting students. One can speculate that entering students' self-appraisals were more varied and less accurate, and that as exiting students' skill increased they developed a more realistic appraisal of their skill. It may be that the higher achieving students underrate their performance, but lacking a benchmark for comparing skill and self-efficacy, it may be they also had the most realistic appraisals. In light of this, more study is needed to determine how self-efficacy can or should be used as an outcome measure for social work education. Although previous authors have stated that self-efficacy does not equate with direct practice, when held out as a possible outcome measure we run the risk of making the false assumption that higher self-efficacy equates to higher skill. It is important to be clear that if using self-efficacy as an outcome in assessment self-efficacy is just that, one's beliefs or confidence in one's ability, not ability itself, until research establishes otherwise.
Further complicating the discussion is the unexpected results of the mediation findings for self-efficacy in this study. Caution is needed in interpreting this finding due to the limited sample size, but interesting questions are raised. In reviewing the literature, several possible explanations are suggested as contributing factors for self-efficacy's negative correlation with skill and subsequently not mediating as expected. An ongoing criticism of self-efficacy as an outcome measure is its reliance on self-report. A person's ability to accurately self-assess has been challenged in the literature on assessment (Kruger & Dunning, 1999; Stone, 1994). Kruger and Dunning (1999) argue that the very skills needed to perform in a domain are the same skills needed to evaluate one's own competence in that domain. They argue that the unskilled lack the metacognition to accurately self-assess, leading to overestimation of their abilities. Kruger and Dunning (1999) found that those in the lowest quartile skills group were most likely to overestimate their abilities, whereas those in the highest quartile group underestimated their performance. They concluded that unskilled individuals not only perform poorly, but also fail to realize it. Further, when poor performers were given more training, the accuracy of their self-assessment improved. This seems possible in the author's study as well, where entering students rated themselves fairly high, whereas some of the best performing students rated themselves no higher than average performing students.
The greater experience of exiting students may have impacted the findings in this study as they could be presumed to have more accurate self-appraisal as they have more knowledge, experience, and developmental maturity; whereas entering students, due to their lack of experience, have less accurate appraisals and are at risk of overestimating their skills, contributing to the poor correlation between appraisal and performance. In particular, students who lack sufficient knowledge about a task are at risk to overestimate their abilities (Bandura, 1997; O'Brian, Heppner, Flores, & Bikos, 1997). Holden et al. (2002, 2003, 2005), in several studies, compared student pretest ratings with a retrospective "then" pretest rating taken at posttest, and found that students overestimated their abilities at pretest as compared to their retrospective rating, in which they rated themselves lower. Urbani et al. (2002), in an interesting study, compared student counseling self-efficacy with trained rater assessment of skill both pre- and poststudent counseling training. Interestingly, students had overestimated their abilities at pretraining, but underestimated their abilities at posttest as compared to the trained rater's assessment. The comparison group rated themselves similarly to the students at pretest, but did not increase in self-efficacy at posttest. One might conclude there is a strong risk of overestimating skill when one is unfamiliar with the skill itself. Similarly, once knowledgeable, cautious students may underrate their abilities.
The complexity of the task assessed may also impact the findings. Bandura (1997) notes that self-efficacy is a stronger predictor in situation-domain-specific situations. When tasks are more complex, multifaceted, or involve distal behaviors, weaker relationships occur (Bandura, 1997). Multon et al. (1991), in a meta-analysis of effect sizes of self-efficacy and academic performance, found greater effect sizes on specific basic skills measures, and smaller effect sizes on broader outcome measures such as a standardized achievement test. Performing an initial assessment of a client is a clearly a multidimensional task.
The counterintuitive findings for mediation may also be due to model specification error. It is possible key variables are missing in the model or variables included are not well specified or relevant to the mediation process. First, BSW education may be too general a predictor to capture specific efficacy processes. Similarly, a summary mean score may also not be specific enough. Second, Baron and Kenney (1986) discuss feedback as a source of bias in the meditational chain. The model is a directional one, presuming that the mediator influences the dependent variable. When we consider that enactive mastery experiences are a source of self-efficacy, it may be that direct practice skill in fact predicts self-efficacy, suggesting perhaps an error in model specification.
Generalization of these results that BSW education predicts higher skill is limited in that the data are drawn from a convenience sample from a single institution, and although the assumption is made that accredited programs offer comparable education, further testing is needed using a random sample from multiple BSW programs. For the future, a longitudinal design testing the same students over time is also needed. The present study, although testing comparable cohorts, is subject to risks of effects of sampling. It is possible that students who volunteered for the study had more experience or felt more confident in their abilities, which would bias results. GPA also is an important variable that was not controlled or tested in this study and needs to be included in future research. The role of experience gained outside of the program also warrants further evaluation and study. Yet this study is a step to establishing what skills students have, allowing us to build a foundation for researching which skills contribute to better client outcomes.
In examining the mediation findings, study limitations such as small sample size, measurement error, and sample bias must all be considered. The sample, comprised of 32 subjects, may have been too small to detect a significant effect of self-efficacy and may have distorted mediation findings. A larger sample would also provide a wider range of scores from which to assess mediation. Limitations of the Social Work Direct Practice Self-Efficacy Scale, the self-efficacy measure compiled for use in this study, must also be considered. Although the individual subscales (except the beginning subscale) had been tested for reliability previously, this is the first time they were used together in combination as a single measure of direct practice self-efficacy.
Study Implications and Future Research
The evidence suggests that BSW education contributes to the acquisition of direct practice social work skills, and its role in the education of social workers is supported. From such studies as this, future research can continue to test the professional skills of the BSW student as compared to the skills of foundation MSW students. Further, it begins to lay the groundwork for comparing BSW educational outcomes with related undergraduate degrees, allowing the profession to begin to self-examine how we are or are not unique in the skill preparation of our students. Finally, BSW programs can be compared to determine if particular curricular aspects contribute to greater skill.
Given the recent widespread interest in and use of self-efficacy measures in social work, further testing of the relationship between social work self-efficacy and skill, as well as its role as a mediator, is needed. Testing to see if findings are replicated with larger samples in different contexts is necessary. Researchers testing self-efficacy could consider adding tests of mediation to their research designs. Testing the contextual variables that impact self-efficacy appraisals, such as experience gained outside the program, may also provide useful information for understanding mechanisms and contexts by which self-efficacy may mediate education and skill.
Although self-efficacy may not predict social work direct practice skill, it may be a useful contributor to such things as student motivation, expended effort, and choice of task. These factors are important aspects of professional practice. Further, Fortune et al. (2005) note that higher self-efficacy was correlated with higher satisfaction in field internship experiences. Larson and Daniels (1998) also note counseling self-efficacy is negatively correlated with anxiety. Further, accurate self-assessment is considered an important aspect of professional practice so that professionals are appropriately confident and cautious, persistent and flexible, experimenting and safe, and independent and collaborating (Eva & Regehr, 2005).
Social work educators may want to consider broadening how self-appraisal is approached and understood as a construct within the profession. Eva and Regehr (2005) challenge the health profession's restricted use of self-appraisal as primarily "summative," noting the poor correlation between student self-appraisal and external outcome measures. Referencing the work of Schon, Eva and Regehr suggest instead that health professions increase their focus on students' abilities to reflect in action. Noting the dynamic nature of ongoing reflection, they argue that student ability to self-assess while in action may be more critical than their summative evaluations of their abilities.
Yet until there is evidence demonstrating that higher social work self-efficacy is predictive of skill, researchers should be clear about what self-efficacy measures: confidence in one's ability to use a skill, not competence in the skill itself. If that is the case, the next question we must ask ourselves is, "What is the outcome we want to assess?" CSWE accreditation standards require us to measure competence, not confidence. Given the limited resources available for educational outcome assessment and the questions raised by this study regarding our understanding of self-efficacy as an outcome, it is important that social work develop and expand assessment options for evaluating direct practice skills that do not rely on self-assessment.
Do we prepare students with the skills they will need for professional practice upon graduation? This question provided the impetus for this study. Students must not only graduate with the knowledge and values of the profession, but with professional skills as well. Although both social work education and the profession have called for the assessment of outcomes and evidence that we are preparing students with skill, limited evidence currently exists. Social work education is well poised with its new focus on competencies to further test its professional claims of skill development. This study advances social work education in validating its claims of training and skill in its graduates, and calls us to further examine how we can best assess social work competencies and skill development.
Anderson, S. L., & Betz, N. E. (2001). Sources of social self-efficacy expectations: Their measurement and relation to career development. Journal of Vocational Behavior, 58, 98-117.
Badger, L., & MacNeil, G. (2002). Standardized clients in the classroom: A novel instructional technique for social work educators. Research on Social Work Practice, 12, 364-374.
Bandura, A. (1997). Self-efficacy: Thought control of action. New York, NY: W. H. Freeman & Company.
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173-1182.
Bell, S., Rawlings, M., & Johnson, B. (2005). Assessing skill, attitudes, and knowledge in gerontology: The results of an infused curriculum project [Special issue on gerontology]. The Journal of Baccalaureate Social Work, 26-37.
Bogo, M., Regehr, C., Hughes, J., Power, R., & Globerman, J. (2002). Evaluating a measure of student field performance in direct service: Testing reliability and validity of explicit criteria. Journal of Social Work Education, 38, 385-401.
Carrillo, D. F., & Thyer, B. A. (1994). Advanced standing and two-year program MSW students: An empirical investigation of foundation interviewing skills. Journal of Social Work Education, 30, 377-387.
Cauble, A. E., & Thurston, L. P. (2000). Effects of interactive multimedia training on knowledge, attitudes, and self-efficacy of social work students. Research on Social Work Practice, 10, 428-437.
Cavazos, A. (1996). Measuring BSW student learning in field instruction. Journal of Social Work Education, 32, 389-398.
Cavazos, A., & Galvan-Posada, S. (1999). Foundation knowledge and field instruction: A replication study. Journal of Baccalaureate Social Work, 5(1), 45-59.
Chang, V. N., & Scott, S. T. (1999). Basic interviewing skills: A workbook for practitioners. Chicago, IL: Nelson-Hall Publishers.
Clark, H. G. (2002). A comparison of the critical thinking skills of BSW and MSW students. Journal of Baccalaureate Social Work, 7, 63-75.
Collins, J. P., & Harden, R. M. (1998). AMEE Medical Education Guide No. 13: Real patients, simulated patients and simulators in clinical examinations. Medical Teacher, 20(6), 508-521.
Council on Social Work Education. (2008). Educational policy and accreditation standards. Alexandria, VA: Author.
Cuzzi, L., Holden, G., Rutter, S., Rosenberg, G., & Chernack, P. (1997). The hospital self-efficacy scale: A replication. Research on Social Work Practice, 7, 490-499.
Eva, K. W., & Regehr, G. (2005). Self-assessment in the health profession: A reformulation and research agenda. Academic Medicine, 80 (10 Supplement), 546-554.
Fortune, A. E., Lee, M., & Cavazos, A. (2005). Achievement motivation and outcome in social work field education. Journal of Social Work Education, 41, 115-129.
Garcia, J. A., & Floyd, C. H. (2002). Addressing evaluative standards related to program assessment: How do we respond? Journal of Social Work Education, 38, 369-382.
Hodges, B., Hanson, M., McNaughton, N., & Regehr, G. (2002). Creating, monitoring, and improving a psychiatry OSCE. Academic Psychiatry, 26(3), 134-160.
Holden, G. (1991). The relationship of self-efficacy appraisals to subsequent health related outcomes: A meta-analysis. Social Work in Health Care, 16(1), 53-93.
Holden, G., Cuzzi, L., Rutter, S., Rosenberg, G., & Chernack, P. (1996). The hospital self-efficacy scale: Initial development. Research on Social Work Practice, 6, 353-365.
Holden, G., Cuzzi, L., Spitzer, W., Rutter, S., Chernack, P., & Rosenberg, G. (1997). The hospital social work self-efficacy scale: A partial replication. Health and Social Work, 22(4), 256-263.
Holden, G., Meenaghan, T., & Anastas, J. (2003). Determining attainment of the EPAS foundation program objectives: Evidence for the use of self-efficacy as an outcome. Journal of Social Work Education, 39, 425-440.
Holden, G., Meenaghan, T. & Anastas, J. (2005). EPAS objectives and foundation practice self-efficacy: A replication. Journal of Social Work Education, 4, 559-570.
Holden, G., Meenaghan, T., Anastas, J., & Metrey, G. (2002). Outcomes of social work education: The case for social work self-efficacy. Journal of Social Work Education, 38, 115-133.
Knight, C. (1993). A comparison of advanced standing and regular master's students' performance in the second-year field practicum: Field instructors' assessments. Journal of Social Work Education, 29, 309-317.
Kruger, J., & Dunning, D. (1999). Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of Personality and Social Psychology, 77(6), 1121-1134.
Ladyshewsky, R. (1999). Simulated patients and assessment. Medical Teacher, 21(3), 266-269.
Larson, L. M., & Daniels, J. A. (1998). Review of the counseling self-efficacy literature. The Counseling Psychologist, 26(2), 179-218.
Lent, R. W., Hill, C. E., & Hoffman, M. A. (2003). Development and validation of the counselor activity self-efficacy scales. Journal of Counseling Psychology, 50(1), 97-108.
Linsk, N. L., & Tunney, K. (1997). Learning to care: Use of practice simulation to train health social workers. Journal of Social Work Education, 33, 473-489.
Maddux, J. E. (1995). Self-efficacy, adaptation, and adjustment: Theory, research, and application. New York, NY: Plenum Press.
Miller, M. (2004). Implementing standardized client education in a combined BSW and MSW program. Journal of Social Work Education, 40, 87-102.
Multon, K. D., Brown, S. D., & Lent, R. W. (1991). Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation. Journal of Counseling Psychology, 38(1), 30-38.
O'Brian, K. M., Heppner, M. M, Flores, L. Y., & Bikos, L. H. (1997). The Career Counseling Self-Efficacy Scale: Instrument development and training applications. Journal of Counseling Psychology, 44, 20-31.
Pajares, E (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543-578.
Pike, C. K., Bennett, R., & Chang, V. (2003). Measuring basic interviewing skills: An instrument validation study. Unpublished manuscript.
Riebschleger, J., & Grettenberger, S. (2006). Assessing graduating BSW field students' preparation for generalist practice. Journal of Baccalaureate Social Work, 12(1), 184-202.
Sharpley, C. F., & Ridgway, I. R. (1993). An evaluation of the effectiveness of self-efficacy as a predictor of trainees' counseling skills performance. British Journal of Guidance & Counseling, 21(1), 73-82.
Shulman, L. S. (1998). Theory, practice, and the education of professionals. The Elementary School Journal, 98(5), 511-526.
Stajkovic, A., & Luthans, F. (1998). Social cognitive theory and self-efficacy: Going beyond traditional motivational and behavioral approaches. Organizational Dynamics, 26(4), 62-73.
Stoltenberg, C. D. (1998). A social cognitive--and developmental--model of counselor training. Counseling Psychologist, 26(2), 317-323.
Stone, D. N. (1994). Overconfidence in initial self-efficacy judgments: Effects on decision processes and performance. Organizational Behavior and Human Decision Processes, 59, 452-474.
Thyer, B. A., Vonk, M. E., & Tandy, C. C. (1995-1996). Are advanced standing and two-year program MSW students equivalently prepared? A comparison of BSW licensure examination scores. Arete, 20, 42-46.
Unrau, Y. A., & Grinnell, R. M. (2005). The impact of social work research courses on research self-efficacy for social work students. Social Work Education, 24, 639-651.
Urbani, S., Smith, M. R., Maddux, C. D., Smaby, M. H., Torres-Rivera, E., & Crews, J. (2002). Skills-based training and counseling self-efficacy. Counselor Education & Supervision, 42(2), 92-106.
Van der Vleuten, C. P. M., & Swansen, D. B. (1990). Assessment of clinical skills with standardized patients: State of the art. Teaching and Learning in Medicine, 2, 58-76.
Vu, N. V., & Barrows, H. S. (1994). Use of standardized patients in clinical assessment: Recent developments and measurement findings. Educational Researcher, 23, 23-50.
Waites, C. (2002). Assessing generalist problem-solving skills: An outcome measure. Journal of Baccalaureate Social Work, 6(1), 67-79.
Wass, V., Van der Vleuten, C., Shatzer, J., & Jones, R. (2001). Assessment of clinical competence. The Lancet, 357, 945-949.
Mary A. Rawlings is associate professor at Azusa Pacific University.
The author would like to thank the members of her dissertation committee, Professors Wally Gingerich (chair), Terry Hokenstad, Aloen Townsend, and Klara Papp of Case Western Reserve University, and Marion Bogo of the University of Toronto for their ongoing support of this project.
Address correspondence to Mary Rawlings, Azusa Pacific University, 901 East Alosta Avenue, Azusa, California, 91702-7000; e-mail: email@example.com.
Mary A. Rawlings
Azusa Pacific University
TABLE 1. Sample Summary of Gender and Ethnicity by BSW Education Entering BSW Exiting BSW (n=16) (n=16) Gender Female 15 15 Male 1 1 Ethnicity Caucasian 13 12 Hispanic 2 1 African American 0 1 Biracial 0 2 Other 1 0 TABLE 2. Group Differences Between Entering and Exiting BSW Students on Age and GPA Entering BSW Exiting BSW ANOVA Variable M SD M SD F df Age 18.75 1.13 22.87 4.63 11.99 * 1.30 GPA 3.3 .58 3.6 .38 1.36 1.25 Note. GPA=grade point average; ANOVA=analysis of variance. * p < .05. TABLE 3. One-Way Analysis of Variance (ANOVA) for Effects of BSW Education on Direct Practice Self-Efficacy Entering BSW Exiting BSW Variable M SD M SD Direct practice self-efficacy (range 0-9) Total scale 4.40 1.38 6.78 .90 Beginning 4.35 1.59 7.58 .90 Exploring 4.79 1.09 6.25 1.16 Contracting 3.42 1.99 6.66 1.03 Case management 3.18 2.12 6.81 1.07 Core conditions 5.50 1.99 7.48 .93 ANOVA Variable F df Direct practice self-efficacy (range 0-9) Total scale 33.36 * 1, 30 Beginning 49.66 * 1, 30 Exploring 13.50 * 1, 30 Contracting 33.28 * 1, 30 Case management Core conditions 13.71 * 1, 30 * p < .05. TABLE 4. One-Way Analysis of Variance (ANOVA) for Effects of BSW Education on Direct Practice Skill Entering BSW Exiting BSW Variable M SD M SD Direct practice skill (range 1-5) Total scale 2.19 .57 3.42 .75 Beginning 1.93 .77 3.53 1.01 Exploring 2.24 .63 3.34 .81 Contracting 2.12 .62 3.56 1.03 Case management 2.23 .65 3.33 .67 Core conditions 2.31 .84 3.39 .85 ANOVA Variable F df Direct practice skill (range 1-5) Total scale 27.35 * 1, 30 Beginning 25.22 * 1, 30 Exploring 19.38 * 1, 30 Contracting Case management 22.42 * 1, 30 Core conditions 13.02 * 1, 30 * p < .05. TABLE 5. Group Differences for Outside BSW Experience and Intent to Work in Direct Practice Entering BSW Exiting BSW Variable M SD M SD F df Intent 8.49 1.82 8.06 2.14 .60 30 Outside BSW experience .19 .54 .94 .85 -2.96 * 25 * p < .01. TABLE 6. Intercorrelations for BSW Education With Direct Practice Self-Efficacy, Direct Practice Skill, Intent to Work in Direct Practice, and Outside BSW Experience Measure 1 2 1. BSW education -- 2. Direct practice self-efficacy .73 ** -- 3. Direct practice skill .69 ** 0.29 4. Intent to work in direct practice -0.11 0.01 5. Outside BSW experience .48 ** 0.32 Measure 3 4 5 1. BSW education 2. Direct practice self-efficacy 3. Direct practice skill -- 4. Intent to work in direct practice -0.31 -- 5. Outside BSW experience .39 * --0.06 -- * p < .05. ** p < .01. TABLE 7. Hierarchical Regression Analysis Summary for BSW Education Predicting Direct Practice Self-Efficacy Variable B SE B [beta] Step 1 Intent (control) 0.03 0.15 0.03 Outside experience (control) 0.66 0.37 0.32 Step 2 Intent (control) 0.08 0.11 0.09 Outside experience (control) -0.07 0.30 -0.04 BSW education 2.47 0.48 .75 ** Variable [R.sup.2] [[DELTA].sup.2] Step 1 0.10 0.10 Intent (control) Outside experience (control) Step 2 .54 * .44 * Intent (control) Outside experience (control) BSW education * p <.05. ** p < .01. TABLE 8. Hierarchical Regression Analysis Summary for BSW Education Predicting Direct Practice Skill Variable B SE B [beta] Step 1 Intent (control) -0.13 0.08 -0.29 Outside experience (control) 0.42 0.18 .37 * Step 2 Intent (control) -0.11 0.06 -0.24 Outside experience (control) 0.08 0.17 0.07 BSW education 1.12 0.26 .63 ** Variable [R.sup.2] [[DELTA].sup.2] Step 1 .23 * .23 * Intent (control) Outside experience (control) Step 2 .54 ** .31 ** Intent (control) Outside experience (control) BSW education * p < .05. ** p < .01. TABLE 9. Hierarchical Regression Analysis for Direct Practice Skill Regressed on BSW Education and Self-Efficacy Variable B SE B [beta] Step 1 Intent (control) -0.13 0.08 -0.29 Prior experience (control) 0.42 0.18 .37 * Step 2 Intent (control) -0.11 0.06 -0.24 Prior experience (control) 0.08 0.17 0.07 BSW education 1.12 0.26 .63 ** Step 3 Intent (control) -0.09 0.06 -0.20 Prior experience (control) 0.06 0.15 0.06 BSW education 1.67 0.34 .94 ** Direct practice self-efficacy -0.22 0.10 -.41 * Variable [R.sup.2] [[DELTA].sup.2] Step 1 .23 * .23 * Intent (control) Prior experience (control) Step 2 .53 ** .31 ** Intent (control) Prior experience (control) BSW education Step 3 .62 ** .08 ** Intent (control) Prior experience (control) BSW education Direct practice self-efficacy * p < .05. ** p < .01.
|Printer friendly Cite/link Email Feedback|
|Author:||Rawlings, Mary A.|
|Publication:||Journal of Social Work Education|
|Date:||Sep 22, 2012|
|Previous Article:||Visualizing the invisible college: community among authors in top social work journals.|
|Next Article:||Field instructors' perceptions of evidence-based practice in BSW field placement sites.|