Printer Friendly

Science or Coaches' Eye?--Both! Beneficial Collaboration of Multidimensional Measurements and Coach Assessments for Efficient Talent Selection in Elite Youth Football.

Introduction

The tremendous popularity of football over the last decades means it is one of the most competitive sports worldwide (Haugaasen and Jordet, 2012). Simultaneously, the development of outstanding football players has become a profitable and prestigious business for clubs and national associations (Relvas et al., 2010). Within this process of talent development, talent identification and talent selection play key roles. Talent selection describes the inclusion of identified talents into a development program (Williams and Reilly, 2000). By subsequently refering only to talent selection, we also imply the process of recognizing participants with the potential to become elite players (talent identification) within that expression. The function of talent selection is to recognize and choose the most promising youth players to receive a superior learning environment (e.g. specialized coaching) within the development systems of football organizations (Williams and Reilly, 2000). In general, it seems to be clear that an optimized and ongoing promotion of any young football participant would be the most promising model of talent development in terms of using the potential of the whole population (Cote and Hancock, 2015). However, resources within football organizations are still limited. Therefore, talent selection and deselections (with the implication of losing potential) have to be taken into account as an inevitable but necessary process of focusing resources on players with the highest potential for future elite performance (Suppiah et al., 2015).

Resulting from the necessity of talent selection, practitioners in the field face the ongoing question of what are the most effective methods for this procedure. Although multidimensional approaches for talent selection have been suggested for some time (Abbott et al., 2005; Vaeyens et al., 2008; Williams and Reilly, 2000), most clubs and associations still rely solely on subjective data from coach assessments (Christensen, 2009; Larkin and Reeves, 2018). Only specific objective data (e.g., from motor performance tests) is common within talent selections in several development programs, in addition to the coaches' eye (Honer et al., 2017). Thus, there is a gap between recommendations of the scientific community and the procedures currently executed in the field (Larkin and Reeves, 2018). In addition to the frequently discussed issues relating to a need for further coach education (Figueiredo et al., 2014), one reason for this gap might be the lack of scientific evidence for the superiority of multidimensional approaches for talent selection over the commonly used coach assessments or motor performance tests. Until now, there has not been a direct comparison between these different methodological approaches to talent selection (coach assessment vs. motor performance tests vs. multidimensional data). The possible differences between the three approaches in their potential to predict future success of young football players remain unclear (Schorer et al., 2017).

Scientific opinion differs on the utility of coach assessments for talent selection in football. On one hand, the holistic nature allows coaches to integrate information from several dimensions and to judge players as a whole (Buekers et al., 2015). Jokuschies et al. (2017) endorse this positive view of the coaches' eye by systematizing talent criteria from five junior national team coaches of the Swiss Football Association. They were able to show that coaches' rating within certain talent criteria were reliable and valid in their appraisal of players overall potential. Furthermore, coaches' ratings of the overall performance of players and overall potential show high interrater reliability (Fenner et al., 2016; Gullich et al., 2017; Zuber and Conzelmann, 2014). However, it could be argued that coaches' decisions within talent selection seem to be guided by subjective feelings (Johansson and Fahlen, 2017; Lund and Soderstrom, 2017), and practitioners in the field do not have a generally accepted talent model (Jokuschies et al., 2017). Additionally, ratings of overall in-game performance are, for example, influenced by the number of actions players have in a game (Tromp et al., 2013). Biological maturation also influences the subjective ratings of in-game performance (Cripps et al., 2016), although an experienced coaches' eye has the potential to be a valid estimator of maturation (Romann et al., 2017).

The value of motor performance tests for talent selection in football has been demonstrated in several cases through the measurement of physiological data and general motor performance (Dodd and Newans, 2018; Gonaus and Muller, 2012; Le Gall et al., 2010; Murr et al., 2018), as well as technical skills (Forsman et al., 2016; Honer and Votteler, 2016; Sarmento et al., 2018). However, the prognostic validity of physiological data (e.g., aerobic capacity) and general motor performance tests (e.g., sprint performance) in the long-term talent prediction of youth players is vigorously questioned due to development-related influences such as biological maturation and relative age (Johnson et al., 2017; Malina et al., 2017; Muller et al., 2017; Romann et al., 2018). For that reason, domain specific test items (e.g., technical skills) are thought to provide higher prognostic validity than general motor performance tests, although the reliability of the former is generally lower (Lidor et al., 2009). Therefore, the overall value of motor performance tests for talent selection in football is still under discussion (Leyhr et al., 2018).

In addition to coach assessments and motor performance tests, common scientific recommendations for multidimensional modelling in talent selection refer to psychological characteristics, familial support, and training history as potential predictors of future success (Figueiredo et al., 2009; Huijgen et al., 2014; Williams and Reilly, 2000). In particular, psychological characteristics are increasingly receiving attention in the field. For example, motivational, volitional, and self-regulation skills are particularly relevant (Gledhill et al., 2017; Zuber et al., 2015). Notably, coaches' perceptions of talent in elite youth football players are predominantly influenced by psychological characteristics (Jokuschies et al., 2017). However, confounding influences (such as limited knowledge about personality changes over time, difficulties with the operationalization of psychological items, the wide variety of designs used in research) inhibit a clear view on the value of psychological characteristics in talent selection in youth football (Gledhill et al., 2017; Sarmento et al., 2018). The influence of familial support, which can be expressed through emotional, financial, or organizational means, is traditionally discussed in the context of talent development (Cote, 1999; Knight et al., 2017), while its predictive power for talent selection has hardly been investigated (Zibung and Conzelmann, 2014). Therefore, a greater understanding of the possible impact of familial support in talent selection is still needed (Sarmento et al., 2018). Finally, the predictive value of training history, especially up to 12 years of age, is vigorously debated. Although there is evidence that some kind of early engagement in a specialized-sampling model, with extensive volume and a broad range of activities within football (Ford and Williams, 2017; Sieghartsleitner et al., 2018), seems to be fruitful for later success, data remain contradictory (Hornig et al., 2016).

Overall, the commonly used and recommended methodological approaches to talent selection each have pros and cons. Coach assessments are inherently subjective, which is always a bone of contention when considering psychometric properties (Johansson and Fahlen, 2017; Jokuschies et al., 2017; Lund and Soderstrom, 2017). However, the holistic character of coach assessments reflects the potential for coaches to integrate information from several different dimensions and to judge players more as a whole. This provides a clear benefit over other assessment methods and leads to easier decision making in terms of overall assessments in selecting or de-selecting a player (Buekers et al., 2015). In contrast, for motor performance tests and multidimensional measurements, psychometric properties in terms of objectivity and reliability are generally accepted by the scientific community and practitioners in the field (Honer et al., 2017). However, there is only limited evidence in support of the prognostic validity of each dimension in predicting later success in football (Sarmento et al., 2018). Furthermore, motor performance tests and multidimensional test batteries provide results consisting of several variables from various items and dimensions, and the issue of integrating these variables into an overall assessment and determining the load of specific variables is critical (Bergman and Trost, 2006; Till et al., 2016; Till et al., 2018). Given the importance of overall decision making on a single player, this is particularly problematic.

Given the current uncertainty on the use of different talent selection instruments, there is an increasing interest and requirement for a direct comparison of their relative values in terms of prognostic validity (Buekers et al., 2015; Schorer et al., 2017). If all methodological approaches have separate strengths and weaknesses, which is most useful in predicting late success: coach assessments, motor performance tests, or multidimensional data? Schorer et al. (2017) considered part of this question in a sample of female team handball players. They found that a logistic regression model of motor performance tests predicted a higher percentage (85.2%) of correctly selected female handball talents over ten years than national team coach assessment (79.3%). However, because of the exploratory nature of their study, they restrained from using inferential statistics and did not test for statistically significant differences between the selection instruments.

Therefore, the current investigation examines whether coach assessments, motor performance tests, or multidimensional data show a higher success rate within talent selection in elite youth football by directly comparing the prognostic validity of these instruments. A second aim was to clarify the evidence underlying the assumption that combinations of the assessment methods may lead to superior predictions, as the use of combinations is either common in the field (coach assessments and motor performance tests; Honer et al., 2017), or provides the most holistic perspective on each player (coach assessments and multidimensional data). The latter seems to be particularly fruitful for predicting later success of sports talents (Reilly, 2006; Zuber et al., 2016).

Methods

Research design and participants

The current research is part of the longitudinal project Talent Selection and Talent Development in Swiss football. It incorporates several dimensions to holistically assess talent development (e.g., motor performance, coach assessments, psychological characteristics, familial support, and training history). The project follows a substantial number of players born in 1999 throughout the talent promoting system of the Swiss Football Association. As is common in other federal talent development programs, the promotion system of the Swiss Football Association follows the pyramidal standard model of talent development (Bailey and Collins, 2013). Early selections into talent bases and regional squads take place from U12 age groups (around six percent of registered players; Romann and Fuchslocher, 2013). The elite youth development program (Swiss junior national teams with around one percent of registered players; Romann and Fuchslocher, 2013) starts at the U15 level. Establishing within the top twelve nations in the FIFA ranking of senior national teams since 2012, the talent promoting system of the Swiss Football Association has to be considered as efficient.

The current research used a sample of 117 players. In the season 2017/2018, 20 of these players (17.1%) participated in the 1st to 3rd league within Switzerland or were nominated for the Swiss U19 junior national team (professional players). The remaining 97 players took part in the 4th league or below and were classified as nonprofessionals. Five years before (during the season 2012/2013, at U14 age category) all of the players, their parents, and club coaches volunteered to participate in coach assessments and multidimensional measurements (see Table 1). The study received approval from the Ethics Committee of the Faculty of Human Sciences of the University of Bern and all players, parents, and coaches provided their written informed consent to participate.

Measures

Coach assessment. The club coaches of the players carried out a visual scale estimation procedure to rate players' current in-game performance. For their rating, the coaches used a visual scale between 0 and 100. With a Kendall's concordance coefficient of W = 0.89 the inter-rater reliability for this instrument of coach assessment can be described as satisfactory (Zuber and Conzelmann, 2014). As the players were part of different regional teams, a total number of fifteen club coaches were involved in the rating of players' in-game performance. To deal with this, there was a standardized procedure to introduce the coaches into the test instrument (e.g., fictitious junior national team players should score between 90 and 100, whereas very poor players would score between 0 and 10; Zuber and Conzelmann, 2014).

For several reasons, we restrained from further standardization within the coach assessment (e.g., a single coach, who should rate all the players). First, the mentioned satisfactory inter-rater reliability of the instrument indicated appropriate objectivity. Second, applied sport science should have ecological validity in mind (Davids, 1988). In terms of talent selection processes for nationwide talent development systems of federations and associations, this may mean that the inclusion of different coaches from different regions is a necessity.

Motor performance tests. During the whole season, players participated twice (autumn 2012 and spring 2013) in a test battery consisting of seven items to determine motor performance. The season performance was calculated using the mean value of both tests. For several reasons (e.g., injury, sickness, and school activities), some players missed one measurement point (29.9%), in which case, the other measurement served as the test score. As no major changes in motor performance within an intra-seasonal period of six months were found in a similar age group (Francioni et al., 2016) and similar procedures are common within long-term development analysis in football (Gonaus and Muller, 2012; Honer et al., 2015), this procedure seemed to be appropriate.

The level 1 Yo-Yo intermittent recovery test (YY) measured the capacity for intermittent endurance performance ([r.sub.tt] = 0.93; Bangsbo et al., 2008). The highest value of five attempts in a vertical counter movement jump test (CMJ, without arm swing) was taken by means of an accelerometric system (Myotest, Sitten, Switzerland; ICC = 0.96; Casartelli et al., 2010). A 40 m-sprint test (40 m) was executed with a twin photoelectric sensor (Microgate, Bolzano, Italy) at the starting and finishing line ([r.sub.tt] = 0.96; Zuber et al., 2016). For the agility test (AG), players took a short sprint, ran around three poles with a change of direction and repeated these actions mirror-inverted before finishing (Honer et al., 2015). As for the sprint test, times were measured using twin photoelectric sensors ([r.sub.tt] = 0.83). A dribbling test (DR) was conducted in the same way as the agility test, the only difference being that it was performed with rather than without a ball ([r.sub.tt] = 0.56; Honer et al., 2015). The passing test (PA) was an adapted version of that used by Honer et al. (2015). Players passed the ball as quickly as possible in a confined zone and bounced it off four walls in turn, one in each direction. After the fourth pass, the same sequence was repeated in reverse order (reaching a total of nine passes). The time was measured manually with stopwatches ([r.sub.tt] = 0.68; Zuber et al., 2016). In the juggling test (JU), players took turns juggling with their left and right foot alternately along a course shaped like the figure 8. For each quarter of a circle they completed, they scored a point. The test was stopped as soon as a mistake was made (e.g., one foot twice in succession, or the ball touching the ground) or at the latest after 45 seconds. The test score was the number of points (Honer et al., 2015), reaching a [r.sub.tt] = 0.79 (Zuber et al., 2016).

The whole test battery followed a standardized protocol (warm-up, order of tests, trained team of testers) and was executed exclusively on dry synthetic turf. In the case of rainy weather conditions, an alternative date was scheduled. For 40 m, AG, DR, PA, and JU, the best of two attempts was used for data analysis. For the all-out YY, only one attempt was possible.

Psychological questionnaires. Consistent with earlier research on psychological assessments in the project Talent Selection and Talent Development in Swiss football (Zuber et al., 2015), the following motivational characteristics were ascertained by means of questionnaires.

Achievement motive was measured using the net hope (hope for success minus fear of failure) by means of the German version of the short scale of the Achievement Motives Scale--Sport (Wenhold et al., 2009). Each scale consists of five items, with a four-point response scale (from 0 = does not apply to me at all to 3 = applies completely to me). The internal consistencies were acceptable with [[alpha].sub.Hope for success] 0.76 and [[alpha].sub.Fear of Failure] = 0.73.

Achievement goal orientations were measured using the German version of the Sport Orientation Questionnaire (average score of competition-, win- and goal-orientation; Elbe, 2004). Each scale consists of six items, with a five-point response scale (from 1 = strongly disagree to 5 = strongly agree). The internal consistencies were acceptable ([[alpha].sub.competition] = 0.80, [[[alpha].sub.win] = 0.72, [[alpha].sub.goal] = 0.81).

Self-determination was measured using a German translation of the Sport Motivation Scale (Demetriou, 2012; Pelletier et al., 1995). This contains seven subscales: intrinsic motivation (three subscales: to know, to accomplish, to experience), external, introjected, and identified regulation, as well as amotivation. Each scale consists of four items, with a seven-point response scale (from 1 = does not correspond at all to 7 = corresponds exactly). The seven subscales were combined to form a self-determination index (Vallerand, 2001). With a = 0.86, the internal consistency was considered acceptable.

Environmental questionnaires. Consistent with earlier research on talent development environment in the project Talent Selection and Talent Development in Swiss football (Sieghartsleitner et al., 2018; Zibung and Conzelmann, 2014), the following aspects were ascertained by means of questionnaires.

The measurement of familial support tried to cover emotional, financial and organizational aspects. They were measured using the items importance of football within family (from 1 = low importance to 5 = high importance), parents' priority of sport versus school (from 1 = school is more important to 5 = football is more important), financial investment (absolute amount of money parents spend for a whole season, e.g. for equipment or practice) and time investment (absolute volume of time parents spend in an average week, e.g. for shuttling or cheering). In relation to the psychometric properties of these items, differential stabilities of 0.52 [less than or equal to] r [less than or equal to] 0.78 have been recorded between measurements within two consecutive seasons.

Training history was measured by the volume of organized in-club football practice and the volume of free play within football up to 12 years of age together. Volumes were collected with retrospective questionnaires by means of hours per week in each age group since the entrance into sports. These values were summed up to a total number of hours up to 12 years of age (Sieghartsleitner et al., 2018). The reliability of such methods has been shown to be acceptable (Helsen et al., 1998; Hopwood, 2015).

Biological maturation. To control for biological maturation, maturity timing (Cumming et al., 2017) expressed as age at peak height velocity (Mirwald et al., 2002) and relative age (RA), in terms of birth months (January = 1, December = 12), were assessed. The psychometric properties of the former procedure have been proven to be acceptable (Muller et al., 2015).

Data analysis

Data analysis calculated five classification models to predict U19 player status (professional or non-professional) by data collected five years before the performance criterion at U14 age category. Table 2 provides an overview on these classification models and the integrated variables. The first model used the in-game performance from coach assessment only (coaches' eye model, one variable). The second model was calculated with data from motor performance tests (motor performance model, two variables) and used the mean z-value of general motor performance items (YY, CMJ, 40 m, AG) and the mean z-value of technical skill items (DR, PA, JU). A third model (multidimensional model, six variables) used the mean z-value of general motor performance items, the mean z-value of technical skill items, the mean z-value of maturity timing (APHV) and relative age (RA, birth month), the mean z-value of all psychological characteristics, the mean z-value of all familial support items and the z-value of training history. A fourth model combined the in-game performance from coach assessment and data from motor performance tests (coaches' eye and motor performance model, three variables), as this combination is common in the field. Finally, the fifth model used all the available information by combining in-game performance from coach assessment and the six z-values from the multidimensional model (holistic model, seven variables), which reflects a holistic perspective on each player (Reilly, 2006; Zuber et al., 2016).

Each model used binary logistic regression (BLR) from R (R Core Team, 2017) as a robust classifier (Antonogeorgos et al., 2009) and receiver operating characteristic (ROC) from the R package pROC (Robin et al., 2011) to determine the discriminative power of the classification. Within this procedure, BLR first calculated the likelihood for each individual to be categorized as professional or non-professional player. The significance of the coefficients which indicate an improvement over a baseline model, and appropriate calibration, which proves if the model fits the data, of BLR models were tested with a likelihood-ratio test (Omnibus tests of model coefficients; Zeileis and Hothorn, 2002) and the Hosmer-Lemeshow test (Hosmer et al., 2013). The alpha level for significance was set atp < 0.05 for both tests. According to the correspondent null hypothesis, significance of coefficients was indicated by p < 0.05 and appropriate calibration by means of p > 0.05. After indicating appropriate calibration, the likelihood from BLR was used to create the ROC. The area under the curve (AUC), an index for measuring the quality of classification, and its standard error were used to compare the ROC models against each other by DeLongs nonparametric test (DeLong et al., 1988; Robin et al., 2011). Again, the alpha level for significance was initially set to p < 0.05. Due to the multiple comparisons between the five classification models a false discovery rate was used to adjust the p-value appropriately (Benjamini and Hochberg, 1995).

In addition to the ability to determine classification quality and the immediate statistical comparability of the models, ROC enhances the statistics from BLR by the descriptive reference to changes of sensitivity, specificity and accuracy, with changes in the discrimination threshold (Robin et al., 2011). These values are better known as proportion of correctly selected talents (sensitivity), correctly de-selected non-talents (specificity) and correct percentage of all selection decisions (accuracy). Compared to the setting of a fixed discrimination threshold in BLR, ROC creates the possibility to ensure one of the most powerful discrimination thresholds, known as Youden index (Youden, 1950). This index maximizes the sum of sensitivity and specificity, and provides an additional benefit over BLR. In the case where a talent development system values correctly selected talents and correctly de-selected non-talents as equal, YI may represent the most efficient talent selection threshold for the system.

Results

Table 3 provides an overview of the descriptive characteristics of the measured variables and items for professional and non-professional players. According to the results of the BLR analysis (see Table 4), all five models were significant (p < 0.01), appropriately calibrated (0.50 < p < 0.95) and showed model fits from 0.14 to 0.55 (Nagelkerkes [R.sup.2]). Table 5 presents the descriptive values from the ROC. The AUC [95% CI] indicates values from 0.71 [0.58; 0.84] to 0.93 [0.87; 0.98]. Sensitivities of the classification models indicate values between 70% and 95%, which means that these models were able to identify 14 to 19 of the 20 professional players correctly. Values for specificity ranged from 66% to 88%, whereby the classification models identified 64 to 85 of the 97 non-professional players correctly.

Sensitivity and specificity together lead to a YI of 0.61 or 71% overall correct talent selection decisions in the coaches' eye model. In other words, from a sample of 117 elite youth football players at the U14 age group, this model would bet on 52 players to become professional (19 valid predictions, one professional missed). In the motor performance model, the resulting accuracy was 73% correct predictions (YI = 0.43), which means a more restrictive bet on 40 players with only 14 valid predictions (six professionals missed). The multidimensional model indicated an accuracy of 82% (YI = 0.67) and therefore a bet on 34 players with 17 valid predictions (three professionals missed). The combined coaches' eye and motor performance model predicted 75% players correct (YI = 0.69), which means a less restrictive bet on 47 players with 19 professionals (one professional missed). Finally, the holistic model had an accuracy of 88% (YI = 0.77) and made the most valid selection decisions. In terms of betting on players who reach professional status, this model would select only 30 players with 18 valid predictions (two professionals missed).

Table 6 displays the results of the nonparametric approach for comparing AUCs. The first three lines refer to the pairwise comparisons between the three methodological approaches for talent selection. Next, the common combination of coach assessments and motor performance tests was compared against the single use of coaches' eye or motor performance tests alone. The final three comparisons target the value of the holistic model, which was examined in contrast to the coaches' eye, the multidimensional data and the combined model of coaches' eye and motor performance tests.

To determine the value of the single dimensions in the classification models, Table 7 shows the BLR regression coefficients for the holistic model, which contains all seven used variables. The in-game performance has by far the highest positive impact with an Odds Ratio (OR) of 12.53 (p < 0.01), whilst general motor performance has a negative relationship with professional player status (OR = 0.11, p < 0.01). Further significant impact on the regression model was found for familial support (OR = 5.20, p < 0.01). Technical skills (p = 0.54), psychological characteristics (p = 0.80), training history (p = 0.83) and maturation (p = 0.93) did not significantly influence the regression model. As in-game performance is dominant in this model, the BLR regression coefficients for the multidimensional model are presented in Table 8, which gives further insight into the value of different measured variables. Familial support (OR = 4.24, p < 0.01) and technical skills (OR = 3.00, p = 0.02) showed a significantly positive impact on getting a professional player, whilst general motor performance was again negatively associated (OR = 0.37, p = 0.04). Psychological characteristics (p = 0.27), maturation (p = 0.32), and training history (p = 0.82) did not significantly influence the multidimensional model.

Discussion

These current findings showed that each of the five classification models for talent selection in elite youth football contributed significantly to the prediction of U19 player status (professional vs. non-professional) through the use of U14 data. Even the lowest YI from the motor performance model (0.43) reached 73% correct selection decisions. This indicates that all investigated methodological approaches for talent selection (coach assessment, motor performance tests, and multidimensional data), and specific combinations of them, are powerful tools. Therefore, it is reasonable that they are commonly used and recommended for use as well (Christensen, 2009; Honer et al., 2017; Williams and Reilly, 2000).

Comparison of the methodological approaches

According to the comparison of prognostic validity of the methodological approaches, the coach assessment did not differ from the motor performance tests or multidimensional data, while the prediction from the motor performance test data was enhanced by multidimensional measurements. Therefore, if a talent development system relies on measured data for talent selection only, multidimensional data is preferable. This is in line with previous scientific suggestions (Abbott et al., 2005; Vaeyens et al., 2008).

Regarding the potential benefits of instrument combinations, the prognostic validity of coaches' eye is not enhanced by motor performance tests, but motor performance tests benefit from the addition of coach assessments in the combined coaches' eye and motor performance model. Further, the combination of coach assessment and multidimensional data into a holistic model is superior over each single part, but does not differ from the combination of coaches' eye and motor performance tests. This provides strong evidence for the general use of either coach assessments and measured data together for talent selection in elite youth football. For measured data, there is only a tendency for the necessity of multidimensional data in the case of a combination with coach assessment, as the holistic model is not significantly better than the combined coaches' eye and motor performance model. All things considered, the idea of a holistic judgement for talent selection seems to be fruitful (Reilly, 2006; Zuber et al., 2016). However, if certain reasons prohibit multidimensional data measurement (e.g. economic reasons), a single addition of motor performance test data on a coach assessment may lead to comparable results.

Coach assessment

In general, the rating of a players' in-game performance by coach assessment is a strong predictor of later success in football in terms of the comparison between different selection models (i.e., only the holistic model significantly outperformed the coaches' eye model). Further, the OR within regression coefficients of the holistic model emphasize this: getting one standard deviation better rated from the coach improves the chance to become a professional player by an OR of 12.53 [2.97; 52.83]. This high impact of the single variable in-game performance is consistent with the common use of coach assessments in the field (Christensen, 2009) and its increasing positive favor within the scientific community (Fenner et al., 2016; Jokuschies et al., 2017; Romann et al., 2017). One reason for the high prognostic validity of coach assessments may be their holistic nature (Buekers et al., 2015). Within the current investigation, this may also have had consequences for the in-game performance variable. As club coaches, who have extensive knowledge about each player (e.g., about familial support and training history), did the rating, in-game performance may not only consist of an integration of game-based aspects (e.g., technical and tactical skills, general motor performance), but may be further influenced by knowledge from different dimensions. This seems to be a relevant limitation to the current investigation in terms of comparability to a short-term coach assessment (i.e., when a coach and player do not know each other). A further limitation to the current results may be a certain dependency between coach assessments and the performance criterion (professional vs. non-professional). As argued before, the use of coach assessments is very common in the field (Christensen, 2009), whereby getting a professional player depends on getting selected to a professional team by means of coach assessment most of the time.

Motor performance tests and multidimensional data

Having the lowest prognostic validity compared to the other methodological approaches in the current investigation, there are two noticeable aspects of motor performance tests. First, general motor performance is negatively associated with becoming a professional player. Although Leyhr et al. (2018) reported non-significant relationships between U12 to U15 sprint performance and adult performance levels, this is still unexpected (Gonaus and Muller, 2012; Murr et al., 2018). However, confounding influences from maturation (Malina et al., 2017), the homogeneity of the sample and the long-term prediction from U14 to U19 may explain it to a certain degree. Further, the assumed predictive value of technical skills for talent selection in elite youth football (Honer and Votteler, 2016; Sarmento et al., 2018) is underlined by its high positive impact in the multidimensional model rather than the holistic one. Therefore, within the holistic model, the dominant in-game performance from coach assessment may explain a certain variance of technical skills, whilst the latter reach a significant and relevant OR of 3.00 [1.20; 7.48] in the multidimensional model with measured data only. As coaches rate technical skills as very important at this age (Larkin and O'Connor, 2017), this is quite reasonable.

Although the multidimensional model is outperformed by the holistic one, the former confirms its positive endorsement in the scientific community (Williams and Reilly, 2000) by demonstrating superiority over the motor performance model. This superiority is primarily based on the contribution of familial support, which shows a meaningful OR of 4.24 [1.63; 11.05] and underpins the so far underestimated usefulness of this area for talent selection (Zibung and Conzelmann, 2014). Although psychological characteristics and training history have shown prognostic validity within different investigations from the current project (Sieghartsleitner et al., 2018; Zuber et al., 2015) and maturation has shown certain relationships with success on the adult level (Ostojic et al., 2014), their current contribution to the predictive value of the multidimensional model was non-significant.

A general limitation to the study affects results regarding motor performance tests as well as multidimensional data: the used statistical methods. As discussed earlier, integrating several variables into and determining the load of certain variables within overall assessments is critical (Bergman and Trost, 2006; Till et al., 2016; Till et al., 2018). The main question is how to put multiple variables or items into an overall motor performance or multidimensional model. As the aim of the current study was to get immediate comparisons of the different talent selection approaches, we built models by means of a curve linear model (BLR) and mean z-values representing the most independent variables. Thus, assumed interaction and compensation phenomena between several variables or items within the holistic nature of a developing sports talent cannot be reproduced appropriately (Meylan et al., 2010). Moving beyond such methods from the general linear model and stepping into non-linear methods (e.g., person-oriented methods; Bergman et al., 2003) would enable such interactions and compensations (Conzelmann et al., 2018). Further, these person-oriented methods would fit into a sound theoretical background for talent development issues from developmental sciences (i.e., young talents are developing humans) and their dynamic interactionist approaches (Zuber et al., 2016). However, non-linear and person-oriented analysis do not enable any possibility of immediate comparison between different talent selection approaches, whereby the use of the current methodology was a corollary from earlier thoughts on the research question. Hence, we have to take into account that the integration of several items into mean z-values impedes further insight and that the value of motor performance tests and multidimensional data may be underestimated--cf. relevant prognostic validity of psychological characteristics and training history within earlier research of the current project (Sieghartsleitner et al., 2018; Zuber et al., 2015). Due to its holistic nature (Buekers et al., 2015) and expression in a single item in the current investigation, in-game performance from coach assessment was not affected by this methodological problem, which may explain its high prognostic validity and that of combined models with coach assessment and measured data.

Conclusion

This study examined the prognostic validity of different methodological approaches for talent selection. Overall, there seems to be a beneficial collaboration of coach assessments and measured data for talent selection in elite youth football, as this may buffer their associated mutual strengths and weaknesses. At best, measured data within this combined strategy is multidimensional rather than based on motor performance only, which provided the highest prognostic validity by means of a holistic model.

These results indicate that the frequently recommended multidimensional approaches (Williams and Reilly, 2000) show superiority over less complex selection strategies. However, this proven benefit of multidimensionality requires further research optimizing the value of these approaches for talent selection in the field. There is still need for further understanding on the relevance of certain dimensions and items within multidimensional data, although several investigations have been completed (Figueiredo et al., 2009; Forsman et al., 2016; Huijgen et al., 2014; Woods et al., 2016; Zibung et al., 2016). Even more important but barely researched is the area of methodological aspects for maximizing the use of multidimensional information in talent selection (Conzelmann et al., 2018; Till et al., 2016). For several reasons, the applied statistical analysis, which was necessary to enable immediate comparisons between different methodological approaches for talent selection, may not be appropriate to analyze multidimensional data. A summation of items as well as the use of curve-linear models inhibit possible interaction and compensation phenomena between certain dimensions (Meylan et al., 2010) and imply the same statistical model being valid for each individual within the sample (Bergman et al., 2003). Some investigations with non-linear (Pfeiffer and Hohmann, 2012; Pion et al., 2017) and person-oriented approaches (Sieghartsleitner et al., 2018; Zibung et al., 2016; Zibung and Conzelmann, 2014; Zuber et al., 2015; Zuber et al., 2016) have tried to resolve these limitations. Nevertheless, there is the need to further implement and develop these methods to get a broader understanding of their possible value in talent selection. Therefore, we hope that the current advocacy for multidimensional approaches may encourage further sport scientists to move beyond linear statistics and utilize the assumed power of multidimensional data to improve the quality of talent selections within talent development systems of clubs and associations.

Key points

* Combined talent selection approaches with coach assessments and measured data are fruitful.

* Multidimensional data (motor performance tests, psychological characteristics, familial support, training history, biological maturation) outperformed motor performance tests only.

* Improvement of non-linear statistics might further enhance the use of multidimensional data for talent selection.

Acknowledgements

We would like to thank the Swiss Football Association for supporting and funding this research project. The experiments comply with the current laws of the country in which they were performed. The authors have no conflicts of interests to declare.

References

Abbott, A., Button, C., Pepping, G.-J. and Collins, D. (2005) Unnatural selection: talent identification and development in sport. Nonlinear Dynamics, Psychology, and Life Sciences 9(1), 61-88.

Antonogeorgos, G., Panagiotakos, D.B., Priftis, K.N. and Tzonou, A. (2009) Logistic Regression and Linear Discriminant Analyses in Evaluating Factors Associated with Asthma Prevalence among 10- to 12-Years-Old Children: Divergence and Similarity of the Two Statistical Methods. International Journal of Pediatrics 2009:952042.

Bailey, R. and Collins, D. (2013). The standard model of talent development and its discontents. Kinesiology Review 2, 248-259.

Bangsbo, J., Iaia, F.M. and Krustrup, P. (2008) The Yo-Yo intermittent recovery test: A useful tool for evaluation of physical performance in intermittent sports. Sports Medicine 38(1), 37-51.

Benjamini, Y. and Hochberg, Y. (1995) Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society 57(1), 289-300.

Bergman, L., Magnusson, D. and El-Khouri, B.M. (2003) Studying individual development in an interindividual context: A person-oriented approach. Mahwah, N.J: Erlbaum.

Bergman, L. and Trost, K. (2006) The Person-Oriented Versus the Variable-Oriented Approach: Are They Complementary, Opposites, or Exploring Different Worlds? Merrill-Palmer Quarterly 52(3), 601-632.

Buekers, M., Borry, P. and Rowe, P. (2015) Talent in sports. Some reflections about the search for future champions. Movement & Sport Sciences 88, 3-12.

Casartelli, N., Muller, R. and Maffiuletti, N.A. (2010) Validity and reliability of the Myotest accelerometric system for the assessment of vertical jump height. Journal of Strength and Conditioning Research 24(11), 3186-3193.

Christensen, M.K. (2009) "An eye for talent": Talent identification and the "practical sense" of top-level soccer coaches. Sociology of Sport Journal 26(3), 365-382.

Conzelmann, A., Zibung, M. and Zuber, C. (2018) Talent identification and talent development in sports. In: Talent management. Ed: Ritz, A. and Thom, N. Wiesbaden: Springer Fachmedien. 87-106. (In German).

Cote, J. (1999) The influence of the family in the development of talent in sport. The Sport Psychologist 13(4), 395-417.

Cote, J. and Hancock, D.J. (2015) Evidence-based policies for youth sport programmes. International Journal of Sport Policy and Politics 8(1), 51-65.

Cripps, A., Hopper, L. and Joyce, C. (2016) Coaches' perceptions of long-term potential are biased by maturational variation. International Journal of Sports Science & Coaching 11(4), 478-481.

Cumming, S.P., Lloyd, R.S., Oliver, J.L., Eisenmann, J.C. and Malina, R.M. (2017) Bio-banding in Sport: Applications to Competition, Talent Identification, and Strength and Conditioning of Youth Athletes. Strength and Conditioning Journal 39(2), 34-47.

Davids, K. (1988) Ecological Validity in Understanding Sport Performance: Some Problems of Definition. Quest 40(2), 126-136.

DeLong, E.R., DeLong, D.M. and Clarke-Pearson, D.L. (1988) Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach. Biometrics 44(3), 837-845.

Demetriou, Y. (2012) Health promotion in physical education: Development and evaluation of the eight week PE programme "healthypep" for sixth grade students in Germany. Forum Sportwissenschaft (Vol. 25). Hamburg: Czwalina.

Dodd, K.D. and Newans, T.J. (2018) Talent identification for soccer: Physiological aspects. Journal of Science and Medicine in Sport 21(10), 1073-1078.

Elbe, A.-M. (2004) Psychometric properties of the German version of the Sport Orientation Questionnaire. Spectrum der Sportwissenschaft 16(1), 96-107. (In German: English abstract).

Fenner, J.S.J., Iga, J. and Unnithan, V. (2016) The evaluation of small-sided games as a talent identification tool in highly trained prepubertal soccer players. Journal of Sports Sciences 34(20), 19831990.

Figueiredo, A.J., Gonjalves, C.E., Coelho-e-Silva, M.J. and Malina, R.M. (2009) Characteristics of youth soccer players who drop out, persist or move up. Journal of Sports Sciences 27(9), 883-891.

Figueiredo, A.J., Gonjalves, C.E. and Tessitore, A. (2014) Bridging the gap between empirical results, actual strategies, and developmental programs in soccer. International Journal of Sports Physiology and Performance 9(3), 540-543.

Ford, P.R. and Williams, A.M. (2017) Sport Activity in childhood: Early specialization and diversification. In: Routledge Handbook of Talent Identification and Development in Sport. Ed: Baker, J. et al. Florence, GB: Routledge. 117-132.

Forsman, H., Blomqvist, M., Davids, K., Liukkonen, J. and Konttinen, N. (2016) Identifying technical, physiological, tactical and psychological characteristics that contribute to career progression in soccer. International Journal of Sports Science & Coaching 11(4), 505-513.

Francioni, F.M., Figueiredo, A.J., Terribili, M. and Tessitore, A. (2016) Analysis of the intraseasonal stability of field test performances in young academy soccer players. Journal of Sports Sciences 34(10), 966-972.

Gledhill, A., Harwood, C. and Forsdyke, D. (2017) Psychosocial factors associated with talent development in football: A systematic review. Psychology of Sport and Exercise 31, 93-112.

Gonaus, C. and Muller, E. (2012) Using physiological data to predict future career progression in 14- to 17-year-old Austrian soccer academy players. Journal of Sports Sciences 30(15), 1673-1682.

Gullich, A., Kovar, P., Zart, S. and Reimann, A. (2017) Sport activities differentiating match-play improvement in elite youth footballers --a 2-year longitudinal study. Journal of Sports Sciences 35(3), 207-215.

Haugaasen, M. and Jordet, G. (2012) Developing football expertise: A football-specific research review. International Review of Sport and Exercise Psychology 5(2), 177-201.

Helsen, W., Starkes, J.L. and Hodges, N.J. (1998) Team sports and the theory of deliberate practice. Journal of Sport and Exercise Psychology 20, 12-34.

Honer, O., Leyhr, D. and Kelava, A. (2017) The influence of speed abilities and technical skills in early adolescence on adult success in soccer: A long-term prospective analysis using ANOVA and SEM approaches. PloS One 12(8):e0182211.

Honer, O. and Votteler, A. (2016) Prognostic relevance of motor talent predictors in early adolescence: A group- and individual-based evaluation considering different levels of achievement in youth football. Journal of Sports Sciences 34(24), 2269-2278.

Honer, O., Votteler, A., Schmid, M., Schultz, F. and Roth, K. (2015) Psychometric properties of the motor diagnostics in the German football talent identification and development programme. Journal of Sports Sciences 33(2), 145-159.

Hopwood, M.J. (2015) Issues in the collection of athlete training histories. In: Routledge Handbook of Sport Expertise. Ed: Baker, J. and Farrow, D. Florence, GB: Routledge. 156-165.

Hornig, M., Aust, F. and Gullich, A. (2016) Practice and play in the development of German top-level professional football players. European Journal of Sport Science 16(1), 96-105.

Hosmer, D.W., Lemeshow, S. and Sturdivant, R.X. (2013) Applied logistic regression. New York: John Wiley.

Huijgen, B.C.H., Elferink-Gemser, M.T., Lemmink, K.A. and Visscher, C. (2014) Multidimensional performance characteristics in selected and deselected talented soccer players. European Journal of Sport Science 14(1), 2-10.

Johansson, A. and Fahlen, J. (2017) Simply the best, better than all the rest?: Validity issues in selections in elite sport. International Journal of Sports Science & Coaching 12(4), 470-480.

Johnson, A., Farooq, A. and Whiteley, R. (2017) Skeletal maturation status is more strongly associated with academy selection than birth quarter. Science and Medicine in Football 1(2), 157-163.

Jokuschies, N., Gut, V. and Conzelmann, A. (2017) Systematizing coaches' 'eye for talent': Player assessments based on expert coaches' subjective talent criteria in top-level youth soccer. International Journal of Sports Science & Coaching 12(5), 565576.

Knight, C.J., Berrow, S.R. and Harwood, C.G. (2017) Parenting in sport. Current Opinion in Psychology 16, 93-97.

Larkin, P. and O'Connor, D. (2017) Talent identification and recruitment in youth soccer: Recruiter's perceptions of the key attributes for player recruitment. PloS One 12(4):e0175716.

Larkin, P. and Reeves, M.J. (2018) Junior-elite football: time to re-position talent identification? Soccer & Society 20(3), 1-10.

Le Gall, F., Carling, C., Williams, A.M. and Reilly, T. (2010) Anthropometric and fitness characteristics of international, professional and amateur male graduate soccer players from an elite youth academy. Journal of Science and Medicine in Sport 13(1), 90-95.

Leyhr, D., Kelava, A., Raabe, J. and Honer, O. (2018) Longitudinal motor performance development in early adolescence and its relationship to adult success: An 8-year prospective study of highly talented soccer players. PloS One 13(5):e0196324.

Lidor, R., Cote, J. and Hackfort, D. (2009) ISSP position stand: To test or not to test? The use of physical skill tests in talent detection and in early phases of sport development. International Journal of Sport and Exercise Psychology 7, 131-146.

Lund, S. and Soderstrom, T. (2017) To See or Not to See: Talent Identification in the Swedish Football Association. Sociology of Sport Journal 34(3), 248-258.

Malina, R.M., Cumming, S.P., Coelho-e-Silva, M.J. and Figueiredo, A.J. (2017) Talent Identification and Development in the Context of "Growing up". In: Routledge Handbook of Talent Identification and Development in Sport. Ed: Baker, J. et al. Florence, GB: Routledge. 150-168.

Meylan, C., Cronin, J.B., Oliver, J. and Hughes, M. (2010) Talent identification in soccer: The role of maturity status on physical, physiological and technical characteristics. International Journal of Sports Science & Coaching 5(4), 571-592.

Mirwald, R.L., Baxter-Jones, A., Bailey, D.A. and Beunen, G.P. (2002) An assessment of maturity from anthropometric measurements. Medicine and Science in Sports and Exercise 34(4), 689-694.

Muller, L., Gonaus, C., Perner, C., Muller, E. and Raschner, C. (2017) Maturity status influences the relative age effect in national top level youth alpine ski racing and soccer. PloS One 12(7):e0181810(7).

Muller, L., Muller, E., Hildebrandt, C., Kapelari, K. and Raschner, C. (2015) The Assessment of Biological Maturation for Talent Selection --Which Method can be used? Sportverletzung Sportschaden 29(1), 56-63. (In German: English abstract).

Murr, D., Raabe, J. and Honer, O. (2018) The prognostic value of physiological and physical characteristics in youth soccer: A systematic review. European Journal of Sport Science 18(1), 62-74.

Ostojic, S.M., Castagna, C., Calleja-Gonzalez, J., Jukic, I., Idrizovic, K. and Stojanovic, M. (2014) The biological age of 14-year-old boys and success in adult soccer: do early maturers predominate in the top-level game? Research in Sports Medicine 22(4), 398-407.

Pelletier, L.G., Tuson, K.M., Fortier, M.S., Vallerand, R.J., Briere, N.M. and Blais, M.R. (1995) Toward a new measure of intrinsic motivation, extrinsic motivation, and amotivation in sports: The Sport Motivation Scale (SMS). Journal of Sport and Exercise Psychology 17(1), 35-53.

Pfeiffer, M. and Hohmann, A. (2012) Applications of neural networks in training science. Human Movement Science 31(2), 344-359.

Pion, J., Hohmann, A., Liu, T., Lenoir, M. and Segers, V. (2017) Predictive models reduce talent development costs in female gymnastics. Journal of Sports Sciences 35(8), 806-811.

R Core Team (2017) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Available from URL: https://www.R-project.org.

Reilly, T. (2006) Assessments of young soccer players: A holistic approach. Perceptual and Motor Skills 103, 229-230.

Relvas, H., Littlewood, M., Nesti, M., Gilbourne, D. and Richardson, D. (2010) Organizational Structures and Working Practices in Elite European Professional Football Clubs: Understanding the Relationship between Youth and Professional Domains. European Sport Management Quarterly 10(2), 165-187.

Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C. and Muller, M. (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC bioinformatics 12:77.

Romann, M. and Fuchslocher, J. (2013) Relative age effects in Swiss junior soccer and their relationship with playing position. European Journal of Sport Science 13(4), 356-363.

Romann, M., Javet, M. and Fuchslocher, J. (2017) Coaches' eye as a valid method to assess biological maturation in youth elite soccer. Talent Development and Excellence 9(1), 3-13.

Romann, M., Rossler, R., Javet, M. and Faude, O. (2018) Relative age effects in Swiss talent development--a nationwide analysis of all sports. Journal of Sports Sciences 36(17), 2025-2031.

Sarmento, H., Anguera, M.T., Pereira, A. and Araujo, D. (2018) Talent Identification and Development in Male Football: A Systematic Review. Sports Medicine 48(4), 907-931.

Schorer, J., Rienhoff, R., Fischer, L. and Baker, J. (2017) Long-Term Prognostic Validity of Talent Selections: Comparing National and Regional Coaches, Laypersons and Novices. Frontiers in Psychology 8:1146.

Sieghartsleitner, R., Zuber, C., Zibung, M. and Conzelmann, A. (2018) "The early specialised bird catches the worm!"--A Specialised Sampling Model in the development of football talents. Frontiers in Psychology 9:188.

Suppiah, H.T., Low, C.Y. and Chia, M. (2015) Detecting and developing youth athlete potential: different strokes for different folks are warranted. British Journal of Sports Medicine 49(13), 878-882.

Till, K., Jones, B.L., Cobley, S., Morley, D., O'Hara, J., Chapman, C., Cooke, C. and Beggs, C.B. (2016) Identifying Talent in Youth Sport: A Novel Methodology Using Higher-Dimensional Analysis. PloS One 11(5):e0155047.

Till, K., Morris, R., Emmonds, S., Jones, B. and Cobley, S. (2018) Enhancing the Evaluation and Interpretation of Fitness Testing data within Youth Athletes. Strength and Conditioning Journal (serial online). Available from URL: 10.1519/SSC.0000000000000414.

Tromp, Y.E.J., Pepping, G.-J., Lyons, J., Elferink-Gemser, M.T. and Visscher, C. (2013) "Let's Pick Him!": Ratings of Skill Level on the Basis of In-Game Playing Behaviour in Bantam League Junior Ice Hockey. International Journal of Sports Science & Coaching 8(4), 641-660.

Vaeyens, R., Lenoir, M., Williams, A.M. and Philippaerts, R.M. (2008) Talent identification and development programmes in sport: Current models and future directions. Sports Medicine 38(9), 703-714.

Vallerand, R.J. (2001) A hierarchical model of intrinsic and extrinsic motivation in sport and exercise. In: Advances in motivation in sport and exercise. Ed: Roberts, G.C. Champaign, IL: Human Kinetics. 263-319.

Wenhold, F., Elbe, A.-M. and Beckmann, J. (2009) Achievement Motives Scale -- Sport (AMS-Sport). Questionnaire on the achievement motive in sports: Manual. Bonn: Bundesinstitut fur Sportwissenschaft. (In German).

Williams, A.M. and Reilly, T. (2000) Talent identification and development in soccer. Journal of Sports Sciences 18(9), 657-667.

Woods, C.T., Raynor, A.J., Bruce, L., McDonald, Z. and Robertson, S. (2016) The application of a multi-dimensional assessment approach to talent identification in Australian football. Journal of Sports Sciences 34(14), 1340-1345.

Youden, W.J. (1950) Index for rating diagnostic tests. Cancer 3, 32-35.

Zeileis, A. and Hothorn, T. (2002) Diagnostic Checking in Regression Relationships. R News 2(3), 7-10.

Zibung, M. and Conzelmann, A. (2014) National youth team football players between the conflicting priorities of sports success and vocational training. European Journal for Sport and Society 11(2), 127-150.

Zibung, M., Zuber, C. and Conzelmann, A. (2016) The Motor Subsystem as a Predictor of Success in Young Football Talents: A Person-Oriented Study. PloS One 11(8):e0161049.

Zuber, C. and Conzelmann, A. (2014) The impact of the achievement motive on athletic performance in adolescent football players. European Journal of Sport Science 14(5), 475-483.

Zuber, C., Zibung, M. and Conzelmann, A. (2016) Holistic patterns as an instrument for predicting the performance of promising young soccer players - a 3-year longitudinal study. Frontiers in Psychology 7:1088.

Zuber, C., Zibung, M. and Conzelmann, A. (2015) Motivational patterns as an instrument for predicting success in promising young football players. Journal of Sports Sciences 33(2), 160--168.

Roland Sieghartsleitner ([mail]), Claudia Zuber, Marc Zibung and Achim Conzelmann

Institute of Sport Science, University of Bern, Switzerland

Received: 08 October 2018 / Accepted: 30 November 2018 / Published (online): 11 February 2019

Roland SIEGHARTSLEITNER

Employment

PhD-Student at the Institute of Sport Science, University of Bern

Degree

MSc

Research interest

Talent Selection and Talent Development in Football

E-mail: roland.sieghartsleitner@ispw.unibe.ch

Claudia ZUBER

Employment

Post-Doc Researcher at the Institute of Sport Science, University of Bern

Degree

PhD

Research interest

Psychological Characteristics in Talent Selection and Talent Development

E-mail: claudia.zuber@ispw.unibe.ch

Marc ZIBUNG

Employment

Lecturer at the Institute of Sport Science, University of Bern

Degree

PhD

Research interest

Person-oriented Research in Talent Selection and Talent Development

E-mail: marc.zibung@ispw.unibe.ch

Achim CONZELMANN

Employment

Full Professor at the Institute of Sport Science at the University of Bern

Degree

PhD

Research interest

Developmental Sciences, Personality

and Talent Research

E-mail: achim.conzelmann@ispw.unibe.ch

([mail]) Roland Sieghartsleitner

Institute of Sport Science, University of Bern, Bremgartenstrasse 145, CH-3012 Bern, Switzerland
Table 1. Overview on measured variables and items.

Dimension         Variable                          Reference

Age,              chronological age (years)
biological        relative age (month)
maturation        age at peak height                Mirwald et al.,
                  velocity (years)                  2002

Anthropometry     height (cm)
                  weight (kg)

Coach             in-game performance               Zuber and
assessment        (points)                          Conzelmann, 2014

General motor     YoYo (m)                          Bangsbo et al.,
performance                                         2008
                  counter-movement-jump (cm)        Casartelli et al.,
                                                    2010
                  40 m-sprint (sec)                 Zuber et al., 2016
                  agility test (sec)                Honer et al., 2015

Technical         dribbling (sec)                   Honer et al., 2015
skills            passing (sec)                     Zuber et al., 2016
                  juggling (points)                 Honer et al.,
                                                    2015; Zuber et
                                                    al., 2016

Psychological     achievement motive (net-hope)     Wenhold et al.,
characteristics                                     2009
                  achievement goal orientations     Elbe, 2004
                  (score)
                  self-determination (index)        Pelletier et al.,
                                                    1995; Demetriou,
                                                    2012

Familial          importance of football within
support           family (score)
                  parents' priority of sport vs.
                  school (score)
                  financial investment (Swiss
                  Francs / year)
                  time investment (h / week)

Training          practice and play up to 12        Hopwood, 2015
history           years of age (h)

Dimension         Variable                             Reliability

Age,              chronological age (years)
biological        relative age (month)
maturation        age at peak height                 [r.sub.tt] = .96
                  velocity (years)

Anthropometry     height (cm)                        [r.sub.tt] = .99
                  weight (kg)                        [r.sub.tt] = .99

Coach             in-game performance                    W = .89
assessment        (points)

General motor     YoYo (m)                           [r.sub.tt] = .93
performance
                  counter-movement-jump (cm)            ICC = .96

                  40 m-sprint (sec)                  [r.sub.tt] = .96
                  agility test (sec)                 [r.sub.tt] = .83

Technical         dribbling (sec)                    [r.sub.tt] = .56
skills            passing (sec)                      [r.sub.tt] = .68
                  juggling (points)                  [r.sub.tt] = .79

Psychological     achievement motive (net-hope)     [alpha] = .76/.73
characteristics
                  achievement goal orientations         [alpha] =
                  (score)                              .80/.72/.81
                  self-determination (index)          [alpha] = .86

Familial          importance of football within      [r.sub.tt] = .63
support           family (score)
                  parents' priority of sport vs.     [r.sub.tt] = .52
                  school (score)
                  financial investment (Swiss        [r.sub.tt] = .78
                  Francs / year)
                  time investment (h / week)         [r.sub.tt] = .59

Training          practice and play up to 12        .59 [less than or
history           years of age (h)                      equal to]
                                                    [r.sub.tt] [less
                                                    than or equal to]
                                                           .97

Table 2. Overview on the five calculated classification models and
included variables.

Classification model   Number of   Variables
                       variables

1   Coaches' eye       1           z-value of in-game performance from
    model                          coach assessment

2   Motor              2           mean z-value of general motor
    performance                    performance items
    model                          mean z-value of technical skill
                                   items
                                   mean z-value of general motor
                                   performance items
                                   mean z-value of technical skill
                                   items

3   Multidimensional   6           mean z-value of maturity timing
    model                          and relative age
                                   mean z-value of all three
                                   psychological characteristics
                                   mean z-value of all familial
                                   support items
                                   z-value of the training history

4   Coaches' eye and   3           combination of models 1 and 2
    motor
    performance
    model

5   Holistic model     7           combination of models 1 and 3

Table 3. Means ([+ or -] standard deviation) for professional and
non-professional players for measured variables and items.

Dimension and variable                             Professional
                                                 players (n = 20)

Age, biological maturation
chronological age (years)                        13.53    (0.38)
relative age (month)                             4.80     (3.89)
age at peak height velocity (years)              13.81    (0.54)

Anthropometry
height (cm)                                      161.1     (7.1)
weight (kg)                                      48.3      (6.0)

Coach assessment
in-game performance (points)                      79       (12)

General motor performance
YoYo (m)                                         1032      (406)
counter-movement-jump (cm)                       29.0      (2.8)
40 m-sprint (sec)                                6.41     (0.34)
agility (sec)                                    8.12     (0.25)

Technical skills
dribbling (sec)                                  10.12    (0.39)
passing (sec)                                    16.2      (1.6)
juggling (points)                                 9.5      (6.1)

Psychological characteristics
achievement motive (net-hope)                    1.91     (0.88)
achievement goal orientations (score)            4.70     (0.23)
self-determination (index)                       10.25    (2.36)

Familial support
importance of football within family (score)     4.70     (0.47)
parents' priority of sport vs. school (score)    3.45     (1.15)
financial investment (Swiss Francs / year)       2625     (2218)
time investment (h / week)                       12.3      (7.0)

Training history
practice and play up to 12 years of age (h)      3311     (1039)

Dimension and variable                           Non-professional
                                                 players (n = 97)

Age, biological maturation
chronological age (years)                        13.54    (0.30)
relative age (month)                             5.07     (3.39)
age at peak height velocity (years)              14.05    (0.60)

Anthropometry
height (cm)                                      160.0     (8.4)
weight (kg)                                      47.5      (8.0)

Coach assessment
in-game performance (points)                      60       (16)

General motor performance
YoYo (m)                                         1101      (352)
counter-movement-jump (cm)                       31.0      (3.8)
40 m-sprint (sec)                                6.44     (0.33)
agility (sec)                                    8.10     (0.29)

Technical skills
dribbling (sec)                                  10.35    (0.59)
passing (sec)                                    16.8      (1.6)
juggling (points)                                 6.2      (5.2)

Psychological characteristics
achievement motive (net-hope)                    1.79     (0.88)
achievement goal orientations (score)            4.49     (0.39)
self-determination (index)                       8.91     (2.85)

Familial support
importance of football within family (score)     4.34     (0.68)
parents' priority of sport vs. school (score)    2.67     (0.97)
financial investment (Swiss Francs / year)       1723     (1940)
time investment (h / week)                        9.1      (8.7)

Training history
practice and play up to 12 years of age (h)      3160     (1017)

Dimension and variable                                 Total
                                                     (n = 117)

Age, biological maturation
chronological age (years)                        13.54    (0.31)
relative age (month)                             5.03     (3.47)
age at peak height velocity (years)              14.01    (0.95)

Anthropometry
height (cm)                                      160.2     (8.1)
weight (kg)                                      47.6      (7.7)

Coach assessment
in-game performance (points)                      63       (17)

General motor performance
YoYo (m)                                         1089      (361)
counter-movement-jump (cm)                       30.6      (3.7)
40 m-sprint (sec)                                6.43     (0.33)
agility (sec)                                    8.11     (0.28)

Technical skills
dribbling (sec)                                  10.31    (0.57)
passing (sec)                                    16.7      (1.6)
juggling (points)                                 6.7      (5.5)

Psychological characteristics
achievement motive (net-hope)                    1.81     (0.87)
achievement goal orientations (score)            4.53     (0.38)
self-determination (index)                       9.14     (2.81)

Familial support
importance of football within family (score)     4.40     (0.66)
parents' priority of sport vs. school (score)    2.80     (1.04)
financial investment (Swiss Francs / year)       1877     (2009)
time investment (h / week)                        9.7      (8.4)

Training history
practice and play up to 12 years of age (h)      3187     (1017)

Table 4. Significance, calibration, and model fit values from the five
binary logistic regression classification models.

Classification model         Omnibus tests of model
                                 coefficients

                          [chi square]   df    P

Coaches' eye model           23.63       1    <.01
Motor performance model      10.19       2    <.01
Multidimensional model       25.47       6    <.01
Coaches' eye and motor       37.02       3    <.01
performance model
Holistic model               47.46       7    <.01

Classification model         Hosmer-Lemeshow test    Model fit

                          [chi square]   df    P    Nagelkerkes
                                                     [R.sup.2]

Coaches' eye model            3.42       7    .84       .31
Motor performance model       2.81       8    .95       .14
Multidimensional model        5.09       8    .75       .33
Coaches' eye and motor        7.34       8    .50       .45
performance model
Holistic model                5.00       8    .76       .55

Table 5. Descriptive values of the receiver operating characteristic
curves for the five classification models.

Classification model      AUC * [95% CI]   Sensitivity [95% CI]

Coaches' eye model        .82 [.74; .90]     .95 [.75; 1.00]
Motor performance model   .71 [.58; .84]      .70 [.35; .95]
Multidimensional model    .85 [.76; .94]     .85 [.60; 1.00]
Coaches' eye and motor    .88 [.81; .95]     .95 [.70; 1.00]
performance model
Holistic model            .93 [.87; .98]     .90 [.75; 1.00]

Classification model      Specificity [95% CI]   Accuracy [95% CI]

Coaches' eye model           .66 [.47; .81]       .71 [.56; .82]
Motor performance model      .73 [.40; .96]       .73 [.53; .89]
Multidimensional model       .82 [.57; .98]       .82 [.63; .93]
Coaches' eye and motor       .71 [.56; .96]       .75 [.63; .93]
performance model
Holistic model               .87 [.70; .99]       .88 [.72; .96]

Classification model      YI ([dagger])

Coaches' eye model             .61
Motor performance model        .43
Multidimensional model         .67
Coaches' eye and motor         .66
performance model
Holistic model                 .77

* Area under the curve, ([dagger]) Youden Index

Table 6. Comparison with DeLongs nonparametric test (DeLong et al.,
1988) between the AUC of the classification models.

Models for AUC                                             AUC [95% CI]
comparison
([double dagger])

Coaches' eye model        vs.   motor performance model   .82 [.74; .90]
Coaches' eye model        vs.   multidimensional model    .82 [.74; .90]
Motor performance         vs.   multidimensional model    .71 [.58; .84]
model
Coaches' eye model        vs.   coaches' eye and motor    .82 [.74; .90]
                                performance model
Motor performance         vs.   coaches' eye and motor    .71 [.58; .84]
model                           performance model
Coaches' eye model        vs.   holistic model            .82 [.74; .90]

Multidimensional model    vs.   holistic model            .85 [.76; .94]

Coaches' eye and motor    vs.   holistic model            .88 [.81; .95]
performance model

Models for AUC                                             AUC [95% CI]
comparison
([double dagger])

Coaches' eye model        vs.   motor performance model   .71 [.58; .84]
Coaches' eye model        vs.   multidimensional model    .85 [.76; .94]
Motor performance         vs.   multidimensional model    .85 [.76; .94]
model
Coaches' eye model        vs.   coaches' eye and motor    .88 [.81; .95]
                                performance model
Motor performance         vs.   coaches' eye and motor    .88 [.81; .95]
model                           performance model
Coaches' eye model        vs.   holistic model            .93 [.87; .98]

Multidimensional model    vs.   holistic model            .93 [.87; .98]

Coaches' eye and motor    vs.   holistic model            .93 [.87; .98]
performance model

Models for AUC                                                  Z
comparison
([double dagger])

Coaches' eye model        vs.   motor performance model        1.50
Coaches' eye model        vs.   multidimensional model        -0.60
Motor performance         vs.   multidimensional model        -2.40
model
Coaches' eye model        vs.   coaches' eye and motor        -1.86
                                performance model
Motor performance         vs.   coaches' eye and motor        -3.23
model                           performance model
Coaches' eye model        vs.   holistic model                -2.97

Multidimensional model    vs.   holistic model                -2.35

Coaches' eye and motor    vs.   holistic model                -1.69
performance model

Models for AUC                                                  p
comparison
([double dagger])

Coaches' eye model        vs.   motor performance model        .13
Coaches' eye model        vs.   multidimensional model         .55
Motor performance         vs.   multidimensional model         .02
model                                                      ([section])
Coaches' eye model        vs.   coaches' eye and motor         .06
                                performance model
Motor performance         vs.   coaches' eye and motor        < .01
model                           performance model          ([section])
Coaches' eye model        vs.   holistic model                < .01
                                                           ([section])
Multidimensional model    vs.   holistic model                 0.02
                                                           ([section])
Coaches' eye and motor    vs.   holistic model                 .09
performance model

([double dagger]) Due to missing practical or theoretical relevance,
two comparisons (motor performance vs. holistic, multidimensional vs.
holistic) have been omitted for economic reasons. ([section])
Significant difference between the classification models (p < .05;
false discovery rate adjusted p-threshold: .031; Benjamini and
Hochberg, 1995).

Table 7. Coefficients of the holistic binary logistic regression
model.

Variable (#)                    [beta]   SE     Wald    df   P

In-game performance             2.53     0.73   11.86   1    < .01
General motor performance       -2.20    0.71   9.66    1    < .01
Familial support                1.65     0.58   8.21    1    < .01
Technical skills                0.35     0.56   0.38    1    .54
Psychological characteristics   -0.15    0.58   0.06    1    .80
Training history                0.08     0.35   0.05    1    .83
Maturation                      -0.05    0.57   0.01    1    .93
Constant                        -3.31    0.71   21.63   1

Variable (#)                    Odds Ratio [95% CI]

In-game performance             12.53 [2.97; 52.83]
General motor performance       0.11 [0.03; 0.44]
Familial support                5.20 [1.68; 16.07]
Technical skills                1.41 [0.47; 4.24]
Psychological characteristics   0.86 [0.28; 2.71]
Training history                1.08 [0.55; 2.12]
Maturation                      0.95 [0.31; 2.92]
Constant                        0.04

(#) Variables ranked by absolute value of beta coefficients.

Table 8. Coefficients of the multidimensional binary logistic
regression model.

Variable 1                      [beta]   SE     Wald    df   p

Familial support                1.45     0.49   8.75    1    < .01
Technical skills                1.10     0.47   5.54    1    .02
General motor performance       -0.99    0.48   4.22    1    .04
Psychological characteristics   0.54     0.49   1.20    1    .27
Maturation                      0.48     0.48   0.99    1    .32
Training history                0.07     0.29   0.05    1    .82
Constant                        -2.21    0.38   33.33   1

Variable 1                      Odds Ratio [95% CI]

Familial support                4.24 [1.63; 11.05]
Technical skills                3.00 [1.20; 7.48]
General motor performance       0.37 [0.15; 0.96]
Psychological characteristics   1.71 [0.66; 4.43]
Maturation                      1.61 [0.63; 4.14]
Training history                1.07 [0.60; 1.90]
Constant                        0.11

([paragraph]) Variables ranked by absolute value of beta coefficients.
COPYRIGHT 2019 Journal of Sports Science and Medicine
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2019 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Research Article
Author:Sieghartsleitner, Roland; Zuber, Claudia; Conzelmann, Marc Zibung Achim
Publication:Journal of Sports Science and Medicine
Article Type:Report
Date:Mar 1, 2019
Words:10882
Previous Article:The Proportion of Lower Limb Running Injuries by Gender, Anatomical Location and Specific Pathology: A Systematic Review.
Next Article:A Comparison of the Maximal Fat Oxidation Rates of Three Different Time Periods in The Fatmax Stage.
Topics:

Terms of use | Privacy policy | Copyright © 2019 Farlex, Inc. | Feedback | For webmasters