Exploring the relationship between modularization ability and performance in the C++ programming language: the case of novice programmers and expert programmers.

The primary purpose of the study was to determine if a treatment that concentrates on building programs from previously written modules can improve the decomposition processes of novice programmers in the C++ programming language and, hence, programming performance. For purposes of performance analysis, a subsample of 23 students was divided into high- and low-performing groups based on calculus test and first C++ programming course scores. Pretest and posttest programming tests were administered to measure performance, and a questionnaire was administered after each test. A significant difference was found between the pretest and posttest mean scores. Students were also videotaped while solving the computer programs. Protocols and audiotaped interviews of these students supported many of the questionnaire findings. A comparison of the 65 questionnaire responses of the two groups indicated several statistical differences as well. Low-performing students improved their ability to divide problems into subproblems, but devoted less time to programming. These students experienced frustration and gave up more often, whereas the high-performing students experienced strong achievement in the ability to think logically and divide a problem into subproblems. Therefore, they were more patient, confident, and persistent in building programs. Although statistical tests reported significant differences between the pretest and posttest scores, some caution must be exercised in interpreting these results. Metacognitive factors such as patience and confidence play roles in the problem-solving performance.

INTRODUCTION

A common finding in the literature is a frequent failure of conventional teaching methods to develop students' programming knowledge to the fullest. The OOP (Object-Oriented Programming) was developed by researchers in the late 1960s but has found its way into the mainstream of computing and the modern computing curriculum. The seminal work by Stroustrup at Bell Labs in C++ is basically the C programming language with OOP extensions. OOP concepts and design are now integral to development in many modern information systems.

Due to the interest in computers and the realization of the seemingly endless growth possibilities in the field, the number of novice programmers is at an all-time high. There is a need to understand planning and the use of other strategies to help these novices in acquiring computer skills and to make the teaching of programming more effective. This study concentrates on the possible relationship between performance and semantic knowledge among novice programmers, with an emphasis on the modularization and planning processes involved in the C++ programming language.

BACKGROUND

Oliver (1993) examined an instrument to measure the achievement of semantic programming knowledge among novice programmers. Results indicated strong evidence of developmental levels among the three knowledge components of the Algorithmic Process Test (APT). Strong achievement in the lower levels of semantic programming knowledge was also shown, but few students demonstrated achievement at the highest levels despite its critical importance for programming success.

Bailie (1990) investigated the modularization ability of students to solve computer programs in introductory Pascal programming from previously existing modules. Results indicated a significant difference between the pretest and posttest means. However, tests to determine if improvement varied according to student ability or problem difficulty were not significant.

Barry (1988) examined the relationship between particular programming skills and task characteristics. A group of novice programmers was given two programs with different levels of difficulty. They used no terminals but wrote the programs as they thought aloud. The proceedings were taped, transcribed, and merged with the written work, and 24 separate strategies were postulated. Two raters established five meaningful and reliable categories with 94% agreement.

Three levels of programmer proficiency were identified for the easier program. The most capable group was abstract, emphasized structure and design, and reviewed their work. The middle group gave little evidence of structure or design, connected subparts poorly, and was concerned with details. The least capable group seemed overwhelmed and attempted to translate the program directly into code.

Four levels of proficiency were identified for the more difficult program. The top and bottom groups had planning strategies similar to their efforts on the easier program. However, two middle groups emerged: the more capable had facility with subparts yet had difficulty connecting these together; the less capable seemed careless, used partial solutions, and often left the program incomplete.

With the more difficult problem, the poorest group was forced to spend greater effort constructing a problem representation at the expense of planning. General planning was a minor part of the protocols, but was the greatest for the capable groups even when the program was easy. Less proficient novices focused on details; capable novices used plan approaches with less concern for details. The groupings obtained differed significantly from each other in average SAT-Math scores and first semester course grades, but not in mathematics achievement measures.

This study demonstrated that changing a program's level of difficulty results in a shift of strategies used. Training novices in representation and general planning is suggested. Restricting activities to one cognitive strategy could be used to observe changes in strategy development.

Maheshwari (1997) developed a model for delivering the first course in computer programming to large and diverse classes. A group of team teachers, tutors, and lab demonstrators implemented this model, which took advantage of the unique characteristics of lectures, tutorials, laboratories, and readings. To improve the learning environment in large programming classes, the environment emphasized program design skills and problem-solving skills, and motivated students to learn by active processes instead of concentrating on language syntactic features.

A series of studies conducted by Perkins, Schwartz, & Simmons (1988) revealed some metacognitive influences involved in programming. The goal of research conducted by Perkins, Martin, and Farady (1986) was the identification of student difficulties in programming. The theoretical basis of their research lay in their perception of programming as both a "problem-solving intensive" and a "precision intensive" domain. They suggested that as a problem-solving activity, programming requires the ability to decompose a problem into subproblems and then to implement the solution in correct programming code. One paradigm used to portray the nature of introductory programming was a "precision intensive" one, in which the smallest error invalidates a program. While other paradigms focusing upon strategies employed could have been used, coding precision instead was considered paramount in this study. The experimenters argued that the combination of these two factors makes programming an extremely demanding subject for pre-college students.

PROJECT DESIGN

Setting and Subjects

The subjects for this study were selected from students enrolled in the course Computer Techniques, which is an Intermediate C++ programming language. The computer course is a required computer science course for Computer Information Systems (CIS) majors. Two complete classes of 46 students were selected at random to participate in the study. Students were informed about the research. A reduced subsample of 23 students who exhibited different levels of understanding and various programming competencies were identified for this study.

Questionnaire Development and Analysis

The aim of this study was twofold: (a) concentrate on the possible relationship between performance and the semantic knowledge among novice programmers in the C++ programming language, and (b) seek a systematic description of the cognitive processes involved in writing a computer program in C++, especially knowledge of algorithms and ability to create algorithms among novice programmers.

A questionnaire was developed to ascertain the understandings of modularization that novice and expert programmers hold regarding problem-solving activity in C++. The questionnaire was designed to elicit information on the understandings, motivations, and experiences of problem-solving activities. The questionnaire was administered immediately after the pretest programming test that preceded the treatment. A revised questionnaire was also administered immediately after the posttest programming at the end of the treatment period.

In order to relate the modularization programming and understandings identified by the questionnaire with programming performance, the students were divided into high- and low-performing groups based on the posttest programming test administered at the conclusion of the treatment.

Test Problem Development

Test questions for the pretest and posttest were selected from various textbooks used in three undergraduate institutions. A panel of five experts from three undergraduate institutions reviewed each of the 10 problems for its appropriateness in testing the modularization ability of high-performing students versus low-performing students. Each expert was asked to rate the difficulty level of each problem on a scale from 1 (very easy) to 5 (very hard).

Pretext 

Problems were selected based on the required chapters in Introduction to C++ Programming (Control Structures to Pointers) and ranged from easy to complex problems. The pretest was administered to students as a proctored, open-book, open-notes test. The 23 students in the subsample were videotaped while solving the pretest problems. The test was graded by the investigator using the scoring scheme presented in Table 1. All pretests were reviewed by the graders and any scores that differed were graded again. A list of calculus scores and programming test scores can be found in Table 2. The module problems were on the control structures in C++ and the use of pointers.

Treatment

The purpose of the treatment was to determine if modularization skills could be improved by an instructional treatment that concentrated on solving problems from previously existing modules. The treatment was integrated into the regular classroom lecture schedule. The modules and programs used in the treatment were drawn from programming texts and from former assignments developed by the instructor. The treatment period followed the pretest and took place during the programming course. The course met twice a week and consisted of two 65-minute lecture periods and one 65-minute computer laboratory per week. Individual modules were solved together in class and students were assigned individual procedures and functions as lab assignments and programming projects. After the modules were written and tested, programs were assigned that utilized the previously written procedures and functions. It was the intention that the pre-written modules would facilitate the solution of these programs and students would become aware of the advantages of the modularization process.

Posttest

Four and a half weeks after the Pretest Data Structures, topics such as classes, friend functions, operator overloading, inheritance, virtual functions, and polymorphism were tested using modules. The investigator and three graders graded each test, and any differences were reconciled during consultation.

ANALYSIS OF RESULTS

Prior to the analysis of the results of the treatment, students were divided into high- and low-ability groups, based upon the total scores in the first course in the C++ programming and scores in calculus courses. The research question that this study attempted to answer was: Does the treatment that emphasizes practice in building programs from previously written modules improve the decomposition processes of novice programmers and, hence, programming performance?

The means and standard deviations of the pretest and posttest for high-ability students and low-ability students and the group as a whole were compared. The high-performing group mean was 75.625, with a standard deviation of 16.132 for the pretest; the low-performing group mean was 60.000, with a standard deviation of 13.228. During the posttest, the high-performing group mean was 87.125, with a standard deviation of 6.993; the low-performing unit had a mean of 68.833, with a standard deviation of 12.120. The scores are displayed in Table 3.

The results indicated that there was an improvement of the modularization strategy for the two groups: low-performing and high-performing. The scores of high-ability students were higher than the scores of low-ability students.

A comparison of the 65 responses on the questionnaire given by each participant supported many of these findings. All analysis referred to the posttest questionnaire. Several statistical differences were found. The questionnaire outline is displayed in Table 4. A summary of statistical significance is displayed in Table 5. The summary of statistical significance was provided by the Kendall correlation and the t-test to explain the improvement of modularization at different levels. The results are reported in Table 5.

The Kendall nonparametric correlations were significant at the 0.01 level. The correlation for the relationship between ability to divide a problem into subproblems and patience, persistence was .733.

A multiple regression analysis test was run to determine the best predictor among persistence, ability to think logically, patience, ability to divide a problem into subproblems, and a dependent: the vast knowledge of programming techniques. The coefficients were positive with F= 7.143 at .001; the results of this test are displayed in Table 6.

F was significant at .001, indicating that the improvement of modularization was due to variables such as persistence, confidence, and so on.

CONCLUSIONS

Although statistical tests reported significance between the pretest and posttest, some caution must be exercised in interpreting these results. The modularization technique improvement was possibly attributable to time, since there was a period of 4 1/2 weeks between the pretest and the posttest, during which programming instruction continued. In addition, the results of the videotaping and the exit interview not only supported and enhanced many of the findings from the questionnaire, but also provided additional insights about metacognitive factors such as confidence, persistence, beliefs, and understandings that play a role in the problem-solving performance of novice computer programmers in the C++ programming language.

INSTRUCTIONAL IMPLICATIONS

The results of this study have certain implications for instructional methodologies employed for novice computer programmers in the C++ programming language. This methodology of planning and designing through modularization seems to advocate guiding students to build libraries and to implement functions, especially for classes that can be employed in the solutions of larger and more complex problems.

References

Bailie, F. K. (1990). The relationship between a novice programmer's set of understanding and programming performance (Doctoral dissertation, Columbia University, 1990). Dissertation Abstracts International, 085B157.

Barry, T. K. (1988). Planning levels of novices in introductory computer programming (Doctoral dissertation, Columbia University, 1988). Dissertation Abstracts International, 085B276.

Maheshwari, P. (1997). Improving the learning environment in first-year programming: Integrating lectures, tutorials, and laboratories. International Journal of Computers in Mathematics and Science Teaching, 16, 111-131.

Oliver, R. (1993). Measuring hierarchical levels of programming knowledge. Journal of Educational Computing Research, 9, 299-312.

Perkins, D. N., Martin, F., & Farady, M. (1986). Loci of difficulty in learning to program_(Technical Report 86-6). Cambridge, MA: Educational Technology Center.

Perkins, D. N., Schwartz, S., & Simmons, R. (1988). Instructional Strategies for the problems of novice programmers. In R. E. Mayer (Ed.), Teaching and learning computer programming: Multiple research perspectives (pp. 153-178). Hillsdale, NJ: Lawrence Erlbaum Associates.

MAURICE A. VODOUNON

[e-mail]

John Jay College of Criminal Justice 

New York 

USA

Table 1 Scoring Scheme

Program Plan  Points  Category

              0       No overt attempt at modularization
              1-2     Modularization attempted but major modules missing
                        or too detailed
              3-4     Modularization good but minor errors
              5-6     Reasonable modularization but not expressed
                        coherently
              7-8     Reasonable modularization and coherently expressed
Program Plan  0       No refinement or negligee refinement
              1-2     Refinement attempted but major errors
              3-4     No major errors but some details overlooked or
                        incorrect
              5-6     Accurate solution with minor errors in logic
              7-8     Complete and accurate solution with no errors in
                        logic

Table 2 List of Students' Ability Measures

Students' group     CALCULUS TEST  COMPUTER TEST  STUDENT

Low-performing #1        60              70          1
High-performing #1       80              75          2
Low-performing #2        65              60          3
Low-performing #3        70              65          4
Low-performing #4        65              60          5
Low-performing #5        55              55          6
Low-performing #6        62              67          7
Low-performing #7        59              70          8
Low-performing #8        58              65          9
Low-performing #9        72              68         10
Low-performing #10       68              63         11
Low-performing #11       61              65         12
High-performing #2       82              80         13
High-performing #3       79              85         14
High-performing #4       83              82         15
High-performing #5       85              88         16
High-performing #6       90              95         17
High-performing #7       92             100         18
High-performing #8       91              97         19
Low-performing #12       57              62         20
Low-performing #13       56              59         21
Low-performing #14       59              63         22
Low-performing #15       66              67         23

Table 3 Pretest and Posttest Scores

                     Pretest         Posttest
Group               Mean    SD      Mean    SD      Difference

Low-performing LP   60.000  13.228  68.833  12.120   8.833  1.108
High-performing HP  75.625  16.132  87.125   6.993  11.5    4.632

Table 4 Questionnaire Outline

  I. Motivations
     A. Reasons to study C++ programming language (Question #1)
     B. Expectations from studying C++ programming (Question #3)
     C. Success in other areas (Question #4)
 II. Understandings and Modularization
     A. Essence of programming (Question #5)
     B. Essentials for success
          1. Qualities for success (Question #2)
          2. Learn by memorizing algorithms (Question #12c)
          3. Learn by memorizing rules (Question #12d)
     C. Strategy models
          1. Teacher expectations (Question #6)
          2. Professional programmers (Question #8)
     D. Opinions
          1. Getting program to work is the only important part of
             activity (Question #12a)
          2. Knowing how to solve a problem is as important as getting
             it to work (Question.#12b)
          3. There is only one correct way to solve a problem (Question
             #12e)
          4. No solution means all time has been wasted (Question #12f)
     E. Personal evaluation of ability (Question #13)
III. Experiences
     A. Personal strategies (Question #7)
     B. Effort expended
          1. Time spent on problem (Question #9)
          2. Time spent on problem when cannot solve (Question #10)
     C. Emotional responses
          1. Frustration or challenge when cannot solve (Question #10)
          2. Other emotions (Question #11)

Table 5 Summary of Statistical Significance

  The t-test indicated that there was a significant difference in the
means of the following questions:
 1. High-performing students ranked sense of humor, patience,
    persistence higher as qualities of a successful programmer.
    (Questions #2h, #2f, #2g)
 2. High-performing students gave a high mark to ability to divide a
    problem into subproblems (modularize). (Question #2b)
 3. High-performing students responded that teachers often expected them
    to use modularization as a strategy. (Question #6d)
 4. High-performing students reported that professional programmers more
    often use modularization as a strategy. (Question #8d)
 5. High-performing students revealed that confidence is a key to
    success in problem solving. (Question #11g)
 6. Low-performing students ranked vast knowledge of programming
    techniques higher as a quality of a successful programmer. (Question
    #2e)
 7. Low-performing students reported that they are frustrated and give
    up immediately more often. (Question #10a)
 8. Low-performing students agreed more often with the statement that
    professional programmers see a solution immediately or in a very
    short time when faced with a programming problem that is difficult
    for them to solve. (Question #8a)
 9. Low-performing students agreed also with the statement that the best
    way to learn how to program is to memorize algorithms for problems.
    (Question #12c)
10. Low-performing students rate themselves poor in problem-solving
    ability in computer programming. (Question #13e)

Table 6 Multiple Regression Analysis Results

Model       Sum of Squares  DF  Mean Square  F
Sig.

Regression  40.865           4  10.216       7.143  .001
Residual    25.744          18   1.430
Total       66.609          22