THE PROBABILITY OF SUCCESS AND FAILURE IN COMPLETING AN ASSEMBLY AND INSTALL OF COMPONENTS BY REFERRING TO WRITTEN MANUALS WITH OR WITHOUT DIAGRAMS: AN EXPERIMENTAL STUDY.
ABSTRACT: It is a practice for aircraft maintenance personnel to refer to aircraft maintenance manuals when conducting maintenance work on aircraft. However, previous studies have shown that the attribution of aircraft maintenance errors were due to documentation errors; incorrect information, insufficient information and unavailable information. This study is the second phase of a four-phased research focusing on the communication media used in an aircraft maintenance organization.
The first phase utilized interview method and risk assessment to investigate which communication media; verbal and written contributed to high risk of conveying critical information between departments. The result revealed that in the maintenance organization, written media such as documentation posed high risk in conveying critical information compared to verbal media. Hence, this paper narrowed its scope to the usage of written instructions among aircraft maintenance to discover the probability of success and failure in completing a maintenance task while referring to written manual, with or without diagrams. Two experiment conditions were employed and the result shows a 100% failure rate for those who were given only a written manual while assembling the components. However, only 60% of those who received written manual and diagrams were able to complete the task in a given time while 40% were unable to finish the task.
In conclusion, it was observed that in both experiments, diagrams play a vital role in enhancing one's understanding of a written manual to complete a maintenance task while being hasty resulted in the incompletion of the task given even though diagrams were available.
Keywords: documentation errors, written communication, lean media, aircraft maintenance task
Written media such as Emails and documentations are widely used when communicating critical information in organisations. The black and white medium can be regarded as proof to what has transpired from the verbal media such as meetings and discussions. Nevertheless, written communication does have flaws when the message is unclear, incomplete or incorrect. The misinterpretation of the written message can also occur when the message is correct. This happens when the readers tend to make up their own understanding of the message. Knowing the fact that humans are prone to making mistakes, especially in the aircraft maintenance field, it is vital to highlight the contributory factors of errors and the type of errors that may lead to aircraft incidence and/or accidents.
According to Shukri et al , the written communication that is conveyed between departments in an aircraft maintenance organisation can contribute to high risk of misunderstanding of critical information compared to verbal communication. This was due to the users' own interpretation of the information extracted from the written media such as documentation, E-mail and Software. The misunderstanding began when critical information was transferred from the Aircraft Management department to the Planning department, leading to the aircraft maintenance department. Hence, maintenance errors occurred when the maintenance personnel did not confirm and verify the information. Reason and Hobbs  regarded this error as "the failure of planned actions to achieve their desired goal, where this occurs without some unforeseeable or chance intervention" (p. 39). Violation, on the other hand, is the "wilful disregard for the rules and regulations that govern safe maintenance operations" .
There are two different ways by which errors and violations can affect aircraft safety:
* Primary Cause - the accident is due to the maintenance system failure.
* Contributing Factor - The accident chain begins with a component or system failure caused by a maintenance error, which was incorrectly handled by the flight crew and ultimately leads to an accident.
Flight crew error is the primary cause of such accidents .
Apart from that, Lattanzio et al.  explained that documentation error could also cause technicians to make mistakes. The procedural errors were due to not understandable, incorrect, insufficient and unavailable of information. Chen et al , on the other hand, conducted a study to identify significant threat and errors that affected aviation safety in Taiwan. It was found that procedural errors were an apparent factor that led to the incorrect installation of components, fitting of wrong parts, electrical wiring discrepancies, landing gear ground lock pins not removed before departure, cowling access panels and fairing not secured and inadequate lubrication. Their study, however, did not indicate the origin of procedural errors. Procedural errors where critical information presents either incorrect or inadequate information could also lead to breakdown of aircraft system .
To reduce the maintenance errors, Rankin  recommended that technical writers of aircraft maintenance manuals to produce user friendly manuals in order to enhance the understanding of the written instructions. This is particularly vital as to ensure the safety and efficiency of the aircraft.
In relation to the effectiveness of communication media in the organisation, Media Richness Theory (MRT) highlights the efficiency of verbal media and the inefficiency of written media [7, 8]. The written media in the context of aircraft maintenance organisation such as letter writing, documentations, emails and fax are considered as lean media as it has the potential to convey vague information. This is especially true when the readers comprehend the information using their own interpretation without verbally verifying their understanding. On the other hand, face to face meeting, phone calls and teleconferencing (verbal media) are considered as "Rich" media. This is due to its nature in allowing the information given or transferred to have an immediate response, use of various cues (pictures), use of natural language and personal focus on the medium .
This is commonly observed when an individual is unsure of the information he receives, he will make the effort to verbally ask and verify the information before taking further actions. The confirmation of the information that was first vague to the receiver is now considered as "rich".
These various studies on maintenance errors and violations particularly when written communication is involved will help researchers to understand in a deeper sense as to how readers interact with the written instruction, with or without the presence of diagrams. The experiment conditions in this study present both lean media (written instructions) and rich media (diagrams), which will hopefully give a new light on the actions taken by the users of technical documents in understanding the critical information in order to assemble and install components.
2. MATERIAL AND METHODS
The aim of this study is to analyse the probability of success and failure in assembling and installing components by referring to documents with or without the presence of diagrams.
Research Subject and Location
Sixty semester six Bachelor degree students at Universiti Kuala Lumpur Malaysian Institute of Aviation Technology (UniKL MIAT) aged eighteen and above were recruited. There were two experiment conditions; Experiment 1 (EC1) and Experiment 2 (EC2), with thirty students for each condition. The students were targeted as they have been exposed to theoretical and practical knowledge of Aircraft Engineering Technology in Mechanical and Avionics. The selection of this group of students was essential as they had the experience in referring to aircraft manual and assembly of aircraft components during their practical lessons.
Experimental Design and Instruments
There were two experiment conditions aimed at shedding light on the probability of success and failure in completing the task using written instructions. Lego was used as the instrument for this experiment to simulate the assembly and install of components on an aircraft. The Lego Red Rotors Set 31003 was chosen as the bricks provided could build three different vehicles; a mini aircraft, a mini helicopter and a mini hovercraft. The mini airplane was selected as the participants were aircraft engineering students and their basic knowledge on aircraft structure was hoped to help them to create the mini aircraft. It was also intended to mimic the Aircraft Maintenance Technicians who possessed theoretical and practical knowledge performing work on aircraft.
Since this study focused on written instructions and the package of Lego blocks only consisted of pictorial diagrams - showing the step-by-step process to build the product, therefore for the purpose of this experiment, written instructions had to be developed to imitate an Aircraft Maintenance Manual that was widely used as reference by Licenced Aircraft Engineers and Aircraft Maintenance Technicians before and during aircraft maintenance work.
The Lego instructions were developed by adopting the rules of Simplified Technical English. The manual consists of short sentences, 20 words for procedural sentences and 25 words for descriptive sentences, use of active voice, use of conjunction "that', introduce a list of item with a dash, and not to use clusters of more than three nouns. However, the rules had to be bent in some ways to suit the Lego diagram. The Lego written instructions were developed by first deciding on the number of stages necessary for the participants to assemble the model. What was given to the participants was actually a partially built mini aircraft. It had twenty six stages in total, however, only nine stages; stage eighteen to twenty six were selected for this experiment. This was due to the consideration of the length of the experiment (fifteen minutes).
Next the Lego instructions were developed. It was important to get the name of the Lego parts correct in accordance with the actual mini plane catalogue so that the participants could identify each item accurately based on the descriptions. The names of the Lego parts were accessed at the Lego website and each part had its individual name, colour and size, which were used in the Lego list and the Lego written manual. By referring to the description of the Lego parts, a list of Lego descriptions of the mini plane was developed. It contained four categories, mainly referring to the stage number, block, colour, quantity, and size. Later the assembly written instructions were developed based on the coloured pictorial diagram provided by the Lego Creator. This was downloaded from the Lego website.
It is important to note that the Lego written instructions which were developed had a combination of procedural and descriptive elements in one sentence, with a minimum of eight words in one sentence and maximum of twenty five words. The written instructions went through rigorous validation by the researcher and the verification was done by language experts. There were three stages of the validation process of technical documents in the aviation industry , and the Lego written instructions simulated these processes:
1. comparison of the written manual with the original Lego mini aircraft diagram, 2. a simulation process where the manual is checked against the Lego components by a native English speaker and 3. The demonstration where the mini plane was physically assembled and installed based on the written instruction and pictorial diagrams.
Three series of pilot tests were conducted to ensure the manual was written exactly as it was described in the Lego pictorial diagram. Ten volunteers participated in the pilot tests. Five of the volunteers were given the written manual only and another five were given the written manual and diagrams. As predicted, all five volunteers in the first group were unable to succeed in completing the task in fifteen minutes. Their failure appeared to have been due to the absence of diagrams. The other five volunteers in the second group, on the other hand, succeeded in creating the mini aircraft. Their success was due to their referring to both the written manual and diagrams simultaneously. The feedback from the pilot tests was very encouraging where suggestions were given to make the documents more user-friendly. Hence, numbers and arrows were inserted into the diagram to help the participants to identify the location of the parts to be assembled and installed as shown in Figure 2.
The Lego Experiment
Prior to the experiment, the researcher confirmed the name of the participant and thanked the participant for taking the time to volunteer for the experiment. Each participant was given a consent form to sign. The participant was then invited to conduct the experiment in a quiet room where a table and a chair were provided for the subject. For Experiment Condition 1 (EC1), using written instructions only, on the table, the participant was given an experiment instruction, a list of Lego parts needed to build the plane and Lego parts. After the subject read the experiment instruction, he/ she was given three minutes to read the list of Lego blocks needed to build the partially completed mini aircraft and placed the blocks on the table. This simulated the actual task of a maintenance crew where they would prepare the tools and aircraft parts and components before performing work.
The subject was then given the written manual and a partially completed mini aircraft and was asked to complete the remaining blocks in fifteen minutes by referring only to the written manual. The participants were instructed to complete the construction from stage eighteen to stage twenty six. After the fifteen minutes ended, the researcher inspected the final product and a verdict was verbally informed to the subject on whether he/ she was able or unable to complete the experiment correctly. The process and procedures for the second experiment was similar to the first experiment, only that the participants were given the written instructions with diagrams.
There was no intervention in the experiment as it was designed to find out the probability of success or failure of the experiment. Therefore, the analysis of the experiment outcome was very straightforward; successful or unsuccessful. The cut-off time for the experiment consisted of two conditions; a) when the fifteen minutes ended, or b) when the participant informed the researcher that he/she had completed the task and this could be at any point in time within the fifteen minutes given. This study intended to observe the participants use of written documents whilst completing the task in fifteen minutes. The Statistical Package for Social Science (SPSS) version 20 was used to analyses the correlation between the experiment conditions with the experiment outcome. The data, including the demography and outcome of the experiment, were keyed in and processed.
Once the experiment had ended, with the participant still present in the room, the final product was inspected thoroughly for its accuracy, making sure that each part was assembled and installed correctly. The experiment result was later verbally informed to the participants; either they were successful or unsuccessful in completing the task. A successful outcome of the experiment was when the participants managed to complete assembling and installing the partially built mini aircraft correctly in accordance with the instructions in fifteen minutes. An unsuccessful outcome was when the participants were unable to assemble and install the partially built aircraft correctly in accordance with the instructions in fifteen minutes. This was to ensure that the task was done in accordance with the Lego instructions.
3. RESULTS AND DISCUSSION
The Participants' Demographic Background
Sixty Aircraft Engineering students from Universiti Kuala Lumpur Malaysian Institute of Aviation Technology (UniKL MIAT) volunteered to take part in this experiment, using a convenient sampling of the volunteers; thirty students in each experiment conditions. This group of students was chosen as they had gone through five semesters of exposure to the theoretical and practical knowledge gained from the Bachelor's Degree in the Aircraft Engineering Technology Mechanical and Avionics programs in classrooms, hangar and workshops. Assembling and installing aircraft components in accordance with the aircraft maintenance manuals are part of their training at UniKL MIAT. Since these experiments simulated the real life of maintenance activities; maintaining, repairing and overhauling aircraft, which involve referring to written manuals and diagrams, their selection seemed practical.
The total number of semester six students was 159; Mechanical engineering students (119) and Avionics (40). All seven classes were approached and invited to take part in the experiment, however only sixty agreed to volunteer while the others declined for reasons of the clash of class schedule and/or lack of interest. In Table 1, the subjects in Experiment Condition 1 (EC1); written manual and Experiment Condition 2 (EC2); written manual with a diagram, consisted of sixty students with twenty seven (90%) males and three (10%) female students in each experiment. In terms of age, twenty eight (93%) of EC1 subjects aged between eighteen and twenty four years, and only two subjects (7%) aged between twenty five and thirty four years old. On the other hand, twenty seven (90%) of EC2 subjects aged between eighteen and twenty four years old, and only three (10%) were between twenty five and thirty four years old.
As for degree programmes, EC1 had twenty seven (90%) Mechanical engineering students but only three (10%) Avionics students, whereas, in EC2, the proportion is slightly lower, with twenty five (83%) Mechanical students and five (17%) Avionics students.
Table 1: Distribution of subjects in Experiment Condition 1 and Experiment Condition 2 based on Gender, Age, Race and Degree Programs (n=60)
Gender###Male###27 (90%)###27 (90%)
###Female###3 (10)###3 (10%)
Age###18-24###28 (93%)###27 (90%)
###25-34###2 (7%)###3 (10%)
Language###20 (100%)###20 (100%)
###Second Language Speaker
Degree Program###27 (90%)###25 (83%)
###Avionics###3 (10%)###5 (17%)
Relationship between Experiment Condition and Experiment Outcome
Based on the outcome of EC1, it was found that all thirty (100%) participants were unable to complete the partially built mini aircraft in fifteen minutes in accordance with the written instructions given. The result concurred to the first hypothesis where it was predicted that there would be a high probability of failure when they refer to written instruction while completing the task. The EC2 results, however, only sixteen (53%) of the thirty participants managed to complete assembling the mini aircraft while fourteen (47%) of them were unsuccessful. Thus, the second hypothesis that stated EC2 would have low probability was rejected. There was also a significant association between the experiment conditions and the outcomes of the experiment, where the value was p = 0.000. The results are shown in figure 4 below. Next, linear regression analysis was performed to further investigate how strong the relationship of both variables was.
Cohen and Pallant  point out that an R-value that is above 0.5 presents a strong correlation among variables. It can be seen from Table 2 that the relationship between both variables is 0.353, which according to Cohen, the strength of the correlation was moderate.
Table 2: Regression Summary for Experiment Conditions and Outcome of Experiment
Model###R###R Square###Adjusted###Std. Error of
The result of the first experiment observed concurred with the first hypothesis, where it was predicted that those who were given only a written instruction as a reference to assemble and install the mini aircraft would have a high probability of failure in assembling the Lego aircraft, with one hundred percent failure. As for the second experiment condition, it was anticipated that those who were given both written instruction and pictorial diagrams as their references would have a low probability of failure. However, the findings suggested that almost half of the EC2 participants (14%) were unable to complete the task given. From the observation of both experiments, it can be concluded that in order to assemble and install a complex object, users should not rely on written instruction alone even though the written manuals had gone through comprehensive validation and verification.
What is more, the study also suggested that the presence of diagrams together with the written instruction could still contribute to assembly error when the users misunderstood the instructions and when the task was done in a hasty manner in order to complete it on time. The overall findings show that the interaction between the readers, written manuals and diagrams can potentially affect aircraft safety and efficiency even though sufficient information such as diagram is provided to enhance the users' understanding in completing a maintenance task in a given time.
Thank you to Universiti Kuala Lumpur Malaysian Institute of Aviation Technology and University of Aberdeen, Scotland for the strong support.
 Shukri, S. A., Millar, R. M., Gratton, G., and Garner, M. The potential risk of communication media in conveying critical information in the aircraft maintenance organisation: a case study. In IOP Conference Series: Material Science and Engineering (Vol. 152, No. 1, p. 012044). IOP Publishing (2016).
 Reason, James and Hobbs, Alan. Managing Maintenance Error: A Practical Guide. Ashgate Publishing Limited (2003).
 Rankin William L., Shappell, Scott and Wiegmann, Douglous. Federal Aviation Administration Human Factor Guide for Aviation Maintenance and Inspection. Chapter 7: Error and Error Reporting System. http://www.hf.faa.gov/hfguide/07/07.html. Retrieved 11 August (2011).
 Lattanzio, D., Patankar, K., and Kanki, B. G. Procedural error in Maintenance: a review of research and methods. The International Journal of Aviation Psychology, 18(1), 17-29 (2008).
 Chen, C. C., Chen, J., and Lin, P. C. Identification of significant threats and errors affecting aviation safety in Taiwan using the analytical hierarchy process. Journal of Air Transport Management, 15(5), 261-263 (2009).
 Chaparro. A and Groff S. L. Human Factors Survey of Aviation Technical Manuals Phase 1: Manual Development Procedures. DOT/FAA/AR-01/43. Interim Report. Available at http://www.faa.gov/about/initiatives/maintenance_hf /library/documents/media/human_factors_maintenan ce/ar01-43.pdf. Retrieved 25 October 2013 (2001).
 Daft, R. L., and Lengel, R. H. Organizational information requirements; Media richness and structural design. Management Science, 32, 554-571 (1986).
 Daft, R. L., Lengel, R. H. and Trevino, L. K., Message equivocality, media selection, and manager performance: Implications for information systems. MIS Quarterly, 11, 355 - 367 (1987).
 Sheer, V. C., and Chen, L. Improving media richness theory. A Study of Interaction Goals, message valence, and task complexity in manager-subordinate communication. Research Notes. Management Communication Quarterly, vol. 18, no 1, pp76 - 93. Sage Publications (2004).
 Wampler, J., Blue, R., Volpe, C., and Rondot, P. (Service Manual Generation: An automated approach to maintenance manual development. Paper presented at the Proceedings of the 16th Symposium on Human Factors in Aviation Maintenance, FAA Human Factors (p. 1) 2002).
 Pallant, J. (SPSS survival manual. McGraw-Hill Education (UK) (2013).
|Printer friendly Cite/link Email Feedback|
|Date:||Oct 31, 2017|
|Previous Article:||AN INVESTIGATION THE EFFECT OF STATOR SLOT OPTIMIZATION IN 0.5HP THREE PHASE INDUCTION MOTOR VIA FEM.|
|Next Article:||FORECASTING CEMENT STOCK PRICES USING ARIMA MODEL: A CASE STUDY OF FLYING CEMENT INDUSTRY.|