Optimal selection of artificial neural network parameters with design of experiments: an application in modeling milling of carbon fiber reinforced composites.
The interest in data driven or non-parametric forms of modeling to create response surfaces of highly nonlinear manufacturing phenomena is high. This is especially true for response surfaces built for the purpose of finite element modeling. The nonparametric approach, artificial neural networks (ANN), is very attractive because it does not require knowledge, distributional, or model assumptions regarding the relation of inputs to outputs. The authors have shown the superiority of an ANN approach to modeling over the standard statistical methods in modeling cutting forces while machining carbon reinforced composites [1-3]. However the ANN approach to modeling is not without its problems. ANN performance can fail when problems are not well posed.
Most literature related to ANNs focused on specific applications and their results rather than the methodology of developing and training the networks. In general, numbers of ANNs' parameters, such as number of hidden node and hidden layer, have to be set during training process and these setting are very crucial to the accuracy of ANNs model. Trial-and-error method is usually used to determine the appropriate setting of these parameters . In building and validating an ANN model of nonlinearity and dimensional space process a decision is required to determine how much effort (data) will be put into building a better performing model (training) or into obtaining a better estimate of the model's true error (testing). Because there are insufficient data to accurately assess the model, performance is often measured according to the training set error. The outcome is the well-known ANN phenomenon of overtraining or overfitting. The applications of Design of Experiment (DoE) techniques to optimize ANNs parameter were reported in literatures [5-7]. Factors that often found to have the significant effect on ANNs accuracy are number of neurons in hidden layers, learning rate and momentum.
Machining fiber reinforced polymers generates force data that is highly nonlinear and dispersed. This is due to the fact that FRPs are inhomogeneous materials consisting of hard and soft phases. The ability to predict the cutting forces during machining FRPs is of great technical significance because tool forces are responsible for creating machining defects such as delamination, matrix smearing and burning. In this case study, both multiple regression and ANN models were used to predict the specific cutting energy for unidirectional FRPs. The cutting forces are determined from specific cutting energy by multiplying the latter with the uncut chip thickness.
ANN was used as a tool to predict the cutting forces for different combination of factor settings. Neural works professional Il/plus  was the ANN software used for training and testing the experimental data. The effect of varying parameters in ANN during the training of data was studied based on the predicted forces. The deviation between experimental and predicted values indicates Root Mean Squared Error (RMSE). This paper purpose was to identify the effect of training parameters in ANN that might influence the RMSE. Furthermore, the study involved arriving at a combination of parameter settings that minimizes the RMSE. It is shown that under conditions of noisy data this model can be constructed and validated without regard to overfit. The outcome is a reduction in effort to determine network size.
Testing Sensitivity of Ann Model S A Prediction Tool
An orthogonal edge trimming experiment on CFRP material was performed in order to study the effects of various process parameters viz. feed rate, fiber orientation and spindle speed . The resulting cutting forces Fx (in normal direction) and Fy (in feed direction) for different combination of factors is utilized in the study. The objective of this work was to identify the training parameters which have significant effects on the prediction capability of ANN. The individual and interaction effects of changes in the training parameters viz. the number of hidden layers, type of learning rule and type of transfer function on the forces predicted by ANN were studied. The study involved testing the sensitivity of Neural Works Professional II/ Plus , an ANN software which was used as a tool to predict forces generated during the machining process. Combinations of feed rate, spindle speed and fiber orientation at different levels and their corresponding responses (Fx and Fy) are fed into ANN for training (refer Table 1). ANN was trained to establish a relationship between the inputs and the outputs. A BP network with three input and two output layers was selected. A learning rate of 0.300 and momentum of 0.400 for all the different combinations of factor settings was utilized to construct neural network. Each combination of input was run for 50000 iterations before the output in terms of predicted forces and their corresponding RMSE values were recorded.
Different combinations of feed rate, spindle speed and fiber orientation as shown in Table 2 was used as inputs for predicting the forces Fx and Fy. For the same levels of input, the predicted output changes by varying the number of hidden layers, type of learning rule and transfer function. The predicted forces were compared with the experimental values and the accuracy in terms of RMSE was determined. A combination of training parameters in ANN which yields minimum RMSE was arrived at.
The ANOVA procedure for factorial designs in Design-Expert 8 software  was utilized to identify the parameters in ANN that have significant effects on the response variable RMSE. Considering the complexities involved in machining composites, employing ANN as a tool to predict the outcome can be very beneficial. Accurate predictions of process outcomes can eliminate the necessity to run pilot process thus saving time and money to the manufacturing industries.
A back propagation type neural network was utilized in the study where in each set of input and corresponding presentation of data sets were compared with desired values of output to minimize the error associated with prediction. Each combination of training parameters (inputs) and corresponding outputs in terms of cutting forces were fed into ANN as shown in Table 1. A Neural Network was trained with different combinations of parameters by running 50000 iterations. A combination of parameters at different levels as shown in Table 2 was fed into the ANN and allowed to predict the output (Fx and Fy). The prediction of output values for a given set of parameters was based on the relationship established during the training.
The ANOVA for different combinations of training parameters and their corresponding RMSE values was constructed. It defines the relationship between the system parameters and the RMSE. Furthermore, it helps identify the factors and interactions that have significant effects on the RMSE value. The main objective was to identify the training parameters which have significant effects of prediction capability of ANN measured in terms of RMSE value. The RMSE was calculated using the same equation as in  and is given by.
[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII]
A minimum RMSE suggests a better prediction model.
Choice of Factors and their Levels
Back propagation type, feed forward neural network was utilized in the study for predicting the cutting forces for different combination of training parameters. Spindle speed, feed rate and fiber orientation were the inputs and the corresponding forces, Fx and Fy, were entered into the system for training the neural networks. A different combination of inputs (spindle speed, feed rate and the fiber orientation) was then entered and the neural network was allowed to predict cutting forces at these levels. The learning co-efficient and momentum were kept constant at 0.300 and 0.400 respectively for all the combinations. Three input layers and two output layers were employed during the training by varying the number of hidden layers between one and two. The control parameters (factors) that were selected for the study were as follows.
Number of Hidden Layers: The hidden layers recode and provide representation for the inputs. More than one hidden layer can be used during the training and testing phase of neural networks. The current study involved evaluating the effect of changes in the number of hidden layers on the response variable. The number of hidden layers was varied between one and two.
Learning Rules: A learning rule is a method of gathering given information and conveying it between the input and output in order to discover optimal point. The learning rule is responsible for making adjustments in the parameters in a systematic manner to reduce the associated error. This is a system parameter that is selected by an engineer designing the neural network. In the current study, six different learning rules were employed to study the effect of each on the response variable and identify the one that minimizes the resulting RMSE. This categorical factor was tested at six different levels viz. Norm-Cum-Delta, Delta, Ext DBD, Quick Propagation, Max Propagation and Delta Bar Delta
Transfer Functions: The function that determines the behavior of input and output variables is referred to as transfer function. A transfer function takes input in any range and produces output within a particular range. They are also referred to as activation functions and the current work involved five different transfer functions. The transfer functions tested were Linear, TanH, Sigmoid, DNNA and Sine. Table 3 shows different levels of training parameters.
Choice of Design
There were three training parameters considered in the study viz. number of hidden layers, type of learning rule and type of transfer function. Each of these factors was tested at different levels to quantify their effects on the predictions made. The predicted forces were then compared with the experimental forces to calculate the RMSE. A full factorial design in these three factors required 2x6x5 = 60 runs.
Results and Discussion
The study involved testing the sensitivity of the Artificial Neural Networks to predict the cutting forces in an edge trimming experiment. Different combinations of training parameters in the ANN software Neural Works Professional II/Plus  were tested. The predicted values were compared against the experimental data to test for error associated with prediction. The ANOVA procedure was utilized in order to identify the parameters which have significant effects on the predictions made and arrive at a combination of system parameters that yields minimum RMSE.
As discussed in the previous section, experimental data from  was utilized for training and testing the prediction capability of the network. Neural Works Professional II/Plus was used in the study. A back propagation type neural network with learning rate of 0.300 and momentum of 0.400 was used in the study with different combination of parameter settings. The training parameters that may affect the prediction capability of the network were the number of hidden layers, type of learning rule and type of transfer function. The study was aimed at identifying the training parameters which have significant effects on the prediction capability. The prediction capability is measured in terms of RMSE (Root mean squared error).
By varying the levels of training parameters, ANN was allowed to predict the forces for test data shown in Table 2. The RMSE corresponding to forces predicted was then utilized to identify the significant training parameters and arrive at an optimal combination that minimizes the RMSE (in other words improve the prediction capability) in neural networks.
From the effects list shown in Table 4, it can be observed that factor C (transfer function) has the highest percentage contribution of 85.6% followed by interaction effect of B (learning rule) and C (transfer function) with a percentage contribution of 7.21%. The main effect of factor B has the next highest percentage contribution of 5.67%.
From the half normal plot shown in Figure 3, it can be observed that factors B (learning rule), C (transfer function) and their interaction are significant. The half-normal plot is for transformed response variable [RMSE.sup.-1] .
[FIGURE 3 OMITTED]
The values of response variable (RMSE) were analyzed using the ANOVA procedure. The ANOVA table for transformed response ([RMSE.sup.-1]) is shown in Table 5. A transformation on the response variable at this stage was required to stabilize the variance thereby satisfying the ANOVA assumptions. It can be observed from Table 5 that the interaction effects between factors B and C is significant with a p value of less than 0.0001. The effects of training parameters B (learning rule) and C (transfer function) also have a p value of less than 0.0001 respectively. The ANOVA verifies the observations made from the effects list and half normal plot discussed above.
Model Adequacy Tests
The normal probability plot of residuals from the fitted model is shown in Figure 4. Figure 4 indicates that most of the residuals fall on the straight line suggesting that they follow the normal probability distribution.
[FIGURE 4 OMITTED]
[FIGURE 5 OMITTED]
The outlier plot in Figure 5 does not detect any outliers in the data suggesting that there are no outliers in the data set.
[FIGURE 6 OMITTED]
The residuals vs. predicted plot in Figure 6, displays a random scatter with no specific pattern. This satisfies the assumption of constant variance of ANOVA indicating a constant range of residuals across the graph.
The Residuals vs. factor plots (Figure 7-9) indicates no trend in the residuals data thereby satisfying the ANOVA assumptions.
[FIGURE 7 OMITTED]
[FIGURE 8 OMITTED]
[FIGURE 9 OMITTED]
Interpretation of Results
The training parameters in the prediction tool that have significant effects on the outcome were identified. The prediction capability was measured in terms of RMSE (Root Mean Squared Error). The objective was to identify the training parameters which can be used to minimize the RMSE (a better prediction model). Different combinations of training parameters were tested in a factorial arrangement and the ANOVA procedure was utilized to arrive at appropriate settings. The study revealed that ANN training is sensitive to changes in the transfer function and learning rule used. The appropriate learning rule cannot be specified without consideration of the transfer function. In the case of predicting forces, the minimum RMSE was obtained by using a "Sigmoid" transfer function with "Quick Propagation" as a learning rule. It was also concluded that the number of hidden layers did not have a significant effect on the prediction capability. From the ANOVA table, it can be seen that the interaction effect between the type of learning rule (B) and type of transfer function (C) is significant. The interaction plot between factors B and C is shown in Figure 10. It can be noticed that the minimum RMSE was obtained when a "Sigmoid" transfer function was used with "Quick propagation" as a learning rule. However it appears that use of sigmoid transfer function with all other learning rules except the Max Propagation would minimize the RMSE.
Figure 11 shows the 3D effects plot for interaction between factors B and. It can be noted from Figure 11 that the minimum RMSE was obtained when a sigmoid transfer function was used with Quick Propagation.
[FIGURE 10 OMITTED]
[FIGURE 11 OMITTED]
Neural networks provide good solutions to a variety of modeling problems in manufacturing where the data are highly dimensional and relationships highly nonlinear. By nature these problems not often well posed because the necessary numbers of data is either too expensive or impossible to collect. The effect of insufficient data is poor generalization and poor estimate of the model's capabilities. The results of this research have shown that DOE to be a superior approach to ANN construction and validation that makes a) the most effective use of nonlinearity data; and b) mitigates overfit in noisy and dimensional space conditions. The outcome for the modeler is a reduction in effort to determine network size. The study dealt with testing the sensitivity of a prediction tool like Artificial Neural Networks. They were utilized in prediction of forces in an edge trimming operation of unidirectional CFRP composites. ANN can predict the forces generated in a machining operation based on experimental data. To test the capability of ANN in predicting cutting forces, the data was used for training and testing. A back propagation type network with a learning rate of 0.300 and momentum of 0.400 was utilized.
 Kalla, D., Sheikh-Ahmad, J. & Twomey, J., (2010). "Prediction of Cutting Forces in Helical End Milling of Fiber Reinforced Polymers", International Journal of Machine Tools & Manufacture, 50 (10), 882-891.
 Sheikh-Ahmad J, Twomey J, Kalla D, Lodhia P (2007) Multiple regression and committee neural network force prediction models in milling FRP. Machining Science and Technology 11(3), 391-412.
 Kalla, D., Nahusha, K. and Dhanasekaran, PS (2011). "Optimization of Cutting Forces in End Milling of CFRP composites using Response Surfaces and Desirability Function Approach", SAMPE, LongBeach, CA, May 23-26.
 Laosiritaworn, W. and Chotchaithanakorn, N (2009). "Artificial Neural Networks Parameters Optimization with Design of Experiments: An Application in Ferromagnetic Materials Modeling", Chiang Mai J. Sci. 36(1), pp. 83-91.
 Kim Y.S., and Yum B.J., Robust Design of Multilayer Feed Forward Neural Networks: An Experimental Approach, Appl. Artif. Intell., 2004; 17: 249-263.
 Yang S.M., and Lee G.S., Neural Network Design by using Taguchi Method, J. Dyn. Syst. Meas. Contr. 1999; 121: 560-563.
 Khaw J.F.C., Lim B.S., and Lim L.E.N., Optimal Design of Neural Networks using the Taguchi Method, Neuralcomputing, 1995; 7: 225-245.
 Neural Works Professional II/Plus software, Neuralware, Pennsylvania.
 Design Expert software, Version 8, Stat-Ease, Inc., Minneapolis.
(1)* Devi K. Kalla, (1) Aaron Brown and (2) Nahusha Kumar
(1)* Mechanical Engineering Technology, Metropolitan State University of Denver Denver, CO, USA (2) Department of Industrial and Manufacturing Engineering, Wichita State University, Wichita, KS, USA
* Corresponding Author E-mail: email@example.com
Table 1: Machining Factors and their corresponding levels (Training Data) Spindle Fiber Sl. No Speed Feed Rate Orientation Avg Fx Avg Fy 1 -1 -1 -1 20.48 19.08 2 0 -1 -1 15.65 15.93 3 0 0 -1 18.27 17.88 4 1 -1 -1 18.98 16.15 5 1 0 -1 25.76 21.27 6 1 1 -1 33.00 26.29 7 -1 -1 0 43.54 30.60 8 -1 1 0 52.31 55.45 9 0 -1 0 35.63 25.21 10 0 1 0 62.01 49.87 11 1 -1 0 44.05 33.00 12 1 1 0 52.51 29.19 13 -1 -1 1 42.84 45.61 14 -1 0 1 96.31 75.06 15 -1 1 1 47.35 62.73 16 0 -1 1 37.03 38.31 17 0 1 1 52.23 55.94 18 1 0 1 45.90 40.62 19 1 1 1 43.55 46.90 Table 2: Machining factors for testing ANN Spindle Speed Feed Rate Fiber Orientation -1 0 -1 -1 1 -1 0 1 -1 -1 0 0 0 0 0 1 0 0 0 0 1 1 -1 1 Table 3: Training parameters and their corresponding levels Factors Level 1 Level 2 Level 3 Number of 1 2 N/A hidden layers Learning Rule Norm Cum Delta Delta Ext DBD Transfer Linear TanH Sigmoid Function Factors Level 4 Level 5 Level 6 Number of N/A N/A N/A hidden layers Learning Rule Quick Max Delta bar Propagation Propagation Delta Transfer DNNA Sine N/A Function Table 4: Effects list for factors Term DOF Require Intercept Error A-Hidden Layer 1 Model B-Learning Rule 5 Model C-Transfer Function 4 Error AB 5 Error AC 4 Model BC 20 Error ABC 20 Sum of Squares % Contribution Require Error 0.182154264 0.05028751 Model 20.57006808 5.678799272 Model 310.3267023 85.67220311 Error 1.89143659 0.522170791 Error 0.610006888 0.168405212 Model 26.12904322 7.213471098 Error 2.516247657 0.694663008 Table 5: ANOVA Table for [RMSE.sup.-1] Source Sum of Squares df Mean Square B-Learning Rule 20.57006808 5 4.1140 C-Transfer Function 310.3267023 4 77.5816 BC 26.12904322 20 1.3064 Residual 5.199845398 30 0.1733 Cor Total 362.225659 59 Source F-Value p-value Prob > F B-Learning Rule 23.7353 < 0.0001 C-Transfer Function 447.5998 < 0.0001 BC 7.5374 < 0.0001 Residual Cor Total
|Printer friendly Cite/link Email Feedback|
|Author:||Kalla, Devi K.; Brown, Aaron; Kumar, Nahusha|
|Publication:||International Journal of Computational Intelligence Research|
|Date:||Apr 1, 2012|
|Previous Article:||Impact of transportation cost for optimum design FMS layout: a genetic algorithm approach.|
|Next Article:||Anonymity Trust Management Scheme (ATMS) for clustered wireless sensor networks.|