# GENETIC ALGORITHM ADAPTIVE MECHANISM FOR EARLIER CONVERGENCE IN NEURAL NETWORK PREDICTION MODEL.

Byline: F. Sh Ismail and N. Abu BakarABSTRACT: This research presents a Neural Network prediction model, developed to learn from seven inputs fed to the network to produce three targets. In order to avoid result from local optimum scenario by the Back Propagation optimizer, Genetic Algorithm optimizes synaptic weights of the network towards reducing prediction error. Adaptive mechanism in genetic algorithm is a suitable approach for it allows fitness monitoring in each generation. The research initially used a fixed probability rates for crossover and mutation. The fixed rates of genetic operators in genetic algorithm will be adjusted accordingly by adding an adaptive mechanism to search for the best probability rates. The fitness value refers to the Sum of Squared Error. The algorithm was used in the Neural Network model to predict the physical properties of the Medium Density Fiberboard.

Performance results show the adaptive mechanism in Genetic Algorithm helps increase capability to converge sooner than the ordinary Genetic Algorithm. The results also suggest replacing the Medium Density Fiberboard testing procedures normally done manually in pilot plants. As a contribution, the simulation has caused some cost reduction in testing procedures of the physical properties of Medium Density Fiberboard.

1. INTRODUCTION

Prediction models contribute solutions for related issues to be able to learn from previous knowledge. As such, this model is able to assist us understand our data better and thus increase our efficiency. One of the popular prediction techniques for non-linear data is Neural Network using Back Propagation optimizer. This technique is reliable to solve similar domain and used by many [1, 2, 3]. The setback of Back Propagation optimizer is the possibility to be stuck at local optimum. This will cause model to produce a non-optimum result. The Genetic Algorithm-Neural Network (GA-NN) hybrid is found to be more efficient in prediction techniques, however needs some improvements to suit the problem domain [4, 5]. Adaptive mechanism in genetic algorithm is a suitable approach for it allows fitness monitoring in each generation [6, 7].

This research will address these issues to improve testing procedures by reducing the amount of time, as well as consequently reduce the cost of the testing procedures. The research will also address the issues on prediction techniques to overcome local optimum scenarios in search of the best solution. The fixed rates of genetic operators in genetic algorithm will be adjusted accordingly by adding an adaptive mechanism; thus allow earlier convergence.

The main objective of this research is to design a prediction model with earliest convergence for Medium Density Fiberboard physical properties. Initial correlation analysis provides information on the interdependency between potential independent predictors and the targets, in order to efficiently search optimum solutions. The contribution will be a cost reduction for the Medium Density Fiberboard testing procedures based on British Standard European Norm (BS EN) standard.

2. MATERIAL AND METHODS

In order to support the country in Medium Density Fiberboard (MDF) industry, the Malaysian Palm Oil Board (MPOB) needs to increase its efficiency in MDF processes, especially in terms of cost and time. Having abundant of fiber resources from the oil palm empty fruit bunch, as well as utilizing the oil palm biomass, Malaysia is able to provide enough supply to the industry. As the demand is vastly increasing, related institutions have to produce the research results fast to be able to compete with competitors from other countries. Data included in the model is MDF properties and its fiber characteristics.

Data Analysis

In any research-based pilot plants, numerous samples with various process combinations need to undergo testing procedures every day. Procedures need to conform to required standard before approval for industrial usage is granted.

Mechanical property testing procedures, namely Internal Bonding test and Bending Strength test, require less than two hours. Physical property testing procedures, on the other hand, need a longer time. Thickness Swelling test involves soaking sample in water for maximum of 24 hours, while Moisture Content test requires sample to be placed in an oven, with specific temperature setting, for maximum of 48 hours (Table 1.1).

Conforming to the British Standard European Norm (BSEN 1993), each MDF test is repeated at least three times to avoid any error or bias in results. Average value of the tests is taken as the final value. Decision on the quality can be made only after all tests are completed. For that reason, the time needed to complete procedures is 78 hours (Table 1.1); including the time for sampling, test pieces cutting as well as handling variety of sample characteristics to be tested.

The number that succeeds the property acronym indicates the hours allocated to run the procedures. Sample weighing is done before and after the test commences to monitor water absorption and moisture retention. Despite having lengthy hours of procedures, physical tests issues remain unhandled. The focus of this study is the Multilayer Perceptron NN model, which is reliable to learn from seven inputs fed to the network, namely IB, MOE, MOR, WA2, TS2 and fiber characteristics (density and percentage of oil palm fiber). The targets are TS24, WA24 and MC48 (Fig.1).

###TABLE 1

###Time Required for MDF Testing Procedures

###(Per Sample)

###Testing###Testing###Time

###Property

###Category###Procedure###(hour)*

###Internal

###IB###2

###Mechanical###Bonding

###Testing

###Bending###MOE,

###2

###Strength###MOR

###TS2, WA2###2

###Thickness

###Swelling

###Physical###TS24,

###Testing###24

###WA24

###Moisture

###MC48

###Content###48

###Total###78

Modeling Methodology

Supervised learning is a machine learning technique that learns from previous trends to produce classification or prediction. One of the approaches in supervised learning is Artificial Intelligence (AI) for automation problems. Among others, Neural Networks (NN) and Genetic Algorithm (GA) are AI techniques well accepted in supervised learning environments. The fitness value refers to Sum of Squared Error.

Neural Network imitates human brain to learn and make decisions. The most crucial parts of NN are the weight and bias values. A reliable prediction is how well a network can adjust its weights and bias to manipulate input to produce target values. It begins with optimizing the initial values of synaptic weights and bias (Chang et al.,2012) and later updating the values of weights and biases (Singh, 2012) which needs an excellent optimizer as GA.

The adaptive technique defers according to the fitness of the data adaptive technique on probabilities of crossover and mutation is to obtain two advantages; maintaining diversity in population and sustaining convergence capacity in GA [6, 7, 8].

GA is responsible to optimize NN weights in search of minimum prediction error [9]. There is a need to learn the population performance of each generation in order to guide in assigning suitable probability rates for crossover and mutation operators. This is possible by scanning through the fitness mean and median in the current population [10].

Adaptive mechanism is an approach where the parameters change accordingly due to the fitness of current population. At the beginning of the GA cycle, an initial population is created through a random process. The population consists of individuals representing the candidate solutions. This population is evaluated for fitness and the GA cycle begins to cross and mutate. A new generation of chromosomes is produced from the parent chromosomes through the actions of the reproduction, crossover and mutation operators. These operators are specially tailored to enable them to adapt with the current performance of the GA. Memberships to the mating pool are constantly re-evaluated by comparing the fitness of the chromosomes from the mating and offspring pool (Fig.2).

The first rule is to check whether the best fitness is above 0.95 (below 5% error), by checking the fitness pool array. Having the best fitness above 0.95 will indicate that the current population contains good chromosomes. If yes, crossover probability rate is assigned with 0.01 (eq.1). Situation 1 refers to a situation where the crossover rate is increased and the mutation rate is retained. Otherwise, mutation probability rate is assigned with 0.001 (eq. 2), indicating the need to increase the mutation rate when having a weak population. This is regarded as Situation 2 of the first rule. In both situations, learning rate formula is adapted to allow polynomial (curve) distribution of update values in crossover probability, Pc and mutation probability, Pm, and at the same time avoiding a linear increase.

updatePc = 0.01 * (1/ (epoch+1)) (1)

updatePm = 0.001 * (1/ (epoch+1)) (2)

The second rule is to use fitness mean and median. Since the fitness is already sorted in descending order, the upper half reflects better fitness as compared to the lower boundary. If fitness mean is greater or equal to its median, it portrays that fitness distribution contains many good populations. Thus, crossover probability, Pc is increased using equation eq.3 and mutation probability, Pm is updated using equation eq.4. Otherwise, if the mean is less than the median, it shows that the fitness distribution contains less number of good populations for on average the fitness lies in the second half of the fitness pool. Therefore, this situation requires the population to be mutated to find better chromosomes. The suitable formula will be to increase mutation probability, Pm (eq.5) and reduce crossover probability, Pc (eq.6).

updatePc = updatePc + 0.01 * (1/ (epoch+1)) (3)

updatePm = updatePm + 0.001 * (1/ (epoch+1)) (4)

UpdatePc = updatePc - 0.01 * (1/ (epoch+1)) (5)

updatePm = updatePm - 0.01 * (1/ (epoch+1)) (6)

Statistics of mean and median provide the general knowledge of data distribution. Knowing that the mean value lies exactly at the median value, or at least reasonably near to each other shows the fitness distribution is normal. When fitness distribution has a mean value in the upper part of median, it is said that the population contains excellent fitness, thus indicating low network error in the GA-NN processes. Parking the adaptive mechanism for each training sample will refine the overall RMSE of each epoch. This means, the model will obtain a good low RMSE even at the very beginning of epoch cycle.

3. RESULTS AND DISCUSSION

The performance achievement was due to the adaptive mechanism; which monitors the best population and changes Pc and Pm rates accordingly based on population situation either having weak or strong chromosomes in each generation.

An increase in crossover rate was to retain them for the use in the next generation. Otherwise, the mutation rate was increased to allow chromosomes to change towards a better population. This action is to reduce the chances of reproducing with non-excellent parents. The reason is to avoid selecting parents from a pool of bad chromosomes.

GA operators adapt the probability rate of crossover and mutation according to model performance of that iteration to optimize weights and biases. This allows a better global optimum. Fig.3 shows crossover probability and Fig.4 shows mutation probability obtained through the adaptive mechanism throughout the epochs.

Crossover probability rates stop changing as early as after the third epoch, while the mutation probability rates reached its optimum rate after epoch 40.

The research uses normalized data for the analysis. On the aspect of the pilot plant, the testing time can be reduced from 48 hours to only two hours, besides reducing the MDF testing cost significantly.

4. CONCLUSION

This research presents a hybrid Genetic Algorithm Neural Network (GA-NN) model to replace the physical tests procedures of Medium Density Fiberboard (MDF). Data included in the model is MDF properties and its fiber characteristics. The emphasis is on applying an adaptive mechanism on GA to enhance model performance.

In order to avoid result from local optimum scenario, GA optimizes synaptic weights of the network towards reducing prediction error. The research used a fixed probability rates for crossover and mutation for hybrid GA-NN model.

The hybrid prediction model is improved further using an adaptive mechanism on GA to determine the most suitable probability of crossover and mutation. The adaptive mechanism in GA helped the GA-NN model to learn the current generation and suggest on suitable rates for crossover and mutation. Applying the adaptive mechanism at each generation has allowed earlier convergence in the hybrid performance. Adaptive mechanism in GA helps increase capability to converge sooner than the ordinary GA.

The reliable model is able to simulate the testing procedure and therefore able to reduce the MDF testing time required as well as to reduce the cost. There is no simulation techniques used by the pilot plant, thus far; therefore, improvement on the time and cost for physical testing. Moisture content testing time was reduced from 48 hours to no experimental task needed. The reduction time for Thickness swelling testing time was 22 hours. With the simulation model, the new total time needed is only four hours; a reduction of 70 hours per sample, representing reduction time of almost three days.

5. ACKNOWLEDGEMENTS

Authors would like to thank Ropandi Mamat and Suchirah Ratmin of the Malaysian Palm Oil Board (MPOB) for supplying the data used in this research.

6. REFERENCES

[1] Khamis, A., Ismail, Z., Haron, K. and Mohamed, A. T. Neural Network Model for Oil Palm Yield Modeling, Journal of Applied Sciences, 6 (2), 391-399, 2006.

[2] Fernandez, F.G., Esteban, L.G., Palacious, P.D., Navarro, N. and Conde, M. Prediction of Std Particleboard Mech properties utilizing an ANN and subsequent comparison with a multivariate regression model, Investigacion Agraria: Sistemas y Recursos Forestales, 17 (2), 178-187, 2008.

[3] Esteban, L.G., Fernandez, F.G., Palacios, P. and Rodrigo, B.G. Use of ANN as a predictive method to determine moisture resistance of particle and fiber boards under cyclic testing conditions, (UNE-EN 321) Wood and Fiber Science, 42 (3), 1-11, 2010.

[4] Mishra, K. K., Singh, B. K., Punhani, A. and Singh, U. Combining NN and GA to Predict and To Maximize Lemon Grass Oil Production, IEEE International Joint Conference on Computational Sciences and Optimization, 2009.

[5] Karegowda, A.G., Manjunath, A.S. and Jayaram, M.A. Application of Genetic Algorithm Neural Network Connection Weights for Medical Diagnosis of PIMA Indians Diabetes. International Journal of Soft Computing (IJSC) 2 (2), 15-23, 2011.

[6] Davis, L. Adapting operator probabilities in genetic algorithms, J. David Schaffer ed., Proceedings of the Third International Conference on Genetic Algorithms, San Mateo, CA: Morgan Kaufman Publishers, Inc., 61-69, 1989.

[7] Back, T. Self Adaption in Genetic Algorithms, Towards a Practice of Autonomous Systems, MIT Press. 263-271, 1992.

[8] De Jong, K.A. Analysis of the behavior of a class of genetic adaptive systems, Ph.D. Dissertation, Dept of Computer and Comm Sciences, Univ of Michigan, Ann Arbor, MI, 1975.

[9] Abu Bakar, N. and Sh Ismail, F. GA++ Enhancing Information Searching in Genetic Algorithm through an Adaptive Mechanism. Accepted for JAIST.

[10] Sh Ismail, F. and Abu Bakar, F. Adaptive GA-NN for MDF Prediction Model. Accepted for Advanced Materials Research Vol. 980, 2013, 2014.

Printer friendly Cite/link Email Feedback | |

Author: | Ismail, F. Sh; Bakar, N. Abu |
---|---|

Publication: | Science International |

Article Type: | Report |

Date: | Jun 30, 2017 |

Words: | 2636 |

Previous Article: | MODELING WEIRS DISCHARGE COEFFICIENT USING EVOLUTIONARY ALGORITHM. |

Next Article: | EFFECT OF SIZE AND GEOMETRY OF POLYCRYSTALLINE DIAMOND CUTTER (PCD) ON WEAR RATE IN MULTILAYER FORMATION. |

Topics: |