# An alternative approach to measure quantity and smoothness of the human limb motions.

1. INTRODUCTIONThe present paper is devoted to the problem of measuring and modelling changes in the human motor functions, during learning new motor activities. Nowadays majority of the results achieved in human motion analysis and gesture recognition is focused on extracting and analysing certain features of the motion [1]. While feature extraction and pattern recognition techniques give precise and robust results, they do not always answer all the needs of academic research. To the best knowledge of the authors, there is no commonly used and widely accepted technique to measure the state of the human movement patterns. This leads to the main goal of the present contribution, namely, to measure numerically the state of the human motor functions. Ability to measure state of the human motor functions would allow to apply a rich variety of mathematical methods to model the learning process which in turn will provide novel opportunities for the computer aided training. For this purpose the notion "motion mass" is introduced. It describes the amount and smoothness of the human limb movements during a certain period of time.

The paper is organized as follows. The process of new motor activity planning is described in Section 2. The notion "motion mass" is formally introduced in Section 3. Application of the motion mass parameters to describe and model the changes in motor functions are explained in Section 4. Advantages and disadvantages of the proposed approach are discussed in Section 5. Conclusions and future plans are outlined in the last section.

2. LEARNING OF A NEW MOTOR ACTIVITY

In order to avoid any confusion caused by the terminology, within the frameworks of the present contribution we use the following terms: movement is any internally generated change in the physical position of the skeletal body parts in relation to one another; motion is a set of movements and action is a purposeful set of movements or motions. The process of learning a new motor activity or optimizing the known one may be seen as a sequence of trials, where each trial is an action aiming to achieve a certain goal. Each action may be a solitary attempt to achieve something, or may have a more complicated structure; for example, sportive training sessions consist of different exercises. From the physiological point of view each action may be seen as a behavioural act. According to Russian neurophysiologist Piotr Anokhin's well-grounded theory [2,3] (see also [4] for a review) every behavioural act of living organisms is based on a 'functional system', which universal structure includes the following components: afferent synthesis (planning of the action), decision making (which includes the formation of two complementary dynamic structures, action program and acceptor of the result), back-afferentation and acceptance of the results (analysis of the achieved results). Comparison of the desired result to the achieved one allows making a decision about the necessity to continue learning and provides feedback for the afferent synthesis of the next step. On the stage of afferent synthesis, the learning individual decides what to do, how to do, and when to do [2-4]. Answering these questions is equivalent to making a choice from all possible options. Creation and adjusting of motor programs is foremost performed by the individual; occasionally supervisors may support this process by proposing modifications of the performed actions. Results of the action may be either changes of the environment or changes in the organism itself. While changes in the environment are usually easy to observe and describe, changes in the organism itself are not so obvious, especially in the unsupervised case. Also description of such changes may be highly subjective. Learning of a motor activity is either an aim by itself (e.g., in case of learning to dance) or is aimed at achieving certain changes in the environment (e.g., learning to use tools). In both cases in the process of learning the movements that comprise the action change; usually the trajectories of motions become smoother and more precise. The working hypothesis of the authors is that the state of the motor functions may be related to the progress of achieving the goal of learning. This leads to the idea to measure the state or condition of the motor functions numerically. On the one hand, such measure will provide objective measurement of the changes in the functioning of the organism (changes of motor functions in our case), which may be used in back-afferentation, and, on the other hand, will allow mapping the processes of motor functions learning for further academic research. Another important point to consider is that final configuration of motor functions is unknown in the beginning of the learning process. Therefore there is no standard to compare and measure the difference between the current state of the motor functions and the goal. Even in such well-studied activities like playing golf or tennis, one may obtain very precise instructions about positions and the motions of the limbs, but final state of the motor functions still will be unique for each individual. In this context, observing numeric changes in motor functions may provide one with the ability to detect changes or lack of them in the course of training or practice of moment patterns.

Let us consider the process of learning the ball throwing into the basket. While this activity may seem quite simple on the first view, the size and the distance to the basket may be chosen in such a way that getting the ball into the basket may require certain amount of practice. Such practice is the optimization of the motor activity. In our study, it was performed in the form of exercises whereas each exercise consisted of ten trials (ball throwing). Preparation for the ball throwing in this case is the afferent synthesis, decision to throw the ball is made once the individual is ready. Immediately after the throwing, information about the result of the action becomes available for the back-afferentation and acceptance of the results. In this experiment, results of the action are expressed numerically, number of the balls got to the basket. This allows using formal methods to check if the progress in achieving the goal of the training is related to the state of the motor functions. The only missing part now is the numerical description of the state of motor functions.

Anyone who has experienced learning new motor activities remembers that in the beginning the movements of the limbs are quite awkward and not very smooth, but during the practice they become smooth and adroit. Intuitively, awkwardness of the motions corresponds to the unnecessary or insufficient limb movements. This leads to two main properties of the motion the desired measure should describe, namely the amount and smoothness of the performed movements. During the learning process the amount of unnecessary movements decreases, which leads to the convergence to some optimal quantity of movements. Amount of the movements may be described by the trajectory lengths of the joints participating in the motion. At the same time, as learning progresses, the action program becomes more precise. This decreases the necessity to adjust movements during the motion which in turn leads to less accelerations to be performed. Therefore, the amount of the accelerations to be performed during the motion may be used to describe smoothness of the motion.

3. "MASS" AND SMOOTHNESS OF THE MOTION

Let us now formalize the idea of the amount and smoothness of the movement. Define motion as the most primitive (indivisible) movement of the limb or group of the joints. Let J = {[j.sub.1], [j.sub.2], ..., [j.sub.n]} be the set of the joints of interest. For each motion one may precisely measure the beginning and ending times, and associate positions of each joint to the beginning and ending time moments. Let t denote the length of the motion in time. Define combined Euclidean distance of the set J as the sum of the Euclidean distances of each joint associated with the motion of interest:

[E.sub.J] = [[n.summation over (i=1)] [E.sub.[j.sub.i]], (1)

where [E.sub.[j.sub.i]] denotes the Euclidean distance between the starting and ending positions of the joint [j.sub,i]. By analogy, denote by [T.sub.[j.sub.i]] the length of the trajectory of the joint [j.sub.i], observed during the motion. Define trajectory mass as the sum of the trajectory lengths computed of each joint of the set J:

[T.sub.J] = [[n.summation over (i=1)] [T.sub.[j.sub.i]]. (2)

As a final step, let us define the acceleration mass in the following way. Associate to each joint the sequence of accelerations computed for each pair of the consequent time moments (while the motion took place). Find the sum of the absolute values of each element of the sequence and denote it as [A.sub.[j.sub.i]]. Define the acceleration mass of the set J as

[A.sub.J] = [[n.summation over (i=1)] [A.sub.[j.sub.i]]. (3)

Trajectory mass describes total amount of performed movements and acceleration mass describes smoothness. Combined with the length of the motion in time, trajectory mass and acceleration mass are the three numerical parameters allowing to compare motions on different stages of the learning process. Main drawback of the trajectory mass and acceleration mass is that they depend on the physiological constitution of the particular individual. Those with the longer limbs would have larger values of the trajectory and acceleration mass. In order to provide possibility to compare motion masses of different individuals, two following parameters are proposed: the ratio of the combined Euclidean distance and the trajectory mass

[R.sub.d] = [E.sub.J] / [T.sub.J], (4)

and the ratio between the combined Euclidean distance and the acceleration mass

[R.sub.a] = [E.sub.J] / [A.sub.J]. (5)

While both parameters [R.sub.d] and [R.sub.a] are necessary to compare motions performed by different individuals, it is enough to know the values of the trajectory mass, acceleration mass and combined Euclidean distance to compute them. Let us now define motion mass as the set of the following four parameters: trajectory mass, acceleration mass, combined Euclidean distance and the motion length in time. Denote the motion mass associated with the joints set J as [M.sub.J]:

[M.sub.J] = {[T.sub.J], [A.sub.J], [E.sub.J] ,t}. (6)

The elements of the motion mass describe the amount and smoothness of the movements associated with certain motion.

4. PRACTICAL APPLICATION

In order to demonstrate abilities of the motion mass to describe learning of new motor activities let us return to the example of ball throwing into the basket. There are two important components to consider: the data acquisition process and the analysis of the motion mass parameters.

4.1. Data acquisition

Motion capture was performed by the Microsoft[TM] Kinect[TM] sensor, connected to a PC. While initially Kinect was designed as a motion capture device for the gaming console, during recent years it has quickly gained popularity not only in the area of human-machine interaction [5] but has also found its way to such a delicate area as medicine. In spite of its simplicity it provides quality, which under certain conditions is in pair with more advanced motion capture systems [6], allowing to use the sensor in physical rehabilitation [7] and computer-aided surgery [8].

Control of the Kinect sensor, data acquisition and storage were performed by means of a specially developed application. This application allows to read the data from the sensor, mark the beginning and the ending times for each motion of interest and store the data for further analysis. Kinect returns three-dimensional coordinates of 19 body joints and the head, with a sampling time of 1/30 s. Within the framework of the present research, MATLAB[TM] was used to analyse the data and perform all necessary computations. Kinect generated schematic diagram of the human skeleton is presented in Fig. 1, where the first two diagrams depict front and side views of the initial posture and the second two--the posture corresponding to the moment of ball throwing.

4.2. Analysis of the experimental data

Totally 20 sessions were performed, whereas each session consisted of 10 trials. The set of joints J included: left shoulder, shoulder centre, right shoulder, right elbow, right wrist and right hand. Selection of the joints was based on the observations, which indicated which joints moved during ball throwing. For each motion of throwing motion mass parameters were computed. Let us first compose the sequence of successful trials only. Changes in the trajectory mass, acceleration mass and time for the successful trials are depicted in Fig. 2.

In Fig. 2, axis x correspond to the overall number of successful attempts, and plots of the trajectory mass, acceleration mass, combined Euclidean distance and length of the motion in time are shown. Unlike to the same motion mass parameters computed for the failed trials (Fig. 3), parameters computed for the successful trials tend to change more during the learning process.

In order to confirm visual observations, statistical hypotheses will be used. Let A be a random sample of 30 successful trials from the beginning of the training and B be the same size random sample of successful trials form the last part of the training. To each sample one may associate five vectors, corresponding to the motion mass parameters, the vector of trajectory masses, the vector of acceleration masses, etc. Now for each pair of vectors, state the pair of hypotheses ([H.sub.0], [H.sub.a]) such that the null hypothesis ([H.sub.0]) states that data in the vectors, corresponding to the samples A and B are independent random samples from normal distributions with equal means and equal but unknown variances, against the alternative ([H.sub.a]) that the means are not equal. Test results for the level of significance [alpha] = 0.05 are presented in Table 1.

In Table 1, value 1 indicates rejection of the null hypothesis at given significance level, and 0 indicates failure to reject the null hypothesis. Notation 0/1 indicates that for some individuals mean values were different and for some not. Test results clearly demonstrate that the motion mass parameters for the successful trials differ between the beginning and the ending parts of the learning process. The same parameters, computed for the unsuccessful trials, do not differ significantly during the learning (see the last column of Table 1). The p-and t-values, corresponding to the test results for one particular individual, are presented in Tables 2 and 3, respectively.

The proposed technique was validated on the data, describing learning of the same motor activity, of two more individuals. Obtained results do not differ much from those, presented in Tables 2 and 3, and are therefore omitted here.

5. DISCUSSION

To a certain extent the proposed approach may seem similar to the notion "motion region" proposed to estimate "quantity of motion" in [9]. While certain similarity undoubtedly exists, the method proposed in the present contribution allows greater freedom of customization, which makes it better suited for the studies of the motor activity learning.

Up to now the question of motion mass units was left undiscussed. While the trajectory mass and the combined Euclidean distance, if necessary, may be expressed in any standard units of international or imperial measurement systems, this is not the case for the acceleration mass. At the present time motion mass parameters are meant to be compared only to their own values at different time moments or between different individuals. Thus there is no necessity to relate those parameters to any other measures. Let us now consider the values of the motion mass parameters as unitless abstract measures (of course, except the motion length in time, which inherits its normal measuring units). Most probably, in the course of future research, relation of the motion mass to other measures will be established which will allow to establish a justified system of units.

Another important point to discuss here is the choice of the elements of the motion mass. While the original idea in [10] was to define motion mass as a set of five parameters: trajectory mass, acceleration mass, ratio [R.sub.d], ratio [R.sub.a] and the motion length in time, our study has demonstrated that the definition, proposed in this paper, is less complex and is better suited for the result interpretation. Also motion mass, defined by Eq. (6), contains all the parameters necessary to compute the ratios [R.sub.d] and [R.sub.a].

As a final note let us briefly discuss the possibility of the proposed approach to be applied in medicine. In those cases where treatment relays on the physical exercises (for example motor functions rehabilitation or treatment of the Parkinson's disease), ability to measure state of the motor functions may provide objective information about progress of the rehabilitation or effectiveness of the treatment.

6. CONCLUSIONS

The notion motion mass was introduced in this paper to describe the state of the human motor functions in the context of learning a new motor activity. The main distinctive feature of the proposed approach is that certain numeric measures are associated with the entire motion, performed by the limb or group of the limbs, and not only with a particular property of the motion. Such numeric measures allow to apply mathematical methods to determine if the state of the human movement patterns is changing from one experiment to another, determine the values of absolute and relative changes and prove convergence of the process. For example the well-known Timed-Up-and-Go test [11,12], in which time needed to perform a certain set of actions is measured, does not describe quantity and the smoothness of the movements. The method we propose allows to record considerably more information about motor functions. For instance, different disorders of motor functions, such as standing up, walking, turning, etc., may look similar if the measures of them are time to perform an act and/or distances passed. Yet the same slow performance of a set of actions may result from opposite kinds of motor disorders. In one case the movements might be small and slow. In another case, on the contrary, due to involuntary movements, the movements might be large and fast but inefficient. In the first situation the motion mass, we measure would be small and in the latter case big even when the time used to perform the task or distance passed had been the same. Learning process of ball throwing into the basket was used to demonstrate the ability of the proposed technique. Future research will be concentrated on practical applications of the proposed technique in different areas of psychology and medicine.

doi:10.3176/eng.2013.4.05

Received 28 October 2013, in revised form 6 December 2013

ACKNOWLEDGEMENTS

This work was supported by the European Union through the European Regional Development Fund and by the target funding project SF0140018s08 of the Estonian Ministry of Education and Research. S. Nomm was additionally supported by the ESF grant No. 8365. A. Toomela was also supported by the ETAG Institutional Research Funding project IUT 3-3.

REFERENCES

[1.] Holte, M. B., Cuong Tran, Trivedi, M. M. and Moeslund, T. B. Human pose estimation and activity recognition from multi-view videos: Comparative explorations of recent developments. IEEE J. Selected Topics in Signal Process., 2012, 6, 538-552.

[2.] Anokhin, P. K. Essays on the Physiology of Functional Systems. Medicina, Moscow, 1975 (in Russian).

[3.] Anokhin, P. K. Selected Works. Philosophical Aspects of the Theory of Functional Systems (Konstantinov, F. V., Lomov, B. F. and Schvyrkov, V. B., eds). Nauka, Moscow, 1978 (in Russian).

[4.] Toomela, A. Biological roots of foresight and mental time travel. Integr. Psych. Behav. Sci., 2010, 44, 97-125.

[5.] Zhao, X., Li, X., Pang, C. and Wang, S. Human action recognition based on semi supervised discriminant analysis with global constraint. Neurocomputing, 2013,105, 45-50.

[6.] Fernandez-Baena, A., Susin, A. and Lligadas, X. Biomechanical validation of upper-body and lower-body joint movements of kinect motion capture data for rehabilitation treatments. In 4th International Conference on Intelligent Networking and Collaborative Systems (INCoS). Bucharest, 2012, 656-661.

[7.] Yao-Jen Chang, Shu-Fang Chen and Jun-Da Huang. A kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities, 2011, 32, 2566-2570.

[8.] Yoshimitsu, K., Muragaki, Y., Maruyama, T., Yamato, M. and Iseki, H. Development and initial clinical testing of 'OPECT', an innovative device for fully intangible control of the intraoperative image displaying monitor by the surgeon. Neurosurgery, 2014. Forthcoming.

[9.] Takai, M. Extracting method of characteristic posture from human behavior for surveillance camera. In Proc. International Joint Conference on Instrumentation, Control and Information Technology. Fukuoka, 2009, 159-164.

[10.] Nomm, S., Toomela, A. and Borushko, J. Alternative approach to model changes of human motor functions. In European Modelling Symposium EMS2013. Manchester, United Kingdom, 2013.

[11.] Wall, J. C., Bell, C., Campbell, S. and Davis, J. The Timed Get-up-and-Go test revisited: measurement of the component tasks. J. Rehabil. Res. Dev., 2000, 37(1), 109-114.

[12.] Herman, T., Weiss, A., Brozgol, M., Giladi, N. and Hausdorff, J. M. Identifying axial and cognitive correlates in patients with Parkinson's disease motor subtype using the instrumented Timed Up and Go. Exp. Brain Res., 2013, 1-9.

Alternatiivne meetod jasemete liigutuste hulga ja sujuvuse mootmiseks

Sven Nomm ja Aaro Toomela

On kasitletud inimese motoorikas oppimise kaigus toimuvate muudatuste mootmist ja kirjeldamist. Tanapaeval pohineb valdav enamik liigutuste analuusi ja zestide tuvastamise meetodeid liigutustunnuste eristamisel, liigutusmustrite tuvastamisel ning klasterdamisel. On esitatud uudne liigutuste kirjeldamise meetod, mis erinevalt senikasutatutest kirjeldab jasemete liigutuste hulka ja sujuvust. Arvuliselt moodetava naitajana on kasutatud liigutuste massi (motion mass), mis kirjeldab kindla liigutuste tsukli kaigus tehtud liigutuste hulka eukleidilises ruumis. Esitatud meetodi voimalusi on demonstreeritud sihtmargi pihta palli viskamise oppimise naitel. Nii teoreetilised kaalutlused kui ka empiirilised andmed osutavad, et uudne meetod voib olla laialt rakendatav erinevate liigutuste oppimisel, sealhulgas sportlaste treenimisel ja ajukahjustuse labi teinud inimeste motoorsete voimete rehabilitatsioonis.

Sven Nomm (a) and Aaro Toomela (b)

(a) Institute of Cybernetics at Tallinn University of Technology, Akadeemia tee 21, 12618 Tallinn, Estonia; sven@cc.ioc.ee

(b) Institute of Psychology, Tallinn University, Narva mnt. 25, 10120 Tallinn, Estonia; toomela@tlu.ee

Table 1. Differences of the motion mass parameters in the beginning and in the ending of the motor activity learning Parameter Successful Unsuccessful trials trials Trajectory mass [T.sub.J] 1 0 Acceleration mass [A.sub.J] 0/1 0 Combined Euclidean distance [E.sub.J] 1 0/1 Motion length in time t 1 0/1 Table 2. p-values, corresponding to the hypothesis testing results for one particular individual Parameter Successful Unsuccessful trials trials Trajectory mass [T.sub.J] < 0.0001 0.1342 Acceleration mass [A.sub.J] 0.0950 0.0776 Combined Euclidean distance [E.sub.J] 0.0172 0.5656 Motion length in time t < 0.0001 < 0.0001 Table 3. t-values, corresponding to the p-values reported in Table 2 Parameter Successful Unsuccessful trials trials Trajectory mass [T.sub.J] 4.1789 1.5189 Acceleration mass [A.sub.J] 1.6972 1.7966 Combined Euclidean distance [E.sub.J] -2.4539 0.5779 Motion length in time t 4.8292 4.5755

Printer friendly Cite/link Email Feedback | |

Author: | Nomm, Sven; Toomela, Aaro |
---|---|

Publication: | Estonian Journal of Engineering |

Article Type: | Report |

Date: | Dec 1, 2013 |

Words: | 3876 |

Previous Article: | Annual variation of air-water temperature difference at three Estonian coastal stations. |

Topics: |