Multidimensional scaling of schematically represented faces based on dissimilarity estimates and evoked potentials of differences amplitudes.
It is obvious that both facial identification and expression detection are determined by their configuration (Paramey, 1996; Bimler & Kirkland, 2001; Ekman & Friesan, 1978). Therefore, research into the role of the linear configuration that forms the pattern of the face in the perception of facial expression and its recognition is of prime interest. The principal characteristics of facial configuration are the outline of the mouth, and eye and brow form. Psychophysical investigations using photographs and schematically presented faces as stimuli have shown that facial expression can be defined as a multidimensional function of the curvature of the lips, angle of eyebrows, and other configuration characteristics (Ekman & Friesan, 1978; Izmailov, Korshunova, & Sokolov, 1999; Paramey, Izmailov, & Babina, 1992). The contour of the mouth, eyes, and brows may in turn possibly be considered as a combination of lines of varying orientation, the detection of which forms the basis for the recognition of facial emotion expression (Abelson & Sermat, 1962; Bimler & Kirkland, 2001; Izmailov et al., 1999; Izmailov, Korshunova, Sokolov, & Chudina Yu, 2004;). This reveals intimate interaction of the occipital cortex neurons aimed at the detection of configuration characteristics of the perceived image (Hubel & Wiesel, 1962; Shvelev, Kamenkovich, & Sharaev, 2000; Supin, 1981) and the temporal cortex neurons associated with facial detection (Perrett et al., 1982; Rolls, 1984, 1998).
The aim of the present study is to reveal the input of the neural networks of the occipital and temporal cortex in the perception of facial expression. Using the method of multidimensional scaling, a spatial model of differentiation of schematic faces is constructed according to the amplitudes of the evoked potentials of differences (EPD) recorded in the occipital and temporal cortex in response to abruptly changing stimuli, and is compared to an analogical space structure based on the results of subjective estimates of emotional differences between the presented faces.
Along these lines, the current study completes the cycle of our research into the perception of emotional content of schematically presented faces in the form of an oval with linearly portrayed brows, eyes, nose, and mouth (see Figure 1). The emotional expression of the face was determined by the curvature of the mouth, which altered from a zero level (horizontal) upwards and downwards in 14[degrees]-steps, and the brows whose angle ranged from the zero level upwards and downwards in 6[degrees]-steps. The aim of these studies was to construct a geometric model of visual differentiation of schematic faces with both EPD amplitudes registered in the cerebral cortex in response to instant substitution of stimuli, and with the results of dissimilarity judgments of the emotional expression of the same faces. In the first study (Izmailov et al., 1999), using the method of multidimensional scaling of the matrix of paired differences between 25 facial stimuli, a geometric model of the perception of schematic facial expression was constructed. In this model, the schematic faces were represented by points in a four-dimensional space so that the Euclidian distances between points were proportional to the perceived differences between the emotional expressions of the presented faces. The points-stimuli do not completely fill the four-dimensional space, but are located on the hypersphere surface. Three angles of the four-dimensional sphere are juxtaposed with the subjective characteristics of facial expressions called emotional quality (joy, sorrow, sadness etc.), emotional intensity, and emotional "saturation" in the studies (Boucsein, Schaefer, Sokolov, Schroder, & Fureby, 2001; Sokolov, 1992). The four Cartesian coordinates of this space are comprised of the activation of neural channels coding mouth and brow linear orientation. The analysis of the data from the evoked potentials of differences has allowed us to specify the model presented.
[FIGURE 1 OMITTED]
The following study (Izmailov, Korshunova, & Sokolov, 2001) demonstrated the measurement of differences between schematic faces based on the recording of evoked potentials in response to abrupt stimulus change. According to the specificity of the components of evoked potentials of differences (Izmailov, Isaichev, Korshunova, & Sokolov, 1998; Izmailov et al., 2004; 2001; Izmailov & Sokolov, 2004), four measures of inter-stimuli differences in each EPD were made--peak amplitudes of medium latency components N180 and P230, and inter-peak amplitudes P120-N180 and N180-P230 (see Figure 2). These amplitudes were compared both with dissimilarity judgments between the same faces and between the angles of the curvature of the mouth and brows of the schematic face.
The study showed that long-latency components of EPD do not contain information either of the configuration differences or the expression differences in the facial stimuli. The medium-latency components, on the other hand, showed high correspondence, and the highest correlation was found for the inter-peak amplitude in comparison to the peak amplitude of N180. Both interpeak amplitudes P120-N180 and N180-P230 differ as a function of electrode site. The eary component P120-N180 has the highest correlation at sties T5 and T6, and the late component N180-P230 at sites O1 and P3. The EPD amplitudes also correlate with dissimilarity judgments as the physical angle differences in mouth curvature and brows decrease (see Table 1).
Results of previous studies led to considering EPD amplitudes to be adequate measures of inter-stimulus differences, in order to build a geometric model for the neural network differentiation of schematic faces in the occipital and temporal cortex, and to compare it with analogical models based on dissimilarity judgments, the so-called subjective space (Izmailov et al., 1999; Shepard, 1964, 1981; Osgood, 1966). The subjective expression differentiation space is an integrative model in the form of a sphere within a four-dimensional Euclidean space in which four Cartesian point coordinates (representing various faces) reflect the neural network for the detection of separate configuration signs of the facial image, whereas other spherical coordinates of the same point reflect the emotional characteristics of the face (Izmailov et al., 1999). These two groups of variables (configuration and expression) are linked together by the dissimilarity estimates and the amplitudes of the EPD obtained for the same stimuli. Within the spherical model, the links are represented by the inter-point distance, which is determined by the direct conformity between perceived differences of facial stimuli and the amplitudes of the EPD, recorded in response to abrupt stimulus change (Izmailov et al., 1998, 2004, 2001). Thus, the spherical model in the present study is the basis for the analysis of the results of EPD recording in the occipital and temporal cortex in order to reveal the participation of these areas in the process of human facial perception.
[FIGURE 2 OMITTED]
Three adult males between 20-30 years of age took part in the study.
The experiment was carried out on a computer consisting of a 16-channel computerized encephalograph, connected by a 16-channel analog-digital converter (ADC) to a personal computer. Stimuli were presented on a separate high-quality computer screen. Both computers involved in the process were synchronized with the "Conan" system which permitted experimentation and primary processing (filtration, summing, and rejection of artifacts) of the recorded potentials.
Nine schematic faces were used as stimuli (see Figure 1), chosen from a bank of 25 stimuli (Bongard, 1955), so that that they expressed the main facial configuration characteristics (curvature of mouth, slant of brows) and the main emotions (joy, surprise, sorrow, anger, and calm) represented on the schematic faces (Osgood, 1966; Paramey, 1996; Paramey et al., 1992; Schlosberg, 1941). In the study of Izmailov et al. (1999), the curvature of the mouth was determined by the angles of the corners upwards or downwards at angles of 28 and 14[degrees]. In relation to the position of mouth angles, each face was characterized by 5 ranges of mouth curvature. Brow angle had five ranges: -12, -6, 0, 6, and 12[degrees]. In the present study, the stimuli had only extreme ranges of mouth curvature (-28, 0, and 28[degrees]) and brow angle (-12, 0, and 12[degrees]). Stimuli were presented at the center of the screen in white lines on a black background. The participants observed the stimuli with both eyes. The screen was placed at a distance of 1m from the participant's eyes. Stimulus size was 7 x 7cm.
Evoked potential recording. Data for the construction of the subjective facial difference space were based on the emotional dissimilarity judgments between presented stimuli, according to the direct estimation method described in our previous study (Izmailov et al., 1999). For recording evoked potentials (EPs), the same conditions of stimulus observation were maintained (Izmailov et al., 2001).
Each stimulus pair (the first stimulus--the referent--, the second--the test stimulus) was presented in clusters. In each cluster, test and referent stimuli were alternately presented 60 times (r-t-r-t-r-t, etc.). The duration of the test and referent stimuli varied within the range of 800-1200 ms in order to avoid rhythmic influence.
EPs were recorded in the occipital (O1, O2), temporal (T5, T6), and parieto-occipital (P3, P4) regions of the left and right hemispheres with earlobe referent electrodes. In order to reject artifacts caused by eye-movements, potentials recorded at sites Fp1 and Fp2 were analyzed with amplitude rejection criteria 50 [micro]V. All denotations were made according to the international 10/20 system. Recording duration for each stimulus change was 400 ms. Digitalized recording was carried out using an ADC with a 5 ms increment (200 Hz). Prior to each change, background EEG was recorded for 90 ms, after the changeover, recording lasted 310 ms. Recording was carried out within the frequency range of 0.3-30 Hz.
Experiments with EPs are different from traditional techniques in two basic particularities. Firstly, instead of presentation of separate light flashes, instant (lasting no more than 1 ms) substitution of one face over the other was used. EP was recorded on this change (Bongard, 1955; Estevez & Spekreijse, 1982; Zimachev, Shehter, Sokolov, & Izmailov, 1986). Secondly, the difference between changed stimulus pairs was monotonically increased along both legs from zero point, at which point, the changed stimuli were identical. Because the recorded EP is not in response to the stimulus itself, but to the difference between the stimuli, we refer to it as the EPD. For the specification of components, series of such functionally related EPs to stimulus substitutions were used rather than separate EPs (Izmailov et al., 1998, 2001, 2004; Izmailov & Sokolov, 2004; Zimachev et al., 1986).
If the stimuli present no difference in the specified cortex region where the EP is being recorded (although, physically, stimuli may be different), the EPD will not differ from background EEG. However, if the visual system differentiates stimuli, at the moment of change, there will be an alteration of activity, as shown by the data (Izmailov et al., 1998, 2001, 2004; Izmailov & Sokolov, 2004; Zimachev et al., 1986), and the greater the difference between stimuli, the greater the visual EPD. Accordingly, a matrix of inter-stimulus differences for the paired changing stimuli can be obtained and, using multidimensional scaling with the recorded data, one can construct a geometric model analogical to the model based on dissimilarity judgments. Thus, by such analysis of the EPs, one can extract reliable information about stimulus differentiation and the cerebral mechanisms leading to the specification and differentiation of stimuli.
The results of psychophysical experimentation (dissimilarity judgments) used for the construction of the geometric model of emotional expression are part of the differentiation matrix from our previous study (Izmailov et al., 1999). EPD results after the rejection of artifacts were averaged for all recordings of each cluster of paired stimuli. According to data regarding the specificity of the visual EPD components shown in other studies (Izmailov et al., 2001, 2004), three measures of inter-stimulus differences were studied: amplitudes of the N180 peak and inter-peak amplitudes of P120-N180 and N180-P230 (see Figure 2).
The Geometric Model of Schematic Face Differentiation
Schematic Face-Differentiation Space
All difference matrices were analyzed using the metric algorithm of multidimensional scaling (Shepard, 1964, 1981). Nine-point coordinates in a nine-dimensional Euclidean space were obtained following analysis of each matrix. The spatial stimulus-differentiation model was defined by two parameters: minimal dimensionality of space and spherical points-stimuli configuration. Minimum dimension was rated using various formal criteria based on the ratings of correspondence between inter-points distances in the resulting space and initial measures of inter-stimuli differences. In the present study, indexes such as Kruskal's "stress" and Pearson's correlation coefficient (Izmailov et al., 1999, 2004; Izmailov & Sokolov, 2004; Shepard, 1981) were used for dimension rating. In this case, according to the spherical model of emotional expression proposed in previous studies (Izmailov et al., 1999; Boucsein et al., 2001; Sokolov, 1992), four first dimensions were studied in each of the analyzed spaces. Each four-dimensional space was checked for conformation with the spherical model as follows.
The optimal location of the frame of reference (geometric center of the points-stimuli configuration) was determined for the obtained point configuration so as to ensure equality of all radius vectors. Due to random errors, the length of the radius-vector has a specific range of values determined by the dispersion of all the radius-vectors. Variation coefficient was used for the numerical rating, it was calculated as the ratio of one standard deviation to the mean radius-vector length. Thus, the lower the variation coefficient, the greater the conformation to the obtained spherical model (Izmailov et al., 2004; Izmailov & Sokolov, 2004; Shepard, 1964, 1981; Sokolov & Izmailov, 1983). Table 2 shows the results of this analysis.
The results show that, formally, the four-dimensional Euclidean space accurately represents all the original EPD matrices. No stress value exceeded 5% error. According to the criteria of the spherical model, two groups of data reveal the magnitude of the variation coefficient, comparable to the magnitude derived from subjective estimate data (5%). They are: the inter-peak amplitude P120-N180, recorded at the occipital (O1) and temporal (T5) electrode sites of the left hemisphere, and the late latency amplitudes N180-P230, recorded at the occipital (O2) and temporal (T6) electrode sites of the right hemisphere. The remaining eight EPD amplitude matrices were characterized by variation coefficients exceeding 10%. Nevertheless, the data provided by these results formally satisfy spherical criteria (Izmailov et al., 1999, 2004; Izmailov & Sokolov, 2004; Sokolov & Izmailov, 1983).
However, formal criteria merely define required solution conditions and are insufficient. Because the initial matrixes of differences consist of real numbers, there will always be a spatial solution for them. Thus, their real goodness of fit is determined by criteria of psychophysical and neurophysiological interpretation in terms of the function of the visual system, as well as established data in this field of research (Izmailov & Sokolov, 2004; Shepard, 1964, 1981; Sokolov & Izmailov, 1983).
Space Rotation and Reference Frame Interpretation
Following location of the point configuration center in the four-dimensional space, the optimal direction of the frame of reference was determined. The original direction of the axes obtained by the multidimensional scaling method is arbitrary, as the solution was based exclusively on inter-point distances which do not depend on the direction of the axes. Here, it also is necessary to use criteria of axis content interpretation. In the present case, characteristics of the spatial axes revealed in previous studies were used (Izmailov et al., 1999; Paramey et al., 1992). The first Cartesian axis of the schematic face-differentiation space is interpreted in these studies as the mechanism to detect mouth curvature. Thus, Axis X1 is oriented so that the projection of the points-stimuli on this axis (values of X1 coordinates of each point) are in accordance with the sign and angle value of the mouth on the schematic face. The second Cartesian axis of the space is interpreted as the mechanism to detect the eyebrow slant (Abelson, 1962; Bimler & Kirkland, 2001). Therefore Axis X2 is oriented to correspond with the sign and angle values of the schematic face brows. Only if both conditions are satisfied can one conclude that the plane of the first two axes corresponds to the spherical model of emotional expressions (Izmailov et al., 1999).
The location of the points-stimuli on Plane X1X2 of the four-dimensional facial- differentiation space, based on the results of dissimilarity judgments, is presented in Figure 3a. The number of points in this figure correspond to the schematic face number shown in Figure 1. In Figure 3a, one can see that points 1, 4, and 7, which represent faces with equal mouth curvature (upturned corners), and point 3, 6, and 9, which represent faces of opposite mouth curvature (down-turned corners), are located at opposite extremes of Axis X1. Points 2, 5, and 8, representing faces with horizontally oriented mouths, have minimal values on Axis X1.
[FIGURE 3 OMITTED]
Similarly, the location of points on Axis X2 reveal the adequacy of interpreting this axis as another facial configuration characteristic: brow slant.
Now, we demonstrate the accordance of the structure of points on Plane X1X2 with the emotional expressions of the corresponding schematic faces. Figure 3a shows the names of the emotions rated highest by the participants by matching the schematic facial expression with an emotional category from a list of 25 categories (Izmailov et al., 1999). According to this information, Face 7 was described with the emotions "joy" and "ardor," Faces 1 and 4 were similarly characterized by the category "surprise." Faces 6 and 3 were described in terms of "sorrow" and "grief," Face 9 of "anger," and "irritation," and Face 5 was described as "calm." According to the emotional characteristics, Face 2 falls between Faces 1 and 3, and Face 8 is close to Face 9, although neither face has clear maximally expressed emotions corresponding to the emotional categories.
The location of the basic emotions in Figure 3a shows a circular trajectory on Plane X1X2 with the "calm" face at the centre of the plane, which corresponds to the well-known circular emotion models of previous works (Ekman & Friesan, 1978; Osgood, 1966; Schlosberg, 1941). One can therefore conclude that the orientation of the first two axes of the schematic face-differentiation space, based on configuration characteristics such as mouth curvature and brow angle, allows us to classify Plane X1X2 of this space as the plane of emotional quality, which, according to the spherical emotion model, is determined by the horizontal angle of the points-stimuli (Fomin, Sokolov,& Vaitkyavichus, 1979; Izmailov et al., 1999; Sokolov, 1992).
We now consider the next two Cartesian axes of the four-dimensional schematic- differentiation space (see Figure 3b). The orientation of Axis X3 is determined by configuration characteristics such as minimal (horizontal brows and mouth) and maximal curvature (summarized curvature of mouth and brows) of the features of the face. Thus, Axis X3 should be oriented so that the value on this axis of point 5, which represents the schematic face with zero mouth curvature and horizontal brows, should be located at maximum distance from points 1, 3, 7, and 9, which represent stimuli with maximal curvature of mouth and brows. As is shown in Figure 3b, these conditions are fulfilled, and furthermore, the remaining points (2, 4, 6, and 8), representing stimuli with intermediate values summarized on the pattern curve, are located on Axis X3 between the two indicated positions. This indicates that Axis X3 can be considered a visual channel that detects the degree of total curvature of facial line features independently of their location and sign of the curvature.
Following the fixation of the direction of the first three axes, the orientation of the fourth axis by the orthogonal system is automatically determined. Thus, its interpretation depends on the derived location of the points-stimuli on the axis. Figure 3b shows that the maximally distanced points on Axis X4 are 2 and 8 on one side, and Points 4 and 6 on the other. All the other points are located between them, practically on one level. Such location of the points may be due to the specific interaction of channels for the detection of the lower half of the face (mouth curvature), and the upper half (brow angle). In contrast to the channel represented by Axis X3, where the curves of the face are summarized regardless of which part of the face the curves are from, in the channel represented by Axis X4, the absolute value of mouth curvature is subtracted from the absolute value of the brow angle.
The validity of the interpretation of axes as visual systems to detect line orientation defining facial configuration was verified by constructing functions of the dependence of the coordinates of the points (ordinate) on the numeral values of the mouth curvature and/or brow angle (abscissa). A strict monotonous link between the specific measurement of graphic characteristics of the schematic face and the value of the coordinates of the corresponding stimulus in the facial differentiation space was defined for all four axes (see Figure 4). This means that the Cartesian coordinates of the spatial facial differentiation space does actually represent the visual mechanisms of facial configuration detection, and, in particular, facial linear patterns.
Thus, dissimilarity estimates between schematic faces contain two types of visual information: On the one hand, they provide information about the configuration differences in the linear pattern, and on the other hand, they provide information about emotional expression. Using the multidimensional scaling technique, these two types of information can be split into the subjective schematic face-differentiation space, which is a spherical surface with a four-dimensional Euclidean space. Four Cartesian coordinates of points-stimuli characterize the input of the facial image configuration features; whereas only two spherical coordinates of the same points (horizontal and vertical angles) characterize the emotional content of the facial expression. This geometrical model is the basis for the analysis of the schematic face EPDs in the present study.
Schematic Face-Differentiation Space Based on the Amplitudes of EPD Components
As mentioned previously, analysis revealed three EPD components--inter-peak amplitudes P120-N180 and N180P230 and the amplitudes of the N180 peak--that we suggest are connected stimulus configuration and categorical characteristics of the face (Bimler & Kirkland, 2001; Izmailov et al., 1998, 1999, 2004; Paramey, 1996; Paramey et al., 1992). The results of this analysis are presented in Figures 5-7 in the form of graphs, similar to the graphs representing the schematic face-differentiation space from Figures 3a and 3b.
[FIGURE 4 OMITTED]
Occipital Electrode Sites O1 and O2
Figures 5 and 6 show graphs representing the projection of points-stimuli (schematic face) on two planes (X1X2 and X3X4) of a four-dimensional space derived from multidimensional scaling of inter-peak P120-N180 (5a, 5d) and N180-P230 (5c, 5f) amplitudes and N180 peak amplitudes (5b, 5e) of the EPD recorded at the left (top row) and right (bottom row) hemisphere occipital sites. The graphs in Figure 5 are made in the same way as the subjective rating graph in Figure 3a, and the graphs in Figure 6, are made similarly to the graphs from Figure 3b. Comparison of the graphs shows that, topographically, the distances between points relative to each other in Figures 3a, 5a, and 5d are identical, as are the distances in the graphs of Figures 3b, 6a, and 6c. This shows that they do not contradict the Cartesian coordinate interpretation of either the configuration characteristics of mouth and brow curvature, or the spherical coordinates of emotional quality and intensity. However, the metric representation (similarity of the inter-point distances) on the graphs reveals definite differences.
In the first group, in Figures 5a and 5d, a change in the distance between points 1, 2, and 3 is noted in comparison to the data in the subjective space (Figure 3a). These changes exceed possible point shift due to ordinary noise of the experimental procedure (in Figure 5a, the noise diapason corresponds to 10% error, represented as a circle around the points-stimuli Comparing the distance change with the graph from Figure 3a, the direction of the changes is opposite to those from the latter Figure. With a general configuration similar to the data in Figure 3a, the graph from Figure 5a reveals a convergence of the stimuli that have a similar configuration (Faces 1, 2, and 3 differ only in mouth curvature), whereas the graph from Figure 5d is more in accordance with the emotional characteristics of the schematic faces. In this case, the points represent faces 1 and 4, and 3, and 6, which have correspondingly paired similar characteristics of "surprise" and "sorrow". One can thus conclude that the inter-peak P120-N180 EPD amplitudes also contain two types of information, similar to that of subjective schematic face-differentiation rating. However, the P120-N180 amplitudes at the O1 (left occipital hemisphere) site were more influenced by the configuration content of the difference between stimuli, whereas the O2 (right hemisphere) site was more influenced by the emotional content of the stimulus.
[FIGURE 5 OMITTED]
[FIGURE 6 OMITTED]
In the second group of graphs of Figures 6a and 6c, there is an increase in the differentiation of the locations of the points on the planes compared to the subjective space (Figure 3b). Whereas in the subjective rating space, all 9 points are located in one quadrant (bearing in mind that in the spherical model, the origin of frame of reference is fixed), in the EPD space, the points are spread out on the plane. As a whole, the interpretation of the coordinate axes, both Cartesian and spherical, is in accordance with the results of the subjective rating. On Axis X3, the order of the points-stimuli is in accordance with the total feature curvature of the schematic faces, from Stimuli 1, 3, 7, and 9 (with maximum curvature) to Stimulus 5 (with zero curvature of mouth and brows). On the other hand, such point location determines the second spherical coordinates as the saturation of emotional expression. The location of these points on Plane X1X3 would obviously be identical to that on Figure 3c. The consecutive long latency EPD components at the left occipital electrode site (N180 peak amplitude and N180-P230 interpeak amplitudes) result in shifts that distort the structure of the stimuli. For Component N180 (Figure 5b), the topography of the graphic characteristics of the stimuli is still maintained. Sets of three of separate stimuli are arranged on Plane X1X2 horizontally and vertically in accordance to the matrices of the stimuli (Figure 1). However, as a whole, the structure of the points is disorganized. For the N180-P230 component though, there is topographic distortion. It is obvious from Figures 5c and 6b, that the points are distributed in space only according to the mouth curvature of the facial stimulus. The axes that are completely (X1) or partially (X3 and X4) determined by this feature maintain the stimulus order, whereas the location of points on Axis X2, which represent another configuration feature (brow decline), is rather chaotic. Thus, we suggest that information about the configuration characteristics is reflected in the left occipital cortex primarily by the first medium latency component (P120-N180), and with increasing EPD component latency, there is increasing distortion, which demonstrates the gradual decrease of the activity of the facial configuration-differentiation neural network.
[FIGURE 7 OMITTED]
At the O2 electrode site (Figure 5e, 5f), the last two EPD components lose all resemblance to the subjective configuration feature rating, but interestingly enough, the influence of emotional opposition "satisfaction-dissatisfaction" is retained. Figures 5e and 5f clearly show that Faces 3, 6, and 9, which express negative emotions, are grouped in one locus, distinctly separate from the rest of the faces that express neutral or positive emotions. This, as well as the information in Figure 5d, reveal that stimuli activate neurons of the occipital region of the right hemisphere that are associated with the analysis of emotional expression.
Temporal Electrode Sites T5 and T6
Whereas the EPD amplitudes at the occipital electrode sites reveal a difference in the values of the left (Figure 5a, 5b, 5c) and right (Figure 5d, 5e, 5f) hemispheres, at the posterior temporal cortex, the results are also asymmetric, but cardinally different from the values obtained at the occipital cortex. Analysis of the results of the data from the posterior temporal cortex are presented in Figures 7a, 7b, 7c, and 7d, 7e, and 7f. The graphs in these Figures show that, for all three components at the T5 electrode site (Figure 7a, 7b, 7c) and the long latency component N180-P230 at the T6 electrode site (Figure 7f), the point configurations obtained are, to a great extent, similar to the results of the P120-N180 component at the O1 electrode site (Figure 5a). In this case, there is also a convergence of Points 1, 2, and 3, and a divergence of Points 4, and 1, and Points 3 and 6, revealing a greater influence on the EPD amplitudes of configuration differences between stimuli than of emotional differences. In turn, the graphs of Figure 7d and 7e, representing the amplitudes of the P120-N180 and N180 components at the T6 electrode site show greater coherence with the results of the early latency P120-N180 component at the O2 electrode site (Figure 5d) and, accordingly, allow us to indicate the noticeable influence caused by the emotional content of the presented stimuli on the amplitudes of these components. A further difference between the posterior temporal and occipital electrode sites concerns the time required for the differentiation process. In the left temporal cortex, this process is reflected in an identical way for a period of time covering all three EPD components. In the right temporal cortex, the process terminates for the emotional element of the EPD (Figure 7d and 7e), but as shown in Figure 7f, continues for the configuration element of the EPD.
The above results can be generalized as follows. The first conclusion is about the visual cortex. It is based on the maintenance of the topographic structure and the local changes in the inter-point distances in the spherical schematic face differentiation model, and on the supposition that the neurons of the visual cortex of both hemispheres form identical channels for the detection of configuration features of the stimulus represented in the spherical model by four Cartesian points-stimuli coordinates. The union of these channels in a differentiation network is different in each hemisphere: In the visual cortex of the right hemisphere, the network is aimed at identification of the expressive features of the stimulus, and in the left hemisphere, at detection of geometric form features.
Comparison of the subjective difference rating of the emotional content of the schematic faces with the inter-peak EPD amplitudes registered at the O1 and O2 electrode sites in response to abrupt change of stimulus reveals that only the earliest of the medium latency EPD components (P120-N180) fully reflects the schematic face-differentiation process of the visual system. This holds true both for the O1 electrode site, where the input of graphical features is greatest (Izmailov et al., 2004; Rudell, 1991), and for the O2 electrode site, where emotional content has greatest input. Analysis of the EPD components shows equally rapid activation of these networks (120-180ms) and abrupt (especially in the right hemisphere) deactivation. With increasing EPD latency, there is increased attenuation of the activity of the "configuration" and "expression" facial-differentiation networks, with attenuation speed greater in the right occipital region than in the left.
The process of activity attenuation of the neural networks reveals that each network is made up of dominant and secondary channels. Dominant channels (detection of mouth curvature in the "configuration" network and detection of emotional signs in the "expression" network) maintain their activity longer than the secondary channels (brow angle detection in the "configuration" network and detection of emotional saturation in the "expression" network). Such channel asymmetry of the neural networks of the visual system was revealed in studies of line orientation (Izmailov et al., 2004) in which dual channel network analysis of line orientation showed that the activity of one of the channels, which detected opposite characteristics such as "vertical-horizontal," was reflected in the amplitudes of several EPD components, whereas the activity of the other opponent channel (left-right decline) was reflected in only one of the components.
The second conclusion is about the temporal cortex. Here, the right hemisphere is shown to be more involved in the emotional network of facial differentiation, although in contrast to the occipital cortex, in the right temporal cortex, the configuration network is simultaneously activated and maintains its activity after the activity of the emotional network is terminated. This can be assumed to be an index of the activity of the neurons of the temporal cortex associated with facial identification, which, in the right hemisphere, starts simultaneously with the activation of the neurons involved in the detection of the emotional expression of the presented face, but lasts longer. Another difference between the results of the temporal cortex and the results of the occipital cortex is that the temporal cortices of both hemispheres are equally involved in facial identification. Judging by the input of the configuration component into the amplitudes of all the EPD components at the T5 electrode site, facial identification process can be assumed to continue for an equal period of time in both hemispheres, which also differentiates the temporal cortex from the occipital cortex, where only the amplitudes of the first two EPD components are influenced by the configuration content of the stimulus.
The results of the present study reveal a greater influence of the right hemisphere on the perception of the emotional content of visual stimuli, and are in accordance with results of spontaneous EEG (Kostandov, 1980) and EP (Component P300) analysis (Herrmann et al., 2002; Jeffreys, 1992) of the presentation of facial stimuli expressing positive and negative emotions, in comparison to neutral facial expressions. Although the stimuli in these studies were facial photographs and not schematic faces, these studies, as well as other similar studies, revealed right hemisphere asymmetry in relation to the visual perception of emotion- laden stimuli. On the other hand, our results are in accordance with the results of studies of the right hemisphere temporal cortex (Perrett et al., 1982; Rolls, 1984), which revealed the presence of neurons that detected not only facial emotional expression, but also were specialized in the identification of the face. Studies of patients with focal lesions of this region (Adolphs et al., 1994; Davidson, 1984; Eirner & McCarthy, 1999) support the suggestion of the involvement of the right hemisphere temporal cortex in facial identification.
Analysis of the amplitudes of medium latency EPD components within the 120-230 ms range using multidimensional scaling shows that EPD contains information about the configuration differences between schematic face stimuli and about differences in the emotional expression of these faces. The spherical schematic face-differentiation model established on the basis of EPD amplitudes is identical to the model based on the dissimilarity judgments between stimuli and allows the separation of configuration and emotional components of the EPD amplitudes.
Analysis of EPD components reveals that, in both occipital lobes, all information is processed in the earliest of the medium latency components (P120-N180 inter-peak amplitude). A greater contribution of the emotional content of the stimuli has been shown for EPD amplitudes of the right hemisphere electrode sites, in comparison to the left hemisphere sites, where configuration differences contribute more. These sites also differ in the duration of activation in response to stimulus change. In the right hemisphere, activation associated with stimulus differences terminates rapidly, with the stimulus differentiation space constructed according to the measurement of the following EPD components being completely destroyed by incidental noise, whereas, in the left hemisphere, activity gradually attenuates, showing that distortion of the space structure monotonously increases with increasing latency of the component.
Analysis of the EPD components recorded at the posterior temporal electrode sites shows that, in contrast to the occipital cortex, the right hemisphere temporal electrode sites are influenced simultaneously by the difference in configuration and emotional content of the stimuli presented, whereas in the left hemisphere, inter-stimulus configuration differences dominate. At both temporal electrode sites, the duration of informational activity increases. Activation at the T6 electrode site related to the facial expression is best revealed in the P120-N180 component, and gradually diminishes with increasing component latency, whereas in the left temporal cortex, activation by the configuration content is maintained at a constant level for all three EPD components.
Received June 9, 2005
Review received August 4, 2005
Accepted August 8, 2005
Abelson, R.P., & Sermat, V (1962). Multidimensional scaling of facial expressions. Journal of Experimental Psychology, 63, 546-554.
Adolphs, R., Tranel, D., Damasio, H., & Damasio, A. (1994). Impaired recognition of emotion in facial expressions following bilateral damage to the human amigdala. Nature, 372, 669-672.
Bimler, D., & Kirkland, J. (2001). Categorical perception of facial expressions of emotion: Evidence from multidimensional scaling. Cognition and Emotion, 15, 633-658.
Bongard, M.M. (1955). Kolorimetriya na jivotnyh (Colorimetry on the animals). Proceedings from Soviet Union Academy of Science, 103, 239-242.
Boucsein, W., Schaefer, F., Sokolov, E.N., Schroder, C., & Fureby, J.J. (2001). The color-vision approach to emotional space: Cortical evoked potential data. Integrative Physiological and Behavioral Sciences, 36, 137-153.
Damasio, A.R , Damasio, H., & Van Hoesen, G.W. (1982). Prosopagnosia: Anatomic basis and behavioral mechanisms. Neurology 32, 331-341
Davidson, J.R. (1984). Hemispheric asymmetry and emotion. In K.R. Sherer & P. Ekman (Eds.), Approaches to emotion (pp. 39-57). London: Erlbaum.
Eirner, M., & McCarthy, R.A. (1999). Prosopagnosia and structural encoding of faces: Evidence from event-related potentials. NeuroReport, 10, 255-259.
Ekman, P., & Friesan, W.V. (1978). Facial action coding system: A technique for the measurement of facial movement. Palo Alto, CA: Consulting Psychology Press.
Estevez, O., & Spekreijse, H. (1982). The "silent substitution" method in visual research. Vision Research, 22, 681-691.
Fomin, S.V., Sokolov, E.N., & Vaitkyavichus, G.G. (1979). Isskustvennye organy chuvstv [Artificial sensory organs]. S. Petersburg: Nauka.
Herrmann, M.J., Aranda, D., Ellgring, H., Mueller, T.J., Strik, W.K., Heidrich, A., & Fallgatter, A.J. (2002). Face-specific event-related potential in humans is independent from facial expression. International Journal of Psychophysiology, 45, 241-244.
Hubel, D.N., & Wiesel, T.N. (1962). Receptive fields, binocular integration and functional architecture in the cat's visual cortex. Journal of Physiology, 160, 106-154.
Izmailov, Ch.A., Isaichev, S.A., Korshunova, C.G., & Sokolov, E.N. (1998). Spetsifikatsia tsvetovogo I iarkostnogo komponentov zritelnogo VP u cheloveka [Specification of color and brightness components of human visual EP]. Zhurnal vysshei nervnoi deiatelnosti, 48, 518-527.
Izmailov, Ch.A., Korshunova, C.G., & Sokolov, E.N. (1999). Sfericheskaia model razlichenia emotsionalnykh vyrazhenii skhematicheskogo litsa [A spherical schematic face emotional expression differentiation model]. Zhurnal vysshei nervnoi deiatelnosti, 49, 186-199.
Izmailov, Ch.A., Korshunova, S.G., & Sokolov, E.N. (2001). Relationship between visual evoked potentials and subjective differences between emotional expressions in "face diagrams." Neuroscience and Behavioral Physiology, 31, 5, 529-538.
Izmailov, Ch.A., Korshunova, C.G., Sokolov, E.N., & Chudina, Yu.A. (2004). Geometricheskaia model razlichenia orientatsii linii, osnovannaia na subektivnykh otsenkakh I zritelnykh vyzvannykh potentsialakh [A geometric model of line orientation based on subjective estimates and visual evoked potentials]. Zhurnal vysshei nervnoi deiatelnosti, 54, 267-279.
Izmailov, Ch.A., & Sokolov, E.N. (2004). Subjective and objective scaling of large color differences. In C. Kaernbach, E. Schroger, & H. Muller (Eds.), Psychophysics beyond sensation. Laws and invariants of human cognition (pp. 27-42). Mahwah, NJ: Erlbaum.
Jeffreys, D.A. (1992). The vertex-positive scalp potential evoked by faces and by objects Experimental Brain Research, 91, 340-350.
Kostandov, E.A. (1980). Hemispheric asymmetry of cortical visual evoked potentials to neutral and emotional stimuli. In H. Lechner (Ed.), EEG and Clinical Neurophysiology (pp. 740-745). Amsterdam: Elsevier.
Osgood, C.E. (1966). Dimensionality of the semantic space for communication via facial expressions. Scandinavian Journal of Psychology, 7, 1-30.
Paramey, G.V. (1996). Konturnye izobrazhenia litsa: mogut li oni peredavat emotsionalnye sostoiania? [Contour facial imaging: Can it portray emotion?]. Psikhologicheski zhurnal, 17, 70-85.
Paramey, G.V., Izmailov, Ch.A., & Babina, V.S. (1992). Emotsionalno-ekspressivnye kharakteristiki skhematicheskogo litsa na displee komputera [Schematic face emotional expression characteristics on computer screens]. Vesti MGU. Ser. 14. Psikhologia, 3, 30-38.
Perrett, D.I., Rolls, E.T., & Caan, W. (1982). Visual neurons responsive to faces in the monkey temporal cortex. Experimental Brain Research, 47, 329-342.
Rolls, E.T. (1984). Neurons in the cortex of the temporal lobe and in the amygdala of the monkey with responses selective to faces. Human Neurobiology, 3, 209-222.
Rolls, E.T. (1998). The brain and emotions. Oxford, UK: Oxford University Press.
Rudell, A.P. (1991). The recognition potential contrasted with the P300. International Journal of Neuroscience, 60, 85-111.
Schlosberg, H.S. (1941). A scale for the judgment of facial expressions. Journal of Experimental Psychology, 29, 497-510.
Shepard, R.N. (1964). Attention and the metric structure of the stimulus space. Journal of Mathematical Psychology, 1, 54-87.
Shepard, R. (1981). Mnogomernoe shkalirovanie i nemetricheskie predstavlenia. Normativnye i deskriptivnye modeli priniatia reshenii [Multidimensional scaling and non-metric representation. Normative and descriptive models of decision making]. Materials of the Soviet-American Symposium (pp. 84-97). Moscow: Nauka.
Shvelev, I.A., Kamenkovich, V.M., & Sharaev, G.A. (2000). Otnositelnoie znachenie linii I uglov geometricheskikh figut dlia ikh opoznania chelovekom [Relative values of lines and angles in human perception]. Zhurnal vysshei nervnoi deiatelnosti, 50, 403-409.
Sokolov, E.N. (1992). Detector mechanisms of perceptions and emotions. In D.G. Forgays, T. Sosnowski, & K. Wrzsiewski (Eds.), Anxiety: Recent developments in cognitive, psychophysiological, and health research(pp. 153-165). Washington, DC: Hemisphere Publishing Corporation.
Sokolov, E.N., & Izmailov, Ch.A. (1983). The conceptual reflex arc: A model of neural processing as developed for color vision. In H.G. Geissler (Ed.), Modern Issues of Perception (pp. 192-216). Berlin: VEB Deutscher Verlag der Wissenschaften.
Supin, A.Ya. (1981). Neirofiziologia zrenia mlekopitaiushchikh [Neurophysiology of vision in mammals]. Moscow: Nauka.
Zimachev, M.M., Shehter, E.D., Sokolov, E.N., & Izmailov, Ch.A. (1986). Hromaticheskaya sostavlyauschaya elektroretinogrammy lyagushki [Chromatic component in frog's ERG]. Zhurnal vysshei nervnoi deiatelnosti, 36, 1100-1107.
Chingiz A. Izmailov, Evgeni N. Sokolov, and Svetlana G. Korshunova
Moscow State University
This study was supported by grants from the RFH Nr. 01-08-00176a and RFBR Nr. 04-06-80245a.
Correspondence should be addressed to Ch. A. Izmailov, Moscow State University. E-mail: email@example.com
English revision: Virginia Navascues Howard
Table 1 Correlation Coefficients between Amplitudes of EPD Components and Dissimilarity Judgments Obtained in Psychophysiological and Psychophysical Experiments of Differentiation of Schematic Faces Site Inter-peaks amplitude Peak amplitude P120-N180 N180-P230 N180 P230 01 .830 .816 .759 .600 02 .739 .769 .742 .489 P3 .723 .830 .667 .752 P4 .730 .791 .644 .709 T5 .832 .774 .783 .603 T6 .839 .792 .794 .579 Note. From Izmailov, Korshunova, & Sokolov (2001). Table 2 Indexes Characterizing the Spherical Structure of Schematic Face-Differentiation Model as a Function of Dissimilarity Judgments and EPD Amplitudes Recorded at Occipital and Temporal Sites Electrode site Difference Spherical space indices judgments O1 O2 T5 T6 Inter-peak amplitude P120-N180 Stress 0.02 0.05 0.05 0.02 0.03 Mean radius 0.99 0.99 0.99 1.01 1.00 Standard deviation 0.053 0.048 0.098 0.058 0.070 Variation coefficient % 5.4 4.9 9.8 5.7 7.0 N180 peak amplitude Stress 0.03 0.02 0.03 0.03 Mean radius 0.99 1.00 1.00 1.00 Standard deviation 0.099 0.100 0.087 0.077 Variation coefficient % 9.9 10.0 8.7 7.7 Inter-peak amplitude N180-P230 Stress 0.05 0.01 0.05 0.04 Mean radius 0.99 0.99 1.01 1.02 Standard deviation 0.083 0.050 0.106 0.027 Variation coefficient % 8.4 5.0 10.5 2.6
|Printer friendly Cite/link Email Feedback|
|Author:||Izmailov, Chingiz A.; Sokolov, Evgeni N.; Korshunova, Svetlana G.|
|Publication:||Spanish Journal of Psychology|
|Date:||Nov 1, 2005|
|Previous Article:||Is the DeVries-Rose to Weber transition empirically possible with sine-wave gratings?|
|Next Article:||A new mathematical model for assessment of memorization dynamics.|