Conscious and unconscious implicit anticipation.
The topic is quite comprehensive and is out of scope of a single article to describe all the 8 levels of my 8-factor anticipation design. Therefore I will focus in this article only on the two basic ones in order to allow more detailed look into my thoughts behind it.
As an introduction I'd like to show few motivational examples to explain what I mean when I talk about levels of anticipation. The first example is taken from work of Mihai Nadin (Nadin, 2003). In this particular piece of work Nadin analyzes the so called the baroreceptor reflex. It affects the heart rate via the nervous system in order to maintain the blood pressure within relatively constant and safe limits. The question asked "how does the heart, or whatever controls the heart, know what the body will do" goes beyond the physics of pumping a liquid through pipes. Please note here that this process works without a voluntary control. This is one of the key observations for my work. I will develop this thought further in the text. As a second example I wish to mention the work of Alexander Riegler (Riegler, 2003). We can use almost any competitive sport, for example tennis. While playing such sports each player is trying to estimate, based on the available information, the opponent's action and also the result of his action (for example trajectory of the ball). This estimate is based on the "model" each player has. It contains among other the human body physiology, the laws of gravity and physics. These may not be expressed as equations but as patterns and rules for reaction. These patterns and responses can be improved by training.
[FIGURE 1 OMITTED]
Please note the significant difference in the examples presented. Both examples seem to be different. The first has no notion of a model, improvement by training or voluntary control while the second has. There is however a common denominator in both--the prediction of future (change in posture, ball trajectory). Another similarity is that the prediction plays important role in closing the regulation loop. In my view the prediction of future that influences the current decision is called anticipation. This clearly suggests that anticipation is not a function of the organism it is built in it and even on different levels.
Researchers categorized the anticipation from different points of view already. The most explanatory and straightforward was presented by Martin Butz (Butz, 2003). Even if it is very comprehensive I was not entirely satisfied with it. I missed a notion about voluntary control observed in the examples shown above. This led me to the key idea of my work--including the consciousness view in all of them. I created thus not four but eight types of anticipation by distinguishing a conscious and unconscious type of the Butz's four anticipation categories.
My idea is sketched out on Figure 1. The complexity in the Figure 1 grows from left to right and from bottom to top. In this article I will focus only on the first two levels. It is due to say that I current anticipatory approaches spans through several levels of my design and I'm aware of that. Still I believe that their logical segregation and detailed analysis can help to analyze the anticipatory behaviour but. The goal of my work is to prove usefulness of this decomposition.
2. Unconscious Implicit Anticipation
I understand unconscious implicit anticipation as the behaviour that the creature was already equipped with either by nature (evolution) or creator (in artificial case) and which is also not voluntary controlled. This may seem like a definition of reactive behaviour in sense of classical robotics but reactive behaviour is often understood as an opposite for anticipatory one. This deserves detailed analysis. Few years back anticipation was perceived as something on higher level of behaviour control, but I'm convinced that anticipation is part of many low level mechanisms (such as the demonstrated blood pressure regulation). Looking at this from the point of my 8-factor anticipation scheme this very basic level has limited possibilities. What I'm trying to show is my different point of view. The reaction cannot be associated with prediction of next sensory value, reward or even state because on this level these are not available in my model (Figure 1). Simply because this level has no means to evaluate the reward or state on the unconscious level. There are dedicated levels in my architecture to do these but their computation might take more time than is needed for reaction. Classical view of implicit anticipation (in sense of Butz's definition) would be satisfied with the statement that it is the prerequisite given to the system either by the long evolution or by the creative mind of architecture designer. I approached this in different and detailed manner.
Let me formalize the approach. The basic reactive artificial creature behaviour consists of the set of inputs I, set of possible outputs O and inference mechanism called action selection. Please note that I'm not labelling these as agent's sensory inputs and actions. This is intentional generalization so it can be used not only for agent as whole, but also for agent internal blocks.
The reactions are captured in form of projection
I [??] O. (1)
The inference mechanism evaluates the input and decides about the output. This can be captured in form of simple IF-THEN rules. This is a couple of decades old approach and there are a lot of possibilities for a technical realization from binary rule base to Anticipatory Classifier Systems (Butz, et.al. 2003)Fehler! Verweisquelle konnte nicht gefunden werden.. I see my contribution in revisiting these principles a modifying them with anticipatory features or discovering the anticipatory features already built within. The anticipation brings into the projection another variable. Newly there is the expected future input after the executed action
I [??] O x I. (2)
Let's forget for a moment about how the expected future input is predicted. First let me continue with the formalism even further. What I shown is a classical anticipatory modification. This is still nothing new however this is the basics of the anticipation. One thing that should be realized at this point is that on unconscious implicit anticipation level there is no mechanism to modify the rule base (only evolution or redesign). This leaves little space for improvements. I would like to show, that even here, there is possible to find some enhancements. There might still be modifications available. I suggest two to the classical approach. One of them is to add to the equation also the expected next action. It will result in the projection of
I x O [??] [O.sup.2] x I. (3)
This describes the expected or as I like to call "prepared" next action. Let's take another example to demonstrate the idea. Imagine the predator evasion situation. The vicinity of predator is input I the reaction might be to move away from predator (output O). Even if the reaction process is fast it takes some time (lookup the rule in database). What will happen in case the agent needs to take another decision? It has to go through the action selection process again to derive new action even if the new action is the same as previous one. This seems to me like a waste of time and resources. On the contrary if this action is prepared, it can be executed faster. For example if the input (or its significant part) did not change then there is no reason to change the output. Except the basic reaction base another area of involving the implicit and unconscious anticipation is the inner agent processes which imitate the creature physiology, energy management, and other internal processes. These are seldom consciously controlled but vital to the overall state of the agent.
3. Conscious Implicit Anticipation
Let me continue with next and for this paper last level of anticipation in my model. This level brings in addition to the previous one the conscious control. How can the implicit anticipation be consciously controlled when I claimed it to be imprinted in the creature? Everything still depends on the design as above, but the results are consciously processed or can be chosen to be controlled. So far I was describing the levels as isolated entities but that is just a logical separation. There is a necessity for the levels to interact. This can be done through a shared media, which can be called memory. The possibilities of this level are broader then of the previous one. The important aspect of behaviour which can be controlled on this level of my design is the attention focus. It decides what object(s) is the agent interested in. Higher levels can influence the attention too for cases where there is enough time for reasoning and decisions like in exploratory behaviour. The reason why I included it also on this level is that even if the animal or agent executes the reactive behaviour it has its focus on something (an object, an agent, on even itself). One may argue that the focus could be influenced by unconscious levels. I have this in my design but in higher levels (unconscious sensory anticipation).
I will describe above said in the formalism used in previous chapter. There is still only the set of inputs I and set of possible [outputs.sup.O]. We can use everything from the previous level up to the next output prediction
I x O [??] [O.sup.2] x I. (4)
What changed on this level is from which object or objects the information for the input I is taken. This depends on how much data can be processed simultaneously which can vary from creature to creature.
I will introduce the second improvement I suggest. With the data available it is possible to capture the rate of change for the input value. The values are typically in ALife discreet often not expressed numerically (action for example). In this case statistical measure such as probability or likelihood can be evaluated. This is another parameter that can help to choose the optimal action. I'm adding two new values the [r.sub.i] and [r.sub.o] which I will call rateability (combination of words rate and probability). This is to have one parameter for input and one for output regardless their continuous or discreet nature. Updating my rule equation will give
I x O [??] [O.sup.2] x I x [[Real part].sup.2] (5)
For continuous or discrete values that are differentiable we can use the rate (first derivation/difference providing the speed of change). For the non-differentiable values we can use probability of their occurrence counted during the agent lifetime to compute the rateability. How can be recognized if the input is continuous variable such as distance or a discreet value from a set such as actions? The answer is straightforward when realized that this level is all about imprinted information in the agent. Simply it is given by the designer for all the valued how they should be treated.
4. Implicit Anticipation Model
The previous chapters described and discussed the first two levels of my anticipatory architecture. The benefit of having this separated is not only in the ease of explanation but also in the possibility of separate implementation and evaluation of the principles in simulation. In this chapter I will describe in detail the implementation model for what was described above. Also it is due to describe what mechanisms and algorithms can be used. I need to address several things including prediction, inference, storing the rules, recalling them, evaluating rateability, generating prepared action and the attention. There are several options but we need to realize that we still cannot use any of the known approaches like reinforced learning (RL) or anticipatory classifier systems (ACS) because these consider reward. On the Fehler! Verweisquelle konnte nicht gefunden werden. there are shown the principles in the collision avoidance situation as an example followed by equations and examples of the values corresponding to these situations. It is interesting how the level of information that can be input to the decision process can vary from just one value from proximity sensor to the several values including expectation, and rate of change of inputs and outputs. This is one of the main messages, the model itself is very simple, but that is just for the sake of explanation how even the simple examples can be seen from different angles and that the number of information about the situation can vary.
[FIGURE 2 OMITTED]
As mentioned the situations on the Fig. 2. are described below with examples reflecting the respective values.
a. The classical reactive approach
IF x THEN y where x [member of] I, y [member of] O (6)
Example: IF distance < 0.5 THEN turn(90)
b. The classical anticipatory approach
IF x THEN y EXPECT zwhere x, z [member of] I, y [member of] O (7)
Example: IF distance < 0.5 THEN turn(90) EXPECT distance >= 0.5
c. First suggested improvement-action anticipation
IF x AND PREVIOUS_ACTION a THEN y EXPECT z AND EXPECT_ACTION b
where x, z [member of] I, y, a, b [member of] O (8)
Example: IF distance < 0.5 AND PREVIOUS_ACTION move THEN turn(90) EXPECT distance >= 0.5 AND EXPECT_ACTION move
d. Second suggested improvement-rateability evaluation
IF x AND PREVIOUS_ACTION a THEN y EXPECT z AND EXPECT_ACTION b
WITH < [r.sub.i], [r.sub.o] > where x, z [member of] I, y, a, b [member of] O, [r.sub.i], [r.sub.o] [member of] [Real part] (9)
Example: IF distance < 0.5 AND PREVIOUS_ACTION move THEN turn(90) EXPECT distance >= 0.5 AND EXPECT_ACTION move WITH <0.1,0.6>
Two things should be taken under consideration. First, the above described two blocks are part of larger architecture and they will interact with other blocks through memory as already mentioned. On the premise of having other blocks we can use above mentioned Anticipatory Classifier System (ACS) and say that it will not only implement the two levels described but also others. This is another important thought of my work, I use the levels to logically discuss the influence of anticipation, but using one implementation such as ACS will implement several levels from my architecture. Second, my goal is to provide as much as possible self-sufficient architecture for each level. Here I needed to realize that at the second discussed level the animate should be in conscious control and hence the desired value should be available (even when produced by higher levels). Hence I chose to use simpler implementation with use the emotivector introduced by Carlos Martinho in his work (Martinho, 2007). This helps to better analyze the details of both levels with too much dependency on other levels, as will be shown I can even get these two levels working independently without the desired value.
For the unconscious implicit anticipation there is in fact little need for experiment.
[FIGURE 3 OMITTED]
From the nature of this level, it is a matter of design, so the experiments would compare different designs of the same agent. I just want to show on the simulation example what this means. I have conducted my experiments in the REPAST simulation tool (Repast, 2008). The setup of the experiment is three different obstacle scenarios with two agents, each trying to reach food. In first experiment b) the unconscious implicit anticipation of "right angles" was used, which means that when meeting the obstacle rotating by 90 degrees is expected to overcome it. We can see that in this case the 3rd scenario is not achievable (agent get stuck, or oscillate between two positions). In this example we can see that simple imprinted mechanism fails, due to incorrect anticipation. In the second experiment I used different mechanism "wall following". We anticipate the wall to continue and not change direction too often. This architecture can successfully complete the third scenario. What was show here is the unconscious implicit anticipation as I understand it, the different behaviour patterns can be either designed or evolved. There is no conclusion from this experiment it was used to demonstrate my view.
With the conscious implicit anticipation two experiments were conducted, the first experiment was devoted to the selection of best predictor for the emotivector. Two possibilities were evaluated SimplePredictor (SP)
[[??].sub.t] = [[??].sub.t-1](1 - [w.sub.t]) + [x.sub.t] x [w.sub.t] where [w.sub.t] = [absolute value of [x.sub.t] - [[??].sub.t-i]] (10)
and LimitedSimplePredictor (LSP) which uses the same equation, but also keeps history of the input values calculates the mean and the deviation and limits the prediction if outside the statistical range (they are described in detail in Fehler! Verweisquelle konnte nicht gefunden werden.). The SimplePredictor convergence speed was slow (convergence to 0.1 [+ or -] 0.01 was 87 steps) while the other predictor was able to converge in 5 steps. Here it is due to say that these values are then further processed. However the properties of predictor play key role in the overall design. The other experiment was designed to test the attention focus mechanism. In this scenario there were two static agents and two moving agents. One static agent was an observer evaluating the environment input and focusing attention (shown as wolf), the second was a non moving agent (shown as flower--top right corner). The other two agents (shown as pigs) had different moving patterns. One was running in circles (gray--on the right) and second was moving randomly (white--on the bottom) the scenario is shown on Fehler! Verweisquelle konnte nicht gefunden werden. below.
[FIGURE 4 OMITTED]
It was confirmed by this experiment that the moving agents are more interesting for the observing agent than the static ones, which was expected based on the fact that emotivector is sensitive to the observed value change in time. It was measured during the experiment that the attention was focused on either of the moving objects. It greatly depended on the moving pattern and amplitude of position change. When the white pig was moving randomly only by 1, the focus was 68% on the pink agent. Increasing the move of the white to 2, then his changed the scenario and the focus was 83% on the white agent. This shows the strength and the weakness of this approach, the amplitude matters and influences the decision. It would require a significant fine tuning to combine with focus on internal needs of the agent. This is acceptable on the basic "reactive" level. However it is due to mention that some higher, attention focus mechanism needs to be employed. Also smoothing would be required as the focus oscillated and for chasing scenario it would not work correctly.
In this paper I have shown details of the unconscious and conscious anticipation from my 8-factor anticipation framework. I have shown that there are more types of behaviour that employs anticipation inherited by the creator (nature, designer). The main messages and contributions of my research are to draw an attention to anticipation as a mechanism involved almost in every aspect of artificial creature behaviour. So far the efforts on these fields were focused on integrating the anticipation in the design, but not integrating it on multiple points. Anticipation is no longer an isolated term, it is an essential part of the artificial creature. Another significant difference is the introduction of the conscious aspect of anticipation. This was described on two levels of my approach to anticipatory behaviour. The presented experiments were used to further underline the principles and what is understood behind them. The basics for more complex multi level anticipatory approach were described in this article. The last conclusion is suggested improvements that can significantly increase the information available to the agent for his decision. The intention of this article is to draw attention to an approach that I consider novel in the field of anticipation. The aim was not to extensively present all the results but to demonstrate the thinking behind. On simple examples I showed that even in the basics are possibilities which can be further extended in the higher levels. Some might object that the presented approach does not provide anything new. I argue that even this simple overview shows a new way of thinking about anticipation not used before. Because my angle of view is usually hard to grasp while there are already defined dogmas, I chose to demonstrate this on the simplest possible examples not to confuse readers and to focus solely on the main ideas of my work. The main conclusion to be made from this article is that consciousness plays important role and that anticipation is not matter of just one mechanism.
Nadin, M. Not Everything We Know We Learned. In Anticipatory Behavior in Adaptive Learning Systems. Lecture Notes in Computer Science 2684. Heidelberg: Springer, 2003
Riegler, A. Whose Anticipations? In Anticipatory Behavior in Adaptive Learning Systems. Lecture Notes in Computer Science 2684. Heidelberg: Springer, 2003
Butz, M. V., Sigaud, O., Gerard, P., Anticipatory Behavior in Adaptive Learning Systems: Foundations, Theories, and Systems, Springer Verlag, 2003. ISBN 3-540-40429-5
Butz, M.V. An algorithmic description of ACS2. In: Lanzi P-L, Stolzmann W, Wilson, S.W. (eds). Advances in learning classifier systems, vol 2321 of LectureNotes in Artificial Intelligence. Springer, Berlin, pp 211-229. 2002
Martinho, C. Emotivector: Affective Anticipatory Mechanism for Synthetic Characters. PhD thesis, Instituto Superior Tecnico, Technical University of Lisbon, Lisbon, Portugal, 2007
REPAST [online]. Last revision 2008. [cit. 2009-05-20]. http://repast.sourceforge.net/
Authors' data: Ing. Kohout, K[arel]; Artificial Life and Mobile Robotics Research, Czech Technical University in Prague, Faculty of Electrical Engineering, Department of Cybernetics, Technicka 2, 166 27 Prague 6, Czech Republic kohoutk@fel .cvut.cz
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||CHAPTER 46|
|Publication:||DAAAM International Scientific Book|
|Date:||Jan 1, 2010|
|Previous Article:||Voltage effect on developing process and B & W reproduction.|
|Next Article:||Disk drive simulation model development.|