Workflow-based personalised document delivery.

New employees in an organisation typically undergo a period of relatively intense training when they commence their employment. Often the quantity of information imparted is too large for the newcomers to assimilate during the short training period. Moreover, much of the information may not be relevant until months or even years after the initial training period, by which time it has long been forgotten. This article describes JITT, a smart personal assistant which delivers training documents to its user in a just-in-time manner. JITT uses workflow technology to drive the delivery of documents in a timely manner, based on organisational processes through which the user is working. User-modelling is incorporated into the system to ensure that redundant or previously known information is not delivered, thereby reducing the problem of information overload. Finally, the teaching agent, which is the engine of JITT, identifies the concepts to be learned, retrieves the best documents for teaching these concepts to the current user and customises the presentation of the documents' links in the user interface.

**********

There are many learning challenges within organisations. Assimilation of organisational knowledge is a problem encountered in almost any organisation (Dieng, Corby, Giboin, Golebiowska, Matta, & Ribiere, 2000). A particularly important group of problems relates to helping people learn things that are already well documented within the organisation. The challenge in supporting such learning is that the people within the organisation often have difficulty in gaining access to that information at the right time. In cases where the person is aware of their lack of knowledge, the primary challenge is to help them find the documents which can answer their needs. Even more challenging is the common case where the person is unaware that they have a learning need. This problem is compounded when people are feeling overloaded, either by the demands of their work or information overload.

One important aspect of this workplace learning relates to the learning load. The timing of teaching is critical to the success of learning. Hoyt pointed out that the Future Search Committee held during the ASTD International Conference and Exposition reported that "E-learning is dominated by an increasing demand for just-in-time learning" (Hoyt, 2002). In an organisational context, it is important to take account of the problems of information overload, especially in the case of people who are new to the organisation or who have recently changed their roles and responsibilities. Such newcomers to roles or organisations have a great deal to learn and are only able to absorb a limited amount of information at one time. In general, teaching should be paced so that a person is given information in manageable chunks, so as not to overload their short-term memory (Sweller, 1993).

It is particularly appealing to improve the delivery of documents at just the time that the learner is ready for them, when they need to learn. So, for example, suppose an employee has been in a job for six months at an organisation which requires him/her to take annual leave within the next six months. This employee would do well to begin planning that leave. So this would be a good time to ensure that they are aware of this leave policy. A simple way to achieve this would be to deliver a short prompt, perhaps via email, pointing them to the particular paragraph that they need know about in the leave policy document. This is just one simple example of a learning need that the employee may not have been aware of.

Another crucial aspect of workplace learning is the appropriateness of the information content given to the user--it should be adapted to the current context of their work, and to their pre-existing knowledge. There is a need to understand the user's needs in order to adapt the delivery of information. Some researchers have adopted a task-driven approach to model the context of the users' needs and build heuristics to tailor the information to them and present it in a just-in-time manner (Budzik, Hammond, & Birnbaum, 2001; Paradis, Crimmins, & Ozkan, 2003). In our project, we use a workflow approach to determine the context of the user's needs.

In organisations where there are automated processes for managing workflow, these processes can provide a framework for supporting learning. For example, our university has a process for managing examination scripts. This process could well be supported by a conventional workflow system. A new academic who is responsible for running a teaching unit in our organisation would need to learn about several aspects of the unit and its related processes. Since newcomers need to learn so many other things when they first join the university, it is not surprising that they generally have problems learning what they need to know to get through this process. It is typical of many learning processes which run over a period of several months and have several stages. The process requires completion of several forms and there are several relevant policy documents. There are many support learning documents, including the policy documents of the institution, and of the school and there are examples of ways to complete sub-tasks. It is very appealing to tap into a workflow for a process like this so that it supports not only the normal management of the documents and approval processes, but also the learning about those processes. This should be in good time as well as just-in-time.

Both the examples mentioned, leave and examination script management, are also typical of many aspects of workplace planning in that there is considerable tacit knowledge involved. It is important for an organisation to have a system in place for employees to acquire this tacit knowledge (Nonaka, & Takeuchi, 1995). It would be valuable to facilitate learning of these aspects by helping newcomers and those transitioning to new roles by helping them make contact with people who have recent relevant experience and willingness to help.

This article describes the current version of JITT, a Just-In-Time Training system which tackles these problems by building upon workflow models and technology (Davis, Kay, Lin, Poon, Quigley, Saunders, & Yacef, 2002). The next section of this article describes the overall architecture of JITT, and the subsequent sections describe the interface, the details on each of the major architectural elements, and related work.

ARCHITECTURE

The overall architecture of JITT is illustrated in Figure 1. At the top is the Human Resources (HR) staff team. They need to initialise the system when a new employee starts. This process can be triggered by the typical administrative tasks associated with this starting stage.

For example, the new employee will have a role within the organisation, and the role will be reflected in an obvious way in their user model. This will show their position and level. Normal duty statements within the organisation would naturally feed into the definition of default tasks for each user. For example, a new member of academic staff would typically have responsibility for teaching and research. Each of these roles can be used to create a default user model for these aspects of a new member of staff. This would include the elements that are a normal part of these roles. The teaching role would, for instance, generally include the need to set examination papers and submit these and associated paperwork. The initial model would include a teaching context and within that, there would be a context for handling examination scripts. For a new staff member, these would initially be modelled with a stereotype (Rich, 1979) to reflect that the user does not know the procedures.

[FIGURE 1 OMITTED]

At the bottom of Figure 1, we show the user. S/he can interact with JITT and update the user model. Although the HR staff creates the default model, the user can alter it. Users interact with JITT via the interface layer shown at the bottom of the main block in the figure.

When the Human Resources staff member establishes a new user in JITT, s/he also initiates the activity engine. This activates a set of workflows, each associated with a set of knowledge components in the JITT user model structure. So, for example, JITT creates user model components for the user's knowledge of leave processes within the organisation. This starts one workflow for that user for each of the forms of relevant leave. Similarly, the user model components for handling examination scripts are associated with a workflow for that process.

The workflow is managed by the activity engine. One important class of these is based on the typical process workflows that are currently used in organisations. One of these workflows could be used for the management of examination scripts. Unlike the case of typical business workflows, the JITT representation captures processes that affect an individual and their learning about the appropriate processes and policies within the workplace. This means that we identify stages in the individual's work even when the stages may not involve other people. The critical factor defining the stages we model in the workflow is that we identify the subtasks or sub-activities that constitute a larger activity. It is this breakdown of a task that makes it possible to order the teaching of relevant aspects with the user being taught how to do the current sub-process just as they need to tackle it.

The final part of the architecture is the teaching agent. This takes the information from the Activity Engine and the user model in order to formulate the presentation of information to the user. Since JITT has been designed to be highly modular, we designed it to allow for several teaching agents, each potentially taking a different strategy for its teaching. We use one of these agents as the default and allow the user to switch to the others.

We now discuss each of these elements in detail.

INTERFACE

We now describe the interface element of the architecture. This also provides an overview of JITT from the user's perspective. We have designed JITT so that it could potentially use a range of mechanisms to communicate with the user. At this stage, as indicated in Figure 1, our prototype implementation uses just one interface element--a web site. We will illustrate the operation of the interface in terms of the following pacing: a user, who we call Jane, starts employment and interacts with JITT on her first day on the job. We then show her the JITT screen at later stages.

JITT is intended to work in either push or pull mode. We first describe JITT's push mode of operation. As indicated in Figure 1, an individual user's JITT agent is initiated by the actions of the Human Resources staff. This means that the new employee's JITT agent can establish an initial user profile and student model and it can use these to formulate an initial set of teaching goals. As soon as JITT is initialised for the user Jane, she can log onto the JITT web site and see the activities that are already activated for her. These activities represent her set of initial recommended learning goals. Figure 2 shows an example of a JITT screen with this set of learning goals, which are organised into categories. From the user's point of view, these goal categories are intended to structure the presentation in terms of major activities in the workplace. In terms of our architecture, each main category maps to one workflow within JITT.

The top left box shows the list of current activities for the user Jane. As we can see in Figure 2, these are composed of "Course planning," "Postgraduate Supervision" and "Administrative Procedures for New Staff."

Directly below, the user can find one box for each current activity, and each box notes the stage she has reached for that activity (not all boxes are visible on the screenshot). In particular, the links of the documents relevant to the user for the current stage of the activity are pushed to the user. Sometimes, depending on the teaching agent, links to documents that the system judges to be already known by the user can also be shown in the same box under "Refresher documents." That way the user can consult them again if needed. At any time, the user can view the workflow that corresponds to the activity and see her current stage. This allows the user not only to visualise which stage she is at in the process, but it also provides the context in which a particular document is relevant now. An example is shown in Figure 3.

[FIGURE 2 OMITTED]

The right hand part of the screen in Figure 2 is composed of two areas. The "Important Items" shows the documents that the user has put aside for later. The "Document Viewing History" lists all the documents recently accessed with the most recently accessed document listed first. If a document is accessed more than once, it appears according to its last access date. These two areas are naturally empty when the user logs in for the first time, but later on in the training they start filling up (see Figure 5).

The results are sorted by activity in JITT. Essentially, the user should see learning recommendations based on a combination of the request and the user's current learning context.

[FIGURE 3 OMITTED]

JITT also operates in a pull mode when the user initiates a search request (in the "Search Learning Centre" text area in the top right corner of the window). The system looks up both documents that are linked to the currently active workflows for this user and other collections of organisational documents. It then returns these documents, as well as the corresponding workflows that teach this concept. For instance in Figure 4, user Jane has just performed a query on "leave." She is provided with a set of available documents related to the concept "leave" (shown in the right frame) and a set of workflows that match the concept "leave" (shown in the top left frame). The user can then either simply consult the documents or choose to activate one of the workflows and decide to follow the system's advice on learning about this concept. In the latter case, the workflow will be added to the list of learning activities.

[FIGURE 4 OMITTED]

Figure 5 indicates the types of changes in the interface in the later visits to the JITT site. Let us suppose that Jane, after her search request on "leave," decided to activate the workflow "Academic Leave: Academic annual leave and annual leave loading." As shown in Figure 5, the workflow is added to the list and she now sees four active workflows. The overall learning categories, as active workflows, will tend be stable over long periods of time. However, at each stage, different learning objectives will be suggested for some of them. For instance, Jane has progressed in the first learning activity (Course Planning). She is at a stage called "Exam Preparation," which corresponds to a node in the workflow. Four document links are pushed to her and she also has a list of refresher documents under that activity. Once she is ready she will click on the button "Preparing the Marking Scheme" and the workflow will progress to the next node, updating the set of links in the box for that activity. We can also see in this figure that the two frames on the right now have information in them. Jane has put two documents aside for later, with a short description as to why (i.e., "Need to find a suitable time"). Her document viewing history has also grown, showing that the most recent document she accessed was "Setting the questions." Documents appear in reverse chronological order, and if a document is accessed a second time it is moved to the top of the list, hence only showing the latest access date.

[FIGURE 5 OMITTED]

ACTIVITY ENGINE

The workflow technology lies in the activity engine. In the literature, when workflow technology is applied to teaching, it is used to handle the administration side of teaching or to let students pace their learning activities by relaxing the time constraints. The Flex-eL project (Lin, Ho, Sadiq, & Orlowska, 2001) is an example of such a system. JITT uses workflows differently. Rather than modeling the teaching workflows, JITT uses the existing business workflows and deduces the pacing of the training from them.

The workflows in JITT are simplified versions of standard business workflows that may exist for the organization, such as a purchasing process or examination script process, except that their nodes may link to metadata about the concepts being dealt with at this point of the workflow. Figure 6 shows a workflow for managing examination scripts in our university. Many nodes have concepts attached to them (we have only shown some of them so as not to clutter the diagram). These concepts are depicted with a dashed box, and can be associated with several nodes across different workflows. For instance, the concept "Large scale copying procedures" (present for the node "Large scale copy" in this workflow) is actually also relevant to two other workflows: Stationery and General teaching. If the user has already seen these procedures in the context of either of the two other workflows (as indicated in the user model), then the procedures will not be suggested again to the user. However, if the user has not seen them, they will be pushed to the user.

The organisational documents are tagged and indexed with metadata representing the concepts explained in the documents. For instance, a document called "Seminar on preparing examinations" contains the concepts "exam aligned with scope and objectives," "setting exam questions," "write exam question answers," "develop marking scheme" and "common exam problems."

The same metadata is also used to tag the various stages of each workflow. So, for example, the node in the previous figure called "Write exam" contains the concepts "Confidential labelling of exam papers," "Setting exam questions" and so on (this node actually corresponds to a smaller grained workflow, so it contains many concepts). When the user reaches the node "Write exam," the teaching agent may decide that the document "Seminar on preparing examinations" should be given to the user.

[FIGURE 6 OMITTED]

The role of the activity engine is to encode the workflows and establish the correspondence between stages of the workflows and the documents that need to be known for each stage. However, the decision about exactly what documents should be shown to the current user is made by the teaching agent.

Workflows are represented using XML files defining the states, the transitions between states and the concepts that need to be known in order to do the work of each state.

USER MODEL

The user model holds the user profile data as well as the student model. For example, it holds profile details such as the user's name and position in the organization. The student model holds the learning concepts. For the workflow described above, these include knowledge of how to organise an in-house examination, a formal examination, as well as the associated concepts such as how to do large-scale copying, how to write an examination, and how to draft a marking scheme.

The user model is implemented using the accretion-resolution representation (Kay, 1995; Kay, 2000). This is based on the same approach as the Personis user model server (Kay, Kummerfeld, & Lauder, 2002). Essentially, this accretes all evidence about the user's knowledge of a concept and then uses a resolver to determine the value of each component as it is needed. This resolution process is very flexible and the design of the system allows for multiple resolvers with the user being able to alter the choice of resolver. This means that the user can maintain some level of control over the way that their student model is interpreted by JITT.

The student model holds the details of the currently active workflows for this user. Each workflow has a set of associated concepts. The student model represents the user's knowledge of each of the concepts for that workflow. When a workflow is initiated for a user, the concepts associated with it in the student model can be initialised to a stereotypical set of values. As in typical uses of stereotypes (Rich, 1979; Rich 1983), these operate as default assumptions which are intended to be overridden once more reliable information becomes available.

The current JITT prototype has two sources of evidence about the user's knowledge. The first applies each time the user accesses a document from the JITT page. This access causes a tell operation to the user model. This form of evidence is distinguished in the student model. Our resolvers treat this as low grade evidence for the user knowing the concept. The second form of evidence is provided directly by the user. At the end of each document, the user is invited to select a popup option indicating how well they have learned each of the concepts in that document. Recall that JITT has been designed to operate with existing documents. This means that one document may have several concepts. The user may feel that they understand some better than others. This will happen most often when there are multiple concepts explained in a document. In this case, it may be the best document available about a concept that is relevant to the user at a particular time. It may also include other content that is not relevant to the current learning goal. The user may skim the irrelevant parts. In such cases, the user can indicate their increased knowledge level for the concepts that they have learned from the document. For others, they can just do nothing (indicating no change in their knowledge) or rate their knowledge. At any time, the user can access their user model and see all the concepts that are relevant to the currently active workflows. The user can also see the system's estimate of how well they know each of these concepts. They can also check the exact evidence used to determine that value and the way its value was resolved.

TEACHING AGENT

One of the advantages of the way we use workflow technology in this just-in-time training system is that the scheduling of documents within a specific workflow process is given precisely by a workflow. That way the workflow provides the basis of the curriculum of the training process. This means that as the learner reaches a particular task in the workflow, that state defines the concepts that they need to know and, in turn, this defines the documents that the learner should read. The heart of JITT is the teaching agent.

The tasks of the teaching agent are to:

* maintain the user's state in relation to each of the workflows that are active for them;

* identify the concepts the learner needs to know for the current state in the workflow;

* find the documents which can teach the concepts relevant to the current step in the workflow;

* select the best of these for the user, based on user model details of preferences;

* customise the presentation of the links to the documents at the user interface.

Let us return to the "large scale copy" example given with the context of the exam preparation workflow (Figure 6). We suppose that the user has reached this stage in the workflow. This means that the teaching agent models this as the user's current state in that workflow. The teaching agent then identifies all the concepts that the user must understand in order to complete this task. In general, for workplace training we would expect that there would only be a small number of such concepts.

Clearly it is critical to model these concepts at the appropriate level of granularity. If for example, we take an excessively fine-grained concept model, we will introduce needless complexity with many concepts always occurring together. In practice, we believe that this problem is naturally addressed when the teacher who creates the metadata chooses the concepts to be as coarse grained as possible. At the same time, it is important to avoid modeling in terms of concepts at too coarse a grain-size. If we did this, it would be impossible to reuse concepts across workflows. This might appear to be a difficult problem. Indeed, in the general case of teaching systems, it can be (McCalla, & Greer, 1994). However, in JITT, the nature of the learning and the use of workflows to define the granularity of learning tasks makes it easier for the teacher to define concepts (and hence metadata) at the right grain size.

Once the teaching agent has identified the concepts to be learned for the "large scale copy," it searches its document collection for those that teach the "large scale copy" concept. In theory, there might be several documents, perhaps a detailed one for new staff, a shorter one for people who partially know this concept, or a document that actually directs the user to a suitable person in the organization. The design of JITT and its underlying architecture is intended to exploit existing document bases within organizations. Certainly there is a significant problem in helping employees to access these at the right time.

Given our approach, the choice of documents is limited to those we are able to find in the organisation. We can expect that this will generally constitute a small set that teaches any one concept. This means that the teacher has only to invest modest amounts of time in marking up documents and it also means that, in practice, JITT tends to have few documents to choose from. If there is a choice of documents, the teaching agent needs to select from these on the basis of the user model combined with the metadata available for each document.

At the push interface of JITT, the teaching agent presents links to the documents it recommends for the current stage of each active workflow. In our current implementation, we customise the presentation by using the architectural feature that allows multiple teaching agents.

For example, one of these only recommends documents that the user is modelled as not knowing. If the JITT model for the user indicates that the user knows all the concepts for the current stage in the workflow, it continues to show the user that stage but does not display any documents. This should be a preferred presentation strategy for users who like JITT as reminder of the overall processes they need to follow, but do not want to see document references where the user knows the concepts well. We designed this presentation for the case of the long term employee who knows the workings of the organisation well. This person still needs assistance in remembering processes that they do not often perform. The important thing is that if the organization does alter the procedures, this must be handled by defining a new concept. In our example, suppose that the "large scale copy" task requires understanding of the concept "authorization to do large scale copying." Suppose further, that after some period of time this concept is changed, perhaps because the earlier mechanism is unworkable. The organization would document this change in a new document (or a new version of an old document). The teacher in the JITT system would create a new concept to represent this new procedure. This concept would be added to the "large scale copy" task in each workflow and when a long term employee next came to this stage of the workflow, JITT would represent them as not knowing this new concept. They would be provided with the new document. Importantly, this long-term employee would see that the JITT was recommending a document--something it normally did not do.

Another teaching agent employs a different presentation strategy. It continues to show the user all documents associated with the current task. However, in the case where the user is deemed to know the concepts associated with a concept, it uses a very small font for the link to the document teaching it. We designed this as the interface that we expected most people to prefer.

Importantly, we have designed JITT so that the user can always change the teaching agent to select the presentation strategy they prefer. This functionality is available at the user profile part of the interface. As described in the interface overview, JITT also maintains two lists of documents the user has accessed. One contains those documents the user has explicitly put aside and the other has the documents the user has accessed in reverse chronological order of access. The teaching agent constructs these lists and controls the associated interface elements. The JITT architecture makes it straight-forward to create new teaching agents that provide variants on this functionality. For example, suppose there were users who preferred that neither of these lists were displayed because they preferred more screen area for the lists of active workflows and associated recommended documents. It is easy to add a new teaching agent with this variant. In organisations, this type of flexibility may be very important since even small deviations from a user's preferred interface may significantly detract from the acceptability of the whole JITT system.

The JITT teaching agents are based upon the Scrutable Adaptive Teaching System approach (Holden, & Kay, 1999; Holden, & Kay, 2001; Lum, Holden, & Kay, 2002).

RELATED WORK

Our research lies at the intersection of workflow systems, user modelling, information delivery and teaching agents. In this section, we briefly relate our work to other projects using these four dimensions.

Workflow

We use workflows to model the tasks within the whole process the user has to learn to do. Another project that has applied workflow as its driving mechanism is Flex-EL (Lin, Ho, Sadiq, & Orlowska, 2001). Their workflow was deployed in a university course environment, whereas the goal of the users in JITT was to accomplish a certain business activity. Although workflow also played an important role in the investigation in the project of Zhao, Kumar and Stohr (Zhao, Kumar, & Stohr, 2001), the goal of the workflow was to deliver information, whereas information delivery is not the goal of our workflow but only a supporting activity to empower a user to carry out the tasks. Many other projects that have not used workflow explicitly, have employed alternative ways to model tasks. For example, Paris, Linden, Colineau and Lu (2004) generated online help from task models. Their task model is coded in XML and basically contains how-to instructions. JITT is somewhat similar in that the information is associated with the intermediate steps in a process. However, their tasks are comparatively simpler and short-lived while the workflow in JITT typically operates over a more lengthy time span. To support opportunistic learning in an organization, Alem and McLean (2004) had explicit representation of the business processes. They associated the documents, electronic discussions between people, their roles and responsibilities in a group/project memory.

User modelling

One of the crucial components in our system is to model the user's knowledge of the concepts needed to do the tasks in the workflow. A lot of the other projects use concept hierarchies or a domain ontology to constrain the operation of their systems. The hyperbook introduced by Falquet and Ziswiler (2004) is comprised of a re-usable document repository that is related to a domain ontology. The concepts in the ontology are defined through a graph-based description logic language. The definition of a concept, according to a point of view, is obtained by selecting those arcs that belong to the desired point of view or to a more general point of view. Their approach is based upon points of view that correspond to a category of users or point of view adopted by a user. A document in our system in linked to the concept(s) that are associated with different activities in a business process; hence, our approach is workflow-driven instead of knowledge-driven. Our system uses a separate user model to adapt to individual needs, rather than using the same point of view representation as in the hyperbook. The knowhow concepts in the online help generation system (Paris, Linden, Colineau, & Lu, 2004) were specified in task models. Their system creates new instructions by combining existing knowledge whereas we explicitly make use of existing document. Mittal, Dixit, Maheshwari and Sung (2004) organised the inter-relationship among concepts in a relational graph. They built a rule-based system to identify and relate the learning objects to these concepts. They claimed that providing the conceptual graph to the users would enhance their understanding and their learning, and it was also more efficient to search for a desired topic.

Interface

In our system we provided push and pull interfaces to existing document bases. This contrasts with, for example, systems which operate purely from information requests initiated by the user in (Alem, & McLean, 2004), (i.e., it is a pull-type of delivery only). A similar approach to work, such as Zhao, Kumar and Stohr (2001), delivered information by email, though the users can also "pull" interesting news from an associated bulletin board. In other words, although that work involves different interface modalities, it too has forms of both the push and pull mode of information delivery available in JITT. The hyperbook by Falquet and Ziswiler (2004) provided an interface with the specified points of view so that a user can select (pull) the one that suits his/her needs.

Teaching agents

We provided multiple teaching agents, each able to support different teaching strategies and information presentation strategies. The point of view in Falquet and Ziswiler's hyperbook (Falquet, & Ziswiler, 2004) provides a similar effect. A view corresponds to a category of users or point of view adopted by a user, but it is not necessarily a teaching strategy. JITT has a different representation to store the models for different users, rather than using the same representation (point of view) as in hyperbook.

CONCLUSION

This paper has described the architecture of JITT, a Just-In-Time Training system which builds upon workflow technology to support organisational learning. In particular, the aims of JITT are to reduce the information overload faced by newcomers by scheduling the delivery of the right documents at the right time for them.

JITT can be implemented as an extra layer on an existing workflow system if one exists. However, whilst workflow management systems can be quite complex and strict in their use, JITT can make use of an arbitrary abstract workflow, defined to distinguish the sub-activities a user needs to be able to complete in order to progress through a complete activity.

We have built a prototype of the system and carried out a careful qualitative evaluation based upon think-aloud protocols and a questionnaire administered to collect affective data. A field trial that ran over several months would be needed for a meaningful assessment of the workplace utility of JITT. The evaluation was designed to be completed in a single session, making modest demands on the time of the participants. The evaluation involved ten participants (staff and students in our university) for the task of preparing a first exam (Holden, Kay, Poon, & Yacef, 2004). This demonstrated the usability of the system and generally positive user response to its value. Users commented that the system was useful, though some added that this obviously depended on the quality of the documents available. They found that having the workflow was useful as it provides an outline of the steps required and the order in which they should be done. Overall, the users thought that the system would also be useful to jog their memory as they worked through the process for later exams as well as for learning how to write the first one. JITT represents a novel combination of personalisation and tutoring controlled by a user-selectable teaching agent, with the sequencing of the teaching being defined by an organisational workflow. JITT has a novel level of user control over the personalisation, enabling the user to alter the teaching processes. It takes workflow approaches beyond the business processes, linking them to the learning processes that an employee must undertake in order to work through the various activities in their workplace. Moreover, it operates within the existing organizational framework in that it makes use of existing documents.

Acknowledgments

This one-year foundation project was funded in part by the Cooperative Research Centre Smart Internet Program through the Department of the Prime Minister and Cabinet of the Commonwealth Government of Australia. We are grateful to Bob Kummerfeld for designing the interface of JITT and implementing its earlier prototype.

References

Alem, L., & McLean, A. (2004, submitted). Supporting organisational learning with technologies for electronic documents. IJEL (Special issue on Technologies for Electronic Document for Learning).

Budzik, J., Hammond, K. J., & Birnbaum, L. (2001). Information access in context. Knowledge Based Systems, 14: 37-53.

Davis, J., Kay, J., Lin, K., Poon, J., Quigley, A., Saunders, G., & Yacef, K. (2002). Workflow based just-in-time training. In J. T. Judy Kay (Eds) Seventh Australasian Document Computing Symposium (ADCS 2002), Sydney, Australia.

Dieng, R., Corby, O., Giboin, A., Golebiowska, J., Matta, N., & Ribiere, M. (2000). Methodes et outils pour la gestion des connaissances. Paris, Dunod.

Falquet, G., & Ziswiler, J.-C. (2004, submitted). A virtual hyperbooks model to support collaborative learning. IJEL (Special issue on Technologies for Electronic Document for Learning).

Holden, S., & Kay, J. (1999). The scrutable user model and beyond. In R. Morales (Eds), Proceedings of Open, interactive, and other overt approaches to learner modelling, held in conjunction with International Conference for Artificial Intelligence in Education, Le Mans, France, 51-62.

Holden, S. & Kay, J. (2001). Document selection in SITS. In A.-M. Vercoustre & J. Kay (Eds) ADCS'2001, Australian Document Computing Symposium, 79-82.

Holden, S., Kay, J., Poon, J., & Yacef, K. (2004). JITT, Just-In-Time Training system technical report. Sydney, University of Sydney: in progress.

Hoyt, B. R. (2002). Facilitating agreement for effective knowledge management systems: The abilene paradox is thriving in the new millenium. International Journal on E-Learning, July-September: 15-20.

Kay, J. (1995). The UM toolkit for cooperative user modelling. User Modeling and User-Adapted Interaction, Kluwer 4(3): 149-196.

Kay, J. (2000). Accretion representation for scrutable student modelling. In G. Gauthier, C. Frasson, K. VanLehn (Eds) Intelligent Tutoring Systems (ITS'2000), Springer-Verlag, 514-523.

Kay, J., Kummerfeld, R. J., & Lauder, P. (2002). Personis: A server for user models. In P. De Bra, P. Brusilovsky, & R. Conejo (Eds) Adaptive Hypertext (AH'2002), Malaga, Springer Verlag, 203-212.

Lin, J., Ho, C., Sadiq, W. & Orlowska, M. (2001). On work enabled e-learning services. In (Eds) IEEE International Conference on Advanced Learning Technologies (ICELT'2001), Madison, USA.

Lum, A., Holden, S., & Kay, J. (2002). Visualisation of document and concept spaces. In J. Kay & J. Thom (Eds) ADCS2002, Australian Document Computing Symposium, Sydney, Australia.

McCalla, G. I., & Greer, J. E. (1994). Granularity-based reasoning and belief revision in student models. In J. E. Greer & G. I. McCalla (Eds) Student Modelling: The Key to Individualized Knowledge-Based Instruction. Springer-Verlag: 39-62.

Mittal, A., Dixit, S., Maheshwari, L. K., & Sung, W.-K. (2004, submitted). Enhanced understanding and retrieval of e-learning documents through relational and conceptual graphs. IJEL (Special issue on Technologies for Electronic Document for Learning).

Nonaka, I., & Takeuchi, H. (1995). The knowledge creating company: How Japanese companies create the dynamic of innovation. New York, NY, Oxford University Press.

Paradis, F., Crimmins, F., & Ozkan, N. (2003). A task oriented approach to delivery in mobile environments. In (Eds) 4th International Conference on Mobile Data Management.

Paris, C., Linden, K. V., Colineau, N., & Lu, S. (2004, submitted). Producing Instructions. IJEL (Special issue on Technologies for Electronic Document for Learning).

Rich, E. (1979). User modeling via stereotypes. Cognitive Science, 3(4): 329-354.

Rich, E. (1983). Users are individuals: Individualizing user models. International Journal of Man-Machine Studies, 18: 199-214.

Sweller, J. (1993). Some cognitive processes and their consequences for the organisation and presentation of information. Australian Journal of Psychology, 45(1): 1-8.

Zhao, J. L., Kumar, A., & Stohr, E. (2001). Workflow-centric information distribution through email. Journal of Management Information Systems, 17(3): 45-73.

S. HOLDEN, JUDY KAY, JOSIAH POON, AND KALINA YACEF

University of Sydney, Australia

[e-mail]

[e-mail]

[e-mail]

[e-mail]