Right menu

APOSDLE - work, learn, collaborate

Files

Activity Report Year 4 (Final)

GPL Copyright Info

COPYING.rtf 90.72 kB

APOSDLE OpenSource License Information

APOSDLE OpenSource Installation Manual

Activity Report Year 3 (Executive Summary)

Publishable Executive Summary of all Project Activities from March 2008 - February 2009

Conceptual Architecture including Component Evaluation

This deliverable gives an overview of the overall APOSDLE design approach to work-integrated learning support. For an in-depth discussion of the work-integrated learning paradigm related to informal learning theories please refer to the integrated Deliverable D II.8 & D III.5 APOSDLE Perspective on Self-Directed Work-integrated Learning. This deliverable focuses on the approach to design support mechanisms for work-integrated learning as prototypically implemented within APOSDLE Prototype 3 and on empirically investigating several features and support mechanisms included.
Specifically this deliverable addresses three major APOSDLE design concepts: models, user?s work context and user profiles; integrated work, learn, and cooperation support; and the knowledge artefact lifecycle. We apply a battery of semantic and scruffy approaches to each of these concepts in order to design learning support flexible enough to cope with challenges of work-integrated learning. By doing so, we draw from approaches to context-aware systems, user modelling, adaptive system design, learning and instruction methods, recommendation systems, retrieval and clustering algorithms as summarized in the following:

  • APOSDLE models and their relationships

Several models are needed to enable APOSDLE to deliver the needed support for self-directed work-integrated learning. This are respectively the domain model (refers to the work-learn domain concepts), the task model (refers to the tasks that are performed in the work-learn domain) and the learning goal model (refers to generic learning goals that can be linked to tasks). Each of the models is described in more detail and methods and techniques for modelling are explained. Also attention is paid to the crucial role these models play in the APOSDLE approach.

  • User’s work context and user profiles

APOSDLE relies on the automatic identification of a user?s current work task and relevant domain concept. This information is utilized to on the one hand trigger a number of work, learn and cooperation support mechanisms (see below). On the other hand this information feeds into a user profile designed as a layered overlay of the domain model. User profile services then allow us to infer the user?s knowledge levels, identify learning goals and compute prerequisite relations, etc. In addition the notion of a knowledge indicating event is explored as a means for unobtrusive profile adjustment and a proxy for outcome assessment.

  • Integrated work, learn and cooperation support

Based on a user?s current work task and/or topic and on her knowledge levels APOSDLE provides a number of recommendation and support mechanisms for learning and cooperation. These support mechanisms utilize on the one hand the underlying semantic models (domain, task, learning goal) and on the other hand advanced retrieval and clustering algorithms based on text and multi-media data. Support is embedded in instructional guidance that intends to help in performing key learning functions in self-directed learning. Cooperation is facilitated by recommendations of knowledgeable persons, context preservation during cooperation and providing communication channels. Outcomes of cooperation activities can be stored and accessed, thus contributing to knowledge creation and knowledge distribution.

  • Knowledge artefact lifecycle

Since APOSDLE re-uses organizational content (text as well as multi-media content) for supporting work-integrated learning this content needs to be turned into knowledge artefacts. We employ advanced retrieval and clustering algorithms based on text and multi-media data in order to do much of this automatically. In addition, the user is empowered to provide feedback and create new knowledge artefacts which over time improve the automatic mechanisms.

Several of the designs were empirically investigated in nine component evaluation studies, covering automated user work context detection, methods for assessing user levels and the validity of these levels, the acceptability of the privacy concept, the effect of manually created learning paths, the difference between several ways to show the domain in terms of work time speeding up, the usability and acceptability of scripted communication, the added value of multimedia and the relative performance of similarity measures for retrieval of resources. Together these studies provided valuable insights into the effectiveness and acceptability of these designs.

Summative Evaluation Report

This document describes the summative evaluation of APOSDLE. Given the scope of APOSDLE it is very important that a comprehensive and well planned evaluation is carried out.
This summative evaluation is directed to the main goals of the project and the application partners. Based on previous work on the i*model, a set of goals is identified that are specific enough to make some kind of measurement possible. These goals are prioritized by the application partners, leading to four goals that are important for all application partners and seven goals that are important for a subset of the application partners. In the summative evaluation achievement of these goals is the central focus.
In addition, a more precise definition is given of what the “APOSDLE product”, or maybe better, the “APOSDLE solution” actually is. It would be a mistake to see only the software embedded in the final prototype to be deployed at a workplace as the “solution”. In the project the “APOSDLE solution” is seen as consisting of the following aspects

  • Modelling the domain and the work processes
  • Annotating documents and other sources of information available in the company repository
  • Training the prospective users of the APOSDLE system
  • Using the APOSDLE system at the workplace

This means that a comprehensive summative evaluation of the APOSDLE solution requires a summative evaluation of each of these aspects taking the goals into consideration. This is even more mandatory as the aspects depend on each other. If the domain modelling has not been done correctly, the annotation will fall short of what is needed; if the annotation is done badly retrieval of relevant information will be unsatisfactory; if the users are not well trained, their use of the APOSDLE system will be sub-optimal.
The summative evaluation was carried out at the Application Partner sites. The workplace evaluation took about three months and involved nineteen persons. A multi-method data collection approach was followed using a questionnaire, interviews, log data, user diaries kept while working with APOSDLE and site visits. This allowed for triangulation of results.
The conclusion of the summative evaluation is that the APOSDLE solution is very useful for learners, especially those in a highly-specialized, scientific and relatively new domain like EADS's Electromagnetism Simulation domain. In those circumstances, APOSDLE delivered an effective work-based learning solution that enabled relatively inexperienced knowledge workers to improve their knowledge in various ways. However, it proved less effective for relatively inexperienced knowledge workers in broad and well-established customer-driven domains where knowledge was shared to a large extent in person and was available in alternative company repositories, employees' computers and the Internet.
Experts in all three domains perceived APOSDLE as a tool aimed mainly at learners or novices in the domain. They valued its benefit as a training tool but compared to learners regarded it a tool they did not need to do their work. They acknowledged their role was to contribute high quality learning material (e.g., Snippets, documents and Learning Paths) but commented such responsibility may not be realistic given their busy schedule and, in some cases, time spent out of the office.
Learning
Overall APOSDLE supported the acquisition of new knowledge by the users by making them aware of learning material, learning opportunities and by providing relevant material. However, this conclusion is not based on more formal knowledge gaining tests, which are not opportune to administer in a working context, but is derived from self reports. Concerning specific support for learning it must be concluded that this is only of limited value in a working context. Apart from learning goals and learning paths, most other supporting facilities, like hints and notes, were rarely used. This shows that supportive measures derived from instructional theories which are focusing on formal learning contexts, are not very relevant for learning at work. This leaves open the question what kind of support, or maybe no more specific support at all, can contribute more to learning at work than providing good material and some help in planning learning and thinking about learning goals.
Working
APOSDLE supported working mainly by making knowledge workers aware of material which helped them complete a task. Such material was discovered as a result of keyword search, browsing the list of topics or tasks or as a result of a topic being detected by the context detection component. In EADS especially, it was reported on numerous occasions in the user diary that explicit and implicit learning material enabled knowledge workers to gain useful insight, improve their knowledge and complete a task they were working on. This type of support was less evident in the two consultancies (ISN and CCI) were knowledge workers relied primarily on each other to support their work.
Collaborating
The workplace evolution of the collaboration component of APOSDLE was limited due to technical problems, which prevented it from being used throughout most of the evaluation period. It did function during the last two weeks of the evaluation and end users were able to test it and form a positive impression, though there no evidence it was used in a real-life scenario either to fill an explicit learning need or complete a task. Apart from any technical problems, it is worth noting that in ISN and CCI end users shared an office space or worked in close proximity. They were used to interacting with each other in person throughout most of the day as part of their work. Under such circumstances one must consider the possibility there was no real need for people working in a small team to structure a collaboration request with a colleague during the evaluation.
Finally, concerning challenges APOSDLE addressed it is concluded that the domain independent approach partly worked. Relying on existing material instead of tailor made learning material provided to be effective and is probably also cost efficient. Crucial for this is having good modelling tools, experienced modellers and high quality annotations of snippets. Providing domain independent learner support was less successful as most of the provided features were hardly used. Probably the gap between the generalized and rather abstract nature of these features and the very specific context of work is too large. It is proposed to put more effort in good annotations which probably brings more benefits than trying to refine or enhance specific learner support.

Dissemination and Standardisation Report IV

This document gives an overview of the activities the APOSDLE project consortium has undertaken in the four project years in the areas of standardization and dissemination as part of work package 7. The focus of this document is a summative one on the more scientific activities. This means, the document is organized as a body of reference for the standardization and scientific publications. In contrast to that, industry-oriented events of the forth year of APOSDLE and the respective activities are described in the final APOSDLE deliverable on exploitation. The former deliverables on exploitation, dissemination and standardization contained also descriptions of scientific events. We restrict our report on summarizing the contributions in standardization and scientific publication.
WP7 promoted the adoption of standards and the awareness of standardisation activities in all the technological aspects relevant to APOSDLE, and to encourage the engagement with standardisation efforts, by contributing to ongoing standardisation groups, and disseminating specifications. The standardisation activity performed in WP7 during the four years of the project has focused mainly on adoption of standards and awareness of standardisation activities across several technological areas of the project in order to ensure the usage of the latest updates and advancements of the standardisation bodies into the design and realisation of the APOSDLE system. During the project, members of the APOSDLE team have also actively contributed to standards and standardisation groups of interest for the project, like in the case of the IEEE LTSC Study Group on Competencies, or the W3C Video on the web standardisation activities. This deliverable reports a summary of the main standardisation activities within the APOSDLE project. These activities were also documented in the APOSDLE Standardisation Wiki, which was used to monitor and report the progress in important standardisation activities and general technological developments of interest to the project.
A further objective of WP 7 was the scientific dissemination of the APOSDLE result, which we summarize here in a table view. APOSDLE succeeded in cross-consortium, integrative work on high-class peer-reviewed publications. Overall sixty scientific dissemination activities i.e. accepted, peer-reviewed publications were reported throughout the project with over a third of these targeting business and industry groups. The impact of these publications was also higher with several more prestigious conference appearances (e.g. CHI, AH, ISWC, EKAW) and a couple of Journal publications.

Update to APOSDLE Approach to Self-directed Work-integrated Learning

This deliverable gives a comprehensive overview of the APOSDLE approach to Self-directed Work-integrated learning. In several deliverables written over the first two years of the project this topic was addressed in a distributed manner. Since this made it difficult for people not part of the project to understand our approach and background we made the effort to provide here a fully integrated perspective on work-integrated learning and how it can be supported. Specifically, this deliverable is the result of the tight integration of two previously separate deliverables: D2.8 “Review of general applicability of learning support over domains” (lead UT, now mainly comprising Chapter 5) and D3.5 “Contextualized Cooperation Concept Version 2” (lead FHG, now mainly comprising Chapter 6).
This updated version is a further and final revision of the integrated deliverable (D2.8 & D3.5). It now also includes the final concept of contextualized cooperation led by FHG, which was integrated through the addition of section 6.2.4 “Social Aspects of Cooperation”. This section considers social aspects and behaviour of knowledge workers and their influence on knowledge sharing in general and cooperation in particular.
The other chapters were left unchanged and describe our common understanding on knowledge work, work-integrated learning, and the overall approach to support them (lead KC).
It starts with outlining how the project conceptualizes „knowledge work?. Following the argumentation of Kelloway & Barling we see knowledge work primarily as an individual, discretionary activity. Thus, knowledge work is characterized as being a dimension of work, not an all or nothing property. To study it, one should focus on the use of knowledge in the workplace and not on properties of individuals or jobs. Next the notion of workplace learning is taken up, more precisely defined as work-integrated learning. It is seen as being truly integrated in current work processes and practices, making use of existing resources with the main goal of improving work performance. In terms of Eraut & Hirsh: learning activities located within work or learning practices. Work-integrated learning is mostly self-directed, meaning that the learner is empowered to take responsibility for their own learning activities, to choose means that allow them to carry out various learning activities and to choose access to various learning resources. In this sense the learner is her own instructor. During this self-directed learning, several learning functions must be carried out by the learner: preparatory functions, executive functions and closing functions. In addition, during work co-workers and other knowledgeable persons are very important sources for learning.
Our design of work-integrated learning support is firmly grounded in the above mentioned theories which together define the design support space for APOSDLE: supporting the learning functions in the context of using (a) written resources and (b) human resources. These resources have to be utilized to support learning activities within knowledge work (acquisition, creation, transfer, application of knowledge) within the computational environment and by taking mediating (ability, motivation, opportunity) and predicting (leadership, job design, social interactions, organizational culture) context factors into account. This design work is made even more challenging given the fact that APOSDLE is intended to be a generic system, not tailor made for a specific task or process and a specific knowledge domain. More in particular this means that APOSDLE has to deal with real time learning, a real computational environment and content already available from the repositories in an organisation.
For using written (or material) resources we especially draw from work by Simons and outline possible support solutions. At the basis lies the notion of a generic information or learning need (or goal) that drives self-directed learning and is linked to work tasks. This generic learning need is linked to available written material trough the notion of the role a piece of information can play when people want to learn: its material use. For example, a definition, an explanation, a procedure. Based on these two concepts hints are provided that can guide the learner. Furthermore, to support extended learning, learning paths can be created that provide a route through the knowledge domain, taking into account prerequisite knowledge.

For using human resources (knowledgeable persons) a cooperation model is developed that describes the phases of a cooperation process: pre-cooperation, cooperation and post-cooperation. Templates containing the essential features of the information that must be available and transmitted during cooperation are described in detail. Moreover, attention is also paid to how learning that takes place in a cooperation process can be made available to the wider organisation by using the notion of cooperation spaces. Finally, actual cooperation is supported by cooperation scripts that guide the more low level communication between the learner and a knowledgeable person. These scripts are needed to make this communication more fruitful. Research has shown that without these scripts, knowledge exchanges tend to deteriorate, or will not take place at all, although scripting might have some disadvantages like disturbing and didactising collaborative interactions (as signalled by Dillenbourg).
Finally attention is paid to how an organisation can find out if APOSDLE could be a suitable solution for workplace learning. A questionnaire and associated decision guidelines are provided.

Final System Architecture for 3rd APOSDLE Prototypes

This document is the final Software Architecture Document for the third prototypes of the APOSDLE (Advanced Process Oriented Self-Directed Learning Environment) System. Based on the requirements documented in Deliverable D6.3 „Use Cases & Application Requirements 2 (Second Prototype)' (APOSDLE consortium 2007a), Deliverable D6.4 „APOSDLE Use Scenarios and Requirements for 3rd Prototype' (APOSDLE consortium 2008a) and on the findings of the formative evaluation of the second prototype of the APOSDLE System documented in Deliverable D6.7 „Formative Evaluation Report of 2nd Prototypes? (APOSDLE consortium 2008b) this document describes the software architecture of APOSDLE by following Standard 1471-2000 of the Institute of Electrical and Electronics Engineers, the IEEE Recommended Practice for Architectural Description of Software-Intensive Systems (Institute of Electrical and Electronics Engineers 2000). The IEEE 1471-2000 standard suggests the use of views for documenting different aspects of a software intensive system, without recommending special views. We base the views model used in this Software Architecture Document in the 4+1 Views Model of Kruchten (1995) (2003).