Right menu

APOSDLE - work, learn, collaborate

Formative Evaluation Guidelines and Plan

The contribution of this deliverable is twofold: First, it presents guidelines for development partners. These guidelines provide the basis for formative usability evaluation during the first iterative development phase until month 12. The target audience for these guidelines are development partners, especially those that implement tools with a user interface (e.g. the APOSDLE toolbar) as the guidelines focus on usability aspects that are relevant when users interact with a tool. Other aspects that are also important, such as testing of precision and recall of the query results, corpora selection or model testing, but not directly related to usability issues need to be considered during development but are not covered in the guidelines.

Secondly, this deliverable provides the framework for formative and summative evaluation tasks throughout the project, in the form of an evaluation matrix. The framework is provided for all of the evaluation tasks in the project to provide the context for the more specific formative evaluation tasks that will take place in months 13 15.

This first formative evaluation phase is in two main parts. Firstly we will test for core usability issues, assuring that users are able to complete APOSDLE core tasks as described in the use cases. The focus here will be on evaluating overall effectiveness of the interaction, overall efficiency of the interaction, overall user satisfaction and checking if usability standards are met. Secondly we will evaluate the prototype for scenario compliance. Here, the focus is on checking for requirements and specification compliance, observing whether features of the prototype are compliant with requirements specified in the DVI.2 deliverable. The outcome of this document is a simple framework for more detailed design of the first formative evaluation phase.