
Intelligent Assessment Technologies' proprietary text marking engine employs state-of-the-art Natural Language Processing (NLP) technology to provide robust computerised marking of short-answer free-text questions. The technology looks for specific content in the answer, without penalising students unduly for poor spelling, grammar or syntax - similar to the way in which human-markers work. IAT's marking engine is now available as part of ExamOnline.
IAT have developed FreeText Author, an authoring tool which makes configuring the marking rules for the marking engine easy. FreeText Author is a sophisticated desktop application which enables you to add machine marked short-answer questions to your ExamOnline tests, examinations, and assessments. With a little practice, you can routinely include automatically marked short-answer free-text questions in your e-assessments, all using a point-and-click interface. To see the main constituent parts of the user interface, click on the image on the above.
With FreeText Author, the mark scheme panel has been designed to resemble a traditional paper-based mark scheme layout, with lists of acceptable and unacceptable top-level mark scheme answers. You can add or delete mark scheme answers using the mark scheme answer toolbar.
In the model answer list you can enter a number of model answers for each top-level mark scheme answer. These model answers represent specific acceptable phrasings or wordings for the corresponding mark scheme answer, and are the basis from which the templates are generated. Template generation is automated - you just have to specify which are the keywords for each model answer.
The synonym editor allows you to add synonyms for the model answer keywords. You can enter your own synonyms, and you can select from lists of synonyms suggested by the authoring tool. The authoring tool provides three list of suggestions:
At any time you can computer mark sample responses using the mark scheme you have created. The response list will be updated in real time as the responses are marked. Responses which have been matched will be shown in green with a tick icon, responses which have not been matched will be shown in red with a cross icon, as shown here.
FreeText Author also includes a marking wizard to help you refine the mark scheme, providing information on why responses are not being matched. See here for screenshots.
When you have finished refining your mark scheme, you can export it to ExamOnline, ready for use.