|
The following is the abstract of the
paper presented at the Loughborough CAA conference on the
8th July 2003. You can also request
the full paper.
Computer
Based Testing of Medical Knowledge.
Tom
Mitchell1, Nicola Aldridge1 ,Walter Williamson2, Peter Broomhead3
1.
Intelligent Assessment Technologies Ltd. www.IntelligentAssessment.com
2.
Faculty of Medicine, Dentistry and Nursing, University of
Dundee.
3. Dept of Systems Engineering, Brunel University.
Abstract
The Medical
School at the University of Dundee offers a high quality teaching
programme, rated Excellent by the SHEFC Quality Assessors.
The assessment of an outcome based curriculum, and the need
to provide rapid student feedback, represents an ongoing challenge
for the School. The recent introduction of a "progress
test" has only added to these challenges. Computerising
the progress test offers obvious advantages to Dundee, particularly
in terms of reducing the marking burden at a time of intense
work with summative assessment, and in providing rapid feedback
to students. However the progress test itself requires marking
of free-text responses. Objective testing is not an acceptable
alternative.
This paper
details the development and roll-out of a computerised system
for delivering and marking the progress test in the Medical
School at Dundee. The system employs an innovative natural
language based assessment engine. The assessment engine has
been developed to perform robust computerised marking of free-text
responses to open-ended items.
The progress
test consists of 270 short-answer free-text response items.
Item presentation is randomised, such that the probability
of any two students receiving the items in the same order
is negligible. Students are allowed up to three hours to complete
the test.
Computerised
marking of student responses is carried out in batch mode,
once the test is complete. The system provides a simple interface
to enable administrators to initiate computerised marking
of student responses, and to provide information on the progress
of marking. The system supports moderation of the marks awarded
by computer. Results of the tests are exported in a flat file
format for subsequent processing and reporting.
The paper
details the experiences gained in testing over 450 medical
students so far at Dundee in 2003. A comparison with the previous
years' paper-based testing approach is provided.
Key
words : Computer Assisted Assessment, Free-Text, Computerised
Marking, Medical.
|
