Test Assembly

Robust Text Similarity and Its Applications for the LSAT (RR 13-04)

by Dmitry I. Belov, Law School Admission Council; Bernard P. Veldkamp, University of Twente, Enschede, the Netherlands; and David A. Kary, Law School Admission Council

Text similarity measurement provides a rich source of information and is increasingly being used in the development of new educational and psychological applications. However, due to the high-stakes nature of educational and psychological testing, it is imperative that a text similarity measure be stable (or robust) to avoid uncertainty in the data. The present research was sparked by this requirement. First, multiple sources of uncertainty that may affect the computation of semantic similarity between two texts are enumerated. Second, a method for achieving the requirement of a robust text similarity measure is proposed and then evaluated by applying it to data from the Law School Admission Test (LSAT). While further evaluation of the proposed method is warranted, the preliminary results were promising.

Back to report gallery

Request the full report

Additional reports in this collection

Some Practical Issues in Computerized Adaptive Testing...

Many standardized tests are now administered via computer rather than paper-and-pencil format. The computer-based delivery mode brings with it certain advantages. One advantage is the ability to adapt the difficulty level of the test to the ability level of the test taker in what has been termed computerized adaptive testing, or CAT. A second advantage is the ability to record not only the test taker’s response to each item (i.e., question), but also the amount of time the test taker spends considering and answering each item.

Read More

Review of Modern Methods for Automated Test Assembly...

Automated methods have been developed for assembling test forms, evaluating a pool of test questions (i.e., items) to determine the number of test form assemblies it can support, and designing an item pool that can most efficiently support the test form assembly process. Automated methods have greatly maintained and improved such activities, all of which are essential to the support of every testing program. This report reviews the major approaches that have been applied in the development of these methods.

Read More