Test Process Data

Bayesian Marginal Joint Models for Responses and Process Data (PR 18-03)

by Konrad Klotzke and Jean-Paul Fox, University of Twente, Enschede, the Netherlands

With computerized testing, it is possible to record not only the responses of test takers to test questions but also other details about the test taker’s activity, such as the amount of time spent responding to each question. These details comprise a new type of data called process data. This report proposes a new approach to modeling responses, response times, and other process data: Test-taker data that naturally belong together are grouped in a cross-classification structure. Five examples of models applying this approach are illustrated. This new approach was evaluated in a simulation study and compared to the classical approach. A model that allows for variations in test-taker speed and accuracy produced satisfactory results. This model also lent insight into changes in the speed–accuracy trade-off over the course of the test.

Back to report gallery

Request the full report

Additional reports in this collection

The Bayesian Covariance Structure Model for Testlets...

Standard item response theory (IRT) models have been extended with testlet effects to account for the nesting of items; these are well known as (Bayesian) testlet models or random effect models for testlets. The testlet modeling framework has several disadvantages. A sufficient number of testlet items are needed to estimate testlet effects, and a sufficient number of individuals are needed to estimate testlet variance. The prior for the testlet variance parameter can only represent a positive association among testlet items.

Read More

Modeling Multilevel Dependence Structures for Responses...

Bayesian covariance structure modeling (BCSM) offers a flexible approach to modeling complex interdependences that arise when gathering test-taker data through computerized testing. In addition to the scored responses, process data such as response times or action patterns are obtained. Data from different sources may be cross-correlated; furthermore, within each data source, blocks of correlated observations may form testlet structures. In previous reports, BCSM was limited to the assumption that all test takers are part of the same group.

Read More