LSAC 2018 Skills Analysis Study: Content Validity of the LSAT (RR 19-01)

The goal of the Law School Admission Council (LSAC) Skills Analysis Study is to identify the skills that law school faculty consider important for success in required law school courses. If certain tasks are required of all or most law school required courses, the skills involved in those tasks can be inferred to be essential to success in law school. This report provides evidence for assessing the validity of the current Law School Admission Test (LSAT), which will guide the development of new item types, item formats, and test specifications for future versions of the LSAT, including digital versions.

The skills analysis survey asked faculty who teach required law school courses to rate the importance of specific tasks to successful performance in those courses. The survey listed 70 law school tasks in 14 skill-related categories. The survey’s importance ratings were “Highly Important,” “Moderately Important,” “Somewhat Important,” or “Not Important/Not Applicable.” Faculty respondents were also asked to describe additional tasks they considered important.

A total of 489 responses were received from law school faculty, representing 87 law schools; 94% of respondents reported that the survey covered “essentially all” or “most” of the tasks involved in successful performance in required law school courses.

Importance ratings for individual tasks were consistent with the LSAT content framework. With one exception, all 15 tasks that were rated “Highly Important” by at least 75% of respondents are skills that are currently assessed on the LSAT; the exception was “Allocating available time based on priorities,” rated “Highly Important” by 78% of respondents.

Skill categories that received the lowest importance ratings are skills not present in the LSAT content framework (i.e., Using Software and Digital Devices and Quantitative Reasoning). Responses from faculty subgroups based on course level, content area specialization, tenure status, institution type (public vs. private), race/ethnicity, and gender were also compared, allowing comparison across different subgroups.