Keeping Up to Data: March 2024

Welcome to the Keeping Up to Data^SM podcast, a space in which we discuss, analyze, and contextualize trends and perspectives in the current law school admission cycle by taking a deeper dive into the most up-to-date data and making sense of the complicated world of legal education.

SUSAN KRINSKY: Welcome back to Keeping Up to Data. I’m Susan Krinsky, LSAC’s executive vice president for operations and chief of staff, with my regular update on the 2024 application cycle and an interview with a special LSAC guest who will tell us about an upcoming development.

But back to the numbers. Based on last year’s experience, we’re at the point when we are just about three-quarters of the way through the admission cycle based on volume, just under 75% of the applicant count, and just over 75% of the application count. Clearly, applications have been coming in slower this year versus last, so it may well be that we are not yet three-quarters of the way through the cycle.

As of today, applicants are 4.4% ahead of last year, with the largest percentage increases in applicants from the Mountain West and those with international addresses. We are also seeing higher than the 4.4% increase in Asian applicants, an increase of 6%; Black/African-American applicants, an increase of 6.7%; and Hispanic/Latinx applicants, an increase of 8.9%. In all, over 45% of this year’s applicants identify as persons of color; over 57% identify as female; 1.3% as gender diverse, which itself is a 3.7% increase over last year; and just over one-quarter as first-generation college students.

As for application volume, applicants have submitted almost 326,000 applications just over last year’s total at this time — a significant recovery from several months ago, when application volume was down 9%, likely the result of some schools’ applications becoming available a little later than usual, as well as a number of new essay prompts from many schools. The largest increases in applications are going to schools in the Northwest and the Great Lakes regions. Over half of law schools, 105, are showing increases in applications today; 81 are seeing decreases; and 10 are showing no change from last year.

As we look at the weekly new application counts, in most weeks, we are seeing the largest volume of new applications for the past five years, with the exception of the unusual 2021 cycle. For the first six LSAT administrations of this cycle — August, September, October, November, January, and February — we saw over 10% more test takers than for the same six tests in last year’s cycle, and an increase in first-time test takers, too. As I record this podcast, we are very close to February score release and the deadline to register for April’s administration, which is itself running almost 20% ahead of last year with respect to the number of registrants. This could be related to the change to the format of the LSAT, which will go into effect in August, and we hope to have more information about that in the coming weeks and months. As always, you can find the latest application trends and numbers on our website, which is updated daily, 365 days a year.

Now, I am very happy to welcome Dan Shaw, director of assessment development at LSAC — which means that he oversees, well, the development of the content of the LSAT, both the multiple-choice test and the writing test. I’ve asked Dan to join us today to talk about an upcoming change to the portion of the test known as LSAT Writing. Dan, welcome to Keeping Up to Data.

DAN SHAW: Thank you, Susan. I’m glad to join you today.

SUSAN: First, I wonder if you could describe briefly what you do at LSAC.

DAN: You bet. Well, as you said, I’m the director of assessment development, which means I oversee the test development group that writes the questions and puts the test together. So, I oversee the quality of items and the process of pretesting items to make sure they’re fair and valid and sound, and revising items to make sure we can put together the best test possible.

SUSAN: Lately, though, you’ve been devoting a lot of time to one project in particular. What is that project?

DAN: Yes, we’re taking a new look at LSAT Writing.

SUSAN: And why would that be?

DAN: Well, LSAC is listening to our law school partners and the assessment industry by innovating to create a new writing task that’s closer to what the legal practice needs. We’ve come up with a new approach that’s designed to be more reflective of the kinds of diverse writing purposes and rhetorical skills that are needed for success in today’s law school and the practice of law. But the test still does not require any specialized skills, knowledge, or experience with legal concepts. Frankly, there’s been a renewed attention to collecting a high-quality writing sample under monitored, proctored test conditions, given emerging concerns about the influence of AI on candidate writing and the use of ChatGPT, potentially, to create writing samples for admissions portfolios. And not that that could be used in the current LSAT Writing, but we just want to make sure we’re giving our law school partners the best possible information about test takers’ writing ability.

SUSAN: How is writing currently assessed on the LSAT? What do the current LSAT Writing prompts look like?

DAN: Good question. So, the current, decision-based prompt format was introduced in 1982, and it’s a focused logical reasoning task that follows preordained lines of reasoning. So, test takers are presented with a couple of proposed solutions to a complex problem, along with a couple of defined criteria for deciding between those two proposed solutions. But this test format doesn’t generalize well to the diverse writing skills and purposes that characterize writing in today’s law schools and the legal profession. Our industry’s understanding of argumentative writing assessment has evolved significantly since that original wave of writing assessment prompt formats, especially in the last 10 to 15 years, with the 2011 NAEP Writing Assessment Framework — I developed it when I was at ACT and the writing assessment group there — in addition to many other key national conversations among writing teachers and scholars of composition and rhetoric about the appropriate learning standards and outcomes for writing instruction.

So, our new argumentative writing task is designed to give test takers a clearer, more authentic writing purpose than the former, decision-based logical reasoning task. We found that when test takers are able to construct an original thesis and defend it based on their own judgment and analytical evaluation, rather than following those preordained lines of reasoning, we can better assess a broader and more complex range of decision-making skills that writers engage in.

SUSAN: So, given that, can you tell us a little bit more about how the new writing prompts will be different from the current ones?

DAN: The new argumentative writing task is still designed to measure the ability to write an argumentative essay, but the updated format presents test takers with a better opportunity to demonstrate that full range of critical thinking and rhetorical skills. So, the new approach is designed to assess a test taker’s ability to construct a cogent argument based on a variety of evidentiary sources. They’re presented with a debatable issue, which is identified in a key question, along with different perspectives that provide additional context for the issue by introducing various claims that have been made within that debate. These perspectives, each of which is about three to four sentences, they’re each representative of a system of beliefs or values. And then together, these perspectives illustrate competing ideologies and arguments and dialogue around this particular issue.

So, test takers will then write an argumentative essay in which they must take a position within this ongoing conversation while addressing at least one of the arguments and ideas presented in the other perspectives. And that gives the test taker a little more freedom to build an argument that suits their rhetorical strengths. And so, we’re not only enabling students to have a more authentic voice in their argument, but we’re also better positioned to evaluate the writer’s ability to employ various rhetorical techniques, evidentiary strategies, and other important aspects of argumentative writing.

It also bears mentioning there’ll be a little more time to complete this task, given the additional reading load required. So, we’ll add 15 minutes to the LSAT Writing test experience, during which students can prepare to write their essay using guided pre-writing analysis questions. These questions are designed to help students analyze the various perspectives and generate productive ideas for their essay. So, test takers will now have a total of 50 minutes to complete the updated LSAT Writing prompt.

SUSAN: That’s an exciting development. How can prospective test takers learn more about these, maybe see an example, and practice?

DAN: So, the new LSAT argumentative writing will become part of the test starting with the August 2024 LSAT administration. To give test takers the opportunity to prepare, we’ll be publishing sample LSAT prompts using this August 2024 format in the coming weeks. So, test takers will have ample time to become familiar with the new approach and take practice LSAT Writing sessions in the official LSAT Writing environment. So, they’ll be able to practice in LawHub with a sample prompt in early March, and we plan to add more opportunities to practice with additional prompts in the coming months.

SUSAN: Now, currently, LSAT Writing responses are simply provided to the law schools without any scoring or grading. Will that change?

DAN: Not this year, meaning not on these eight tests between August 2024 and June 2025. So, for this 2024-25 testing cycle, nothing will change in terms of how LSAT Writing will be administered. LSAT Writing will continue to be offered in connection with each LSAT administration, and test takers will continue to complete their LSAT Writing sample in the same online, proctored format. For each administration, LSAT Writing opens eight days prior to the primary test dates, so the test takers can plan and complete it at any time. However, test takers will not be able to receive their score, just as now, will not be able to receive their score on the multiple-choice test, nor will scores be sent to law schools, until a completed and approved writing sample is on file.

SUSAN: Which leads me to ask whether anything will change after June of 2025 — that is, after the upcoming testing year.

DAN: Good question. So, let me back up and talk a little bit about what we’ll do during this coming year and then speak to the potential to possibly provide scores starting in the tests that are administered in August 2025 and beyond.

So, we do plan to use this first year of essays collected during the unscored administrations to evaluate the efficacy, reliability, the validity of a potential scoring framework based on existing research about writing competencies and best practices in graduate admissions assessment. I led writing assessment, development and scoring at ACT for 15 years, including the GMAT graduate admissions test, MCAT graduate admissions, and, of course, the ACT writing. So, we’re well versed in the types of research that are needed to assess the validity and reliability of providing scores for the new assessment.

And we should be very clear that this is really an effort to gather information about the validity and impact of writing scores on test takers, especially our different demographic subgroups. A key reason to wait until the 2025-26 testing year is to provide plenty of time to collect such data and make sure the test is achieving its design goals of providing a fair and accurate writing skills assessment.

SUSAN: So, just to review the bidding here: During the upcoming testing year, that is, starting in August ’24 and ending in June of 2025, the writing prompt will be a little different than it is currently, but it will not be scored.

DAN: Yes, that’s exactly right. And we hope this new task provides richer information and a more comprehensive assessment of our test takers’ writing skills.

SUSAN: Thank you, Dan, for a very informative conversation.

DAN: Yeah, it’s a pleasure to be here and to get this exciting new information out. We’re really excited to see how this can benefit our test-taking community.

SUSAN: To our listeners, thank you for joining us at Keeping Up to Data. We look forward to your joining our next episode, when we will continue to take a close look at the data from the current admission cycle. Until next time, stay well.

Thank you for joining us. Keeping Up to Data^SM is a production of LSAC. If you want to learn more about the current law school admission cycle and the latest trends and news, visit us at LSAC.org.

Back to Keeping Up to Data