Keeping Up to Data: October 2021

Welcome to the Keeping Up to Data podcast, a new space in which we discuss, analyze and contextualize trends and perspectives in the current law school admission cycle by taking a deeper dive into the most up-to-date data and making sense of the complicated world of legal education.

SUSAN KRINSKY: Welcome to LSAC's Keeping Up to Data podcast, where each month we will share perspectives and trends related to law school admission and cover other topics of interest. I'm Susan Krinsky, a former admission dean and former LSAC board chair — and now chief operating officer and chief of staff at LSAC. I started the Keeping Up to Data podcast a few years ago, but put it on pause at the start of the COVID-19 pandemic. I am pleased to bring the podcast back to life in a new format and to welcome LSAC's president and CEO, Kellye Testy.

KELLYE Y. TESTY: Thank you so much, Susan. It is great to be with you today, and I am so glad we're reinstating this podcast. It will be a fun way to connect with our community and to advance our mission together.

SUSAN: And that's probably a good place to start. Kellye. LSAC's mission has always been grounded in a deep commitment to opening up access and encouraging diverse and talented individuals to study law. We recently expanded our mission to focus not only on admission, but to support individuals from pre-law through practice. Why is this so important?

KELLYE: Well, Susan, I'll start by telling you, as I know you know, that when I joined LSAC about four years ago, I really didn't realize all that LSAC did for schools and for candidates, even though I'd been a law school dean and professor for a long time. So, part of what our expanded mission is about is really capturing not only what we have traditionally done, but also expanding somewhat our reach. And it's really important for us to work across that whole journey from pre-law to practice because until we do so, we're not going to make the change we want to make for legal education.

We need to support the candidates every step of the way: access, in law school, at licensure, and on into practice. We want to make sure that we're reaching our goals in helping diversify the profession and making society better. And the best way to do that is to really support our candidates along that whole "pre-law to practice" journey. We do that in so many ways, including the wonderful program we ran this summer, called Law School Unmasked, that really helped open up what it means to go to law school, how to get there, and how to do well once you're there.

Susan, I want to turn to you for a minute. You wear many hats at LSAC, and managing the LSAT is one of those, a really big part of what you do. I want to ask you to look ahead to the upcoming admission cycle and talk about what listeners ought to know, both about it and the cycle we just came through. But before you get into some of that detail, what was it like to lead the LSAT during COVID? What are some of the things that surprised you and what are you most proud of when you think about that period?

SUSAN: The most important thing was this: We knew we needed to do everything possible to allow prospective students to continue their journeys to become lawyers, and to make it possible for law schools to continue their admission processes. So, as it became clear that we could not endanger anyone's health by continuing to deliver the LSAT in person, we were able in a matter of weeks to deliver the LSAT online, allowing test takers to use their own devices in their own homes, overseen by live remote proctors.

In the end, we administered 170,000 tests between May of 2020 and June of 2021 — 10 administrations. In addition, however, because we were well aware that not every test taker has access to a suitable computer (or to a suitable space in which to take the test with reliable internet), we provided loaner devices free of charge and reimbursed test takers who needed to take the test at a hotel. Applications to the class entering in 2020, as you know, had been lagging, but that year ended up ahead of the year before.

And this year, as we all know, we saw a significant increase in applicants: 12.3 percent more individuals applied to law school, and they submitted a total of 26.1 percent more applications. I am so proud of the way we all came together, LSAC and law schools, to make this happen. Kellye, you spend a lot of time with law school deans and other leaders in the industry. What did you observe? What surprised or inspired you?

KELLYE: There was really so much that inspired me as I saw how legal education came together in this period, Susan. I think so much about why law matters to all of us. It's a pathway for society to thrive, and getting a law degree is really getting a license to help other people and to help build our society in a way where everyone can truly thrive. So, when I saw the number of people expressing their interest in becoming part of law and justice, in taking steps to address social issues that matter so much to all of us, that was truly inspiring.

And furthermore, I was deeply inspired by the way our law schools responded. You know, Susan, in a matter of moments, everyone went to online education in a way that made sure that the students could continue their studies and continue their pathways to becoming empowered to help people. And so, I was so proud of the law school deans and faculty and students and the way they came together with us and to make sure that everything just kept moving in a wonderful way so that COVID could not limit what we could accomplish for justice.

I loved convening the groups of candidates and deans and admission leaders, pre-law advisers. We did that in so many ways to just keep information flowing and to keep all of those journeys rolling right on time. And Susan, I know that when we talk about law school journeys, one of the things that LSAC does is sponsor national Forums each year all across the country, and you had to move those online, too, during COVID. And now we're offering, as I understand it, a mix of digital and in-person Forums this year. How do you think that's going, Susan?

SUSAN: Yes, that's right. We hosted our first digital Forum on September 10th, and close to 4,500 individuals interested in law school attended. They were able to meet with representatives from the 212 law schools present. We have two more digital Forums scheduled, one on December 4 and one on February 5. And we have five in-person Forums scheduled between September 25 and November 13 in Chicago, San Francisco, Atlanta, New York, and Los Angeles.

I think adding the digital forums to the in-person schedule is such a good thing, because it enables everyone to attend even if you can't travel to an in-person event. These events are such a great way to get more information about the law school admission process generally and about individual schools in particular. Not only can you talk with representatives of the law schools, but there are educational workshops for more general information. Speaking of resources, there is so much information out there about legal education. It makes me wonder, Kellye, where do you go to keep up to date on what's happening with legal and higher education?

KELLYE: Well, you're right, Susan. There are a lot of options when it comes to getting good information, but one of the things that I really like to do is make sure I'm talking directly to the people actually participating in legal education. So I want to give a big shout-out to the around 200 law school deans out there, and then all their admission deans and admission professionals, because they're all on the front lines every day, understanding what candidates are looking for and then helping make sure that journey from pre-law through practice is a successful one.

And that success is what we're all after and what we're joined in seeking. And as I work with the law school deans and admission deans, I've been really pleased that LSAC has been able to beef up the many channels of current data and information that's available to that community, and there's many examples of that in the emails and webinars and different kinds of newsletters we do. I would list at the top of that heap what's called LSAC Insights, because that really helps our community understand what's happening on the ground and where things are going so that as they're making decisions in real time, they have the information and the insights they need to make sure that they're informed as possible and making great decisions.

In a lot of ways. I think about LSAC as a huge help to our schools in making great decisions, and also to our candidates in just saying, "We're here for you. We have our arms outstretched, we want you, we want your voice in law, and we're here to help you find your pathway, your own unique pathway to put law and justice together for our world." I want to note, Susan, that as we think about that, I know that one of the things that all of our candidates obviously are thinking about, and our schools too, is the LSAT itself, which is the way that the students can know that they're ready for law school, ready to really learn more, as it assesses the core skills of reading and writing and reasoning that are fundamental for success in law school. So maybe, Susan, we could take a minute to talk about the LSAT itself. Anything in LSAC Insights or elsewhere that they should be sure to check out to learn more about the LSAT?

SUSAN: Certainly, throughout our website, there is extensive information about the LSAT, and in particular, there are links to at least three different ways to prepare for the LSAT. What we know is that preparation time is positively correlated with higher scores. Candidates can go to our web site to learn about Khan Academy. There are also written materials that we produce, and then there is LawHub, which has on it both a couple of free tests and, for just $99 a year, more than 70 prep tests that candidates can use to prepare for the LSAT.

And everything we learn is that it's absolutely worth the time. With respect to LSAC Insights, however, there are some very excellent Law:Fully blog posts and some very persuasive and compelling commentary on topics related to diversity, equity, and inclusion. However, I have to say that the Data Library section is the place to be. You can find current volumes (updated daily), test registration and test taker volumes, lots of historical data, and much more.

KELLYE: That's fantastic, Susan. Thank you so much for that terrific insight that you have into the data, and I know that you're going to be digging deeper with Josiah Evans, one of our leading psychometricians, for the next few minutes. I'll let you get to that, but before I do, I wanted to just thank you and thank our listeners for spending time with us today. We are here to help the enrollment journey and to help the law school journey toward practice be a successful one.

And I want to, on that note, put a quick plug in for a wonderful conference we're hosting October 15. It is called Black Lawyers Matter, and we are working with a wonderful group of law schools and their deans to organize that conference with excellent representation from a number of people on the planning committee and from our sponsors. That's a wonderful event that is in Year Two, and we plan to keep helping to bring that wonderful conference forward. So I hope you'll join us October 15 for Black Lawyers Matter, and you can register and also find so much more information about legal education and admission at LSAC.org.

I hope candidates especially know that they can always start at LSAC.org and get the help they need for their enrollment journey.

SUSAN: Thanks, Kelly. I attended the Black Lawyers Matter conference last year, and I am really looking forward to this year's conference, so thanks for reminding us about that.

And now, it's my pleasure to welcome Josiah Evans, our director of assessment science, which is another way of saying that he heads up our psychometrics group. So glad you could join me, Josiah. How are you?

JOSIAH EVANS: I'm great, Susan. I'm pleased to be here today and to share what we're seeing so far, even though it's early days and we don't want to read too much into anything we see yet. That said, August LSAT scores were released September 10, and while we always see cyclical variation in scoring from one test administration to another based on the candidate population testing at different times of the year, the August 2021 score distribution is very much in line with historical patterns we've seen for late-summer tests. Aggregated August 2021 scores are in line with and slightly lower than the aggregated August 2020 scores, but that's to be expected because we did not have a July 2021 test, so some candidates who would have tested in July are likely part of the August test taker population, and historically, July test takers have tended to score slightly lower in aggregate than the August test takers.

August 2021 scores are also very much in line with the scoring patterns we observed for the in-person, five-section 2019 LSAT. We did not have an August 2019 administration, but the aggregated August 2021 scores are very similar to the aggregated scores for the September 2019 administration.

SUSAN: Thanks, Josiah. And this is helpful because we know some people were curious about what might happen when we transitioned from the three-section LSAT-Flex back to the pre-pandemic format of including one unscored variable section that allows us to validate new test questions for future use and ensure that they're free from any form of bias.

JOSIAH: That's right, and the longer test did not have an impact on scores. And this might also be the place to remind people that the LSAT is not scored on a curve. We compared test taker performance on three scored sections that were administered during an undisclosed LSAT-Flex in 2020 and test taker performance on the exact same three scored sections, plus an unscored section, in the recent August exam. Test taker scores did not change in any meaningful way because of the addition of the unscored section.

We looked at whether scores were affected by where the unscored section occurs on the four-section tests, also. We compared test takers who took the same three scored sections and one unscored section with the unscored section last, similar to the LSAT-Flex testing experience, to those who received the unscored section first, second, or third. We found no significant difference in test performance no matter where the UN scored section occurred within their four section tests. These results are entirely consistent with our expectations based on many years of offering the five-section tests with the unscored section.

SUSAN: You mentioned a minute ago that the LSAT is not scored on a curve. Sometimes we hear, from test takers and others, speculation about whether we at LSAC "tightened up the curve" or "changed the curve" in response to whatever might have happened on a previous test. What about that?

JOSIAH: Every test is scored the same way, regardless of the population. In essence, we do not impose a curve on the scores. If, for example, we see a large number of highly proficient test takers who show up for a given test administration, then everyone receives the score they earned, even if many others also received that score.

SUSAN: So perhaps unsurprising, but something I took away from all the research LSAT does on test taker behavior is that August LSAT takers continued to devote more time to prep than test takers in pre-pandemic administrations.

JOSIAH: That's right. Test takers reported preparing 25 to 30 percent more during the pandemic, and for applicants as well. Undergrad GPAs were also up compared to last year. We also found that usage of free, official LSAT Prep through the Khan Academy and LSAC LawHub is growing rapidly. We also expanded our fee waiver program as yet another example of how we're trying to open up access and support.

SUSAN: Josiah, we had close to 25,000 test takers for August, and we're expecting similar numbers for the October LSAT, so we continue to see very strong interest among candidates. It's still early on in the cycle, but the large number of prospective candidates who appeared for the June and August tests, plus the high number of applicants from previous years, suggests we may see another strong admission cycle this year.

JOSIAH: That's right. Again, we're really cautious about making predictions this early in the cycle, but so far, so good. The most important thing is that behind these numbers are the individuals and their career aspirations, and that's something we take very seriously.

SUSAN: Absolutely. And as we close out the podcast, our listeners may want to encourage themselves and others to check out LawHub for the free test prep tools that were mentioned, as well as additional resources they can take advantage of.

Next month, I'm excited to speak with LSAC's chief diversity officer, Angela Winfield, who's on the road right now, conducting a comprehensive listening tour. I know she's planning to share some of her experiences on LSAC.org along the way, and I'll be eager to get a more thorough update on how things are going and what she's hearing from our valued community. Until then, take care and thanks for tuning in.

Thank you for joining us. Keeping Up to Data is a production of LSAC. If you want to learn more about the current law school admission cycle and the latest trends and news, visit us at LSAC.org.

Back to Keeping Up to Data