Top law schools’ rejection of ranking should inspire others

There has been a recent exodus of elite law schools participating in the US News and World Report (hereafter US News) rankings. Law schools at Yale, Harvard and a range of other top institutions do not believe the rankings accurately reflect their values.

For those who research rankings, the exodus is welcome news, as we have raised the alarm bells about the negative impacts these metrics have had in higher education, especially in the international education arena.

The law school ranking is a fairly complex metric composed of over a dozen categories. Below are all of the indicators US News included in the most recent rank metric:

• Peer assessment score (25% or rank)

• Lawyers and judges assessment score (15%)

• Employment rates for 2020 graduates 10 months after graduation (14%) and at graduation (4%)

• Bar passage rate (3%)

• Average debt incurred obtaining a Juris Doctor (JD) at graduation (3%) and the percent of law school graduates incurring JD law school debt (2%)

• Median Law School Admission Test and Graduate Record Examination scores (11.25%)

• Median undergraduate grade point average (8.75%)

• Acceptance rate (1%)

• The average spending on instruction, library and supporting services (9%) and the average spending on all other items, including financial aid (1%)

• Student-faculty ratio (2%)

• Library resources and operations (1%).

US News provides rationalisations for each indicator, along with changes it has made year to year. The metric was built with sector-wide outreach along with other indicators relevant to the legal profession, all weighted by the rankers. Despite all of this effort, law schools across the country are still rejecting the veracity of the ranking.

Faulty programme rankings

Our law school colleagues’ protest should inspire the rest of us, especially as other sectors do not even get close to the care that has gone into the US News law school ranking.

US News also provides a ranking for each of the following areas: first-year experiences, co-ops or internships, learning communities, senior capstone project, service learning, undergraduate research or creative projects, writing in the disciplines and study abroad.

Rather than a well-explained metric of elements that might be important in each of these areas, the magazine simply sends out a questionnaire to “college presidents, chief academic officers, deans of students and deans of admissions”, asking them to list up to “15 institutions with stellar examples” of these programme areas.

The rankers then simply add the responses together to compile their yearly ranking of universities that received at least 10 nominations in the questionnaire.

The current construction of the metrics in these academic areas is merely an afterthought for the magazine. The questionnaire mostly does not even target the people working in these areas. Our work is not just an afterthought; these areas are essential parts of the university and crucial to student life, and they deserve better care.

Study abroad as a core value

The US News ranking for study abroad caught our attention because this area is a central tenet of our mission and culture at Soka University of America. It is not simply something that some students may or may not choose during their time at Soka. Instead, all undergraduate students must study abroad to graduate. The institution provides generous funding to ensure that all students study abroad.

Our strong value in what it means to study abroad was not a simple afterthought in the founding of the university. There were fights to make sure all students did study abroad – yes, even international students. And 50% of Soka’s undergraduate population are international students.

But Soka can’t make the study abroad ranking by US News, although it’s always placed at the top of its Most Students Studying Abroad list, with 100% participation as study abroad has been mandatory since 2003.

At Soka, every student is required to bring back 12 credits from one full semester abroad. The cost is included in tuition, so students don’t have to pay anything extra. It’s a unique and highly impactful programme.

Unfortunately, Soka is still very new and the experts US News consults don’t know us, meaning we are never ranked among the ‘Best Study Abroad programmes’. It’s frustrating to be evaluated by such a simplistic measure.

Those working in study abroad have probably had these same fights carried out at their institutions. Pre-COVID, fewer than 11% of American undergraduates studied abroad and that number plummeted during COVID. As a sector, we would love to increase those numbers, boosting the prominence of study abroad in US higher education.

It is common for those of us working in the study abroad sector to have had experiences abroad that changed our lives. The ranking does not reflect the weight and value we put into the experience of going to an unfamiliar place, surviving, learning the language, culture and customs and then thriving there. It is truly life changing.

Better ways to measure study abroad

The study abroad ranking is currently an afterthought for US News, tacked on at the end of a larger survey sent to presidents, provosts and admissions deans. We are not suggesting that university administrators are unworthy of evaluating study abroad, only that US News is not using the real experts. The study abroad or international student offices employ loads of people on the ground, working with various programmes worldwide.

Study abroad staff gather at events hosted by NAFSA and-or organise through Study Abroad and International Students Special Interest Group in the Comparative and International Education Society (CIES). These professionals would at least have credibility in evaluating study abroad programmes for young people. University presidents do not.

Even if high-level administrators have had experience abroad recently, how does their experience relate to an undergraduate? When the executive lands in a foreign country, a private car is waiting at the airport for them to be chauffeured directly to campus.

Students studying abroad may get some support upon arrival or may have to find out everything on their own. Perhaps the type of welcome they receive or the ease of travel to their destination would be a reasonable metric to include? Though, sometimes getting lost in a foreign country is half the point of ever going – another complication in trying to measure something so personal.

Likewise, when the university executive stays on campus abroad, they are put up in the finest digs, likely in the best nearby hotel. They are taken to the top restaurants around campus.

Students studying abroad usually eat what the local students eat. The canteen abroad could be amazing, introducing students to wonderful new foods. In other cases, students may find local cuisine hard to stomach. It likely varies according to each person – we’ve had students who fell in love with natto (fermented soybeans)!

A university president likely cannot go abroad for more than a week or so due to obligations at their home institution. Generally, over half of the students in the US who study abroad go for a quarter, a semester or longer. There is little comparison between dipping into a foreign country for a couple of days versus living there for an entire semester or more.

We concede that not everyone can afford to study abroad. In fact, the sector is highly skewed toward affluence, often reflected in racial disparities. NAFSA reports that black, Hispanic/Latino, Asian and Native American students are all under-represented in study abroad compared to their proportion in population enrolment numbers.

Any ranking of study abroad programmes should consider how institutions have helped groups historically left behind. Providing guaranteed funding and other assistance is crucial to expand traditional study-abroad populations. We are certainly considering this at Soka.

There is a plethora of other areas that matter for students when evaluating study abroad – selection of locales, accommodation, volunteer opportunities, etc. We could even just ask students themselves upon their return to rate their experiences. In the end, though, each person will have a highly personal experience that is likely impossible to capture, especially in a singular indicator.

Riding the law school wave

The ranking magazine responded to the law school exodus by saying: “US News has a responsibility to prospective students to provide comparative information that allows them to assess these institutions”. Furthermore, they claimed that the ranking is “designed for students seeking to make the best decision” and to provide “the best and most accurate information”.

For colleagues in study abroad, first-year experiences, co-ops or internships, learning communities, senior capstone, service learning, undergraduate research or creative projects, and writing in the disciplines, does the current US News ranking of our sectors reflect “the best and most accurate information”? We do not believe that it does.

Those working in study abroad and beyond should ride the current moment with our law school colleagues and reject a ranking that does not reflect our values. We owe it to our students to try to evaluate life-changing decisions and give them better, open and honest information. The current US News ranking does not do any of those things.

Ryan M Allen is an assistant professor of comparative and international education and leadership, Graduate School, Soka University of America. He tweets from @PoliticsAndEd. Tomoko Takahashi is the vice-president for institutional research and assessment, dean of the Graduate School and professor of linguistics and education at Soka University of America, United States.