A Nate Silver-Take On The Most Influential MBA Ranking

Just a few days ago, Bloomberg BusinessWeek published the 13th edition of its always controversial rankings of the world's best business schools. This year, the University of Chicago's Booth School of Business stayed atop the list of the best U.S. schools for the fourth time in a row. London Business School, meantime, nudged aside INSEAD as the best non-U.S. school.

Columbia Business School and UC-Berkeley Haas had notable declines in the ranking, each falling five places to finish 14th and 13th, respectively. For Columbia, which suffered a 19% fall in MBA applications last year, it was the worst showing since 1988 when the BusinessWeek ranking debuted. The school’s standing declined in two of three of the magazine’s key metrics: student satisfaction and intellectual capital. The biggest drop was in student satisfaction where Columbia lost 10 places to finish 20th this year. It also fell seven places in intellectual capital to 19th from 12th. Those are fairly significant setbacks in the space of one survey.

Ever since I had the dubious distinction of creating what became the first regularly published ranking of business schools, I've sometimes wondered if I created the proverbial Frankenstein monster.

MBA applicants tend to put far too much importance on rankings in deciding where to get their degrees, and schools take the whole exercise far too seriously, given the limitations and flaws of this or any other ranking. The results of these lists is entirely a function of the chosen methodology. 

Back in 1988, when most business schools were as customer friendly as AT&T when it had a monopoly, it made sense to me to subject the community to the discerning opinions of its core customers: the graduate students who today spend more than $250,000 in tuition, fees and opportunity costs to get an MBA degree and the companies that recruit the vast majority of them. After all, the attitude on most B-school campuses was self-satisfied, arrogant, and we know better than anybody else. (The metric of academic research, which accounts for only 10% of the ranking, was added later).

But what if someone like Nate Silver, who so artfully dissected political polls through the current Presidential election for The New York Times, cast his evaluative skills on the BusinessWeek ranking now on the newsstands? What would he say about the relative strengths and weaknesses of the effort to assess the quality of the world's best business schools?

With apologies to Nate, I think I have a sense of what he would say. That as a measurement of what recent graduates think and how MBAs have fared inside the organizations that hire them, it's not all that bad. As a true measure of quality, however, it might not be so good. How come? By and large, the graduate satisfaction numbers are so closely clustered together that the difference among the schools often isn't great enough to be statistically meaningful. The quality of the recruiter judgments, moreover, is so dependent on both survey design and who responds that it is hard to know how solid those results are as well--and how comparable they are survey over survey.

Nate would in all probability raise issues about the underlying index numbers between each school's actual numerical rank. Is No. 2 Harvard Business School, which an index score of 97.5, all that much different from No. 3 Wharton, which has an index score of 96.5? After all, a mere 1.0 separates the two. Depending on your point of view, you could think the methodology fails to account for Harvard's superiority, with its $2.8 billion endowment, 34 buildings, superstar faculty, and more CEO alums than any other institution in the world.

Still, this remains a sincere and genuine effort to hold business schools accountable to their key stakeholders--and it quickly uncovers problems on campus that would otherwise go unnoticed and unaccounted for. To that end, it remains a valuable if annoying service to deans, professors, students, alumni--and, of course, the applicants who are about to make the single biggest investment of their professional lives.

Our complete analysis of the results of this year's ranking is at PoetsandQuants.com:

Winners & Losers in BusinessWeek's 2012 Best Business School Ranking

Chicago Booth Retains No. 1 Status in 2012 BusinessWeek MBA Ranking

London Business School Captures Top BusinessWeek Rank Among Non-U.S. Schools