No AI could pass a professional licensing exam like the bar
The old claim turned out to be wrong and was rejected by evidence.
TaughtA gruelling professional licensing exam (the bar for lawyers, the medical boards for doctors) tests years of specialized human judgement. A computer might answer trivia, but it could not pass the kind of exam that certifies a person to practice a profession.
NowIn March 2023, OpenAI reported that GPT-4 scored high enough on the Uniform Bar Exam to pass in most US jurisdictions, around 75%, and it cleared other professional and graduate exams too. A machine passing the bar went from thought experiment to done.
What actually happened
GPT-4's launch materials showed it clearing a battery of real human exams, the bar among the most striking: a licensing test that gates entry to an entire profession. The raw fact, that its answers met the passing threshold used by most states, holds up and is the reason this claim is genuinely disproved.
The famous headline number, though, deserves the site's usual honesty. OpenAI said GPT-4 landed around the 90th percentile of bar takers; a 2024 re-analysis by MIT's Eric Martínez argued that figure was inflated because it was measured against a pool skewed toward people re-sitting after failing. Compared with first-time takers who passed, GPT-4 sat closer to the middle of the pack, and weakest on the essays.
So the precise correction is layered: 'no AI can pass the bar' is false; it passed. 'AI is a top-decile lawyer' was oversold. Passing the exam is also not the same as being fit to practice law. But the specific wall your schooling implied, that these exams were beyond any machine, came down in 2023.
Newsletter
Get new weekly revisions
Sources
Who was taught this
Still standard through 2023, so anyone who finished school between 1950 and 2023 learned the earlier version.