‘Going Beyond’ and Grading Elvis

Three lawyers share their experiences and advice as bar exam graders.

By JAMES CARLSON

Summer 2014

Around 900 people, many still gripping a stack of worn note-cards, will file into the Colorado Convention Center next week for the summer bar exam. The annual rite of passage for attorney hopefuls is a marathon event — 12 hours spread over two days testing knowledge and intellect gathered over three years of law school. In the end, the exams go to the graders.

Graders work in teams, each team focusing on either one essay question or one Multistate Performance Test question. Three such MPT graders recently sat down to discuss what goes on behind the scenes. (To shield the graders from any undue attention, we have used pseudonyms below.) Collectively they have more than half a century of experience in law.

They discussed the elements of a good answer, how simply following directions is key to success and why trying to be funny in an answer can backfire. An edited version of the conversation is below.

The new holistic grading system gives a more global assessment of a potential lawyer’s capabilities. I’ve heard some people say that unlike in the past, answers in the new system have to go beyond simply restating the law. What does it mean to “go beyond”?

Steve: Do they understand and lay out the elements of the legal analysis? Do they apply those elements to reach the proper legal conclusion? Do they put it in the right format? Do they write it well?

What about the inverse, then: What does it look like for an answer that doesn’t go beyond?

Joan: One of the most basic problems is that they don’t follow directions. We have what we term the “call of the question.” What is it that the test takers are supposed to be responding to and do they actually respond to that?

Mary: Right. Some people go off on a tangent. They’ve got a constitutional issue and they talk about something else, which immediately drops you down on your points.

Joan: Or they talk very informally — “By the way” or “Have you thought about?”

Mary: You don’t get to give a lot of bull and pretend you know what you’re talking about. We’re looking for specific, concrete answers.

Steve: Quite often the prompt is “Write a letter advocating for this,” and then they write a memo that talks about both sides. That misses the concept of the question which was a letter advocating for one position.

Mary: It’s simple: Look at what the question is asking you to do.

I’m picturing my elementary school teachers telling us over and over, “Remember to follow directions.”

Mary: Right, in that sense, it’s pretty simple. Or at least you would think so.

Steve: They’re given everything they need for the MPT. Generally, everything you’re given has a meaning, so if you ignore some portion of it, that’s to your detriment.

Joan: I will say it’s a lot of material to digest at one time.

Steve: That’s true. And if you panic, you’re in trouble. I think outlining helps. I don’t think people spend enough time outlining before they write.

How incomplete can an answer be and still receive a good score?

Steve: It depends on in what area they’re incomplete because different portions are weighted differently. If you’re incomplete on a 10-percent point, that’s better than being incomplete on a 30-percent point.

You mentioned earlier that some answers are too informal. How important is proper grammar and punctuation?

Steve: Improper punctuation and grammar aren’t going to get you down that much, but the structure of a sentence can matter. If you write really well, that can be the difference between a 5 and a 6.

Mary: Formality can mean something, too. The call of the question often asks, “Write a memo to your senior law partner.” You’re not going to address your senior law partner, “Hey you.” And you’re not going to sign off as Elvis Presley. In a test situation, as in real life, you need to treat your senior law partner with a great bit of formality. We might laugh at your test, but sense of humor isn’t going to get you the point.

Did you actually encounter someone who responded as Elvis Presley?

Mary: Yes, I had someone put “To: Senior Law Partner. From: Elvis Presley.” I will give that answer the attention it deserves, but I’m going to look at it differently because they’re already showing that they’re not taking it seriously.

How well can someone do on a question if their reasoning is sound but the outcome of their analysis is incorrect?

Mary: If you’re wrong but have a good reason for it, we take that into account. You’re not going to get a 6, but if you explain yourself well and understand how you got there, you can do ok.

I get the sense that some law students see the exam as a hurdle simply to get over and not something that’s relevant to real-life practice.

Mary: To me, the performance test addresses that concern. The (MPT) is what it’s like to practice law. It’s applying facts and the law in a coherent and intelligent analysis. A question inherent in these performance tests is, “How well would you help the senior law partner with this issue?”

Steve: I think (the MPT) is probably one of the more relevant parts of the test. I’ll read answers and say, “Wow, I’d hire this person” and then read others and say, “No way.” It does reflect on how they’re probably going to perform as a junior associate.

Is there something distinct about the answers that make you say, “I’d hire this person”?

Joan: They’re organized.

Mary: They answer what the question asked. They made it easy for me to understand the law and determine my course of action.

Steve: And it’s a pleasure to read. One point follows the next.

Matt Samuelson, Chief Deputy Regulation Counsel who oversees Attorney Admissions, jumps in: It’s not just throwing a bunch of words on the page in order to get points. A big element of the holistic grading system is the ability to communicate your answer. It’s about being able to understand the law, understand how it applies to the situation and being able to communicate that to a client or to another lawyer.

Steve: In that sense, I think the bar exam is now harder. It requires more communication skills.

Mary: It’s meant to make sure you can be a real lawyer in the real world.

James Carlson is the Information Resources Coordinator for the Office of Attorney Regulation Counsel.