J of A: How the CPA Exam is Scored

The Journal of Accountancy has published an article in their May 2011 issue titled “How the CPA Exam is Scored”.

Some items of note that CPA Exam candidates should be aware of:

The article states that BEC essays are currently graded by humans.

“Because a new section of essays was introduced into the Business Environment and Concepts (BEC) section this year, initially, those essays will still have to be scored by humans, Mattar said. The Examinations Team will need to build computer models after it receives enough sample responses. If a test taker’s total score is close to the passing score, the candidate’s written responses will be automatically regarded [sic] by human graders.”

Another71.com asks: Where is the AICPA getting this army of CPAs to hand-grade tens of thousands of essays? 8,478 BEC exams were taken in the January/February 2011 testing window. BEC has three essays and let's assume that one of them gets tossed out as a pre-test essay. This leaves nearly 17,000 essays to grade by CPAs during tax season? I'm sure there is a reasonable answer…that just seems like an incredible amount of essays to grade by hand.

If anyone out there in AICPA-land wants to edjukate me ~ shoot me an e-mail at jeff@another71.com and I'll post the explanation. Again – I'm sure I'm missing the obvious.

Update: The article says BEC essays are human graded, but the PDF on the AICPA Website says they are computer graded?

Most responses are scored by a computer grading program, which is calibrated using human scorers. In some cases, responses are scored by a network of human graders (all CPAs). If your score is close to the passing score, your written responses will be automatically re-graded by human graders. When there is more than one grader for a response, the average of the scores is used as the final grade.

The simulations you receive have nothing to do with your performance on the MCQs

“Task-based simulation (TBS) questions are pre-assigned and are not chosen based on performance on the multiple-choice testlets.”

I still see people fretting about their terrible sims they saw on their exam and whether it had anything to do with their MCQs. This just reinforces this fact.

Source: Journal of Accountancy

Medium Testlets have difficult questions and Difficult Testlets can have “easier” questions.

“How do you decide which questions are difficult and which are medium?

The difficulty levels of the test questions (and other statistics that are used to describe each test question) are determined through statistical analysis of candidate responses. At the question level, difficulty is not quantified as a category (for example, moderate or difficult), but as a numeric value along a scale. Testlets are classified as either medium or difficult based on the average difficulty of the questions within that testlet.

Does that mean difficult testlets can have easier questions and medium testlets can have difficult questions?

Yes. All testlets have questions ranging in difficulty. Questions in difficult testlets just have a higher average level of difficulty than those in medium testlets.”

“Discrimination” – a back-door method of curving the exam?

“Three statistics describe the questions: difficulty—whether the question is generally easier or more difficult for candidates; discrimination—how well the question differentiates between more able and less able candidates; and guessing—the chances of candidates answering the question correctly just by guessing.”

If you do well on the BEC MCQs, you can bomb the essays and still pass.

“However, it would be very difficult to do so as you would have to perform exceedingly well. It is advisable to be prepared for both the multiple-choice and the written communication questions.”

If you do well on the AUD, FAR, and REG MCQs, you cannot bomb the sims and still pass.

“The portion contribution from task-based simulations in those sections is too large. You would need to get some of the task-based simulations questions correct to pass.”

Source: Journal of Accountancy

Read the entire article here.

Related Posts

NASBA & AICPA Invoke Emergency CPA Exam Testing Period Due to Coronavirus Covid-19

“In anticipation of the reopening of Prometric test centers and in response to the COVID-19 situation, NASBA, AICPA and Prometric have made the decision to invoke an emergency testing period. We understand candidates are concerned about their testing options and, therefore, we are implementing this emergency testing period to provide additional testing opportunities. As part […]

Prometric Closings: January 28

  Prometric has closed some testing centers due to winter weather. January 28, 2015 Bangor, ME 5119 Boston, MA 5135 Boston,MA 5235 Brockton, MA 5194 Brookline , MA 5332 BROOKLINE, MA 5331 Burlington, MA 5855 Concord, NH 5148 Cumberland, RI 3903 Glastonbury CT 0048 Glastonbury CT 5480 LOWELL, MA 5180 North Andover, MA 2504 Warwick, […]

Prometric Closings: January 27

  Prometric has closed some testing centers due to winter weather. January 27, 2015 ALBANY New York 5110 Allentown PA 1710 BANGOR Maine 5119 BOSTON MA 5135 Boston MASSACHUSETTS 5235 Bridgewater MA 7569 BROCKTON MA 5194 Brookline MA 5332 Brookline MA 5331 Brooklyn New York Unites States 5204 Brooklyn NY 5203 Burlington MA 5855 Clark […]

Prometric Closings – January 26, 2015

  Prometric has closed some testing centers due to winter weather. January 26, 2015 Allentown, PA 1710 Boston, MA 5135 Boston, MA 5235 Brooklyn, NY 5204 Brooklyn, NY 5203 Canton, MD 5001 Clark, NJ 5858 Clark, NJ 5871 Clarks Summit, PA 1736 Columbia, MD 0084 Hamilton, NJ 4113 Hamilton, NJ 4114 Lancaster, PA 1731 Laurel […]

Leave a Reply

2 comments

Celyn 10 years ago

I thought the essays were graded by non CPAs. That would go along with the notion that they are graded for structure and grammar, not on content. But I don't know if that's true, or another CPA urban legend.

Jason 10 years ago

So is there such a thing as "partial credit" on FAR, AUD, and REG simulations?