How are SIMs actually graded???

  • Creator
    Topic
  • #2347338
    baylor.bear95
    Participant

    I’ve been using Becker to prepare and notice a lot of the time Becker grades the Sims ridiculously by counting an entire line as wrong if just one answer in that line is wrong even if it has multiple aspects. Curious if this is the same for the actual exam and how the CPA grades the simulation section as a whole. Is each TBS in a testlet converted to a percentage then averaged for a total average for that testlet then averaged against the rest. Or is a testlest averaged based on the total correct answers / Total questions then the those testlet averages are aggregated against the other testlets. Or is the whole sim section just a percentage of total corrects answers / Total questions. trying to figure out if i can adjust my Becker mock exams results to more accurately reflect the true CPA exam scoring.

    AUD - NINJA in Training
    BEC - 96
    FAR - 92
    REG - 89
    2 to go
Viewing 9 replies - 1 through 9 (of 9 total)
  • Author
    Replies
  • #2347473
    Pork Flavored Bacon
    Participant

    Unfortunately, no one knows how they are graded. The only advice I can give you is to read the instructions carefully. Sometimes they will tell you to enter a zero in unused spaces or leave them blank. Make sure you read the instructions to know what they want.

    FAR - 75 | REG - 87 | AUD - 82 | LAW - 81

    #2348205
    alloverit
    Participant

    I didn't use Becker so I'm not completely sure what you mean, but…

    With one exception, each “box” on the SIMS portion of the exam is graded separately. So let's say you perform a calculation and enter an answer for that calculation, and then a second calculation based on the first, and then a third calculation based on the first two calculations…you can earn credit for all three, two of three, one of three, or zero of three.

    The exception is the research question. BOTH boxes must be answered correctly or you get nothing. https://www.another71.com/cpa-exam-forum/topic/partial-credit-for-research-question/

    What is unknown is how much WEIGHT is assigned and where. Using the example above, an easier SIM may only award you 50% of the points possible if you answer 2 of 3 boxes correctly, but a harder SIM may award you 80% of the points in the same scenario.

    Also, even though it is known that the research question is all or nothing, the question's weight is unknown.

    I also didn't use Surgent. But they posted an article recently about partial credit on SIMS along with other info.

    Good luck

    FAR 81

    AUD 83

    BEC 93

    REG 84

    Ross

    #2349456
    animalwithin
    Participant

    I'd like to get a definitive answer to this as well as Roger grades SIMs the same way you are describing. I'm missing an entire rows in almost all SIMs when I'm only missing one of the boxes and I don't know if this is how it is on test day.



    @Alloverit
    , are you sure each box is graded separately? If so my score is significantly higher than it is now on SIMS.

    #2349459
    baylor.bear95
    Participant

    The AICPA is so vague and ambiguous about the grading, which is probably on purpose because I don't believe they are totally honest about their grading system. They try and make it seem like the points value for each question is assigned before it is tested, but the fact it is a 100% computer graded test and the scores are released in blocks doesn't coincide with that. The only reason they would be accumulating tests before releasing would be to “curve” the exams in some way, because if its graded the way they say you should receive your score as soon as you submit the test.

    AUD - NINJA in Training
    BEC - 96
    FAR - 92
    REG - 89
    2 to go
    #2350689
    Jimmy Dugan
    Participant

    We will never get a good answer for this question, or even how the exam is graded in general. The review courses can't be grading them like the exam, simply because they don't know the weighting each question gets on the exam. Don't worry about it too much. Just make sure you are paying attention to the instructions about how to enter the amounts (negative, positive, rounding, etc.).

    AUD - 95
    BEC - 87
    FAR - 84
    REG - 90
    You're killing me Smalls

     

    #2350806
    alloverit
    Participant

    A Google search shows plenty of results indicating that partial credit is received on SIMS (except for the research question). But again, the WEIGHT assigned to the partial credit is the key we don't have.

    So if you get 4 of 10 boxes correct is that 40% credit? 60% credit? 32.4187% credit? Does it change based on difficulty or pass rates for that window?? No one knows.

    Common sense would indicate that partial credit is given. Otherwise the pass rate for the exams would be about 5%

    FAR 81

    AUD 83

    BEC 93

    REG 84

    Ross

    #2351502
    animalwithin
    Participant

    I agree with you that common sense would indicate that partial credit is given. It's just frustrating getting 20% on SIMs when practicing just because you missed a box here and there and the whole row is getting counted as wrong.

    #2352891
    jeff
    Keymaster

    The AICPA reached out to me about this thread to clarify…

    “A candidate receives credit for each measurement opportunity (question) in each simulation. The total simulation score is based on the number of questions answered correctly across all the simulations that the candidate takes.

    Of course, we also recommend that they also look at this information on our website:
    https://www.aicpa.org/becomeacpa/cpaexam/psychometricsandscoring.html

    AUD - 79
    BEC - 80
    FAR - 76
    REG - 92
    Jeff Elliott, CPA (KS)
    NINJA CPA | NINJA CMA | NINJA CPE | Another71
    #2353176
    animalwithin
    Participant

    Thank you Jeff! Would be great it test prep courses would reflect what you've stated above. My experience with Roger and according to others, with Becker, we're getting graded on the entire row of questions as a whole rather than each question separately and thus are getting low scores on SIMs not reflective of how the test is actually graded.

Viewing 9 replies - 1 through 9 (of 9 total)
  • You must be logged in to reply to this topic.