How testmakers construct the test

[interestedintacos]

I do best on flaw, method of reasoning and necessary assumption questions (of course categorizing just based on the question stem, which I understand might be different from the MLSAT strategy).

Lately I've been going through flaw questions. Perhaps it's just this question type in particular, which can be very formulaic, but I'm starting to pick up on the way the test makers construct these questions and answer choices. It gets very predictable.

It seems to me that test makers put in the basic components, perhaps an argument in simple, easy to understand form, perhaps taken from a real source, perhaps not. Then they tailor it a little to ensure that the argument will clearly contain one particular flaw (they seem to test around 10-15 different flaw types in all). Then it seems they often add in a few touches to make sure they can add a little bit of viability to incorrect answer choices (SIDE POINT: if there was NOTHING at all to support AT LEAST ONE of the incorrect choices the test wouldn't succeed in differentiating between different sets of test takers). They also might rearrange or take advantage of some other tactic in order to increase the difficulty. Easier questions will have the least amount of fiddling, and more difficult questions will often have more, sometimes by messing with the stimulus, other times by wording the answer choices in a confusing way.

For instance, in a question where the correct answer will be that the arguer committed an ad hominem attack, the testmakers will find a way to insert language/cues meant to look like another flaw, or vice versa: sometimes a few words, sometimes a sentence, sometimes a single word. The additions will be irrelevant to the core flaw going on.

My point is this: Doesn't this indicate that students who want to master the exam shouldn't rely on methods of memorizing cues, etc. ? Even learning cues to recognize premises and conclusions, a foundation of analyzing arguments, can be misleading and at the very least, it doesn't seem to be a way to master the test. Nevertheless, I know Powerscore, Kaplan and others emphasize that approach; I'm not sure about MLSAT.

In all the materials I've looked at I haven't once seen someone breaking down the LR section based on a hypothetical way that test makers construct the test. To me that would be the best way to actually master the LSAT. I think the test makers certainly have a formula, at the very least for each question type (divided by question stem)--which they even hint at in their official guide (the Superprep). In fact, in the Superprep they even TELL YOU their formula for picking incorrect answer choices, etc. (for instance that often incorrect answer choices will be the complete opposite of the correct choice). Not only that--they tell you WHY they do it! They explain to a limited degree the psychology behind why wrong answer choices are attractive. So far I've found it to be the best test guide by a huge margin.

In my studying so far I've found major patterns. Sometimes test makers have even gone so far as to duplicate the logic in a question, the subject, AND the answer choices. Clearly the test makers have a lot of room for variation that they DO NOT take advantage of; there's certainly no excuse imaginable for using essentially the same question twice (even with the same subject matter--homeless people and low-income housing, for instance) given the amount of variation possible. But the test makers have done it.

[noah]

I agree particularly with this point: understanding the way the testmakers write wrong answers is useful. I dare say that we teach a lot of this. It's actually a fundamental part of our approach - working wrong-to-right (which is slightly different than the usual P.O.E. in that you actively seek out wrong answers).

I appreciate your efforts to understand the LSAT at a deeper level. Not only is it admirably geeky, it'll no doubt prove effective in taking you beyond your typical score.

[interestedintacos]

I haven't looked at your materials for logical reasoning, but that sounds like an interesting approach.

The background behind that approach would probably be studying the methods the test engineers have pulled out from cognitive science in order to ensare test takers. As I've noted, I think it's interesting that the test makers even touch on those methods in their own book.

Aside from that, I think the three most fundamental categories of questions are must be true/inference (MBT), flaw and method of reasoning.

MBT simply tests putting one and two together with precision. All the incorrect answers essentially test whether you are interpreting the information precisely. The reverse and opposite choices (which again test precision, especially of the question stem and answer choices), choices where the answer choice is either the correct answer in reverse or the exact opposite of the correct answer, are bread and butter standardized test fare. The most common type of incorrect answer is the simple "not germane" choice (again, precision).

The test makers describe method of reasoning questions first in their own book, and you can see why. By studying these questions you can unlock the various argument structures they use. And this will be necessary before moving on to flaw questions.

Flaw questions allow you to unlock pretty much all the ways in which the test makers can poke holes in arguments. Mastering this type allows you to master weaken, strengthen, assumption, justify, etc. I think flaw questions really are the most fundamental; I've yet to encounter an argument with reasoning not addressed in a flaw question at some point in the LSAT's history. But I haven't faced the most modern LSATs, so let me know if something has changed.

Method of reasoning and flaw questions in my mind are sort of questions straight out of the "template book" of the test makers. By practicing those types you will be learn how arguments are constructed and taken apart, and that kind of thinking will help on all the questions.

I'm not sure how this either agrees or disagrees with the MLSAT method.