Thoughts on the MGMAT CAT

[vectorSpace]

I have a couple of observations about the MGMAT CAT Quantitative section. I'd like to admit that these observations were based only on 2 MGMAT CATs that I took in the last 2 days. I'd like to know - a) Other test-taker perspectives and b) MGMAT's thoughts on my observations.

1. Once you start well, the difficulty-level of questions (call it Q-level) jumps quickly to the 700-800 range. While it's understandable why the algorithm pulls questions out of the 700-800 bin, I found that ALL remaining questions (if you maintain a good strike rate) are of the same 700-800 bin difficulty level. This is unlike GMATPREP, which always randomizes questions, not sticking to the hard bin even though a candidate's strike rate may be high. In fact, I've noticed that the GMATPREP 'settles down' to an average to medium hard level after about 60% of the Quant section. In fact, there are also easy questions mixed throughout the test, even for a strongly performing candidate. This 'diversity' in the Q-level allows the average candidate to complete the real GMAT without rushing through the final questions.

2. Given the above, a candidate's 'mistake-level' is bound to be higher on the MGMAT than the GMATPREP. I found that consistently getting the 700-800 Q-level problems make it difficult to stick to the 2-min average. As a result, towards the end of the test, candidates may have to guess/skim problems. While I agree that this is a great strategy during the preparation-stage, consistently practicing on MGMAT CATs could entail a tendency in the average candidate to attempt problems at a faster than required pace in the real GMAT. This could have an unintended consequence of sacrificing the accuracy for speed on the real GMAT.

3. On the other hand, the 700-800 Q-level problems greatly improve a candidate's understanding of concepts through difficult problems under timed conditions. Hence I feel that the MGMAT CATs are a great learning tool during the prep-stage - however, because of (1) and (2) above, it may not be an accurate predictor of the real GMAT score, unless the algorithm somehow 'compensates' for this difference in testing conditions on the two CATs while evaluating the scaled score.

Thoughts?
vectorSpace

[StaceyKoprince]

The official practice test (GMATPrep), like the real test, includes "experimental" type questions, which our CATs don't use (we are building in this capability, but it will take a while - it's so complicated that test prep companies just haven't done this in the past). The higher level you are, the more likely you are to have little "breaks" in the form of experimental questions that are easier for you - on GMATPrep and the real test, but not our test.

We also attempt to address a specific situation: most students (used to) report feeling more time pressure on the official math section than on practice tests - ours and others. This had a potentially very significant downward impact on performance, so we wanted people to be prepared for that pressure before they went in, and the best way to do that is to force you to deal with more severe timing issues on practice tests.

We do attempt to adjust the algorithm for this as we can, but GMATPrep is still a more accurate gauge of your potential official score than our practice test (or any other test prep company practice test, for that matter). So you want to use both - ours primarily as a study tool for both content and strategy and GMATPrep as a more accurate gauge of potential score.

Our standard deviation is about 50 points, compared to a roughly 30 point SD for the official test, and an unknown (not published) SD for GMATPrep. GMATPrep will be higher than the official test, but probably lower than our own 50 point SD. I think of it as in the 40-point range (though I have no hard data to back that up).

[Guest]

Great answer - thank you!

[Guest]

I see that you said people used to report having more time pressure on the official test than on GMATPrep, and MGMAT corrects that. But do you have any feedback on how the time pressure on the official test compares to MGMAT? I would hope people are reporting feeling less time pressure on the real thing, at least in Quantitative. I'm at a pretty high math level (I get around 50 on all non-MGMAT practice tests including GMATPrep) but I can never finish your Quant section in time. One time I had three questions untouched!

[StaceyKoprince]

More of our students now feel that the time pressure on the official test is the same as or less than our test. Some people, of course, still have timing issues on the official test, sometimes more issues than they had on our tests. But it used to be pretty universal that people would get slammed on the official test and that's happening a lot less now.

[Guest]

I just finished MGMAT CAT #4 and noticed a problem that I've heard contemplated in another string on this forum: namely, that the computer seems to run out of hard questions. Actually, it may even be exacerbating the problem by using up too many 700+ level questions on earlier tests.

For example, on CAT #3, in the math section, 35 out of my 37 questions were 700+ level, even though I was getting enough of them wrong. I finished with a 42 on that section, putting me in the 66th percentile. This was only five days ago.

Then today, on GMAT #4 math section, I had 17 questions at the 600-700 and 18 questions in the 700+ level and scored a 50 (95th percentile).

What is even more telling is the fact that the first half was practically all 700+ and the second half was practically all 600-700. I got my last 11 questions in a row correct, and all were 600-700 (you'd think it would bump me up after a few correct ones in a row, especially since my previous test was virtually all 700+).

I will admit I am no algorithm expert but its safe to say I shouldn't be getting all of my questions at the 700 level when I'm scoring in the 66th percentile and, also, that I shouldn't be getting half my questions at the 600 level when I'm in the 95th percentile. Just letting you guys know - would be great if this could be fixed soon so that the current batch of test takers can get accurate readings on their last few CAT's. Thanks.

[JAMGAJR]

I finished 2 tests
MGMAT1 q51 v32
MGMAT2 q51 v35

The number of 700-800 level questions was around 35.

Your questions are very, veru good. I did the PowerPrep1 (only quant) and finished in 38 minutes. When I did your tests, i finished in 74min...............

I think the CR questions were much more difficult than those of og11..........is it correct?


I am going to do the test in November...... i am very worried about Verbal. How can I improve?
I have been studying for 2 weeks (og11)
I aim for a 740 (at least)....................

[JAMGAJR]

Only to give you additional information
I am from Brazil .........and my english is not veru good.........

[StaceyKoprince]

Well, it looks like you don't have to worry about the math! :)

For verbal, you're currently scoring at about the 65-70th percentile. A 740 overall is the 98th percentile (combined) - which is VERY ambitious. Only 2 percent of test takers score that well. (Not to say you shouldn't go for it! But don't compare this to a normal test in school in which you can just study more and get an A. This as a forced-curve test in which only 2% of us will be allowed to score 740+, no matter how well we all do.)

If you maintain your current math level, you will still have to raise your verbal level to the 99th percentile (or very close to it) in order to get a 740.

Given that English is not your first language, you may want to seriously consider a class or private tutoring. It's touch even for native speakers to hit 99th percentile; it's obviously even more challenging for non-native speakers. Take the GMATPrep test - the whole thing, including the opening essays, because those affect your stamina, and therefore your score, and see how you do. GMATPrep is the closest indicator to an official GMAT score (though it is by no means exact). You can get it from www.mba.com. (PowerPrep is the older version - GMATPrep is a more accurate test.)

[chrissy]

Hi I just bought two MGMAT books and tried to go online to take my first practice test from the 6 included online tests. However when I tried to register and access the tests I got to a screen that asked for the 32 digit serial code on the INSIDE back cover. But there IS no number on the inside back cover of my book. Nor anywhere else in the book that I can find. What gives? I bought the books from the manhattan gmat website so I know they're real! Help ....

[StaceyKoprince]

Hi, Chrissy, sorry for the confusion! Please either email tech support or call them (techsupport@manhattangmat.com or 800.576.GMAT and ask for tech support) and they'll get you straightened out!

[JAMGAJR]

Stacey,

First of all, Iwould like to thank you for your advices about get verbal tutoring.
As I am form Brazil, I really need a special verbal preparation. Last week i started these verbal classes.

I have already finished OG11. I bought the MGMAT SC and CR books, as well as LSAT EXPLAINED. I am going to start them as soon as possible.

Following your advice, after 5 weeks of study, today I took the GMATprep1.

My result: 760 (q50 v42).

I could not understand both of the results..........
I was expecting a q51, but had 6 questions wrong (31 correct og 37) and got a 50. I did stop to see what kind of mistakes I had..........
In verbal, I had only 7 mistakes (34 correct of 41........i found 4 questions from og11).
Yesterday I tried sets 1 to 3.....on average I had 36 correct/1wrong 1 quant and 29 correct/12 worng in verbal. This is why i am surprised.

Is it possible that I was lucky today? Or Gmatprep1 is easier than other tests?

I did the essays before the test and i believe that gmatprep1 is easier than MGMAT.

Next week i will take mgmat3.....just to compare the results.

[Guest]

Jamgjr,

I feel GMATPREP is quite representative of the real GMAT, with a (an unpublished and approximate) 40 point standard deviation from your true ability. I won't be surprised by your scores, in fact, my own scores were in the same range when I took GMATPREP 1 and sets. I got 7 incorrect in Quant and 11 in Verbal - and my score was 760. Likewise, I averaged 2 incorrect on Quant in sets, and about 8 on Verbal.

The reason for your high score despite making mistakes include:
1. You are getting many 'hard' questions, and getting hard questions wrong does less damage to your score than messing up an easy one. The number of incorrect questions is therefore a deceptive indicator - you should always compare the difficulty along with the numbers. The reason you are getting hard questions is because you are doing really well in the first place.

2. Also, it is well-known that getting many questions wrong in a row hurts you more than spreading your mistakes. Look at your pattern of mistakes - I'm pretty sure these are well-spread (you probably did not get 3 wrong in a row).

Thus getting 6 well-spread, hard questions wrong is perfectly permissible for a Q50 - I'd say you are pretty comfortably placed! Get a GMAT date, and shoot for 780 :)

Good luck,
vectorSpace

[JAMGAJR]

Vectorspace,

Than you for the information.

In quant I feel i am in a good shape. But I bought the 4gmat books in order to do more difficult exercises.

But in verbal, despite the fact i had a good result in gmatprep, i am not confident. This is why i intend to take the offical the test only in october or november.

[StaceyKoprince]

There's also a bit of a design flaw that does make GMATPrep a bit easier at the very highest levels. Essentially, there isn't a huge pool of questions for GMATPrep - they don't publish how many are in the database, but it is certainly nowhere near the number in the database during the official test (and that's part of the reason why the standard deviation for GMATPrep can be expected to be higher than that of the official test, even though they use the same algorithm).

So, at the highest levels, the test doesn't have as many Very Difficult questions to go to in terms of giving you a very hard test experience. Just be aware of that - the official test will have a deeper pool of Very Difficult questions from which to choose, and you will see them on the real thing.

But, overall, excellent job!