I took the Miller Analogies Test yesterday because I had nothing better to do on a Saturday and because I wanted to see if I could get into the Prometheus Society. I can't. I fell short by a mile, actually by many, many miles, scoring only a 476. Here's a blog post by a guy who got a 506, and here's another by a gentleman who got a 486.
I took the GRE 3 or 4 times over the years, getting almost identical results each time. There are tables on the Internet that show equivalent scores between different standardized tests, and a 476 on the MAT is almost identical to my old GRE scores.
I've taken a specific work-related test 4 different times by now. One of those times I screwed up the timing and didn't get to the last dozen or so questions. But the other 3 times my scores were almost identical to each other. It's eerie. Also funny, considering the amount of money made by test prep companies and publishers every year. How much of the medical profession operates on the same principle?
About 5 years ago I had a bout with cancer. Half of my hair fell out because of chemotherapy, I couldn't keep any food inside me for a week at a time, and for long periods a weird, chemo-related fog spread over my mind - an amazingly crappy sensation that I've never experienced before or since.
But even that failed to permanently alter my scores. I got pretty much the same result on those professional tests before and after chemo.
Complex mechanisms tend to be fragile, but the mind apparently isn't. Kingsley Amis, my favorite English-language author, drank heavily all his life, and yet his last novel, written in his early 70s, was just as witty as the ones he wrote in his 30s. If hundreds of gallons of whiskey won't screw it up, what can?
By the way, according to this PDF (p.40), the highest MAT score during the 2001-2003 period was 563. Good God! Why is Lady Gaga a celebrity, but the guy who scored a 563 on the MAT isn't? That's supposed to be 6.52σ above the mean, which gives us a right-tailed p-value of 3.515*10^-11, which translates into a frequency of roughly one in 28.45 billion people. Well, perhaps the testing isn't as reliable at the extreme right tail as elsewhere, and perhaps the extreme right tail of the IQ distribution isn't even very normal to begin with. Regardless, my hat goes off to the geek who managed to score that high. Has anyone outscored him since 2003?
The highest level that tests like the LAIT claimed to reliably measure was, if I'm not mistaken, around 175 IQ. 6.52σ, assuming a mean of 100, implies something like 198. Of course the mean here should be higher than 100 because only people who want to go to graduate school take the MAT. The company that operates the MAT has a lot more resources than Mr. Langdon or Mr. Hoeflin. For example, the 563 guy was the best in a sample of 126,082 people. However, since I'm not a psychometrician, I'll stop there.
What of the test itself? I was surprised by how un-PC it was. You had to know who people like Camus, Renoire and Degas were. As far as I remember, all the cultural references were Western and all of the culture referenced was high, not TV-based. However, it was a real IQ test because sometimes I couldn't get the analogies even though I recognized all of the terms in a question.