(reprinted from Noesis #137)
In his letter to the Editor, Chris Harding wrote, ``I believe the cut-off level should remain at the 99.9999th %ile.'' If we are to do this, we must also change our qualifying score on the Mega Test to 45 (allowing one point for ceiling bumping).Chris' remarks about selecting candidates for membership with 98-percent accuracy make no sense; clearly we can't come close to this kind of accuracy. His opinion that we should select members at the 99.9999th percentile without a demonstration of a means for doing so means little.
Chris wrote:
The final test need only to be a quite short one--say 20 to 30 items. The diffi-culty would be set so that 50% of items scored correct would reach the 99.9999 %ile. The difficulty level can be placed above the level of the final people--thus almost all candidates would go bust doing it.
How does Chris intend to calibrate a test on which ``almost all candidates go bust''?
A further requirement would be that the multiple choices given would leave them all with the impression they'd won through.
Obviously, the distractors have to be plausible. I put a great deal of work into constructing distractors which would lure those who failed to think through the problems on my tests.
Chris suggested that the two of us and Ron Hoeflin could construct a ``final'' test. While I appreciate this vote of confidence, it seems preferrable to me that there should be alternatives. Anyone with an interest in psychometrics can atttempt to construct an intelligence test, but it isn't easy to do it well. The quality of the better tests is generally recognized in the higher-I.Q.-societies community and a high score on a Langdon or a Hoeflin test is respected. As there are few competent to construct high-range tests, it is best that we should continue to work independently and let the psychometric experts and the marketplace judge of the success of our efforts.
But at the present time, no one has presented a credible case for discrimination at the 99.9999th percentile by any test. And there's something suspicious about the Ferguson formula (see "The Statistical Technique for Combining I.Q. Scores," by Ronald K. Hoeflin, [page 4]). A lot more people can score 164 on four different tests than 176 on one test. One factor that may make the use of the formula problematical for our purposes is that there is some indication that highly g-loaded tests are more highly correlated at the high end than they are over their full range, requiring a higher average score than that produced by the Ferguson formula for true equivalence to a score on a single test.
We also need to do something to bring our admission scores into line with the comparative statistics for the LAIT and the Mega Test published in Noesis #136.
I'd like to hear more opinions from the membership about these matters.