Noah Mancino

Recently I’ve gone through the process of applying to law school, and I have observed that in the neurotic world of admissions forums, no topic inspires bellyaching and division quite like testing accommodations. The common way discussions proceed on this topic is that someone calls people who are granted accommodations cheaters, then another person responds by calling that person a self-centered bigot. If the interlocutors decide to reason at all, they mostly do so by providing their own vague standard for fairness, without either side explaining why their standard is superior to the other’s.

Here I would like to articulate the standard of fairness common in disability law and psychometric testing and briefly defend it. I will also defend the thesis that this standard leaves many commonplace accommodations in a gray area.

Articulating the Standard

A DOJ provision derived from the ADA offers a concise statement of the kind of standard I would like to defend:

“The examination is selected and administered so as to best ensure that, when the examination is administered to an individual with a disability that impairs sensory, manual, or speaking skills, the examination results accurately reflect the individual’s aptitude or achievement level or whatever other factor the examination purports to measure, rather than reflecting the individual’s impaired sensory, manual, or speaking skills (except where those skills are the factors that the examination purports to measure)”

— 28 C.F.R. § 36.309(b)(1)(i)

The only area where I would modify the provision is the list of categories of impairments to be accommodated, to which I would add “cognitive”. Without this modification the provision does not apply to most accommodated disabilities without an unreasonably broad definition of the term “sensory”.

Defending the Standard

This standard is best understood as forcing examiners to act in a way that is prudent in the pursuit of accuracy of assessment. It is not motivated by any substantive sense of what is a just outcome for the individual test-taker. This is a virtue because it is sensitive to the actual purpose of examinations. In the first instance, the purpose of a test is not to confer prestige or shame, nor to allocate resources. It is to measure aptitude or knowledge in a specified area. Concerns about group inequality in post-test outcomes may be legitimate, but to bias the test on this basis is to confuse an issue with the use of the measurement for an issue with the measurement itself.

On the other hand, complaints that accommodations are inherently unfair because they violate a ‘level playing field’ principle rely on an overly procedural view of test-taking as an activity. In general, we as society do not force people to take tests because we value the sanctity of the procedure, but to predict ability in a broader domain. This explains why, for example, the forms of medical licensure exams vary among developed countries, but they all have one. It is true that we want a level playing field, but this is for reasons of measurement, and what exactly it means for the playing field to be level is a function of what is measured.

Clear and Penumbral Cases

The principal problem in the application of the standard is that what most tests purport to measure is defined loosely, and in the case of cognitive disability the interpretation of those loose definitions usually turns on a modular view of the mind which is itself vague. I’ll end with an examination of two cases, one clear and one that sits in the large gray area.

Case 1: Dyslexia on a Math Test

Suppose the explicit goal of a typical math test is to measure understanding of the classroom material. In the context of a mathematics examination, the material in question generally falls under the umbrella of quantitative reasoning. Dyslexia, a cognitive impairment affecting the decoding of written language, introduces a confounding variable when mathematical concepts are presented via word problems. It is easy in practice to provide accommodations that would only seem to assist the easy-to-isolate word-decoding module, and empirically we observe that dyslexic test-takers can greatly benefit from interventions that would be meaningless for the neurotypical, increasing our confidence that language decoding is a real (whatever that means) module which plays no role in quantitative reasoning. Because the examination does not purport to measure phonetic decoding or reading speed, the accommodation ensures the results accurately reflect the individual’s mathematical achievement rather than their cognitive impairment, making this a clear and unproblematic application of the standard.

Case 2: ADHD on the LSAT

In the context of the LSAT, an exam which purports to measure reading comprehension and logical reasoning, ADHD presents a penumbral case. The exam-affecting impairments of ADHD are typically attributed to attention and working memory, but it is fundamentally unclear whether an “attention module” can be cleanly separated from the reading comprehension and logical reasoning aptitude the test purports to measure. Furthermore, working memory probably is a direct factor in the skills the test is meant to measure. In practice, it is difficult to know how much of a test-taker’s impairment stems from a deficit in working memory versus a deficit in attention, or indeed, how separable those two constructs actually are.

The specific accommodation usually under consideration here is extra time. Problematically, the empirical literature is not clear about whether extra time benefits ADHD sufferers more than it benefits neurotypicals, casting doubt on the idea that the accommodation targets an underlying difference. However, even if extra time eliminates the influence of an impaired, measurement-irrelevant module, time is a blunt instrument—it also seems to affect measurement relevant factors. Because we cannot determine the extent to which the impaired modules are factors that the test purports to measure and because it’s not clear how those modules could possibly be isolated anyway, the measurement prudence standard doesn’t seem to give clear guidance about how to properly administer the LSAT to someone with ADHD.