Language Module

Joshua K. Harshorne and Michael T. Ullman, “Why girls say ‘holded’ more than boys,” Developmental Science 9:1 (2006): 21-32. [PDF]

One of the reasons I feel confident that the pendulum has started its swing back toward symbolic approaches in language research is that the recutionist crowd now regularly engages in all the reckless conclusion by assumption on which they (rightly, in many cases) originally based their criticisms of the symbolic approach. The paper reviewed here is as brazen an example of Asserting the Consequent as one is likely to find in which the authors still bother to collect data.

The overall problem is this: a series of recent studies have shown that in general females outperform males on verbal memory tasks such as recalling words from a list. For this reason, we might expect young girls to overgeneralize less often than boys when producing past tense forms. That is, we might expect that girls would be less likely to produce the ungrammatical holded in place of the grammatical held, and similarly for similar examples. The basis of such a hypothesis is the intuitive belief that regular forms are produced by a rule (e.g. of the form “add -ed to a stem to form the simple past tense), whereas irregular forms must simply be memorized. This is an appealing notion primarily for reasons of memory efficiency: while there is perhaps a performance gain in memorizing frequent regular forms for rapid retrieval (e.g. worked), it seems a waste of brain space to bother explicitly storing multiple forms for infrequent items (e.g. pardon and pardons and pardoned) when the manifest regularity of the lexicon provides such an obvious optimization opportunity.

Nothing, of course, can be asserted without confirmation in science, and the researchers found, in the course of trying to document this assumption, that in fact just the opposite was the case. It seems girls are significantly more likely than boys to produce the overregularized forms, even addressing all the obvious confounds (age of speaker, priming by adult conversation partner, token frequency of use, number of utterances produced, etc.). This obviously poses something of a puzzle. Either the studies showing female superiority in verbal memory are flawed, or the relative inferiority at the task in question is a clue to the mechanism behind female superiority in verbal memory tasks generally.

Taking the second route, the authors hypothesize that if the girls have greater associative memory skills - at least for linguistic forms - they may in fact be producing generalizations on that basis which are extended to forms which should not be generalized. That is, forms like folded get the in way for girls, who have generally greater facility in retrieving them, when trying to produce held than they do for boys.

This yields a testable hypothesis. If such interference is in fact occuring, then it should be predicted by neighborhood effects: items that “sound like” many other regular forms should be more likely to interfere than those that “sound like” comparatively fewer regular forms. Defining “sounds like” gets into controversial territory, of course, so the authors ran tests on three separate interpretations:

Rhyme:

the forms in question rhyme (“sinked” - linked, blinked)

Final Coda:

the forms in question end in the same string (“sinked” - linked, blinked, flunked)

Final Consonant:

the forms in question end in the same phoneme (“sinked” - linked, blinked, flunked, barked)

And since irregulars tend to be monosyllabic, these numbers were calculated twice - once for all relevant forms, once only for monosyllabic forms. In total, six conditions, then.

The correlations were significant and positive for girls in all 6 conditions save one (rhyme measure over monosyllabic verbs - for which it was positive but not significant) - which is to say, girls are more likely to overregularize those irregular verbs for which there are lots of highly frequent similar-sounding regular examples. Boys showed no such correlations at all, let alone significant ones.

There is no reason to raise questions about the data. All of it comes from the widely-available CHILDES Corpus and is therefore easily replicable by anyone who has or is willing to design appropriate software. What is interesting here is not so much what was found but how it is interpreted.

Put crudely, what the authors claim on the basis of these correlations is that girls are generally better than boys at verbal memory tasks, even when they’re not, and that indeed when they’re not it’s because they are. Their greater facility with retrieving stored memory items by association means that girls are more easily confused by regularities in similar-sounding forms (having stored folded and molded, she reasons, on the basis of sound, that there is likely to be a holded, and this short-circuits the retrieval of the dissimilar held). Boys, who either are not as gifted at these sound-based associations or else are simply not as efficient at retrieving their exemplars, are less likely to be misled by their ability and thus are doomed by their verbal inferiority to retrieve the correct answer more reliably. Something is obviously in need of more explanation.

While there is no basis for doubting the data, there is perhaps reason to assume that this interpretation is a bit selective. To get the tiresome and obvious out of the way - yes, there is some evidence for politically correct bias. From page 30:

However, we are not claiming that females depend only on lexical memory for processing complex forms. Even with their excellent memory abilities, females are expected to compose many types of complex forms, including new and lower frequency regulars, and highly complex linguistic representations, including most phrases and sentences (citations).

The word “excellent” stands out here. “Excellent” by what standard? Surely it is the case, as with all such distributions over populations, that there is great variation within groups on level of ability. An individual can have “excellent” memory ability by standing out among his peers; “girls” as a group simply have a somewhat greater tendency averaged over the group to excel in this area than do boys.

But the concern here is not so much with possible politically correct biases as with research biases. Notice the potential explanations we’re ruling out without properly addressing. Foremost is the possibility that it isn’t so much that girls are better at verbal memory as that boys are better at rule-based learning. If indeed rules are in part a method of data compression - the ability to leverage regularities in the lexicon in the reduction of processing load - then the fact that boys store fewer exemplars is a feature and not a bug. “Less efficient at retrieving” (spurious) exemplars may simply be a theoretically-biased way of saying “have internal search engines with greater precision.” Boys’ brains are better-attuned to capturing the real regularities in the lexicon. That’s an important point, since any measure of competence surely concerns itself as much with the appropriateness of the method employed to the task at hand as with realtime performance at that task (though of course the two are related). It’s a bit like calling a computer that evaluates loans on the basis of the applicant’s credit score “equally competent” as a loan adjustor. Given its performance on a certain set of data, it may appear that it is (and given a lazy loan adjustor, it may even be so in a particular case). The reality remains that the computer is using proxy data to approximate the real task. As with credit scores, of course, sound associations are a very good approximation of the actual regularity because they are themselves symptoms of that regularity. The “Final Consonant” condition in that task above, for example, would be explained by a traditional grammarian with reference to productive phonological rules. You know - the past tense morpheme is voiced when it attaches to a stem that ends in a voiced segment, voiceless otherwise, and there’s epenthesis in some cases (or however the official version goes - I’m not a phonologist). Whether or not one believes in such a rule is largely a matter of academic preference, the verdict on which is ultimately dependent on some as-yet-to-be-completed reserach and philosophizing. What is not in question by either side of that debate is that such a rule, if a psychological reality operational in a language, would produce a dataset ripe for exploitation by the kind of “Final Consonant” sound-based associative method described above. One could ape the regularity before he had learned the actual rule in exactly the way that this study suggests that girls have a tendency to do. That the study chooses to phrase its conclusions in terms of a superior verbal ability on the part of girls, rather than a deficit at rule-based learning that is being compensated for by a proxy crutch, owes to a reserach bias that favors reductive explanations. (To see that it is a bias, notice that in the passage quoted above the fact that girls can employ linguistic rules is treated as evidence that they always do. Whereas we are asked to assume that there are differentiated relative levels of ability at associative memory, no such assumption seems to be in play for these authors about rule application.)

Reductionism is a kind of unavoidable disease of science. It is the result of twin concerns, each legitimate in its own right: (1) the need to avoid circularity and (2) the need for a transparent mechanism to underlie our explanations. The first needs no justification: explaining something by naming it is no explanation at all. The second is of course related to the first: we don’t feel that we’ve really understood something unless we can replicate it. Reductionist explanations are often appealing here as a way of capturing noise along with the regularities. But the Turing Test for “thinking” is inappropriate for exactly this reason: testing whether something is human is not the same as testing whether it is conscious; humans have some characteristics that are probably incidental to consciousness per se. If we convince someone that something is human by concentrating on where to build in the pauses/hesitations in its speech, we’ve passed the test without really answering the question. Of course it may turn out that the pauses/hesitations are inevitable consequences of the mechanisms that underly consciousness (in which case modeling them is undeniably useful), but there is no a priori reason to believe so other than assumption. Because of the kind of hair-splitting nonsense that philosophical discussions often produce, I think we’re right to give some weight to operational definitions in science. What concerns me is that they not take the place of real explanation when such is possible. Laboratory word-association tasks are, after all, not real-world linguistic tasks so much as tools for approaching answers to questions about how such real-world tasks are done. It is an error to confound ability on verbal word association tasks in the laboratory with real verbal ability. The one is merely a proxy for the other. The only thing the alternate explanation offered for the phenomenon under discussion - that it is a comparative deficiency in the rule-application abilities of girls rather than in the sound-associative memory of boys that carries the weight of the explanation - has working against it that I can see is that it would tend to associate the bearer with a currently-unfashionable belief in a mental “rule application” mechanism of symbolic flavor. What it has going for it, of course, is that you don’t run into the absurdity of claiming that evidence of one’s lack of linguistic ability shows just how good at language she really is…

To be fair, the authors do note some of these problems as outstanding issues that will need to be addressed by future research. Noting this has, however, not prevented them for titling their paper “Why girls say ‘holded’ more than boys,” as if this were a question they’d answered decisively. More to the point, nor has it prevented them from talking as though they had done so throughout their paper, tempered only by a single caveat near the end. It’s a classic research error. If P then Q, observe Q, conclude, on that basis, P. It’s a named fallacy, fellas.

Tags: Psycholinguistics