Meta-Analysis

Meta-analysis is a statistical method for aggregating research findings across many studies of the same question (Hedges and Becker, 1986). It is ideal for synthesizing research on gender differences, an area in which often dozens or even hundreds of studies of a particular question have been conducted.

Crucial to meta-analysis is the concept of effect size, which measures the magnitude of the effect—in this case, the magnitude of the gender difference. In gender meta-analyses, the measure of effect size typically is d (Cohen, 1988).

where M_M is the mean score for males, M_F is the mean score for females, and s_w is the within-sex standard deviation. That is, d measures how far apart the male and female means are, in standardized units. In meta-analysis, the effect sizes computed from all individual studies are then averaged to obtain an overall effect size reflecting the magnitude of gender differences across all studies. Here I follow the convention that negative values of d mean that females scored higher and positive values of d indicate that males scored higher.

Although there is some disagreement among experts, a general guide is that an effect size d of 0.20 is a small difference, a d of 0.50 is moderate, and a d of 0.80 is a large difference (Cohen, 1988). As an example of a large effect, for the gender difference in throwing distance, d = +1.98 (Thomas and French, 1985).

Meta-analyses generally proceed in three steps: (1) The researchers locate all studies on the topic being reviewed, typically using databases such as PsychINFO and carefully chosen search terms. (2) Statistics are extracted from each report and an effect size is computed for each study. (3) An average of the effect sizes is computed to obtain an overall assessment of the direction and magnitude of the gender difference when all studies are combined.

Conclusions based on meta-analyses are almost always more powerful than conclusions based on an individual study, for two reasons. First, because meta-analysis aggregates over numerous studies, a meta-analysis typically represents the testing of tens of thousands—sometimes even millions—of participants. As such, the results should be far more reliable than those from any individual study. Second, findings from gender differences research are notoriously inconsistent across studies. For example, in the meta-analysis of gender differences in mathematics performance discussed later in this paper, 51% of the studies showed males scoring higher, 6% showed exactly no difference between males and females, and 43% showed females scoring higher (Hyde, Fennema, and Lamon, 1990). This makes it very easy to find a single study that supports one’s prejudices. Meta-analysis overcomes this problem by synthesizing all available studies.

Gender Differences in Mathematics Performance

A major meta-analysis of studies of gender differences in mathematics performance surveyed 100 studies, representing the testing of more than 3 million persons (Hyde, Fennema, and Lamon, 1990). Averaged over all samples of the general population, d = –0.05, a negligible difference favoring females.

An independent meta-analysis confirmed the results of the first meta-analysis (Hedges and Nowell, 1995). It found effect sizes for gender differences in mathematics performance ranging between 0.03 and 0.26 across large samples of adolescents—all differences in the negligible to small range. Results from the International Assessment of Educational Progress also confirm that gender differences in mathematics performance are small across numerous countries including Hungary, Ireland, Israel, and Spain (Beller and Gafni, 1996).

For issues of the underrepresentation of women in the physical sciences, however, this broad assessment of the magnitude of gender differences is probably less useful than an analysis by both age and cognitive level tapped by the mathematics test. These results from one meta-analysis are shown in Table 2-1. Ages were grouped roughly into elementary school (ages 5-10 years), middle school (11-14), high school (15-18), and college age (19-25). Insufficient studies were available for older ages to compute mean effect sizes. Cognitive level of the test was coded as assessing either simple computation (requires the use of only memorized math facts, such as 7 × 8 = 56), conceptual (involves analysis or comprehension of mathematical ideas), problem solving (involves extending knowledge or applying it to new situations), or mixed. The results indicated that girls outperform boys by a small margin in computation in elementary school and middle school and there is no gender difference in high school. For understanding of mathematical concepts, there is no gender difference at any age level. For problem solving there is no gender difference in elementary or middle school, but a small gender difference favoring males emerges in high school and the college years. There are no gender differences, then, or girls perform better, in all areas except problem solving beginning in the high school years.

TABLE 2-1

The Magnitude of Gender Differences in Mathematics Performance as a Function of Age and Cognitive Level of the Test.

This gender difference in problem solving favoring males deserves attention because problem solving is essential to success in occupations in engineering and the physical sciences. Perhaps the best explanation for this gender difference, in view of the absence of a gender difference at earlier ages, is that it is a result of gender differences in course choice, i.e., the tendency of girls not to select optional advanced mathematics courses and science courses in high school. The failure to take advanced science courses may be particularly crucial because mathematics curricula often do not teach problem solving, whereas it typically is taught in chemistry and physics.

Gender Differences in Verbal Ability

A meta-analysis of studies of gender differences in verbal ability indicated that, overall, the difference was so small as to be negligible, d = –0.11 (Hyde and Linn, 1988). The negative value indicates better performance by females, but the magnitude of the difference is quite small. There are many aspects to verbal ability, of course. When analyzed according to type of verbal ability, the results were as follows: for vocabulary, d = –0.02; for reading comprehension d = –0.03; for speech production d = –0.33; and for essay writing d = –0.09. The gender difference in speech production favoring females is the largest and confirms females’ better performance on measures of verbal fluency (not to be confused with measures of talking time). The remaining effects range from small to zero. Moreover, the magnitude of the effect was consistently small at all ages. Overall, then, gender difference in verbal ability are tiny and, if anything, favor females on measures such as essay writing and speech production, which should contribute to success in science. A second meta-analysis confirmed these findings using somewhat different methods (Hedges and Nowell, 1995).

Gender Differences in Spatial Ability

Spatial ability tests may tap any of several distinct skills: spatial visualization (finding a figure in a more complex one, like hidden-figures tests), spatial perception (identifying the true vertical or true horizontal when there is distracting information, such as the rod-and-frame task), and mental rotation (mentally rotating an object in 3 dimensions). Two meta-analyses are available on the question of gender differences in spatial performance. One found that the magnitude of gender differences varied substantially across the different types of spatial performance: d = 0.13 for spatial visualization, 0.44 for spatial perception, and 0.73 for mental rotation, all effects favoring males (Linn and Peterson, 1985). The last difference is large and potentially influential. The other meta-analysis found d = 0.56 for mental rotation (Voyer, Voyer, and Bryden, 1995), a somewhat smaller effect but nonetheless a substantial one. Gender differences in spatial performance—specifically, mental rotation—are important because mental rotation is crucial to success in several fields of engineering, chemistry, and physics (Hegarty and Sims, 1994).

Sociocultural Influences on Gender Differences in Mathematical and Spatial Abilities

The evidence on social and cultural influences on gender differences in mathematical and spatial abilities is plentiful and varied. I consider three categories of evidence: research on family and school influences, training studies, and cross-cultural analyses.

Family and School Influences

Abundant evidence exists for the multiple influences of parents and the schools on children’s development. Here I focus on these influences specifically in the domains of abilities and academic performance. A limitation to some of these studies is that they report simply a correlation, for example, between parents’ estimates of the child’s mathematics ability and the child’s score on a standardized test. From this correlation, we cannot infer the direction of causality with complete certainty. We cannot tell whether the parents’ beliefs in the child influence the child’s performance or whether the opposite process occurs—that children’s test scores influence their parents’ estimates of abilities. Moreover, it may be that both processes occur.

Numerous studies have confirmed the finding that parents’ expectations for their children’s academic abilities and success predict the children’s self-concept of their own ability and their subsequent performance (e.g., Bleeker and Jacobs, 2004; Eccles, 1994). When engaged in a science task—playing with magnets— mothers talk about the science process (e.g., use explanations, generate hypotheses) more with boys than with girls (Tenenbaum et al., 2005). Moreover, the amount of mothers’ science-process talk predicts children’s comprehension of readings about science 2 years later. Observations of parents and children using interactive science exhibits at a museum showed that parents were three times more likely to explain science to boys than to girls (Crowley et al., 2001). Girls essentially grow up in a different family science environment than boys do.

Schools may exert their influence in multiple ways, including teachers’ attitudes and behaviors, curriculum, ability grouping, and sex composition of the classroom. The availability of hands-on laboratory experiences is especially critical for learning in the physical sciences in middle school and high school. An important point is that, although laboratory experiences do not improve the physical science achievement of boys, they do improve the achievement of girls, thereby helping to close the gender gap in achievement in the physical sciences (Burkam, Lee, and Smerdon, 1997; Lee and Burkam, 1996). In science and mathematics classes, teachers are more likely to encourage boys than girls to ask questions and to explain (American Association of University Women, 1995; Jones and Wheatley, 1990; Kelly, 1988). In one study of high school geometry classrooms, teachers directed 61% of their praise comments to boys and 55% of their high-level open questions to boys (Becker, 1981). Experiences such as these are thought to give children a deeper conceptual knowledge of and more interest in science.

Students also exercise choice in school activities. Crucial to this discussion is their choice in high school to take advanced mathematics and science courses. The gender gap in mathematics course taking has narrowed over the last decade, so that by 1998 girls were as likely as boys to have taken advanced mathematics courses, including AP/IB calculus (National Science Foundation, 2005). Girls were actually slightly more likely than boys to take advanced biology (40.8% of girls, 33.8% of boys), AP biology (5.8% of girls, 5.0% of boys), and chemistry (59.2%, 53.3%). Boys, however, were more likely to take AP chemistry (3.3% of boys, 2.6% of girls) and physics (31% of boys, 26.6% of girls), and were twice as likely to take AP physics (2.3% of boys, 1.2% of girls) (National Science Foundation, 2005). The science pipeline heading toward physics, then, begins to leak early as fewer girls take the necessary high school courses to prepare themselves for college-level physics. It is beyond the scope of this article to review what psychologists know about the reasons why adolescents choose or do not choose to take challenging math and science courses. Readers wanting more information can look to a massive program of research conducted by Eccles (e.g., Eccles, 1994).

Training Studies

Environmental input is essential to the development of spatial and mathematical abilities (Baenninger and Newcombe, 1995; Newcombe, 2002; Spelke, 2005). Babies are not born knowing how to work calculus problems. Children acquire these skills through schooling and other experiences.

A meta-analysis found that spatial ability can indeed be improved with training, with effect sizes ranging between d = 0.40 to 0.80, depending on the length and specificity of the training (Baenninger and Newcombe, 1989). The effects of training were similar for males and females; that is, both groups benefited about equally from the training, and there was little evidence that the gender gap was closed or widened by training. A more recent study showed that the gender difference could be eliminated by carefully conceptualized training (Vasta et al., 1996). Unfortunately, most school curricula contain little or no emphasis on spatial learning. Girls, especially, could benefit greatly from such a curriculum.

The most recent development is multimedia software that provides training in 3-dimensional spatial visualization skills (Gerson, Sorby, Wysocki, and Baartmans, 2001). It has been used successfully with first-year engineering students. Most notably for the topic under discussion, there were improvements in the retention of women engineering students who took the spatial visualization course; without the course, the retention rate for women was 47%, whereas with the course it was 77%.

Cross-Cultural Analyses

The International Assessment of Educational Progress (IAEP) tested the math and science performance of 9- and 13-year-olds in 20 nations around the world. The effect sizes for gender differences for selected countries are shown in Table 2-2 (Beller and Gafni, 1996). Focusing first on the results for mathematics, we see that the gender differences are small in all cases. Most importantly, effect sizes are positive (favoring males) in some countries, negative (favoring females) in other countries, and several are essentially zero. The Trends in International Mathematics and Science Study (TIMSS, 2003, formerly the Third International Mathematics Study) found similar results, with some positive and some negative effect sizes, and most < 0.10. In the TIMSS data for eighth graders, the magnitude of the gender difference was 0.09 in Chile (country average score 379), 0.02 in the United States (country average 502), 0.01 in Japan (country average 569), and –0.05 in Singapore (country average 611). That not only the magnitude, but also the direction of gender differences in mathematics performance varies from country to country is powerful testimony to the importance of sociocultural factors in shaping those differences. Perhaps most importantly, though, the gender difference is very small in most nations.

TABLE 2-2

Effect Sizes for Gender Differences in Mathematics and Science Test Performance Across Countries.

Focusing next on the results for science performance (Table 2-2), we can see that the effect sizes more consistently favor males and are somewhat larger, although not large for any nation. When the results are broken down by science, gender differences are smaller in life sciences knowledge (0.11 and 0.20 at ages 9 and 13, respectively, averaged over all countries) and somewhat larger for physical sciences (0.22 and 0.33) (Beller and Gafni, 1996).

It is important to note that cross-cultural differences in mathematics performance are enormous compared with gender differences in any one country. For example, in one cross-national study of 5th graders, American boys (M = 13.1) performed better than American girls (M = 12.4) on word problems, but 5th grade Taiwanese girls (M = 16.1) and Japanese girls (M = 18.1) performed far better than American boys (Lummis and Stevenson, 1990). Culture is considerably more important than gender in determining mathematics performance.

In perhaps the most sophisticated analysis of cross-national patterns of gender differences in mathematics performance, the researchers found that, across nations, the magnitude of the gender difference in mathematics performance for eighth graders correlated significantly with a variety of measures of gender stratification in the countries (Baker and Jones, 1993). For example, the magnitude of the gender difference in math performance correlated –0.55, across nations, with the percentage of women in the workforce in those nations. That is, the more that women participate in the labor force (an index of gender equality), the smaller the gender difference in mathematics achievement.

The Gender Similarities Hypothesis

I propose an alternative to our cultural and scientific obsession with gender differences. The alternative is the Gender Similarities Hypothesis, which I formalized in an article that appeared in the American Psychologist this year (Hyde, 2005). For that paper, I essentially meta-analyzed meta-analyses. That is, I found all the meta-analyses of psychological gender differences that I could. I found 46 relevant meta-analyses, and from them I extracted 124 effect sizes—d’s—for gender differences. The meta-analyses spanned a wide range of psychological characteristics, including abilities, communication, aggression, leadership, personality, and self-esteem.

I organized those 124 effect sizes into ranges— those that are close to zero, i.e., in the range 0 to 0.10, those that are small, 0.11 to 0.35, those that are moderate in magnitude, 0.36 to 0.65, those that are large, 0.66 to 1.00, and those that are very large, > 1.00. The results indicated that 30% of those effect sizes were in the close-to-zero range, and another 48% were small. So, 78% of the effect sizes were small or close to zero—that is the gender similarities hypothesis—psychologically, women and men are more similar than they are different. There are a few exceptions of large differences, but the big picture is one of gender similarities.

Implications: How Can We Close the Gender Gap in Engineering and the Physical Sciences?

One conclusion of this review is that, overall, there are no gender differences in math performance, but a gender difference favoring males in complex problem solving does emerge in high school. Mathematical problem solving is crucial to success in the physical sciences, so this gap must be addressed. The evidence also indicates a gender gap in favor of males in spatial ability, specifically in mental rotation. This ability, too, is crucial to success in the physical sciences and must be addressed.

The following policy recommendations flow from the data reviewed here:

1. Focusing on the gender difference in spatial skill, we need to institute a spatial learning curriculum in the schools. Girls are seriously disadvantaged by its absence.
2. Colleges of engineering should have a spatial skills training program for entering students. Theoretically, such a program should help in physics and chemistry as well.
3. We should require 4 years of math and 4 years of science in high school— or at least require it for university admission. Otherwise, girls will elect not to take some advanced science courses and, without carefully making the decision, close themselves out of outstanding careers in engineering and the sciences.
4. The mathematics curriculum in many states continues to need attention. It needs far more emphasis on real problem solving, and that approach will benefit not only girls, but boys as well.
5. Hands-on science labs will benefit girls and help to close the gender gap. And, they represent good science education practice.
6. Teachers and high-school guidance counselors need to be educated about the findings on gender similarities in math performance. Otherwise, teachers will believe the stereotypes about girls’ math inferiority that pervade our culture, the teachers will have lower expectations for girls’ math performance, and those expectations will convey themselves to the students.

If we do all this—and much more—we can all look forward to a day when girls and women will have equal access to careers in engineering and the sciences. And our nation will benefit from maximizing women’s contributions.

References

1. American Association of University Women . How schools shortchange girls. Washington, DC: AAUW; 1995.
2. Baenninger M, Newcombe N. The role of experience in spatial test performance: A meta-analysis. Sex Roles. 1989;20:327–344.
3. Baenninger M, Newcombe N. Environmental input to the development of sex-related differences in spatial and mathematical ability. Learning and Individual Differences. 1995;7:363–379.
4. Baker DP, Jones DP. Creating gender equality: Cross-national gender stratification and mathematical performance. Sociology of Education. 1993;66:91–103.
5. Becker JR. Differential treatment of females and males in mathematics classes. Journal for Research in Mathematics Education. 1981;12:40–53.
6. Beller M, Gafni N. The 1991 International Assessment of Educational Progress in Mathematics and Sciences: The gender differences perspective. Journal of Educational Psychology. 1996;88:365–377.
7. Bleeker MM, Jacobs JE. Achievement in math and science: Do mothers’ beliefs matter 12 years later? Journal of Educational Psychology. 2004;96:97–109.
8. Burkam DT, Lee VE, Smerdon BA. Gender and science learning early in high school: Subject matter and laboratory experiences. American Educational Research Journal. 1997;34:297–331.
9. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum; 1988.
10. Crowley K, Callanan MA, Tenenbaum HR, Allen E. Parents explain more often to boys than to girls during shared scientific thinking. Psychological Science. 2001;12:258–261. [PubMed: 11437311]
11. Eccles JS. Understanding women’s educational and occupational choices: Applying the Eccles et al. model of achievement-related choices. Psychology of Women Quarterly. 1994;18:585–610.
12. Gerson H, Sorby SA, Wysocki A, Baartmans BJ. The development and assessment of multimedia software for improving 3-D spatial visualization skills. Computer Applications in Engineering Education. 2001;9:105–113.
13. Handelsman J, Cantor N, Carnes M, Denton D, Fine E, Grosz B, Hinshaw V, Marrett C, Rosser S, Shalala D, Sheridan J. More women in science. Science. 2005;309:1190–1191. [PubMed: 16109868]
14. Hedges LV, Becker BJ. Statistical methods in the meta-analysis of research on gender differences. In: Hyde JS, Linn MC, editors. The psychology of gender: Advances through meta-analysis. Baltimore: Johns Hopkins University Press; 1986. pp. 14–50.
15. Hedges LV, Nowell A. Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science. 1995;269:41–45. [PubMed: 7604277]
16. Hegarty M, Sims VK. Individual differences in mental animation during mechanical reasoning. Memory and Cognition. 1994;22(4):411–430. [PubMed: 7934947]
17. Hyde JS. The gender similarities hypothesis. American Psychologist. 2005;60:581–592. [PubMed: 16173891]
18. Hyde JS, Fennema E, Lamon SJ. Gender differences in mathematics performance: A meta-analysis. Psychological Bulletin. 1990;107:139–155. [PubMed: 2138794]
19. Hyde JS, Linn MC. Gender differences in verbal ability: A meta-analysis. Psychological Bulletin. 1988;104:53–69.
20. Jones MG, Wheatley J. Gender differences in teacher-student interactions in science classrooms. Journal of Research in Science Teaching. 1990;27:861–874.
21. Kelly A. Gender differences in teacher-pupil interactions: A meta-analytic review. Research in Education. 1988;39:1–23.
22. Lee VE, Burkam DT. Gender differences in middle grade science achievement: Subject domain, ability level, and course emphasis. Science Education. 1996;80:613–650.
23. Linn MC, Petersen AC. Emergence and characterization of sex differences in spatial ability: A meta-analysis. Child Development. 1985;56:1479–1498. [PubMed: 4075870]
24. Lummis M, Stevenson HW. Gender differences in beliefs and achievement: A cross-cultural study. Developmental Psychology. 1990;26:254–263.
25. Hill Susan T. National Science Foundation. Science and engineering degrees, 1966-2000. Arlington, VA: National Science Foundation; 2002. http://ww​.nsf.gov/sbe/srs/stats.htm.
26. National Science Foundation. Science and engineering indicators 2004: Elementary and secondary education, mathematics and science coursework and student achievement. Arlington, VA: National Science Foundation; 2005. http://www​.nsf.gov/statistics/seind04/
27. Newcombe NS. The nativist-empiricist controversy in the context of recent research on spatial and quantitative development. Psychological Science. 2002;13:395–401. [PubMed: 12219804]
28. Spelke E. Sex differences in intrinsic aptitude for mathematics and science? A critical review. American Psychologist. 2005;60(9):950–958. [PubMed: 16366817]
29. Tenenbaum HR, Snow CE, Roach KA, Kurland B. Talking and reading science: Longitudinal data on sex differences in mother-child conversations in low-income families. Applied Developmental Psychology. 2005;26:1–19.
30. Thomas JK, French KE. Gender differences across age in motor performance: A meta-analysis. Psychological Bulletin. 1985;98:260–282. [PubMed: 3901062]
31. Trends in International Mathematics and Science Study (TIMSS). 2003. http://nces​.ed.gov/pubs2005/timss03.
32. Vasta R, Knott JA, Gaze CE. Can spatial training erase the gender differences on the water-level task? Psychology of Women Quarterly. 1996;20:549–568.
33. Voyer D, Voyer S, Bryden MP. Magnitude of sex differences in spatial abilities: A meta-analysis and consideration of critical variables. Psychological Bulletin. 1995;117:250–270. [PubMed: 7724690]

Footnotes

*

Paper presented at the National Academies Convocation on Maximizing the Success of Women in Science and Engineering: Biological, Social, and Organizational Components of Success, held December 9, 2005, in Washington, DC. Preparation of this paper was supported in part by the National Science Foundation, Grant REC 0207109.