WASHINGTON, DC, May 7, 2007 -- The American Education Research Association (AERA) has released its latest publication directed to advancing education research and scholarship of the highest quality in this field. Its publication comes as education research faces new challenges and opportunities due to legislation calling for scientifically based research and as new research methods and datasets emerge.
The report, Estimating Causal Effects Using Experimental and Observational Designs, is the product of a think tank of the AERA Grants Program led by authors Barbara Schneider and William Schmidt, both of Michigan State University. Coauthors are Martin Carnoy and Richard Shavelson, both of Stanford University, and Jeremy Kilpatrick of the University of Georgia.
The new report is designed to help researchers, policymakers and funders understand the capacities and limits of examining the causes of educational outcomes with large-scale databases. It is intended to help researchers consider a range of methods designed to establish causality, from experimental to non-experimental designs. For example, the ordinary question "Did A cause B" takes on great significance when applied to education intervention programs. Can small class size, or highly qualified teachers, be shown to have caused increased or decreased learning? With what degree of confidence?
"This report is especially timely as the education research community strives to increase researchers’ capacity to study education problems scientifically and to eliminate fragmented and often unreliable findings," states William H. Schmidt of Michigan State University, chair of the Governing Board of the AERA Grants Program.
Professor Schmidt emphasizes that the report will help address the many questions that researchers and policymakers are confronting about the logic of causality and appropriate research methods for establishing causality. He acknowledges that the report is, in part, a response to recent federal initiatives designed to promote the accumulation of scientific evidence in education that rely on randomized controlled trials, which apply rigorous, systematic, and objective methods to obtain reliable and valid knowledge.
As part of the AERA Report, Barbara L. Schneider of Michigan State University, an AERA Grants Board member and the report’s senior author, points out that the report is not intended to be a "how to" manual for designing research studies or analyzing experimental or observational data. However, it is intended to help researchers, educators, and policymakers understand causal estimation by describing the logic of causal inference and reviewing designs and methods that allow researchers to draw causal inferences about the effectiveness of educational interventions.
The report specifically:
- Considers key issues involved in selecting research designs that allow researchers to draw valid causal inferences about treatment effects using large-scale observational datasets.
- Addresses why key issues of establishing causal inference are of particular interest to education researchers, briefly explains how causality is commonly defined in scholarly literature, and describes tools that analysts use to approximate randomized experiments with observational data.
- Reviews four National Science Foundation-funded studies which illustrate the difficulties of and possibilities for making causal inferences when conducting studies focused on significant education issues.
The authors have provided decision rules specific to the evaluation of studies based on large-scale, nationally representative datasets. In their report, they conclude that "existing large-scale datasets remain a rich resource for descriptive statistics on nationally representative samples and of students and subgroups….However, these datasets have been underutilized for purposes of study replication. Properly analyzed, they present cost-effective alternatives for addressing causal questions about the effectiveness of educational interventions."
The AERA volume was released recently during AERA’s 88th Annual Meeting at an Association-wide session, "How Large-Scale Data Sets Can Address Causal Questions in Education—Guidance from a New AERA Report." At the Annual Meeting, more than 15,700 education researchers from 68 countries attended this Chicago meeting.
Editor’s Note: Established 16 years ago, the AERA Grants Board enhances capacity for conducting quantitative analyses of national and international datasets that have implications for education policy, with a special emphasis on science and mathematics. Through grants from the National Center for Education Statistics, a component of the U.S. Department of Education, and the National Science Foundation, the AERA Grants Board funds pre- and postdoctoral fellows, as well as researchers who pursue questions on the effects of instruction and curricula, organizational practices and policies, and teacher development on student learning, achievement, and educational attainment.
The American Educational Research Association (AERA) is the national interdisciplinary research association for approximately 25,000 scholars who undertake research in education. Founded in 1916, the AERA aims to advance knowledge about education, to encourage scholarly inquiry related to education, and to promote the use of research to improve education and serve the public good. AERA is dedicated to strengthening education research capacity by promoting research of the highest quality, undertaking education and training programs, and advancing sound research and science policy.