A new study offers evidence that simple and inexpensive changes to existing courses can help students learn more effectively.
The study from Rice University and Duke University found that making a few changes to homework assignments in an upper-level undergraduate engineering course at Rice led to improved scores on exams. The study appears this week in the journal Educational Psychology Review.
The findings by a team from Rice's Center for Digital Learning and Scholarship and Duke's Department of Psychology and Neuroscience demonstrate how technology and cognitive science can be combined to develop inexpensive but effective educational changes that required no changes to course curriculum.
"Based on laboratory studies, we know a lot about how people learn," said lead author Andrew Butler, a postdoctoral researcher at Duke. "To test how well some of those cognitive science principles worked in a classroom, we made subtle changes to standard homework practice."
"The results exceeded everyone's expectations," said Rice co-author Richard Baraniuk, the instructor of the upper-level "signals and systems" engineering course where the experiment took place. "These simple changes produced a larger effect than the average improvement for classroom interventions that require a complete overhaul of curricula and/or teaching methods."
In the study, students switched back and forth from week to week between two different styles of homework. One style, which followed the standard practice that Baraniuk has used for years, consisted of one homework assignment per week, which was graded and returned the following week. The second style, which was called the "intervention," incorporated three principles from cognitive science that have been shown to promote learning and increase long-term retention.
The principles were implemented in the following way:
1. Repeated retrieval practice – In addition to receiving the standard homework assignment, students were given follow-up problems on the same topic in two additional assignments that counted only toward their course participation grade.
2. Spacing -- Rather than giving all the problem sets for a week's lectures in one assignment, the researchers spaced the problems over three weeks of assignments.
3. Feedback -- Rather than waiting one week to learn how they did, students received immediate feedback on intervention homework, and they were required to view the feedback to get credit for the assignment.
"Giving students multiple opportunities to practice retrieving and applying their knowledge on new problems is a very powerful way to promote learning, especially when this practice is spaced out over time," said study co-author Elizabeth Marsh, associate professor of psychology and neuroscience at Duke. "Feedback also is critical to learning, and previous studies have shown that students will often skip looking at feedback."
To alleviate any concerns that individual differences in ability might skew the study results, the researchers split the class into two groups and assigned each group standard homework and intervention homework during alternating weeks; in any given week, half of the students were assigned to the intervention and half to the standard practice homework.
The course covered 11 broad topics and approximately five core concepts per topic. Questions on the midterm and final exams covered specific concepts and topics, so by comparing how the two groups fared on those questions, the research team could compare whether students learned more doing one style of homework or the other. The research showed that students scored about 7 percent higher on the portions of the final exams that were taught with the intervention.
"In any experiment, it's important to try to minimize the random sources of variance that might undermine the ability to detect whether an intervention had a significant effect," said Baraniuk, the Victor E. Cameron Professor of Engineering in the Department of Electrical and Computer Engineering and director of Rice's RDLS. "In signal processing, we refer to this as 'noise,' and classrooms are inherently 'noisy' because each student will engage with the material in many different ways.
"Besides homework assignments, students can also learn from lectures, do extra reading, go to review sessions and the like," he said. "In general, the 'noise' created by all this outside learning would be expected to reduce or eliminate the kind of positive effects that we measured, so it's noteworthy that we did not control for noise and still saw a significant effect."
The study showed that the combination of repeated retrieval practice, spacing and immediate, required feedback had a powerful effect on student learning of complex engineering material.
Marsh said further research is needed to determine whether the results are broadly applicable across disciplines and grade levels, but the researchers made a conscious effort to design the study using broadly applicable methods.
"We aimed for changes that could easily be applied across disciplines and grade levels with minimal cost and disruption," she said.
The software used in the test was developed in Baraniuk's lab and is an academic research tool that has many of the same features as powerful learning aids that are currently on the market. These tools, which are sometimes called "digital tutors," can be used as study aids or as platforms for administering homework and tests. In addition to collecting metrics that instructors can use to evaluate individual students or entire course sections, the most sophisticated tutors are "intelligent" in that they learn about each student and tailor instruction for each individual.
"We could have implemented these same principles in the classroom without technology, but the digital tutor made it much easier," Butler said. "Moreover, technology has the potential to implement these principles in a more powerful way by providing personalized instruction to each student."
Marsh, Butler and Baraniuk will present details of the study at Rice's second annual Personalized Learning Workshop April 2 in Houston. The workshop will focus on how the science of learning can inform the development of personalized learning systems and transform learning in dynamic and complicated educational environments. The scope of the workshop will encompass all levels of learning from pre-K to postcollege. For webcast details, visit http://conference.cnx.org/.
The research was funded by the National Science Foundation and by Google's Faculty Research Award program. Rice research engineer J.P. Slavinsky was a co-author.
High-resolution IMAGES are available for download at:
http://news.rice.edu/wp-content/uploads/2014/03/0318_COGSCI-grades-orig.jpg
CAPTION: A study from Rice University and Duke University has found intriguing evidence that simple and inexpensive changes to existing courses can help students learn more effectively.
CREDIT: Rice University
http://news.rice.edu/wp-content/uploads/2014/03/0318_COGSCI-baraniuk-lg.jpg
CAPTION: Rich Baraniuk
CREDIT: Rice University
http://news.rice.edu/wp-content/uploads/2014/03/0318_COGSCI-marsh-lg.jpg
CAPTION: Elizabeth Marsh
CREDIT: Duke University
http://news.rice.edu/wp-content/uploads/2014/03/0318_COGSCI-butler-lg.jpg
CAPTION: Andrew Butler
CREDIT: Duke University
A copy of the Educational Psychology Review paper is available at: http://link.springer.com/article/10.1007/s10648-014-9256-4
This release can be found online at news.rice.edu.
Follow Rice News and Media Relations via Twitter @RiceUNews
Located on a 300-acre forested campus in Houston, Rice University is consistently ranked among the nation's top 20 universities by U.S. News & World Report. Rice has highly respected schools of Architecture, Business, Continuing Studies, Engineering, Humanities, Music, Natural Sciences and Social Sciences and is home to the Baker Institute for Public Policy. With 3,920 undergraduates and 2,567 graduate students, Rice's undergraduate student-to-faculty ratio is 6.3-to-1. Its residential college system builds close-knit communities and lifelong friendships, just one reason why Rice has been ranked No. 1 for best quality of life multiple times by the Princeton Review and No. 2 for "best value" among private universities by Kiplinger's Personal Finance.
Journal
Educational Psychology Review