On the first national assessment of technology and engineering skills, 8th grade girls scored higher on average than 8th grade boys, while suburban and rural students significantly outscored their peers in cities.
But those gaps were significantly smaller than the familiar gaps between wealthier students and their less-affluent peers, and among racial and ethnic groups, that often crop up on national assessments in other subject areas.
The technology and engineering results come from the first new test in a decade from the National Assessment of Educational Progress, also known as NAEP or the “nation’s report card.” The Technology and Engineering Literacy exam, or TEL, was piloted in 2013 and administered to 21,500 students in approximately 840 public and private schools around the country in 2014.
SOURCE: National Center for Education Statistics
The test was designed to give educators and policymakers a glimpse of students’ skills in what William Bushaw, the executive director of the National Assessment Governing Board, which sets policy for NAEP, referred to as the “T and E in STEM.”
In addition to gauging problem-solving skills, the new test also surveyed students about their in- and out-of-school experiences with technology, engineering, and problem-solving.
The TEL is a harbinger of elements to come in future NAEP exams. Rather than answering a series of multiple-choice questions or writing essays, the 8th grade test-takers worked through a series of virtual scenarios aimed at testing their problem-solving abilities and their ability to use information about technology and engineering. Officials said similar scenarios would likely appear on NAEP in the future. The test was also the first entirely digital NAEP. The National Center for Education Statistics, which administers Naep, plans to administer all its tests digitally by 2017. The NCES avoided some of the technical difficulties that have plagued other computer-based tests by bringing its own Internet service and devices into schools, according to Peggy Carr, the NCES acting commissioner.
Insights From Home and School
Overall, 43 percent of the tested 8th graders scored proficient or above on the test. These results will be a benchmark for future administrations of the TEL. (The next is slotted for 2018 and also will be given only to 8th graders.)
SOURCE: National Center for Education Statistics
Carr drew attention to the fact that 45 percent of female students scored proficient or advanced on the test, compared to 42 percent of their male peers. Carr said that made a strong case that “girls are better able to understand and evaluate technology and then use it to solve problems and achieve goals.” On a call with reporters, she said the results counter conventional skepticism about the technology skills of girls and women.
Other observers said such results shouldn’t come as a shock. Stacie Gregory, a research fellow at the American Association of University Women who has studied women in engineering, noted, for example, that while girls and women are underrepresented in top STEM classes and fields, research shows that “the reasons they leave have very little to do with academic performance.”
“It’s often social pressures,” she said.
Jack Buckley, the senior vice president for research at the College Board and a former NCES commissioner, noted that girls had outscored boys on a 2009 NAEP science test that involved simulated tasks. He said the complexity of the TEL scenarios also makes it difficult to determine exactly what skills were being measured. The TEL included questions on everything from information and communication technology to the role of technology in society.
In and Out Of School
The breakdown by geographic area also yielded striking differences in scores: Suburban and rural students scored a full 10 and 8 scale-score points higher on average than students in cities (154 and 152 out of 300, compared to 144 for urban students).
But gaps between socioeconomic and racial groups outstripped the gender and geographic differences: More-affluent students scored 28 points higher on average than lower-income students (163 points out of 300, compared to 135). And while Asian and white students scored 160 points on average, the average Latino student scored 138 points and the average black student, 128 points.
NAGB’s Bushaw said that when considering gaps between other groups of students, it is also worth exploring “gaps of opportunity” for students.
According to the results, 48 percent of students were not taking any technology, computer, engineering, or industrial technology course. But about two-thirds said they had studied the topics in science class.
Many students reported out-of-school experiences in problem-solving: Eighty-seven percent of students said that they had at some point figured out what was wrong with something in order to fix it. Two-thirds said that what they knew about problem-solving and fixing things had been taught by family members, not at school.
And students who had regular out-of-school experiences with learning about technology had higher scores overall, the report notes.
The National Assessment Governing Board spent more than two years developing a framework for what kinds of skills and abilities should be tested in a technology and engineering test. The framework focused on three main topics: technology and society; design and systems; and information and communication technology.
As they take the test, students work through multistep scenarios that range from creating a historically accurate museum exhibit about a drought to developing safe bike lanes. Students are provided with background knowledge before they are asked to complete the task, and are scored separately on each step.
Not everyone has confidence in the new approach to test design, though. Tom Loveless, a senior fellow with the nonprofit Brookings Institution, said he is skeptical of a test that tries to measure skills like problem-solving outside of a particular discipline.
“I question the value of what it’s measuring, … and I wonder if they’re measuring what they think they’re measuring,” he said.
Carr said the NCES plans to introduce similar scenario-based tasks on its other exams in the near future; the first is likely to be in social studies or history.
