Join the Meeting Place for Moms!
Talk to other moms, share advice, and have fun!

(minimum 6 characters)

Moms for Education Moms for Education

Poor ranking on international test misleading about U.S. student performance, Stanford researcher finds

Posted by on Jan. 20, 2013 at 9:35 PM
  • 64 Replies
http://news.stanford.edu/news/2013/january/test-scores-ranking-011513.html




Poor ranking on international test misleading about U.S. student performance, Stanford researcher finds

A comprehensive analysis of international tests by Stanford and the Economic Policy Institute shows that U.S. schools aren't being outpaced by international competition.

BY JONATHAN RABINOVITZ

Socioeconomic inequality among U.S. students skews international comparisons of test scores, finds a new report released today by the Stanford Graduate School of Education and the Economic Policy Institute. When differences in countries' social class compositions are adequately taken into account, the performance of U.S. students in relation to students in other countries improves markedly.

Shutterstock.com

An accurate comparison of nations' test scores must include a look at the social class characteristics of the students who take the test in each country, says Stanford education Professor Martin Carnoy.
The report, What do international tests really show about U.S. student performance?, also details how errors in selecting sample populations of test-takers and arbitrary choices regarding test content contribute to results that appear to show U.S. students lagging.

In conducting the research, report co-authors Martin Carnoy, a professor of education at Stanford, and Richard Rothstein, a research associate at the Economic Policy Institute, examined adolescent reading and mathematics results from four test series over the last decade, sorting scores by social class for the Program on International Student Assessment (PISA), Trends in International Mathematics and Science Study (TIMSS), and two forms of the domestic National Assessment of Educational Progress (NAEP).

Based on their analysis, the co-authors found that average U.S. scores in reading and math on the PISA are low partly because a disproportionately greater share of U.S. students comes from disadvantaged social class groups, whose performance is relatively low in every country.

As part of the study, Carnoy and Rothstein calculated how international rankings on the most recent PISA might change if the United States had a social class composition similar to that of top-ranking nations: U.S. rankings would rise to fourth from 14th in reading and to 10th from 25th in math. The gap between U.S. students and those from the highest-achieving countries would be cut in half in reading and by at least a third in math.

"You can't compare nations' test scores without looking at the social class characteristics of students who take the test in different countries," said Carnoy. "Nations with more lower social class students will have lower overall scores, because these students don't perform as well academically, even in good schools. Policymakers should understand how our lower and higher social class students perform in comparison to similar students in other countries before recommending sweeping school reforms."

The report also found:

There is an achievement gap between more and less disadvantaged students in every country; surprisingly, that gap is smaller in the United States than in similar post-industrial countries, and not much larger than in the very highest scoring countries.
Achievement of U.S. disadvantaged students has been rising rapidly over time, while achievement of disadvantaged students in countries to which the United States is frequently unfavorably compared – Canada, Finland and Korea, for example – has been falling rapidly.
But the highest social class students in United States do worse than their peers in other nations, and this gap widened from 2000 to 2009 on the PISA.
U.S. PISA scores are depressed partly because of a sampling flaw resulting in a disproportionate number of students from high-poverty schools among the test-takers. About 40 percent of the PISA sample in the United States was drawn from schools where half or more of the students are eligible for the free lunch program, though only 23 percent of students nationwide attend such schools.
With each release of international test scores, many education leaders assert that American students are unprepared to compete in the new global economy, largely because of U.S. schools' shortcomings in educating disadvantaged students.

"Such conclusions are oversimplified, frequently exaggerated and misleading," said Rothstein, who is also senior fellow at the Chief Justice Earl Warren Institute of Law and Social Policy at the University of California – Berkeley School of Law. "They ignore the complexity of test results and may lead policymakers to pursue inappropriate and even harmful reforms."

Carnoy and Rothstein examined test results in detail from the United States and six other nations: three of the highest scorers (Canada, Finland and South Korea) and three economically comparable nations (France, Germany and the United Kingdom). In cases where U.S. states voluntarily participated in the TIMSS for 8th grade mathematics, the researchers compared trends in these states' scores with trends in 8th grade mathematics on the NAEP.

The researchers show that score trends on these different tests can be very inconsistent, suggesting need for greater caution in interpreting any single test. For example, declining trends in U.S. average PISA math scores do not track with trends in TIMSS and NAEP, which show substantial math improvements for all U.S. social classes.

Carnoy and Rothstein say that the differences in average scores on these tests reflect arbitrary decisions about content by the designers of the tests. For example, although it has been widely reported that U.S. 15-year-olds perform worse on average than students in Finland in mathematics, U.S. students perform better than students in Finland in algebra but worse in number properties (e.g., fractions). If algebra had greater weight in tests, and numbers less weight, test scores could show that U.S. overall performance was superior to that of Finland.

The report comes as the administrators of TIMSS are preparing to make public more detailed data underlying 2011 test results, following last month's release of average national scores. PISA plans to release detailed data on its 2012 test in December 2013. Carnoy said that he and Rothstein will then be able to supplement their present report by comparing the most recent TIMSS and PISA results by social class across countries. He invites other researchers to conduct similar analyses, to see if their findings confirm those of the present report.

A copy of the report is available to download at http://www.epi.org/publication/us-student-performance-testing/.

Jonathan Rabinovitz is director of communications at the Stanford Graduate School of Education.

Media Contact
Martin Carnoy, Graduate School of Education: (650) 906-7469, carnoy@stanford.edu

Jonathan Rabinovitz, Graduate School of Education: (650) 724-9440, (415) 601-1811, jrabin@stanford.edu
by on Jan. 20, 2013 at 9:35 PM
Add your quick reply below:
You must be a member to reply to this post.
Replies (1-10):
Kris_PBG
by on Jan. 20, 2013 at 9:37 PM
I've heard this before in other studies as well.
mjande4
by Member on Jan. 20, 2013 at 9:41 PM
2 moms liked this

Not to be crass, but DUH!

Kris_PBG
by on Jan. 20, 2013 at 9:51 PM
Quoting mjande4:

Not to be crass, but DUH!




Lol!!!!! No doubt, but so many say how our public schools are just so dismal,especially when compared to others internationally...
tuplamama
by on Jan. 21, 2013 at 12:37 AM
1 mom liked this

 I wonder how this study is done. To what do they compare since for example Finland does not have poverty levels that USA has.

 

 

 


Your clitoris has over 8000 sensitive nerve endings.......... foreskin has over 20 000.


 




 

Kris_PBG
by on Jan. 21, 2013 at 12:41 AM


It is not hard to adjust out the numbers for those in poverty.  I have seen similar studies quite a few times now. 

Quoting tuplamama:

 I wonder how this study is done. To what do they compare since for example Finland does not have poverty levels that USA has.

 

 



tuplamama
by on Jan. 21, 2013 at 12:48 AM
1 mom liked this

 I wasn't clear what I meant. What I meant was that we do not have the type of poor USA has, I was not talking about how many poor there were but the type. Am I making any sense?

 

Quoting Kris_PBG:

 

It is not hard to adjust out the numbers for those in poverty.  I have seen similar studies quite a few times now. 

Quoting tuplamama:

 I wonder how this study is done. To what do they compare since for example Finland does not have poverty levels that USA has.

 

 

 

 

 

 


Your clitoris has over 8000 sensitive nerve endings.......... foreskin has over 20 000.


 




 

Kris_PBG
by on Jan. 21, 2013 at 12:52 AM

It's late for me here and I'm not sure my brain is computing!  LOL!

I'll try again in the am when I have my wits about me!

I was answering in regards to how they compared the data by removing the poverty children in the study - my brain needs until the am to wrap itself around your point!  :)


Quoting tuplamama:

 I wasn't clear what I meant. What I meant was that we do not have the type of poor USA has, I was not talking about how many poor there were but the type. Am I making any sense?

 

Quoting Kris_PBG:


It is not hard to adjust out the numbers for those in poverty.  I have seen similar studies quite a few times now. 

Quoting tuplamama:

 I wonder how this study is done. To what do they compare since for example Finland does not have poverty levels that USA has.

 

 



 



Kris_PBG
by on Jan. 21, 2013 at 12:54 AM


I am curious though - what is the "poor" population like in Finland compared to the US - I don't have a good point of reference for that.

Quoting tuplamama:

 I wasn't clear what I meant. What I meant was that we do not have the type of poor USA has, I was not talking about how many poor there were but the type. Am I making any sense?

 

Quoting Kris_PBG:


It is not hard to adjust out the numbers for those in poverty.  I have seen similar studies quite a few times now. 

Quoting tuplamama:

 I wonder how this study is done. To what do they compare since for example Finland does not have poverty levels that USA has.

 

 



 



tuplamama
by on Jan. 21, 2013 at 1:09 AM
1 mom liked this

 That's my thing in this because in Finland there is no fear for anyone to be without roof over their head, without healthcare or being hungry. By law everyone has to have a certain level of income and if you have kids that income must be higher. So the whole definition of poor is very different here than in USA. Even if you are so called "low income" family or single parent, you don't have to worry about things Americans have to worry. Schools get money/student, not how schools perform. So little village school in Northern Finland is treated equally with urban school in Helsinki, it is about the amount of students, not about grades.

Clarifying also that I have no opinion about the study OP posted, I'm just curious how they are able to compare countries like USA and Finland in this matter.

Quoting Kris_PBG:

 

I am curious though - what is the "poor" population like in Finland compared to the US - I don't have a good point of reference for that.

Quoting tuplamama:

 I wasn't clear what I meant. What I meant was that we do not have the type of poor USA has, I was not talking about how many poor there were but the type. Am I making any sense?

 

Quoting Kris_PBG:

 

It is not hard to adjust out the numbers for those in poverty.  I have seen similar studies quite a few times now. 

Quoting tuplamama:

 I wonder how this study is done. To what do they compare since for example Finland does not have poverty levels that USA has.

 

 

 

 

 

 

 

 

 


Your clitoris has over 8000 sensitive nerve endings.......... foreskin has over 20 000.


 




 

SusanTheWriter
by on Jan. 21, 2013 at 7:33 AM
2 moms liked this

I'm wondering why we should discount students from disadvantaged families in the study. I thought the point was to see how the US *as a whole* did when stacked up against other countries. Sure, take the poor kids out and the numbers look better. It doesn't take a statistician to figure that out. But it artificially skews the numbers higher.

I guess I don't understand the point they're trying to make.

Add your quick reply below:
You must be a member to reply to this post.
Join the Meeting Place for Moms!
Talk to other moms, share advice, and have fun!

(minimum 6 characters)