In the Nation's Interest

Are You Smarter Than An Italian Baker?

How about an Estonian bookkeeper? Disturbingly, the skills of US adults – and especially the skills of those young millennials who are just entering the workforce – do not stack up well against those of their peers in other developed countries. Their literacy, numeracy, and problem solving skills rank near the bottom for OECD countries.

In February, the Educational Testing Service (ETS) released an analysis of data from a large OECD study that compared the skills of adults ages 16-65 across 22 developed countries in 2012.

The results were not pretty.

Handwringing about the low level of skills of our lowest performing young adults is nothing new. And this latest analysis confirms that our least-skilled millennials – the generation born after 1980 and aged 16-34 in 2012 – trail their low-skilled peers in other developed countries. But the disturbing surprise in this study is that even our highest performing young adults do not stack up well against their counterparts abroad.

For example, at the 90th percentile, the numeracy score for US millennials ages 16-34 falls below the equivalent score for millennials in every other country except Ireland, Italy, and Spain. Put another way, in almost every developed country, the top 10% of millennials in numeracy score better than our top 10% in numeracy score.

And at the other end of educational spectrum, the US does even worse. Our 10th percentile score for numeracy ranked dead last among the 22 countries studied.

As the final piece in this troubling trifecta, the gap in numeracy scores between our high and low performing millennials – that is, the difference between the 90th and 10th percentile scores – is the largest among all the countries studied.

And just in case you think this is a problem of Americans being “bad in math,” the patterns are generally similar when it comes to literacy and “problem-solving in a technology-rich environment.”

This report really does not contain any good news for the US. Our top-performing millennials compare poorly to their peers in developed countries. Our lowest skilled young adults are outperformed by their counterparts abroad. And the gap between our top and bottom is the largest among any of the countries studied. In short, our young adults perform badly across the talent spectrum, and we have the greatest inequality.

The data for this analysis come from the Programme for the International Assessment of Adult Competencies (PIAAC), and given the strengths of the data, the findings in this report are not easily dismissed. PIAAC assessed the numeracy, literacy, and problem-solving skills of a representative sample of over 166,000 adults ages 16-65 in 22 developed countries in 2012. The US sample alone included 5,010 adults. The study is notable because it assessed the skills of a representative sample of all adults in these countries. Much of the data we have from other sources applies only to younger individuals who are still in school. PIAAC gives a much more comprehensive view of all adults – including those young adults who are not in school.

To reach such a broad population, the study was conducted in the homes of the randomly selected participants. When possible, respondents completed the study questions directly on PIAAC laptops. When this was not possible due to literacy or other issues, the PIAAC representative assisted the respondent in entering their responses.

The assessments were developed to tap those aspects of literacy, numeracy, and problem solving using technology that are broadly applicable to successful adult functioning. All questions were translated into the appropriate languages of the 22 countries in the study.

To illustrate with examples from each category, here's a mid-level question from the problem solving in technology-rich environments (PS-TRE) assessment. The question asks the test-taker to respond to a request for information by locating the information in a spreadsheet and emailing the answer to the person who asked for it. The difficulty score of this item is 296. The average score for US millennials ages 16-24 was 285 – that is, this problem is slightly more difficult than what the average millennial in this age group could complete correctly.

A mid-level numeracy problem presents a page from a motor vehicle logbook with information on the odometer reading (start and finish), distance traveled, and other details. The test-taker is asked to calculate how much a particular employee should be reimbursed if the employer pays $0.35 per mile plus $40 per day for additional costs. The difficulty score of this item is 250, which is very close to the average score of 249 for US millennials ages 16-24. Also note that young millennials' average score of 249 is actually below that for all US adults ages 16-65, implying that young millennials' numeracy skills are below those of older adults.

Finally, the average literacy score of US millennials ages 16-24 was 272, the lowest of all the countries studied except England & Northern Ireland, Italy, Ireland, and Spain. A sample test item with a difficulty score of 289 – that is, slightly above the skill level of the average US millennial – requires the test-taker to find the name of the author of a book called Ecomyth. To do so, the test-taker must scroll through a list of entries on a simulated library website and find the name of the author listed under the book title. The task requires scrolling and clicking to the second page of listings. Making the task harder, each listing contains considerable irrelevant information.

So what are we to make of all this? First, the report's implications for the US's productivity and competitiveness in the coming decades are alarming. Second, in light of the US's poor performance regardless of which apples-to-apples comparisons we conduct – high-performing US millennials versus high-performing millennials internationally, or our low-performing millennials against their low-performing millennials – it becomes harder to maintain that the US's poor performance is somehow due to our demographic mix. In fact, the full ETS report does a good job of slicing and dicing the data to control for all sorts of possible demographic explanations (e.g., the US has more immigrants). No matter, how you analyze it, we still look bad. Even controlling for respondents' level of education, Americans score below their international counterparts.

Given these results, it's hard to conclude anything other than that our educational system does not measure up to our competitors'. That has disturbing implications for our future as an economy and a country. But at least it's something we can change through public policy, if we muster the will and wisdom to act.

* The countries are Australia, Austria, Canada, Czech Republic, Denmark, England and Northern Ireland, Estonia, Finland, Flanders (Belgium), France, Germany, Ireland, Italy, Japan, Netherlands, Norway, Poland, Republic of Korea, Slovak Republic, Spain, Sweden, and the United States.