Q&A

Q: What is the purpose of the Global Report Card (GRC)?

A: The GRC provides information on the average level of student achievement in math and reading in virtually all U.S. school districts relative to the student achievement in a set of international peers.

Q: What do the measures of achievement for a district mean?

A: The numbers reported to indicate the level of math and reading achievement are percentile rankings. A percentile ranking describes where the average student in a district is in achievement relative to where students in the average developed economy would be performing. For example, a score of 60 means that the average student in a district does better than 60% of the students in the average developed economy.

Q: How can we manage to compare the achievement in U.S. school districts to achievement in other countries?

A: Students in every district take state accountability tests in math and reading. We use the results from those tests in every grade to generate an average level of achievement in each district. We then adjust that level of achievement up or down based on the extent to which the average achievement in that state, as measured by the National Assessment of Educational Progress (NAEP), is higher or lower than the national average. We then adjust each districts achievement up or down again based on the extent to which the U.S. does better or worse than students in a set of countries with developed economies, as measured by the Program for International Student Assessment (PISA).

For a more detailed explanation of the methodology, click here.

Q: What if I can't find a district that I am looking for?

A: You may need to refine the way in which you are searching for the district. You can identify the district from a list of districts for each county and for each state. You can also try to search by the name of the district. Remember that the names of the districts are not always the same as the name of the city or town in which the district is located.

In addition, the American Institutes for Research (AIR), the organization that collected the state accountability test results we used for the Global Report Card, did not have results for every district and every state in every year. The AIR data set is the most comprehensive collection of district test results, but unfortunately it has its limitations.

Q: What is a good score?

A: We leave that up to you to judge.

Q: What if the score for a district just doesn't seem right?

A: The results may surprise you because the comparison we are making here is not the one that you are accustomed to seeing. Normally we compare student achievement in a district relative to the average achievement in a state or relative to a stated goal for achievement (such as basic, proficient, advanced, etc…). In the Global Report Card we are comparing against a set of international peers, so the results may be different from what you have seen in the past.

It is also possible that any particular result could be distorted by an error in the data set. With almost 14,000 school districts, it is inevitable that some errors will find their way into our results. We are also dependent on AIR data base for district test results, so any errors in that data set will be reproduced here. Overall, errors should be rare, but we will do our best to identify and correct any errors that are uncovered.

Q: Does a district with a low score mean that it has bad schools?

A: Not necessarily. The Global Report Card scores simply indicate the level of achievement without attempting to isolate the various factors that contributed to that level of achievement, such as the roles played by families, communities, cultures, and school quality. Future versions of the Global Report Card may begin to disentangle the unique contribution of each of those factors to student achievement.

Q: Don't we care about more than just student test scores when considering education outcomes?

A: Of course we do. But student achievement in math and reading are predictive of the future success of those students. We should care about all of the factors that contribute to the success of our students, including math and reading test scores.

Q: Why do I see charter schools listed for some states but not for others?

A: The AIR data set from which we are working has test scores for virtually every school district. In some states charter schools are classified as if they were their own school district, and would be included in our results. But in other states charters are not treated as if they were school districts and those charters are not reported in our results. We apologize for the inconsistency but it is a reflection of different state systems for charter schools.

Q: Where are the results for private schools?

A: The Global Report Card only has information on public school districts.

Q: How were the rankings developed?

A: To construct the Global Report Card we combine testing information at three separate levels of aggregation: state, national, and international. At each level we use the available testing information to estimate the distribution of student achievement.

The calculations begin by evaluating the student achievement distributions at the state, national, and international level. To allow for direct comparisons across state and national borders, as well as testing instruments, all data is mapped to the standard normal curve using the appropriate student level mean and standard deviation. The lowest level of aggregation is then calculated by estimating average district quality within each state. Each state's average quality is evaluated using national testing data. Finally, the average national quality is determined using international testing data. This evaluation re-centers the distribution of district quality based upon the relative performance of the individual state when compared to the nation as a whole, as well as the relative performance of the nation when compared to the international set of competitors.

The methodology was also reviewed by outside peer reviewers.

Additional information about the methodology can be found in the Technical Appendix. The material and rankings provided are the opinion of the George W. Bush Presidential Center based on this study's methodology and the most recent data that was available for this analysis.

"One key to improving our schools is ensuring transparency of performance across schools -- but this has never happened.  Greene and McGee have made tremendous strides toward correcting this by putting every school in the country on a world standard.  Instead of hiding levels and differences in performance in opaque terms of points on standardized tests, they have told us where our schools -- by name -- stand in the worldwide distribution of performance.  This clever and compelling analysis lays bare the challenges that we as a nation face."

Eric Hanushek
Stanford University