Is the SAT accurate in predicting student performance?

By Dr. Darrin Hanson

I come at this question from a somewhat unique perspective.  I used to serve on a university admissions committee.  I was also an assistant professor for three years who had access to the SAT/ACT scores of my advisees.  I am a social scientist who has studied much of the literature on standardized testing.  I currently co-own a side business, Madison Learns, tutoring high school students to score higher on the SAT.  Given this background, I've given a lot of thought to the effectiveness of standardized tests such as the SAT in predicting how well a student will perform in college.

Today I'm writing because a recent study came out arguing that the SAT does nearly as well at predicting college success for lower "socio-economic-status" students (SES) as it does for higher SES students.  The study has a relatively large data set and the statistical analysis isn't crazy, but I am somewhat skeptical of the findings.

Overall, from my experience as a college instructor and a member of an admissions committee, I have to say that by and large the SAT and ACT do a relatively good job at predicting success in college.  There are outliers on both sides (such as one student with an SAT combined score of 1270 who got straight A's, or another student with a combined score of 2250 who flunked out his first semester), but in aggregate both test do relatively well predicting college performance.  The academic studies largely support this finding.

The concern that the recent study tries to address is whether the SAT is a better predictor of those with higher SES background.  For purposes of the study, they looked at both income and parental educational attainment (highest level of school, etc).  The study found that there was a slightly smaller correlation between SAT scores and college success for lower SES students than for higher SES students, but that the difference was not statistically significant.

It's at this point that I start to wonder what is going on.  My own suspicion, honestly, is that SAT scores would significantly under-predict performance for low-SES students.  My reasoning is this: Higher SES students frequently get special training to prepare for the exams.  Looking at my current test-prep students, all of them have at least one parent with a graduate degree and all of them have a combined household income in the six figures.  To put it bluntly, parents who earn less can't afford my rates and parents who never went to college might not realize the value of getting extra help preparing for these exams.  High-SES students have an inherent advantage in taking the test because their parents understand the value of and can pay for extra help.

But, to be fully honest, the skills I teach to increase scores on the standardized tests are not the same as the study skills I teach to help students do better in school.  If you utilize the stuff I teach you to get a better score on the SAT for your college class, it will actually hurt your grades.  If you utilize the study skills I teach you to improve your grades while taking the SAT, then you will do poorly on the SAT (largely because you will run out of time).  I have yet to figure out how doing well on the SAT could truly be an accurate measure of success in college, as they involve so many contrasting skills.  (Perhaps someone from the College Board can comment and explain it to me.)

The final thing that makes me suspicious of the recent study is that it was funded by the College Board, the company that owns the SAT.  This doesn't necessarily mean that the study was corrupted, but it does make me wonder.  I am one of few social scientists will confirm the old saying, "Lies. Damned lies. And Statistics."  By asking certain questions in certain ways, you can manipulate results.  Again, let me emphasize that I have not found anything like this (yet) in the study that just came out.  But when the findings of a study don't match up with common sense, it does make me want to sniff around a bit more.