Monday, November 16, 2009

Yeah, counselor, That's just what I want to do. Pay a whole lot of money for college, go for two years and flunk!

Many of you are aware by now that blogger Nate Silver discovered apparently phony survey results by the polling company Strategic Vision using statistical analysis. In particular he looked at a survey which was given to high school students in both Oklahoma and Arizona (which had ridiculous results, such as claiming that only a quarter of public high school students knew that George Washington was the first president.) In fact, an Oklahoma state senator named David Cannady replicated the test in that state and showed that the actual results were far, far from what the original data showed.

The results however have been trumpeted both by a conservative think tank in Oklahoma and by our very own Goldwater Institute here in Arizona.

Arizona blogger David Safier has a nice writeup on it here.

In fairness, Matthew Ladner, the lead statistician at the G.I. has said he would own up to it if the numbers turn out to be fake, and in fact it's looking like he's going to be overwhelmed with evidence.

In Safier's write up he links to the public school and private school surveys.

Looking at the entire survey, not just the ten questions on civics adds even more evidence.

For example, when asked about such matters as whether their school treats everyone well, you get a very nice statistical distribution-- however I know from having conducted surveys in issues of opinion like this in the past that a significantly higher number of people (and I'd venture even of schoolkids) are likely to answer the to the extremes (i.e. 'strongly agree' or 'strongly disagree' than the low numbers shown by the survey. Even on questions that should provoke a strong opinion, all the data seems to fit a nice, normal distribution when it should more likely be bimodal. For example, I'd think even the G.I. would question whether 2% of public school students give their teachers an 'F.' I could name more than that percent of my kids' classmates who want to blame the teachers for all their problems. But hey, once again look at the nice normal distribution there-- on a question that should very likely not be normal.

Even more strongly, look at the public school survey question on what the students want to do after high school. Note that vocational school is listed separately from the military. So if you look at just those who plan to attend college, 27% say they expect to attend college but not finish, compared with 39% (30% + 9%) who do plan to finish. This may fit the actual numbers (after all a lot of people do flunk out of college) but I don't think I've yet met a single young person who enters college intending to not graduate! Yeah, that's the ticket-- run up a lot of student loan debt and flunk! And they are claiming that that is the INTENT of 27/66 = 41% of public school students who DO plan to go to college?

Clearly THAT is a fabricated number-- the person who did it looked at the approximate number of kids who actually do flunk out of college (or leave early due to a family emergency or other reason) and extrapolated to that's what they were planning in high school.


sandyh said...

Name me any other country on earth which constantly belittles and tries to make their youth look dumb.

Of course, most surveys are skewed. How else could they be anything else when the choice of questions and the wording used is completely subjective? It's always easy to get the results you want.

A 15-second sound byte world demands such outlandish claims. Most people have become numb to any findings that come from self-funded polls or surveys. You'd be surprised how many illiterate people understand this concept better than the "experts" and particularly the media.

We are no longer shocked by alarmist claims nor do we trust them. It's the color-coded terrorist alert concept. All the experts cry wolf when they are only selling something we don't need.

Our kids are just as smart as any others in the world. It's our media which is dumb when it continually allows itself to be manipulated and exploited into thinking otherwise.

I use the term "thinking" here subjectively.

Eli Blake said...


The issue here though is not just a matter of bias, but of an intentionally fabricated survey that is being bandied about as 'real' results.

In other words, a bald-faced lie.

sandyh said...


I guess I was trying to be polite? No matter how they do it, it's still a fabrication based on a pre-conceived lie.