Tuesday, 16th February, 2010
BBC News Statistics
Recently, I’ve begun to notice some deficiencies with the reporting of “studies” and “surveys” by the BBC. I have never taken any of the statistics or probability theory courses here at university, so in effect, I am a layman on these matters. Regardless, I shall post my opinions on these issues.
Violence against women is justified, says pupil study
Page last updated at 14:59 GMT, Monday, 15 February 2010 According to the BBC News Website, at this time of writing.
It’s a good headline, bold and shocking, it gets your attention and will probably trigger a reaction in any one you read it to. But not that informative.
I’ll get the non-stats criticisms out of the way very quickly. The article only gentle implies that it is possible to give reasons for doing an unreasonable action. Reasons, as given by real people are rarely reasonable, and almost always biased. Ignoring this little issue, we’ll move on.
The majority of the pupils said it was justified if the woman had an affair, or if she was late in making the tea.
The “majority” 51% is a majority. If there are 4 choices, only 26% is a majority. Granted, in this case, it was essentially a dichotomy; justified or not justified. Still, they’re not actually representing their data here. They’re making very ambiguous statements, and not really doing anything about it.
The study by a researcher from Edinburgh Napier University …
This must scream out to most people that this article must be taken with a grain of salt. A single researcher, at a single institution. I see the peer review a mile off, the control groups and the repeated surveys in other major cities. That’ll give it the credibility needed to be published to the entirety of the public. That is, published by a news organisation that many hold in very high esteem, and take a good majority of what is published as fact.
The research involved 89 primary seven children at five Glasgow primaries.
89 Children? Wolfram|Alpha’s current population statistics pin Scotland’s population at 5.17 million people. I do not know what portion of them are children, but this is a painfully small sample size, in a very localised area.
The author of the article neglects to say how deprived the areas near the schools are, if they’re comprehensive or public, co-educational or other pertinent facts that may help the reader form a reasoned judgement. Simplistic “facts”, held up to generate shock.
Men ‘need better-fitting condoms’
Page last updated at 01:23 GMT, Tuesday, 16 February 2010 According to the BBC News Website, at this time of writing.
Simple headline, and directly quotes from the research. Doesn’t appear to be crying out for attention.
Men are twice as likely to take the condom off midway through sex due to a poor fit, a poll of 436 in Sexually Transmitted Infections journal reveals.
Due to poor fit? Who determines what is poor fit? Can a man who is not thoroughly educated in the matters really deduce what constitutes a poor fit? Either way, it’s not bad, but it does leave some questions unanswered.
Sexual Transmitted Infections Journal is a peer-reviewed paper, so if they published this paper, there must be some merit to it. That is to say, this article, which is based on the paper published in the journal, must inherit some of the credibility. Huzzah, the BBC doing it right.
Sadly, the BBC does not do it right too much. Although they acknowledge at the beginning of the article to having a sample size of 436, which is tiny, they still proceeds to publish this article without too much worry as to the veracity of the source of the information.
Women say some rape victims should take blame - survey
Page last updated at 14:04 GMT, Monday, 15 February 2010 According to the BBC News Website, at this time of writing.
Wow. That’s an attention seeking headline if I ever saw one. At least they note it’s a survey, not a study or portray it as fact.
Most of the start of the article is simply raw summations of the figures dumped, for example:
The main reasons were being too embarrassed or ashamed (55%), wanting to forget it had happened (41%) and not wanting to go to court (38%).
This would not be to bad in and of itself, however, there’s little actually said at the start of the article about the source of these figures. You have to dig a tiny bit deeper into the article. Specifically, the crucial information is given in the 17th paragraph. Given that a lot of people will simply glance at the headline, and the first few paragraphs, this really isn’t good enough. There’s little possibility of the casual reader being able to make a reasoned judgement as the the reliability of these numbers.
The online survey, titled Wake Up To Rape, polled 1,061 people aged 18 to 50, comprising 712 women and 349 men.
Most people will know that online surveys are self-selecting by their nature. You choose to fill out the survey, you choose because you have something to say on the subject. Perhaps those who found the questionnaire were rape victims, many rape victims do blame themselves before seeking professional help. Many people who were unaffected by rape would’ve simply passed it by. In the process, taking valuable data away from the survey.
Also, they have a slightly skewed result set. The results for those from women would be more reliable than those for me, simply because more women answered. This could lend itself to more views about rape from women being put forward.
Once again the sample size demon rears it’s ugly head. 1,061 people in a country of over 60 million is not an acceptable sample size. Especially when combined with the self-selecting nature of the survey, and the skew in the genders.
An Amnesty International report five years ago found that a significant minority of British people laid the blame for rape at victims themselves.
Once again, the BBC using ambiguous language for what appears to be simple shock value. More so because of the prevalence of data earlier in the article. How large must a minority be to be significant? 1%? 5%? 15%?
The rest of the article is simply quoting spokespeople for various government institutions.
Survey Complete!
So, I’ve done my survey of some relevant articles on the BBC News website. Given that BBC News is using similar tactics, I’ll use there here:
A study, published earlier today by a largely unverified source states that 100% of BBC News articles have statistical flaws in their data.
Only 33% of the articles showed any real statistical merit, but those ones were correlated with the articles which had non-provocative headlines.
Almost all of the articles used statistics from a very small sample size.
By the way, the sample size was 3, self-selecting articles on the BBC News Website.
Do you get it now BBC News? Misusing statistics is not big, it’s not hard, and it’s the opposite of clever. Especially when it appears to be primarily based on drawing traffic to your site through shock-tactics.
Oh, and cite your sodding sources correctly next time. Googling for “Sexually Transmitted Infections Journal” is not cool.
Done! :-D