Sunday, March 21, 2010

MSD misrepresenting statistics and misleading the public

A report just published by the Ministry of Social Development examines efforts in Counties Manukau to reduce youth offending.

Under key findings the following appears;

Homicides in Counties Manukau reduced from an unprecedented peak of 34 in 2005 to 7 in 2008.

The figure of 34 looks too high. For further verification I went to the actual report;

The number of homicides tends to vary considerably from year to year. However, the 2005 figure (34) in Counties Manukau for young offenders was the highest recorded in the last decade. The numbers of homicides in 2007 and 2008 were considerably lower at 6 and 7 respectively. Nationally, excluding Counties Manukau, homicides by young offenders remained fairly stable between 2004 and 2008 at approximately 20 a year.

Here are the police statistics for Counties Manukau;

There were 19 homicides in 2005. How can there have been 34 homicides by youth offenders in the same year?

Of course there may have been more youth apprehended for homicide but that isn't what the report says.

The media would simply cut and paste the statement under key findings which is an error of significant magnitude.


big news said...
This comment has been removed by a blog administrator.
mojo said...

Mmmmm ... an uncommon phenomenon??

Anonymous said...

It looks like they're referring to homicide youth apprehensions to get the 34 figure (p20 of the full report), but have screwed up the summary and referred simply to "homicides". Which roughly doubles the homicide incidence rate. Whoops, as they say.

Although it doesn't actually affect the general gist of the report (on a quick scan) which was that the youth programmes do appear to have contributed to a reduction in youth offending and recidivism.


Lindsay said...

I haven't disputed the "general gist" of the report.

This particular key finding was wrong.

And the main body of the report (quoted in post) was wrong.

Anonymous said...

yeah, it's a bit of a cock-up, no debate there.

But the particular key finding was probably wrong in scale, not substance (i.e. the label was wrong, not the data).

At least they don't seem (might want to double-check that, I skim-read) to be comparing apprehended youths 2005 vs homicide victims 2006/7. Which would be A Major Cock-up (tm) and suggest a disproportionate "reduction" in "homicides".

I guess it's the difference between "whoops" and "whoops, all our data is crap and doesn't justify the incredibly expensive recommendation we've just made, or even the correct data opposes it". Which is my personal nightmare.

But go for broke on them - it's the only way people learn. Write the MP for clarification: that always sends s$%t downhill with gathering speed. You can guarantee *that* section manager will be properly proofing drafts before dissemination after Questions are Asked (tm). Especially if you threaten to write the opposition spokesperson.