Tuesday, October 29, 2013

CPAG still not owning their errors

A note has been added at the link to the CPAG report I highlighted a couple days back:

Please note: Due to feedback CPAG has become aware that there is an error on page 9 of the report. The relevant paragraphs should read: 

"The percent of distinct clients with substantiated abuse or neglect for each site office from 2008-2012 was averaged and divided by the estimated number of 0-17 year olds in each site office (2012 population)  in order to calculate the proportion of children who were victims of substantiated abuse (see Table 3 in the Appendix).

Figure 3 shows the 10 site offices with the highest proportion of children being victims of substantiated abuse. Even within this small group there is considerable variability with Papakura having the highest percentage of substantiated abuse (4%) and the Far North having a rate slightly over half that (2.2%) (a ranked listing of site offices is at in Table 4 the Appendix)."

The error does not affect the results or the substance of the report. A corrected full version of the report will be available shortly.


Let's recall what the original report said:

"The percent of distinct clients with substantiated abuse or neglect for each site office from 2008-
2012 was averaged and divided by the estimated number of 0-17 year olds in each site office (2012
population) in order to calculate the proportion of children who were victims of substantiated abuse
(see Table 3 in the Appendix).

This figure was multiplied by 100 to give an easy-to-read number. It is therefore important to exercise caution when interpreting the numbers. For example the proportion of 0-17 year olds who were victims of abuse in Papakura was not 4.0% but 0.40 of 1% (my emphasis)
 
Figure 3 shows the 10 site offices with the highest proportion of children being victims of substantiated abuse. Even within this small group there is considerable variability with Papakura having the highest rates of substantiated abuse (0.040%) and the Far North having a rate slightly over half that (0.022%) (a ranked listing of site offices is at in Table 4 the Appendix)".
 
Note the middle paragraph was omitted from their correction. That was the section that illustrated their inability to mentally calculate a proportion.

In any case, this correction owns only one of three mistakes I have written to them about.

If the author had acknowledged that the denominator population used to calculate benefit dependence is wrong, then the relationship between the rate of child abuse and benefit dependence - the "substance of the report" -  would change. (I don't have the information yet to show by how much but I'm working on it.)

While CPAG can attempt to minimise their errors in this fashion, they cannot minimise the negative impact on their credibility.


1 comment:

thor42 said...

Hah.... were you ever a teacher by any chance, Lindsay? :)

"CPAG - D-minus. Must do better. See me."