what is it about child protection plans and statistics?

Ofsted warns of abuse risks to disabled children” says the BBC headline about the report entitled Protecting disabled children; thematic inspection published last week. And indeed the report opens by reminding us that research “indicates that disabled children are more at risk of being abused than non-disabled children”. It goes on:

However, they are less likely than other children in need to become the subject of child protection plans. This suggests either that risks to disabled children are not well identified or that support effectively reduces risks and helps to keep them safe.

The thematic inspection was undertaken to find out more. I’ll come back to the findings later, but in this post I want to take a closer look at the data about risk that appear to have prompted Ofsted’s interest. The research cited in support of Ofsted’s opening claim consists of one paper (Sullivan and Knutson, 2000) describing a large-scale study of children in Omaha, Nebraska during 1994-95. Disabled children were found to be 3.4 times more likely to have experienced maltreatment than non-disabled children.

By contrast, Ofsted points out;

The children in need census as at 31 March 2011 showed that there were 382,400 children in need in England of whom 54,100 (14.2%) were recorded as having a disability. At that time 42,700 (11%) children were subject to a child protection plan of whom 1,600 (3.8%) were recorded as having a disability. Children with a recorded disability were therefore less likely to be the subject of a child protection plan than other children in need.” (p.8)

If I’ve understood this correctly, it’s saying that if disabled children are at a higher risk of maltreatment than non-disabled children, you’d expect to see a higher rate of child protection plans (CPPs) amongst disabled children. But the frequency of CPPs amongst disabled children is lower (3.8% as compared to 11%), prompting concerns about children falling through the net.

There was something that didn’t look quite right about Ofsted’s analysis and I think I’ve figured out what it is. (I should point out that statistical analysis isn’t my forte, so please let me know if I’ve got it wrong.) I think there are three problems with Ofsted’s data;

– the level of risk to disabled children is based on an assumption
– the figures for children in need and disabled children are unreliable
– the level of risk and level of CPPs are derived from qualitatively different populations.

Assumptions about disabled children and the risk of maltreatment

First, let’s look at the figures for the risk of maltreatment. Little relevant research has been undertaken. Sullivan and Knutson’s study did indeed find that disabled children were 3.4 times more likely than non-disabled children to experience maltreatment. But they also cite an earlier finding that the risk was 1.7 times higher (p.1258). It’s clear from previous research that determining the prevalence of maltreatment is dependent on factors such as the quality of available data, quality of support services, definitions of disability and maltreatment, and whether disability has led to maltreatment or maltreatment has led to disability. This means that the prevalence of maltreatment is likely to vary widely between communities. The figure Ofsted took as its baseline for England was one that emerged from data gathered almost 20 years ago and 5,000 miles away in a community with a very different healthcare system. We don’t actually know the risk of maltreatment for disabled children in England.

Who qualifies as a child in need or a disabled child?

Let’s assume for the moment that disabled children are at a higher risk of maltreatment than non-disabled children. Whatever the level of increased risk, we’re still left with the question of the lower proportion of disabled children with a CPP.

Sullivan and Knutson’s sample consisted of 50,278 children enrolled in the Public (OPS) and Archdiocese schools of Omaha, Nebraska during the 1994–95 school year. It included children in other educational programmes and therefore covered an age range between 0-21. So the sample represented almost all children in a specific geographical area. But the sample Ofsted refers to consists, not of all children in a geographical area (in this case England), but of children in need in that location.

Section 17 (10) of the 1989 Children Act defines children in need as follows:

“For the purposes of this Part a child shall be taken to be in need if—
(a) he is unlikely to achieve or maintain, or to have the opportunity of achieving or maintaining, a reasonable standard of health or development without the provision for him of services by a local authority under this Part;
(b) his health or development is likely to be significantly impaired, or further impaired, without the provision for him of such services; or
(c) he is disabled”

and s. 17 (11) defines ‘disabled’ as;

For the purposes of this Part, a child is disabled if he is blind, deaf or dumb or suffers from mental disorder of any kind or is substantially and permanently handicapped by illness, injury or congenital deformity or such other disability as may be prescribed; and in this Part—
* “development” means physical, intellectual, emotional, social or behavioural development; and
* “health” means physical or mental health.”

The first problem that arises is one of classification. The Children Act appears to class all disabled children as ‘children in need’, whether or not they are considered to be otherwise at risk. But local authorities don’t necessarily do that. DirectGov site says ‘local councils must identify the extent of need in their area and make decisions about levels of service they provide’, and it’s clear from local authority websites that their criteria for ‘children in need’ and criteria for ‘disabled children’ vary somewhat. One local authority officer told me that her LA classifies all disabled children as children in need whereas Cheshire East, for example, carries out an assessment to determine whether a disabled child qualifies as a child in need.

Then there’s the issue of the reliability of local authority records. The Citizen’s Advice Bureau tells us that ‘the local authority must keep a register of children with disabilities in its area but does not have to keep a register of all children in need’. Local authorities might be required to keep a register of children with disabilities, but that doesn’t mean that all children with disabilities will be registered. A recurring complaint from my own local authority is that although they have a register, it’s perpetually out of date and incomplete. In short, if LAs are unclear about how many disabled children and how many children in need are living in their area, the census data cited by Ofsted might not be entirely reliable. I did try to find out what data were collected by the census, but failed. If you can figure it out, let me know.

Qualitatively different populations

A key problem with Ofsted’s data is that if some LAs are treating all disabled children as children in need, but are treating non-disabled children as children in need only if their health or development is at risk, this would result in non-disabled-children-in-need being at a higher risk of maltreatment than non-disabled children-in-the-general-population. Ofsted is comparing data drawn from two qualitatively different populations – the general population vs children in need.

Child protection plans

What difference does that make to Ofsted’s concern about CPPs? It means that you’d expect non-disabled-children-in-need to have a disproportionately high level of CPPs compared to non-disabled-children-overall. That would widen the gap between CPPs for non-disabled children and CPPs for disabled children, making the proportion of CPPs for disabled children look disproportionately low. Some calculations might clarify the point:

Sullivan and Knutson set out their data (for all children) in a 2 x 2 matrix as shown below:

If Ofsted’s data (for children in need) are set out in the same way, they look like this:

In Table 2, the proportion of non-disabled children with CPPs is 4 times the proportion of disabled children with CPPs, exactly the opposite of Sullivan and Knutson’s trend, and that’s what Ofsted is concerned about.

But Sullivan and Knutson’s finding was based on data from all children in a location, not just children in need. If we put Ofsted’s figures into the context of all children in a location (in this case, England), a different picture emerges. (The total figures for all children and for all disabled children are estimates – the figure for disabled children is the Aiming High estimate).

If my calculations are correct, the proportion of non-disabled children with CPPs in the whole population of non-disabled children is only 0.39%, whereas the proportion of disabled children with CPPs in the whole population of disabled children is slightly lower at 0.28%, considerably less than the 4-fold ratio if children in need only are taken into consideration. When the population differences are corrected for (as far as is possible with the data provided), the discrepancy between CPP rates for disabled and non-disabled children almost disappears.

*********

There’s something about child protection plans that seems to have an adverse effect on statistical analysis. UK readers who were home-educating their children during the Badman review in 2009 will probably recall the disagreement between Graham Badman and Graham Stuart (later to become Chair of the Education Select Committee) about the proportion of home-educated children subject to CPPs. The central issue was that the total number of home-educated children in the UK was unknown; estimates had varied between 20,000 and 150,000. You can read the full account here, but I thought it was worth reproducing some of the exchange in full:

Graham Badman: I reflected a great deal on our exchange of views, I promise you. I did go back and look at the figures and I came up with exactly the same conclusion. In fact, if you want me to qualify it, when surveying the number of child protection plans in the authorities that we covered in the last survey, we came up with a figure of 0.4 per cent., which is double that within the normal population. In fact, that figure could be fractionally higher, because we discounted any child protection plan that was there as a consequence of disability. I know you argued that because I did not know about the other half, that therefore negated it and made it even again. My argument in return, as you will recall, was that because we did not know about the other half, that did not mean to say that they were all safe either. It could be exactly the same figure or more.

Q 86Mr. Stuart: May I interrupt you there? The question was not about the letter to the Select Committee in which you suggested that I had said that any child who was not known about was automatically safe. I said no such thing, so you have your facts wrong there—again. What we were establishing was the rate of child protection plans in the home-educating community. We went through it slowly, and you want to rehearse it, so I will rehearse it.

If you have a child protection plan, automatically you are known to the local authority, so children previously not known to the local authority who become subject to a child protection plan are then known to the local authority. It does not by any means guarantee that children who are not known to the local authority and do not have a child protection plan are necessarily safe. But what you do know is that every child with a child protection plan is known to the local authority, and you claimed, completely erroneously—and although you have been corrected, here you are months later still getting it wrong, which is pretty frightening—that the percentage of children who are home educated who have a child protection plan is double the national average, when your own report suggests that the numbers are at least double the 20,000, which is the number registered with the local authority, and the Secretary of State said on Second Reading that the Government estimated the number was 70,000. They do not know for sure, but they think it is 70,000.

If you take the number of home-educated children with a child protection plan and you see that as a percentage of 70,000, it comes out at a great deal less than the average for the population, and that I believe is the definitive statement of what we know. It does not tell us everything we need to know, which is why I would not overstate my reliance on it. Could you please comment on whether you believe that we can truly say, as you have repeatedly said, that there is double the rate of the most serious level of child protection plan among children who are home educated?

Graham Badman: I fear we are in danger of going round in the same circle. I am afraid I fundamentally disagree with you. You think I am wrong; I think you are wrong.

Mr. Stuart: It is maths.

Graham Badman: Well, fine; we might want to debate that later—perhaps you went to a better school than me. The fact remains that I do not agree with your assumptions. On the basis of the data collected from the local authorities and from two surveys that have been validated by DCSF statisticians, I believe that we are quite safe in saying that on the basis of the child protection plan analysis that they carried out, there are twice as many young people on a child protection plan known to local authorities within that population as are within the general population. That is a fact. I am sorry that you do not agree with me, but we could go on for ever disagreeing.

I also have concerns about other aspects of the report, which I’ll discuss in the next post.

Reference

Sullivan, P.M & Knutson, J.F (2000). Maltreatment and disabilities: A population based epidemiological study. Child Abuse & Neglect, 24, 1257–1273,

Advertisements