Sunday, 8 November 2009

Paula Rothermel and the thirty five children

Last week it was suggested on one of the home education lists that I was responsible for Graham Badman's mistaken views about the research carried out by Paula Rothermel. What are these mistaken views? Specifically, the allegation is that Graham Badman ignored the fact that Dr. Rothermel had conducted extensive research into home education and obstinately insisted that her work only involved thirty five children. Perhaps it is time to revisit this research and establish first whether or not Graham Badman was wrong in dismissing it.

In February 1997 Paula Rothermel, a student at Durham University, sent out two and a half thousand questionnaires to home educators belonging to Education Otherwise. the following year she sent out the same number again. A small number were also sent to LEAs and a few other places. The result was a little over a thousand responses. Two important points need to be made here. Firstly, this survey is a snapshot of the situation almost thirteen years ago, before online communities and before the exponential growth of home education in this country. It is well past its sell-by date. The second point is that only a fifth of parents responded. This immediately skews the findings. What sort of people replied? Parents who were struggling? Parents who were educated? parents who were articulate and successful at home educating? This is a classic case of a self selecting sample.

These questionnaires did not yield any information about the academic achievement of the children. We will come to that in a moment. They did reveal motives for home educating, how parents found the experience, what jobs they had and things like that. Now most home educating parents have the vague idea that Paula Rothermel showed that home educated children did better than schoolchildren. This is where the thirty five children that Graham Badman mentioned enter the picture. Paula Rothermel selected thirty five home educating families whose children she tested using the PIPS (Performance Indicators in Primary Schools) Baseline Assessments. These look at academic achievement in young children. It is unclear how many children were actually involved and in part of the work, Dr. Rothermel seems to suggest that it was thirty five children rather than thirty five families. This is where Graham badman got the figure of thirty five children. He did not pluck it from thin air. I have to say, thirty five children is a very small sample indeed. It is upon these assessments, carried out on a sample of children whose families self selected themselves over twelve years ago, that all of Paula Rothermel's conclusions about how well home educated children do, are based.

I do not think that Graham Badman can be blamed for stating that Dr. Rothermel's work was based upon thirty five children. Essentially, he was correct. As to whether it was I who told him this, well he saw Paula Rothermel twice before I had even spoken to him. Any objective person who reads Dr. Rothermel's work will probably come to pretty much the same conclusion that Graham Badman did.

22 comments:

  1. I would have thought Paula, as a university student, would have known about sampling to ensure validity, data collection/analysis, presenting findings etc. Maybe Paula herself would like to comment on this?

    ReplyDelete
  2. Just had a quick look at Paula's reply to the Select comm; she claims there 196 children were assessed - so where did the 35 come from?

    ReplyDelete
  3. This figure of 196 is reached by adding up the PIPS Baseline number of 35 to the lietracy assessments which were carried out. There were 49 of these. A big problem is that only five were conducted under proper conditions by Paula Rothermel herself. The others were sent out by post and the parents were supposed to do them. I think most people can see how that makes it difficult to be confident about all but five of the literacy assessments. The rest of the assessments were not concerned with educational outcomes, they were social stuff, Rutter and so on.

    ReplyDelete
  4. Here is where the figure of thirty five that Graham Badman mnetioned actually comes from. This is from Paula Rothermel's Phd disertation;

    "From the 35 families in the study, 33 took part in the full 'Start' and 'End of Reception' research and 2 families assisted with the PIPS 'Start of Reception' only. Section 4.6.2 describes the process by which, of 45 families approached, 35 went on to take part in this assessment. All the PIPS Baseline assessments were administered by the researcher in person."

    Later on, it becomes unclear whther this is thrity five children or thirty five families. In the 196 which she mentiones in her submission to the select committee, some were involved in the PIPS, some in the literacy assessments and others in pyscho-social stuff. It is those thirty five children though, upon whom all the conclusions about how well home educated children achieve academically are based.

    ReplyDelete
  5. I don't really care one way or the other, because I don't see the education of my children as some kind of a competition. I'm far more concerned about Graham Badman's attempt to smear us by asking her whether she thinks home educating mothers might have Munchhausen's syndrome by proxy. Such a dangerous, nasty thing to say when people have lost their children because of that allegation. It looks like a crass attempt to shut us all up and make us compliant.

    It's good if a child can read and write by the end of its education (though sadly lots of school children don't seem to manage that) but beyond that, education is and should be a very personal, private thing.

    It seems like this government views normal, private family life as a nut to crack open with a sledgehammer, or whatever tool will work.

    I hope the next government will be different.

    ReplyDelete
  6. I thought this was Paula Rothermel's PhD thesis. Not quite the same as her being 'a student', although technically she was of course.

    In addition, a small sample is not necessarily unrepresentative. Peope often assume that the more data points you have, the more reliable your research is. That's not so; you can quickly hit diminishing returns. In other words, unless you have reason to suspect that your sample is significantly different to your population, the more data points you collect, the more you get the same information.

    ReplyDelete
  7. I described Paula Rothermel as "a student" because when she planned and distributed these questionnaires she had been studying with the Open University for two years and been at Durham for less than a year. Since this piece of work is almost invariably quoted whenever anybody is talking about home education in this country, it is worth noting that the sample was tiny. The main point I was making was that Graham Badman was not wrong about what he said of the research.

    ReplyDelete
  8. He insisted her work was of little significance; since it contained a considerably more information about home education than his surveys of LAs, presumably his was of such miniscule significance that we can discount it? Or is he playing 'my sample was bigger than your sample'?

    ReplyDelete
  9. "I hope the next government will be different."

    _______________________________________


    Having trawled through the 2007 review and memorandums/transcripts re the 2009 review I have come to the tentative conclusion that -

    1) neutralising liberal HE laws to head off a potential future boom in "white flight" from state schools.

    2) neutralising liberal HE laws to head off making the UK more attractive than more obvious destinations for Roma gypsies, skewing the migration predictions made when Romania entered the EU.

    3) neutralising liberal HE laws to head off large numbers of British nationals being brought up in and educated in communities where the language and culture of the country of origin of their parents is prioritised to the extent that their ability to identify as British is compromised.

    ...are three possible broader political motivations behind the 2009 review. I doubt they are would be the ONLY motivations, but certainly they are three that I think might feature strongly.

    If that is the case then I wouldn't expect any huge shift in perspective from the next government, whoever it may be.

    (Please note - I am not stating what I think the political aims should be, nor am I stating that the above are valid political aims, just stating what I think the government's position might be, based on what I have read. )

    ReplyDelete
  10. Graham Badman may well have insisted that Paula Rothermel's work was of little significance to the present day situation, suzyg. I for one would agree with him. It is based on a study of thirty five children drawn from a self selected sample made twelve or thirteen years ago. Home education has changed enormously since then. The main point that I was making was that firstly, this was nothing to do with me and secondly, that Graham Badman had good reason for saying that the study was of thirty five children, rather than one hundred and ninety six or four hundred and nineteen.

    ReplyDelete
  11. Hmmm... Philip Gardner's 1984 study of home education and small private schools in the 19th century could be dismissed as of little significance to the present day situation, but it sheds a good deal of light on why people might choose to educate their children otherwise than at state schools. I don't see why a cross-sectional study undertaken 13 years ago shouldn't be informative.

    ReplyDelete
  12. Had the study been a randomly selected, reasonably large selection of home educating families, then it might well be useful today. It wasn't. The children looked at were the ones whose parents were the sort of people who fill in detailed questionnaires and wanted others to know what they were doing. How likely is it that somebody whose children were really struggling, would fill in such a form? Or a mother with poor literacy skills herself? Already, before you even analyse the results, bias has crept in. This in turn will mean that the children at whom you are looking may be atypical. As I remarked above, only five of the literacy assessments were conducted under controlled conditions; this in itself fatally compromises the results. We can draw no useful conclusion about how home educated children are doing compared to schoolchildren. As for the size of the sample, when you are reading about surveys of this sort, you will somewhere see a note saying (N=2000) or something similar. This tells you how many subjects there were. In the case of schools, one might reasonably expect to see N=2000 or N=10000. These are good samples. In the case of this piece of work, we see things like (N=17) or even (N=15). The samples are simply to small.

    ReplyDelete
  13. "Having trawled through the 2007 review and memorandums/transcripts re the 2009 review I have come to the tentative conclusion.."

    Thanks for this, Sarah. It helps to try and see the wider picture. We can get very insular on this little island and then a bit bewildered when policy changes seem to come out of the blue. I can, anyway, even though I've been tracking this 'rolling out' process for nearly a decade now.

    ReplyDelete
  14. "We can get very insular on this little island and then a bit bewildered when policy changes seem to come out of the blue."

    I don't think this is us/you being insular, I am shocked rigid by the content of the link below, but not shocked that I have never "met" or even "heard of" this type of HEing person online via forums and blogs. That type will be very much of the "keeping ourselves to ourselves" ilk and doing their promoting of HE exclusively via contexts with a common political flavour. Which means they are rather "invisible" to the greater HEing community.

    WARNING - link is NOT work safe, or kid safe, or not feeling a bit sick safe.

    This was in the search results when I was looking for the perspectives of adults who had been HEed as kids. I had already posted about "white flight" before I found it and this is way more extreme that what I had in mind. I was thinking "concerned middle classes with sub conscious, fearful tribal reactions who resent other's needs taking up resources to the perceived detriment of their kiddies", not white supremacists that make the BNP look tame.

    http://www.stormfront.org/forum/showthread.php?p=5094175&mode=linear#post5094175

    ReplyDelete
  15. This has been a depressing way to spend an evening. Shoot me now.

    This is from the Student BNP, urging people to sign the petition against the review/proposals.

    NOTE - I did say BNP, you may not wish to click this on a work 'puter or in the presence of others just in case they jump to conclusions.

    http://student.bnp.org.uk/2009/02/home-education-petition/

    And oh lucky day, guess which party is firmly placing itself against the proposals...

    "The British National Party has come out firmly against Government proposals to interfere in the rights of parents who home educate their children. In a statement released today, Joanna Jordan, BNP spokesman on Home Education,...."

    again, not nice link, click with care

    http://bnp.org.uk/2009/09/bnp-will-oppose-interference-in-home-education-rights/

    ReplyDelete
  16. Sarah, as far as I know (having read a few of them) there have been a few people within the HE lists providing links to those same political groups, on the grounds that they (the BNP mostly) are opposing the proposals.

    That some people within the HE community might actually be naïve enough to be sucked in by this, or worse, might not actually care (because, hey the BNP are promising to *not* do this...."), is sickening. Surely the fools touting the BNP (and worse) links on the HE lists realise how crass and racist they're being?

    There are times when I wonder whether some of the people on those lists could be any more ignorant. This is one of them.

    ReplyDelete
  17. "Surely the fools touting the BNP (and worse) links on the HE lists realise how crass and racist they're being"?

    I need more lists. I appear to be the last to know anything rather a lot.

    I'd leave room for the waters being tested from the outside for reaction. There was at least one self-declared, longer term, HEing parent posting on the BNP forum who appeared to support the party and talked the HE lingo, wouldn't shock me if the BNP trawled various forums, posting as a way of looking for potential recruits or at least a softening up towards them from various sub sections of the UK gen-pub.

    Unless the HEing parent who was posting on the BNP forum was counter espionage from HE lists trying to infiltrate the BNP and out any errant HEing parents who played for both teams.

    OK. I defiantly spent too long on the white supremacist's site, I am seeing black helicopters everywhere. I am going to the LOL Cats site for a few minutes (hours/all night) to try and find my happy place again.

    ReplyDelete
  18. I haven't seen anyone touting or promoting racist links on home ed lists. In fact, the most racist links I've ever seen in relation to home ed appear right here in these comments! (Way to go, Simon..) And I'm a member of all the main HE national lists and groups.

    Happy LOL cats, Sarah ;-)

    ReplyDelete
  19. Ah I see, Anonymous. You mean that this is the sort of place where people might feel more comfortable about posting racist links? Interesting idea. Actually there have been discussions on the HE-UK list about BNP policy on home education and somebody did post a link there.

    ReplyDelete
  20. Of course there have been discussions on HE-UK; the BNP have a policy on HE. Of course someone posted a link. So what?
    This is a discussion about the BNP. There are links here. So what?

    ReplyDelete
  21. Above, Anonymous said;

    "I haven't seen anyone touting or promoting racist links on home ed lists."

    I pointed out that links had been posted to the BNP on the HE-UK site. Anonymous' responses was to say;

    "Of course there have been discussions on HE-UK; the BNP have a policy on HE. Of course someone posted a link. So what?"

    I feel that we are talking at cross purposes! I was simply pointing out that some people have posted links to the BNP here and also on the HE-UK site. I took it to mean that when Anonymous said that the most racist links had been posted here and followed it by the remark, "Way to go Simon", the impliation was that there were more links to racist sites here than elsewhere.

    ReplyDelete
  22. Mam'Goudig used the phrase "fools touting the BNP (and worse) links on the HE lists". She was suggesting that the people who did this are crass and racist. I was trying to point out that while the BNP have been discussed and linked to, no-one has been *touting or promoting* them. You have also been discussed and linked to, but no-one has been "touting or promoting" you either!

    ReplyDelete