Thursday 13 August 2009

Paula Rothermel and Alan Thomas

Whenever home educators are trying to prove that their lifestyle is good for children, they invariably quote research by Paula Rothermel or Alan Thomas. The problem comes when you actually read the research, as as opposed to bandying about snippets on the internet. Because one of these authorities, or perhaps both, must be mistaken.

Let us start with Alan Thomas, a respected psychologist who has written key works on the subject of home education. In 1998 his book, "Educating Children at Home" was published. Based upon work with a hundred home educating families, he drew two important conclusions; both of which were enthusiastically received by home educators. Firstly, he suggested that although many parents began by teaching their children formally, most slipped into a more relaxed style, without lessons, timetables or conventional teaching. Secondly, he noticed that the children tended to be late in reading, but that when they did start reading, they rapidly caught up with school educated children. I think that most home educating parents would be inclined to agree with both propositions. In short, he believed that children taught informally often were late in reading, sometimes not doing so until eleven, twelve or even a little later.

Let us now look at Paula Rothermel's much quoted work, which contains the only real evidence that children home educated in the UK perform as well as those at school. At once, we see a problem. Rothermel comes to a completely different conclusion to Thomas. When she looks at the reading ability of young children, from five or six upwards, she finds that far from lagging behind the school children, they are in fact extraordinarily advanced for their age! Actually, her data are absolutely astonishing. Without going into too much detail, the children whom she investigated were given a reading test, (the NLS Assessment). In schools, one would expect 16% of children to reach the top band. Among the home educated children 94% of six year olds managed it. The figures are also extremely high for seven, eight, nine and ten year olds. In other words, rather than being delayed in their reading, according to Rothermel home educated children are fantastically ahead of those educated at school.

Clearly, both Thomas and Rothermel cannot be right about this. Home educated children cannot be both marvellously early readers and also remarkably late ones! I think that most home educators fall into the Thomas camp, believing that their children may learn to read late, but that it does not matter. This does not of course prevent them brandishing Rothermel's figures when debating with non home educators!

So what caused such a difference in findings? Well, to begin with Alan Thomas was already a very well established and experienced psychologist when he undertook his research. Paula Rothermel, on the other hand, was undertaking a thesis for her doctorate. When one looks closely at her work one finds that the literacy tests were not conducted under controlled conditions as they are in schools. She posted them out to parents, who then did them unobserved with their children. Hands up anybody who can imagine a mother leaning over little Johnny's shoulder saying irritably, "Come on, you do know that word! Look, it begins with C."

As far as can be gauged, Rothermel and Thomas carried out their work more or less at the same time, using precisely the same sort of subjects, found mostly via EO and other groups. Any difference in results is likely therefore to be caused by methodology. Because Paula Rothermel's work is the only research undertaken within the UK which supports the notion that the educational progress of home educated children equals or even exceeds that of those educated at school, I shall in the next few days focus on this and see what it might tell us.

30 comments:

  1. Hands up anybody who can imagine a mother leaning over little Johnny's shoulder saying irritably, "Come on, you do know that word! Look, it begins with C."
    You can see straight away old Simon mistrust of parents from the above quote.Thats why he such a big fan of uncle Graham Badman frist he wants to invade you home which most people work very hard to pay for then he want to take child away and interview with questions to make parent look bad.You should write to Graham and ask him if you can go on his panel when he gets new work from DCSF and if your very good you may get a CBE! only one problem Labour are going to lose next election voters will be taking revenge on Ed balls i wonder if he hang on to his seat?

    ReplyDelete
  2. No, the difference Thomas's and Rothermel's research is that you feel that you can interpret the former, but not the latter, in a way that matches your prejudices.

    The fact is that your "explanations" are simply made up in order to justify conclusions that you'd already reached. If instead it was Rothermel's findings that aligned with your views then you might have said "So what caused such a difference in findings? Paula Rothermel's research was peer-reviewed and published at international conferences and in scientific journals; Alan Thomas published his opinions in a less closely scrutinised setting." It's not a difficult game to play.

    It is, of course, pretty ridiculous for you to criticise either piece of research when you yourself simply make things up and present them as fact, like your bold pronouncement that autonomous education is "the most popular educational method" among home educators.

    You are determined to discredit Dr Rothermel, but the only explanations you can come up with are that Rothermel is incompetent and that home educators are dishonest. Have you considered the more plausible explanation: that it is your opinions that are wrong?

    ReplyDelete
  3. Well, I am not at all determined to discredit Paula Rothermel's work. I simply took the trouble to plough through the evidence which she presents and weigh it up carefully. Since it is almost exclusively upon her work that the claimed efficacy of home education in the UK is based, I think that this is quite a reasonable thing to do.

    I did not make up the "bold pronouncement" that autonomous education is the most popular educational method among home educators. This is the conclusion which Alan Thomas reaches in his books. Read the books and then comment. He says that no matter how determined home educating parents were to stick to a highly structured method, almost without exception they tended after a while to adopt more informal methods. He worked with a couple of hundred families; upon what do you base your belief that this is not correct?

    ReplyDelete
  4. write to Graham for a job on his crap review panel!

    ReplyDelete
  5. I do not have a "belief that it is not correct". I don't know whether it is correct and neither do you, which is the point.

    Whether or not Alan Thomas made the claim about the popularity of autonomous education is irrelevant. The fact is that you have no substantial grounds for believing it to be correct, and yet you state it as if it were indisputable fact. When it comes to Paula Rothermel's conclusions, which do not fit with your prejudices, you suddenly become very concerned about evidence.

    ReplyDelete
  6. You say that I have no substantial grounds for believing it to be the case that autonomous education is the most popular method among British home educators. Of course, one cannot be sure. A couple of years on the HE-UK message board, EO and Herts-HE boards all tend to suggest that this is so. Of course these are not all home educators, but I also come into contact with a fairly random sample in the boroughs where I work in East London. The picture in Essex, where I live is similar. Books by Thomas, Durbin and Mountney all champion this idea. Perhaps you could let me know the details of some sites for or books by structured educators in this country? I spent a good deal of time hunting for such groups in the past without success.

    In any case, my prejudices, if present, are neither here nor there. If the research is sound, then the point of view of the person reading it should not matter. On the subject of educational attainment by home educated children in the UK, I am completely open minded. I am simply looking for some evidence. Perhaps you can help provide me with some?

    ReplyDelete
  7. What is your job? is it teacher by any chance?
    Graham Badman review is rubbish and every one knows it

    Come on Uncle Graham try geting into this house? I want you to watch us burning your review on the coal fire! or what about Uncle Ed Balls M.P coming to join us you think he try and stop us burning it on the fire it makes for good lighting paper!

    ReplyDelete
  8. How high do you jump Simon when Ed Balls /DCSF gives out its orders every day? check press releases from DCSF? your go far and may get on to Graham rubbish review pannel you be with like minded people! you could then find out if it is true that Graham hobby is wasp watching?

    ReplyDelete
  9. I am really not prepared to discuss my personal life. If you wish to discuss Rothermel and Thomas' research, then fire away!

    ReplyDelete
  10. Simon said,
    "When one looks closely at her work one finds that the literacy tests were not conducted under controlled conditions as they are in schools. She posted them out to parents, who then did them unobserved with their children."

    I haven't had time to re-read all of Rothermel's study so far so I may have missed the part you are referring to. Can you point out where it says the children took the literacy tests with their parents, unobserved? Are you referring to a different test to those mentioned in the quotes below?

    "All the PIPS Baseline assessments were administered by the researcher in person.... Administering the 35 PIPS Baseline assessments to home-educated children in their own homes twice over the assessment period, provided an opportunity for the researcher to conduct interviews with both them and their families."

    http://www.pjrothermel.com/phd/6%20Pips%20Baseline%20Chapter%206.htm

    ReplyDelete
  11. Are you o a teacher Simon? come on just say so or are you ashamed to admit that you do work for the DCSF?

    ReplyDelete
  12. Sharon, here is what Rothermel says about the reading tests, the PIPS Baseline assessments are a separate matter about which I shall have more to say later;

    Following initial contact to obtain consent, letters containing the assessments were distributed to the participants by post, together with stamped addressed envelopes for their return. An example of a letter relating to the NLP assessment is provided at Appendix 4.11.



    Parents, according to the instructions supplied to them, administered all assessments. These instructions were those provided by the CEM Centre/PIPS Project to the researcher, but modified to reflect the home, as opposed to classroom, environment.

    It is upon this evidence that an awful lot of the assumptions about educational attainment of home educated UK children are founded. I dare say you can see the problem. I am sure that if GCSEs were sent to pupils by post with instructions to go to their rooms and do them alone for the specified time, then eyebrows might be raised about how much the results were worth! I will be going a little deeper into this in another post, but it is enough for now to say that combined with self assessment of social class, I am beginning to wonder about some of the facts and figures that we routinely see quoted about this research.

    ReplyDelete
  13. I am sure that if GCSEs were sent to pupils by post with instructions to go to their rooms and do them alone for the specified time, then eyebrows might be raised about how much the results were worth!

    What about course work?

    ReplyDelete
  14. Course work? Precisely why my own daughter took International GCSEs........

    ReplyDelete
  15. Whoops! You forgot to mention that Paula Rothermel subsequently re-assessed five of the children herself in order "to ascertain inter-rater reliability". It's odd how you missed that, since it's in the paragraph that follows the sections you quoted.

    It's a shame that it somehow escaped your notice. If you'd seen it then you might have realised how silly your little scenarios about home educators irritably manipulating the test results would sound.

    For the benefit of your readers, you might like to point out that the children re-assessed by Rothermel scored almost exactly the same as they did when assessed by their parents (Table 4.2 of Chapter 4 of Rothermel's dissertation). There was, in Rothermel's words "remarkable consistency" between the two assessments.

    It is not difficult to imagine why you are so quick to suspect dishonesty in others, Simon Webb.

    ReplyDelete
  16. Simon said,
    "Parents, according to the instructions supplied to them, administered all assessments. These instructions were those provided by the CEM Centre/PIPS Project to the researcher, but modified to reflect the home, as opposed to classroom, environment."

    Why didn't you include the next paragraph in your quote?

    "Once completed, assessments were returned to the researcher. From the NLP cohort, 5 children[54] were re-assessed by the researcher, prior to any marking, to ascertain inter-rater reliability. The time-lapse between initial and subsequent administration varied between 3 and 5 weeks. The results of this second assessment can be seen at Appendices 7.1, 7.2 and 7.3. The differences revealed remarkable consistency between parent administered and researcher administered tests, as illustrated by Table 4.2."

    http://www.pjrothermel.com/phd/4%20Method%20Chapter.htm

    ReplyDelete
  17. Simon said,
    ""Course work? Precisely why my own daughter took International GCSEs........"

    But it doesn't appear to stop Universities valuing normal GCSEs(as you constantly point out). Yet you are happy to disparage the idea of parents being trustworthy enough to carry out literacy tests on their children for the Rothermel study, despite a random sample of 5 confirming that they were trustworthy. Have you ever considered that your requirements for levels of proof and your lack of trust in others are unreasonable, effectively above and beyond that required by universities as demonstrated by your preference for IGCSEs?

    ReplyDelete
  18. There was no intention to deceive here, I simply lifted off the paragraph that explained how the NLP assessments were conducted. We are given no further information about the five re-tests, nor why the initial tests were not done by the researcher. I am curious to know whether these five were picked at random, lived near Durham or were personally known to Rothermel. I could not get into the appendices which might have given this information. There are a number of questions here that I am looking at. Bear in mind that this is the only piece of research that supports the contention that home educated children in the UK achieve as well as school educated ones. It is important and needs to be examined.

    ReplyDelete
  19. "Bear in mind that this is the only piece of research that supports the contention that home educated children in the UK achieve as well as school educated ones. It is important and needs to be examined."

    Only if you completely dismiss all the research from other countries. Yes, they should be treated with some caution because they are from a different countries, but you have not given a convincing reason for dismissing them completely, especially when the evidence available in the UK agrees with their findings. I don't think you can dismiss Alan Thomas' research either. Case studies are a valid form of research and he concluded that education at home is a viable alternative to school.

    ReplyDelete
  20. By the way, I am not suspecting dishonesty on the part of parents. Just as one parent mentioned in the thesis told the researcher that there was no point asking her daughter about the letters of the alphabet because she would not know them, so another might have assumed knowledge without checking. This is human nature. That is why it so vital that the tests are administered by an impartial person. I am still very interested to know the explanation for the hugely advanced readers of Paula Rothermel's group as compared with those the Alan Thomas found.

    ReplyDelete
  21. Sorry, Sharon, did not notice your last post. I don't dismiss Alan Thomas at all. I find his work very good. Rather the same problem as Paula Rothermel though, small samples.

    ReplyDelete
  22. "That is why it so vital that the tests are administered by an impartial person."

    So why dismiss the 5 that were checked by an impartial researcher as evidence that the test results are accurate? As a PhD dissertation Rothermel's work was overseen by Dr Petrie and Professor Aubrey at the University of Durham. It seems likely that they would have been unhappy if Rothermel took this approach so I'm not sure why you would assume this purely because every detail is not given in the report, especially when parts of the research itself could not be included due to word count limits. Even if the 5 lived near Durham or were personally known to Rothermel (which seems unlikely if you read the section on how home educators were contacted), how would this increase the accuracy of the parent administered tests?

    ReplyDelete
  23. I said,
    "if Rothermel took this approach"

    Which isn't very clear.

    I'm talking about the approach of choosing respondents who lived near Durham or were personally known to Rothermel for re-testing by a researcher to check for inter-rater reliability.

    ReplyDelete
  24. Sharon, let me try and make clear what I am driving at. I have no doubt at all that home education can achieve brilliant results. The question is, how often does it do so and how often does it give worse results than schooling would have done. I have looked at the evidence for one aspect of education which can be compared betwen school and home educated children. This aspect is reading. I pointed out that the two main researchers in the field seem to have produced very differnet results and I am wondering why. Possibly, the methodolgy is at fault, perhaps the samples were very small; 35 as opposed to over 20,000 in one American survery. Perhaps there is another explanation.

    I would like you to bear in mind that I have been a home educator myself for many years and am not at all opposed to the practice. I would simply like to see more and better research conducted in this country. Five verified reading tests is not a large enough number to draw any firm conclusions about the state of British home education today.

    ReplyDelete
  25. Despite what you say about your intentions, your blog post is utterly deceitful. You claim to have looked closely at Rothermel's work, and yet you blatantly misrepresent her methodology. Your inaccuracies have been pointed out more than once in the comments, and yet you have done nothing to correct or remove your false claims.

    ReplyDelete
  26. Simon said,
    I am still very interested to know the explanation for the hugely advanced readers of Paula Rothermel's group as compared with those the Alan Thomas found.

    I would guess it had something to do with the self selected nature of the respondents in Paula Rothermel's study. If you are happy for your child to learn to read when they are ready and are against testing and measuring children you are not likely to volunteer for this type of study. But the least we can take from the two studies together is that both of these broad approaches to education can work, and work well.

    I'm still not sure why you insist on dismissing foreign research as irrelevant to the UK?

    ReplyDelete
  27. I don't dismiss foreign reseacr at all, Sharon. I found the recnet HSLDA findings fascinating, particularly because it was such a huge sample.What other foreign research are you talking about, apart from the American stuff? I think you are probably right about the self selection bias in Rothermel's work, but there is a lot more there that I find puzzling. I shall post more on this in due course.

    ReplyDelete
  28. What other foreign research are you talking about, apart from the American stuff?

    http://www.fraserinstitute.org/commerce.web/product_files/Homeschooling2007.pdf

    Home Schooling: From the Extreme to the Mainstream, 2nd Edition

    ReplyDelete
  29. "
    I did not make up the "bold pronouncement" that autonomous education is the most popular educational method among home educators. This is the conclusion which Alan Thomas reaches in his books. Read the books and then comment. He says that no matter how determined home educating parents were to stick to a highly structured method, almost without exception they tended after a while to adopt more informal methods."

    You do realise that informal learning is not the same as autonomous education? Autonomous education can include informal learning and structured learning. Informal learning will not be autonomous education if the parent decided what to teach informally. You seem exceptionally confused about autonomous education for someone who claims to have been around home education for 20 years.

    ReplyDelete
  30. Does it matter if by the age of 8 home educated kids are at the same/higher or lower level of reading than school kids? I don't actually class any child at the age of 8 as 'fully educated' yet, and as far as I'm aware, neither do schools. Otherwise they would be graduating at this age. Isn't the real question what they are like when they are say, 18yo? Do they enjoy learning? Do they want to learn? Do they have the skills to learn what they need to, when they need to, and find the resources and interest to do so? I would be asking these questions. I think you'll find Simon that generally speaking home educated kids have the ability to do these things, and generally, school kids do not. If it is not handed to them, they don't know how or what to do to learn, even if they had the desire. They have mostly lost the desire to learn anything, they will only do it if its required of them, and then only the bare minimum most of the time. There are of course exceptions from both sides of the fence, but generally this is the case.
    Also Simon, I have to agree with Anonymous here, autonomous and informal learning are definitely not the same thing. They can be, but they don't have to be.

    ReplyDelete