12 September 2011 in Mominformation.blog

People ask me about homeschooling outcomes, and until now I haven’t had much to say.

There have been published studies reporting high achievement among homeschoolers. But those studies had problems: They focused on a select group of homeschoolers — the children of highly-involved, affluent, educated parents who are eager to participate in a study about homeschooling.

If you compare such kids with the whole range of children in public school, you can’t be surprised when the homeschoolers perform better on standardized tests. They’ve got advantages that aren’t necessarily related to homeschooling.

The studies aren’t useless. They demonstrate that homeschoolers can flourish. But if you really want to compare apples to apples, instead of apples to oranges, you have to design a different sort of study.

That’s what Sandra Martin-Chang of Concordia University did in Canada. She and her colleagues decided to compare homeschoolers with public school kids matched for age and place of residence. They advertised for volunteers from both populations, so we’d expect homeschool and public school parents to feel similarly involved and enthusiastic about their children’s education.

Because homeschooling can take radically different forms, the researchers also decided to divide homeschoolers into two categories:

• Kids who “often” or “always” got structured, organized lesson plans, and

• Kids who “rarely” or “never” got structure

Then the researchers administered achievement tests to the kids, who ranged in age from 5 to 10 years old.

The results were quite marked. The structured homeschool kids were at least one grade level ahead in phonics, science, social science, and the humanities. In math, they were about half a grade level ahead.

And remember, their competition wasn’t a cross section of public school students, but a select group of kids who matched them in many ways. Like the structured homeschoolers, these public schoolers were performing above grade level. But not as much. And even after controlling for family income and mother’s education level, the structured homeschoolers still had the edge.

There’s a wrinkle, however, and it’s this: Unstructured homeschoolers didn’t perform nearly as well. In fact, they were the only students in the study to test a bit below grade level. If structured homeschoolers got an A+, unstructured homeschoolers got a C or C-.

Why? It might be a statistical blip. This is a small study (just 37 homeschoolers, and only 12 of those were “unstructured” homeschoolers). Because there were so few unstructured homeschoolers, we can’t conclude that unstructured homeschoolers in general get a C grade.

But there is reason to think that this study tells us something about the relative standing of structured homeschoolers.

Statistical analysis permits researchers to estimate the probability of getting these results — structured homeschoolers outscoring both public schoolers and unstructured homeschoolers—as a result of random chance.

It varies from subject to subject. But for several areas (like decoding words), the probability of a misleading outcome was less than 0.007, or seven-tenths of one percent.

The bottom line? Homeschoolers can say there is better evidence than ever before in support of their efforts. But as we always knew, approaches to homeschooling can vary widely, and these may be linked with very different outcomes.

Does this study mean that unstructured homeschooling–sometimes called “unschooling”–is a bad idea? No. The sample size is too small, and we lack information about causation.

Maybe some parents choose unschooling because their kids have characteristics than make them perform poorly on standardized achievement tests. Future research will help us figure that out.

More reading

To read more about this research, check out my new Parenting Science article Homeschooling outcomes: How do they compare?

"I’m a biological anthropologist and creator of Parenting Science, a website for parents who want scientific information about kids and parenting. Parentingscience.com has been praised by researchers and hailed as “a welcome antidote to the opinion dressed as science that parents are constantly being fed.” I earned my Ph.D. from the University of Michigan at Ann Arbor, where I was trained in evolutionary anthropology, behavioral ecology, and psychology. My interests in social learning and intelligence have led me to study the cognitive development in children and nonhuman primates. I’m also a keen student of evolutionary biology—especially the evolution of parenting. I hate dogma and preachy advice, and I’ve been really exasperated by the sorry state of popular parenting information, which is mostly personal opinions, folk theories, and pseudoscience. I make lots of mistakes and forgive the boo-boos of others. But authoritarian pronouncements give me hives. Before people tell us what to do, they need to explain their reasoning and cite the evidence. (http://blogs.babycenter.com/author/gdewar/)

The following are comments made by various readers of this article which I have cur and paste. If nothing else this and many, many other homeschooling blog demonstrate time and time again is that all kinds of parents are both competent and experts in their own way, something the Swedish government refuses to recognise because it refuses to accept the genius of each individual who is motivated, and refuses to believe that anyone who is not "certified" by them is incapable of teaching. One thing I know from experience is this: dedicated homeschooling parents make excellent teachers! Nature does the rest. (CCMW)

"I know that homeschool kids are usually ahead of the game because of the basic fact that they have one on one attention that public school kids do not get. One thing that needs to be taken into consideration in a study like this is the child's learning style. Some kids learn better with a more “unstructured” approach to learning while others need that structured school type environment. I think the most important thing to know is if your child will learn the best under a structured or unstructured schooling environment so that your child gets the best education possible if you are homeschooling" (Shelly)

"Kids who don’t spend tons of time practicing standardized tests don’t do as well on standardized tests? Shocking! The real question is, what correlation does success on these tests have with whatever your desired outcomes are? Unschoolers are usually hoping to cultivate independent, curious, creative adults – not necessarily kids who get perfect scores on their SATs" (Tory).

"It may be a good thing to be a creative person, but it also helps if you can handle standardized tests so that you can get into a good college" (Amanda).

"I know this sounds strange - but the information you need to pass the SAT or ACT can be picked up in 3-4 months of study, including how to fill in the little dots. And what constitutes a good college nowadays? The most expensive? The college that has the lowest drinking parties? Many unschoolers are taking 2-3 years of community college in their teens and are already suited for a degree by the time they are 17. Many unschoolers don’t see college as a goal. They go on to start businesses or work in apprenticeships. Rather than going $100,000 in debt only to come out with a degree and no job they are choosing to find what they enjoy first and then take any means possible to pursue that. Unschoolers are not “at grade level” on achievement tests because they don’t have academics and grades as goals. They play a completely different game - with different outcomes. The author was right stating that you cannot compare apples to oranges" (Aadel).

"I have not done “un-schooling”, but I am the product of a combination of public, private and homeschooling. And as such I think there are benefits to each type of schooling and success can be found if utilizing each one versus just one option. I had private school early in Kindergarten and again in 7-8th grade, I was home-schooled 4-5th and the rest was public. Homeschooling early gives the kids a boost, and a freedom that most kids don’t get. Structure is important especially very early I was probably more than a grade ahead, but then I am a nerd. Private schools (most, but probably not all) deliver specialized and more academic training and push your kids harder to excel. With that foundation, I say send them on to Public High School so they can learn social protocol. Because honestly most kids in high school don’t care about the school part, though as a nerd, again I did. I think this study helps show some relieve to those that get gruff for homeschooling and that more needs to be done on the “un-schooling” personally as a nerd, I’m not for that, but to each their own (Momof4).

"Completely agree with Aadel’s and Tory’s statements (respectively) above. For someone who believes that high scores on achievement tests are the the goal of education, this study certainly makes it look as if unschoolers don’t measure up. But if test scores aren’t your goal, then this study has no real relevance. We do homeschool, we lean heavily toward unschooling, and my son performs roughly average for his grade level on most standardized tests (we’re required in our state to submit test results, or I wouldn’t bother administering them). Sure, I could drill harder, hit the textbooks more, and focus on raising his scores, and he would probably perform better on the tests, but that’s not what I’m worried about, and it’s not what he’s worried about either. We have life to live, field trips to take, books to read, things to make and explore, volunteer work to do, and friends to hang out with. The goal of our family’s homeschool isn’t to fill our kids’ heads with all the facts they’ll ever need to know. Our goal is to raise our kids to have lots of life skills, to have character, to be thinkers, to love learning, and to be prepared to keep learning and growing throughout their lifetime" (Stephanie).

"I strongly (and unscientifically) suspect that if you (1) have strong principles and priorities and you live by them daily, and (2) stay involved with your child’s life and education so they will see that it’s important to you and you will spot problems before they balloon out of control, then whatever method you choose will yield good, happy, successful children. I personally recognize the (unfortunate) importance of standardized tests and will do my best to help my daughters do well on them, because while doing so may take a bit of our time, not doing so will probably rob them of a lot of opportunities for college, jobs, and higher pay. If they decide to be teachers, doctors, engineers, businesswomen, lawyers, accountants, or any number of other things, they’ll be pretty much out of luck if they haven’t done their best on those tests. If they decide to do something else, then they’ll be overprepared for the challenge, and hopefully comfortable with deadlines, structure, working in teams, keeping a schedule, and all those other grown-up skills that serve *me* so well on a daily basis. I think whatever path you choose, a parent’s job is to help find balance. As they go through the machine of public school, I’ll be striving to provide some of those unstructured moments of blissful raw discovery that can be so exhiliarating and so important. If I chose to “unschool,” I know I’d need to create opportunities for playing and working with others, and to impose some form of structure or schedule in their lives so they could deal with it when they got out on their own" (Jeremy).

"I am a product of total Unschooling. I stumbled upon this article while researching my Masters thesis and I had to smile. Before I came to college I did need to be taught the “rules” of school (I was used to writing to create a constructive, useful and pleasing document - not to touch upon the points a teacher keeps secretly in mind) so I took a few basic college courses as an early teen before enrolling full-time. Being unfamiliar with the “structure” of compulsory school has been a blessing and an advantage, and I hope to raise my children with the same opportunities" (Rachel).

"I wonder what the difference would be if you evaluated them using something other than a test. If, for example, you reviewed a writing sample rather than gave a multiple choice phonics test. Also, I’d be curious how unschoolers do as they get older. Are they not at grade level because their education is inefficient/ineffective or is it because they haven’t done the grade level material on that schedule because they’re doing other things instead" (Beth ).

I am a HUGE believer in structured homeschooling…..HOWEVER with that said….I also see nothing wrong with unschooling as long as they incorperate the important skills (like some math, reading, etc) b/c I know most unschooling parents do teach those important skills either by play, field trips, etc. Now for those who don’t then I would worry for the child only because of their future (such as would they be able to make it in college if they chose to go or be educated enough to get a good job). Of course I believe in 100% freedom how to raise your child the way you want so I will never judge another person’s choice!" (Bridget).
"“Are they not at grade level because their education is inefficient/ineffective or is it because they haven’t done the grade level material on that schedule because they’re doing other things instead.” Good question, Beth. In this study, researchers didn’t collect information on what parents were trying to teach or present to their children. To evaluate kids, researchers used questions from a popular achievement test known as the Woodcock-Johnson. Questions were arranged in order of difficulty, and covered 7 subject areas: Word identification, phonic decoding, reading comprehension, science, social science, humanities, and calculation (math). In my experience, standardized tests usually include a certain number of questionable “correct” answers, too. Sharing the cultural assumptions associated with these answers may help test takers discover the favored answer. And formal instruction may expose kids to these assumptions…" (Gwen Dewar).

"I do structured homeschooling with my children and it is awesome! My kids are ahead of the game AND they have most of the day (only about 2 hours of “school time”) to explore/play/read/whatever. In our state we have to test them, and my son who is 7 tested waayyyy ahead. One on one makes a huge difference!" (Sarah).

"I read your original post about this study from your Parenting Science website… and it was quite an interesting read! Especially this part: “Unstructured homeschoolers also performed worse than the public school kids did, though not by enough margin to rule out chance.” So, the students who were taught in public schools, with their standard curriculum and academic setting, didn’t do significantly better on standard tests than unschoolers, who are not usually taught anything standard or purposefully academic. Good to know all of the tax-payers’ money is going to good use…" (Lucy).

Three of my four unschooled children only learnt to read after the age of 8, so I suspect they would have been below grade level in the three tests relating to reading if they had been participating in this study. Now, at the ages of 9, nearly 11 and 14, they are all reading way above grade level" (Rina).

"I was an “unstructured” home schooled kid and ended up graduating when I was 15. My parents purchased a software called Switched on Schoolhouse that was wonderful to use during junior high and high school, and gave them the ability to monitor me and decide where I needed extra help. They also played educational games with me, though, and sent me to public school for foreign language so I could be around kids my own age. And I was given music lessons outside the home. When my kids reach junior high I plan on home schooling them, I would now but they both need the social life much more than I ever did" (Maria).

A more accurate picture of outcome between the two methods might be better obtained when the kids are older (17-18ish) rather than 5-10. One of my unschooled daughters did not really take to reading until she was nine years-old and now, two years later, she prefers books that might be included on a freshman/sophomore reading list. She also reads confidently out loud (from the same books) to her little brothers and sisters. I believe this is because I never forced her to read to me and because she has grown up hearing good literature. Skills do not need to be rehearsed year after year to acquire them if we just learn to wait for when the individual is ready. It is sort of like waiting for the ground to thaw before digging; we can dig and dig in frozen ground, make no progress, become frustrated as well as exhausted, or we can wait till the soil is ready. Under the right circumstances a break-through can happen almost overnight" (Kathi).

"“It may be a good thing to be a creative person, but it also helps if you can handle standardized tests so that you can get into a good college.” A person doesn’t need 12 years of standardized tests to prepare for taking standardized tests. Aadel is absolutely right in that takes only a few weeks or months (if that – it depends on the student!) of preparation. Of course the test scores would be lower; unschoolers don’t look to what some random testing company considers important in the world, to figure out where to focus their attention. Our attention is on living full, happy lives, learning as we go. Covering “the basics” happens as a side effect of living engaged lives. In other words, if you truly need it in the world, you’ll come across it – and learn it – by living in the world. Nothing needs to be separated out. I’d test the students a few years out of college. What will they have retained? What will they have used in the world? That’s what kids need to succeed. And every person will have different needs. AND different interests and passions. Ever seen “Who’s Smarter Than a 5th Grader?” Some people see that as, “Oh, we adults are so dumb, we’ve forgotten so much”, when the reality is, if the material was needed in day-to-day life, no one would forget it; we’d be using it all the time. It’s just that my kids get to skip the whole “preparing for life” sham, and go out and live real lives" (Unschooling Mom).

It’s too small of a study to mean anything really. But, giving it the benefit of the doubt, look at the ages! Ages 5-10? Seriously? If you bother to write about unschoolers or do a study about them, you should bother to know something about them! Clearly, someone missed something. Unschoolers are big fans of the “better late than early” philosophy of education. Not all of them, but many of them. They don’t push reading or phonics at an early age. So to do a test on a child that probably has been doing a large number of good things, but focused in the physical world, not the written word, and compare that to another child that has been pushed to read before Kindergarden- LOL, apples and oranges indeed! But, I’d love to see a study when kids are in their teens. That study would actually mean something, not simply be a result of some children being allowed to enjoy their childhood in an age appropriate manner. We’re not unschoolers. We wouldn’t want to be. But I hate to see a study (of unreasonably small size and lack of controls and understanding) be used to malign unschoolers" (MP).

I am excited to read all this research. I am a public school teacher, but I did a thesis on homeschooling. In my research I did see a difference between homeschoolers and unschoolers. Major! So this does not surprise me. Thanks for the information" (LK).

In the end, the tests don’t matter. I’d like to see someone follow up with all these groups in 15-20 years and see if they are having success in the real world (after college, even). Public school kid of prepares you for an office or factory type job - where you are expected to be doing certain things at certain times in a certain way. Un-schoolers need to “learn” that kind of structure. Unschoolers have an advantage in unstructured time. If they are not given specific direction, they can still produce a product. Great for unconventional or creative jobs. How are all these kids making it work for them, is my question. It’s wonderful to think that we need to focus on the process, but part of learning is the outcome" (MB).

"It won’t help unschoolers, though, so it’s frustrating to have one more thing out there that might turn away parents whose kids would really benefit from that approach. John Holt, who coined the term ‘unschooling’ said something that I thought was very a propos: “We can tell a good deal about the competence of a particular group of experts by the kinds of research they do or do not do.” It’s from his book ‘Teach Your Own’. If people want to know how kids are faring – look at the kids! Are they learning? (You don’t have to test to see learning! You just have to know the kid!) Are they engaged? Excited? Interested? I don’t know that one can test for creativity and divergent thinking… I’m sure someone, somewhere has designed a test that purports to do so, probably with government funding! But that’s also not a test I’d be interested in seeing" (Unschooling Mom).

There are indeed tests for creativity and divergent thinking, and they reveal many interesting things (including the fascinating finding that most people in our society become LESS creative and divergent over time, something I’d think unschoolers would be interested in). That’s the sort of stuff that enriches all of us, helps us understand how humans develop, and helps us investigate new ways of doing things. If you eliminate all tests, then we can’t study kids at all except through impressionistic, qualitative studies (that depend on subjective experiences of whoever is in charge of the study). We need both approaches to learn and stay flexible. I don’t want to live in a world where I have to trust self-described experts because they are the most charismatic or most traditional or most popular or most adept at speechifying. And how is it that this study doesn’t help unschoolers? It hints at new questions and suggests new directions for research, several of which have been mentioned by you and other readers. When somebody takes the time and care to run a controlled study, and then explains it so you can draw your own conclusions and ask your own questions, that’s constructive, fair, and democratic. It’s a process that’s helped improve a lot about education. And the truth is, there has been too little of it–of rigorous, controlled research about education" (Gwen Dewar).

Thank you Gwen for an interesting article. I’m starting a PhD in educational research myself and have been annoyed by the small number of well-done studies in many areas of “educational research”. It seems as though many homeschoolers/unschoolers who have read this article are not in fact reading the results correctly. There are two main points – if the study hadn’t differentiated between the two main types of homeschooling, then homeschoolers as a group would have done marginally better than public school if at all. By differentiating, it shows that those who tend to focus on what schools do tend to do much better. Secondly, for the unstructured/unschoolers, why did they do worse than structured homeschoolers? Is it the testing format? Is it that in a few years they will catch up? Is it that the tests focused on, as many people say, knowledge that unless you enter that profession you will never actually use but is considered age-appropriate in a public school system? After all, the sample size is so low that you can’t really tell, therefore this study isn’t something to get angry at, but to use as a springboard for future studies and areas to question. Even if kids are excited about what they’re doing, there is a phenomenon that all teachers who work with kids have seen – “hands-on, minds-off”. You may have an amazing experiment teaching kids about buoyancy and how boats work, but all they see is playing with clay or foil to make a cool-looking boat. Your planned lesson take-away? Buoyancy knowledge. Their take-away? I got to play with clay! Minimal learning, but a lot of excitement and engagement. You have to have a way to find out what the students have learned, and while standardized testing may not be perfect, it is one reasonable way to figure out what people know. As an engineer, I wish more people had the background in scientific reasoning and were able to read and understand the results from scientific studies rather than just reacting to sensational newspaper headlines throwing the real results and correlation/causation out the window. I’m quite happy to have found your site and will be perusing your archives!" (Ana).

"Thank you, Ana, for contributing. It’s very encouraging to hear from people like you. And thank you for spelling out a point I had meant to make earlier [but forgot once we got into the uses of scientific inquiry ]: If the researchers hadn’t distinguished these two groups of parents, they might not have detected any advantages for homeschoolers. When I’ve looked at the reaction to this study around the web, I’ve been unpleasantly surprised. The people I’d expect to be the most open-minded and interested in educational research (How do kids learn? How can we make learning more effective? What are the costs and benefits of each approach?) are expressing the least curiosity and the least familiarity with basic principles of scientific problem-solving. Oh–and I almost forgot to say how much I enjoyed your example about the buoyancy lesson!" (Gwen Dewar).