Hester Takes the Gen2 Survey

In December, my mother, who is still on several homeschooling mailing lists even though she’s officially retired from support group leadership, got a notice in her inbox about Brian Ray’s Gen2 Survey. Neither of us had heard of it, though apparently word had been circulating for several months (that’s what you get, I guess, for finally withdrawing, in large part, from the homeschooling community). Ray is the founder of the National Home Education Research Institute (NHERI), and as you might expect has released many studies of homeschoolers in the past, almost all of which have been criticized for methodological flaws (see here and here). His newest study is described as follows on its own website:

This is a survey of those between the ages of 18-38 years old that grew up in religious homes. Our goal is to come up with data points of key influences that either encouraged or deterred the participants from practicing the same faith as their parents. Dr. Brian Ray, a decades-long experienced researcher, is the principal investigator and will confidentially analyze the data. We will use the statistics from this survey to help equip parents to make more informed decisions in the education and spiritual guidance of their children. All results are anonymous. We greatly appreciate your participation in this effort.[1]

Personally, in light of the information discussed in the final section of this article by Kathryn Joyce, I find this new study to be interestingly timed (though this could, of course, be entirely coincidental). What I find even more interesting, however, is who is apparently helping Ray promote it. From the aforementioned email from Ray (emphasis mine):

Dear Homeschool Leader,

You have probably heard of this exciting Gen2 study that is in full swing – http://www.gen2survey.com. We are studying adults who were churched while growing up, including the homeschooled. Kevin Swanson and I explained this to many of you at the national conferences.

For those who may not know, Kevin Swanson is a prominent homeschool leader and the executive director of Christian Home Educators of Colorado (CHEC), and has come out swinging against groups like Homeschoolers Anonymous on his radio program, Generations with Vision, where he recently referred to them as “homeschool apostates.” He’s also known for saying downright bizarre things on occasion, most notably his claim that the wombs of women who take birth control pills are lined with tiny dead embryos. Nope, not kidding.

But enough about Kevin Swanson (at least for the moment). We were talking about the Gen2 Study. After rereading the email from Ray this week, I poked around a bit and found that the study is already garnering criticism, once again for its methodology. Well, since I rarely let the call of morbid curiosity go unanswered, I decided to take Ray’s survey for myself. I do, after all, land right smack dab in the middle of his target group (18-38-year-olds raised in religious homes). Unfortunately, in the process, I learned much more about Brian Ray’s assumptions than I did about myself or my upbringing. And today I’m going to share my revelations with you, my dear patient readers, and what they tell us about Christian homeschooling culture. I’ll be going through the survey page by page (there were 10). I will skip a page if it contained nothing relevant or problematic.

Page 1

The first question that jumps out as potentially interesting on this page is the third one, “What kind of church or religion do you currently associate yourself with?” Remember that in the description of the study’s goal above, Ray claims to be studying those who grew up in “religious” homes generally, not specifically Christian homes. Thus, when the list of religions and denominations under this question includes options like “atheist,” “Buddhist” and “Muslim,” it seems to fit with the stated goal of the study. What does not fit, however, are the some of the other questions on the page:

How often do you share the Bible with your spouse and/or children (family worship, family devotions)?

What is your favorite inspirational/religious book (or series) besides the Bible?

600px-Thanksgiving_Day_Service_Held_in_English_Country_Church-_Americans_in_Cransley,_Northamptonshire,_England,_UK,_23_November_1944_D22914In fairness to Ray, he did list “not applicable” as an option under the question about sharing the Bible with your family, but the assumption is clear nonetheless: Ray is thinking primarily of those raised in Christian homes, even though the study is technically framed to include other religions. The question about inspirational books makes this especially obvious, as its very wording assumes that the Bible is already the participant’s favorite book, or at least liked and highly valued. This is, needless to say, a bad assumption to make, given how many of the “homeschool apostates” have abandoned Christianity due to abuse or other factors.

Ray also made another interesting choice in his list of religions and denominations, though if you weren’t up on the doctrinal distinctives of various Protestant denominations, you would probably miss it entirely. Most of the other denominations are listed monolithically (Lutheran, Methodist, etc.), but for whatever reason Ray chose to divide Presbyterians into two groups: “Presbyterian such as EPC or PCUSA” and “Reformed Presbyterian” (and yes, I know that last one is inherently redundant). I’m not perfectly certain why Ray did this. It does, however, happen to align perfectly with the line in the sand drawn in modern Presbyterianism over female pastors. To the right of the line lie the PCA and various smaller groups, who reject the ordination of women, and to the left are the EPC and PCUSA, who either accept the ordination of women outright or have labeled it a debatable issue over which Christians should not divide.

This could mean nothing, of course, and I certainly can’t claim to have a direct window into Ray’s mind. Women pastors, however, are generally verboten in Christian homeschooling culture, so it wouldn’t be unusual at all if Ray chose to separate the “good” Presbyterians from the “bad” ones.

Page 2

The assumption of Christianity only continues on this page, in an even more obvious manner. Despite the fact that the possible answers to the question “Generally, what kind of religious service did you attend as a child?” include non-Christian religions (in fact, it even includes atheism and agnosticism – I didn’t know atheists and agnostics held religious services 😉 ), every other question on the page uses words like “church,” “pastor,” “youth group” and “Sunday School,” which are all flagrantly Christian. How someone raised in a religion other than Christianity would answer these questions is anyone’s guess.

Even more telling, however, are Ray’s assumptions about Christianity itself. Three of the questions were especially revealing, and obviously slanted toward the patriarchy-influenced Family-Integrated Church movement:

Did you attend a church that was family-integrated, age-integrated, or homeschooling-friendly?

When you were between 13 and 18 years of age, were you part of a youth group?

Did you attend a church Sunday School while growing up (ages 0-18)?

These seem to be “flag questions” designed to mark out those raised in an FIC-influenced environment, and the first is frankly just amusing. If only “family-integrated” and “age-integrated” churches are “homeschooling-friendly,” does that make non-FIC churches “homeschooling-hostile”? Ray’s also trafficking in loaded language, which has a specific meaning inside the homeschooling community. For instance, the church I attended growing up was “family-integrated” in the sense that both children and parents were present in the worship service. But anyone who “speaks homeschooler” knows that that’s not really what the phrase “family-integrated” means. It entails a number of other beliefs as well, such as mandatory homeschooling, opposition to Sunday School and an extremely strict, conservative view of male headship. Ray should have capitalized the term (as it is technically a proper name) to convey these connotations, or explained what he meant by it so those outside the movement could properly answer the question.

Finally there is the last question on the page, which made me quite literally laugh out loud:

My current church is serious about applying the biblical practices of eldership, shepherding, mentoring, and church discipline.

Ray might as well have rephrased this to read “my current church practices an extremely strict, controlling, patriarchal brand of Reformed theology and thinks all churches that don’t are neglecting their Biblical responsibilities.” There are so many theological assumptions left unexplained here, it’s hilarious. Other churches do practice things like eldership and church discipline, of course – but not in the way intended when all four of those elements appear together as Ray has done here. For just a taste of the necessary background, I recommend this brief history of the shepherding movement and its various cultic control tactics.

And for my more curious readers, yes, I did answer “strongly disagree” to this question, as the theological brew alluded to here is so profoundly contrary to Lutheranism at so many levels, that I suspect an extremely dangerous explosive reaction would occur were the two compounds to come into even the slightest contact. In fact, I wish I could have appended my “strongly disagree” with a hearty “alleluia” and a rebellious rendition of A Mighty Fortress Is Our God!

Page 3

On this page the focus shifts away from religion and more toward education. The second question asks participants if they received any “worldview training,” which is another example of homeschool-culture specific loaded language. (“Worldview training” often amounts to little more than indoctrination in young-earth creationism, strict gender roles and conservative political talking points in practice.) The answers to the last question (“Did your parents use corporal discipline (spanking) with you?”) are also quite revealing:

No

Yes, consistently and they were generally under loving control

Yes, consistently and often they were not under loving control

Yes, inconsistently and they were generally under loving control

Yes, inconsistently and often they were not under loving control

paddleI have two takeaways from this question. First, this seems to be a subtle way for Ray to ask whether the participant was physically abused by their parents – abuse being a failure to stay “under loving control” when disciplining your children. This, of course, betrays massive ignorance about the nature of child abuse, which isn’t always connected to spanking or even discipline.

Second, I find it interesting that Ray chose to label parents who spank less often as “inconsistent,” as opposed to using a more neutral phrase like “spanked rarely” or “spanked occasionally.” In reality, a parent who spanks rarely may be completely consistent; they may just believe that a spanking is warranted far less often than the authors of conservative Christian parenting books. Ray may not have intended this label as a value judgment on the frequency of spanking, of course, but it still smacks strongly of pro-spanking rhetoric, whether he realizes it or not.

Page 4

We find even more of the assumption of Christianity on this page, when Ray asks how often parents read the Bible to their children or “explained biblical principles” to them. However, I’m more concerned with the question “How would you describe yourself when you were a child?” and its possible answers:

I was very rebellious

I struggled with rebellion, but overcame it

I was always fairly obedient and honoring as a child

Notice first of all that question is not about how the participant behaved as a teenager, but as a child. Thus, Ray may not be talking about “traditional” teenage rebellion here, but “rebellion” as conservative Christian parenting books tend to define it – basically, a child frequently displaying “bad attitudes” and having any kind of long-term trouble obeying or respecting their parents. To be honest, I’m really not certain which kind of rebellion Ray means, and I find the question confusing, possibly because I never did fit any of Ray’s prescribed boxes here.

Page 5

This appears to be a “didn’t fit anywhere else” page, as it contains an odd assortment of questions about all sorts of topics. The first few questions look like part of a bad personality test, as Ray asks participants to name their favorite movies, video games, TV shows and musical genres. So to get into the spirit of the survey, yeah, Brian, I suppose I do like Lord of the Rings, and The Big Bang Theory. And for what it’s worth I am listening to the soundtrack to The Little Mermaid right now, though for all I know I could decide to switch to Renaissance polyphony in five minutes. Pretty much whatever keeps away the writer’s block will do. As for video games, Myst is pretty cool. You should try it, Brian. Amateria and Spire will knock your socks off. (Readers who actually understand those Myst references will win the Nerd of the Day award, and may be entitled to free cookies.) I do, however, remain puzzled as to how any of this relates to the goal of the study.

Ray next asks a series of standard hot-button political questions (legalization of gay marriage, abortion, etc.) before moving on to sex and alcohol:

During the past six months, how many times have you been under the influence of alcohol or drunk with alcohol?

Have you ever had a sexual encounter or physical relationship with some to whom you are not married?

Were you ever sexually abused before age 18?

On the one hand, I’m glad Ray mentioned sexual abuse, as the answers to the physical relationship questions didn’t allow for it (the only possible answers were “no,” “heterosexual,” “homosexual” and “bisexual”). On the other hand, he once again fails to define his terms, and in fact we find ourselves asking the same questions we asked after reading Doug Phillips’ resignation letter. What is a “physical relationship”? Sex? Petting? A kiss? It may sound like a silly question, but there are many in the homeschool community who include just about anything in the term “physical relationship” – for instance, we learned here that S. M. Davis thinks innocent hand-holding qualifies. If Ray is targeting this community (and it’s obvious that he is, at least in part), he should have worded his questions to reflect these huge differences in definition.

499px-Marsden_Hartley_-_Handsome_Drinks_-_Google_Art_ProjectThe same criticism applies to Ray’s alcohol question. What does he mean by “under the influence” of alcohol? My cheeks flush and I become more talkative after about four ounces of wine – am I “under the influence” when this happens? And if so, why am I required to enumerate it in the same column as outright drunkenness? I’ve never gotten drunk, and never intend to. But Ray has no way of knowing this, so if I write, say, the number eight in the box after this question, Ray may assume that I’ve been raving drunk eight times since July, when all I’ve done is had an occasional glass of wine after dinner.

A similar wrong impression would be left by my answer to “Growing up, who did you spend most of your time with?” I was a homeschooled only child, so taken in absolute numerical terms, I spent most of my time with my parents. But this makes it sound as if I were locked up in the house with little or no contact with the outside world, when in fact I regularly saw homeschool friends, other homeschool parents, my music teachers, and adults at church and AWANA. So the question may have been more informative if Ray had asked something like “With whom did you regularly come in contact as a child?”, as opposed to focusing on a simple majority of time spent.

Page 6

This page is similar to the last one in that it appears to have no pattern at all. Ray wants to know if I smoke, how many texts I receive per day, whether I’ve ever had an abortion, and how much volunteer work I do. There is another stray personality test question (favorite novel or series of novels), so for Ray’s sake I’ll just confess now and admit that I like Harry Potter, Lord of the Flies and The Scarlet Letter. You know, because that last one was such a big secret.

One question in particular that reveals Ray’s biases is this one about creation and evolution:

What is your perspective on the origin of life and species?

a. I believe in a literal six 24-hour-day creation.

b. I believe in God’s influence on origins, but not literal six-day creation

c. I believe in evolution

d. I don’t know

At least Ray allowed for “I don’t know” here, which he did not for equally controversial questions like abortion. More to the point, however, notice that Ray paints “God’s influence on origins” and “evolution” as polar opposites which can never mix, as if they were oil and water. Even a glance at theistic evolutionists and evolutionary creationists’ beliefs, however, reveals that this isn’t necessarily true. Ray may believe it is, but this is not a survey about Ray’s beliefs. Thus, he should at least fairly portray the potential positions he may encounter. If he doesn’t, he may get misleading answers that call into question the accuracy of his research.

Page 7

On this page Ray attempts, mainly, to delve into his participants’ view of their situation in life:

You have little control over the things that happen to you.

In most ways your life is close to ideal.

Your life often seems to lack any clear goals or sense of direction.

There is little you can do to change many of the important things in your life.

Overall Ray seems to have done better here, since all the above questions can be answered along a sliding scale from “strongly agree” to “strongly disagree.” Unfortunately, when he strays from this topic, he falls right back into imprecision and false dichotomies. Take, for instance, the second question on the page:

In general, how happy or unhappy are you with your body and physical appearance?

I suspect Ray thought this was a clear-cut question about body image. His wording, however, is far too general for that, because there are many different reasons people may be unhappy with their body and physical appearance. Maybe they don’t like their nose or their hair. Maybe they have a disease, genetic condition or injury that causes them pain or inconvenience. Maybe they want to lose weight, or they’re anorexic. They might even be transgender (though I suspect that didn’t even occur to Ray). So while Ray may find out how many of his participants self-describe as “unhappy” with their bodies, he won’t get much insight into why they’re unhappy, or even what they’re unhappy about.

The religion-related questions on this page had similar problems:

If you were unsure of what was right or wrong in a particular situation, how would you decide what to do?

a. Do what would make you feel happy

b. Do what would help you get ahead

c. Follow the advice of a parent or teacher, or other adult you respect

d. Do what you think God or the scripture tells you is right

e. Something else

800px-Old_latvian_bibleIn my opinion, this is probably one of the worst constructed questions in the survey. Most people, even Christians (and notice again that Ray uses only Christian terms for those who make decisions based on their religious beliefs), don’t use only one method to decide what is right and wrong. But in Ray’s world, we’re presented only with a hedonist, an opportunist, an advice-seeker, a Christian, and “other.” Ray also doesn’t seem to make allowance for cases where the Bible is unclear or silent on a particular topic (which would definitely affect my answer to this question in a pretty profound way). In fact, if we’ve already reached this level of moral uncertainity, there’s a good chance we’re dealing with one of those very issues, at which point, in the absence of clear Biblical guidance, we might choose a different option. Doing that, however, would make even a committed Christian look like they didn’t care about the Bible, due to the terrible construction of this question.

How distant or close do you feel to God most of the time?

This question, of course, is plagued with subjectivism – one Christian’s closeness to God, may feel like distance to another, and vice versa. Another problem is that Ray didn’t allow for an answer of “not applicable,” which is strange since he allowed participants to self-describe as “atheist” earlier in the survey. An atheist isn’t going to feel either distant or close to God, since they don’t even believe in His existence, so how are they supposed to answer this question? Lie?

Page 9

The first thing we encounter when we click into page 9 is a miniature doctrine test, in which participants are asked to rate their level of agreement with the following statements:

Jesus Christ is the divine Son of God

The Bible may be an important book of moral teachings, but it was no more inspired by God than were other such books in human history.

The concept of God is an old superstition that is no longer needed to explain things in the modern era.

Through the life, death, and resurrection of Jesus, God provided a way for the forgiveness of people’s sins.

Despite what many people believe, there is no such thing as a God who is aware of our actions.

Jesus was crucified, died, and was buried but on the third day He arose from the dead.

If Ray intended this list to be an “atheist detector,” then he’s fallen prey to imprecision yet again. For instance, it’s not only atheists who would disagree with the statements about Jesus; a Muslim or a Jew would as well. And Ray doesn’t really want to lump Muslims and Jews in with atheists, does he? That would hardly give him an accurate picture of his study group – unless, of course, he only wanted to separate Christians from non-Christians. And if that’s the case, he should have made that clear the from the beginning, rather than using the seemingly inclusive language in the study’s description.

Homeschoolers only

On page 3, participants were asked to indicate how they were educated in each grade (public school, Christian school, etc.). Since I was homeschooled from kindergarten on, I was directed to a page of the survey accessible only to those who indicated they were homeschooled at some point. Most of the questions on this page are pretty basic – for instance, participants are asked why their parents chose to homeschool them, and asked to rate their overall homeschooling experience. One, however, I found puzzling, though you wouldn’t know why unless you were intimately familiar with the internal politics of homeschooling.

The question is, “What curriculum was the predominant form used in your homeschooling?” This appears simple on the surface, and is followed by a list of curriculum publishers, including many recognizable names (A Beka, Bob Jones, Hewitt, etc.). One, however, is conspicuously absent: Sonlight. This seems odd to me, as Sonlight has been popular and well-known for years, and their booth at even smaller state conventions is usually quite large and prominently placed.

A few years ago, John Holzmann, co-owner of Sonlight, revealed that the company had been banned from the Christian Home Educators of Colorado’s convention. I strongly encourage you to read the whole story here and here, but here is the pertinent section in which Holzmann discusses the reasoning behind the ban:

…I would say my impression was correct that “evolution” and “youth-earth creationism” is, at this point, hardly the basis for CHEC’s objection to Sonlight. There is a very much more fundamental difference of opinion. Indeed, a whole slew of differences of opinion – about the kinds of books a Christian curriculum should use, the emphases of such a curriculum, etc. And Sonlight is clearly on the “outs” with CHEC’s current/newly developed (and perhaps, still developing) vision.[2]

Kevin Swanson, who is helping Ray promote the Gen2 study, is the executive director of CHEC, and appears to have been intimately involved in the aforementioned banning (see emails published by Holzmann).

So does this mean anything?

I really don’t know at this point. Truth be told, I have no idea how, or whether, Swanson was involved in the actual construction of the survey. He may not have been involved at all. What I do know – because Brian Ray himself said so – is that Swanson is helping to promote the survey. Thus, his name is prominently attached to and associated with the Gen2 effort (which allows for participation by the same “homeschool apostates” he recently criticized). And for the moment, unless I get more solid information, that’s all I can say with any certainty.

As a final note, another glaring flaw in the survey was that it appears to not remember IP addresses. In other words, it can be taken multiple times from the same computer, and since all participation is anonymous unless you decide to submit your name and email address for the iPad Mini drawing, this leaves the door wide open for the same person to take the survey more than once. If this occurred, of course, it would completely destroy the study’s credibility, as there would be no way of knowing which results were real, and which were “clones” trying to skew the results in favor of a particular outcome.

That, then, marks the end of my little romp through the Gen2 survey. Overall, due to bad question construction, faulty assumptions, and an obvious questioner bias, I’d have to say I was unimpressed. Oh, well. At least I got to enter to win a free iPad Mini. Though to be honest, I might have more fun with one of these…

Advertisements

7 comments on “Hester Takes the Gen2 Survey

  1. Jeff S says:

    “Presbyterian such as EPC or PCUSA” and ‘Reformed Presbyterian'”

    Wow- what a terrible question. The EPC is far closer to the PCA than PC(USA). Not even all EPC churches allow women pastors. At ones that don’t, they would be virtually identical to PCA.

    And yeah, “Reformed Presbyterian” is hysterical. I should inform my sister (who is PC(USA)) that she isn’t Reformed.

    • Jeff S says:

      Because clearly “Reformed” means “no female pastors”, despite the focus given to the issue within the WCF (which is, of course, none).

    • Hester says:

      Yeah, when I wrote that paragraph I was wondering how long it would take you to show up. 🙂 It also makes me wonder if Ray himself is Presbyterian, as he clearly didn’t bother to divide up the other denominations in such detail. What was even weirder was that the full survey answer was “Reformed Presbyterian or Reformed Baptist,” so apparently as long as you’re Calvinist and don’t believe in women pastors, nothing else matters. Not even doctrinal differences (baptism, church government, etc.) dating back almost 400 years.

    • Hester says:

      Oh, and as for “Reformed Presbyterian,” I think this picture sums it up nicely. 😉

  2. Wasn’t Myst created by homeschoolers? (Uru is my fave 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s