Category: Culture

Unreasonable expectations, part 2

Posted on by Suki

This is a second in a two-part piece about the current state of educational testing in our K-12 schools. The first half covered the reliability of standardized testing, whether we should be using standardized tests for younger children, and digital educational design. Click here to read Part 1.

The tests don’t test what we think they test

My informant pointed out a huge problem with her third graders taking the test: much of the test had no audio component, and assumed that they could all read and write well enough. But as anyone who has taught children to read will tell you, some kids just learn later. They don’t learn worse and it has nothing to do with their intelligence overall. Late readers are not less successful in life.

The test will now assess one skill ahead of all others: typing.

But here’s what she had to say about her group of kids: “There was no audio component to the math, so a lot of the test was really a reading test. If they couldn’t read the paragraphs, they couldn’t answer the questions. And they sure as heck couldn’t write a paragraph.” The Common Core assumes that if you understand something, you should be able to write about it. (I won’t get into the question of why any reasonable 8-year-old would actually want to write about math!) But clearly, the less able readers were not being tested on their understanding of math—they were being tested on reading, which depressed their math scores.

On top of that, this test is also a test of a skill most kids don’t learn until middle school: typing. “And OMG they have no typing skills. I’m not sure a 3rd or 4th grader needs typing skills in general, but they were not ready to type for a grade. It was painful to watch.” Again, their math skills took a backseat to something the test designers didn’t even take into account. If we really wanted to find out their mastery of math, we’d let the teachers read the instructions out loud and type for the kids, or install voice recognition software so they could dictate.

Unreasonable expectations:
Standardized tests have been around for a long time, and over those long years, we have learned a lot about them. Here are some things we know about the tests themselves:

  • They are inherently biased. They can be made better and better through tinkering, but they can never reach the stated goal of being instruments that find out “what a child knows” because some children, for a variety of reasons, will never do well on them regardless of their mastery of a subject.
  • They are not good predictors of much of anything except how well a child will do on his next standardized test. The SAT, a much better test than any ever designed by a state government, is retooling itself because of the much-publicized research that shows, conclusively, that a good SAT score predicts absolutely nothing. Except, maybe, a good GRE or LSAT score!
  • They do not measure the worth of a teacher. Great teachers have all sorts of effects on their students’ lives, but improving their students’ standardized test scores is not a given effect. You can have a great teacher who does amazing things with kids who does not bring up their test scores.
  • They do not measure the effectiveness of a school. There are so many other factors that are as important or even more important than test scores. Test scores are one tiny factor that administrators can use to judge schools, but they are not the most important factor by far.

Yet our unreasonable expectations of this test are that it will somehow:

  • Be better at testing all children at their own level. See the point above about the inevitable bias. These tests won’t do any better than other tests. Sure, the kids who have trouble tracking from a test booklet to the correct bubble to fill in might do better, but these tests will inevitably end up biased against some other group of kids.
  • Predict a child’s success outside of test-taking. No, these tests will not predict any such thing. They will merely predict how well the child will do on the next standardized test. Period.
  • Show how well a teacher is teaching. This is absolute idiocy and any idea that teachers should be punished or rewarded based on test scores is rooted in a deep cultural distrust of teachers, not in any sound educational theory. Some teachers may indeed bring up their students’ test scores, but I sure hope those teachers are also doing something useful for their students.
  • Give us a way to “rate” schools. I have my own personal way to rate a school. I walk into the school and watch. In a great school, the students will be happy and relaxed. Yes, they may also be deeply focused on what they are doing, but that doesn’t mean they’re not also happy and relaxed. The parents will enjoy the school and feel welcome there. The teachers will feel energized to come to work; they will feel a partnership with the school administrators, other teachers, their students, and the parents. None of these important factors is represented in a composite test score. Yes, the score is a useful piece of information, but it alone does not rate a school.

Until we as a culture deal with our unreasonable expectations, it doesn’t matter how “good” the test is. A standardized test is a measure of how well students take standardized tests. In other words, it’s a measure of how much vocabulary they have heard in their few years on this earth. It’s a measure of what their parents discuss at the dinner table, assuming they have parents, a dinner table, and food to put on it. It’s a measure of how often the people they spend the most time with (and this is not teachers) talk about numbers in real life so that they become comfortable with number sense before being required to learn other skills that build on number sense.

A standardized test is also a measure of a child’s personality—nervous, anxious children don’t test as well regardless of their background. A child who didn’t have protein with breakfast won’t test as well. A child in the first day or two of coming down with the flu won’t test as well as she would otherwise. A child who lives daily with the fear that his older brother will be shot by his friends won’t test as well as he should. A child who is told he is too stupid to learn won’t do well on tests, and a child who has been overpraised about her intelligence (ironically enough) won’t test as well.

In conclusion, there are simply too many factors within the messiness of one person’s little life to put such weight on the results of a test. Sure, let’s make a better test, because we always need to improve the information we gather. But let’s not think that this test is going to solve any educational problems we have. It’s just a test, imperfect, limited in scope, and vulnerable to bias and technical problems. Education is just too important and complex to be judged by such a narrow, flawed instrument.

Unreasonable expectations, part 1

Posted on by Suki

This is the first in a two-part post about the new tests being administered through the Common Core. To find out more background on these tests, visit the Smarter Balance Assessment Consortium.

It’s that season again, the one that used to involve lots of filling in of bubbles. This spring, Google is giggling all the way to the bank as our schools purchase carts full of Chromebooks to have their students take the new, Common Core-aligned computerized tests.

Reports have been filtering in from around the country, with tales of crying children, broken software and hardware, and lots of overworked IT guys. But I wondered how things were going locally and talked to a teacher from a shall-remain-unnamed local public school. (Not my daughter’s school—her class hasn’t gotten to take the test yet because the district is worried about too much net bandwidth at one time so they’re spreading out the pain.)

The tests are new, and this year they “don’t count,” which actually doesn’t mean that they aren’t taking data from the results. The data, in fact, will be very important. We the parents, however, will not get to see our children’s scores, nor will the scores be used to fire our beloved, hardworking teachers. Not yet, at least. The data they’re taking is supposedly going to improve the test itself, and from what my teacher-informant tells me, there’s room approximately the size of California for improvement.

Reliability of the test itself

This is the issue that, it seems, the state is most concerned about, but frankly, it’s the least of our worries. My informant tells me that there were questions that required answers to proceed, but the test offered no spaces in which to put answers so the students couldn’t proceed. OK, that’s a simple software problem, but since the teachers aren’t supposed to “help” the students in any way, kids like my very literal daughter would have just sat there, unable to proceed.

There was no way for my informant to judge the quality of the content of the tests, but I’m sure we’ll find out that these tests have all the same problems as other standardized tests: multiple choice questions for which there are two, truly valid answers; deliberately misleading questions; fuzzily worded questions that don’t actually have a valid answer, etc. That’s par for the course in state-designed tests, and I really don’t know that there is a fix for it.

Appropriateness of the test for the age group
examcomicFrankly, I don’t think any standardized test should be administered to any child under the age of, say, 12 except in situations where you really need certain specific information. The very word “standardized” says it all—by creating a common standard you end up judging seals by how well they climb trees.

However, that said, if we must test younger children we can do two important things to make sure the test is appropriate:
1) Don’t make the test too long.
Let’s face it, even if the above-average 3rd-grader can sit for an 8-hour test over three days, most kids suffer.
2) Don’t create a test the requires tools that some kids might not have mastered.
For example, the old bubbles were a challenge for some kids, especially those with trouble tracking their eyes from the booklet to the answer sheet.

This test fails miserably on both counts. This year’s test was shorter and my informant said her 3rd-4th graders did OK, but she can’t imagine them hanging on for next year’s 8-hour test without some of them suffering terribly. Just because we adults have become office drones attached to our computers doesn’t mean our 8-year-olds need to be! If we really want to know their achievement level, why do we administer tests in such a way that will make it impossible for them to do their best?

And then there’s the whole question of asking young children with varying degrees of familiarity with technology to be able to use a computer with a trackpad, little tiny icons, and little tiny boxes they have to click in. Imagine the difference between the speed of a well-off kid who owns her own iPad and a kid who has no computers in the home—this is clearly not fair and clearly not developmentally appropriate. The number of hours of exposure in school is not enough by third grade to expect mastery of these physical skills by kids who don’t practice at home.

Digital educational design
I had a very bad feeling when it was announced that our tests would all be delivered by computer. Yes, there are some great aspects of this. No more tracking from booklet to answer sheet. No more one-test-fits-all since computers can adaptively offer questions at each student’s level. No more checking patterns of erasure after the teachers have had unmonitored access to the tests.

On the other hand, I started in digital educational design in the 90’s, creating the first online classroom materials for our local community college. The teacher I worked with on one project had learning disabilities and was a passionate advocate for his learning disabled students. Instead of a paper textbook, he and I created a website that had resizable text and also audio versions of the text. (Since screen reading software wasn’t advanced at the time, he recorded the whole thing!)

This experience led me to be keenly aware of the fact that online educational tools create very different challenges, and not everyone who is hired to design these tools is really qualified to do it. (I’ll save my rant about the quality of educational IT in general for another time!)

My teacher-informant reported a shocking first fact: Her school had “chosen” not to let the students take the tutorial that teaches them how to use the test environment first. How is the state letting this be a choice? Obviously, any school administrator who looks at the enormous pile of curriculum they’re required to get through is going to try to “save” tutorial time for something else. But in order for the tests to be effective, each and every student should be required to use a tutorial until s/he reaches a minimum standard of proficiency on the tools. Any student who can’t get up to speed on a tutorial should not be allowed to continue with the test.

This should be obvious to the people who designed the test, since (theoretically) we’re not designing these tests to prove that economically disadvantaged students are “stupid,” right? (Or are we?) You might think that I’m exaggerating how much trouble these kids have with the technology. However, my informant’s students are largely not low-income, yet she reported a number of problems, most of which she was not allowed to help with:

  • In the first part of the test, the students themselves are required to type their name in all caps (Chromebooks don’t have a caps lock key), an i.d. number with mixed numbers and letters, and a session passcode that had both 0’s and O’s in indistinguishable type.
  • And then there’s the use of icons with no text, one of my major pet peeves. Yes, there are those who think in pictures, and they all love Ikea’s instruction sheets. The rest of us, though, need language. I’ll let my informant describe what it was like to watch kids with varying levels of exposure to modern technology deal with this: “The kids don’t know the speaker icon is for hearing stuff. Some can’t read the directions. For example, they are given a paragraph and the directions are, Highlight the sentence that is out of place. They don’t know that they are supposed to highlight a sentence. They are looking for the dot to click or the space to type something. AND I CAN’T TELL THEM they are supposed to highlight a sentence.” Cuz that would be helping, right? And God forbid we let teachers help… the kids might learn something.

Continued:

Click here to read why the tests don’t test what we think they test, and why our expectations for this test really are unreasonable.

What’s a parent to do about health and diet?

Posted on by Suki

I think parents around the Internet responded strongly to the article Puberty Before Age 10: A New Normal?, published in the New York Times, because it hit on how vulnerable we feel when it comes to making decisions about food. It’s extremely hard to decide what to believe on the issue of how to feed our children a healthy diet and avoid dangerous substances in their food. There is so much competing information out there, and so much “evidence” that is cited that really isn’t evidence at all. How can we make decisions when the information we get is so confusing?

First of all, what sort of evidence should be considered “hard” evidence? The difference between “hard evidence” and “evidence” is that hard evidence withstands rigorous inquiry:

  • It’s reproducible
  • It can be seen on a large scale
  • It doesn’t go away in double-blind testing

There’s lots of “evidence” out there that indicates that some things in our environment may be making our kids sick. But how do we know how to react to all this information?

The problem with knowing whether we are “causing” these problems with the chemicals in our environment is that human bodies are so incredibly complex. And the problems we’re seeing are complex. Put those two things together, and designing a strong study becomes nearly impossible. Sometimes scientists can look at epidemiological evidence (longterm evidence, for example, that a certain type of cancer is rising), but that doesn’t answer the important question: Why? What’s causing it? Is only one cause or a combination?

Here’s an example of this complexity: I wrote at one point about the clear evidence, from the experience of many parents and with some evidence from studies, that gifted kids are more likely to have what is called “reactive hypoglycemia” and that they are likely to respond well to adding Omega-3 oils to their diets. Someone wrote back telling me that if my child was acting crazy when he was hungry, then there must be something deeply wrong with my child and he needed a full medical work-up. But no, many parents see that if we keep enough O-3 in our kids’ diets, we see marked improvement. And since that’s an easy, healthy way to fix the problem, and it’s very hard to find “hard” evidence past that, parents have to be satisfied that at least there’s a workaround.

But how would we get “hard” evidence for something as complex as the behavior of certain kids when they are hungry? First of all, they would have to take parents’ opinions out of the mix, because we are unable to be truly objective observers. But next, they’d have to find a way to verify that they have a large sample of kids who have this problem, and since it’s hard to test the problem without living with the kids, you’d have to put parents back into the formula. In other words, there’s no “hard” evidence for this phenomenon or the cure, but that doesn’t mean that the problem doesn’t exist or that the cure doesn’t work.

In the case of what’s causing this “early puberty” and how it can be fixed, we have even greater complexities. Is the onset of puberty really getting earlier in our country? Is early breast development the same as early puberty or does it just have one obvious characteristic in common? And even if we can prove it’s happening, how can we prove that a substance nearly ubiquitous in our food supply is the cause? One of the reasons it took so long to prove that smoking causes lung cancer is that smoking was so ubiquitous, scientists had trouble proving that it could be a factor. (Even one of the lead scientists on the first major study doubted they’d get any clear results… the day they started compiling their data, he quit smoking, but still died of lung cancer a few years later!)

I talked to my kids’ pediatrician about this and he said he believes, as I do, that the influence of diet on our health is the major new frontier in medical science. The way our healthcare and scientific systems are set up, however, it’s really hard to get funding for this type of research: the “cure” doesn’t involve a drug that a company can benefit from. Unfortunately, this leads to a lot of non-MD/scientists getting into this field — people who are into alternative medicine and are seeing marked improvements in patients just by dietary control. But since that field is also rife with charlatans (who just want to make money) and true believers (who are willing suspend rational judgment), the insistence on rigor and sound data goes out the window.

Correlation vs causation
From XKCD.com

As to how we parents should respond to this confusion, it’s clear that each parent has to make her own decision based on principles she believes in. Some parents are willing to make huge, difficult changes in their lives in case the people who espouse radical opinions are correct. My family’s approach involves straightforward lifestyle decisions:

  • Avoid milk/meat raised on hormones and antibiotics
  • Avoid BPA-lined cans and no heating in plastic containers.
  • Mostly organic produce, with an emphasis on all organic for plants that show high residues (root veges, fruits that you eat the skin of)

But I am not going to turn our lives upside-down for this, because I also believe in a balanced, enjoyable life. Thus, yes, sugar is pretty bad for us, but we love desserts and do eat them in moderation. And yes, we would be better off eating only whole grains, but pleasure in food is also part of health and frankly, sometimes a lovely croissant is just what your body and soul needs!

Finally, I think it’s best for parents to avoid reading inflammatory articles if these articles affect them negatively. Definitely don’t go looking at these magazines whose sole purpose, it seems, is to make us fearful of modern life. We are living longer and healthier lives because, in many cases, of the very same advances in applied science that may be making some of us sick. It’s very complicated, but with reasonable people taking reasonable care to sort it all out, things will continue to improve. As parents, we have an obligation to make our concerns known. But we don’t have to torture ourselves about the decisions we make – sometimes we just have to go forward with what we’ve got.

 

Neither good nor bad, but very different

Posted on by Suki

Recently I spoke about blogging at my son’s journalism class, and we talked about a lot of aspects of how journalists’ lives have changed due to the Internet. I talked about how my blogging is probably the equivalent of the ‘zines that used to pass between people who shared similar interest. When I was in my 20’s, for example, I read zines about music. They were little photocopied pamphlets that a person would make and sell to others with similar interests. The likelihood that a zine would reach even 1000 people was very small—back then, it was so hard to find people with similar interests because you were confined to the physical world.

These days, every blog is like a zine but with some obvious differences: for one, it’s free, and more importantly, you can reach so many more people. Blogs range from those with no readers (save the writer) to those with millions of readers. The effort that the bloggers put forth ranges from occasional musings to the equivalent of a full-time job. And the rewards of a blog range from a charming pastime to, in a few cases, a decent income.

What blogs don’t have, what has largely been dumped from the process altogether, are editors. Blogs are a journalistic free-for-all—taken as seriously by many people as serious news websites, they are not held to any journalistic standard at all. This gives bloggers incredible freedom, but also means that bloggers don’t benefit from the intellectual back-and-forth that characterizes a good writer-editor relationship.

Do I think this is a bad thing? Well, on the one hand, yes, we all suffer from the loss of editorial control. Writers suffer because editors force us to be better writers and to think more deeply about what we’re writing about. Readers suffer because we are presented with such a range of content that it’s hard to discern what has been written thoughtfully and with attention to facts and what has been dashed off by someone in a steam. Media outlets suffer because they think that they’re gaining—cool, we don’t have to pay those pesky editors anymore!—but actually they’re losing quality, credibility, and the maturity of writers who have someone to answer to.

Back in the day of ‘zines, the zine writers didn’t have to answer to anyone, and that made them exciting. It was so fun to read someone’s uncensored opinion. At the same time that I might pick up Spin Magazine from a newsstand, I’d read some photocopied zine with a person’s bold thoughts in it. Spin would have access to all the stars and all the opinion makers, but the zine would be much more fun.

Blogs have blown the world of journalism wide open. The world has changed and will never be as it was. Is this a good thing? Sure. Are there things I think we’ve lost? Most definitely. I hope that as the Internet matures, we all learn to find both the fun and excitement of uncensored opinions and also the challenge and importance of well-reasoned, well-edited journalism. Right now, I fear that such journalism is just going to go away, but I hope that things will shift as we all become more savvy consumers of online media.

How the world has changed 

A girl dying of leukemia made a request to hear holiday caroling outside of her home. 6000 people turned up. Would this have happened in the days before social media? Certainly, spontaneous, large gestures did happen, but behind the seeming spontaneity was a group of hard-working people who had to physically round up the participants. Social media, blogs, and the immediacy of the Internet has changed the landscape of our lives.

iPotty, uPotty, we all scream for iPotty!

Posted on by Suki

Well, the votes have been tallied up and the winner has been decided. The award? The TOADY, given by the Campaign for a Commercial-Free Childhood to the toy most deserving of our parental disdain. The competition is really tough out there, but this year the award has again gone to a toy built around a screen, the iPotty.

iPottyWhy, you might wonder, do these parents hate screens so much? Are we all luddites, looking back to the past before technology took over our lives?

That is, perhaps, one explanation. It’s natural for parents to compare their children’s lives with their own and wonder whether they’re doing the right thing. We remember climbing trees when we were kids though now we’re raising our kids in a treeless desert, or we remember how much we loved our public school though we have chosen a private Montessori, or we remember the joy of eating a bologna and American cheese sandwich on Wonder Bread while we’re raising our kids whole grain vegan organic.

But most parents are pretty quick to separate silly nostalgia from serious comparison. We may expound on the delights of eating Ho-ho’s while watching Gilligan’s Island, but that doesn’t mean we think it’s the right choice for our kids. Most of us actually make choices with some amount of thought, and we know that we make compromises each and every day. If we didn’t come to peace with our compromises, parenting would lead directly to a padded cell.

There is nowhere so fraught with compromise than how we parents have allowed screens into our children’s lives. Most of us probably grew up with television, but none of us grew up with cellphones that play high-resolution video games. The change that our society has gone through is extraordinary, with today’s children facing an adult future dominated by jobs that didn’t even exist when we were kids.

My own parenting life has straddled this change. A very useful book I got during my first pregnancy reviewed various brands of baby equipment and noted that some of the companies even had websites! If I were pregnant now, I wouldn’t buy such a book—I’d be reading blogs, consulting reviews submitted by thousands of parents, and subscribing to Facebook pages.

But despite our longing for a past when our kids actually wanted to go outside and play, there’s a much bigger and better reason for parents to reject a product like this: Our small children simply don’t need screens. Every piece of evidence gathered about babies and toddlers is that they learn through human interaction with the real world. Babies who are regularly put in front of screens have measurably lower IQs. They don’t bond as well with the adults in their lives. They don’t get on the business of learning what children their age should be learning. [Read this great piece by Media Mom.]

Apptivity Seat
And hey, how ’bout the “Apptivity Seat” from Fisher-Price? Yet another bad idea to put kids in front of screens.

Yes, I’m sure that some study will come out showing that babies who use iPads have quicker reflexes or learn to track small objects earlier than other babies. But that’s not the point. Babies with screens are hitting the pause button on the business of being babies. In my family, we use technology as useful tools for learning, working, and entertainment. But when technology gets in the way of life, it’s time to turn it off and get back to real life.

And how much more “real” can you get than potty training? Integrating screens into an essential physical learning process is silly at best, psychologically damaging at worst. If I had small children now, this is one compromise I personally wouldn’t be able to come to terms with. Introducing screens into the bathroom is an idea that simply…eh…stinks.

Related posts:

Now available