Infrastructure: canalization of society

Some human phenotypes have very low levels of canalization, such as many gender roles (e.g., pink is for girls). Others, such as having two arms and two legs, are quite canalized.

If the environment suddenly changed such that walking upright or having a large cerebral cortex was dangerous, humans would be in big trouble.

I think societal infrastructure is like the canalization of traits of a species.

Education reform

Not long ago it was necessary to travel to have access to knowledge.  Libraries were built.  Schools were built.  Eventually, buses were purchased and children all over the country were transported to these schools each day.  Thousands of teachers teach essentially the same lessons to kids everyday, with varying degrees of quality.  Housing kids at schools during daylight hours made it much easier for parents to be employed outside of the home.  Our society is very much structured around this.

However, we now live in a world where anyone with access to the internet does not need to travel to have access to knowledge. Further, it is extremely inefficient (horrifyingly so) to have thousands of teachers simultaneously developing lectures for the same topic.  In case this is not clear, consider the following two approaches for teaching algebra I to students:

scenario 1:

• each algebra teacher (there are thousands of them) develops her/his own lectures (this is many hours of work per teacher, where each teacher is essentially replicating the work of the others)
• they use whatever standard textbook the school district chooses

scenario 2:

• people around the country develop interactive algebra 1 lessons (including animations) and submit them online (to something like Khan academy)
• users vote on which lecture is most effective (similar to how memrise gives you several options, and you can rate them)
• algebra students can go to the site and work through the best lesson or lessons

In scenario 2 all it takes is one extremely talented and passionate person to make a great algebra lesson.  This tool can then be used by everyone for many years.  That is extremely efficient, relative to scenario 1, where there is a lot of replication of much lower quality work.

However, it is very difficult to design a new educational system, when the current infrastructure is so canalized.  We are starting to see resources such as Khan academy used in classrooms, but the basic setup of children being bussed to public schools to be segregated by age, to deal with chimp politics, to follow a rigid schedule, while learning from dry textbooks, hasn’t really changed.    It’s all small adaptations while maintaining the key features of the species.

In general, the richest societies build with the technology of the day.  Decades or centuries later, it’s potentially easier for a society that is developing for the first time to build with the new technologies, than it is for an older society to rebuild.

Read Full Post »

Teacher and school markets

I am not in favor of giving teachers pay raises strictly based on years of experience and training/education, which I believe is the current standard (for public schools).  Ideally, salary would be proportional to quality for teachers and principals.  However, evaluating teachers and schools can be challenging.  For example, there are several major problems/challenges with using student test scores as the marker of success.

Problems with test score based evaluations:

1. teachers teach to the test; the focus would likely be on skills and memorization; this could lead to a more homogeneous and less creative group of students than is ideal for society

2. selection bias:  substantial variation in student quality across schools (by student quality here I mean independent from the effect of the school)

3.  incentive to cheat:  both teachers and administrators have incentive to cheat.  if the metric is a change score, then there is incentive to do poorly on the pre-test.

4.  at the state level there is incentive to make the test easier to show ‘improvement’ (e.g., link)

5.  kids are self-centered;  they have little personal incentive to try hard on these tests (when I was in school, I recall other students admitting that they were just going to randomly fill in dots, since they weren’t going to be graded on it)

Alternative proposal:

I’m going to ignore the logistics of this (how to pay for it, how to implement it, etc) for now.  Think of it as something more like a thought experiment.

Suppose we have a lot of fairly small schools, so that parents could choose between about 3 local schools (without having to travel long distances).  Parents would have a choice of which school to send their kids to.  A lottery would be used for schools that got more applicants than they could admit.

Similarly, within a school parents could choose between approximately 3 teachers.  A lottery would be used if a teach got too many applicants.  (note that this is a very different model than we see in public schools now, where parents are discouraged from asking for a specific teacher)

Ideally, these schools would all serve a single community, and therefore eliminates problem #2 above.

Schools would be  judged primarily based on how many people wanted to go to that school.  That is, schools would be judged based on demand for that school.  I think it’s likely that demand would be correlated with quality (a school with a good local reputation would most desirable).  It would be difficult to game this system (eliminating concerns #2 and 3).  If a school was getting very few applicants, that would be a reason to consider hiring a new principal.

Teachers would be judged primarily based on demand as well.  A teacher with a good reputation would get the most applicants.  This would likely be correlated with quality.

In addition, the schools would likely reflect local preferences.  If a community preferred skills and memorization to creativity, they’d send their kids to teachers and schools with those values.  I could easily see different types of schools emerging in a single community, reflecting diverse preferences of parents.

One possible concern is that teachers and schools that were more generous with their grades (e.g., pass everybody) would be more popular.  I’m not sure if that’s a problem, however.  It would reflect the preferences of the community.   In addition, the diploma would eventually be devalued (if these students were not having success post-high school).

Another possible problem is the small sample size.  Suppose all of the local schools were excellent.  Even if one had much lower demand than the others, that might not be reason to make major changes.  So, having both a relative metric (like demand) and an absolute metric might be necessary.  The same issue exists for teachers (i.e., a high demand teacher still might be bad, if all teachers in that grade are bad; a low demand teacher still might be good, if all teachers in that grade are good).

Read Full Post »

Cohen on education reform

Columnists from our nation’s largest newspapers and magazines love to write about education reform.  Even though they apparently know nothing about it, they are very quick to not only propose solutions, but to make it clear that these solutions would work.  They imply (or directly state) that the only barriers to implementation are teacher’s unions or people who have too low of expectations for underprivileged kids.  I’ve written about this before (link), and will do so again. 

A good example is from Richard Cohen’s column in the Washington Post (link).  I’ll let Bob Somerby do the dirty work for me (link): 

COHEN (2/3/09): Do your reading on education and you will find an emerging consensus. Abolish tenure. There are other ways to ensure that teachers are fairly treated without guaranteeing the jobs of the inept. (Cops don’t have tenure, and neither do columnists.) Ensure that the best teachers teach at the most challenging schools and ensure also that they get paid lavishly for doing so.

“Abolish tenure,” Cohen advises, assuring us that columnists lack it. In fact, Cohen has lasted so long at the Post, through so many comical blunders, that his career seems to stand as clear proof that big pundits simply don’t need it. “Ensure that teachers teach at the most challenging schools,” he further lectures Obama—without explaining how we can “ensure” such a thing, especially once we’ve removed their tenure. After all: For decades, “the best teachers” have been leaving urban systems in favor of suburban districts. Question: Might that exodus increase if these “best teachers” are forced to teach in schools they’d rather avoid?Further question: Could Obama possibly deal with a problem like that as part of an emergency measure, one he hopes to complete in two weeks? And by the way: Are “the best teachers” in one school setting necessarily “the best teachers” somewhere else? If Teacher X is great in an upper-end AP program, will he necessarily be “the best” when it comes to teaching low-income kids who are years below grade level? Such questions have never occurred to Cohen—yet he somehow thinks they can be addressed as part of the two-week stimulus effort. Just a guess: Because his columns take fifteen minutes, he may believe that quandries like this can be settled by this time next week.

Read Full Post »

Washington Post article

I wrote a letter to the Washington Post in response to their misleading article on progress in education in the DC area (link to the article).  It doesn’t appear that they are going to publish it.  Perhaps next I’ll try writing to the authors.  Anyway, here is my letter (I thought it was pretty good!):

This letter is in response to the October 2, 2008 column “Needy Students Closing Test Gap Under ‘No Child.’

The story begins with the exciting claim that “students from poor families in the Washington area have made major gains…and are starting to catch up with those from middle-class and affluent backgrounds.” As a statistician, however, I could not help but look carefully at whether the data supported the conclusion. They do not.

There are several problems with the presentation and with the analysis. For sake of brevity, I will focus on the major problem. Performance was measured by the percentage of students who passed the exams. In 2003, the majority of students from middle-upper income families passed the exams (in some counties the percentage was close to 100). That was not the case for students of low-income families. So, there was not much room for upper income students to improve (since the outcome was pass/fail). There was a lot more opportunity for lower income students to improve, as their pass rates in 2003 were much lower. In statistics, this is known as a ceiling effect.

From 2003 to 2007 pass rates improved. Also, the ‘gap’ in pass rates between economic groups have decreased. Does that imply that students from poor families are “starting to catch up?” Absolutely not. There are several alternative explanations that could explain the trends. An obvious one is that the tests have gotten easier. In that case, pass rates would increase, and the gap would decrease (due to the above mentioned ceiling effect). Another possible explanation is that all students have improved at the same rate. In that case, the ‘pass rate gap’ would decrease (due to the ceiling effect), but students from poor families would not be catching up with students from more affluent families.

I appreciate that The Washington Post covered such an important topic. In the future, however, I hope that more care will be taken with the data analysis and with the interpretation of the results.

Read Full Post »

Achievement gap narrowed!

Or so proclaimed Maria Glod and Daniel de Vise of the Washington Post (link):

Since enactment of the No Child Left Behind law, students from poor families in the Washington area have made major gains on reading and math tests and are starting to catch up with those from middle-class and affluent backgrounds, a Washington Post analysis shows.

The achievement gap between economic groups, long a major frustration for educators, has narrowed in the region’s suburban schools since President Bush signed the law in 2002, according to Maryland and Virginia test data.

Isn’t that wonderful?  Students from poor families are catching up to students from more affluent backgrounds.   Or are they?  Let’s take a look at their data.

The astute viewer might notice that they are comparing ‘economically disadvantaged’ students with ‘all students’.  The latter category includes students who are in the former category.  If interest is in comparing ‘economically disadvantaged’ students with middle or upper income students, then your graph should reflect that.  By including low income students in both curves, you’re necessarily reducing the gap between them.  Standard practice is to use mutually exclusive groups when stratifying.

There is a larger problem, however.  Notice that they are not graphing test scores; rather, they are graphing the percentage of students who ‘passed’ the exam.  The majority of upper income students passed the exam in 2003 (it was close to 100% in some counties).  So, there wasn’t much room for upper income students to improve (since the outcome is pass/fail). There was a lot more opportunity for lower income students to improve, as their pass rate in 2003 was much lower.  In statistics, this is known as a ceiling effect.  The Post analysts didn’t seem to notice this problem.

Bob Somerby discussed both of these problems on Saturday here. Today, he posted an email from a very wise statistician (me!) here. 

From the graphs, it’s clear that pass rates are going up.  Well, it is safe to say that there is less of a gap in pass rates between lower and upper income students. 

Does that mean achievement gaps have narrowed?  Are students from poor families in the Washington area really “starting to catch up with those from middle-class and affluent backgrounds”?

Let’s consider some possibilities:

1.  The test has gotten easier.  If the test has gotten easier, then we would likely see graphs like the ones above, even if students don’t know more now than students did in 2003.  The typical upper income student could pass either exam, while the lower income students might have failed the 2003 exam, but passed the 2007 exam.  Did the Post even consider this (likely) possibility?  As I mentioned in my email to Somerby, “if you make the test easy enough (where everyone can pass) there will be no achievement gap at all!” (based on Post logic)

2.  Both upper and lower income students are testing better now than they were in 2003.  Because the graphs are pass/fail, it would look like lower income students have narrowed the gap.  In fact, it’s entirely possible that upper income students have improved even more during that time, widening the gap, but a pass/fail test cannot capture such an effect.

3.  Lower income students really are catching up with upper income students.  It’s possible, but there is not more evidence for this than for options 1 or 2.

Is it even desirable to narrow achievement gaps?

That depends.  Again, quoting myself:  ”There are ways to close achievement gaps that are not necessarily good. For example, schools could decide to use all of their resources on kids who are not meeting minimal standards, and ignore kids who are. Certainly achievement gaps could be narrowed, but at the expense of kids who started out ahead.”

There is no question that some kids enter school way behind other children.  The school’s job is to teach those kids as much as possible.  If they maximize learning from all children, will these gaps narrow?  I have no idea.

Somerby mentioned an interesting theory: 

When I was a teacher, we were taught a naughty theory (I think it was conventional wisdom at the time): Good teaching increases achievement gaps. If you miraculously create a situation where everyone grows at his or her maximum potential, everyone will have advanced by the end of the year. But the smarter kids will be farther ahead of the less gifted kids than they were at the start of the year.

Now, some of the most talented kids will be in the lower SES group, and therefore might make some gains on less talented kids who got a head start (due to higher SES). But, on average, will the optimal teaching situation narrow the gaps?  I don’t think we know the answer to that.

Read Full Post »

Education reform

As often pointed out by Bob Somerby at The Daily Howler, when politicians, or writers for the Times or Post, write about eduction reform, they always have the same message.  Basically, feel good stories about how schools will improve if we hold them to higher standards and raise expectations — you know, no real ideas.  Last year Bob commented on articles in the Times and Post.

Blame the teachers (and the administrators) well! As we’ve explained in the past, it has been the foundational theory of upper-class “school reform” since we entered the Baltimore City Schools way back in 1969. From way-on-the-outside peering in, high-minded elites have always insisted that everything would turn out fine if those teachers would just start doing their jobs. At first, we were told the teachers weren’t doing their jobs because they were such slobbering racists. When these elites finally learned that many black kids were actually being taught by black teachers, we were then told they weren’t doing their jobs because they were so g*d-damned lazy. And basically, that’s the theory that obtains today among these lofty, high-minded reformers. We’ll just stamp our feet at the teachers, they say—we’ll just “insist” they “become more accountable!” Throw in high-minded statements about “standing up for the children” and you have a formula for more decades of fumbling improvement.

The people who write these article typically do not have a clue about what is wrong with schools.  Do they even care about these kids?  If so, why do they always write articles in which the only proposed solution is raising standards and holding teachers/schools accountable?

Bob has written a lot about what is really wrong with schools.  Here’s a good example (link):

…many low-income students are able to read; they just can’t read anywhere near traditional “grade level.” In Monday’s Times, Bob Herbert quoted a new study commissioned by two liberal think tanks. “By the fourth grade, low-income students read about three grade levels behind non-poor students,” the study reports. “Across the nation, only 15 percent of low-income fourth graders achieved proficiency in reading in 2003, compared to 41 percent of non-poor students.” It’s hard to know just what these statistics mean—“proficiency” is a subjective measure—but they do seem to describe a familiar situation, in which deserving, low-income kids achieve far below traditional norms. And for kids like this—kids whose reading skills are far below “grade level”—school can be a bath of frustration, due to the massive failure of their schools to provide them with instructional programs actually geared to their existing skill-levels. They’re handed books they can’t possibly read; as they struggle and fail with these inappropriate texts, a good listener—a listener like Johnson—will often hear them saying things like, “I can’t read—I can’t do this!”

How frustrating can the classroom be for kids who are reading far below traditional grade level? In 1982, we wrote on this topic in the Baltimore Evening Sun, describing results of a study we’d done of the Baltimore school system’s elementary grade curriculum (see THE DAILY HOWLER, 3/17/04). We had spent a year studying the system’s prescribed textbooks, using standard measures of “readability” (formulas used to determine the “grade level” on which a given book is written).

….

kids who are “three grade levels behind” by fourth grade simply can’t read any regular textbooks. Nor can they really read library books which were written for their age group. As a result, their teachers can give them few reading materials—and in the absence of reading experiences, we see a much-lamented (but rarely explained) “failure to thrive.” 

 And what is the popular education reform model?  Why, higher standards, of course.  Bob gives one detailed example about the disaster that is “higher standards” (link).  “at some point in the late 1990s, the Los Angeles school board had gotten the latest buzz-word—“higher standards”—lodged inside its collective head. Result? They declared that students would have to pass Algebra 1 in order to graduate from high school.”  The result?  An increase in enrollment in Algebra by kids who weren’t ready for it.  After failing, they had to drop out of school:

almost surely, most of those kids had no business in a first-year algebra course to begin with. They were there to serve a misapplied dream—the pleasing dream of “higher standards.” And now, the result of this ill-advised program is a bunch of kids out on the street.

What do we mean when we say that these kids didn’t belong in an algebra class? Simple: We mean that they lacked the basic prerequisites—that they almost surely lacked proficiency in basic pre-algebra math. There’s nothing worse than putting kids into a class where they’re destined to fail, but this is happening all over the country as school boards break every pedagogic rule in the book to serve that great god, “higher standards.” They put perfectly decent kids, like Gabriela Ocampo, into classes for which they are unprepared—then stand back and watch them fail. Incredibly, they watched Ocampo fail six times—then watched her become one more drop-out.

In the past, Ocampo wouldn’t have been assigned to this class, for which she was likely unprepared. Traditionally, Ocampo would have been taking a basic high school math course, from which she might have gained actual knowledge. (Who knows? She might have been ready for algebra after that.) Instead, she went into Algebra 1—then went there again and again and again. According to Helfand, this is the standard, cock-eyed practice now in effect at Birmingham High. “Like other schools in the nation’s second-largest district, Birmingham High deals with failing students by shuttling them back into algebra, often with the same teachers,” he writes. And this practice is just as dumb as it seems.

It will be great if we reach a point where education reform is being dicussed by intelligent people who are serious about helping kids.

Read Full Post »