NECN's Peter Howe has more on this progress report:

"My observation in Massachusetts is that we have absolutely failed to take advantage of the charter school agenda in terms of learning from those charter schools that in fact are doing well.''

That was Massachusetts' Elementary and Secondary Education Commissioner, Mitchell Chester, reacting to a major new study of some innovative approaches to school.

It's been 15 years since Massachusetts education officials launched charter schools -- independent of teacher unions, free to innovate on curriculum and schedules. Boston followed two years later with 'pilot schools,' meant to include some of the same benefits of charter schools while maintaining union representation and city oversight.

The Boston Foundation, the city's biggest charitable foundation, has just studied how they're doing. The findings are complex and full of statistical and methodological nuance, but here are two key messages:

- Charter schools are helping Boston kids get a much better education than they would in regular city schools.

- But pilot schools -- especially in middle-school years -- aren't helping much.

Thomas Kane, a Harvard Education School professor who led a team of Harvard, MIT, and Duke University researchers, found a big impact for kids in Boston charter schools: "As they move from 4th grade through middle school, 6th 7th 8th grade, those differences dramatically expand. People who won the lottery to attend the charter schools have outcomes on the 8th grade MCAS (state standardized tests) that are only slightly lower than the average test score of the kids from the Brookline public schools -- not the Boston public schools.''

(Brookline is an affluent suburb right next door to Boston that has superb schools, just the kind of schools you'd expect in a town where according to the 2000 Census 77 percent of adults over age 25 have graduated from college, and nearly 45 percent have advanced or professional degrees.)

But Boston's pilot schools, the report found, aren't having anything like that impact. Pilot schools were conceived of in 1995 by teachers' unions as a kind of middle ground between charter schools and regular city schools. In both, the principal and teachers control curriculum and schedule. but pilot school teachers remain unionized and overseen by the school board and superintendent. At both schools, the special status can be revoked if kids' test scores lag.

"Each year spent in a pilot school was associated with lower achievement in English language arts and math,'' Kane said. "It's in the middle school area where the charters are having the largest positive effect.''

Commissioner Chester says, "This study shows us that there are a number of charter schools that are just getting outstanding results, particularly in grades that are really stumping us, those middle grades, and in mathematics, and we need to learn from that.'' But Chester doesn't consider the Boston Foundation study the last word: "I don't think charter schools have a monopoly on having strong schools. I've seen strong schools in the traditional sector. I've visited a number in Boston and other cities. And, not all charter schools are strong.''

Boston Foundation President Paul S. Grogan agrees: "It is not going to end the debate about charters and pilots and traditional public schools, or anything close to that.''

Kane acknowledges that "our study was focused on the 'whether or not' they made a difference. Answering the 'why' question is a much harder problem than trying to answer the whether or not they made a difference.''

Could it be charters' main benefit is more motivated and involved parents who've pushed to get their kids in? Or that many have a longer school day, more homework, and a lower student-teacher ratio than regular schools? And how much -- really -- do teachers, administrators, and students benefit from the union-free flexibility?

Boston School Superintendent Carol R. Johnson half-seriously raises another factor: "the charter schools do have more girls, and if in fact boys tend to be most disruptive than girls, I won't stereotype that .... '' But Johnson adds that "there may some 'peer effects' that may be worth noting,'' the question of how much charter-school kids benefit just from being with other kids who are motivated enough to get into a charter school.

Johnson and others had plenty of quibbles about which charters and which pilots Kane's group chose to study. But she and other Boston officials and Chrester agree the report shows there's much for all schools to learn from the charters.

Kane adds: "I do hope that there is some process by which we systematically go and try to learn from what the charter schools are doing and to translate those lessons back into the public schools and not have this be just a sort of charter school or pilot school versus traditional public school debate, which is the one that sort of comes naturally to people.''