Mark Guzdial's Amazon Blog

Hi Bill --

Suppose we take as a premise that the following results of surveys over the last 20 years were gathered well enough to form a real generalization:

• only 20% of American adults can read a well written important ideas essay (they used Tom Paine's "Common Sense"), and understand it well enough to discuss it, write about it, criticize it, advocate it, etc. (National Literacy Foundation)
• only 5% of American adults are "literate/fluent" enough in math & science to deal with mainstream ideas, have an extended conversation with a mathematician or scientist, be operational enough to put a mathematical map on a set of ideas and do something with them, etc.

If we weed a few more artifacts out of this survey (and the surveyors did some of this already -- such as not counting those who were not educated in American schools, etc.), then we can pretty reasonably conclude that the schemes of education employed in the US have failed (miserably) to meet the goals of education in America.

The reason I say "suppose ... premise" above is that we have to be very careful about "scientific studies" outside of "science", because sometimes only the trappings and not the substance of "science" remains. In the above case, it looks as though (especially) the NLF did a comprehensive job of sampling and concluding.

However, this is not often the case in most studies of educational methods and results. And, it is very difficult to separate out and test any method from the testing that is also going on of evaluating how well randomly chosen teachers in America can teach anything in any style. I would posit that trying to do this in single trials is essentially intractable.

I've mentioned before that just validating a piece of our curriculum requires three years of doing it with a teacher in a classroom before enough of the artifacts and distracters can be nudged out to get even a qualitative judgment. (In our case, this has very often been a "no, this is not a good way to approach this that will get more than 90% of the children above a real fluency threshold" -- i.e. failure.) No-one wants to pay for these extra years, so we use our discretionary research budget to support the extra costs and time.

A much more important kind of investigation in education is a "transfer study", which is all about whether enough of both practical and abstract understanding is retained and operationalized enough to be applied by the learner in later contexts (both related to the original learning and in areas where there could be very fruitful analogies). For example, in the 70s Adele Goldberg and I designed an extensive transfer study to see if "powerful ideas in programming/active-math" could be foundations for more powerful learning and thinking in other fields (we chose major parts of Biology). Because of the kind of setups and testing needed, we thought that at least 7 years should be devoted to this. For example, there were several overlapping three year implementations in the programming/active-math ideas, and the kind of testing we had already been doing, and then there would have to be another series of these in the later experimental and control classes when the children started learning about Biology.

Needless to say NSF turned this down flat, and turned down several subsequent requests we made.

However, even if they had funded the study, we realized that it would be adding more of the largest problem of doing anything in a school with math or science, which is working with teachers who don't remotely understand their subjects -- and (even in the case of reading and writing) don't do the activities with the children (when was the last time you saw a 5th grade teacher assign a composition to the students and then let the students pick a topic for them and write an essay along with the students?). (Actually, given the excellence of your blog, you might be an exception!)

This, along with many other reasons, is why I don't worry about the "C" word or the "D" word, or any other simple scheme. As Marvin Minsky once pointed out, every educational method works for some students. This is because another deeply important factor is that children in a single classroom exhibit a wide variation in motivations, knowledge, skills, maturity and "wiring". Different children need different approaches. A classroom is a tough place to learn anything (as an orchestra is a tough place to learn how to play an instrument). The US factory approach to education was hoping for economies of scales via method, but it forgot that it wasn't about just turning out Model-T's, but every kind and variation of vehicle using every kind and variation of materials and design.

Long (very long) ago I was a professional musician (jazz guitar) and also taught guitar for a few years. The basics for musical learning are rather similar to sports learning, and they involve rather different approaches and mixes of processes than in formal schools. (Of course, they might be so different from learning math that no analogies will hold -- but let's pretend that they aren't so different.)

The goal in music-sports is fluent playing. It is not known how to do this without having the learners undergo a lot of "doing of playing". However, there is not a lot of discovery to be done early on that is going to help and not hinder later on (i.e. most ideas are mediocre down to bad -- this is why good ideas are so rare and precious). But, as Tim Gallwey the great tennis teacher says, "The problem with most theories of learning is that the parts of your body that need to learn, don't understand English!" Saying it a different way, the parts of our mind that do understand natural language, aren't often able to do other subjects well. We can see this is also true for math and science -- otherwise we could just write the best expositional essay on each subject (called "great books") and just get the learners to read them! And, imagine how easy it would be to teach teen-agers to drive a car! Obviously, other elements are vital.

If we combine a few ideas -- e.g. discovery is really difficult, it's hard to learn via language, we have limited capacity for dealing with ideas at one time (7+-2 according to George Miller), etc. -- then we can see that Jerry Bruner's notion of "scaffolding" starts to come front and center as a way to devise strategies for learning sequences. For example, a teacher can set things up so that only a few degrees of freedom remain, and now there is a much higher chance of actual discovery, or homing in on what is best to concentrate on. This is done all the time in music-sports.

For example, Ted Williams introduced the batting tee into professional baseball and was pooh-poohed for "silly, unmanly, etc.". But he was the greatest hitter of his day (and one of the greatest of all time) so gradually others began to surreptitiously practice. His idea was that it was almost impossible even getting the muscular feeling and memory for a level swing if you are going against a moving target of "round thing against round thing". Now the batting tee is found in every training facility for all levels of baseball and there is even a league for very young players.

Scaffolding has to be carefully vetted. For example, short skis really seem to work for learning beginning skiing, but putting frets on a violin doesn't (even though they seem to help in the beginning - then they hurt badly). However, "multiperson African Drumming" really does help all aspects of music learning, including classical music.

"Showing" often helps. If you can't feel the phrasing of a musical sequence, sometimes it's just best for the teacher to play various phrasings to be judged. Or to get you to watch them serve (the flip side of this is that the top tennis pros have rather different strokes and serves -- i.e. personal wirings and idiosyncrasies have to be tolerated -- it is very difficult to learn exactly what someone else does -- but one can learn "just as well though a little differently").

This hurts badly in school when the teachers don't know enough math or science to be flexible about perspectives, etc. We would be surprised if our music or tennis teacher weren't fluent and refused to play with us (for one thing, that's the best way for them to assess where their students are) -- we would doubtless drop a "non-doing" teacher. But the opposite is egregiously true for most school teachers, most are not and have never been practitioners. However, we only see a few parents take their kids out of public school for such reasons.

We could well imagine that one form of instruction might score better than another if teachers are not up to snuff (however, as mentioned above, the "better" is not nearly good enough to get the eventual American adults above any reasonable threshold). If we are going for "evidence" and "scientific evaluation", then we have to include getting to real thresholds, not just relative differences. Here, all methods currently fail -- and probably will until better conceptions and thresholds are created for teachers.

Gallwey again: "You still have to hit thousands of balls to learn tennis, the difference is what you are thinking about and how you are focusing while doing".

This is as good a key to progress as any.

An interesting paper by one of your countrymen that Mark Guzdial pointed me to (After the Gold Rush: Toward Sustainable Scholarship in Computing, by Raymond Lister, University of Technology, Sydney, Australia --http://crpit.com/confpapers/CRPITV78Lister.pdf ) shows some of the difficulties of dealing with this very complex area. I don't know quite how to do justice to a counter argument in a very short space here, but I think there are real parallels with what happens with learning programming (he gives his POV as a college teacher of programming) to what happens with learning music, sports, and even driving a car, if the learners don't do enough of the actual processes. For example, he makes the (to me) astounding statement that:

They couldn't write code at the end of a semester? Was the course about anything else? What kind of grades could he be giving? "Air guitar" grades for "air programming"?

Much other of interest will be found herein.

There aren't enough details in the paper to comment on his teaching style or to guess why his students couldn't program at the end of a semester. (This is not the only such story that has been written up over the years.) In some of the latter cases, I knew some of the instructors and they were not dunderheads by any means. So we could certainly give Mr. Lister the benefit of the doubt, and wonder instead about the processes in his class and in universities in general.

Now, if we do the (so far unwarranted) act of substituting music, sports or even driving a car, we might guess that the main reason the students were in this unhappy state at the end of a semester is because of the pace, depth and amount (if not also the nature) of the doing experience.

Another unwarranted comparison is to the way programmers were created in the military services in the 50s and 60s. Virtually all participants were enlisted personnel without college educations and some without high school. Programming was needed, but was not glamorous enough to be within the ken of the college educated officers.

Prospects (in this case, the Air Force) were given a short aptitude test (about 45 minutes) made up by IBM that essentially assessed interests and latent abilities in patterns of various kinds. Only people who got through this went to the next state -- which was a one week wall-to-wall (40 hours of class plus lots of assignments) of instruction in how to program a computer. This was also conducted by IBM, and in my memory was just about perfect in the balance of description, advice, examples, and many doings with one's own code. (I had similar favorable impressions with the rest of the training I got while in the military -- the only thing left out was "education", meaning that "theory" was scant -- every other aspect could not have been better thought through and presented.)

One hectic week later, one knew the machine code and assembler and could write many programs for the real computer that was back on base, and that was what we did to other's goals for several months. This was intensive and literally "on the job training". One thing that people find unusual today, was that not only was there no interactive programming (punched cards were submitted for a batch run), but one was allowed a maximum of five minutes actual contact each day, not with the machine, but via an operator who ran the machine, could punch in addresses, etc. One had one's listing draped over the card reader and was kept well away from the console. Basically, the only way you could get a program to run was to have it be "almost perfect" before testing. This was accomplished via another developed skill called "desk checking" (Don Knuth attributes his facility with programming to this quaint process as well.)

Then there was another intensive week of wall-to-wall "Advanced Programming" in which one learned a little more architecture and how to use the extensive macro facility in the assembler, etc. I will only compare the first intensive week and month or so which resulted in real programming skills to Lister's very different experience in university.

The point here is that the armed services scheme had almost no failures, everyone who went through it was successful. The instructors weren't any better than the college professors, but the process really was. And the goals were very different. There wasn't any class to pass, no multiple choice tests to take, no grading on the curve, only a few hours of "lecture" (and just when needed), and (no small matter) there was nothing to do but to learn programming that "semester". The basic idea here in 1961 (I think) was that if you can think a little, then a "summer music camp" approach is the best way to really get going on something. If you can't think a little (play a musical instrument a little) then you should get across this threshold and then go to summer camp.


Again, this successful scheme doesn't necessarily generalize to every subject. But it's strong enough to be worth considering in areas where "doing skills" are an important part of the subject. (One problem with "math" in the US is that it isn't actually "math" but only simple calculation skills. This isn't enough to help with actual math thinking (which is a special skill all its own that can indeed be taught, but isn't.)

An important aspect of this approach is that it nicely avoids having to categorize methods: it is really about a somewhat vague but readily understandable approach in which the only real goal is to help the learner achieve fluency in "something that is done". We can safely ignore the C and D words in favor of the E and T words (or the M and S words).

Best wishes,

Alan