I've been using PSE3 heavily for the past year, and now have PSE4. It's a nice improvement over PSE3 in several key areas, notably performance and documentation. The other features you can read about elsewhere, so I'll touch on them only briefly.
First, the documentation has been completely rewritten. You know get a good-sized User Manual with many how-to's, instead of the too-brief Quickstart Guide. I wish Adobe had more color in the guide, especially in some places where color is needed to tell blue from red in the Magic Selection Brush, for instance, but at least there is a full-color section at the end.
Second, the performance has been dramatically sped up. The launch time of the Editor, by my timing, has gone from something around 28 seconds on my laptop, to about 8 seconds. My laptop has a 5400rpm drive, so it's fast for a laptop, but slow compared to a desktop. This should really sing on a much faster desktop drive! Also, the scrolling speed in the organizer is really smooth now...the scrollbar handle acts a bit odd, but you get this really smooth continuous scrolling even in a very large catalog. (My catalog is >13,000 photos and growing fast.)
Finally, some of the new features are pretty amazing. The slideshow editor has gotten major work, and now has nice pan-and-zoom Ken Burns effects, plus just a nicer set of templates. "Face Tagging" automatically finds the faces in your photos, and lets you tag by dragging tags onto the faces themselves, instead of onto a batch of images. When you tag, the faces disappear so you can just keep tagging the top few faces and go through a batch of family photos really quite quickly. Why doesn't any other app do this...it makes sense!
The Magic Selection Brush is pretty darn cool, though I think there are cases where magic just doesn't work very well. Maybe more pixie dust next time, Adobe folks? When it does work, it's really cool -- especially since they linked it up to the great Quick Fix panel, and you can select with the wand, then edit in the Quick Fix panel.
Overall, this is a really nice set of improvements, and based on the feedback from PSE3, you can tell Adobe was really listening. It can't have been easy to release a full manual that fast. Maybe next year we'll even get more color!