First of all, the reviews here probably made the difference between me loving and hating this player. To turn it on, you do have to make sure that a) the switch in the back is ON, that b)you've pressed and held the green button until the thing is on and then c)you have to press the green button *again* to start the songs playing. This is actually not a bad design, it just isn't well described in the instructions.
One of the features I love is the "hold" function. I can flick the switch on the back to "hold" and my special needs kiddo then doesn't end up turning it off and crying because she can't figure out how to turn it back on again. Hold prevents any changes to the settings on the front. It is not inaccessible to kids, but it is "non obvious" and not the kind of button she feels a need to play with constantly.
We chucked about 3/4 of the included songs. While it is possible to add an SD card, the card slot is not child-proof, so it won't get a lot of use in a house with kids who still like putting stuff in their mouths. Some of the included songs are great, others were cloying enough to have me wanting to claw my ears off, but the interface to remove and put new songs on is so straightforward and easy (plug in, open folder, move songs in and out at will) that it was straightforward to pull the annoying "Marshmallow" song, among others. We replaced it with Queen, They Might be Giants and Dr. Horrible. :)
The player has seen nearly constant use since she got it a few days ago, and thus far is holding up well to being "loved". The price was outstanding, and we're thrilled with the product.
ETA: Not so thrilled anymore. We've had it 45 days, and it's started turning off at random, even following a fresh set of batteries.