They're actually nothing alike. About the only thing they have in common is that they're both portable electronic devices. Gaming on the ipod touch or ipad requires touching the screen with your fingers to control games, while the PSP (and Sony's newer handheld, the Vita, which is basically the "PSP 2") uses traditional button input for controlling games (think Xbox, Nintendo, PS3). The PSP is more like a modern Gameboy than an ipad/ipod. It's basically a portable Playstation 2. Apple's products have tons of inexpensive downloadable touch-based games that can get boring quickly for those who play video games on a regular basis. Handheld consoles like the PSP, Vita, and Nintendo's Gameboy, DS, and 3DS lines are built for more serious gaming. While the PSP technically can store photos, music, etc., like and ipod/ipad, I've never heard of anyone buying one for those features -- they're basically just fluff to make the PSP sound more versatile. Sounds like your son may be itching to play something above and beyond the simplistic games from the Apple store (he wants to ditch the training wheels).
There's also the fact that you own games you buy for the Vita or PSP (or 3DS) and you don't own those downloads for Apple's devices. Plus the game systems are stable for years. i.e. the PSP was released in 2005 and still has some development for it. Vita was released earlier this year and will probably have a good 4-5 years before being replaced, while Apple's specs get upgraded yearly.