The fitbit one is water resistant whereas the fitbit ultra, apparently, is not. Additionally the fitbit one syncs automatically with some phones whereas the fitbit ultra only syncs when you are close to a base station thing you have to plug into our computer. Finally, the fitbit one has a silent alarm (it vibrates to wake you up), it's a little bit smaller than the ultra, and you can't clip it directly to your clothes like you can with the ultra so you have to use the holster that it comes with. It seems like a lot of people have had trouble with the one falling out of the holster, though, so be really careful to wear it somewhere you won't lose it.
I've had both. Essentially they both operate in the same way, but the One is smaller and syncs better for me. And I like the silent alarm. The holder is a bit clumsy, but it doesn't fall off as easily as the ultra did (I lost my first one, it managed to edge its way of my belt loop at some point of the day). I like that I can now sync to my droid phone or ipad too (wirelessly). Love my fitbit, can't live without it, it keeps me on track.