For better or for worse, Vita will be pitted against the Nintendo 3DS. But the comparison doesn't make a whole lot of sense, and even if you were to pit them against each other, Vita is still destined to sell better. That's because in addition to games (which Nintendo 3DS is still sorely lacking), it's also a much more powerful machine capable of doing a whole lot more than 3DS.
Even Nintendo, with its buttons and triggers and touch pad admitted that leaving off a second analog stick from its handheld was a huge mistake. Now, you have to buy it separately. Vita comes with one right off the bat because it's probably the most glaringly obvious thing a handheld needs to compete these days.
The Nintendo 3DS is certainly not going to take the Vita out with N64 ports and tired, worn series. The iPhone and Android phones are in everyone's pockets, but who's going to play a competent first-person shooter or action-heavy games like Arkham City on them any time soon? The Vita seems to be the best of both worlds -a technologically-advanced and connected handheld that's tailor-made for gamers.
Should Sony feel threatened by its competition? Sure. But Nintendo is the least of Sony's concerns these days.