I do not know what version of Android you use, but for 2.2 and 3.0 go into settings from the phone/tablet homescreen. Select "applications" -> "manage applications" -> select the app you want to shut down -> hit "force stop". This is how I shut down any app at any time, it will warn you that it could cause problems, but I have never really run into any with a force close. You will also see options like "uninstall", that is how you can remove it entirely. I had a friend recently comment he could not remove an amazon app he did not like, this menu was news to him. He mentioned that the android market has the uninstall button on the page.
Shawn, I had the same concern - but got referred to a developer website that explained how Android works. Apparently, the "running apps" listed aren't really running (unless you are actually using them), i.e. they aren't using any processor power, but are just sitting in memory, waiting to be used quickly if called on. The Android system actively manages the memory space also, so if another program needs more memory, it will dump apps that aren't actively being used. Check out an app called Watchdog which will explain all this and which will watch the "running" apps and kill them if they are causing problems.
What if the Uniinstall link doesn't appear. In my case, there's just a Disable link where Uninstall usually appears. This on a recently purchased Kindle Fire HD 7" Second Generation. I actually downloaded or punched Office Suite, but there it is -- version 7.3.1513 -- in my Manage All Applications list.