I cut my teeth on something as simple as the Lee Handloading tool and I still got and use that cheap little press!
Lee Breech Lock Hand Press Kit
This is the newer version of it with breech lock which makes it easier since you dont have to adjust dies as much. Kinda like the Hornady Lock n Load. Its a little slower than a bench mounted single stage but it has the advatage that I can reload just about anywhere I might find myself. My little kit short of a tumber for the cases fits in a small little plastic breif case like box. I can throw it in the truck and reload ammo while I am at work often times. I can reload at the range if I am doing load development too. I do have about 4-5 bench mounted presses though that I use most of the time. But there is no substitute for this little gem and for the survivalist I think you ought to have one in your collection so you can reload on the fly if you need to.
Lee Auto Prime Ergo Prime Hand Priming Tool
This is the hand priming tool I use when using the Lee handloading tool. Yes I have a bench mounted RCBS one that is quicker and easier to use when loading a large volume of ammo but this little guy works fine and it was pretty cheap.
http://www.midwayusa.com/find?userSearchQuery=Lee+improved+powder+dipper
This is what I use to charge the cases with powder when I am using the Lee Hand Loading Tool. Heck its so easy I use it a lot when I am at the bench mounted presses too. Its not as fast a way to measure out powder on my electronic scale and powder despenser but it is pretty easy and pretty quick to use. No I will grant you its not going to be accurate enough to load precision ammo for 1000 yard matches, but it plenty good enough for punching paper and minute of deer out there about as far as you got any business shooting in the first place.
Other than primers powder bullets and case prep tools this is really about all you NEED. Dont get me wrong there are a gang of things you can get to make it faster and more convienent but you can also easily spend a grand plus in the process. Probably not what every beginer wants to invest right off the bat on something they may not like or really get the hang of doing. The Lee Hand Loading tool lets you get in on the ground floor cheap but works plenty good. Just not the fastest set up out there. But you can learn the basics off it and then you can invest in a bench mounted press later if thats your cup of tea. Lots of videos on You Tube on using this and other presses, check them out!