This excellent series by the EEVblog have 3 parts. The first is about tools, the second about through hole soldering and the third about surface mounts.
Here's what I have for general through hole work. I have never had a problem with soldering using the following tools. This will bring your costs down significantly.
I've got a 25W Antex iron. Go for the silicon cable. The other one will work, too, but the silicon is more convenient. A 18W is more than enough for PCBs only. The power rating is not comparable with the cheaper irons. Those might have a higher rating, but they can't actually deliver the heat onto the board. I sometimes do other DIY, so I needed a bit more power. You can spare the temperature control for general PCB work in my experience. Don't go cheaper, though. The cheaper irons last for a couple of hours, this one has been reliable since I have it. I didn't find the need going more expensive, yet.
The solder tips are important for the good thermal contact. It makes sense to buy something you can buy standard tips for - this again rules out the very cheap irons. I only used the standard tip for all my through hole projects, it works well.
You might want to consider a holder and tip cleaner.
The solder itself is the most important bit, I use this thin, multi cored, leaded one 0.5mm 250g from Cynel. This was best value among the high quality solders on market when I searched.
The wick is a must when you consider desoldering parts - this will come by time.
For the trimming of the through hole legs I use an old nail clipper; it does the job perfectly. It's also good for trimming your jumper wires. I provides the perfect control on the tension to apply.
Board cleaners are not necessary if you use quality solder/flux. Your boards will look better, though if you clean them.
Wear eye protection and do the soldering by an open window and that's it. In my experience you don't have to spend tonnes of money just to get started with soldering.