I'm in the process of upgrading from RedHead and denim to nicer stuff, but it definitely puts a hurting on the wallet. I suggest identifying the biggest gaps in your current system (e.g., don't have a good puffy vest) and upgrading those first. Good base layers are more important than many guys think, since you'll almost always be cold if you got sweaty walking in. Breaking the wind is also critical. So, my opinion is that, unless you're just flush with cash, don't bother spending up on expensive gear that doesn't either get/keep you dry or keep the wind out. Pieces that do those things well, though... Can't hardly put a price tag on that, in my view.
If you decide to buy nicer clothes, which definitely enhance comfort, make sure you don't pay full price for them. Sitka is regularly on sale on Black Ovis and other sites; First Lite runs 20+% off sales periodically; and you can usually find pretty large discounts on Nomad stuff (on sale on CamoFire today, also could use code MTNOMAD50 to get 50% off everything on the Nomad website).