Chris,
You don't need to defrag, unless somehow you have a 1TB primary drive that somehow fragments to Hell and back when you format (a little excessive, but you get the point). Even if you did a "quick format," you'd want to defrag before adding to many files to the system; but, it's not critical.
I don't recall Chris actually saying which version of Vista he had. Chris, could you tell us if you actually did install Vista Business, or if it's a different version?
Bottom Line: Do you need Vista? If you have such a formatting addiction, why not just format and go back to XP? That way, as well, you'd know if this system slow-down is a result of Vista or if it's truly your hardware. It's only another 20-30 minutes of your time, and would benefit you greatly at this point, it sounds like.