The general recommendation is to not do this. However, if you just want to try it out, here are some sources for general information.
http://www.startbitcoin.com has some nice introductory information.
You will need to download a wallet from http://www.bitcoin.org. This download will potentially take a long time.
ASIC miners (Application Specific Integrated Circuit) have taken the place of GPU miners. ASIC miners are generally expensive and can become dated very quickly.
http://www.butterflylabs.com has some ASIC miners for sale at prices starting at $50 (not recommended).