The standing order for Magento is "never modify core files"
Always create modules to do any code overrides, always create your own theme folders in app/design/frontend/default/ and skin/frontend/default/ and always put your layout changes in local.xml in your theme folder
The easiest method is to merely download the zip file for your version, expand it out on your hard drive on your local workstation and peruse the file system. Everything contained in that archive will overwrite everything on your system you will be upgrading.
Archived installer files availabe here under the download and release archive tabs.
Here are some of the files/folder trees to keep your changes out of:
Install will overwrite all .htaccess files in any folder tree necessary to preserve Magento's security including the one you will add a lot of customization to in your Magento root folder (always have a backup stored off server)
app/code/core/*
app/design/adminhtml/*
app/design/frontend/base/*
app/design/frontend/default/* <= all Magento supplied themes but your own
app/design/install/*
downloader/*
errors/*
includes/*
js/* <= except any separate subfolders you may add
lib/*
shell/*
skin/admin/html/*
skin/frontend/base/*
skin/frontend/default/* <= all Magento supplied themes but your own
skin/frontend/install/*
var/connect
var/package
