I've added my websites, stores, and store views through the admin panel, set my prefered default and then deleted the website that Magento had by default.
I've now read this;
http://magento.stackexchange.com/questions/5894/deleting-website-in-magento-gives-page-not-found-err...
Given that I now no longer have a store with an ID of 1; this would suggest that I've now killed my Magento instance ensuring that I'll never be able to run upgrades. Is this the case?