There doesn’t seem to be a proper way to upgrade from XCode 3 to XCode 4, so the best way to do this is to completely uninstall XCode 3.
I found that the best way to do this is by running the following command from a Terminal Window
sudo /Developer/Library/uninstall-developer-folder
