If it's the Mustang in your signature, with NO cats, then you might as well ditch it. The only reason it's there is to help with the oxidation process in the catalyst beds, so have at it if they're not installed.
btw, I'm sure you know this, but don't expect any HP gains from removing it.
Oh, almost forgot: are you gonna take out the piping as well?