[SteveK]

I agree with Blue00GT. It is most likely your alignment.
If your alignment is toed out, the car will seem to follow the rut or unevenness of the road. I have mine set at 1/16" toe out and it will do that quite a bit on the highway when I drive over uneven or grooved road, even without the brakes. But for track, it works great.

For the street, you may want to toe in slightly. It will feel straighter and give you more nimble feel at low speed.