I lived in Chattanooga, TN for a while; near the tn/alabama/georgia border and we heard them all the time when they were on training runs; there's an airfield in north-west GA or something. more than once a year but it was still a jolting enough experience that it freaked you out. Once we had one early morning on a weekend and my ex was walking down his stairs at the time. He fell and a picture on the wall fell and broke. He and his family thought someone had crashed into the house because of how much shaking went on. That particular one caused a lot of trouble because they were flying too low over a metropolitan area. At least in the states there's regulations on how low they can be in populated areas because it can be a big disruption.
edit: I was just on the phone with my mom and the deal down there was that they were developing a new plane down there. They've finished it now and no longer have to deal with it. She did say that another sound that can scare the crap out of you is when they hit an air pocket in a rock quarry. The result is nearly the same as breaking the sound barrier.