Wow! Seems like everybody mentioned Tai Chi, which I also agree to. Other than that, low impact exercises, such as walking and swimming, which you already mentioned, are good for muscle pains, too.
The breathing and stretching exercises done in Yoga are also said to be very effective.
Pilates is an excellent exercise regimen specially if you have lower back pain.
Stretching is probably the simplest among these exercises, but they're actually very much effective. Stretching improves your muscle and joint circulation, so you know it's good for you.