I think part of the challenge in audio is that anything you change will change the sound, and usually, if you're making an upgrade, you'll upgrade the sound (but not always).
If your SA20 is loud enough for your B&W CM5, it should be loud enough for your SF. If you never thought of buying the PA240 before, why would you suddenly want to buy the PA240 now?
Sure if you like what you're hearing, buying the PA240 would probably upgrade your sound to a more refined version of what you're getting now and you'll probably be very happy with it. The question ultimately to me is options and best bang for the buck. What if you sell the SA20 and take the money to buy Peachtree Carina or Cambridge Audio Eco 150 or Musical Fidelity M6si?
It's really hard to make recommendations sometimes because we don't know your personal sonic preferences. Ideally, you want to be able to listen to these amps in your home. If not, you should try to hear them at a dealer's demo just to get a feel of how it'll work with your system. If none of these are options, and you already like the Arcam SA20 sound with the SF, then I guess upgrading to the PA240 is your safest bet.