A few comments to supplement Amir's excellent posts:
1. Nyquist has to do with information bandwidth and, in the case of audio, the highest frequency content of a waveform. So, as Amir states, we need more bandwidth and thus a higher sampling rate to reproduce a signal that has more than a single tone at the fundamental frequency. Sawtooth, square, etc. signals require more bandwidth than single-tone sine waves at the same fundamental frequency. (Amir, in your post you should say one must filter signals >= 20 kHz for a 40 ksps system -- a signal exactly at 20 kHz gos to d.c. Yes, I know you know that!)
2. While the arguments rage eternal, most of the testing I recall says harmonics in the signal above your range of hearing do not add to the sound. In that sense, if your hearing falls off over 10 kHz, then a 10 kHz sine and square wave would sound the same. That does not include the impact of nonlinear mixing that may occur in the electronics or in your ears, however.
3. Tony, the digital system cannot accurately capture and reproduce signals above the Nyquist limit, that is 1/2 the sampling frequency. Frequencies over that, if not filtered, will appear down in the audio band based upon their relationship between the signal and clock frequencies. This can make for bad sound and is one reason higher frequencies must be filtered before the ADC sees them. It's a digital thing...
4. Having a higher sampling rate does in theory make it easier to reconstruct signals when non-ideal filters are considered. That is, non brick-wall filters. However, higher rates mean wider noise bandwidth (and thus more noise), can lead to more ringing and other artifacts, make power supply design and filtering more difficult, and make it harder to realize highly-linear (low-distortion) analog circuits in the signal chain. Speaking as a designer, I want no more bandwidth than required.
5. A digital system can produce a d.c. output; a phono cartridge or tape head cannot. This probably matters to nobody, just thought I'd mention it. The phono cartridge and tape head can produce much higher frequencies than 20 kHz, where a CD must filter those frequencies out, of course. Same comment.