Ok I see where you are coming from and I think that we can agree on this. To make my point clearer let me word it another way: every component has a transfer function; sure their transfer function may change as components age, are burned in, or wear out (tubes for instance) but if you look at components at any moment in time, they have a specific transfer function, or in audiophile lingo a “sonic signature”. I believe that the transfer function of each component is very telling in what impact it has on the spectral content/distribution or tonal balance. Like any other transfer function, these can be analyzed and characterized with a step function stimuli.This is how room acoustics are measured and analyzed.
Here again I agree with you that the auditory system is a very powerful, complex, and sophisticated system. Is everything known by science? No, but enough is to address the phenomena that we regularly refer to in this hobby.
I know we can use both method of Frequency (fourier) and time domain analyses for linear systems, so the Time domain step response or impulse response gives us some information about an audio system transfer function but it is not enough for accurate modeling of complex non-linear systems .