I'd be more concerned about the coloration of the SET than the horns.
I'm new here, but I think you have pretty much nailed it...at least on the face of things. I use SET amps and horns.
Most SET systems I have heard have sounded compromised in at least one or more ways. Being driven too hard, unsuitable loudspeaker impedance loads, feedback from the bass driver affecting the other channels to name a few. The Single Ended Triode is near the simplest and oldest electrical amplifying circuit we humans have devised and its use comes with some serious caveats which most users tend to either be unaware of or ignore.
For any given SET amp the following things must be overcome:
(a) Output impedance is generally, but not always high. This means that the frequency response follows the speaker impedance plot. Unless the speaker is designed to be agnostic (flat impedance curve) or specifically designed to be used with the output impedance in question, the frequency response is going to be problematic.
(b) Distortion increases with power output. Run them light and they are fantastic. Run them hard and you will hear it.
(c) Electromotive force from bass and larger drivers can feed back via the output transformer and alter the frequency response in other loudspeaker channels, eg. tweeters or mids.
(d) There is not much power on tap to start with so your speaker will need to be big.
(e) Output transformers for bass need huge cores, thick wire and as much inductance as you can jam in there. Output transformers for mids and highs need much smaller cores, thinner wire, less inductance for optimum sound. The two never shall meet in a single output transformer which is why the great OPTs are only really good, and the good ones and not so good...if you want great sound.
All of these caveats can be beaten.
Anyway, one advantage of SETs is that as power is decreased, distortion falls off to unmeasurable. This is all about that 'first watt' which has to be clean. SETs are pretty good about this. By comparison, large push-pull amps are not, since below a certain power level (usually around 5% or so), distortion starts to go back up. This is why SET users often talk about that 'magical inner detail'.
Yes. If you have 100dB/w/m sensitivity loudspeakers the smallest SET I would recommend is 15w. Absolute minimum so that the amp retains plenty of headroom. My personal opinion is that any loudspeaker less than 100dB/w/m sensitivity is unsuitable for SET amps. If the loudspeaker does not have a benign impedance load (i.e. is a normal commercial loudspeaker, not one design to be agnostic toward higher output impedance amplifiers) then the SET amp needs to have low output impedance. This can be done, but it limits the power tubes that can be used. Low plate resistance tubes must be found and used here. My zero feedback SET amp for my bass channel has 0.19ohm output impedance, which is pretty decent.
If you want great sound from flea-powered SET amps then you are going to need much more sensitive speakers than 100dB/w/m...think 110dB/w/m. Horns. Big horns.
So IMO/IME there is no argument for SETs, other than someone simply not having heard something that is both sonically and measurably better.
In most situations I agree with you. However, if you have a loudspeaker that is not adversely impacted by the shortfalls of the SET topology, then I will guarantee that there is nothing sonically better. I'll explain below.
Some argue that SETs are very dynamic and it is often true that they sound that way. But in reality, what is happening is the amp is being driven hard enough that the higher ordered harmonics are showing up as the distortion of the amp is increased (SETs are often 10% THD at full power), and where the power is needed is often transients. Since the ear uses these harmonics to sense sound pressure, and since they are showing up on transients, presto- you have an amp that sounds 'dynamic' but its really distortion masquerading as such. Simply by you're reading this, I may have ruined it for you because of how our brains process music. To avoid this phenom, the speaker should be efficient enough that this does not occur: hence horns.
A mark of any good system is the quality where it does not sound loud even when it is. That takes clean power which SETs cannot provide except at low power. This is often why SET owners will tell you that '90dB is plenty loud enough for me'. If the higher ordered harmonics were not present, it would be natural to turn the volume up higher.
SET's are ridiculously dynamic when paired with the correct speaker, but it is not the type of flawed dynamics that you correctly describe when overdriven, they are no-compromise dynamics.
A mark of any good system is the quality where it does not sound loud even when it is. That takes clean power which SETs cannot provide except at low power. This is often why SET owners will tell you that '90dB is plenty loud enough for me'. If the higher ordered harmonics were not present, it would be natural to turn the volume up higher.
This can be done with SET, but it takes a big effort. My listening is usually at concert levels, 105dB peaks, and it does not sound loud or even with a hint of discomfort in the ears. The opposite actually happens to what you desrribe...the louder my system gets the softer it sounds. I attribute this to in-room response flat to 18dB and loads of headroom in the amplifiers. The Fletcher Munson curves of equal loudness show that as SPL increases we able able to hear a higher proportion of the bass frequencies. If the speaker output is clean and capable of producing these big SPLs at low distortion then the sound should get softer as volume increases, but most playback systems I have heard are unable to achieve this.
So, how do you side-step the topological flaws of SET amps?
(a) Match amplifier output impedance to the speaker load.
(b) Run the amp at a low percentage of its potential power output. This means very high sensitivity loadspeakers.
(c) Multi-amping. There are a number of ways this is possible: SET for mids/highs and SS for bass drivers; SET for mids/highs and another SET suitable for bass drivers.
(d) Great big speakers that need a fraction of watt to be really loud. Minimum 100dB/w/m for high powered SETS, 110dB/w/m or better for low powered SETs.
(e) Multi-amping so you can use the best output transformer (and amplifier circuit) for the job. Big core, high inductance OPT for bass duties. Smaller core, lower inductance OPT for mids/highs.
By my judgement, a "good" SET system uses at least two amplifiers (bass can perhaps be SS) to separate the influence of the bass drivers from the mids/highs and to optimise the OPT's, has a high sensitivity loudspeaker preferably with a benign/flat impedance curve.
A great SET system goes further. One SET amp for each driver with transducers directly driven (no passive crossover between amp and driver). Each amplifier circuit is optimised for each driver in terms of output power and output transformer (among other things). Higher frequencies need much less power than lower frequencies so requirements change with frequency.
Do smaller output SET amps sound better? Usually, within limits and the other compromises of the playback system. My system is dual mono, six channels per side, mixed two stage and single stage SET amps with output varying from 17w to perhaps 1w, output impedance from 0.19r for the bass channels to much higher for the highs, direct coupled, passive line level crossovers at the input to the amplifiers, horns circa 110dB/w/m with bass channel 100dB/w/m. The speakers are 2.4m tall.
To make a "great" SET based system that side-steps the amplifier shortfalls is a lot of work...but it is worth it.