This topic came up in Marty’s amplifier thread. Member cableman questioned Roy Gregory’s review methods. Mr. Gregory stated that he will change speaker positioning within the room to better optimize the sound of various amplifiers being reviewed.
Interesting topic, Peter. Presumably, readers read reviews for varying purposes or combinations thereof.
Hadn't really thought about it but since a component's design including constructions, materials, etc are all over the map to varying degrees, I suppose it makes sense if a component were able to allow more or less bass to remain audible at the speaker, then repositioning the speaker to compensate for that altered bass output could also make sense. This could also apply to the mids and highs too. Though not sure doing so in the middle of auditioning a component is the best time as things could go south as well as north and the reviewer's potential listening references could be lost or compromised.
My question is: does changing more than one variable at a time change your opinion about the value of the review of the component being compared?
Another interesting topic, Peter. I'd venture yes and no. Anything the reviewer does to the component will not be the exact same what potential buyers might do. Hence, I would think the first half of auditioning time should be with the component right out of the box as the designer designed it. After all, we're really buying the designer's philosophies consolidated into their end product.
At this juncture, the reviewer hopefully knows his own system well enough and he simply replaces a single component with another. This would seem the best hope for a reasonable perhaps best attempt at an objective review. But what if the reviewer was reviewing and/or purchased Center Stage footers positioned under the old component? Should the reviewer also use the same footers under the new component right from the start? My knee-jerk response would be yes, the new component should have the same opportunity to perform as the old component.
But what if when the reviewer installed the footers on the old component when all the planets were in alignment and just happened to obtain the best mechanical mating possible between the old component, the footers, and associated platform? And when he installed the new component under review none of the planets were in alignment and he unknowingly did a horrible job of mating the component with the footers, platform, etc? Or vice versa? IME, something as simple as this could drastically alter a component's performance potential and the reviewer may not even realize the performance difference is a direct result of his knowledge or lack thereof but he still writes the results. What if the mod is a power cable and power cable is actually a POS but the reviewer doesn't know it? What about racking systems? Same thing there too. As many would attest, some products simply aren't worth owning, yet somebody's buying / auditioning those products and reviewers are no exception.
For reasons like that I would think it best if all components and speakers of the reviewer's reference system were configured to play right out of the box without tweaks or mods of any sort. But even then everything has to be placed/installed on some type of platform. The problem with this scenario is the playback system probably wouldn't be worth listening to, much less auditioning products with.
But for accuracy sake, I would still think the first half a reviewer's audition time with a product should be as straight out of the box as possible and then do their initial write up under those conditions. Then perhaps the back half of audition time would be a bit of Crowleyism i.e. "Do what thou wilt shall be the whole of the law." and do the back half of the review on that. Because every component has its actual as well as potential performance and in the end, hopefully not many care about a component's base performance but rather its optimal performance potential. Then again, we're assuming that the reviewer knows what he's doing and that can be quite a gamble itself.
Even excluding all the potential variables a designer used during their component's R&D and testing, I'm guessing all these reviewer-only variables are just the tip of the iceberg and this a primary reason I don't give much credence to most reviews or reviewers as there are just too many unkowns and without really knowing the reviewer, isn't it all just a potential box of chocolates? Even if one thinks they know the reviewer well? I think these are good examples why it's at least as important that we try to read between the lines of a review as well as reading the review on its face.
What would you think of a turntable review if the reviewer changed the tonearm and cartridge to optimize the sound of the turntable in a given system context?
Another interesting point. Nobody should presume any designer knows everything there is know about their designs. Some could even have a tin ear and they too must work within their own budgets and price points. Maybe it's a fabulous TT but if it came with a tonearm and cartridge, maybe the designer came up short in their choices there and the reviewer greatly improved things? Or maybe the reviewer made the tt much less a performer than intended? It can always go either way, right?
This is my response to Ron Resnick in the thread which I deleted because it is off-topic to Marty’s thread.
Ron, the reader is free to believe whatever he wants from a review. Personally, I don’t want to see the reviewer change more than one variable at a time before describing the effect on the sound. The exception I can think of is replacing an entire chain of electronics from one brand to another brand.
I can only say it drives me a bit nuts if I happen to alter 2 variables instead of 1 and realize a performance gain and unable to discern what did what. So I always try to keep changes to a single variable whenever possible.
Personally, I think it requires way too much faith in a reviewer to rely solely on their review. I can only think of a few that I'd put such great faith in. But all good points and good questions.
