Why negative reviews can be unethical and misleading: 10audio.com

[Synaxis]

Most of the reviewer I've dealt with have scruples, care a lot about their work and that they do it to the best of their ability.

I'm confused.

In post #97 you stated "The problem I have is over the last 35 years I've only seen one that didn't have an ethical breach."

If I read that correctly - and I am open to the possibility I did not - your stating almost EVERY REVEIW you've read had an "ethical breech" of some sort.

That's a pretty strong statement - again assuming it means exactly as it appears.

 

[Atmasphere]

I'm confused.

In post #97 you stated "The problem I have is over the last 35 years I've only seen one that didn't have an ethical breach."

If I read that correctly - and I am open to the possibility I did not - your stating almost EVERY REVEIW you've read had an "ethical breech" of some sort.

That's a pretty strong statement - again assuming it means exactly as it appears.

Apparently I wasn't clear. What I meant by that comment was "The problem I have is over the last 35 years I've only seen one negative review that didn't have an ethical breach." I described that negative review and also its impact.

 

[Synaxis]

Apparently I wasn't clear. What I meant by that comment was "The problem I have is over the last 35 years I've only seen one negative review that didn't have an ethical breach." I described that negative review and also its impact.

OK - thank you for clarifying.

That is why I left it open that It may not have meant as typed.

 

[DaveC]

It seems engineers are far less dogmatic than audiophiles !

Some are... I think engineers are more likely to understand the inherent compromises in different designs, and how those compromises have pros and cons in different use cases. Product design requires, hopefully, a deep understanding of the compromises made by design decisions. They're also more likely to understand bias and thus see things more neutrally, and with an understanding of subjective preference. People don't value the exact same things as everyone else in life, and so there's no agreement on what the perfect speaker is, or the perfect car, or well.. anything.

Speakers are a good example because it's very easy to see the differences in speakers. Audio in general has not converged on the perfect speaker design, as everyone has different preferences that match up with the compromises made by different speaker designs. Preference for a particular speaker design is often a result of what music the listener enjoys the most. So unless you can determine what the best music is, you can't say what speaker is best, and it's pretty obvious that isn't possible.

As a result, anyone who has a dogmatic view of audio is wrong, because it's all based on subjective preference. For example, if you think balanced is always better, it's not. It's better in some use cases, but provides a solution to a problem that doesn't exist in others, in which case all it does is complicate the system for no benefit. It's often a detriment to add complexity without a reason for the complexity. This is the real truth in most home systems, there is no interference present that requires using balanced gear, a simple high-eff speaker like a single driver with one source might be best with a SET and single ended cabling. Balanced anything is a downgrade in that use case. OTOH, if you're setting up a big PA system that has tons of gear, tons of cables, many of which are often very long, and setup in a way and place that often has external interference, you need to use balanced gear to avoid problems. This is why balanced was invented and why you probably don't NEED it for a home 2-ch system, but you might need it for a big home theater with all the power and electronics setup in a large closet.

With SET amps, we all know it's favorability is entirely dependent on the speaker. With the right speaker, it may be the best choice. Other speakers will require more power and a lower output impedance to produce a superior result.

This is also the reason audio reviews are difficult and negative reviews are often not thorough enough to hold credibility. I can say from over 10 years of sending people demo cables and hopefully helping them achieve their goals, that for the reasons previously stated not everyone prefers the same thing. Despite that, there are better and worse products overall.

What I've noticed is the better I can make a cable in objective terms, the more the feedback on that product converges. Objective improvement in cables means the electrical and material properties of the cable improve, the result of this is the cable adds less of it's own character to the sound. When a cable adds its own character to the sound, it means it's not as good in some objective way, and is less neutral. The result of this is the divergence of subjective preference. in other words the reports I get back from people trying my cables differs more, if the character the cable adds to the sound is preferred people like it, if not they don't. The less character the cable adds, the more the feedback converges. I've found this equates to more predictability in recommendations and an increase in subjective preference for that cable. But there are always exceptions as some people want a particular character. In cables, this character is almost always warmth. So in some cases I do offer a warmer version of a cable... I'm starting to get away from that though, and just focus on objective superiority as I believe this will result in the best long term result. Over time preferences almost universally trend towards neutrality, meaning people value warmth less as their system improves, which makes sense as warmth covers imperfections. However, imperfections can come from the recording its self, for example a warm system is great for blasting Led Zeppelin at realistic concert SPLs. It can also come from hearing damage. So the subjective preference for warmth is NOT WRONG, it's a result of being a human with their own preferences.

Ok, I only intended to write a short reply to the quoted statement, so I'll stop here, but I hope this helps a bit with understanding the exceptional complexity of reviewing individual components used in complicated systems when combined with personal preferences. It's challenging and complex, but that's what also makes it interesting and rewarding.

 

[Jack Pot]

Some are... I think engineers are more likely to understand the inherent compromises in different designs, and how those compromises have pros and cons in different use cases. Product design requires, hopefully, a deep understanding of the compromises made by design decisions. They're also more likely to understand bias and thus see things more neutrally, and with an understanding of subjective preference. People don't value the exact same things as everyone else in life, and so there's no agreement on what the perfect speaker is, or the perfect car, or well.. anything.

Speakers are a good example because it's very easy to see the differences in speakers. Audio in general has not converged on the perfect speaker design, as everyone has different preferences that match up with the compromises made by different speaker designs. Preference for a particular speaker design is often a result of what music the listener enjoys the most. So unless you can determine what the best music is, you can't say what speaker is best, and it's pretty obvious that isn't possible.

As a result, anyone who has a dogmatic view of audio is wrong, because it's all based on subjective preference. For example, if you think balanced is always better, it's not. It's better in some use cases, but provides a solution to a problem that doesn't exist in others, in which case all it does is complicate the system for no benefit. It's often a detriment to add complexity without a reason for the complexity. This is the real truth in most home systems, there is no interference present that requires using balanced gear, a simple high-eff speaker like a single driver with one source might be best with a SET and single ended cabling. Balanced anything is a downgrade in that use case. OTOH, if you're setting up a big PA system that has tons of gear, tons of cables, many of which are often very long, and setup in a way and place that often has external interference, you need to use balanced gear to avoid problems. This is why balanced was invented and why you probably don't NEED it for a home 2-ch system, but you might need it for a big home theater with all the power and electronics setup in a large closet.

With SET amps, we all know it's favorability is entirely dependent on the speaker. With the right speaker, it may be the best choice. Other speakers will require more power and a lower output impedance to produce a superior result.

This is also the reason audio reviews are difficult and negative reviews are often not thorough enough to hold credibility. I can say from over 10 years of sending people demo cables and hopefully helping them achieve their goals, that for the reasons previously stated not everyone prefers the same thing. Despite that, there are better and worse products overall.

What I've noticed is the better I can make a cable in objective terms, the more the feedback on that product converges. Objective improvement in cables means the electrical and material properties of the cable improve, the result of this is the cable adds less of it's own character to the sound. When a cable adds its own character to the sound, it means it's not as good in some objective way, and is less neutral. The result of this is the divergence of subjective preference. in other words the reports I get back from people trying my cables differs more, if the character the cable adds to the sound is preferred people like it, if not they don't. The less character the cable adds, the more the feedback converges. I've found this equates to more predictability in recommendations and an increase in subjective preference for that cable. But there are always exceptions as some people want a particular character. In cables, this character is almost always warmth. So in some cases I do offer a warmer version of a cable... I'm starting to get away from that though, and just focus on objective superiority as I believe this will result in the best long term result. Over time preferences almost universally trend towards neutrality, meaning people value warmth less as their system improves, which makes sense as warmth covers imperfections. However, imperfections can come from the recording its self, for example a warm system is great for blasting Led Zeppelin at realistic concert SPLs. It can also come from hearing damage. So the subjective preference for warmth is NOT WRONG, it's a result of being a human with their own preferences.

Ok, I only intended to write a short reply to the quoted statement, so I'll stop here, but I hope this helps a bit with understanding the exceptional complexity of reviewing individual components used in complicated systems when combined with personal preferences. It's challenging and complex, but that's what also makes it interesting and rewarding.

I cannot agree more. I would like to add that, as consumers, we should applaud and celebrate the wonderful diversity of Hifi "perspectives". Such diversity of choice would not exist without the plethora of manufacturers who investigate sound reproduction. And at the risk of being ostracized, I consider the cable loom as one of the more important components of a hifi-system. Invest in a good cable loom and it will survive many other upgrades and consistently bring out the best of purchased equipment. Good power management, good equipment supports, good cable loom, and where necessary, good room treatment constitute the infrastructure of a hifi-system on which ALL else depends. Spend wisely, invest in boring and mostly "invisible" infrastructure and save fortunes on pointless other equipment upgrades.

 

[Tangram]

In my experience, any overtly negative review should be read with caution simply because it’s a significant outlier, in the order of two standard deviations. A positive review with no nitpicks (far more common) should also be read with caution. How many times have we read speaker reviews where there’s a clear mismatch with the amp (because that’s all the reviewer had on hand) yet you’d never know it from the review? Or a reviewer living in a tiny New York apartment ascribes sonic attributes to a speaker that would be challenging to achieve in a much better room? For me, the best reviews give the bad with the good, but they give reasons why another person may get a different outcome. I also care about quality and ease of use, and enjoy truly insightful design commentary if a novel approach is used.

For me, as a customer, any extreme review, negative OR positive, is taken with a grain of salt. It’s actually pretty easy for me to be objective. After all, my heart and soul haven’t been poured into a project over many years, the success or failure of which has real implications for those involved in its design, production, marketing and sales. So I truly empathize when a company feels slighted by a negative review. But rest assured, most end users are astute enough to recognize that rare hit piece when they see it.

 

[mtemur]

Most of the audio component reviews (I’m not saying all, but most) are there to convince readers that it justifies its price.

A review always has a conclusion like “it’s not the best but well deserves its price tag” or “punches way above its league”. This way, the reviewer plays safe and avoids clear statements about whether the component under review really sounds good or bad and tries to convince you how logical and rational it is to spend money on it.

It’s a win-win situation for both the manufacturer and the magazine. More importantly, you as a reader, cannot claim the reviewer is misleading because nobody can prove it doesn’t punch above its league or what the actual league it’s on.

 

[PeterA]

Most of the audio component reviews (I’m not saying all, but most) are there to convince readers that it justifies its price.

A review always has a conclusion like “it’s not the best but well deserves its price tag” or “punches way above its league”. This way, the reviewer plays safe and avoids clear statements about whether the component under review really sounds good or bad and tries to convince you how logical and rational it is to spend money on it.

It’s a win-win situation for both the manufacturer and the magazine. More importantly, you as a reader, cannot claim the reviewer is misleading because nobody can prove it doesn’t punch above its league or what the actual league it’s on.

Very true. I like reviews that tell me something about the actual company and component and then describes what the reviewer actually hears from his system with the component while listening to specific music recordings, and then ideally, how what he hears may differ from another component he has on hand.

 

[tima]

Most of the audio component reviews (I’m not saying all, but most) are there to convince readers that it justifies its price.

A review always has a conclusion like “it’s not the best but well deserves its price tag” or “punches way above its league”. This way, the reviewer plays safe and avoids clear statements about whether the component under review really sounds good or bad and tries to convince you how logical and rational it is to spend money on it.

It’s a win-win situation for both the manufacturer and the magazine. More importantly, you as a reader, cannot claim the reviewer is misleading because nobody can prove it doesn’t punch above its league or what the actual league it’s on.

Interesting -- I think this is the case some of the time though I'm sceptical of 'most'. Your comments could be compelling if you included links to reviews from well known reviewers that are examples. Should be easy to find.

I'll list msrp but almost never talk about price or value because I typically don't consider those factors in gauging a component. Can't recall if I've used language similar to what you cite.

What do people look for or want to read in a review's conclusion?

 

[PeterA]

What do people look for or want to read in a review's conclusion?

A short summary of the main characteristics of the component’s sound, and a very clear expression of whether or not the reviewer likes the sound of the component.

 

[mtemur]

What do people look for or want to read in a review's conclusion?

That’s a fair question. I don’t know what others want, but here’s what I want to read:

• How do the instruments sound? How are the dynamics? Is it lifelike?

Instead of vague descriptions like “blacker backgrounds,” “deep soundstage,” or “quietness” tell me how the piano sounds? Does the kick drum really sound something is kicking under the couch? Soundstage is more related to the room than the equipment itself, yet reviews still mention it. Additionally, nobody knows if a deep soundstage is inherently good or bad. If it’s a multi-miked recording (which almost all recordings are, except 1950s–60s classical), where is that soundstage even coming from? Similarly, how black is “black enough” for a blacker background?

• Regarding comparisons to other equipment, I want to know which one the reviewer prefers to keep listening to. There’s often a mixed comparison, like, “This one is better at X, and the other is better at Y.” Just tell us—what do you prefer? Which one is better overall? When comparing similarly priced components, you almost always end up preferring one over the others. One wins, even if it doesn’t win in every single area.

Bottom line; most reviews are very long, fun to read but hardly say if the component under review is any good. Vague descriptions and comparisons left you confused without a clear answer to that question. In my opinion only one thing is clear with those reviews; if you have the money you should buy this.

BTW I don’t think your reviews are like the ones I’m complaining.

 

[PeterA]

That’s a fair question. I don’t know what others want, but here’s what I want to read:

• How do the instruments sound? How are the dynamics? Is it lifelike?

Instead of vague descriptions like “blacker backgrounds,” “deep soundstage,” or “quietness” tell me how the piano sounds? Does the kick drum really sound something is kicking under the couch? Soundstage is more related to the room than the equipment itself, yet reviews still mention it. Additionally, nobody knows if a deep soundstage is inherently good or bad. If it’s a multi-miked recording (which almost all recordings are, except 1950s–60s classical), where is that soundstage even coming from? Similarly, how black is “black enough” for a blacker background?

• Regarding comparisons to other equipment, I want to know which one the reviewer prefers to keep listening to. There’s often a mixed comparison, like, “This one is better at X, and the other is better at Y.” Just tell us—what do you prefer? Which one is better overall? When comparing similarly priced components, you almost always end up preferring one over the others. One wins, even if it doesn’t win in every single area.

Bottom line; most reviews are very long, fun to read but hardly say if the component under review is any good. Vague descriptions and comparisons left you confused without a clear answer to that question. In my opinion only one thing is clear with those reviews; if you have the money you should buy this.

BTW I don’t think your reviews are like the ones I’m complaining.

Excellent post. I agree with all of your points about reviews. I think Tim was asking specifically about the conclusion at the end of the review. I also agree with you that Tim's reviews are excellent and do in fact describe what he hears when listening to specific recordings. He goes into good detail about the music and sound. And you do not read or hear about him going to lavish dinners at the shows or with the manufacturers. There are some very good reviewers out there. At this point, I rarely read reviews. If someone sends me a link, I read to learn more about the music than I do the component, as my system is basically done.