Criticism of the Hifi media: What would you do different?

[morricab]

How do you handle the issue of opinion quality? Who decides which audiophile friends have enough experience and critical listening skills?

The reviewer decides who is invited of course and so what if there are weak members of the panel? The breadth of opinion is what you want. You can find clear points where the whole panel agrees and some where no one is in agreement. You can assign likely characteristics to the ones with the strongest statistical agreement and the more speculative characteristics to the ones where people agreed less. A table with traits and agreement level could be quite useful for someone to see what most people who heard the piece of gear in that system thought it sounded like.

All I can say is try it out and see what works. Happy to discuss with you privately about a framework for conducting the tests and tabulating the results.

 

[morricab]

No one. But it’s a different thing to have friends do reviews and someone like Alan Sircom who has done hundreds of reviews do a review.

You are not having your friends do the review. You are gathering a statistical sample of listening observations from somewhat experienced listeners.

 

[Lee]

The reviewer decides who is invited of course and so what if there are weak members of the panel? The breadth of opinion is what you want. You can find clear points where the whole panel agrees and some where no one is in agreement. You can assign likely characteristics to the ones with the strongest statistical agreement and the more speculative characteristics to the ones where people agreed less. A table with traits and agreement level could be quite useful for someone to see what most people who heard the piece of gear in that system thought it sounded like.

All I can say is try it out and see what works. Happy to discuss with you privately about a framework for conducting the tests and tabulating the results.

Just thinking through how you operationalize it…would there be one central playback system that is used?

 

[Tangram]

I'd love to see a little sketch accompanying each review that shows the size and layout of the room showing speaker position and listening position. This, in addition to an explanation of the equipment actually used for the review, not just a list of equipment owned/loaned/in for review. I'd also like some comfort that if it's an analog review, that the equipment has been properly set up and know HOW it was set up, specifically cartridges. I'm not convinced that some of the sound reviewers describe, especially when cartridges are being compared, is because they didn't nail the alignment. It takes a lot of effort to do this and I question whether some reviewers have the time/skill/tools to do it properly. Finally, reviewers should be willing to subject themselves to a hearing test annually and publish the results. Some of these guys are getting pretty long in the tooth!

 

[Elliot G.]

Harry had a listening panel and all of his major reviewers came to his home a few times a year to listen. That is how one learns and how one sharpens his skills. The other reason of is to have a REFERENCE that all can aspire to. If you have everyone on their own and no vision or goal then you have IMO nothing. One persons opinion in a void isnt worth anything to me. Every Industry has plans and a target.
I learned a lot from listening with Harry and with others who were more experineced and knowledgeable than me. Arnie Nudel, Jon Dahlquist, Dick Sequerra, Mitch Cotter, Jonas Miller, Bill Johnson etc. All of these guys listened with HP as well.
A magazine makes a profit. IMO they should have a corporate reference room that all thier reviewers can have access too and should be required to listen there at least once every year. If RH has a reference, and JV has a reference, and JKH has a reference etc and they are all different then what is the actual reference when they are all different?
I listen with my friends often. We discuss. we question and we critic.

 

[MarkusBarkus]

...perhaps even a master-reference list of tracks/versions/mixes that forms a core base is a start. Maybe held at/distributed from the mag's audio-vault. Easier with digital, I recognize.

Even if a mag has a 20-30 song list, and reviewers are not obligated to use all of them, perhaps they need to select 3-5, and whatever else they like from their ref collection.

I recognize there will be many dissimilar aspects, including the component itself (!) but any standardization would help.

Would it lead to false conclusions? I suppose it could, but it looks like that is happening all-day, every-day anyway. Carry on.

 

[Kal Rubinson]

A few of the specified aspects, such as DIY or hobbies fun are better served by alternative interactive media - only a minimal number or readers would be interested in their printed form.

And some of them are fortunately present in some current magazines.

Sure but it was a rhetorical question.

 

[Rexp]

...perhaps even a master-reference list of tracks/versions/mixes that forms a core base is a start. Maybe held at/distributed from the mag's audio-vault. Easier with digital, I recognize.

Even if a mag has a 20-30 song list, and reviewers are not obligated to use all of them, perhaps they need to select 3-5, and whatever else they like from their ref collection.

I recognize there will be many dissimilar aspects, including the component itself (!) but any standardization would help.

Would it lead to false conclusions? I suppose it could, but it looks like that is happening all-day, every-day anyway. Carry on.

Yes, particularly if each track was chosen to demonstrate a specific sonic attribute like, imaging, dynamics, flow etc

 

[morricab]

Just thinking through how you operationalize it…would there be one central playback system that is used?

Yes, the reviewers system.

A magazine could have a reference system but as reviewers are not coming into the office everyday, I don't think it really makes sense for a magazine to have a single reference system that basically no one will go in to use.

 

[bonzo75]

That would be great. I grew up with such mags.
I wonder why they have all disappeared?

The fact that someone succeeded in being commercially successful does not mean they are right for the hobbyists. Otherwise Coke and Pepsi would be health drinks to lower HBA1c and LDL

 

[morricab]

The fact that someone succeeded in being commercially successful does not mean they are right for the hobbyists. Otherwise Coke and Pepsi would be health drinks to lower HBA1c and LDL

Coke was probably originally sold as a health tonic! Keeps one well preserved on the inside...

 

[bonzo75]

Coke was probably originally sold as a health tonic! Keeps one well preserved on the inside...

I can see who here would have paid a million for it, if their understanding of health was similar to their understanding of sound and music.

And would have argued on what degree to chill it to for maximum benefit

 

[ssfas]

I remember having a conversation decades ago with a friend about TAS reviewers and my friend was critical of Anthony Cordesman not making more regular reviews, as he went through a period of time where he was less prolific. I was like, "You do know he has a real job - like being a leading expert in national security, advising the White House, and using his doctoral degree to be a professor at Georgetown University?"

He was one of my favorite TAS reviewers. Been gone from this earth for a year now

If I recommended a doctor on the basis that he was a leading expert in national security, would you trust his opinion? How does expertise in national security contribute towards an opinion on audio?

I first heard of Mr Cordesman when PS Audio referenced a review he'd done of their new speakers, maybe 2 years ago. I looked him up. He was 80+ years old. At that age his hearing would have been seriously impaired. I didn't read the review. To be honest it was not just his age that put me off, I just struggle to summon the willpower to read any audio reviews. It suggested to me that his name was being used to promote the product.

 

[ssfas]

The only reviewer I ever paid much attention to was Jimmy Hughes.

For years he was head of customer care at Leica UK. So I'd take a camera in for a sensor clean and I'd get a lecture about some audio topic. There was nothing he didn't know about audio or cameras. I sometimes buy cameras from a shop next to the Savoy and knew the head salesman well. He was a friend of Jimmy's, who came in frequently and had tried just about every camera ever made.

Jimmy knew even more about recorded classical music, he was simply astonishing. He has what is considered the largest classical collection in the UK, vinyl and CD.

On one occasion I had a Leica tele-elmarit in for recalibration. It was my son's barmitzvah that evening at the Mayfair Hotel, not far from Jimmy's shop in Bruton Place. We were at the hotel and my wife was having her hair done, I'd got a message the lens was ready for collection, so I popped out to have a coffee and collect the lens. He gave me a lecture about something, I think it was mono cartridges, and almost missed my son's barmitzvah. The wife was not happy.

I bought a turntable he'd reviewed. I'd met the manufacturer, Jimmy had reviewed it and I went to see him to discuss it. He'd had it in his system for 3 months. The review was short and to the point. I bought something else he used himself. Both products I used for over 10 years, I still own the turntable.

 

[italia]

If I recommended a doctor on the basis that he was a leading expert in national security, would you trust his opinion? How does expertise in national security contribute towards an opinion on audio?

Thanks for your question. If you read the quote at the top of my post, you'll see that I was referring to the OP's statement that most reviewers are part-time employees, and relaying my experience in discussing that part-time nature with a friend of mine decades ago. My comment was not attempting to establish reviewing credibility based on his full-time job or educational credentials.

With respect to your comment about a recent Cordesman review, I've not read any of his reviews in the past 25 years so I cannot comment about his ability to assess sound quality given his age. But yeah, 80+ years old would give me pause.

 

[morricab]

If I recommended a doctor on the basis that he was a leading expert in national security, would you trust his opinion? How does expertise in national security contribute towards an opinion on audio?

I first heard of Mr Cordesman when PS Audio referenced a review he'd done of their new speakers, maybe 2 years ago. I looked him up. He was 80+ years old. At that age his hearing would have been seriously impaired. I didn't read the review. To be honest it was not just his age that put me off, I just struggle to summon the willpower to read any audio reviews. It suggested to me that his name was being used to promote the product.

Ability for critical thinking and analysis? His hearing might have been diminished but if his mind was still sharp I think his analytical approach would still have some validity.

 

[Lee]

If I recommended a doctor on the basis that he was a leading expert in national security, would you trust his opinion? How does expertise in national security contribute towards an opinion on audio?

I first heard of Mr Cordesman when PS Audio referenced a review he'd done of their new speakers, maybe 2 years ago. I looked him up. He was 80+ years old. At that age his hearing would have been seriously impaired. I didn't read the review. To be honest it was not just his age that put me off, I just struggle to summon the willpower to read any audio reviews. It suggested to me that his name was being used to promote the product.

Cordesman was one of the nicest gentlemen in the hobby and one of the best reviewers imho.

 

[MarkusBarkus]

...I enjoyed his reviews. At some point, the penny dropped, and I wondered: Can this be *that* Tony Cordesman? I had seen him in interviews over the years, read editorials, etc..

I thought he was thoughtful and thorough as a reviewer. However, as would be the case for me with any reviewer, I did not/would not purchase because of the reviewer/review, regardless of the day job they held.

 

[ssfas]

Thanks for your question. If you read the quote at the top of my post, you'll see that I was referring to the OP's statement that most reviewers are part-time employees, and relaying my experience in discussing that part-time nature with a friend of mine decades ago. My comment was not attempting to establish reviewing credibility based on his full-time job or educational credentials.

With respect to your comment about a recent Cordesman review, I've not read any of his reviews in the past 25 years so I cannot comment about his ability to assess sound quality given his age. But yeah, 80+ years old would give me pause.

It's strange how that thing about him being in national security popped up so many times when I searched him, like it adds credibility. I saw two movies on Saturday, "A Real Pain" and "The Brutalist". We thought A Real Pain was a better movie, the main character was a real pain of an individual, but he was far more perceptive and honest than anyone else in a group of 6 people. It was a very perceptive and quite brilliant film.

A lot of reviews are related to the reviewer's personality and style. If only they were anonymous, I might give them attention. I wouldn't care if they were written by Mr Personality or a total sociopath.

 

[microstrip]

(...) I first heard of Mr Cordesman when PS Audio referenced a review he'd done of their new speakers, maybe 2 years ago. I looked him up. He was 80+ years old. At that age his hearing would have been seriously impaired. I didn't read the review. To be honest it was not just his age that put me off, I just struggle to summon the willpower to read any audio reviews. It suggested to me that his name was being used to promote the product.

I do not expect someone reading for the first time an isolated review of a reviewer to get the proper value of it - I am not surprised by your reaction.

In fact you missed an excellent review by one the best audio reviewers I had the pleasure to read along decades. Anthony Cordeseman died a year ago, someone told it much better than me

https://www.psaudio.com/blogs/coppe...HnPDSzhK3P7vk-lSyeOXdJ3OSf1lmdWmsMFh_lpDCz_pK

IMO Anthony Cordeseman exquisite reviews do not tell us everything on how equipment sound - they teach us how to listen to this particular component, and surely what he things about it. Each of his long reviews is a lesson on the high-end - he even openly exposes the limitations the reviews in their text.

A generation of knowledge audiophiles benefitted from reading his reviews and articles - IMO time much better spent than seeing movies of high-end influencers. For those interested in his TAS reviews of the last ten years, please see: https://www.theabsolutesound.com/author/anthony-cordesman/