What is a reviewer?

[morricab]

It's not the same at all, because accessibility to those is easy and volumes are high, so lots of checks and balances. There is no exclusivity to watching a movie and writing a review about it and comparing IMDB user ratings of tens of thousands. On the other hand, how many reviewers and users have heard AF0? Or any average priced TT? 1 professional reviewer, definitely 2, an completely sway things.

I am not talking about that...I am talking about the general lack of qualifications for critics of all artistic output...not how easy it is to get hold of the media or item to review. Sure the number of reviews is less for very expensive high end gear...just like it is for a high end car...usually on the bigger magazines or YouTubers get hold of them...and those are often loaners from owners and not from the manufacturer.

So, from the perspective of who is qualified to be a reviewer I think it is pretty much the same.

 

[ssfas]

Okay.

All I"m saying is that to achieve what you describe as a "perfect review" (below) will require considerable time and expertise and I believe that won't happen for free. You might get some of this from a Martin Collums HiFiCritic subscription (which takes no advertising), although I believe he closed his magazine a year or so ago.

"For me, a perfect review would
— Describe what the product does
– Describe how it does it, with some detail on the technology employed
– Test the machine on the workbench to confirm whether or not the published performance measurements are accurate and, if any critical measurements are not provided, provide them.
– Explain what complementary products the item being tested would work with, cartridges with arms, amplifiers with speakers etcetera
– provide some subjective listening observations that are musically meaningful and repeatable.
– Provide full details of specifications, pricing, distribution, relevant websites.
– provide a range of comparable and/or competing products.
– avoid any flowery language, attempts at humour etcetera. "

Usually you get what you pay for.

The people I mentioned have/had separate careers and whether or not they got paid for their assessments was irrelevant. John Borwick was succeeded at Gramophone by Andrew Everard, who is a professional journalist so his work is done for commercial gain. He only has a few pages at the back of Gramophone, but it is well focused on the needs of the mainstream classical music lover. Jimmy Hughes used to use a component for months before writing up his review. I saw he is now more active on a site called Stereonet (he used to publish elsewhere), possibly because he has recently retired from role at Leica.

I write expert reports for a living. There are probably quite a lot of people with my expertise, but very few who can write a good report and even less who can adequately defend their opinion under cross-examination in court by some of the brightest trial lawyers. I've been doing it for 33 years, so I must be doing something right. The report itself is usually the visible tip of the iceberg, the investigation is the far larger chuck hidden underwater. To use another metaphor, the report has to be watertight, defensible to the last detail. Any weak points and it will be like hitting an iceberg, you will sink like the Titanic.

No, my reports don't come cheap. They come very expensive. Some clients want cheap preliminary assessments. I generally don't do them. Without a certain level of investigation, it would be easy to mislead or just get it completely wrong.

I find that, even after a lot of work, some of my colleagues fail to identify the motive behind a business or activity we are investigating. You often get this with audio reviews, where the reviewer has not got to grips with what the designer was intending to achieve. The easiest way is to ask them, but so many reviewers don't bother. As we do, it should be normal for the manufacturer to have the opportunity to confirm that the technical and factual aspects are correct and complete.

So much is now personality driven. The @Ron Resnick idea that you need to do your own profiling of each reviewer sadly may be true, but more sadly it's a bizarre way to have to find out something honest about audio. In John Borwick's day reviews (they were called Technical Reports) were done by him or a couple of staff engineers, and you would be hard pressed to identify the author without looking to the ending credit. Now that you have mostly independent contracted reviewers, it seems often to be more about them than the product.

 

[ssfas]

You could ask the same about book, movie or music critics. Some of them have actually worked in some way in that industry and therefore have a deeper than average knowledge on the subject. However, the key attribute a reviewer must have is critical thinking skills and enough background education on the subject at hand to make meaningful constructive criticism and ability to critically compare the item under test to similar items of a similar or above (or below) class. This also means a lot of exposure to gear...I mean a lot of exposure and thinking critically about what does and doesn't work and with what it goes with and what it might not go with.

This is why just being able to write well, while also important to convey the impressions generated, is far from sufficient to be a good reviewer. I think a lot of reviewers come from a journalism background and well...that just makes for flowery writing with very little insight or understanding. Might make for good ad copy but not a serious critique of seriously expensive gear.

Critical thinking and ability to compare what one hears in one thing with another thing (or things) is crucial to having meaningful reviews that potential customers can actually use.

My wife and I mostly go to ballet and dance. First you have the quality of the choreography, then the technical and artistic skills (which are quite different) of the dancers. We try and avoid reviews because they can induce bias. Last Saturday was the opening night of the Royal Ballet season, so no reviews, we got served Alice in Wonderland, a 3-act ballet from about 10 years ago. Our conclusion was that it would have been great if they scrapped the 2nd act completely. The corps de ballet, for whom the 2nd act was largely contrived, were technically well below their usual impeccable standard. This happens at the start of the season as they are often not match fit. So when the reviews came out, one said the 2nd act should be scrapped, another said it was their favourite part of the whole thing.

Lesson One There's no accounting for taste.

Last year we saw New York City Ballet, a top US company, with some very good dancers. They had massive hype in the press. At the interval the gentleman next me was commenting on how good they were, especially one of the stars, I think it was Megan Fairchild. I said I thought she was terrible, she looked like she had a stomach muscle injury, she could barely jump and hold her position. My wife had gone outside and found her leaning against a wall, throwing up. She actually apologised on twitter after the show, saying she had food poisoning and probably should not have performed.

Lesson Two Some people just can't blank out the hype, and will believe it, come what may, even if demonstrably false

My wife was a ballet and contemporary dancer, in Europe and the USA, so she can spot every technical aspect of dance. I've no training, but I've been to maybe 2,000 shows, so I have a pretty good idea about what's good or bad and when things are or are not working or go wrong. I don't need to read critics and I can tell their favouritism or bias.

Lesson Three If you're an amateur, how ever much experience you have, you're still an amateur. It will never make up for technical, professional knowledge.

 

[Mike Lavigne]

It's not the same at all, because accessibility to those is easy and volumes are high, so lots of checks and balances. There is no exclusivity to watching a movie and writing a review about it and comparing IMDB user ratings of tens of thousands. On the other hand, how many reviewers and users have heard AF0? Or any average priced TT? 1 professional reviewer, definitely 2, an completely sway things.

agree. for me personally; hifi reviews start and end about references. systems, sources, media. otherwise they have little value.....for me. and they don't need to do the compares in the review, but i need to know that they have those references relative to their thoughts. because that is what brings value to me in my thinking.

which eliminates my interest in many reviews or reviewers. and they might be great writers, and very effective reviewers. but for hifi they also have to have the right tools. which is a challenge; to have access and experience with the right tools.....for relevance to me.

as far as defining the tools; i know it when i see it. it's not an exact thing.

and not saying that i think everyone should approach it like i do. but i'm where i'm at in my journey, and others are where they are at. so the problems being solved, and answers given, are relative to that position.

i'd much rather have casual/anecdotal feedback from someone who i know has the proper references and tools than reviews without those references. which is the underpinning and value of this forum. i do respect talent and effort given to proper communication and writing too, but it is secondary.

 

[morricab]

My wife and I mostly go to ballet and dance. First you have the quality of the choreography, then the technical and artistic skills (which are quite different) of the dancers. We try and avoid reviews because they can induce bias. Last Saturday was the opening night of the Royal Ballet season, so no reviews, we got served Alice in Wonderland, a 3-act ballet from about 10 years ago. Our conclusion was that it would have been great if they scrapped the 2nd act completely. The corps de ballet, for whom the 2nd act was largely contrived, were technically well below their usual impeccable standard. This happens at the start of the season as they are often not match fit. So when the reviews came out, one said the 2nd act should be scrapped, another said it was their favourite part of the whole thing.

Lesson One There's no accounting for taste.

Last year we saw New York City Ballet, a top US company, with some very good dancers. They had massive hype in the press. At the interval the gentleman next me was commenting on how good they were, especially one of the stars, I think it was Megan Fairchild. I said I thought she was terrible, she looked like she had a stomach muscle injury, she could barely jump and hold her position. My wife had gone outside and found her leaning against a wall, throwing up. She actually apologised on twitter after the show, saying she had food poisoning and probably should not have performed.

Lesson Two Some people just can't blank out the hype, and will believe it, come what may, even if demonstrably false

My wife was a ballet and contemporary dancer, in Europe and the USA, so she can spot every technical aspect of dance. I've no training, but I've been to maybe 2,000 shows, so I have a pretty good idea about what's good or bad and when things are or are not working or go wrong. I don't need to read critics and I can tell their favouritism or bias.

Lesson Three If you're an amateur, how ever much experience you have, you're still an amateur. It will never make up for technical, professional knowledge.

Thanks, I have a similar experience with an ex who is a professional violinist. What I would ask you is this: Do you think you have the requisite knowledge and now critical eye to be able to critique performances?

I read a scientific article where they were comparing functional MRI scans of people's brains listening to music. They looked at untrained non-musicians and also professional musicians. There is a profound difference in the brain usage between the two with the professional, unsurprisingly, using many more areas of their brains (hearing processing is not nearly as localized as vision...perhaps why it is easier to agree on visual facts) than the untrained non-musicians. A funny thing happened though when they started training the non-musicians how to be more critical listeners...their brain patterns started to converge on the brain patterns of the professionals! So much so that there was little to separate them after a couple of weeks of training. That doesn't mean that they could pick up instruments and start playing, or if they could already play a bit, suddenly become professionals. But for something like observation and critique they might have gotten very good and probably able to point out rather subtle differences in sound and performance.

So, I do think if one has been heavily exposed in the right way it could go a long way towards making one a good critic if not a good performer.
FWIW, I was also a reviewer for Positive Feedback on and off for over a decade.

 

[Kingrex]

Usually you get what you pay for.

I know a guy that replaced his $70,000 amp with a $12,000 amp because the $12,000 was better. In truth he could not help himself and bought the $20,000 replacement. He heard the $12,000 stereo version in his system that trumped the $70,000, so he then ordered the monoblock set for $20,000.

I am beginning to sense there is less and less connection between price and performance once you get beyond say $20,000. It may be more about esthetics and flowery language tief to specifications. Lots of watts. Low distortion. You know. But actually sounding better????????

 

[Kingrex]

I'm frustrated with one well know reiewer as I went to his place and saw his electrical. Its really bad. We set a date to change it. I said, why don't you fix the walls in the room while your at it. They are bad too and I'm going to have the installer rip them out. He thought that fantastic but never got rolling. Last I asked he said he had to finish a couple speaker reviews before starting work.

There is 0 chance he is actually heating what the speakers he is reviewing are capable of. Not a chance. They will absolutely change when his amps are receiving good power. I heard it many times personally. So what are you getting from a reviewer. There is 0 standard for the "laboratory " environment a review is performed in. I said to Fremer I sould be contracted by TAS,Stereophile etc to.go.to each of their staff locations and submit a study of the space. A room specialist should also be contracted to do.the same.

I.would go so far as to say, an accurate review of much gear would include going to owners homes that have well set up spaces to hear the gear in a variety of settings.

I put more faith in what Bonzo says about how something sounds than any reviewer. Simply because he has been exposed to.such a broad variety of room settings and systems. He's probably the best reviewer out there.

And Timas DIY record cleaning thread was probably the best education on how and what to do to properly clean a record. The stuff in magazines is toilet reading. Fun, not of value.

 

[Mike Lavigne]

I'm frustrated with one well know reiewer as I went to his place and saw his electrical. Its really bad. We set a date to change it. I said, why don't you fix the walls in the room while your at it. They are bad too and I'm going to have the installer rip them out. He thought that fantastic but never got rolling. Last I asked he said he had to finish a couple speaker reviews before starting work.

There is 0 chance he is actually heating what the speakers he is reviewing are capable of. Not a chance. They will absolutely change when his amps are receiving good power. I heard it many times personally. So what are you getting from a reviewer. There is 0 standard for the "laboratory " environment a review is performed in. I said to Fremer I sould be contracted by TAS,Stereophile etc to.go.to each of their staff locations and submit a study of the space. A room specialist should also be contracted to do.the same.

I.would go so far as to say, an accurate review of much gear would include going to owners homes that have well set up spaces to hear the gear in a variety of settings.

I put more faith in what Bonzo says about how something sounds than any reviewer. Simply because he has been exposed to.such a broad variety of room settings and systems. He's probably the best reviewer out there.

And Timas DIY record cleaning thread was probably the best education on how and what to do to properly clean a record. The stuff in magazines is toilet reading. Fun, not of value.

the problem with connecting dots is that it's not that simple. just like some normal rooms can sound good, you can hear around some issues, if you have the gear and media references. so it's a mistake to take that too far. if you are reviewing room design, sure. or reviewing AC grid issues. agree.

we can objectively agree that those compromises exist. but over time skills get refined and truth comes through. dismissing results would require evidence that it happened. when the evidence is that the results have been pretty (or very, very) good, then that is the truth. their tools apparently work.

over decades some true experts have established a track record, in spite of what we see. they excel sufficiently all around to compensate.

we see lots of 'mad scientist' sort of environments that = genius. i know i don't possess the talent to pull that off myself. i need all the help i can get for my best result.

 

[Ron Resnick]

– Provide full details of specifications, pricing, distribution, relevant websites

This is just totally personal preference, of course, but I actually find it annoying when a review is lengthened by elements like this which I can look up easily and instantly on the manufacturer's website. (I apply this preference to my Masters & Makers interviews myself: I almost never ask a question the answer to which can be found easily by a viewer on the designer's website.)

– provide a range of comparable and/or competing products

Unless the reviewer has actually heard such comparable and/or competing products and compared them to the component under review -- or at least some indirect listening experience with such components at a dealer or a show or something -- I find a mere, cold listing of competing components to be another section of review text I don't want to waste time reading.

– avoid any flowery language

I don't know exactly what you mean by "flowery language." (If you had said avoid turgid language I would have known exactly what and which reviewer you were talking about!)

attempts at humour etcetera.

The natural humor Michael Fremer sprinkles in his reviews is one of the reasons I love reading his work! On the other hand for some reason Jason Serinus' attempts at humor usually fall flat with me.

 

[Ron Resnick]

The @Ron Resnick idea that you need to do your own profiling of each reviewer sadly may be true, but more sadly it's a bizarre way to have to find out something honest about audio. In John Borwick's day reviews (they were called Technical Reports) were done by him or a couple of staff engineers, and you would be hard pressed to identify the author without looking to the ending credit.

Respectfully I feel that you may not be distinguishing sufficiently between subjective reviews and objective reviews. In subjective audio reporting there is nothing objectively "honest" to find.

If subjectivist high-end audio reviewers were engaged in objective reporting and measurements analysis then, yes, profiling reviewers would be a bizarre way to have to find out objective reports and measurements about audio components.

My suggestion results wholly from the subjective nature of what a subjectivist high-end audio reviewer is reporting upon. This endeavor is at the opposite end of the reporting spectrum from "technical reports."

 

[Ron Resnick]

I said, why don't you fix the walls in the room while your at it. They are bad too and I'm going to have the installer rip them out.

There is 0 chance he is actually heating what the speakers he is reviewing are capable of. Not a chance. They will absolutely change when his amps are receiving good power.

I don't see sub-optimal sub-panel to outlet wiring as being material. Whatever it is, it is an even playing field for all components.

I put more faith in what Bonzo says about how something sounds than any reviewer. Simply because he has been exposed to.such a broad variety of room settings and systems. He's probably the best reviewer out there.

With this you have made Kedar's year!

 

[Rt66indierock]

What is a reviewer? Qualifications? Purpose? responsibilities? are they critics? reporters? journalists? marketers? cheerleaders? influencers?
What should there place in the Industry be and how do we get there?
Limits? rules?
What are your opinions?

I read in so many threads comments about all the "reviewers" maybe a place to actually discuss it will be useful

Elliot, when I walked into my office suite this morning, I walked past the TV monitor showing four of us have education, experience, training and passed a really hard test so we can hold out to the public that we know how to test stuff. Three of us have extensive experience and additional training.

Without that skill set any audio review is just marketing material.

 

[wil]

I like best the reviewers who are good writers, know what they are talking about and let their humor, humanity and occasional wisdom come through.

Herb Reichert, Fremer and the late Art Dudley come to mind.

Six Moons guy is practically unreadable but the content can be rich. (interestingly, when he speaks on Darko’s podcast, he’s much more coherent).

The fragrant infomercials on Mono/Stereo are ludicrous, imo.

 

[Kingrex]

I get the concept that if your room and electrical power are deficient and all you know, then everything you hear is based upon the same poor foundation. So, comparisons or opinions over the years will be accurate as to what you perceive as superior to something else.

But, with inferior power and.room, you won't have knowledge of the true voice of the equipment. It will contain distortions and veiling that will rob dynamics, clarity and add an edgyness. An edgyness or distortion that others with good power won't hear. I still remember the Estelon speaker at Fremers. Before the power update Ed.and I went to dinner and both felt they were ok. After the power update we were surprised how good they sounded. Point being, if Im a reviewer and hearing systems at shows and factories and other people systems, and I take a speaker into my hobbled room and listen, I would believe all the reviewers past experiences with other systems will influence.their perceptions. And even if the reviewer only heard say a Magico at some other well set up place, the memory will remain and some comparison on some level as they asses a different speaker in their room will be influenced.

 

[OrigTweaker]

Perhaps the answer regarding what makes a good reviewer is that the reviewer agrees with me

Meaning, if you purchased a piece of equipment and the reviewer hears it like you do, then you might follow his reviews later. I found that with Hans Beekhuyzen. After I had the Grimm MU1 streamer for a while, a friend sent me his review of the MU1 (I had no idea who he was). Spot on, I thought. Later, he reviewed and purchased the Grimm MU2 (streamer/DAC). Time for an in-home demo, I thought. That worked out well, as did his recommendations for footers and a passive ethernet filter for the Grimm equipment.

Hans B. was the one who turned me on to the world of Raspberry Pi audio when he recommended the Allo DigiOne transport six years ago. I was looking for affordable alternatives to the expensive Meridian electronics I had been using, and these relatively cheap designs got me thinking about -and subsequently exploring- outboard power supplies, operating systems and network issues. More recently, like you, I happily took his advice on Stack Audio footers and Network Acoustics products.

He also alerted me to the value of Digital to Digital Converters (I use Singxer SU-6) as an inexpensive glimpse into the sonic stratosphere occupied by brands like Antipodes, Aurender, Grimm, Innous and Taiko once the basics of power, interconnects and speaker/room interaction/correction have been addressed.

Now that I've gone over the edge into esoteric RPi configurations directly feeding my new active digital loudspeakers, I ignore most (analog) talk about turntables, amps etc., and don't follow reviewers like I used to. Are there any out there today who reliably report on the most promising digital innovations so that I might discover the next Allo, NA, Stack or UpTone?

 

[stehno]

...

I agree with Stehno 100% when he said of a good reviewer: "possessing the ability to discern / interpret what they hear and conveying what they hear to the reader."

...

What makes a good reviewer?

Is it their ability to ‘sell’ a product? Absolute transparency? Technical prowess? Golden ears? Product discovery? When you read (or watch) a review, what is it about the content that makes you want to come back and take in more of that creator’s work?

www.whatsbestforum.com

Sorry, tima, but this attribute requires more than mere intellectual agreement.

 

[ssfas]

Thanks, I have a similar experience with an ex who is a professional violinist. What I would ask you is this: Do you think you have the requisite knowledge and now critical eye to be able to critique performances?

I read a scientific article where they were comparing functional MRI scans of people's brains listening to music. They looked at untrained non-musicians and also professional musicians. There is a profound difference in the brain usage between the two with the professional, unsurprisingly, using many more areas of their brains (hearing processing is not nearly as localized as vision...perhaps why it is easier to agree on visual facts) than the untrained non-musicians. A funny thing happened though when they started training the non-musicians how to be more critical listeners...their brain patterns started to converge on the brain patterns of the professionals! So much so that there was little to separate them after a couple of weeks of training. That doesn't mean that they could pick up instruments and start playing, or if they could already play a bit, suddenly become professionals. But for something like observation and critique they might have gotten very good and probably able to point out rather subtle differences in sound and performance.

So, I do think if one has been heavily exposed in the right way it could go a long way towards making one a good critic if not a good performer.
FWIW, I was also a reviewer for Positive Feedback on and off for over a decade.

Dance knowledge is a curious one. Most people who see something like McMillan‘s Romeo and Juliet with Prokofiev’s amazing music will be completely blown away, even if it’s the first time they’ve ever seen a ballet. You don’t need to know the nuances and details. For us, our first decision is which cast to see, because The Royal Ballet will have five or six pairings doing three or four performances each. We know all the dancers and we have to decide who we want to see doing which role. We know the choreography and music backwards, so our thought processes will be completely different to the novice, as we will be anticipating sections and elements in the dance.

You then have new pieces by choreographers whose dance language you know. We have one tomorrow, a piece by Crystal Pite for the National Ballet of Canada. There will be some bias, because she’s never put on a show that isn’t fantastic. Next Wednesday is a new piece by Hofesh Schechter, who we also know well, but there is some trepidation because he has been lacking in originality.

Finally, you have new pieces by new directors and choreographers who you know nothing about or some who produce pieces that are all totally different. One of the best last year was a combination of dance and rock climbing with a high wire act thrown in.

A reviewer of established ballet repertoire, just like someone reviewing a Beethoven Symphony performance, would be expected to have a profound knowledge of the subject matter, being able to critique both the overall nature and specific details of the performance. When you got something totally new, experience counts, but two people may have a completely different reaction.

The greatest dance critic ever was Clement Crisp. He wrote dance reviews for the Financial Times for 64 years. He used a pen more like a scalpel, and could cut very deep. There was nothing he didn’t know about dance, classical and modern, but before becoming a dance critic, he was a French teacher. He knew everyone in classical dance, and had a very close relationship with Alicia Markova for decades, perhaps the greatest dancer of her era, which gave him a direct link back to 19th century St Petersburg.

I was just reading his review of the first performance by Cloud Gate from Taiwan, who first came to London in 1999. The show was an acquired taste. We liked it, he obviously didn’t. So he just made a lot of jokes about rice. It’s a brilliant example of a review where most readers would not have seen the piece and would not do so, so was largely written to entertain. that’s all well and good for a dance show, but if a review will influence whether or not someone is likely to spend a large trunk of money on a piece of audio equipment, there is agreater responsibility.

 

[PYP]

I almost never buy anything new to the market. For me it has to have several years of market approval. Here’s an example of a review by Raphael Todes of a speaker he’s used for decades, the review ticks most boxes.

Bowers & Wilkins 802 D4 Floorstanding Loudspeaker Review

Bowers & Wilkins 802 D4 Floorstanding Loudspeaker Review

www.stereonet.com

If I was interested, I’d call him and ask to go and have a listen. As I sold him a perfect pair of ESL63, he owes me one.

Thanks. I've read a few reviews of the 802D4 since that is a logical upgrade for me. This review was particularly useful to me since he directly compares the D3 iteration to the D4 (the other reviewers either did not make that comparison or did so based upon memory). That is exactly the kind of information useful for someone considering an upgrade.

Of course, I need to listen to them before a purchase, but this review makes it more likely I will do so (hard to get inspired for a change when the current setup pulls me into the music).

 

[bonzo75]

I like best the reviewers who are good writers, know what they are talking about and let their humor, humanity and occasional wisdom come through

Thanks. I will try to dial back to occasional from regular

 

[tima]

And Timas DIY record cleaning thread was probably the best education on how and what to do to properly clean a record. The stuff in magazines is toilet reading. Fun, not of value.

Thank you!