Phelonious: I assume you've read our policy. Then you'll already know two things. A/ do your due diligence on the reviewer, B/ if you're not happy with the outcome, you can request a second opinion from a second writer. These are both mentioned clearly in my article.
About A/, our archives are listed by writer. It's very easy for you to 'vet' a writer on their style, their biases, what they own, what their room is like... it's all there. Doing that due diligence is your job, not ours. So let's say you've identified our resident DAC expert; or the one whose tastes most coincide with what you think your DAC sounds like. You've committed with your payment and the assignment is a go.
If the review is *scathing* as you put it, your DAC must have been terrible. Because we wouldn't write such a review unless there was concrete evidence to support such a claim. It wouldn't lock to signal. It dropped out. It had a channel imbalance. It was shoddily made. Its transformer hummed. Its display failed. The paint was peeling. The list of things that would have to be wrong to merit a scathing review is quite long. Phelonious, you're a crap DAC designer! -
What's far more likely is that everything is at least competent, works as it should, is priced as it should for what it is and all that. Now it's down to small niggles and sonic commentary. Small niggles couldn't possibly make for a scathing review. Could sonic commentary? Unless the DAC was truly terrible, I don't think so. A writer's biases always show up as preferences but that doesn't make for a scathing review either. And we won't have any compatibility issues with a DAC as one might have with the wrong speaker for a given room; or an underpowered amp for an application. Such matters are handled upfront and part of your own vetting. Does the reviewer have the right gear, the right room and the proper experience with your type device? It's also part of my job as the publisher to assign the right writer to the right gear and consider his exposure/experience level.
I wouldn't publish a review where a writer got personal and forgot that it's not about what he likes; but to describe what something sounds like. It's perfectly okay to interject personal likes. It's not okay to get personal and condemn a product just because it fails to match your taste. If you describe its sonics to the best of your ability and then state "this wasn't for me, I like my DAC better and here's why"... then I'd not call it a scathing review. I'd call it a qualified review that didn't end up in the rave you may have liked -
B/ If regardless you felt the review failed to do justice to your equipment and you have really serious issues with it... then you'd get a second writer (and no, you'd not pay extra). If a product was truly bad of course, a second writer would merely say so again. Now you've got a double whammy. Bad idea but your call.
To be honest, to me these are mostly abstract concerns. We don't accept everything for review we're offered. We do a certain amount of pre-vetting ourselves. Our staff and abilities are limited. We can only process so much. Do we want to waste our time on stuff that reeks of trouble? No. Do we accept stuff we know we're gonna hate just so we can write the scathing review the perennial conspiracy theorists are waiting for as proof of a magazine's objectivity? Hell no.
Of course none of it eliminates the
potential for a lemon to slip through here and there but in my experience, outright lemons really are very rare. A product simply doesn't survive in the market place if it's that bad. And if a reviewer does come across one and doesn't say so, the boards will be on a witch hunt when first owners post on blown-up units and the lot. Rightly so too.
In the real world, there are plenty of normal checks and balances along the way. Anyone working on the internet is subject to real-time instant inspection, commentary, criticisms and comparisons. Anyone playing games, being way off and demonstrably wrong most of the time just doesn't make it. If he's not fired, people simply stop reading him. Or those who do do so for pure amusement to ridicule the guy on the forums afterwards.
If you actually
were a DAC maker, you'd look at all your review exposure options carefully. You'd not come to me/us if you thought we were whack jobs, would you? With 12 years of reviews in our archives and at your finger tips, you can vet us to your heart's content. If you don't do that work; and if don't know that I've got a Vega and Hex and Eximus and Aqua DAC in inventory; and then act surprised that I compared yours to four class leaders which cost the same as your DAC and and found yours wanting... well, then you didn't do your home work on multiple fronts, did you? That's why, unlike most, we publish photos of our systems too. It's all about data points to relate to a given writer and his work.
I feel very confident saying that we've been nothing but consistent. You might like us, you might not but we're not arbitrary. You pretty much know going in what you're going to get.