Yup.
Haven't tried since. I see no reason to do so.
Knowledge and levels, John - all your average music-lover needs to account for in order to determine whether A or B might improve his/her real-world music listening enjoyment.
Conclusive "Proof" that higher resolution audio sounds different
- Thread starter amirm
- Start date
Yup.
Haven't tried since. I see no reason to do so.
Knowledge and levels, John - all your average music-lover needs to account for in order to determine whether A or B might improve his/her real-world music listening enjoyment.
So you are an example of someone who hears differences in Winer's files but doesn't need a small section to be isolated & listened to repeatedly to be able to identify the files. Contradicts Tim's assertion about how irrelevant this test is.
Sure, I knew this would be your answer - same as you see no reason to look at BS1116, I guess.
Keep repeating the mantra, Max - it has nothing to do with this thread, however & is also wildly mistaken when it comes to correct listening testing.
Positive results do nothing to verify a test procedure. Obtaining the same results as a valid procedure obtains doesn't make a questionable procedure suddenly become valid.
David I think you're confusing experience with "trained" in the context of this thread.
And yes, gentlemen, my position is very straightforward. You can complicate things as much as you like. You can assume I don't understand what's being discussed if that makes you feel better, but all I'm saying is that playing repeated passages of jangling keys while listening for something you've been trained (or at least told) to listen for is a very different exercise than listening to music. I would like to see blind listening tests conducted using a variety of experienced listeners (say audiophiles, audio engineers, audio engineering students and simple music lovers), a variety of high quality systems, a variety of high quality recordings...and see how audible the difference between hi-res and 16/44.1 is under those conditions and to whom. Think Meyer & Moran under BS1116 guidelines. Pretty straightforward, but continue diverting if you must.
Tim
Hi Tim. First, let me say that members should not keep addressing each other and saying they don't understand this and that. Let's just express our own opinion as Tim is doing above or the technical topic.
Back to Tim's post, we do research in a specific way based on collective experience we have had in the field. It might seem odd at first but you have to trust us that we know what we are doing Tim
. Here is an example how we have learned these lessons the hard way from JJ's presentation that Arny linked to the video of:
JJ is recounting his first music lossy codec he designed. How it sounded good, sounded good, and then it didn't. Notice that it sounded good on far more tracks than the ones it didn't. That doesn't matter. The codec must have even response across ALL content and ALL listeners. That is of course is not remotely practical. So what to do? We test difficult tracks using trained listeners for the core R&D. We do occasionally do larger scale tests with broader set of music and listeners but in general, science is made from the former.
As JJ explains we aim to find the threshold of hearing for these artifacts. That is the science we want to discover. It goes without saying that there are people who are not that good, and content that is not as revealing. We don't need to keep testing for that. I can test a million people who think MP3 at 128 kbps is the same as CD. When developing a new codec, why would I want to bother testing those people?
Also, a point of correction which hopefully was not created by me
. My training is in finding audio compression artifacts. Those artifacts I know forward and backward. What I used here is not that training. Going into Arny's test or that of Scott/Mark, I did not know what to look for compared to the clear picture I have of compression artifacts. What did help me, is being a critical listener, and not one experienced in hearing these specific artifacts (again JJ's video explains the same).Think of me as a detective at a murder scene. Every case is new so he doesn't walk in knowing who dun it. He knows how to gather evidence and hopefully solve the case. The other people who found differences without formal training either have critical listening abilities or developed the first phase of it through this experience. If they passed these tests, they are above ordinary listeners.
One of the "proven" values of these tests/thread is just that. That it is important, as is done routinely in the industry to deploy difficult content and critical listeners. Data otherwise for small differences is of no value. We know most people can't hear those artifacts or care to hear them. So your wish to have more music samples, and more ordinary listeners is just something that researchers in this field don't care about for reasons I have explained.
Indeed & this type of wished for experiment will guarantee a null result. Hence the mass of null results so prevalent in this area - the status quo, if you like. I don't believe anybody who has followed this thread & absorbed any of it is still in any doubt just what correct test procedure is (of course they can deny it as some continue to do). Therefore, Amir, I think you are wasting your breath explaining all the above - what is really being asked for is a return to the status quo situation of pretend science & null results.
Last edited:
.Hi Tim. First, let me say that members should not keep addressing each other and saying they don't understand this and that. Let's just express our own opinion as Tim is doing above or the technical topic.
Back to Tim's post, we do research in a specific way based on collective experience we have had in the field. It might seem odd at first but you have to trust us that we know what we are doing Tim
JJ is recounting his first music lossy codec he designed. How it sounded good, sounded good, and then it didn't. Notice that it sounded good on far more tracks than the ones it didn't. That doesn't matter. The codec must have even response across ALL content and ALL listeners. That is of course is not remotely practical. So what to do? We test difficult tracks using trained listeners for the core R&D. We do occasionally do larger scale tests with broader set of music and listeners but in general, science is made from the former.
As JJ explains we aim to find the threshold of hearing for these artifacts. That is the science we want to discover. It goes without saying that there are people who are not that good, and content that is not as revealing. We don't need to keep testing for that. I can test a million people who think MP3 at 128 kbps is the same as CD. When developing a new codec, why would I want to bother testing those people?
Also, a point of correction which hopefully was not created by me
Think of me as a detective at a murder scene. Every case is new so he doesn't walk in knowing who dun it. He knows how to gather evidence and hopefully solve the case. The other people who found differences without formal training either have critical listening abilities or developed the first phase of it through this experience. If they passed these tests, they are above ordinary listeners.
One of the "proven" values of these tests/thread is just that. That it is important, as is done routinely in the industry to deploy difficult content and critical listeners. Data otherwise for small differences is of no value. We know most people can't hear those artifacts or care to hear them. So your wish to have more music samples, and more ordinary listeners is just something that researchers in this field don't care about for reasons I have explained.
I get all of that, Amir. What you and John and micro don't seem to get is thar I'm not talking about what scientists care about. I accept that you and the others heard the difference, even though we are far from anything approaching proof. Let's call it done; you heard it. Let's even assume this difference is positive, in favor of hi-res. That's certainly not established, but it doesn't matter because I'm not talking about what scientists care about as revealed by trained listeners. I'm talking about comparing that to what listeners hear when listening to music.
What field? Because if the overwhelming majority of listeners will never hear this difference, it may be interesting from a purely academic point of view, but there won't be a market for a product. I've known plenty of research professionals who care about that. If they have no impact outside of the lab, interesting data points become moot points very quickly.
Tim
Last edited:
I suspect that the key to the validity of this test is the recordings that were involved. No problem contriving recordings and playback contexts that show CDA isn't transparent. Doing so with mainstream commercial recordings played as real world people play them is a different story.
So it IS a "why bother" situation after all
Yep, it appears to be the case Jack.
I can't find it now but someone recounted a story about delaying the woofer response by up to a second or two (memory vague of this) in a music system that was in operation in the workplace & nobody noticed it, even when it was pointed out to them. So, there is a lot of scope for ignoring the obvious in "normal listening" as "real world people play them". This thread is now bordering on the ridiculous with such non-science posts as the above.
One of the recordings that has turned out to be a codec buster that is a very crisp a capella female voice is "Tom's Diner" from "Solitude Standing" by Suzanne Vega. The ABX team formed a list of recordings for the purpose of making equipment misbehave in the early 1990s that included it (it was released in 1987). It was used in the Stereo Review amplifier and CD player tests.
http://www.npr.org/blogs/therecord/2011/03/23/134622940/the-mp3-a-history-of-innovation-and-betrayal
""In 1988 when I thought, 'OK, that's near perfect, what we've done,' I heard about this song and then heard it myself — it's the acapella version [of Suzanne Vega's 'Tom's Diner']. The CD is Solitude Standing. The way it's recorded — with Suzanne Vega in the middle and little bit of ambience and no other instruments — is really a worst case for the system as we had it in 1988. Everything else sounded quite OK, and Suzanne Vega's voice was destroyed."
The major dependent variable in listening tests is the recordings used for the evaluation. This is usually far more important than the associated equipment given that its quality is reasonable. It is even more important than listener training because the best trained listener can't hear an artifact that isn't there. When the artifact is stimulated by the recording, it is often right in the listener's face.
John, your typical forum blind-test (half-arsed you'd say) is ideal for demonstrating the effects of the expectation bias served up by the audio press and manufacturers. Vital's tests being a perfect example.
If definitive proof of something is the goal then using BS1116 protocols alone is not enough. You need to get a lot more sciencey, utilise many, many people and spend lots of money. So it'll not be happening much, if at all.
So it IS a "why bother" situation after all
If this is a response to my post just above, I don't think this is established either. But let's say you run the test I'd like to see and the results are the same as Meyer and Moran: Regardless of source, system, or the experience of the listeners, no difference is perceived in statistically significant numbers. Yes, that would probably mean that there is not much of a market for hi-res. The mainstream would not bother. But there doesn't need to be a mass market to make something a good model for a small business. HD Tracks sold upsampled 16.44.1 as hi-res for years before anyone noticed.
Tim
This reminds me of the studies of Dr. Bose. Statistically it was gleaned that the majority of people preferred (I am oversimplifying I know) rolled off sound and lots of indirect sound. Beats, another company that relied on extensive market research, goes the other way with boom and sizzle and they too are doing well. Throughout history you've got the UK sound, the East Coast Sound, the West Coast sound, all tailored to markets.
You don't consider yourself an audiophile but these starts with a B products wouldn't float your boat any more than it would an audiophile. We want steel strings sounding like steel strings, brass cymbals sounding like brass not sand paper, etc. etc.
These guys above, regardless of whether or not I agree or not, are zooming in on the little things that keep realism from happening. An artifact here, IM distortion there, clipping over there, the little things that add up. I think if there is one potentially sad error it would be to single out one thing and attribute everything to that. It just doesn't happen in complex systems thus the need for rigorous protocols.
We on the sidelines on the other hand are limited by circumstance, so we choose what we like the ordinary way. We try, we decide, we live with whatever consequences those decisions bring. It's not like we have a team of white coats and a mobile lab on call to help us choose a friggin' component.
You don't consider yourself an audiophile but these starts with a B products wouldn't float your boat any more than it would an audiophile. We want steel strings sounding like steel strings, brass cymbals sounding like brass not sand paper, etc. etc.
These guys above, regardless of whether or not I agree or not, are zooming in on the little things that keep realism from happening. An artifact here, IM distortion there, clipping over there, the little things that add up. I think if there is one potentially sad error it would be to single out one thing and attribute everything to that. It just doesn't happen in complex systems thus the need for rigorous protocols.
We on the sidelines on the other hand are limited by circumstance, so we choose what we like the ordinary way. We try, we decide, we live with whatever consequences those decisions bring. It's not like we have a team of white coats and a mobile lab on call to help us choose a friggin' component.
??? I thought this discussion was about the Meridian/Stuart paper to come at AES, not about my results. The paper is being presented to other professionals in the field, not to consumers. As such, it has to follow the standards of how we report new findings.
As an individual, you can consider the data to be moot. From the point of discussing audibility of distortions, it is not. If a human, trained or otherwise, hears distortions at a certain level, and our goal is to have a distortion-free system, we need to get rid of that distortion. The distortion is above the threshold of audibility.
As an individual, you can dial that however you like. You can choose to listen to your music on AM radio, FM Radio, Digital FM, MP3, AAC, CD or high res. Nothing about this discussion is about forcing anyone to make a choice there. But rather, showing proper data that differentiates these formats based on audibility. If we throw out such data, then someone who does care about being at a specific level of fidelity won't have the information to make that decision.
I think many, myself included, have repeatedly taken this position.
Thank you for apparently acknowledging the "real world" validity of this position.
Could be making progress.
Tim,
So much variety without any selection criteria ...
BTW, are you aware that BS 1116 asks for specific training of the listeners and replacing those who do not show capability of passing the controls, pre-screening and post-screening? I can not see how these requirements fit in your "common people" dream tests or ""Meyer & Moran under BS1116 guidelines" as you now call them.
Amir,
Even worse, some one has chosen the format for me, as more than 90% of the music I currently enjoy listening only exists in RedBook.
I bought what I consider one of the best players for CDs and you insist on spoiling my bias expectation of listening to the best format. Fortunately a few members are doing their best to show me my fears are not real.
It would be reminiscent of all of that if anyone was talking about preference testing. I don't think anyone is.
Tim
This discussion has been all over the place, and I'm not questioning your results.
I'm not talking about my view as an individual. I'm saying if these differences heard by trained listeners are inaudible to untrained listeners that data point -- "difference heard by trained listeners" -- is moot outside of the lab; there will be no market for a product, it isn't likely the research will have any impact outside of the lab. Which is, of course, why you would test audibility with a broader variety of listeners.
Well that completely misses the point.
I haven't suggested that we throw out any data. I'm suggesting that more needs to be gathered. And unless I missed a big discovery in this thread, we're not yet talking about reaching a specific level of fidelity. Has the source of the difference been identified?
Tim
Similar threads
