Why 24/192 is a bad idea?

[arnyk]

You may remember I provided a lot of scientific papers highlighting the complexity of any testing including ABX that still suffers from AB order bias

ABX suffers from AB order bias?

Do tell!

 

[jkeny]

Tim just using yours as it is the last post on this subject in the thread.

Can I suggest we leave the JND, double-blind testing,etc to the many other threads we have on such discussions.
You may remember I provided a lot of scientific papers highlighting the complexity of any testing including ABX that still suffers from AB order bias, and other biases similar to more traditional JDN double blind testing methods used in science outside the audio community.
Not going to say anymore on that subject here as I feel it is not the right place and it has been discussed many times already, but I have provided a lot of information in the past forums on this, including how with perception differences becoming more difficult then the number of passes reduce....
None of this is really relevant to this thread anyway as the focus should be processes-functionality-algorithms-etc with the consideration from the studio all the way to the consumer DAC (or anywhere in between including aspect of chip-DAC embedded/external architecture)
Thanks
Orb

I agree ORB, DBX should not be discussed on this thread - I just had to comment on the testing procedures used (I had specifically asked to listen first, measure later) & how I saw it!

 

[Ethan Winer]

Unquestionably the best is DBT AB/X, when you don't even know what you're listening for, other than to identify whether X is A or B. Hard to expect anything then.

Has John proven that he can hear 0.2 dB ripple in an ABX test? Not that this can be proven across the Atlantic ocean. But has he even claimed he can hear that reliably?

--Ethan

 

[xiphmont]

It's amusing that Monty can state that he can't cheat by using ABX when in fact he's already cheated by looking at the measurements. Not truly blind now is it?

This is completely wrong, you're mocking me when you're wrong, and I'm pretty much done taking you seriously.

If you're wrong on ten points and right on two, the two don't make up for the ten. Why all the self-congratulation for failing to explain anything? You stopped testing when you got a result you didn't like. Debating the other points doesn't even matter when you're only willing to discuss the points where you think you're right.

 

[xiphmont]

ABX suffers from AB order bias?

It can, or at least, there's concern it can. That's the idea behind XXY tests.

 

[xiphmont]

I had specifically asked to listen first, measure later

I had flipped only three or four times when I realized something was wrong (the time shift you'd introduced), and so I looked at them. That was pretty much instant. And once I looked, I looked good and hard.

Nonetheless we're once again talking about people claiming 'immediate' and 'obvious' differences... and of course I can't hear them because I don't have good enough equipment or I'm just refusing to admit it. That's been the audiophile party line for 40 years. When a discrepancy appears in your own testing results (can't ABX in foobar, can tell the difference in jplay) you're uninterested in why. Yeah, I don't believe you anymore, and you're not exactly earning benefit of doubt at this point either.

Can a personal bias influence someone to not hear something as well as believe they hear something that's not there? Sure! But you're not interested in approaching that methodically or with any nuance at all, it seems to support you on its face so it's victory lap time!

You obviously want to try to convince us... so why not bother doing it right? I tell you what-- I'll add blind samples into the mix that absolutely should be audible and repeat the tests. Because I'm actually interested in knowing the right answer, not bragging rights.

 

[jkeny]

This is completely wrong, you're mocking me when you're wrong, and I'm pretty much done taking you seriously.

Here's our previous exchange on this :

I know at least half of the people who did this test & know that they did not analyse the files first. I specifically asked you or Arny to listen first & look later. A bit like Ethan's reply - he subsequently wanted knowledge about the files & I asked why? SO you guys really don't trust your ears - even though you proclaim blind testing for others, you won't abide by this yourself?

The file sizes make guessing easy in retrospect too.

Yes if you want to peek but as I said I & others trust our ears & don't tend to peek . This is hilarious, I ask you to do a blind test & you try to cheat

To which you later replied:

Did not! I used the ABX tool.

For the record, every human cheats. That's what big powerful brains are for. That's why the tools that don't let you cheat are useful.

No mention here of not peeking at measurements first - your statement is that the ABX tool stops all cheating

If you're wrong on ten points and right on two, the two don't make up for the ten. Why all the self-congratulation for failing to explain anything? You stopped testing when you got a result you didn't like. Debating the other points doesn't even matter when you're only willing to discuss the points where you think you're right.

Don't know what you're talking about, sorry??

 

[xiphmont]

I had no clue what was going on before I heard the tracks and that the differences were really obvious immediately, I replayed them a few times but really I only needed a first play to hear the differences - John either cheated royally or the differences are easily audible with a good system!

What was your ABX score, Clive?

 

[xiphmont]

Here;s our previosus exchange on this :To which you later replied:
No mention here of not peeking at measurements first - your statement is that the ABX tool stops all cheating

Spelling out the obvious: ABX and ABX-like tests stop any chance of rigging a positive result during the test. It removes the expectation bias in the given sample set during a test. It does not, and cannot, prevent rigging a null result.

It cannot remove all foreknowledge; you told us ahead of time what the samples were; that bias existed already. All you didn't tell us was which samples were which, and all that looking did was confirm: A) you weren't a liar and B) which samples were the original, 20 sample offset, and 60 sample offset. I had already assumed A, and the ABX tool removed the bias of B. So looking added no new bias.

Don't know what you're talking about, sorry??

You're not being methodical, so being right occasionally... or even mostly... on the singleton points does not prove your assertions.

 

[xiphmont]

Has John proven that he can hear 0.2 dB ripple in an ABX test? Not that this can be proven across the Atlantic ocean. But has he even claimed he can hear that reliably?

--Ethan

No, I had proffered it as a more likely explanation of audibility than the preecho.

And, worse, I'd screwed that up--- .2dB _fluctuations_ in amplitude are at the edge of detectable, not .2dB ripples in the FR. So I think I'm retracting my expectation that this is at the edge of audible for particularly skilled listeners. That mistake is primarily what I'd based it on.

BTW, Ethan it is +/-.1dB not .1dB P-P? OK. I was sure I'd read the latter.

 

[jkeny]

Spelling out the obvious: ABX and ABX-like tests stop any chance of rigging a positive result during the test. It removes the expectation bias in the given sample set during a test. It does not, and cannot, prevent rigging a null result.

Nobody's talking about rigging anything - we're talking about psychology & the traps of biasing. Sighted tests can be a source of bias, as can expectation bias (whether due to pre-knowledge of measurements or otherwise). The fact that you would do measurements before listening does say a lot about your approach. I specifically asked that listening be done first!! Measurements gave you a lot more information about the files than I provided so it doesn't equate to the pre-knowledge you already gleaned from me! You do remember the example I gave of a well known audio manufacturer who ran a test saying that it was a test of mains cables in audio. One of the participants was a staunch anti-cable guy & he found no differences in the tests whereas everybody else 100% heard differences. What the tester actually had done was reverse the polarity of the speaker cable during the tests. So you can see how strong expectation bias can be! I doubt it has anything to do with rigging the test.

It cannot remove all foreknowledge; you told us ahead of time what the samples were; that bias existed already. All you didn't tell us was which samples were which, and all that looking did was confirm: A) you weren't a liar and B) which samples were the original, 20 sample offset, and 60 sample offset. I had already assumed A, and the ABX tool removed the bias of B. So looking added no new bias.

Yes, it added information that was very relevant to what could/should be audible & this is a biasing factor. Unless expectation bias has been accounted for & eliminated then why would null results be of value in a DBT?

You're not being methodical, so being right occasionally... or even mostly... on the singleton points does not prove your assertions.

Still don't know where you are going with this without examples but let's not go down that road!

 

[MarinJim]

If these discussions happened during the Reformation and/or Spanish Inquisition , heads would roll!

 

[xiphmont]

Nobody's talking about rigging anything - we're talking about psychology & the traps of biasing.

I'm not only talking about consciously cheating (in this case, I'm not talking about deliberate cheating at all). I'm also talking about subconscious biases rigging the test.

Sighted tests can be a source of bias, as can expectation bias (whether due to pre-knowledge of measurements or otherwise).

In this case you're complaining that the sighted test resulted in a subconscious expectation of a null result. However, ABX tests cannot guard against a null result anyway. ABX tests are for guarding against spurious positive results.

The fact that you would do measurements before listening does say a lot about your approach. I specifically asked that listening be done first!!

You've conveniently ignored now the several times where I said I did listen first-- but the time shift made testing impossible until I determined what it was.

Measurements gave you a lot more information about the files than I provided so it doesn't equate to the pre-knowledge you already gleaned from me!

OK, you're upset, there's an easy way to fix that. I will happily take the test again. This time please remember to trim the padding.

You do remember the example I gave of a well known audio manufacturer who ran a test saying that it was a test of mains cables in audio. One of the participants was a staunch anti-cable guy & he found no differences in the tests whereas everybody else 100% heard differences. What the tester actually had done was reverse the polarity of the speaker cable during the tests. So you can see how strong expectation bias can be! I doubt it has anything to do with rigging the test.

A) ABX tests do not prevent 'rigging' a null result
B) The positive ABX result from the other participants was far and away sufficient to provide proof.

This is why I'm saying that your thinking [on these testing matters] is not methodical or logical. You can't just mime the process and say 'see?'

Yes, it added information that was very relevant to what could/should be audible & this is a biasing factor. Unless expectation bias has been accounted for & eliminated then why would null results be of value in a DBT?

They're valuable only in the relative absence of positive results, and positive results must be checked carefully for error, as most audible testing errors cause positive results, not negative (eg, the positive results shows there was a difference heard-- then it's vital to verify what the difference was). Eg, the example above.. when I listened to your samples, ABX was trivial (a real positive result!) because of the inadvertent time shift you'd introduced into the test samples (...but the difference wasn't what you were testing for).

 

[jkeny]

If these discussions happened during the Reformation, heads would roll!

 

[jkeny]

Monty,
I don't want to keep quoting & replying to each point so I'll do it this way:
- what you are saying effectively is that BT favours null results. Explains the preponderance of null results in DBTs
- I'm not missing the point about the cable test & the outliers. But this is not a group test, it's a singleton test - the only ones posting about their results on here are you, Ethan, Arny & Clive & me. There's not enough numbers to use statistical averaging as you are suggesting.
- you had the ability to block the sound of time shift jump when switching between tracks so no need to go looking & measuring files. Sorry if I'm mistaken in wrongly labelling you as one who goes for the measurements first but I have noticed this as a trend with some. It was even stated in a post & I replied I would prefer listen first & measure later.
- glad you are genuinely interested in this - I'm not running victory laps & I'm not upset but I will say again, you won't hear the difference with Foobar or indeed a Sonos player. Sorry if you don't like that fact but it's true!! So unless you have a PC & can download Jplay, & have a reasonable replay system following that, I'm afraid you will not to hear any differences.
- as I said if you want to get into a discussion about Jplay Vs Foobar or why bit perfect output can sound different, I would direct you to the many threads about it here. I just didn't want to go over that well trodden ground in this thread. It's nothing to do with lack of logic!

- The file size is not immediately obvious on PC, maybe it is on Mac?

 

[xiphmont]

jkeny,
Yeah, the quote mechanism is clumsy. I think I'll do it this way too.

-- It's not that DBTs are 'skewed' toward null results, it's a fundamental aspect of the logic. It's a tool for finding evidence. A preponderance of null results eventually indicates what you're looking for isn't there. That preponderance can be overturned by a small set of verifiable positive results.
-- Yes, this is a signleton test, but you asserted the matter 'settled' last night [perhaps jokingly]. It's nothing of the sort, especially when even this short run of tests presented an inconsistency you were happy to ignore.
-- I had the ability to block the time shift once I knew what it was. The phase plot told me it was a time shift. I possibly could have intuited without, but I was already in debugging mode.
-- It would not have been 'cheating' if I'd still gotten a positive result, and it is not cheating when I get a negative result. You cannot test only with blind tests; blind tests are used to confirm sighted tests. This is and always has been standard practice (the sighted tests _increase_ positive results; the blind test confirms the result is true).
-- I'm a linux person, so I'll not be using jplay (and can't even check it). However, I wrote the kernel driver I'm using (including parts of the USB stack), I wrote the playback software, the OS translation layer and the ABX tool I'm using (Squishyball). I'm confident in the software's function. Whether the confidence is well placed or not is matter for a different argument
-- I'm a command line person primarily. File size can be pretty obvious in a shell.

 

[jkeny]

Monty,
- I never said the matter was settled jokingly or otherwise, AFAIR.
- The "inconsistency" you talk about is a red herring. As a software developer, you might be interested in the work that the Jplay guys have done - check out a thread on CompterAudiophile about what they have done under Windows - I know Linux is a whole different ballgame, more logical & more efficient an OS. All I can say for sure is that Jplay is superior sounding to any other playback software that I have heard on a PC & is needed for this test.
- Ah, Linux. Yes, it should be a favourable environment for audio but I don't know, I haven't done the test under Linux & nobody else has either. Yes file sizes would be very obvious if ls of files!

 

[xiphmont]

jkeny,
-- the digital bits jplay is sending to the hardware are different from the digital bits foobar is sending? Is it doing additional signal processing?
-- Linux is what you make it. I hope that statement is sufficiently vague, damning, and inspiring at the same time. It is the Bill Belichick of operating systems.

 

[jkeny]

This requires quoting

jkeny,
-- the digital bits jplay is sending to the hardware are different from the digital bits foobar is sending? Is it doing additional signal processing?

Yes same bits, i.e bit-perfect therefore no manipulation of bits. If we get into this area it will go all per-shaped so either a different thread or have a read of the existing bits are bits threads already on this forum.

-- Linux is what you make it. I hope that statement is sufficiently vague, damning, and inspiring at the same time. It is the Bill Belichick of operating systems.

 

[xiphmont]

This requires quoting
Yes same bits, i.e bit-perfect therefore no manipulation of bits. If we get into this area it will go all per-shaped

I'm sensing that and backing slowly away with my hands where you can see them. That will be a joust for a different day.