DBT testing is it accurate?

[steve]

Hi Gents,

I just finished an article on subjective AB/ABX testing, is it accurate? Here is the link.

http://www.sasaudiolabs.com/theory14a.htm

Take care.
Steve

[_Scotty_]

I think you have covered the bases pretty well with your article. My problem with ABX testing lies with the objective that the testing usually revolves around,that the differences audiophiles claim to hear between, fill in the blank here, can be proven not to exist when using this test methodology. I could care less what someone thinks about the reality of sonic differences between pieces of audio gear or the electronic components they constructed from. I have demonstrated to my own satisfaction via single blind testing that subtle differences I have heard between various pieces of audio gear are real and that even pieces of wire six inches long between a RCA jack and the selector switch in a preamp can have a profound impact on how it sounds.
 You might mention that AB/ABX testing usually provides more insight into the world view or psychology of the test designers than the DUT and that occasionally it identifies a truly GOLDEN EARED listener who can consistently identify X in spite of the flawed test conditions and make monkeys out of testers with the "there is no difference" axe to grind.
In your "Power Supplies and Capacitors - Their effect on audio:" article I would have stressed that the goal of a power supply for audio amplifiers is to have as low a dynamic impedance as possible so that it's rise time is faster that that of the circuit it supplies power to. It should also not ring when asked to supply the bursts of current required for the reproduction of music with high frequency content. You might also touch on the advantages of having a rectifier bridge with as large a current rating as is possible and the positive effects it has on lowering the dynamic impedance.
As always I must question the wisdom of any manufacturer who supplies factual information to potential customers as this may tend to confuse them with the truth and lead to fewer sales. 
Scotty

[Carlman]

I tend to do highly objective and blind testing on my own when I listen for differences in an A/B situation.  I can easily forget which input has which device and just ask that the source be changed from whoever has the remote.  For splitting-hairs differences, it's nice to do it that way... because sometimes those minor differences eat at you after you live with it for a while.

There are so many factors to listening tests... Many are psychological.  I know what's important to me so I know how to do my own testing.  Which is also why I don't find reviewing gear all that helpful, it's just so personally subjective.   (Plus I don't think what I write is all that helpful after reading others' reviews.)

In any case, I have no opinion of debating test types.  That seems like a different world to me.  Going down this road is more scientific and analytical and I could see how some would enjoy it.  But actually debating which listening method is best to analyze sounds?  I think that would drive me crazy.

-C

[spudco]

As a research psychologist, I can tell you that DBT works. 

I can also tell you that preceived differences are just as real to a listener as actual differences.  The human mind is great at reinforcing our belief systems with "actual" experiences.  This is the essence of the placiebo effect (which is very real).

One problem with DBT for audio gear is that it is difficult to conduct in an environment that a subject is "used to".  Therefore, very subtle effects can be missed.  Often, simply different or louder can be precieved as better - especially if you are in an unfamiliar listening environment.

Also, a good system is one you enjoy.  This may of may not float someone else's boat.  After all, we all have different preferences, auditory perceptions and unique measurement devices (ears/brains/bone structures/...).

[bpape]

Sorry, I still fail to see the relevance.  SBT is more valid IMO. 

I don't need to know which is A or which is B.  However, if I can consistently identify that I prefer A over B, what difference does it make if I can identify X as A or B?

Bryan

[steve]

Thanks for your input gentlemen. I appreciate it. Yes, DBT testing can and does work in some situations, but when it comes to audio perception habituation, as well as other problems, do occur. Any repetition of a selection, whether sited or not, will cause habituation to rear its ugly head.

It is hard to find partial results such at the beginning, middle and conclusion of such testing. But once in a while such data is given. Here is one example (hopefully this article will stay online).

http://www.bostonaudiosociety.org/bas_speaker/abx_testing.htm

Notice the comment
"The tests we took showed clear audibility to a confidence level well over 95% for the first three tests, and null results for the last three."

This is a good example of habituation influencing audio perception and the resultant outcome. (For those who have not read the article, I also cite other problems with AB/ABX testing that need to be accounted for.)

As always I must question the wisdom of any manufacturer who supplies factual information to potential customers as this may tend to confuse them with the truth and lead to fewer sales. 

You are probably right Scotty. Maybe I should not have posted the article.

[richidoo]

Any repetition of a selection, whether sited or not, will cause habituation to rear its ugly head.

A young child will deliberately sing the same song 100 times in a day with no sign of fatigue. Audiophiles consumers are middle aged professional men, they can handle hearing a song repeated 5 times without becoming fatigued or disoriented.

Notice the comment
"The tests we took showed clear audibility to a confidence level well over 95% for the first three tests, and null results for the last three."

This is a good example of habituation influencing audio perception and the resultant outcome. (For those who have not read the article, I also cite other problems with AB/ABX testing that need to be accounted for.)

It's just as likely a good example of the effect of beer at the BAS G2G where this 'test' took place.

To me it is all academic. Real people compare everything AB with their eyes open for everything they buy, because it is common sense. Many factors affect the choice audio gear, so limiting the testing to sound is of academic value only.

[steve]

Any repetition of a selection, whether sited or not, will cause habituation to rear its ugly head.

A young child will deliberately sing the same song 100 times in a day with no sign of fatigue. Audiophiles consumers are middle aged professional men, they can handle hearing a song repeated 5 times without becoming fatigued or disoriented.

Notice the comment
"The tests we took showed clear audibility to a confidence level well over 95% for the first three tests, and null results for the last three."

This is a good example of habituation influencing audio perception and the resultant outcome. (For those who have not read the article, I also cite other problems with AB/ABX testing that need to be accounted for.)

It's just as likely a good example of the effect of beer at the BAS G2G where this 'test' took place.

To me it is all academic. Real people compare everything AB with their eyes open for everything they buy, because it is common sense. Many factors affect the choice audio gear, so limiting the testing to sound is of academic value only.

I think we both crossed wires Rich. We all compare different components by listening, whether at the store or elsewhere. No problem is done a couple of times. However, as the PHD insinuates, as one listens over and over and over again to the same selection, habituation does come into play. This is quite different then when a child sings the same song over and over as the child is not discerning sonic differences.

It is an unconscience type of thing, as mentioned below.

Expedia has a good explanation of non verbal habituation as well.

http://en.wikipedia.org/wiki/Habituation

"Habituation need not be conscious - for example, a short time after a human dresses in clothing, the stimulus clothing creates disappears from our nervous systems and we become unaware of it."

Hope this helps Rich.

[richidoo]

Yes thanks Steve... It is a weird topic, so removed from the meat and potatos of the hobby. But interesting.

[steve]

Yes thanks Steve... It is a weird topic, so removed from the meat and potatos of the hobby. But interesting.

Hi Rich,

I think I was not as clear as I probably could have been Rich. Also medical info seems rarely mentioned when discussing audio testing. Hopefully it gets out there so the public can be better informed when making comparisons in the store or at home. I usually do limited comparisons over many days, weeks or months (depending upon the situation) to make sure I do not get caught in the habituation trap (and other variables).

Take care.

[steve]

Sorry, I still fail to see the relevance.  SBT is more valid IMO. 

I don't need to know which is A or which is B.  However, if I can consistently identify that I prefer A over B, what difference does it make if I can identify X as A or B?

Bryan

I agree Bryan, what difference does it make.
I meant the article for those who are undecided whether testing is accurate or not.