Computer Audio Asylum

Music servers and other computer based digital audio technologies.

Return to Computer Audio Asylum

Message Sort: Post Order or Asylum Reverse Threaded

Is this test valueable?

79.3.70.82
Posted on December 19, 2009 at 01:35:21
Bibo01
Audiophile

Posts: 648
Joined: December 18, 2008



The following test was done to compare the playback of two different media players - Samplitude and Foobar - on the same PC.
Do you think this test has any value or are there bottlenecks somewhere?

Digital signal was taken directly from spdif output.
Firtsly, it was acquired Samplitude's playback of a song - The Swedish jazz Kings "It's Right Here For You", song n.12 "melancholy Blues" 6,38 min. Secondly, the same song was acquired from Foobar playback (v.0.9.5.5).
Then, similar portions of the recording were cut at the same spot in order to play them back simoultaneously with opposite phases.
Conclusions: null test level at -94 db....therefore no difference between the two players!
On top of it, Samplitude played through ASIO, whereas Foobar through DS.

 

Hide full thread outline!
    ...
RE: Is this test valueable?, posted on December 19, 2009 at 03:48:17
fmak
Audiophile

Posts: 13158
Location: Kent
Joined: June 1, 2002
 Did they sound the same, and on what system?

 

RE: Is this test valueable?, posted on December 19, 2009 at 04:31:30
Roseval
Audiophile

Posts: 1875
Joined: March 31, 2008
 SPDIF and SPDIF is two.
There is the signal (the bits) and there is the timing (the sample rate)
Your test measures differences in signal and proves there is none.

There are claims that ‘electrical activity’ in side a PC affects sound quality.
Probably it causes small disturbances in the cycle time of the clock of the DAC (jitter).
Players can vary in the amount of system resources needed and therefore can generated more or less ‘electrical activity’ and therefore more or less jitter.

In principle, your test measures differences in bits, not in timing.
I think you can safely conclude that your test proves that both players+drivers produce the same signal.
The test doesn’t prove that both have exactly the same timing.
So you can’t conclude they sound the same.

I don’t think one can say there is a bottle neck but like all experiments, its scope is limited by design. In this case to the signal part. If we keep this scope in mind, it is perfectly valid (and interesting) to test differences between players/drivers on the signal part.

The Well Tempered Computer

 

RE: Is this test valueable?, posted on December 19, 2009 at 04:43:35
aljordan
Audiophile

Posts: 1252
Location: Southern Maine
Joined: November 4, 2003
 Just to be sure I follow you, are you saying that the spdif interface homogenizes the timing of the software players output such that any minute timing differences cant show up during the recording phase? Or are you saying that the timing differences between players is at a much smaller scale than the sampling rate of the recording, so timing differences couldn't show in this test?

Thanks,
Alan

 

RE: Is this test valueable?, posted on December 19, 2009 at 05:18:29
Roseval
Audiophile

Posts: 1875
Joined: March 31, 2008
 If you send a stream of bits over a bus, the receiver locks on the stream.
There will always be jitter as no clock is perfect.
However, as long as the receiver keeps its lock, it will read all the bits right regardless of the (very) small deviations in timing.
This is more or less how moving digital data works, you use the speed of the bus as a transport mechanism, not as a piece of information.

In case of SPDIF all above apply but there is more.
If you feed it to a DAC, the speed of the bus represents the sample rate.
In principle, if a bit arrives a bit sooner or later, now does matter as the DAC can only process the bit the moment it arrives.
So now we are not only using the speed of the bus as a transport mechanism but we are using its speed (and in the process, the unavoidable small variations) as information.
That is the funny thing about SPDIF, the signal is digital (the bits) the time (sample rate) is analogue (the clock).

In practice: if you record from the SPDIF, as long as the recorder can keep a lock on the signal it will read all the bits perfect. All the jitter won’t affect the result.
To phrase it different: as long as you stay in the digital domain, jitter won’t have any impact as you are using the rate of the bus as a method of transport only.
Moving any file (including audio) from one disk to another won’t have any impact on its contents even if the bus is high on jitter.
The moment you use the rate to time a DAC, any imperfection in the cycle to cycle time maps into sample rate jitter.

In my opinion: the test by Bibo01 (nice piece of work, I like this type of testing) stays fully in the digital domain. It is a perfect test to measure differences in the bits.
As a consequence, it won’t tell you anything about differences in timing (severe errors beside) as this part of the signal is not used as information but as transport mechanism only

The Well Tempered Computer

 

No, posted on December 19, 2009 at 05:19:21
Dynaudio_Rules
 The only real test that has any real value is listening with your ears. Don't let your eyes tell your brain what you are hearing.

 

RE: Thanks for the excellent explanation. nt, posted on December 19, 2009 at 05:32:08
aljordan
Audiophile

Posts: 1252
Location: Southern Maine
Joined: November 4, 2003
 Good explanation. Thanks for taking the time to write it up.

Alan

 

RE: Don't you want to try to learn why something sounds the way it does?, posted on December 19, 2009 at 05:41:42
aljordan
Audiophile

Posts: 1252
Location: Southern Maine
Joined: November 4, 2003




Feel the power!

 

RE: Is this test valueable?, posted on December 19, 2009 at 06:40:45
fmak
Audiophile

Posts: 13158
Location: Kent
Joined: June 1, 2002
 It is not just a matter of lock and arrival time; it is a matter of the way dacs are triggered by edge transitions of the signal. If there are irregularities such as imposed oscillations or reflections, then the whole thing can go haywire.

One of the first things one learns about gradient dependent phenomena is that it is very difficult to precisely control the exact trigger points.

This is why the dCS U clock has a dither function which adds a small amount of noise near the transition edge of the clock.

See the latest test in TAS of the Puccini/U Clock for explanation and review.

 

Probably, posted on December 19, 2009 at 06:54:57
Tony Lauck
Audiophile

Posts: 13629
Location: Vermont
Joined: November 12, 2007
 If the two digital outputs have the identical bits, then the two streams should null out to zero. Perhaps that is what you got, but with Soundforge, which I use, a zero bit stream would have been reported at "minus infinity dB". Look at the maximum and minimum values of the mix and display them as integer values if you can. If you have lined up the timing and chopped off any short glitches at the start and finish of the original tracks you should get a result with each and every sample zero. If not, then the two original files are different or something is wrong with your analysis (e.g. the mixing software itself isn't bit perfect).

So the test is definitely telling you something, possibly that one of the players isn't bit perfect or possibly about the way you are doing your measurement. Take a track (any non-silent track) and make a file copy to create an identical track. Then repeat your comparison test using these two tracks. This will provide verification that your comparison method is working correctly. If this nulls out, then take one of the files, go in and modify one sample by tweaking the least significant bit, and save this modified file. Then run your comparison again. This time it should show that they two files do not null out.





Tony Lauck

"Diversity is the law of nature; no two entities in this universe are uniform." - P.R. Sarkar

 

LOL...Cool, But, posted on December 19, 2009 at 07:34:05
Dynaudio_Rules
 LOL....sure I love to learn.

But knowledge also changes the way you perceive reality. So be careful of what you learn, you may wish in the end that you had remained ignorant of certain truths.

In this case learning will decrease may satisfaction with what I am hearing UNTIL certain numbers, measurements or other non-audible factors add up. Like Ummmmm, shall we say certain measurements of Jitter.

Try not to say that word too often or else the Ji..er monster will show up...

 

RE: Is this test valueable?, posted on December 19, 2009 at 07:46:10
Bibo01
Audiophile

Posts: 648
Joined: December 18, 2008
 The test was done using a M-Audio PCI board in a well set-up PC system.
It was run to proove that what comes out of spdif was the same regardless of player involved. So much so, that Samplitude was using ASIO and Foobar was using DS.

Is this test conclusive? Is it possible to claim that differences depend on the DAC involved only?

 

RE: Is this test valueable?, posted on December 19, 2009 at 07:47:41
Bibo01
Audiophile

Posts: 648
Joined: December 18, 2008
 The test was done using a M-Audio PCI board in a well set-up PC system.
It was run to proove that what comes out of spdif was the same regardless of player involved. So much so, that Samplitude was using ASIO and Foobar was using DS.

Is this test conclusive? Is it possible to claim that differences depend on the DAC involved only?

BTW, during a listening test, both tracks recorded sounded the same.

 

Is -94db enough? Do you hear a difference? Perhaps your ears are better than your measurements. nt, posted on December 19, 2009 at 08:59:33
Norm
Reviewer

Posts: 31024
Joined: September 6, 2000
 a

 

Interesting philosophy but not unusual, posted on December 19, 2009 at 09:01:12
mls-stl
 >> But knowledge also changes the way you perceive reality. So be careful
>> of what you learn, you may wish in the end that you had remained
>> ignorant of certain truths.

Humans are very protective of what they believe and it can be disconcerting to find a strongly held tenet doesn't hold water. As such it is not unusual to find people unwilling to do certain experiments lest their convictions be disturbed.

In audio, that is particularly true if an expensive or exclusive piece is being challenged by something more ordinary. (One also sees the opposite where the expensive item is simply dismissed as "audio jewelry.")

It takes a conscious effort to overcome our instinct to protect a current belief structure when we aren't sure of the outcome. However, that is the price of advancing knowledge.

Of course, what one does with that knowledge is a separate issue.

I may well decide that the $5,000 widget is better than the $500 one but conclude the difference is so small or ephemeral as to not be worth it. Or, I may decide the $500 one is just as good (or better) than the $5,000 one, but decide to keep the $5,000 on in my system. I may just like the way it looks or decide it has some other psychological benefit.

Or, my definition of "better" may not be the same as someone else's.

 

RE: Is -94db enough? Do you hear a difference? Perhaps your ears are better than your measurements. nt, posted on December 19, 2009 at 09:08:56
fmak
Audiophile

Posts: 13158
Location: Kent
Joined: June 1, 2002
 No, not for hirez

 

Nice explaination., posted on December 19, 2009 at 09:41:40
rick_m
Audiophile

Posts: 6230
Location: Oregon
Joined: August 11, 2005
 "One of the first things one learns about gradient dependent phenomena is that it is very difficult to precisely control the exact trigger points."

YES! Boy does one ever. However if I may expand it a little, it's actually rather easy to set the trigger (or as my old industry used to call it, the slicing) point. What's tough is knowing just where it belongs at any point in time.

Injecting noise doesn't increase the accuracy, but at least it decorrelates the error.

Good post,

Rick



 

Depends on the Recording., posted on December 19, 2009 at 13:01:38
Tony Lauck
Audiophile

Posts: 13629
Location: Vermont
Joined: November 12, 2007
 There are some recordings that I have where -94 dB would be easily audible with the volume set at realistic levels. To name the first one that come to mind: Bob Katz's 24 bit live recording of a space shuttle launch, where the "ten, nine, eight" of the count down peaks at -45 dBfs, while the launch itself peaks at -1 dBfs. To name the second one, any good recording of Mahler's 8th symphony. (There probably will not be a problem with a solo harpsichord recording.)

Unless deliberately changing the bits for a purpose, no credible software player should fail to be bit perfect. There is simply no reason for it, only lame excuses for incompetent "software engineering". No point in saying that -94 dB is (generally) inaudible. If the bits are wrong the software is junk.

Tony Lauck

"Diversity is the law of nature; no two entities in this universe are uniform." - P.R. Sarkar

 

Agree completely. Some more claims., posted on December 19, 2009 at 13:26:03
carcass93
Audiophile

Posts: 7181
Location: NJ
Joined: September 20, 2006
 I would prefer not to use that word, "claim", since it bears a lot of negative connotations, especially around these quarters - but here goes anyway:

Many of the optimization and linker settings in C++ compiler, that's used to compile code for media player, or relevant to the sound (for instance, output) plugin, drastically affect sound. I'm speaking from experience of using Intel C++ compiler to build Winamp ASIO plugin (Otachan). Not to mention huge difference utilization of SSE instructions makes, even if they're not used explicitly in the code, but compiler is directed to use them if possible.

The most interesting detail - practically ALL kinds of code optimization (there are many available in Intel compiler) ADVERSELY affect sound - to my ears, in my system, of course.

 

"If the bits are wrong the software is junk" - true. But if they are "right",..., posted on December 19, 2009 at 13:35:11
carcass93
Audiophile

Posts: 7181
Location: NJ
Joined: September 20, 2006
 ... it doesn't mean squat, not in the sense of "how it sounds" anyway.

That's what some here fail to understand, to my puzzlement - despite the fact that explanations are readily available (by Roseval below, for instance).

 

RE: "If the bits are wrong the software is junk" - true. But if they are "right",..., posted on December 19, 2009 at 13:48:42
Tony Lauck
Audiophile

Posts: 13629
Location: Vermont
Joined: November 12, 2007
 If the bits are right and the sound is still bad, then the problem can be fixed down stream (or at least substantially improved) by reclockers and other methods of isolation (possibly yet to be devised). While it may be easier and cheaper to tweak the transport, the fact is that if the bits are right and the sound isn't good then the problem is elsewhere in the system, e.g. in the DAC (especially its clock circuitry) or the analog amplification.

If the bits are wrong, then there is no point in proceeding further. It's hopeless.

Tony Lauck

"Diversity is the law of nature; no two entities in this universe are uniform." - P.R. Sarkar

 

No It Isn't....... Because............, posted on December 19, 2009 at 14:08:54
Todd Krieger
Audiophile

Posts: 37473
Location: SW United States
Joined: November 2, 2000
 The comparator samples the two signals digitally...... Any variations in jitter or noise during playback, which is where the audible differences take place, don't make it to the comparator.

The only thing the test shows is at least both players don't alter the original signal digitally.

Now if you were to somehow take the analog outputs and do this in the analog domain, I'm certain the difference signal (one signal subtracted from the other) *would* be audible.

 

RE: No It Isn't....... Because............, posted on December 19, 2009 at 15:15:37
Bibo01
Audiophile

Posts: 648
Joined: December 18, 2008
 The idea was to test 2 different players and demonstrate that they do not affect sound quality.
Do you agree with this?

 

RE: Is this test valueable?, posted on December 19, 2009 at 15:41:19
Bibo01
Audiophile

Posts: 648
Joined: December 18, 2008
 If I send the same signal with 2 different players to the same DAC and record its output, should I have 2 different files or the same?

If the 2 files are different, it means that the players were sending the same signal out but with different timing. Correct?

 

RE: No It Isn't....... Because............NO, posted on December 19, 2009 at 21:58:50
fmak
Audiophile

Posts: 13158
Location: Kent
Joined: June 1, 2002
 No, you need to listen thru the same chain and at the same level with the same settings; not asio v DS

 

RE: "If the bits are wrong the software is junk" - true. But if they are "right",..., posted on December 19, 2009 at 22:00:44
fmak
Audiophile

Posts: 13158
Location: Kent
Joined: June 1, 2002
 How is this possible if the same repro chain is used? What would you fix?

 

RE: No It Isn't....... Because............, posted on December 19, 2009 at 22:09:10
Todd Krieger
Audiophile

Posts: 37473
Location: SW United States
Joined: November 2, 2000
 "The idea was to test 2 different players and demonstrate that they do not affect sound quality.
"Do you agree with this?"

No I don't. The outputs being "cancelled out" are still in the digital domain- The effects of jitter and noise, which I think accounts for the sonic differences, are not taken into account.

 

RE: Is this test valueable?, posted on December 20, 2009 at 03:04:13
Roseval
Audiophile

Posts: 1875
Joined: March 31, 2008
 Recording the analogue out of the DAC should in principle tell us all.
This signal is what we are going to hear.
However it might be technically difficult.
I should do a null test first. Play same the song a couple of times using exactly the same settings.
This should yield the same files. If not the testing method is not valid.
I have no experiences with this type of testing but I can imagine that using different resolutions on the AD, say 16/44.1 or 24/96 might have an impact on the results.

The Well Tempered Computer

 

RE: No, posted on December 20, 2009 at 03:04:36
audioAl
Audiophile

Posts: 1462
Location: So. Texas
Joined: December 16, 2007
 That's a big Ten-Four!
Vista Ultimate 64 bit/e5300 Intel 45nm cpu/ASRock G41M-LE/Asus Xonar DS R 7.1/YamahaRX-V465 HT receiver/ Infinity RS1001 & Cambridge SoundWorks speakers/Yamaha YST-SW216 subwolf

 

RE: "If the bits are wrong the software is junk" - true. But if they are "right",..., posted on December 20, 2009 at 06:04:57
Tony Lauck
Audiophile

Posts: 13629
Location: Vermont
Joined: November 12, 2007
 If the bits are right and the sound is bad, then the software is interacting with the hardware in such a way as to cause the hardware to do more of something undesired (e.g. create more ripple on power or ground and hence create jitter in clocks and clock drivers). (This presumes that one of the software programs sounds better. If both are bad then it may just be that the hardware is bad, period.)

Thus, there are two approaches: improve the hardware to the point where it does a better job of implementing the software abstraction, or change the software so that it interacts in a different way that has less adverse consequences from a sonic standpoint. I would say the choice is purely pragmatic, according to one's skills, preferences and test equipment. However, in the real world, best results are likely to require a combination of both approaches: upgrading the hardware in various ways to make it perform (e.g. with less noise) and upgrading the software to minimize its footprint.

Both of these represent flawed approaches so long as it is necessary to evaluate changes by subjective listening to sound quality. Not because this isn't the purpose and end point of evaluation, but because this method of evaluation is so finicky and unreliable for evaluating minor tweaks. So the first point of the effort really ought to be to come up with some objective method of measuring the sound quality that is consistent and reliable. These measurements can then be used to guide the hardware/software decisions. The problem with this approach is that the result will be biased in favor of optimizing what can be measured at the expense of what can't be measured, so it is not guaranteed to produce a perfect result either. (I have in mind things like evaluating the analog jitter spectrum of various test signals, so that you measure timing jitter where it matters, at the actual point of conversion rather than elsewhere, e.g. at the oscillator output where it might be good but corrupted inside the DAC chip.)

Unfortunately, real progress with this approach will require deep knowledge of hardware and software coupled with expensive test equipment. Those who aren't prepared to make the necessary investments are best advised to procure high quality ready built gear (expensive) and get expert advice in its setup, or just relax and enjoy the music using more modest gear.



Tony Lauck

"Diversity is the law of nature; no two entities in this universe are uniform." - P.R. Sarkar

 

Squeezebox tests, posted on December 20, 2009 at 17:44:31
John Swenson
Audiophile

Posts: 2422
Location: No. California
Joined: October 13, 2002
 There are a group of people who have been doing some of these tests over in the squeezebox forums concerning claims of sonic differences between decoding PCM or flac encoded streams.

Several people have been using a difference test looking at analog outs and I have tried looking at clock spectrum. So far nobody has found a clear measured difference in any of these tests, but I and others can clearly hear the difference, others cannot hear the difference. I have even performed simple blind tests and have been able to distinguish the two.

It turns out that doing the analog diff tests is very difficult due to slow subtle clock changes mucking up things (you can either call it slow jitter or fast drift). Some indications are that this might be thermal, its very hard to tell. We are dealing with things WAY down in level. Equipment to test these things is very hard to come by and its difficult to setup proper tests.

My guess (and it IS a guess) is that we will eventually find that the human ear/nervous system is incredibly sensitive to certain things, it has sophisticated filters to zero in on these certain aspects of sound. Until we find out what these filters are and devise a way to mimic them in our instrumentation we will be wandering in the dark trying to measure these phenomena. I liken it trying to measure the bandwidth of a cellphone call by hooking a scope up to a coat hanger, the signal is there, but you need massive appropriate filtering to be able to isolate what you are after from all the noise that antenna is picking up.

At this point there is no obvious "smoking gun" pointing at a particular mechanism. It might be jitter, it might not. It might be other things such as noise on supply or ground planes, it might be noise injected onto the mains wiring, or maybe radiated EMI. Nobody knows at this point. What we have now is people coming up with hypothesis and experiments being performed to try and determine if the hypothesis is true. So far there have been no proved hypothesis. Either we haven't tested the right one, or the tests are not sensitive enough.

This is made much more difficult because we have no idea what the human perception system is responding to, so its difficult to try and measure something. Right now we are trying to look at a bunch of possible mechanisms and see if any of them change with different stream type. The expectation is that IF we can find something that actually changes with stream type then we MIGHT have found the mechanism, but there is no guarantee that it would be. So far we haven't even found any differences.

I HAVE found that the jitter inside the squeezebox gets worse when you plug in headphones, but still I can't measure any differences when changing stream types.

I think its very important that tests like these be done, even if they don't come up with significant conclusions. Eventually I think we WILL find out whats happening and figure out how to improve the hardware, but it might take some time.

I think that in the past people have discovered various mechanisms that adversely affect sound quality and these have been addressed to a large extent and what is left is much more subtle and difficult to measure. And because the higher order mechanisms have been addressed to a large extent, the human perception system can now "focus" on the much more subtle affects that were not obvious before.

I would like to point out that a lot of these things I have been talking about are not about bad and good sound but between great and WOW! That there are some things that make the sound even better than it already was, if you hadn't of heard the "better" you might believe that the previous was as good as it gets. Then the new software, cable etc comes along and you realize that it really can get better. That does NOT mean the previous sounded BAD. It just wasn't the best there is.

John S.



 

RE: No, posted on December 20, 2009 at 18:28:33
Phelonious Ponk
Audiophile

Posts: 992
Location: NC
Joined: September 3, 2007
 Which is, of course, why the only valid listening test is a blind listening test. Anything less is no test at all. It's just listening. Pleasant. The point, even. But indicative of nothing.

P

 

Page processed in 0.039 seconds.

[ Top Top of Page ] [ Contact Us ] [ Support/Wish List ] [ Copyright © 2025, Audio Asylum, all rights reserved ]
[ General ] [ Speakers ] [ Tubes ] [ Vinyl ] [ Digital ] [ Hi-Rez ] [ Video Asylum ] [ Cables ] [ Tweaks/DIY ] [ Music ] [ Films ]