 
|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
|
|
Audio Asylum Thread Printer Get a view of an entire thread on one page |
For Sale Ads |
209.130.140.130
OK, this should be simple. Please do not reply to this post until you have the results of your test ready.Here's what to do:
1) Find a music track that you know well that is already in FLAC or WAV format. Convert it to the other format so that you have a file with the same song in both formats. You can use Foobar2000 to convert a track to another format by selecting it, right clicking it, and selecting the "Convert To..." option in the Convert menu.
2) Open up Foobar2000 and add both tracks to your playlist (preferably next to each other).
3) Select them both (hold down shift to select the second one), right click on one of them, go to the Utils menu, and select the "ABX Two Tracks..." option. Do not select the ReplayGain or DSP options.
4) Use the "Play A", "Play B", "Play X", and "Play Y" buttons to listen to the selections. Once you've determined which match, make your choice using the "X is A, Y is B" or "Y is A, X is B" buttons. Hit the "Next trial" button when you are ready to try again.
5) Repeat step 4 at least 10 times, preferably 20 or 30 times if possible.
6) Post here with your results! It would be helpful to know what equipment and music you were using for the test. If you can't hear any difference, please post that result as well. The posts can be as long as you guys want to make them, but put the relevant information right at the top, please.
Edits: 06/01/07 06/01/07 06/01/07Follow Ups:
Downloaded both files from Linn's site:
http://www.linnrecords.com/linn-downloads-testfiles.aspx
ABX'd em' on a very nice rig consisting of an authentic Intel G965 Series main-board with only one card installed in the first PCI slot: a new ASUS Xonar PCI audio, (supposedly the current "Best" soundcard). The main-board is hosting one gigabyte of DDR RAM in the form of a single chip and a 3.6 Gig Core 2 Duo CPU with large, slow-speed fan. The motherboard is running BIOS Rev. 0806.
I’m running Windows Vista Business Edition with all visual tweaks turned off, all unnecessary services disabled and all updates / patches installed. The Operating System is a fresh load onto a new Seagate 160 Gigabyte hard disk drive operating at 7200 RPM. Aside from Audacity & Foobar 2000, there are [no] other applications installed on the drives.
The PC is sporting an [external power supply] on a separate line all it’s own from the panel. Between this effort and the uncanny noise floor of the shielded Xonar audio card, I have electronic noise to the absolute minimum possible. The PC features two of the aforementioned SATA hard disk drives, the second of which houses the data files in question. Both drives have been quadruple disk defragmented and the PC has been clean rebooted after each defrag, including the final one.
I'm a long-time audiophile who is beyond anal about music recording and playback. I am a practicing musician on the acoustic guitar as is my spouce. I have my ears checked about every five years, and can say that at this moment, I have excellent hearing: dropping off beyond 17 KHz - pretty much par for the course at 40 years of age. I build PCs for a living, and have done so professionally for nearly a decade and a half. I am a Microsoft certified Software Technician as well as a Computer Trade industry Association certified hardware technician.
I am using Foobar 2000 version 0.9.4.4 and have left all setting at factory default values.
I have ABX’d the hell out of these files using a set of Audio Technica ATH-M40fs headphones, in a dead-quiet room… and I here no difference…
I’ve also tried to ABX this same set of files, with the same version of Foobar 2000, on a new Hot Rod Toshiba G35 Qosmio, multi-media specific laptop, running XP Media Center Edition, on a built-in high resolution audio chipset. With an older set of AKG ‘K Series’ headphones: the results… no difference is to be discerned between these two audio files.
In summary, I feel that more folks should take a bit (or as much time as they wish, days perhaps), of time and conduct unsighted ABX evaluations like this one for themselves. If more people would do so, silly discussions such as this thread would not exist.
Andrew D.
www.cdnav.com
Well, actually ~ try ripping a CD into MP3 at 320KB/s with Audacity using the latest version of LAME and ABX it vs. the same WAV file [for yourself]. Don't attempt to discredit or belittle me: try an honest ABX and you'll hear what I mean: 320Kb/s, if done with the latest codec’s is 'transparent'.
Andrew D.
Just read on your homepage (the link you posted) that you couldn't hear a difference between .MP3 and .WAV either!?!? ;)
Now you need to tell us, how you seriously expected to hear a difference between flac and wav.
Incidentally,
if one uses the foobar ABX plugin to conduct the test, one is not comparing flac decoded on the fly to wav. The foobar plugins saves the wav and the wav obtained from "uncompressing" the flac file and plays either of the two (or selections from). So it does not play from your flac file on the fly.
To check that the wav file obtained from the flac one is identical to the original one all one needs is a diff (or a binary compare) utility. Nobody is doubting that.
If you want to do a meaningful ABX use a friend/significant other (but if you cannot hear the difference between wav and MP3 why bother ...).
Best
Giulio
NT
This is not a valid test because it writes the two files to a temporary folder and they're both WAV files. I don't think anyone has disputed that when decompressing a FLAC file to a WAV file, then that file is bit-identical to the original WAV file. That's easy to verify with EAC's compare WAV feature.
I think the suggested possibility of them "sounding" different is strictly based on the playback and the possibility that during playback of FLAC from foobar there may be a different signal path (DSP, CPU usage) than a WAV file. Thus you can only compare when playing the original FLAC file and not by using foobar's ABX.
I'm going to keep my opinion, of whether they "sound" different, to myself, however I will say that if I play a FLAC file from foobar then I have to increase my ASIO buffer or else I get stuttering. So obviously it's chewing up more resources or CPU usage or something as compared to just playing WAVs.
The problem is, it's not possible in any music player to play back a FLAC file without first decoding it to WAV. That can be done by creating a temporary copy on the hard disk, or it can be done by creating a temporary copy in memory, but there is always a WAV copy made by the player at some point.
So while it's possible that the increased processing load caused by decoding FLAC in real time could mess up your playback, it's not really about differences between FLAC and WAV per se - it's just that the computer is doing something else while it plays music and that could affect what you hear. If that does happen, I suggest you update that 386 you're using :-) (FLAC decoding is really not much of a load - the standard was designed that way).
Are in a thread I started at HydrogenAudio after complaining about how Foobar's ABX test converts both files to WAV:
http://www.hydrogenaudio.org/forums/index.php?showtopic=55319
If you want to try another non-ABX but possibly meaningful test - try playing back a Wav or Flac file with and without your antivirus/firewall turned on.
OK, it seems it is converting both files to WAV. Great...I will go complain at HydrogenAudio now.
Oh well, I guess we get to keep living with those who have the seemingly ridiculous notion that FLAC playback sounds different than WAV until I find some other comparison mechanism. :(
*** Oh well, I guess we get to keep living with those who have the seemingly ridiculous notion that FLAC playback sounds different than WAV until I find some other comparison mechanism. :( ***
Here I was thinking it's great that someone has an open mind, but this line now makes me suspect you are only interested in debunking theories you don't believe in. Sigh :-(
Anyway, my thoughts on this are: suppose that FLAC playback sounds different from WAV due to changes in jitter, EMI/RFI, whatever. Note: I am not claiming this is 100% true, but it seems plausible.
Then any attempt to measure this difference using software based ABX switching will affect jitter, EMI/RFI etc. So in other words you are changing what you are trying to measure.
Maybe better to have two identical machines side by side - one playing FLAC, the other playing WAV, and have a friend randomly switch between the two. But then you are likely to encounter differences between the machines, which may be greater than the difference between FLAC and WAV playback.
I don't think you will be able to easily prove or disprove the hypothesis. So relax and enjoy the music.
I was the one who said that, on the Linn site, short previews of 24/88.2 WMA appeared to sound different , and better than 24/8.2 FLAC. This is with the new Gilcrist album where one can try different levels of resolution before buying.
This then exploded into a FLAC v WAV argument whereas anyone could have gone to the website and hear for themself. If it was my imagination, they were invited to say so. The fact that no one seems to have done so, seems to support what you have said.
I actually compared the downloaded the WMA 24/88.2 files to decompressed WAV files using Audition 1.5 and compared the two. The WAV file has better treble although not done listening in my hifi system yet; just through my well known Terratec 2496 card and decent amplification system thru my download computer.
Can I invite people to try on the Linn site?
Actually I started this thread before reading your post over on the digital forum (otherwise I probably would have posted it there, and it would have been the FLAC vs. WMA test). :-)
I tried downloading the test files from the Linn site for comparison and it *seemed* like they sounded a bit different to me, but without some type of ABX test there is no way for me to quantify exactly how different they are to my ears on my equipment (if at all).
My personal position on ABX testing is a bit mixed. On the one hand, I have incorrectly concluded that two different items sounded different (in this case it was cables), only to come to the realization during an extended A/B test that they sounded exactly the same (I had to switch back and forth at least 50 times on a song that I was SURE I could tell the difference on before formalizing an A/B test setup with a special switchbox set up for blind testing).
But I've also been burned by ABX testing. For example, during one test I couldn't tell the difference between two settings in Foobar2000 despite a lengthy comparison. But I ended up keeping the new setting (a certain resampling DSP in Foobar2000) without thinking about the test much for a couple of weeks, until one day I heard something in a familiar song that sounded different for some reason. I then started up the A/B test again using that part of that song and, voila, I could clearly hear the difference.
And this is the thing the people whose posts I read at HydrogenAudio often fail to consider: an ABX test does not PROVE that the two test subjects are the same, it only shows that, for one particular song and one particular set of ears using one particular set of equipment in one particular location, there was or was not an obvious difference. I believe the best test is one like I was trying to set up here, where numerous people use the same methodology to test the same two hardware, software, or data format options, after which we might begin to draw some conclusions if enough people respond with valid test data.
Foobar sounds different with different ASIO plugins, buffer settings etc etc. It is difficult to use it as a neutral tools for comparisons. also ABX tests are just what they are; short term statistical impressions - much longer long term listening is to me more important.
If two compressed versions of a 24/88.2 track sound different, and you seem to confirm this, it is more profitable to find out why, rather thamn try to quantitfy that difference statistically.
My own short experiences with FLAC is that the the hirez files I possess (including download on hirez tape transfer) do not sound good even played back on a high end computer/dac system and imagination or not, I prefer uncompressed WAV or AIFF, even though the files are massive. With HD cheap, who cares?
My other comment is that. to hear clear differences, one needs to be very creful with PC settings and the quality of the replay chain. The higher the quality, the easier it is to hear differences.
You bring up a really good point which a lot of people fail to appreciate: our ability to spot differences in ABX testing depends on how well we have trained our ears to recognise the specific differences.
For example, because I play around with resampling a lot, I can easily recognise the sort of artefacts produced by a resampling algorithm. I can also hear some types of distortion (like clipping for example) more than perhaps an average person, because I do a lot of recording.
Some of the guys on HydrogenAudio (guruboolez for example) have obviously trained their ears very well to be able to spot tell tale artefacts from various lossy compression schemes. Even I, with little or no training, can spot the warbliness that you get in low bitrate MP3, the slight ringing in Vorbis, and high frequency attenuation plus noise from WMA.
The problem with ABX tests involving a lot of people is that most of them won't be "trained" to spot differences, so the likely result is that on average no differences are reported.
There's been a few "studies" over the past years that have tried to show for example differences between amplifiers are not audible, or there's no difference between SA-CD and DVD-Audio, or no difference between SA-CD and CD etc. and they all suffer from the same flaw.
Even the student thesis from a year or so ago that tried to show there is no difference between DSD and PCM failed to appreciate that for a small number of people in the test (I think it was only 1-2 people) - they *were* able to tell the difference in a statistically meaningful way. However, instead of exploring *why* these people could hear a difference and most couldn't, the study decided to ignore them and suggest differences between DSD and PCM are not significant.
I think you are confused about the ABX test feature in Foobar2000. It does not convert both files to WAV; that would go against everything this test code was designed for in the first place!The idea that FLAC decoding consumes so many resources that it makes your computer stutter leads me to conclude that you have a seriously messed up computer, or some kind of audio configuration that is FUBAR. I base this on the fact that I can listen to FLAC files without any problem whatsoever while playing games like World of Warcraft (which consumes more CPU resources than 50 Foobar2000s playing FLAC files, by the way).
Finally, the whole point of the test in the first place is to remove the "opinion" about whether one lossless music format sounds better than the other with actual facts. I understand that those who have heavily voiced an opinion in the past here that is now disputed by the facts may be hesitant to post their results here...but we really can't make any progress on our mutual quest to make music coming from a computer sound as good as it can unless everyone is ready to stop sharing their opinions and start sharing some unbiased listening test results.
As far as I know, all Flac plugins in all the music players use essentially the same source (from the FLAC developers kit) and all convert the Flac to Wav before playing it. So you are always hearing a Wav file.
Hey Scrith,
Ed may be running programs or plugins that do consume a ton of resources, like upsamplers, crossovers, and room correction, that combined, could be very taxing on a system and cause stutters in some cases.
I for one found it very difficult to run both crossovers and upsamplers (forget about secret rabbit) at the same time on my dedicated 1ghz machine.
Ah, that could be true. But his post lead me to believe that it was the difference in CPU resources between WAV and FLAC decoding, which are so small as to be almost non-existent (e.g. I get less than 1% CPU usage when playing FLAC files in Foobar2000).
If you search on the general forum under "Foobar ABX" there is a thread that talks about how the ABX feature in Foobar makes them sound less transparent and louder. Perhaps what you mention is what was being reffered to.
Where did you see that it turns them into wavs.?
OK first let me just point out that I'm using foobar v0.8.3 - just so we don't have confusion about versions.
In foobar - go to "Preferences" - then under "Components" select ABX Comparator. At the right you'll see a space to indicate a path for the temporary folder (C:\). Then before you select your two files to ABX, open up the folder in windows explorer (that you indicated for the temporary folder) and size/place that window side by side foobar (so the foobar window is not blocking it so you can see what happens). Now, in foobar, select your WAV & FLAC file and right-click and select ABX and then you'll see a progress bar while it "prepares" the files. After you see the ABX window pop-up you'll see over in your explorer window that it has written two files named "abx_temp_xxxxxxxxxx.wav" (where the x represent a random 10 digit number). Viola!
Same on 9.4.2 also.
I finally got a chance to try it myself. I gave up after 5 attempts...I can't tell them apart.
I tested using a Benchmark DAC1 USB, an SFT Dynamight balanced headphone amp, and Qualia 010 headphones with a custom balanced headphone cable. I was testing using various parts of track 4 ("Time") from an AccurateRip (via EAC) of Mobile Fidelity Sounds Labs CD of Pink Floyd's Dark Side of the Moon.
I am questioning headphone setups. With these there is no way to judge the correct room information, which IMO will be the biggest difference between both formats during playback.
My headphones are the most detailed listening devices I have ever heard. I also have some nice Dynaudio Contour S 1.4 speakers, which sound wonderful for general listening. But I've never heard anything (even $50K+ speakers) that can touch the crazy detail I hear from these (admittedly expensive and now impossible to purchase) headphones. And the amp (also impossible to purchase) takes them to another level altogether. Trust me, this headphone setup I have is among the best on this planet. :)
> With these [headphones] there is no way to judge the correct room information...
Not sure I follow your logic. If the recording contains hall or studio information from the original recording session (reflections, echoes, decay info) it will be present in the headphones as well as the speakers. (In fact, good binaural recordings are very realistic with headphones, the catch being you can't turn your head while listening without screwing up the cues.)
If you are talking about room information that is added by the room the stereo is playing in, what does that have to do with the info on the recording? Those reflections, decay and echoes will be added regardless of playback format.
Could you clarify?
I am NOT questioning the level of detail a headphone is able to deliver.
It is just lacking the ability to present the room or stage-image information in front of you.
I got music playing in the middle of my head when listening to headphone music. (I got Etymotics ER4). If I use some binaural pluggins, the original sound gets messed up.
Bass in the lower ends is not really the strongest disciplin for headphones either. (at least my ER4s give up on that.)
And lack of low-level bass might also be another issue popping up when comparing flac and wav.
Cheers
> I am NOT questioning the level of detail a headphone is able to deliver.
> It is just lacking the ability to present the room or stage-image information in front of you.
> I got music playing in the middle of my head when listening to headphone music.
> (I got Etymotics ER4). If I use some binaural pluggins, the original sound gets messed up.
> Bass in the lower ends is not really the strongest disciplin for headphones either.
> (at least my ER4s give up on that.) And lack of low-level bass might also be another
> issue popping up when comparing flac and wav.
Headphones are certainly a "different" experience than speakers which is why some people prefer one over the other. However, for the pitfalls you mention, headphones can actually be closer to what is on the recording than speakers. For example, they are usually a single full range driver with no crossover which means better phase response. I find differences in recordings easier to spot on headphones since there is less extraneous interference. (I've got a pair of old Stax electrostatics which have a very nice sense of "space.")
BTW, "binaural" is not a plug-in. It refers to a type of recording where the stereo microphones are placed in a dummy head as if they were ears and the assembly placed in an optimal seat in a concert hall. The recording does not sound good over speakers but presents a stunning image in headphones. (As noted, though, since the dummy head doesn't move during recording, if you move your head significantly during playback, the audio "image" doesn't move correctly with your head so your brain gets a confused set of cues. However, the effect is startling if you hold your head reasonably steady.) I don't think anyone does these recordings commercially (since they don't sound right on speakers) but I'm sure you could run down an example if you tried. A standard recording cannot be converted with a plug-in or in any other manner.
| FAQ |
Post a Message! |
Forgot Password? |
|
||||||||||||||
|
||||||||||||||
This post is made possible by the generous support of people like you and our sponsors:
Top of Page ]
[ Contact Us ]
[ Support/Wish List ]
[ Copyright © 2025, Audio Asylum, all rights reserved ]
