View Single Post
  #60  
Old 2020-02-19, 10:41 PM
Five's Avatar
Five Five is offline
 
Join Date: Oct 2004
Location: Canada
Re: The Validity of MD5 Checksums

ctree.org, circa 2002-06-04

CDRs used to be this important...
Quote:
Originally Posted by bobby GeeWOW
bobby geeWOW's CDR Rant..

This is basically a general overview of CDR technology culled from a buncha sources. Its intended to pull together all the divergent bits of info out there and kind of put things into context for joe non-computer-professional lotus-eating shorten trader.

While I do of course degenerate into offering my own deluded opinions here and there, I urge you bear in mind that I know nothing and claim no expertise in anything whatsoever - except for some propensity towards maximising sheer wonder and enjoyment in the beauty this great country of ours is blessed with.

So set yer bullshit filters to fine grained and kick back if yer gunna read it, 'cause this got WAY longer than I thought:

Episode 1 Is a general introduction to CDR technology.
Episode 2 Gets into the technology in a little more depth in order to address some of my ludicrous claims.
Episode 3 Here come things like the Plextor/Mitsui fetish.
Episode 4 Deals with software. Yowtch, get yer flame retardent ready..

If you have any comments/questions/open redicule to offer, please direct them to me or the list.


Episode 1: Myths and Facts in CDR burning.

First off the accelerated age testing that supported the industry's claim of 100 year CDR life spans turned out to be rediculous in the real world. And remember when they hyped us with the "you can scratch the bejeesus out of a CD and still play it so forget about albums" spiel?

Basically when you burn a disk the laser discolours the dyes on the disk in such a fashion that the data on them will be recognized and read for playback. Turns out that light and heat over the life of a disk will continue to change the state of these dyes until they eventually, and inevitably, become unreadable. I have a large batch of 3 year old audio disks burned to cheapo CDR's that are starting to just fail to play back audio at all - sitting happily ensconced in the darkness of binders at room temperature. Then hold a disk that's just been whizzing around for a while in your computer's CDOM drive between the palm of your hands...

So there are two basic types of disks:

Cyanine dye (type 0 and type 1) or Azo disks use a `long write' strategy type: low laser power for long periods of time. Typically blue in color.

Phtalocyanine dye use a `short write' strategy type: high laser power for a short time. This corresponds to the increasing speed at which modern burners operate. Typically light greeny/goldish in color.

By the way, Verbatim has the patent on Azo, Mitsui on Phtalocyanine, and Tayo Yuden has Cyanine (also the first CDR's made and the basis of the "Orange Book" specifications).

There are also key differences in what happens when yer burning the disks: Cyanine dyes, the original, do a phase change which discolours the burned parts that a reader then decodes into the ones and zeros that make up digital audio or data. I urge the gentle reader to think about this and picture just how "permanent" such a disk will be.

When Phtalocyanine disks are burned, a lower layer oozes up and hardens, actually imitating the pits of a real silver factory pressed disk. As you can imagine, these disks will be more robust and tolerant to environmental changes, as well as give the reading laser a little more to bite on.

On to the chaos:

Bear in mind there are a lot of factors involved when performing a burn - ambient temperature, temperature in the burner, type of dye on the disk, write strategy, humidity (go ahead and laugh..), speed of rotation, etc.

Also throw in such physical factors as groove width (which is different for factory audio disks and CDR specs) and the literally microscopic accuracy of the manufacturing necessary to make a good consistent disk. Utter flatness and resistance to warping (unlike what we strive for in our lives) are needed. As a side note, some CDR manufacturers are replacing their stamper dyes (molds) almost every 4-6 months to keep up with the changing dye formulations.

So even a good old Tayo Yuden Cyanine CDR is not the same disk it was two years ago.

Oh yeah:

I strongly encourage you next time yer bored, to grab some coaster you made and break it. Go ahead, its good fun - it blows up with surprising force. *DO NOT LOOK!* - in fact close your eyes while you break it. Point it at a corner of the room or the pieces'll be all over the place.

Then find some shard that's scattered about and check it out. You'll find the actual micro-thin burning surface is in fact glued to the TOP of the disk, and the plastic underneath makes this surface flat, like pressing your face against a window. Sometimes there is writable paint on top of the reflective coat, obscuring but not protecting it.

From this you'll see just how obviously the most delicate part of a disk is actually the TOP of the disk, and you'll see how incredibly easy it is to instantly and permanently destroy a disk by scratching this layer. You will need to find a shard with a frayed bit of reflective layer to really see what I mean here.

Kodak Ultimas, Mitsuis, cdrecordable disks all have a protective coating on top for this. cdrecordable has confirmed they license their dyes from Mitsui by the way, so they have a firm stake as one of the cheapest sources of quality CDR's on the net.


Episode 2: The Depths and Depravity in Burning Plastic

Here's where it gets weird:

The pits in a factory pressed CD vary in length to denote information. Fine. But the length is *relative to time* - in other words to the rate of spin on the disk. This is bizzarre since they must remain readable within very tight (microscopic!) specs. Consequently we have long and short write strategy types to compensate for this, and associated media to make it all happen. Its chaos and deeply weird, but it works.

The whole idea is to imitate these microscopic varying-length pits made by CD presses as per Phillips's patent. Not a trivial undertaking when using a laser to burn dyes embedded in plastic travelling at high speeds - especially considering the margin for error and this wacky relative pit length scenario..


However, read on:

When actually creating a pit of a particular length, everything varies with the whim of the tree spirits. A certain setting on a laser of a certain strength that produces a certain length pit on one disk at such a spin rate will not necessarily produce the same length pit when a different dye is used, or the ambient temperature goes up, or the sun peeks out from behind the clouds in Tuktayuktuk for that matter...

Nothing is linear here either, and doubling the setting for a 3T pit does not produce a pit twice as long. Its insanely finnicky stuff to get right. Additionally, your burner is optimized for certain dye types and write strategies, which are themselves in a continuous state of flux.

Then of course the reader has to interpret these pits exactly in order to accurately reproduce the data contained within. Enter the mayhem of accurate ripping and the birth of such tools as EAC and cdparanoia to combat all this ambiguity, essentially by re-reading the data until its satisfied it statistically speaking has the correct bits...this is not trivial either, as is evidenced by some cheap readers not even being able to maintain a consistent offset, literally making it impossible to get accurate audio rips.

Another factor here is the relative reflectivity of the dyes used. If its brighter it requires more laser or more time to make a good impression, and still has to be read by the stereo CD player, which of course is built to read a certain relectivity at different specs than those called for in CDR technology.

So its entirely possible that your Phytloyanine disk burned with a long write strategy on your old burner with the weaker laser won't be playable on your stereo, but might be fine if burned at a slower speed on a full moon in May over a fine pint of Guinness.

As a side note, the various CDR identifiers out there of course can't actually tell what the dye on the disk is - only what the stamper that made the disk is designed to produce, and consequently enter into the pregroove's ATIP info.

That may well change with the new specs. The good news is that you can be sure that if it says Cyanine, it is Cyanine, since the differences between dyes is so pronounced that a you can't use a Phyalocyanine stamper to make a Cyanine disk (or visa versa). However, a stamper made for Cyanine type 1 might well be used to make a type 0 disk, or use a reformulation of the originally intended dye.


Episode 3: Gearhead Love

So what colour disk should I get for my burner then?

As you can see, there are no hard and fast answers to this. "Whatever works for you" is good advice. Older burners tend to be designed for a Cyanine long write strategy, and newer ones for Phytlocyanine short write strategies.

Or go with the highend manufacturers that are consistently on the cutting edge of this rapidly developing technology and have always out-performed mandatory specifications, like the famous Plextor/Mitsui combo.



Aw christ, why Plextor?

Basically they have been at the forefront of CDR technology since the beginning. Their drives are of universally high quality, and tend to not only define, but exceed standards imposed on CDR(OM) drives. It is not uncommon to have Plextors burning thousends upon thousends of disks correctly and without a complaint. Their ripping accuracy is similiarly without peer. They actually design their CDROM's with accurate audio extraction in mind.

Now immediately after saying this, you should know that shawn has burned thousends of disks spread over many years with a crappy old P133 laptop and an external USB 4x HP burner, about the most nightmarish setup going, so go figure. Mind you, the ol' beast refuses to extract audio accurately, so its a good thing he archives his shns.

Here's the scoop on what Plextor's up to right now. They invented BurnProof technology to prevent coasters caused by buffer underruns. Works beautifully. Essentially if your buffer runs out of data because you've got seventeen other apps open at the same time, (Plextors have nice big buffers too) then the laser stops at an ECC or header spot away from the data part of the disk and hangs out until it can continue burning. I can actually burn data images directly from drives located across a 100Mbit network connection using CDRDAO with BurnProof turned on as long as I don't bog down the network by streaming the latest shn seed from the server across the house at the same time..

Next up from Plextor:

PowerRec II - supposed to be a couple years off, but according to a Plextor beta tester *cough!*, they already have drives that can recognize the media type and calibrate themselves with respect to write speed, write strategy, and laser strength, integrated with BurnProof in order to provide the ideal burn. Other manufacturers like Yamaha are also working on this logical next step.

But THEN we have OPC. Get this..

What happens here is that the burner monitors the reflected light coming back from the disk as the burned pits form, comitting a signature to memory. As the burn progresses, the Plextors will monitor the new marks as they are created, and actually modify the burnspeed *on the fly!* to maintain the optimum burn. No kidding. The technology has to mature for a while before going public, but our friendly brotha at Plextor claims to be playing with pre-alpha firmware that already works. It could be that all you gotta do is upgrade yer current Plextor's firmware to get in on this good stuff when its ready..



So what's with the Mitsui trip then?

Mitsui directly controls every aspect of their production process. They do not outsource any part of the manufacturing process, technology, or even raw materials(!). Their quality control is second to none, and one of the key trips is to make CDR's as flat as possible to enable stability at high RPM's. That's kind of the main deal with 24x and now the Plextor-only 40x certification. Its also crucial for longevity, since a disk that is easily warped will also become unreadable very quickly.

The gold disks come with a lifetime guarantee. In all the industry testing for Orange Book compliance (involving block error rate (BER) tests on abused disks, Mitsui's consistently outperform the specs, in particular the light exposure tests. (CDR masters going in for mass production are made to Red Book specs by the way.) They also coat their disks with a protective layer to guard against scratching, including the top side of the disk.

They also own the patent on Phytlocyanine dye, which as I explained earlier is far superior to Cyanine/Azo dyes.

That said, they are the pinnacle of CDR's. They gained their reputation in the early stages and have held true. They now license their dyes to people like Kodak and cdrecordable, who make excellent disks themselves.

Update:

Unfortunately Kodak has now sold off their CDR assets, including Matsushita Manufacturing LLC of America at the end of last year. They made good stuff, get 'em while you can.


Episode 4: Let's take 'er for a spin then!


...and wrecklessly forge on to the software side of things..

When you are ripping audio disks, use EAC for windows or cdparanoia for *nix. Mac users have Trackthief. This software knows all about CDROM offsets, CDR sector boundaries, and so on. You will have to tell it manually what your drive's offset is. The resultant wavs will be as bit accurate as possible, and when burned back to disk will yield non-skipping, non-popping audio CD's if everything else is right.

Anything else will result in inaccurate rips, and you will be corrupting the community's gene pool, bringing disease and pestilence on the land. You can easily check your ripping by making an md5sum of some wavs, burning them to audio, then ripping them back to wav and checking the md5sum you made for the originals. If they fail you need to get your ripping trip together pronto or live with your own karma...or archive your shns!

EAC/cdparanoia will also tell you a little about your reader, because they require setting and checking your CDROM's offset and will give you a log file of the ripping process to let you know how many potential corrupted bits are in your wav files.

Of course shn files completely eliminate all the ripping ambiguity. Shn disks are burned as data CD's, and file integrity can be verified with an included md5sum, ensuring precise clones of the original master tapes. Places like etree.org archive the md5sums from the original seeds of shows, so you can check the lineage of your sources directly.

It might be worth archiving your shn files and burning audio disks as needed to throw in the truck, give to friends, etc. This way you maintain a perfect clone of the source recording for future audio burns, your next trade - and future generations if yer thinking of propagating the species with that special someone in yer life on a groovy afternoon on a sunny hillside..



Burning. Ouch. I must be out of my mind to even go here. Passions run high with this one, so please review the opening disclaimer before proceeding. There are lots of considerations to take into account here, including the robustness of the burning engine itself, supported drives, the ability to create bit-accurate data copies, conformity to standards and so on.

If you actually read all this drivel so far, you probably appreciate that the process of interfacing with the burning device to provide an accurate, steady bitstream to be burned into a disk is not quite so trivial as might appear. And I haven't even touched on file systems, sectors, EEC headers, zones, and all that dry technical grind that define the exact format of a CDR. Yech, don't ask.

For our purposes, the number one factor is the ability to burn in Disk-At-Once (DAO) mode. Anything else and yer playing with fire - especially that abomination called DirectCD that used to be bundled with burners. It uses a packet writing technique, and is to be treated as the very incarnation of evil itself and purged from your system accordingly. I'm serious, I've seen 70+ packet written CD's suddenly simply stop working all at once, representing a devastating loss of data to a friend of mine. He now believes in my DAO fetish.

Speaking of DAO, I'd say that any software that supports it (and you can dig out the place to make that your default setting) will do fine. If you're wondering, the absolute cadillac of burning apps for windows types is CDRWin. Rumoured to be based on opensource unix code, it makes good bit-accurate rips of data CD's into a format called bin/cue, and is very good at burning data reliably, bit by bit, onto a CD. All CD's rigorously conform to standards, with no convienience kludges to make it work. It supports BurnProof, raw read/writing, and ONLY does DAO writing. It doesn't even know what TAO, packet writing, or any of those other hokey methods are.

Unix users have CDRDAO for all that stuff. Needless to say it works perfectly out of the box, including BurnProof support, leaving them once again to wonder why people would even use an inferior OS. (Then casually engage them in a linux vs. BSD debate and watch the sparks fly..)

If you've been using Nero for audio:

PLEASE check out the integrity of your audio disks! A bug has been discovered wherein Nero consistently *loses frames of audio*, and writes corrupt tunes to disk! Ahead software has confirmed this, and pomises a fix sometime. This really hoses Nero users who burn audio disks and didn't archive their shns. One can only wonder how many shows out there have been affected by this rediculous bug. The original report is gone, but have a search of google to get an idea.

You can check your disks with md5sum as I described above, and shntool will give you sector boundarys and lenths of the wavs.



One sneaky little issue that has caused immense headaches for people is the cursed ASPI layer. This is a standard device interface layer that is owned by Adaptec. Huh? You heard me, a standard that is proprietrary. Sort of like the Cisco/VRRP debacle, but that's a different trip all together. The ASPI layer basically makes an ATAPI (IDE) device look like a SCSI device to the burning util in question. Only Adaptec products (like EZCD) are allowed to ship with an ASPI layer.

This causes mayhem in itself, with differing versions and other manufacturers making their own ASPI knockoffs. Adaptec has gone so far now as to make their installers check if you have an Adaptec product (hardware or software) installed before it will throw the thing in there. You must load an ASPI layer to burn disks if your app doesn't bring its own, which CDRWin now does.

As an aside, I hear some dilligent freedom hackers have modified this installer to load the ASPI layer regardless of such Adaptec created environmental concerns and released it on the 'net for public download. This will also install ASPI layers onto XP machines, which have their own issues regarding burning with their "integrated" burning facility junk. I also gather that you can goto your "Control Panel - Add/Remove Hardware" and pretend to install an Adaptec SCSI card to fool the official Adaptec ASPI installer.

Update: Adaptec received so much flak for this move that they have relented, and released the latest ASPI installer to load on non-Adaptec endowed PC's and it supports XP too.



So what can we conclude with all this? Its a crazy, fast-changing, dynamic little corner of technology, that's for sure. Are there any guaranteed solutions? No. But it does go some way towards explaining the rabid brand conciousness some people exhibit when confronted with this debate. You will have to experiment, try this and that, until you have a dependably working combination of ingredients.

I would say though, if you go Plextor+Mitsui+CDRWin and have a reasonably fast hard drive, you can't go wrong - and based on my experience, will never regret the reliable, consistent burns you will achieve for years to come. You will also have the comfort of knowing that people you trade with won't groan when they open their package to find badly burned cheapo CDR's that pop and click, wrecking their eagerly awaited can't-miss jam of the year.



Instead they'll throw the new disks on, crank it up, and invitingly start to slink about the kitchen in a suggestive manner as the groove begins to move their bones and everything erupts in titanic sexy mayhem, encouraging everyone to get on up, feel free, and groove!
Attached Images
File Type: jpg ctree_banner.jpg
( 50.6 KB, 22 views)
 
__________________
Checksums Demystified | ask for help in Technobabble

thetradersden.org | ttd recommended free software/freeware webring
shntool tlh eac foobar2000 spek audacity cdwave vlc

Quote:
Originally posted by oxymoron
Here you are in a place of permanent madness, be careful!
Reply With Quote Reply with Nested Quotes