Guides CHD Format: Lossless Compression to Save Massive Storage Space!

[RashFaustinho]

The CHD format was developed by the MAME team (yes, those same crazy folks whose project is to emulate every machine possible, starting with arcade systems).
It can be used with most of the CD-DVD based consoles

It’s an image file, much like an ISO backup file for a PlayStation 2 game, but with the major difference of being massively optimized for compression.

Here’s a quick comparison using MY PlayStation 2 game folder before and after compression:

Before compression: about 30-35 games

After compression: same number of games

The size reduction isn’t the only advantage. For instance, the CHD format can also compress bin/cue files (used by consoles like the original PlayStation) without losing the .cue file information in the process. (This is something that can happen when trying to convert a CUE+BIN into an ISO)

So, there’s no audio loss for games that use systems with multiple audio tracks. The CHD format preserves everything and is much tidier compared to having multiple pairs of scattered files.

Major emulators for the following systems support the CHD format:​

• 3DO
• Arcade (MAME)
• CD-i Philips
• Dreamcast
• PlayStation
• PlayStation 2
• PlayStation Portable
• Sega Saturn
• Sega CD
• TurboGrafx-CD

---

How to Use It​

You can download the ZIP file attached to this post.
There is also this Mediafire Link (just in case):
CHDMAN (RetroPie User-Friendly)
This ZIP has .bat files, but they are all innocuous. Feel free to virus scan them before opening.

Extract the contents into the SAME folder where all the ISO, BIN, and CUE files are located (or move the ISOs into its folder — it’s up to you).

Run the corresponding bat file: CreateCD or CreateDVD, depending on whether it’s a CD-based or DVD-based game. (PSP's UMDs count as DVDs)
Wait for the process to finish (it might stress the CPU of your PC a bit, depending on the size of the files), and you got your compressed game!

Obviously, being a loseless compression, you can get back your original ISO file (in case you need it) by running the ExtractCD or ExtractDVD bat files

 

[Tbest25]

That's awesome, thanks for the explanation and the files!

 

[bloodywing]

I used this in the past, because I had a "few" disc to convert: https://github.com/thingsiplay/tochd You still need chdman, but it makes working in bulk faster and easier.

 

[otogirisou]

is it smaller compared to zipped files?

 

[RaypuiaASCII]

is it smaller compared to zipped files?

Yes, and decompress way more faster.
So, you'll have barely any framedrops on games during decompression (unless you have slow storage)

 

[Glimbo]

If I use this to convert a multi-bin cd image to chd and convert it back to bin/cue, it ends up being a merged bin which is not redump verified. Is there a setting I'm missing or do I have to fix it with another tool?

Edit: Found what I was looking for here.
chdman can not convert chd to multi-bin/cue properly so you need to use something like binmerge to split it into separate bins then download the proper cue file from redump.

 

[ThatsNotRich]

Just wanted a tad bit of clarification if possible.

So you would take your ISO and create DVD.
That then turns said ISO into .chd?

Also, for .bin/.cue, are we wanting to make sure that all corresponding .bin and .cue files are in there right folders (games) and then create DVD using the folder or does it auto select all corresponding .bin and .cue in that folder?

I mean I'm all for saving space and compression so I am a tad curious about it.

EDIT : I've re read this post and comments and believe I understand this is a .bat and not app or .exe

 

[ATenderLad]

chdman – CHD (Compressed Hunks of Data) File Manager — MAME Documentation 0.273 documentation

The official documentation for the chdman utility for those looking to mess with it.

 

[AnonymousCookieMonster]

is it smaller compared to zipped files?

Yes, and decompress way more faster.
So, you'll have barely any framedrops on games during decompression (unless you have slow storage)

Yes and no. Regular zip files? yes CHD will do better as it's lzma. 7z files? I regularly see CHD to be 10-20% larger. But it's still better than uncompressed.

If the game includes a LOT of music, the FLAC audio compression is superior to 7z, but if it's mostly game data 7zip beats CHD, especially if you ecm the bin files.

I'm working on a way to store it with the best of both worlds. Got a prototype for Dark Stone at 307Mb, just not sure the exact way to do the shell script and how to store/name track 1.

Just wanted a tad bit of clarification if possible.

So you would take your ISO and create DVD.
That then turns said ISO into .chd?

Not quite following. CHD works fine on iso's as well as cue/bin files.

Though i'd like to see good examples of storing non-game data, like cover images and readme's.

Also, for .bin/.cue, are we wanting to make sure that all corresponding .bin and .cue files are in there right folders (games) and then create DVD using the folder or does it auto select all corresponding .bin and .cue in that folder?

I mean I'm all for saving space and compression so I am a tad curious about it.

EDIT : I've re read this post and comments and believe I understand this is a .bat and not app or .exe

The exe is the app that does the work. The bat file that is included just scans for cue/bin files and makes all the chd files it can.

Curiously if you due chd to cue, it merges all the bin files together. using binmerge you can resplit it to individual files.

 

[DanCalibur]

Very useful tool for saving space and keeping those pesky multi track discs tidy. For those using retrobat, you can access CHDMAN thru BatGui.exe > CHD manager.

 

[McDoob]

The CHD format was developed by the MAME team (yes, those same crazy folks whose project is to emulate every machine possible, starting with arcade systems).
It can be used with most of the CD-DVD based consoles

It’s an image file, much like an ISO backup file for a PlayStation 2 game, but with the major difference of being massively optimized for compression.

Here’s a quick comparison using MY PlayStation 2 game folder before and after compression:

Before compression: about 30-35 games

After compression: same number of games

The size reduction isn’t the only advantage. For instance, the CHD format can also compress bin/cue files (used by consoles like the original PlayStation) without losing the .cue file information in the process. (This is something that can happen when trying to convert a CUE+BIN into an ISO)

So, there’s no audio loss for games that use systems with multiple audio tracks. The CHD format preserves everything and is much tidier compared to having multiple pairs of scattered files.

Major emulators for the following systems support the CHD format:​

• 3DO
• Arcade (MAME)
• CD-i Philips
• Dreamcast
• PlayStation
• PlayStation 2
• PlayStation Portable
• Sega Saturn
• Sega CD
• TurboGrafx-CD

---

How to Use It​

You can download the ZIP file attached to this post.
There is also this Mediafire Link (just in case):
CHDMAN (RetroPie User-Friendly)
This ZIP has .bat files, but they are all innocuous. Feel free to virus scan them before opening.

Extract the contents into the SAME folder where all the ISO, BIN, and CUE files are located (or move the ISOs into its folder — it’s up to you).

Run the corresponding bat file: CreateCD or CreateDVD, depending on whether it’s a CD-based or DVD-based game. (PSP's UMDs count as DVDs)
Wait for the process to finish (it might stress the CPU of your PC a bit, depending on the size of the files), and you got your compressed game!

Obviously, being a loseless compression, you can get back your original ISO file (in case you need it) by running the ExtractCD or ExtractDVD bat files

Do you know if this has a Linux (and ARM) compatible binary? I'd definitely like to use it in my Pi 4. Storage space is a problem, even with a 500gb SSD, since I use this thing for literally everything. Currently, my game library is using 66gb, with almost all of the cartridge roms stored as zip files. The biggest ones, of course, are the CDs/DVDs.

-McD

 

[bloodywing]

Do you know if this has a Linux (and ARM) compatible binary? I'd definitely like to use it in my Pi 4. Storage space is a problem, even with a 500gb SSD, since I use this thing for literally everything. Currently, my game library is using 66gb, with almost all of the cartridge roms stored as zip files. The biggest ones, of course, are the CDs/DVDs.

-McD

Which Linux distribution do you use? There is mame-tools in arch and debian, which has chdman in it.

 

[McDoob]

Which Linux distribution do you use? There is mame-tools in arch and debian, which has chdman in it.

RasPiOS, which is a debian derivative. The issue is the ARM CPU in my Pi, which is not x86 compatible. I'd have to...emulate, and with x86, performance really suffers. I once tried to set up a virtual machine with Win9x, and that was a complete disaster...

I will check it out, though, and see if that package is available.
-McD

EDIT: Turns out there is a mame-tools for ARM64...it is a half-gig install, which isn't great, but if it can cut my game library size in half, it's worth it!

 

[AnonymousCookieMonster]

Turns out there is a mame-tools for ARM64...it is a half-gig install,

When i was trying to get updated tools i was running into something similar. Kept running into it being the 0.64 package of emulators; As for a chd management that shouldn't be nearly that much space. Aside from dependencies of LZMA compression (which may pull libraries from 7z or xz i'm not sure) FLAC and a couple others, it really shouldn't be anywhere near that size.

Course depending on what libraries it uses, you could install, copy chdman and then uninstall so long as the other dependencies are met.... (Hell even if some of them aren't. I remember Ogglib in 2005 complaining about not having some driver dependencies and warning me; But i only ever did encoding/decoding not feeding to hardware so i never needed it and it worked fine)

 

[thecrow2313]

Just wanted a tad bit of clarification if possible.

So you would take your ISO and create DVD.
That then turns said ISO into .chd?

Also, for .bin/.cue, are we wanting to make sure that all corresponding .bin and .cue files are in there right folders (games) and then create DVD using the folder or does it auto select all corresponding .bin and .cue in that folder?

I mean I'm all for saving space and compression so I am a tad curious about it.

EDIT : I've re read this post and comments and believe I understand this is a .bat and not app or .exe

createcd is for .gdi & .cue
createdvd is for .iso

 

[RageBurner]

This is greatly appreciated.
Can chd be ran directly by emulators or is it strictly a storage format? I didn't understand that exactly. Also, it would be nice to know which emulators for each system support this.

 

[AnonymousCookieMonster]

Can chd be ran directly by emulators or is it strictly a storage format? I didn't understand that exactly.

A number of emulators directly support it, as sectors (or groups of sectors) are compressed as a unit, then decompressed by the emulator and passed as though it were the sectors requested. Check your emulator to see if it's supported, but i've seen several that support it. From what i see PCSX2 (at least on Retroarch, Batocera and Lakka) support CHD, and the PS1 emulators too.

I've heard a few games don't work right as CHD, has to do with the ECC it uses for some internal checks that don't add up, but i'm sure those if you come across them you just decompress them.

 

[RageBurner]

A number of emulators directly support it, as sectors (or groups of sectors) are compressed as a unit, then decompressed by the emulator and passed as though it were the sectors requested. Check your emulator to see if it's supported, but i've seen several that support it. From what i see PCSX2 (at least on Retroarch, Batocera and Lakka) support CHD, and the PS1 emulators too.

I've heard a few games don't work right as CHD, has to do with the ECC it uses for some internal checks that don't add up, but i'm sure those if you come across them you just decompress them.

Thank you for the explanation! I just hope the support widens to standalone versions of emulators too, as using Retroarch on PC for me has the equivalent comfort level of sitting on a cacti couch.

 

[AnonymousCookieMonster]

Thank you for the explanation! I just hope the support widens to standalone versions of emulators too, as using Retroarch on PC for me has the equivalent comfort level of sitting on a cacti couch.

Understood. I don't use Retroarch on PC (well, not on a main PC anyways, an emulation box i do batocera at present). From what I've seen on Linux RetroArch systems is a LOT of them support some type of compressed iso format. ePSXe supported a CDZ format, still got the tool to compress/decompress PS1 games with that.

CHD is common, but there's also CSO/CISO (which seems rather anemic on compression), and Lakka a number of them even supported zip/7z (mind you, it just decompresses the whole thing, then runs it from the tmp folder, so starting up may take 2-5 minutes but then runs quite well. Downside is if you exit out, it has to decompress it all over again. Doesn't cache the uncompressed game)

 

[RageBurner]

Mind you, retroarch on emu boxes and devices is wonderful (I feel it was conceived with this in mind), but on desktop it feels clunky. It's gotten better, but it's not quite there yet.

 

[Cheela]

Great info!! really glad for these helpful treads you are making :3

 

[thecrow2313]

I'll leave this here

for /r %%i in (*.cue, *.gdi) do chdman createcd -i "%%i" -o "%%~ni.chd"

for /r %%i in (*.iso) do chdman createdvd -i "%%i" -o "%%~ni.chd"

 

[AnonymousCookieMonster]

I'll leave this here

Looks suspiciously similar to the 'cue/gdi to chd' bat file...

 

[thecrow2313]

Looks suspiciously similar to the 'cue/gdi to chd' bat file...

Yes that's correct ps1, dc (gdi), or disc image with .cue & .bin use createcd
PS2, PSP, .iso disc image use createdvd

Just to clear the confusion by using the correct compression method using chdman