[This information was collected using Digital Photo Professional 3.9.3 on Windows. It may not be applicable to other versions.]

Meet Roadblock

In my current attempt at storing my data for posterity (as if anyone cared), I needed to convert all my photos from RAW-format to JPEG. Part of the problem was, that I had more than 13.000 RAW files in over 370 directories. Luckily DPP supports batch conversion, but only for files within a single directory. So did I really want to initiate a batch process for 370 directories manually? You sir, second row, wearing a yellow Bikini, are right. The answer to that question is, of course: “No frelling way!”

Have a Sandwich

What were my options?

• Find another software that works the way I want Well, the thing is, that RAW conversion is not a well defined process, so each RAW-processor “develops” pictures in a slightly different way. And I happen to like DPP’s way. So that’s a “no go”.
• Force DPP to work the way I want Sounds intriguing to me, so I had found my winner. Checking for any command-line parameters for DPPViewer.exe resulted in.. well, nothing. But there were other executables and the most promising was DPPBatch.exe. Starting DPPBatch without any parameters even displays a menu where .vbf files can be loaded. Aha! As it turns out DPPViewer communicates with DPPBatch via temporarily created .vbf-files in the output-directory. A rather old-school approach to IPC, but something that can be exploited for my needs. I only needed to figure out how to create my own .vbf-files to use the DPP converter from the command-line.

Recipe for a Solution

However, after analysing the file format, it turned out to be a bit more complicated than expected, because of recipes that are required in .vbf-files. Analysing a recipe’s format and how it interacts with settings stored within a RAW file itself are a completely different beast altogether and nothing that is accomplished within a few minutes.

So I decided to quickly whip up a Perl script that scans for all RAW files in a directory recursively, stores their original location and moves them to a single directory. Then a single batch process could be started from within the UI of DPP to convert all files at once, after which the script can move the RAW files (and newly created JPEG files) back to their respective directories.

Actually it was not that easy because DPP is still a 32-bit application and ran out of memory when it tried to load that number of files, metadata and thumbnails it. So I had to spread them over a few directories with 2.000 files each. Which allowed me to convert all files by starting only 7 batch processes manually. Compared to the original number of 370 this was highly acceptable however.

.vbf File Format

Nonetheless here is what I’ve found out about the format of .vbf-files. Perhaps someone else can fill in the blanks or find something useful to do with that information.

Data Types

• Numbers: Unsigned 32-bit big endian integers
• Strings: The null-terminated string is preceeded by a 32-bit big endian integer specifying the byte-length of the string (including the terminating \0). The characters in the string are 1 byte wide and use the current Windows codepage, not UTF-8 encoding. Way to go!
• Recipe: Just like strings, recipes start with the number of bytes for the following recipe-chunk. For DPP 3.9.3 that are 1026 bytes.

Update 2012-01-05

Thanks to commenter DSops for pointing out Canon’s free Digital Imaging Developer Programme, which includes an SDK for converting RAW files. However the FAQ states:

Q: What are the criteria for joining the programme?
A: To be eligible to join Canon's Digital Imaging Developer Programme,
   you should be operating as a legally registered company.
   Applications from other users will be considered at Canon's discretion.
   [...] 

So it might or might not work out for you.