In my current attempt at storing my data for posterity (as if anyone cared), I needed to convert all my photos from RAW-format to JPEG.
Part of the problem was, that I had more than 13.000 RAW files in over 370 directories. Luckily DPP supports batch conversion, but only for files within a single directory. So did I really want to initiate a batch process for 370 directories manually? You sir, second row, wearing a yellow Bikini, are right. The answer to that question is, of course: "No frelling way!"
Have a Sandwich
What were my options?
- Find another software that works the way I want
Well, the thing is, that RAW conversion is not a well defined process, so each RAW-processor "develops" pictures in a slightly different way. And I happen to like DPP's way.
So that's a "no go".
- Force DPP to work the way I want
Sounds intriguing to me, so I had found my winner.
Checking for any command-line parameters for
DPPViewer.exeresulted in.. well, nothing. But there were other executables and the most promising was
DPPBatchwithout any parameters even displays a menu where
.vbffiles can be loaded. Aha!
As it turns out
DPPBatchvia temporarily created
.vbf-files in the output-directory. A rather old-school approach to IPC, but something that can be exploited for my needs. I only needed to figure out how to create my own
.vbf-files to use the DPP converter from the command-line.
Recipe for Solution
However, after analysing the file format, it turned out to be a bit more complicated than expected, because of recipes that are required in
Analysing a recipe's format and how it interacts with settings stored within a RAW file itself are a completely different beast altogether and nothing that is accomplished within a few minutes.
So I decided to quickly whip up a Perl script that scans for all RAW files in a directory recursively, stores their original location and moves them to a single directory. Then a single batch process could be started from within the UI of DPP to convert all files at once, after which the script can move the RAW files (and newly created JPEG files) back to their respective directories.
Actually it was not that easy because DPP is still a 32-bit application and ran out of memory when it tried to load that number of files, metadata and thumbnails it. So I had to spread them over a few directories with 2.000 files each. Which allowed me to convert all files by starting only 7 batch processes manually. Compared to the original number of 370 this was highly acceptable however.
.vbf File Format
Nonetheless here is what I've found out about the format of
.vbf-files. Perhaps someone else can fill in the blanks or find something useful to do with that information.
Unsigned 32-bit big endian integers
The null-terminated string is preceeded by a 32-bit big endian integer specifying the byte-length of the string (including the terminating \0). The characters in the string are 1 byte wide and use the current Windows codepage, not UTF-8 encoding. Way to go!
Just like strings, recipes start with the number of bytes for the following recipe-chunk. For DPP 3.9.3 that are 1026 bytes.
|S||The string "|
|S||Directory where the converted files will be saved|
IF filename IS "|
|S||Absolute paths to 5 different colour profiles|
|N||Output file format just like in the combo box:|
0 = JPEG
1 = TIFF 8-bit
2 = TIFF 16-bit
3 = TIFF 8-bit + JPEG
4 = TIFF 16-bit + JPEG
|N||JPEG compression quality ranging from 10 to 100|
affects only JPEGs by reducing their size. So my guess is: optimisation of huffman tables. And JPEGsnoop shows different distributions within those tables when using one or the other setting.
When resize is activated it is actually 4, so maybe this is more than just a boolean flag but a bitfield.
|N||Width and height are always in pixels. Any settings in the UI regarding the units of measurement are only used in combination with the DPI-setting to calculate the pixel-size|
|N||set to the same value as 'external' by the UI. However changing this value to an arbitrary value doesn't change anything, by the look of things.|
|R||Some kind of global recipe?|
|N||Number of image-chunks following:|
Update 2012-01-05Thanks to commenter DSops for pointing out Canon's free Digital Imaging Developer Programme, which includes an SDK for converting RAW files. However the FAQ states:
Q: What are the criteria for joining the programme?So it might or might not work out for you.
A. To be eligible to join Canon's Digital Imaging Developer Programme, you should be operating as a legally registered company. Applications from other users will be considered at Canon's discretion. [...]