# Question about RAW



## skitron (Apr 20, 2011)

My understanding of RAW is that it is simply the data "as-is" read from the sensor and saved on the memory card with no processing applied. However, I found that the color temp as set in a 50d directly affects the color temp of the saved RAW image. Are other in-camera functions applied to RAW as well? 

Sorry for such a basic question but I'm new to DSLR and the last time I was serious about cameras was back in the film days. I thought the description of RAW in the manual was pretty straight forward until I tested color temp on it.


----------



## prestonpalmer (Apr 20, 2011)

skitron,

A RAW file contains virtually all of the camera and sensor data when it is written to your card. It is saved with a "loss-less" compression algorithm unlike JPG which actually throws out a TON of image data.

Before I continue, I recommend that you never again shoot a JPG on your camera, set it to RAW and RAW only, no RAW+JPG.

The color temp you set on your camera will change the way the image appears on your computer screen. When you set a "Color Temp" you are essentially telling the computer that is translating the raw image, just how exactly to display that image, even though ALL the color temp data still exists in the file. A good habit you can get into now is NEVER changing the white balance setting on your camera, you can change it AFTER you take the photo as long as you are shooting in RAW. Set your camera to AWB, and leave it there....forever. When you get your photos onto a computer you can use Lightroom, Aperture, Photoshop, Bridge, or even DPP to adjust the White balance to EXACTLY what you want after you take the photo by selecting a true white or true black. You will get the most accurate white balance possible with this method, AND you can apply the color temp change to a bunch of photos. IE, you have 50 photos in a room with florescent lights, you can select all 50 and correct the white balance, if the next bunch of photos are in a incandescent lit reception all, select all those photos and change the color temp to that situation. This will prevent you from repeatedly "guessing" as to what color temp to set your camera to, and will make ALL your photos more accurate in much less time. I know this concept may be difficult for you to grasp at this time, but trust me when I tell you to leave your White balance alone and shoot in RAW. Ive taken more than 500,000 wedding pictures and I can tell you what works and what doesn't. Fast and efficient work flow is a HUGE part of my business.


----------



## jeremymerriam (Apr 20, 2011)

prestonpalmer said:


> skitron,
> 
> A RAW file contains virtually all of the camera and sensor data when it is written to your card. It is saved with a "loss-less" compression algorithm unlike JPG which actually throws out a TON of image data.
> 
> ...



Before you give advice you really should be positive on your facts.

Just setting the color temp to AWB and forgetting it is not always a wise solution. RAW doesn't save all the gamuts of color that always completely represent the true colors. It works sometimes but not all of the times. This I know from experience shoot all type of jobs. AWB has its place in a pinch but if you know the true color temp you are dealing with, images ALWAYS have better colors at especially higher ISO if shot properly. The closer you are to the mark, the better the colors will be when using LR and PS. AWB does an awful job with tungsten lighting at times and LR doesn't always do a decent job recovering that error. I shoot a lot of restaurants and I have learned from my mistakes.


----------



## neuroanatomist (Apr 21, 2011)

jeremymerriam said:


> Just setting the color temp to AWB and forgetting it is not always a wise solution. RAW doesn't save all the gamuts of color that always completely represent the true colors.
> 
> *Before you give advice you really should be positive on your facts.*



Yes, you should. _Your_ 'facts' are wrong. A RAW file does not have a color temperature applied _to the data_ (only the metadata). RAW image data contains the readouts from individual pixels - each pixel is a monochromatic sensor (it counts photons regardless of their wavelength), with color data coming from the overlying filter in the Bayer mask - but even though that limits the photons counted to those of a particular part of the visible light spectrum, even that 'color' is artificial - pseudocolor, if you will, based on the RAW conversion software knowing which color (R/G/B) mask was over that particular pixel). Thus, the setting you choose on your camera for the white balance has no effect whatsoever on the image data in the RAW file. 

It's true that at high ISO, dynamic range is compressed and the ability of the camera to accurately capture the full color gamut is compromised - but for a RAW image that results from the lighting conditions and the ISO setting, not the irrelevant WB setting on the camera.

If your white balance setting is affecting your RAW images, it's an indirect effect that could be called operator error (if not error, then a lack of understanding). Although the WB setting does not affect the RAW image data, it _does_ affect the JPG conversion. But wait, you say, "I shoot RAW!" Even if you're shooting RAW, a JPG preview image is embedded into the RAW file, and that JPG image, converted in-camera based on your settings (including WB, picture style, etc.) is what's displayed on your camera's LCD, and also what's used to generate the histogram displayed on your camera's LCD. So, if you alter your exposure settings based on that preview image/histogram on-camera, that change in exposure will affect your RAW file image data. 

For example, a large red object under mixed but mostly tungsten light, with AWB set on-camera, may not saturate the red channel of your reduced bit-depth JPG preview/histogram, even if the 'real' red channel is actually saturated (real meaning the RAW data from the pixels under the red filters in the Bayer mask). So, you may feel comfortable bumping your exposure up a little more (ETTR), and end up significantly clipping the red channel in the RAW image. But that wasn't the WB setting affecting the RAW file, that was you changing the exposure. This is a situation where knowing how your equipment works, and it's limitations, can help you avoid problems.

*Skitron*, you are (mostly) correct. The RAW file is the _mostly_ 'as is' data from the sensor. The sensor itself is analog, and the analog data (photons stored as an electirc charge in each pixel well) must be converted to digital information - some minimal processing is also applied as part of that process (differs by manufacturer and model, but they don't disclose that information). Data for each pixel are stored separately in the RAW file (and later must be demosaiced, i.e. the Bayer color mask of red-green-blue filters over the pixels must be interpolated to merge the color channels). The settings selected during shooting (white balance, picture style, ALO, etc.) are recorded as metadata in the RAW file, but do not affect the image data (one exception to that is highlight tone priority, which does affect the RAW file). Depending on the software you use on your computer, some or all of those parameters may be applied as defaults to the RAW file during conversion/viewing - Canon's DPP applies Picture Style, ALO, etc.; 3rd party RAW converters (Adobe ACR, Lightroom, Aperture/iPhoto, etc.) read and apply the WB but little else, and in some cases not even the full WB data (for example, with WB set to AWB or Flash, if you use a Canon Speedlite the color temperature of the flash is reported by the flash to the camera and added to the metadata; if the WB was set to Flash, DPP will use the 'custom' color temperature of the light, whereas 3rd party software will just use the Flash default, e.g. 6000K). But regardless, you can change the WB setting in the RAW file with no penalty to image quality, since you're just changing metadata.

Hope that helps...


----------



## unfocused (Apr 21, 2011)

> A good habit you can get into now is NEVER changing the white balance setting on your camera, you can change it AFTER you take the photo as long as you are shooting in RAW.



Just to be clear, though, you might want to adjust the white balance setting if you are trying to balance light from more than one source. May not be absolutely necessary, but it can help you judge the colors better and make adjustments, such as adding a gel to the strobe. RAW will let you change the overall color balance of the scene, but it can't do anything about individual light sources.


----------



## gmrza (Apr 21, 2011)

prestonpalmer said:


> A good habit you can get into now is NEVER changing the white balance setting on your camera, you can change it AFTER you take the photo as long as you are shooting in RAW. Set your camera to AWB, and leave it there....forever.


This may not work, from a workflow perspective, for a wedding photographer for instance:
A typical job may result in around 800 images. The closer those images are to perfect, straight out of the camera, the less work you have afterwards. - After all, your client does not pay you for the time you spend editing in Lightroom/Aperture/DPP etc.

As for shooting JPGs - I can understand why some wedding photographers shoot JPGs, but my wife and I do not subscribe to their point of view. (They probably don't have UDMA CF cards, and have not realised that storage is cheap.)
If you have a 1D/1Ds, then it makes absolute sense to write RAW to CF and JPG to SD. That way you have a level of backup in the event of a card failure.


----------



## neuroanatomist (Apr 21, 2011)

gmrza said:


> The closer those images are to perfect, straight out of the camera, the less work you have afterwards.



I'm not a wedding photographer and not under that sort of time pressure for processing images. But the problem with 'close to perfect' is that it's not perfect...but if it looks too close, human nature will tempt one to not go that last short distance to perfect. If the WB noticeably off, you'll probably get it right. But if it's close, you may leave it alone...


----------



## gmrza (Apr 21, 2011)

neuroanatomist said:


> gmrza said:
> 
> 
> > The closer those images are to perfect, straight out of the camera, the less work you have afterwards.
> ...



You've probably hit the nail on the head: it's horses for courses. Architectural, landscape and commercial work needs to be perfect (i.e absolutely perfect) - so it pays to spend the time on WB.
For a wedding, after your client has selected the 60 or 70 photos from the 400 odd initial proofs, it is worth working on the 60 to 70 photos that make it into the album, more so for that 20"x30" enlargement that your client is paying you hundreds of dollars for. That is where I think the wedding photogs who shoot JPG will fall short - when it comes to making those large prints RAW gives you the edge to show details you will not see on the screen or in a small print - probably about 1 stop worth more shadow detail (YMMV).
For wedding formals, it is often even worth it to give the bride a white balance card to hold first - that way that framed print that hangs over the mantlepiece will be perfect because you can get the white balance absolutely right in lightroom afterwards. (Forget trying to do that with JPG.)


----------



## skitron (Apr 21, 2011)

Thanks everyone for your replies. I'm a software developer and also have a background in audio so I 'get it' about the data and meta data being seperate, using a lossless format etc and it all makes perfect sense. 

So now that I understand that is what is going on, do all of the in-camera processing functions get applied to the RAW's meta data or just some of them? Just trying to get a practical handle on the capabilities so I can put together a workflow that fits me. 

Also, are there any differences in processing quality for any given function (say noise reduction for example) between in camera processing and the Canon software?

Thanks again everyone for chiming in, it is very helpful.


----------



## neuroanatomist (Apr 21, 2011)

skitron said:


> ...do all of the in-camera processing functions get applied to the RAW's meta data or just some of them? Just trying to get a practical handle on the capabilities so I can put together a workflow that fits me.
> 
> Also, are there any differences in processing quality for any given function (say noise reduction for example) between in camera processing and the Canon software?



They all get written into the metadata, yes, including things like which AF point was selected, etc. But, the .CR2 format is proprietary, so even though settings are written to the metadata, they cannot necessarily be read by your processing software. DPP is from Canon, so it can read everything (you can display your AF point for each shot in DPP, to continue with the example). But things like Picture Style, ALO, etc., are not read/used by 3rd party software (which may offer similar options, e.g. a 'portrait style', but they aren't the same as Canon's, and if you set one in-camera, the 3rd party software won't apply their version by default). Hope that makes sense.

Yes, there are differences in processing between in-camera and DPP. In the camera, the RAW file must be converted to JPG (which includes NR, etc.) using only the DIGIC processor which, while sophisticated, is relatively weak (compared to a computer's CPU), and it must be done fast enough to keep up with the frame rate of the camera. DPP running on your computer has a lot more processing power, and still takes a lot more time - it's doing a better job of the JPG conversion.


----------



## dougkerr (Apr 21, 2011)

skitron said:


> My understanding of RAW is that it is simply the data "as-is" read from the sensor and saved on the memory card with no processing applied. However, I found that the color temp as set in a 50d directly affects the color temp of the saved RAW image. Are other in-camera functions applied to RAW as well?


The white balance color correction setting in the camera does not have any effect on the raw data.

However, that setting is saved (in a technically-detailed way) in the metadata of the raw file.

Then, when the raw data is "developed" in external software to a JPEG or TIFF file, color correction is then applied, as set by the operator in any of various ways. One option is for the operator to tell the software to use the color correction that was set in the camera (which the software can read from the metadata) - the same color correction that the camera would have (or did) use in the in-camera generation of a JPEG file for the shot. But that is an explicit choice by the operator.

Best regards,

Doug


----------



## te4o (Apr 21, 2011)

May I through in my questions about RAW too:

Are Picture Settings (Landscape, Portrait etc, i.e. in-camera sharpness, contrast etc. adjustments) applied in RAW or is this only for JPEG (i.e. heavier DIGIC-processed files) ?

Importing RAW in any RAW editor applies certain settings (software manufacturers work on this every day, skitron). Now, once imported in one RAW editor (DPP f.i.) can I get the original RAW file (as delivered from the camera) back out of the editor (not from the CF) like an export and import this "precious original Canon CR.2 RAW" into another RAW-editor (like LR, or Aperture). I know, this doesn't make me a better photographer, but would be interesting in changing RAW-editing systems and reworking some files.

Thanks


----------



## prestonpalmer (Apr 21, 2011)

neuroanatomist, thanks for the support and additional RAW info!

gmrza, I completely disagree with your theory on white balance and that "getting close to perfect" is better for workflow. I AM speaking from a workflow and experience perspective here! I shoot 40 or so weddings per year in addition to engagement shoots and other photo work, and delivering a $4,000 wedding shoot to a client in 10 days or less starting with 1800 or so unprocessed photos is truly a demanding task. Ive taken OVER 500,000 wedding photos, I can assure you, my workflow is EXTREMELY EFFICIENT! My photo editors all agree, leaving the AWB ALONE on default makes for MUCH MUCH quicker and more accurate post processing. It simply takes too much work chasing the WB and still having to correct it in the end anyway. As neuroanatomist pointed out, the JPG's "false" representation on the back of your camera doesn't help you anyway as you try to CHASE the right color temp. Remember the back of the camera is lying to you anyway. While the average photographer is fiddling with white balance in situations that are changing minute to minute, I am off taking beautiful photos, getting all the shots the fiddler is missing. AND in the end my product will be superior. every. single. time!


----------



## NormanBates (Apr 21, 2011)

not long ago I was told that RAW gets you a better RAW than RAW+JPG

I think that's wrong, but maybe someone here knows why this person had that idea in his mind; my guess is that it might have been so, maybe, in something like "nikon cameras from 1999", or some other early step in the evolution of RAW

anybody?


----------



## jeremymerriam (Apr 21, 2011)

neuroanatomist said:


> jeremymerriam said:
> 
> 
> > Just setting the color temp to AWB and forgetting it is not always a wise solution. RAW doesn't save all the gamuts of color that always completely represent the true colors.
> ...



Help me understand this then. When i shoot with the wrong white balance setting (lets say its set at AWB when the lighting is tungsten), why is there less play (fine tuning) room in lightroom when the image is overly red? I know what you said is supposidly right but getting the right color balance is much more difficult and always feeling muddier than when i get the right WB. In lightroom, it will go from being real warm to real cold in just a small (shiftkey + downarrow) move.

My processing goes much faster and to me looks better when I am closer to the mark. Maybe the original 5d isn't as accurate as the mark2 AWB. AWB is always way off on my camera in tungsten situations.


----------



## neuroanatomist (Apr 21, 2011)

te4o said:


> Are Picture Settings (Landscape, Portrait etc, i.e. in-camera sharpness, contrast etc. adjustments) applied in RAW or is this only for JPEG (i.e. heavier DIGIC-processed files) ?
> 
> Importing RAW in any RAW editor applies certain settings (software manufacturers work on this every day, skitron). Now, once imported in one RAW editor (DPP f.i.) can I get the original RAW file (as delivered from the camera) back out of the editor (not from the CF) like an export and import this "precious original Canon CR.2 RAW" into another RAW-editor (like LR, or Aperture). I know, this doesn't make me a better photographer, but would be interesting in changing RAW-editing systems and reworking some files.



Like most other settings (the exception being Highlight Tone Priority), the Picture Style setting for the shot is written to the RAW metadata without affecting the RAW image data. If you open the image in DPP, the in-camera setting will be automatically selected for the image (and applied during conversion), but you can also change it in DPP with no effect on the RAW image data.

RAW converters do not alter the RAW image data - none of them do, that's the point of a RAW file. So, you can always get the original RAW file back out. Of all the RAW editors, only DPP modifies the RAW file itself - but only the metadata. The other converters (ACR, DxO, etc.), write the conversion settings to 'sidecar' files (usually linked in a behind-the-scenes database). So, if you convert a RAW image with one converter, you can later process the image with a different converter and you'll be starting from the same _original_ image. 



NormanBates said:


> not long ago I was told that RAW gets you a better RAW than RAW+JPG
> 
> I think that's wrong, but maybe someone here knows why this person had that idea in his mind; my guess is that it might have been so, maybe, in something like "nikon cameras from 1999", or some other early step in the evolution of RAW



RAW gets you the same RAW as RAW+JPG (but shooting RAW alone gives more RAW than RAW+JPG - i.e., you have a higher burst capacity, although nowhere near as high as JPG alone). Not all RAW is created equal, though - for example, older Nikon cameras applied lossy compression to the RAW data (newer ones give you the option of compressed or uncompressed).



jeremymerriam said:


> Help me understand this then. When i shoot with the wrong white balance setting (lets say its set at AWB when the lighting is tungsten), why is there less play (fine tuning) room in lightroom when the image is overly red? I know what you said is supposidly right but getting the right color balance is much more difficult and always feeling muddier than when i get the right WB. In lightroom, it will go from being real warm to real cold in just a small (shiftkey + downarrow) move.



Great question - and I have no idea. ??? I'd guess it has to do with the way LR applies the WB setting from the metadata. Adobe software does other 'stealth' alterations (e.g. ACR applies some NR even if you have NR turned off). I haven't noticed any reduced latitude for color temperature alterations with AWB-shot tungsten scenes in DxO - I still have to drag the slider quite a ways to cool off the image.



jeremymerriam said:


> My processing goes much faster and to me looks better when I am closer to the mark. Maybe the original 5d isn't as accurate as the mark2 AWB. AWB is always way off on my camera in tungsten situations.



I think there's no 'right' or 'wrong' workflow - even though the in-camera WB setting does not affect the RAW image data, having it close may save time (e.g. if you know you're going to set a bacth of images to Cloudy in post, setting it in-camera saves a step in post). So, what goes faster for you is 'right' at least for you, but clearly not for prestonpalmer.

I don't know how far off the original 5D is with AWB under tungsten...but I really hope Canon _didn't_ try to improve it with the 5DII, because if they did try to improve it, they failed miserably.  (Or the 5D was way, way, way off, and they improved it significantly, leaving the 5DII only way, way off...so maybe the 5DIII will merely be way off?!?) Regardless, the 5DII delivers very orange images with AWB under tungsten.


----------



## skitron (Apr 21, 2011)

neuroanatomist said:


> Hope that makes sense.
> 
> Yes, there are differences in processing between in-camera and DPP...it's doing a better job of the JPG conversion.



That makes sense and I'll probably continue to shoot only RAW and just mess with the color temp and balance in-cam and leave the rest to DPP, That seemed to be working great and LOL now I know why. Thanks again for your input.


----------



## autochrome (Apr 21, 2011)

For reference, the tags associated with the CR2 files, and their descriptions (the ones that were reverse engineered so far that is), are included here:

http://lclevy.free.fr/cr2/

Also covered is information on the sRAW and mRAW format. It's mostly useful to software engineers and researchers (the section on demosaicing is specially juicy), but it goes into some depth in explaining some key concepts.


----------



## John Smith (Apr 26, 2011)

NormanBates said:


> not long ago I was told that RAW gets you a better RAW than RAW+JPG
> 
> I think that's wrong, but maybe someone here knows why this person had that idea in his mind; my guess is that it might have been so, maybe, in something like "nikon cameras from 1999", or some other early step in the evolution of RAW
> 
> anybody?



I don't see it helping any, beside a higher FPS.


----------



## skitron (Apr 27, 2011)

I posted this in the software section but will also follow up here since I started this thread.

I bought the PhaseONE Capture One Pro 6 RAW processor software and tried it out tonight. It renders in a totally different league than in-cam or DPP. The difference is literally bigger than getting a new L over a kit lens to my eyes, no kidding.


----------



## branden (Apr 29, 2011)

Cool -- post some examples! I'm eager to see. Thanks


----------



## bvukich (Apr 29, 2011)

branden said:


> Cool -- post some examples! I'm eager to see. Thanks



X2


----------



## skitron (Apr 29, 2011)

I'll see if I can post some shots over the weekend. The biggest two things I noticed are the color renditions and sharpening/NR.

Colors are absolutely amazing. I'll post a pic of one of my guitars that has 24K gold plated hardware that I never could get to look right until Capture One. Also, skin and hair tones same thing.

Second is the sharpening and NR. DPP will "out sharpen/NR" it at first glance, but when looking at details in the bokeh, DPP is not doing well at all in comparison. Meanwhile Capture One nails it and the bokeh still looks great when sharpen/NR and DPP never gets there even with as similar as possible settings. Also, DPP loses more detail at similar NR levels. I'll post crops of the afforementioned guitar that show it.

I would also encourage you to download their 30 day demo and try for yourself! They have two versions (pro, light) and either can be installed using the single download. Plus the demo can be activated without uninstalling/reinstalling.


----------

