# Histogram question regarding clipping



## kat.hayes (Aug 18, 2017)

On a 5DM3, how do you know if it is clipping.

1. Does the histogram have to touch either the left side to clip the blacks or the right side to clip the highlights? Does it matter how high the spike is, or is it clipped if it touches the edge of either side regardless of spike height?

2. Can it clip if it comes close to either side but the spike is real high, but it is not technically touching either side?

Thanks in advance.


----------



## mnclayshooter (Aug 18, 2017)

Think of a historgram as a stack of blocks, each block representing a pixel, or in the case of the very small histogram we can see on the camera, a representation of a subset of pixels. All of the blocks are arranged by the color value or luminosity of that set of pixels and they are sorted and stacked in matching groups. There's really no limit to height as the height is relative to all of the other stacks of blocks within that image. Width is what describes gamut from pure black to pure white. IF the image is pushed perfectly up against either end, in theory, it still could be not clipped. Think of a frame that is all white with one single blue pixel (or any other color)... the histogram for that image would be entirely stacked at the furthest right with a very tall spike and one relatively very small bump further to the left. 


For a more normal scenario, in general, if you are seeing an overall average of the image pushing more to the right or more to the left in terms of the overall "shape" of the histogram, it could indicate clipping. A visual inspection of the image can also help to tell if some area is very dark/muddy or completely white and washed out. 


Sorry for such a round-about answer, but a histogram is what it is... a graphical representation of how often a specific color/luminosity occurs with in the image.


----------



## neuroanatomist (Aug 18, 2017)

1. Yes, but just because it touches doesn't mean clipping in the RAW file, because the histogram is based on the in-camera jpg (according to picture style, etc.). The height of the spike tells you how many of the pixels are (potentially) clipped. 

2. No, if it's not touching then no clipping is occurring (although if you are looking at the luminance histogram, you may be clipping in a specific channel; I recommend using the RGB histogram).


----------



## Joules (Aug 18, 2017)

I am not sure if this applies to all Canon cameras but at least on the 600D and 1100D the histogram is based on the JPEG displayed on the screen and therefore doesn't really show clipping at all if you shoot in RAW. In Lightroom, for example, the supposedly clipped information can be brought back to some extend from either end. While pushing shadows, the visible noise is more of a limit than the histogram I think.

I think this also applies to the 5D3, as neuro says, as Magic Lantern offers a feature called RAW Histogram, which shows a histogram based on the true RAW file, and I think I remember somebody with a 5D3 posting about using this feature on the ML forum.

If the question is about JPEG, i have nothing to say about that.


----------



## Ray-uk (Aug 19, 2017)

I would imagine the answer to raw/jpg histogram would be to take a single shot with the camera set to save in both raw & jpg and then compare the two histograms on the computer. I've never tried this but it could be interesting to see just how much of a guide the in-camera histogram is when you are shooting raw.


----------



## sulla (Aug 19, 2017)

Unfortunately, neither in-camera histograms nor Lightroom histograms are "true": They do not show clipping in the 14-bit world of Canon RAW, but in an 8-bit world (jpg). (for LR I am not 100% sure if it is 8bit, it is definitely not 14-bit, but I believe this is true also). This is a weird behaviour, because in my view the camera should present you with the true histogram: If you shoot RAW it should be RAW-based, if you shoot JPG... Perhaps ist has something to do with calculation power of the processor not being sufficient for histogram calculation form RAW, but then, this is not such a difficult process either. In LR this is also strange: If you export to JPG, an 8 bit histogram is ok, but if you export to 16 bit or if you edit an image, the histogram should be accurate.

To cut a long story short, no, there is now way to know (during shooting) if your exposure actually clips. In my experience, there will be full highlight details in the shot even with a clipped in-camera-histogram very often.


----------



## SecureGSM (Aug 19, 2017)

I am sorry, but I have to make a slight adjustment to your statement.

yes, you can tell reliably if your clipped highlights or not.
how:
1. step one: measure your camera's usable ( with safety margins ) dynamic range. for 6D is around 5.7 stops. yes, 5.7 stops.
step two: observe the scene and locate the brightest highlight area. do not worry about the sun, it is going to clip highlights there, no matter you tried.

step three: spot meter for your absolute highlights. get the reading.

your camera treats everything as 18% (mid grey) and will calculate exposure to place your absolute highlights in the dead centre of the histogram.

step four: take the reading of the step three and add half of your camera's usable dynamic range less 1/3 EV (safety margin) to the reading. in my case with 6D : 5.7 /2 = 2.85 EV, less say 0.35EV. that's +2.5EV for Canon 6D
My exposure compensation for the spot metered highlights will be : +2.5EV

2. birders will tell you to add +2 EV if you shoot bird in flight to compensate for the exposure bias. that's simplistic way of doing what I suggested in the method #1.

3. method number three (trial and fail):

blinkies (Overexposed highlights) indicators come _earlier_ than actual clipping occurs by approximately 1/3 EV on my 6D camera. 
take a shot, observe the image on lcd screen, look for the blinkies. if present , decrease exposure by 2/3 of a stop. take another shot. if blinkies gone, then increase exposure by 1/3 of a stop. take another one.
blinkies may return but that is fine. that's is what we need. done!

4. method four ( my preferred method when time permits): Buy good light meter (Sekonic, for example).

calibrate light meter to your camera, on location spot meter for highlights and save exposure to the light meter memory, then spot meter for shadows and save exposure to light meters memory.
based on this two reading evaluate the scene dynamic range. if the dynamic range of the scene is wider then you cameras usable dynamic range, than you have to use exposure bracketing.
if scene dynamic range is within the your cameras usable dynamic range, then set you absolute highlights to the right edge of your histogram , see method #1.
alternatively, spot meter for the grey card (18%) and use the reading to set your perfect exposure.

My preferred method for run and gun situation is method #3



sulla said:


> To cut a long story short, no, there is now way to know (during shooting) if your exposure actually clips. In my experience, there will be full highlight details in the shot even with a clipped in-camera-histogram very often.


----------



## snoke (Aug 19, 2017)

kat.hayes said:


> On a 5DM3, how do you know if it is clipping.



Use Magic Lantern. ML has raw histogram. Calibrate Canon histogram to ML raw histogram. Voila. Or just use ML everywhere. Choose.

ML not for your camera? Get better camera


----------



## sulla (Aug 19, 2017)

SecureGSM said:


> I am sorry, but I have to make a slight adjustment to your statement.
> yes, you can tell reliably if your clipped highlights or not.
> how:
> [...]


Indeed...
esp. method 4 allows for an exposure even without any camera metering... :
So I correct: "there's no (straightforward) way to tell _from a Canon Histogram_ whether your're clipping..."

The suggestion of using ML is a good one, actually.


----------



## Joules (Aug 19, 2017)

sulla said:


> In LR this is also strange: If you export to JPG, an 8 bit histogram is ok, but if you export to 16 bit or if you edit an image, the histogram should be accurate.


Why would a 16 bit histogram differ from an 8 bit one in terms of clipping? Would it differ at all?

I read so many people talking about file bit-depth as if it is linked to dynamic range, but it isn't, right? The bit-depth is just a number indicating the highest value a pixel of a given color could have (256 for 8-bit and 65536 for 16-bit). But a higher value doesn't have to mean a higher brigthness, that's a matter of how that value is interpreted. And the lowest value - 0 - should be treated as black, the highest - 256 / 65536 - as white. That's the same range of brightness levels, so the same dynamic range, just the amount of steps in between differs.

I do a bit of programming and plan to tackle a little image editor soon, so if I'm mistaken here, that would be helpfull to know.



sulla said:


> Perhaps ist has something to do with calculation power of the processor not being sufficient for histogram calculation form RAW, but then, this is not such a difficult process either.


The processors in Canon cameras can do a lot more than they do. As said before, the firmware addon Magic Lantern actually has a raw histogram that also shows the percentage of clipped pixels per color channel, as well as a raw spotmeter and raw zebras. All of it works perfectly fine on my T3i, even though it only has the old Digic 4 chip.

I would love to have ML come to some of the newer cameras at some point. Having it on a 6DII or 80D would certainly be cool.


----------



## sulla (Aug 19, 2017)

Joules said:


> I read so many people talking about file bit-depth as if it is linked to dynamic range, but it isn't, right?


Indeed, you are right. And actually an 8 bit JPG can cover about 11 stops of dynamic rage, as the mapping of brightness to bit-value is non-linear, but a gamma-curve.
I was sloppy with my wording. I somehow implied that high-dynamic-range image formats come in higher bit depths.
What I should have said was: _The histogram should cover the whole dynamic range of the image data. It does neither in Canon cameras nor in LR, which is sad._


----------



## SecureGSM (Aug 19, 2017)

there is a problem with ML implementation for DPAF enabled sensors, apparently.



Joules said:


> I would love to have ML come to some of the newer cameras at some point. Having it on a 6DII or 80D would certainly be cool.


----------



## Joules (Aug 19, 2017)

SecureGSM said:


> there is a problem with ML implementation for DPAF enabled sensors, apparently.
> 
> 
> 
> ...


I don't think DPAF has something to do with it, or do you have some link about that? I'm always interested in the Magic Lantern projects progress, although I don#t follow it as closely any more.

As far as I can tell, it works with the 70D, which has DPAF, although there are a lot of broken features. There's also work in progress on the 80D and it sounds like the issues with that are more related to the new processor and the new ways Canon handles their stuff.

There is a lot of handy stuff in ML, although it is the kind of stuff that I use rarely. So I'll stay hopefull.


----------



## SecureGSM (Aug 19, 2017)

thanks for the clarification. I am not a software developer and this is only what i have heard so far. hence I used word "apparently". I stay corrected.


----------



## snoke (Aug 19, 2017)

sulla said:


> What I should have said was: _The histogram should cover the whole dynamic range of the image data. It does neither in Canon cameras nor in LR, which is sad._



Read

http://www.adobepress.com/articles/article.asp?p=1930486

Lightroom histogram = ACR histogram. Same code.

Who tell you Lightroom histogram lie?


----------



## sulla (Aug 19, 2017)

snoke said:


> Who tell you Lightroom histogram lie?


Lie? In a sense of not-telling-the-whole-truth: yes.
Try the following experiment: From a perfectly exposed RAW, i.e. without over- and under-exposed parts (no clipping on either side), if you import it in LR and push the highlights/whites, then the histogram will clip, although the RAW contains full data and does not clip. Yes, LR histogram lies.
Following real situation: Image exposed such, that some highlight clipping is shown in-camera, but actually not in RAW. Import it to LR, and histogram will still clip. bring down highlights/whites will bring back all the details. Now histogram will not clip any more. So yes: LR histogram lies in some sense.

What the histogram should do: It should show the full scale of the RAW and then overlay to this borders on the high and low side which will show you the constraints of the current image format, JPG, say. This way you would see that there is no clipping in the RAW-data and that by bringing down highlights / pushing shadows you can bring those details back.


----------



## SecureGSM (Aug 19, 2017)

Sulla, not quite correct. The Canon RAW 14-bit file format contains an extra headroom in highlights area. So clipped highlights still can be recovered in many cases. That has nothing to do with Lightroom at all. Ask Private by Design. He should be able to explain this better.


----------



## Joules (Aug 19, 2017)

SecureGSM said:


> Sulla, not quite correct. The Canon RAW 14-bit file format contains an extra headroom in highlights area. So clipped highlights still can be recovered in many cases. That has nothing to do with Lightroom at all. Ask Private by Design. He should be able to explain this better.


I think all Sulla is saying is that he'd like to have a histogramm that displays the entire dynamic range of the RAW file, including this extra headroom, to know how much he can recover in post.

I'd like to have that too. When I go trough my RAWs, I sometimeshave to bring down their exposure a little before being able to select which one is exposed the best, because I can't tell in LR based on the initial histogram. (I shoot multiple exposure settings when I can, because i'm not great at it so I like to have some options).

That's also why I'd like to try putting together a little RAW editor that's tweaked for my taste. I'll try to implement an option for saving the images as non-debayered dng files, so that I can still do all I usually do in LR.


----------



## Mikehit (Aug 20, 2017)

On all canon cameras I have owned, I find that the raw file has about 1 stop headroom compared to what the in-camera histogram shows. And when viewing the image on the screen I will allow a small amount of blinkies and find that the data there is recoverable. 

The luminance histogram can be deceptive especially regards the red channel - I have had luminance histogram look fine but red poppies turn orange because the red channel has blown badly.


----------



## snoke (Aug 21, 2017)

Mikehit said:


> The luminance histogram can be deceptive especially regards the red channel - I have had luminance histogram look fine but red poppies turn orange because the red channel has blown badly.



Use RGB histogram, not luminance and you see channel clip.


----------



## snoke (Aug 21, 2017)

sulla said:


> snoke said:
> 
> 
> > Who tell you Lightroom histogram lie?
> ...



No. Who tell you lie about Lightroom histogram?

LR histogram = ACR histogram. Same logic.



> Try the following experiment: From a perfectly exposed RAW, i.e. without over- and under-exposed parts (no clipping on either side), if you import it in LR and push the highlights/whites, then the histogram will clip, although the RAW contains full data and does not clip.
> 
> Following real situation: Image exposed such, that some highlight clipping is shown in-camera, but actually not in RAW. Import it to LR, and histogram will still clip. bring down highlights/whites will bring back all the details. Now histogram will not clip any more. So yes: LR histogram lies in some sense.
> 
> What the histogram should do: It should show the full scale of the RAW and then overlay to this borders on the high and low side which will show you the constraints of the current image format, JPG, say.



Raw engine is same for LR & ACR. Both lie or not lie. Not one.

But you confused.


----------



## zim (Aug 21, 2017)

Can any topic in CR actually end without insults any more..... sigh


----------



## Joules (Aug 21, 2017)

snoke said:


> Raw engine is same for LR & ACR. Both lie or not lie. Not one.


As far as i can tell there is no debate here about whether or not the Lightroom and Adobe Camera RAW Histograms are the same. That is something you read into the conversation, so it seems you are confused. or am I missing something? I think it's common knowledge that both programs share some technical components, including the RAW engine. I see nobody claiming something else.

Sulla's point was that Lightroom's Histogram shows the information from the current state of the edited picture and not the information from the underlying RAW file. Which can be seen as a distortion of information, and therefore be called a "lie" in some sense. Another term that you brought into the conversation, by the way.

So what's your point? I feel like we're discussing a misunderstanding.


----------



## snoke (Aug 21, 2017)

Joules said:


> snoke said:
> 
> 
> > Raw engine is same for LR & ACR. Both lie or not lie. Not one.
> ...



I know what you say. CR forum say histograms different. Adobe say histograms same.



> Sulla's point was that Lightroom's Histogram shows the information from the current state of the edited picture and not the information from the underlying RAW file. Which can be seen as a distortion of information, and therefore be called a "lie" in some sense.



ACR is same. It show histogram after changes too. You never make change to image in ACR?


----------



## Joules (Aug 21, 2017)

snoke said:


> ACR is same. It show histogram after changes too. You never make change to image in ACR?


Precisely. The histogram for both programs changes as you make edits to your image. The reason for that is that the histogram on display shows the information of the edited picture. It shows the information of the output image, if you will. Just like the camera shows the histogram of the JPEG it would output.

Neither shows all the information from the actual RAW file. That's all that was ever said here. Do you disagree with that? It that case, run the experiment described by Sulla to see the proof. I still think there is a misunderstanding here.


----------



## sulla (Aug 21, 2017)

_I already wanted to retreat from this discussion, but now that I see it becoming more productive again, I somehow feel that I am obliged to reengage, as my earlier contributions lead to considerable misunderstandings. So, I would like to add this:_

I don't consider the "additional headroom of ~1 stop" in the RAW files over the in-camera displayed histogram an "additional bonus" you get from shooting RAW, but I consider the canon / adobe algorithm of calculating and displaying the histogram as hiding from the user ~1stop of dynamic range that is actually present in the sensor data. That algorithm is what I - and apparently Juoules, too - am not really happy with and that leads to the situation that there is no Canon way to know in the field whether an exposure acutally clips or not and that makes exposure-to-the-right quite difficult.

But we learned from earlier contributions that there is a MagicLantern way to tell right away whether sensor data clip.

I take away from this passionate discussion that I really should give MagicLantern a try. So far I unfortunately haven't.


----------



## scyrene (Aug 22, 2017)

zim said:


> Can any topic in CR actually end without insults any more..... sigh



Insults? I must have missed that. A fair bit of talking at cross purposes, which is par for the course.


----------



## privatebydesign (Aug 22, 2017)

A large part of the dissonance in this thread is based on an opinion that is not entirely correct, which is based on a misunderstanding of what Lightroom is actually showing you. In the Library module it is, effectively, showing you a jpeg in a small color space and the histogram reflects that, in the Develop module it is showing you something entirely different.

Lightroom does a lot of heavy lifting under the bonnet to make things seem effortless in the interface, trouble with that is it leads to many false assumptions. If you want to see your histogram from the RAW data look at the Develop module, the histogram there is the full depth information mapped into the Melissa RGB color space, which is the ProPhoto color range with an sRGB gamma curve applied (it is a custom color space used in the Develop and Print modules).

Here is an illustration. First screenshot from the Library Module with a clearly clipped (but missing spike) histogram, second screenshot same image no adjustments but from the Develop Module with highlights that are clipped but present.

So why does it do this?

The Library module is working in a different color space with a smaller gamut and the histogram is a reflection of the image as you see it, WYSIWY have. The Develop module is working in a much bigger color space and again is showing you a real time analysis of what is on the screen. By the way, the Develop module only does this for RAW files, it is quite amazing how much thought Adobe put into color management within the software so we don't need to give it a thought.


----------



## sulla (Aug 22, 2017)

I knew that the develop module uses different data than the library module, as, e.g., it builds its own preview, during which at first it displays the library-module preview. I did not know, however, that it uses a different colour space and that it displayed a histogram with a greater dynamic range than the library module. I never noticed that colours and tones shift as soon as the develop module switches from the library-module-preview to its own preview. Next time I'll try to observe that carefully.


----------



## neuroanatomist (Aug 22, 2017)

@sulla - for in-camera exposure, you might Google 'Canon UniWB'.


----------



## snoke (Aug 22, 2017)

sulla said:


> I knew that the develop module uses different data than the library module, as, e.g., it builds its own preview, during which at first it displays the library-module preview. I did not know, however, that it uses a different colour space and that it displayed a histogram with a greater dynamic range than the library module. I never noticed that colours and tones shift as soon as the develop module switches from the library-module-preview to its own preview. Next time I'll try to observe that carefully.



Try two monitor next each other. 100% sRGB and >99% AdobeRGB. Then you see what you don't see.


----------



## Joules (Aug 22, 2017)

neuroanatomist said:


> @sulla - for in-camera exposure, you might Google 'Canon UniWB'.


That's insanely helpfull suggestioon, thanks a lot! For non-Magic Lantern cameras and people who understandingly don't feel like trying it, that seems like a nice work around from my first little test.

I just gave it a try. I shot the blue, sunny sky at 1 second, f/1.4, ISO 100 to get a completely blown out image and used that to set my custom white balance. Works the way it should, now the Live View looks terrible but the histogram is a deal more helpfull.

I put my Rebel T3i on a tripod and aimed it at a big white cloud. I made three observations:

1) If I expose the image so that the Canon RGB LiveView histogram shows that I'm just shy from clipping highlights and take the shot, the histogram in the Image Viewer shows a great deal extra headroom, about 1.5 stops. The LiveView hsitogram is less wide than the Image Viewer histogram! They just cut away the right side! In other words, the LiveView histogram on the Rebel T3i / 600D is entirely useless for ETTR.

EDIT: Seems like additionally using the Technicolor CineStyle Picture Style reduces the difference between the Image Viewer and Live View histogram.

2) Taking shots at increasing increasing shutter speed (Longer exposure time) and looking at the Image Viewer histogram afterwards to determine the brightest exposure possible, I arrived at 1/1250 seconds, f/2.8, ISO 100. That's pretty much precisely what I would have chosen with the Magic Lantern RAW RGB histogram, which showed the clipping indicator, but without a number in it, meaning less than 1% of pixels were blown out.

Switching back to daylight WB and taking the same shot, the Image Viewer histogram touched the right border and the blinkies where on, so it thought the image was clipping even though it wasn't. Pulling back the exposure in LR, the cloud had tons of detail... I wonder if even the ML RAW histogram might not show the entire dynamic range ... a mostly white cloud isn't the best DR test subject though, is it? ;D

3) I have yet another dust spot on my sensor :-\ (I tried some smaller apertures before going to 2.8 )

Also thanks to privatebydesign, some more knowledge about Lightroom is very nice too. I guess I really have to look into color and the science behind it more at some point.


----------



## zim (Aug 22, 2017)

scyrene said:


> zim said:
> 
> 
> > Can any topic in CR actually end without insults any more..... sigh
> ...



yip, line was removed.
I only commented because I was really enjoying the thread, thanks to OP for posting!


----------

