# 5Ds/5DsR & Lens resolution - anomaly (Lensrentals ref.)



## Andyx01 (Jun 17, 2015)

I want to bring to light an odd anomaly on some initial resolution tests published by lensrentals.

Ref: http://www.lensrentals.com/blog/2015/06/canon-5ds-and-5ds-r-initial-resolution-tests

Specifically the Canon 300 2.8 IS II, and Zeiss 85 1.4 Otus results.

Below are the Center sharpness scores of the two lenses on three different bodies:

_________ 300mm _________ 85mm_________ 85>300 result.
5d-III_____1050___________1300___________23.8%
5Ds_______1375___________1570___________14.2%
5Ds-r______1545___________1660___________7.4%


So here is the anomaly. If the center sharpness of the 300mm lens is within 8% of the 85mm lens on a high resolution sensor - one would think that as the sensor resolution DECREASED - the differences would become smaller, but the opposite is happening.

How is this score being generated?

What are the chances of something happening behind the scenes that is affecting results? e.g. in camera distortion correction.

I would like to see the 5D III - 300mm re-tested with the lens not fully seated, (or pins taped over) to eliminate anything that may be occurring due to the camera body identifying the lens.

Even if the results are the same, it brings in to question why this is happening?

It's like examining the paint on a car with a really good magnifying glass and saying paint A is 7.4% smoother than paint B, then having someone else examine the paint with a lower resolution magnifying glass and saying, no paint A is 23.8% smoother than paint B. If anything the magnifying glass that is of lesser quality, should have a harder time detecting the 7.4% difference resulting in a closer score, not a larger gap.

???


----------



## Andyx01 (Jun 17, 2015)

I just want to add the wonderful tool Bryan has over at the-digital-picture.com for comparing lenses:

http://www.the-digital-picture.com/Reviews/ISO-12233-Sample-Crops.aspx?Lens=739&Camera=453&FLI=0&API=0&LensComp=957&CameraComp=453&SampleComp=0&FLIComp=0&APIComp=3

To my eye, the center sharpness between the two (both at f/2.8) is roughly the same in the center (not 23.8% worse on the 300.)

While I did not include the corner results, the 300 did score better in the corners than the 85, and the images hosted by Bryan do appear to back that up.

But again, I want to focus on the center descrepancy, and where that score is coming from. It doesn't look right purely by the numbers, and it doesn't look right on the ISO chart either.


----------



## Eldar (Jun 17, 2015)

This is not my area of expertise, so I´m sure others can be more scientific in their response. My assumptions would be:

The sharpness from a lens-sensor combination will not be linear when you change the resolution of the sensor. At some point, when you increase sensor resolution, you reach the maximum resolution you can get from a lens. With the 5DSR you are moving up the lesser steep part of the curve and sharpness improvements will be less and less. The sensor is less of a limiting factor and the limitations in the lens more dominant.. On the 5DIII, it is the sensor resolution that is more of a limiting factor and you get full effect of the lens quality.

i would also guess that if you did the same measurement on a lower resolution sensor, like a 5DC, the curves would get closer again, since the lesser quality lenses would have enough resolving power.

Looking forward to the qualified answers


----------



## Andyx01 (Jun 17, 2015)

Eldar said:


> The sensor is less of a limiting factor and the limitations in the lens more dominant.. On the 5DIII, it is the sensor resolution that is more of a limiting factor and you get full effect of the lens quality.



Exactly! The 5D III sensor isn't nearly as discriminating as the 5Dsr. Thus the differences should be getting bigger. Not smaller.

i.e. if you Used a horrid lens on a full frame sensor that only had 1MP of resolution, then compared it to a near perfect lens, the differences would be limited by 1MP and thus not very big, but on a 50MP sensor, the differences should become huge.

Yet.... The opposite is being reported in these tests.

???


----------



## Eldar (Jun 17, 2015)

Andyx01 said:
 

> Eldar said:
> 
> 
> > The sensor is less of a limiting factor and the limitations in the lens more dominant.. On the 5DIII, it is the sensor resolution that is more of a limiting factor and you get full effect of the lens quality.
> ...


No, that is not it.

You´re right about the low resolution sensor, where you´ll see little difference, because of the limiting performance of the sensor. But with the 5DSR you are approaching the maximum performance of the lens and It is the limiting factor. So the performance curve for a lens-sensor combination, with sharpness on the Y-axis and sensor resolution on they X-axis will get an S-shape. The 5DIII sensor resolution will most likely be somewhere close to the steepest part of the curve.


----------



## mistaspeedy (Jun 17, 2015)

A possible explanation is the ability of the sensor to resolve fine detail.
What if the 5DsR and 5Ds sensors can only resolve detail up to about a score of 1700 and 1600 respectively ?
Think of this as a 'cap' or max value, no matter the lens.
Therefore, on the 300mm lens, they have no problem resolving all of the detail the lens gives them, because it is lower than the values of about 1600-1700.
But when presented with an extremely high quality lens like that Otus, they have hit their limit in resolving power.
The sensors have hit their resolution ceiling and are not resolving significantly more than they do on the 300mm lens.
Now for some calculations!
The Otus is resolving 23.8% more detail on the 5D mark III than the 300mm F2.8.
If we take that 23.8% value and apply them to the 5Ds and 5DsR numbers for the 300mm lens, we should get the following (what the 5Ds and 5DsR 'should be resolving')
5Ds with 85mm Otus: 1702 score [match below with 1701 score]
5DsR with 85mm Otus: 1912 score [match below with 1912 score]
I further confirm these two scores with the following 3 pieces of information listed on the site:
5D mark III with 300mm F2.8 = 1050
5DS with 300mm F2.8 = 1375 = + 30.9% over 5D mark III
5DsR with 300mm F2.8 = 1545 = 47.1% over 5D mark III

Let's now apply these 30.9% and 47.1% differences to the 5D mark III score for the Otus:
1300 + 30.9% = 1701 score
1300 + 47.1% = 1912 score
I'd say those numbers match up nicely!

So I have gone two different routes and gotten the same numbers.
I used the 5D mark III to compare 2 lenses = 23.8% difference
I used the scores on all 3 cameras with the 300mm lens to measure sensor resolving power... it seems that the 300mm lens is sharp enough to show these differences, but not sharp enough to run into a sensor limit.
What about the other Zeiss lens?
5Ds = 27.8% better than 5D mark III
5dsR = 41% better than 5D mark III
What about the Canon 50mm F1.4 ?
5Ds = 19% better than 5D mark III
5DsR = 21% better than 5D mark III

CONCLUSION:
The better the lens, the bigger the differences between sensors! [unless sensor limited]
With the 5DsR, we have:
21% better than 5D mark III on Canon 50mm F1.4
41% better than 5D mark III on Zeiss 21mm F2.8
47.1% over 5D mark III on Canon 300mm F2.8
The 85mm F1.4 Otus did not show these larger differences because we have hit a sensor-resolving limit.
Thank you Andyx01 for bringing this to my attention... you are all free to double-check my math, my logic or to bring your own conclusions!


----------



## Andyx01 (Jun 17, 2015)

Re: S-shaped

The resolving power of a lens is a fixed thing. It is what it is. It wouldn't plot out as a curve. (unless you were referencing from the center to a corner.)

Pretend the sensor had an infinite number of pixels, or something like a billion MP. The lens projection onto the sensor is what it is; it doesn't change. As the resolution of the sensor is reduced from a billion MP to 1MP the amount of detail you can record will be reduced as well. Plot the reduction on a chart and you would end up with a curve, (not an S curve.) The steepest angle of the curve would be limited to 45 degrees, this would be the area where resolution is the limit, the curve would go nearly flat as resolution continued to increase because the lens is only delivering so much.

e.g. horizontal = sensor res and vertical = optical res (see dots below)
.
.
.
.
.
.
.


Compare that curve to another curve from a lesser lens, and you would get a similar shaped curve.

As you work toward 1MP the differences would get smaller (not larger.)

That is my point.

The numerical value being reported as 'resolution' in the chart doesn't add up. Something is definatly skewing the results and no one seems to know what it is.


----------



## dcm (Jun 17, 2015)

Why don't you post your question in the comments on Roger's blog item and get a response directly from the source of the numbers? I imagine he can shed some light on it for you.


----------



## Andyx01 (Jun 17, 2015)

mistaspeedy said:


> A possible explanation is the ability of the sensor to resolve fine detail.
> What if the 5DsR and 5Ds sensors can only resolve detail up to about a score of 1700 and 1600 respectively ?
> Think of this as a 'cap' or max value, no matter the lens.



That's actually a good point! (Limiting factors - I'm making an assumption that the test chart isn't limiting things)

I still find it odd however that if a lens CAN score 1300 on a 5D III (which the 85 does) - that a competing lens that is nearly as sharp (on a high res sensor) scores so much lower. I almost want to say the contrast of the resolution is somehow affecting the resolution score. However -- If it was related to contrast (which is obviously better on the stopped down 85 per Bryans chart) it wouldn't explain why the 300mm score would rise significnatly on the 5Dsr (unless the contrast was less of a factor given the way the scoring system biases total resolution?)

It would be nice to know exactly how these scores come to be.

Like you point out though, perhaps given a better chart, the otus would score substantially higher? (the reason I say limiting chart, and not limiting sensor is because of the large discrepancy between the 5D III sensor and those two lenses.) The 5D III sensor is clearly limiting, yet the score difference is dramatic enough to take note. (esp considering a visual comparison (at least to my eye) shows no resolution difference, but it does show a contrast difference.)

If contrast is indeed the answer - how is the score weighted?


----------



## mistaspeedy (Jun 17, 2015)

Well, we both have different views on that 300mm lens from Canon... I straight up believe that the Zeiss is simply much better. There is only so much Canon can do with a lens that big... it is simply huge and costs a lot to make having that much glass, having autofocus, having image stabilization, having weather sealing.... the Otus does not have all these things, and the money can be invested into optical quality.

The HUGE Canon 300mm F2.8 costs $6100
The Otus 85mm costs $4500.


----------



## Andyx01 (Jun 17, 2015)

My (unconfirmed) take away from this is:

a) Resolution is measured and assigned a score (up to the resolving power of the test image and sensor.)

e.g. perhaps the 300, and 85 score the same score for for part (a) above.

b) Contrast is measured, and score is reduced accordingly

e.g. the 85 has better contrast and the score isn't reduced as much

Result: The 5D III the scores end up 1050 for the 300mm, and 1300 for the 85mm. (difference due to contrast)



Now increase the sensor resolution by going to the 5Dsr:

a) resolution is again measured but this time perhaps the 300 scores better then the 85 as they are no longer sensor limited.

b) the added resolution has no effect on the vignetting, and again the 300 takes a bigger hit than the 85 does.

The end result still goes to the 85, but by a narrow margin as the 300 scores higher in the resolution test before taking a contrast hit.



This is conjecture of course, but it's the only thing that makes sense. (although even this doesn't make a lot of sense when you bring corner scores into it *sigh*)


----------



## Andyx01 (Jun 17, 2015)

mistaspeedy said:


> Well, we both have different views on that 300mm lens from Canon... I straight up believe that the Zeiss is simply much better. There is only so much Canon can do with a lens that big... it is simply huge and costs a lot to make having that much glass, having autofocus, having image stabilization, having weather sealing.... the Otus does not have all these things, and the money can be invested into optical quality.
> 
> The HUGE Canon 300mm F2.8 costs $6100
> The Otus 85mm costs $4500.



The 300mm actually scores better then the Otus 85 though. (Only the center falls 7.4% short, the corner is actually scoring 18.7% better.) The average score goes to the 300mm by 4.8%

Not only does it score better, but it visibly looks better wide open compared to the Zeiss stopped down 2 stops.

Lucky for the Zeiss it's not the same focal length, or zeiss would be in serious trouble. (Edit: Actually they wouldn't be in trouble, I think enough people buy things based on how they feel physologically based on advertising, etc.)

But this brings up another point. (at f/2.8 )

The 85 scores better on a resolution limited body (1015) then the 300 does (920)

-however-

The 300 scores better on a high resolution body (1300) then the 85 does (1240)

Which goes back to the contrast v.s. resolution scenario.

What do you guys think?


----------



## mistaspeedy (Jun 17, 2015)

Well, my comments were for the 'center' sharpness mentioned in the beginning.

As far as these results you mention... the 300mm is simply sharper in the corners, bringing the average up.
The average for the Otus is being limited by the sensor.. since those 'center' scores should be significantly higher.

In short... they are both excellent lenses, with the Otus having better center sharpness, and the Canon having better corner sharpness.


----------



## longtallkarl (Jun 17, 2015)

one thing to remember is that image resolution is a function of lens and sensor (someone else can supply the actual equation if they wish - i just know i've read it on other threads.) so even if a lens scores a 1300 on a high resolution sensor that doesn't mean that it will on a lower resolution sensor, even if that sensor is capable of generating a 1300 with a higher quality lens. 

just like a higher resolution sensor will get more image resolution out of any quality lens, so too will the image resolution always fall when moving to a lower resolution sensor.

i think mistaspeedy has provided the correct answer.


----------



## Andyx01 (Jun 17, 2015)

longtallkarl said:


> one thing to remember is that image resolution is a function of lens and sensor (someone else can supply the actual equation if they wish - i just know i've read it on other threads.) so even if a lens scores a 1300 on a high resolution sensor that doesn't mean that it will on a lower resolution sensor, even if that sensor is capable of generating a 1300 with a higher quality lens.
> 
> just like a higher resolution sensor will get more image resolution out of any quality lens, so too will the image resolution always fall when moving to a lower resolution sensor.
> 
> i think mistaspeedy has provided the correct answer.



correct answer for what?


----------



## longtallkarl (Jun 17, 2015)

this:



mistaspeedy said:


> CONCLUSION:
> The better the lens, the bigger the differences between sensors! [unless sensor limited]
> With the 5DsR, we have:
> 21% better than 5D mark III on Canon 50mm F1.4
> ...


----------



## bchernicoff (Jun 17, 2015)

Another thought. Could false detail be an issue? Here is a quote from the Amateur Photographer review of the 5DS R (http://www.amateurphotographer.co.uk/reviews/dslrs/canon-eos-5ds-r-review/7)

"As we’d expect from a 50MP sensor with no optical low-pass filter, the 5DS R gives remarkable results in our resolution tests. At ISO 50 it resolves very close to its theoretical maximum of 5600l/ph, although with some aliasing and false colour around this point, *and plenty of false detail at higher frequencies.*"

Is it possible Imatest is getting confused? Lensrentals says in the comments that they decided not to publish Otis D810 numbers because: "There was a big enough disparity between the expected result on the Canon and Nikon systems that it through big red flags, so we did not publish that data."

This could be an indication that they are missing something important with their testing methodology.


----------



## Andyx01 (Jun 17, 2015)

bchernicoff said:


> Another thought. Could false detail be an issue? Here is a quote from the Amateur Photographer review of the 5DS R (http://www.amateurphotographer.co.uk/reviews/dslrs/canon-eos-5ds-r-review/7)
> 
> "As we’d expect from a 50MP sensor with no optical low-pass filter, the 5DS R gives remarkable results in our resolution tests. At ISO 50 it resolves very close to its theoretical maximum of 5600l/ph, although with some aliasing and false colour around this point, *and plenty of false detail at higher frequencies.*"
> 
> ...



Yeah something is definately not right. The fact that they don't post certain results because of a dramatic disparity with expectations along with scores that don't add up, really make me wonder.

Reminds me of Monitor Calibration scores.

I've seen horrible screens score dramatically higher then amazing screens due to the values assigned to where some of the plot points were.

If you buy into the score, you can increase your score by re-mapping look up tables.

Even though you now score higher with the adjusted table - the actual image quality suffered dramatically.

Without knowing how they are scoring these, along with the amount of objectionable data, leads me to question the value of these results.

Thanks for the post - makes me feel better about the wonky data


----------



## Andyx01 (Jun 17, 2015)

longtallkarl said:


> this:
> 
> 
> 
> ...



Keep in mind the Canon lens leveraged the increased resolution much better than the Otus on these tests. This would suggest it is the better performing lens as it was impeeded by a larger percent.

Additionally, if the sensor was the limiting factor, the two lenses would have been much closer on the 5D3. The fact that they weren't closer means something other than resolution was boosting the Otus score.

For example, the center scores:
_________ 300mm _________ 85mm_________ 85>300 result.
5d-III_____1050___________1300___________23.8%
+31% +21%
5Ds_______1375___________1570___________14.2%
+12% +6%
5Ds-r______1545___________1660___________7.4%




v.s. The corner scores:
_________ 300mm _________ 85mm_________ 85>300 result.
5d-III_____895___________835___________-7.2%
+20% +16% 
5Ds_______1070___________970___________-10.3%
+10% +2%
5Ds-r______1175___________990___________-18.7%



The ever increasing difference as the sensor density increases is how the center table should look as well.

The larger gains on the Canon mean it's leveraging the sensor better. The fact that the baseline score of the Otus is higher suggests another factor is boasting it up (less vignetting due to being stopped down? / contrast?)


----------



## bchernicoff (Jun 17, 2015)

Whoever suggested you post your concerns in a comment on the lensrentals blog post was right. I'd like to see if they respond.


No registration required to comment: http://www.lensrentals.com/blog/2015/06/canon-5ds-and-5ds-r-initial-resolution-tests


----------



## Ph0t0 (Jun 18, 2015)

mistaspeedy said:


> A possible explanation is the ability of the sensor to resolve fine detail.
> What if the 5DsR and 5Ds sensors can only resolve detail up to about a score of 1700 and 1600 respectively ?
> Think of this as a 'cap' or max value, no matter the lens.
> Therefore, on the 300mm lens, they have no problem resolving all of the detail the lens gives them, because it is lower than the values of about 1600-1700.
> ...



So the bigger the sensor resolution the smaller the difference between lenses?
So are we expected to see the biggest difference between good and bad lenses on sensors with fewer MP?




mistaspeedy said:


> So I have gone two different routes and gotten the same numbers.
> I used the 5D mark III to compare 2 lenses = 23.8% difference
> I used the scores on all 3 cameras with the 300mm lens to measure sensor resolving power... it seems that the 300mm lens is sharp enough to show these differences, but not sharp enough to run into a sensor limit.



Wouldn't the lenses run into the sensor limit even sooner with the low MP body?


----------



## mistaspeedy (Jun 18, 2015)

Well, the original title of this thread is quite good, since it states 'anomaly'.



> So the bigger the sensor resolution the smaller the difference between lenses?
> So are we expected to see the biggest difference between good and bad lenses on sensors with fewer MP?



The data seems to show the opposite, the bigger the sensor the resolution, the bigger the difference between lenses. Which sounds logical as well.
However, we have an anomaly with the scores from the Otus... which I was trying to explain somehow.

If we change the sensor scores for the 5Ds and 5DsR to higher values on the Otus, then everything makes sense, is logical and is as-expected.

That's why I concluded that there was something limiting the scores on the sensor side (which doesn't have to be true).

Just trying to make sense out of something that doesnt make sense (the Otus results with 5Ds and 5DsR)


----------



## bdunbar79 (Jun 18, 2015)

<Wouldn't the lenses run into the sensor limit even sooner with the low MP body?>

No! Higher resolution sensors will be able to resolve the diffraction sooner (wider apertures, all else equal).


----------



## mike b (Jun 18, 2015)

Andyx01 said:


> bchernicoff said:
> 
> 
> > Another thought. Could false detail be an issue? Here is a quote from the Amateur Photographer review of the 5DS R (http://www.amateurphotographer.co.uk/reviews/dslrs/canon-eos-5ds-r-review/7)
> ...


Hi All
please refer to my otus 55 review comments and also to my comment in the lens rental blog on filter stack thickness problems for what i believe could be causing SOME of the irregularities ........ie that the otus could be generically designed with nominal filter packet size and not specifically for each versions individual size.While controversy reigns this is my somewhat tongue in cheek take on the latest cameras.

Conan Photographic Corp camera body dept meeting in the recent past:
“Money”grunted the ageing president to a hushed and reverent audience not used to meeting the esteemed man. “While it is true that the Conan corp international group has had so much success in so many other fields that we have perhaps overlooked your small department i thought it wise to conduct some extensive surveys” piped up the newly appointed and enthusiastic CEO.”I sent a couple of people to hang around the street corner near that Nokin Pony place and do you know what ,they saw a lot of familiar faces popping in to see their latest cameras”.”Money” repeated the ageing president conscious(but only barely) of the approaching deadline for the next huge payment to the aptly named “Immortality Gardens”built in his honour.He acutely remembered his cousin Monte Olympus’s memorial gardens being reduced to couple of old pot plants and he certainly wasn’t going to allow that to happen to him. The lanky R and D chief slowly rose”I think i have a solution .Why don’t we just glue a couple of 7d sensors together,a bit of extra computing power and we could have er let me see……50.6 megapixels”.The ever so slightly slimey marketing boss immediately shrieked “ OMG 50 megaaaaapixels “ at the thought of all the marketing opportunities and then promptly wet himself.The cool headed and down to earth accountant finally took the stage .” Well yes but we have all those old mk3 bodies that nobodys going want anymore and the new body will never be designed if you lot in R and D don’t stop playing computer games and do some work , so lets pop it in them, problem solved”.Charlie ,the junior in R and D couldn’t contain himself any longer and needing at least to say something ,jumped up and blurted out “ we could like have 2 models like,sort of like one with and sort of like one without ”.The marketing boss looked as if he needed hospital treatment as he salivated over the idea of selling two models maybe even one of each to the same person.The R and D chiefs head slowly crumpled into his hands while computing the potential problems of aa filter removal,focus stack thickness and a myriad of other thoughts.
HAS CHARLIE THE JUNIOR BITTEN OFF TOO MUCH?
IS THE R and D CHIEFS WIFE GOING TO SEE HIM ANYTIME SOON?
HAS CHARLIE STILL GOT A JOB?
STAY TUNED FOR A THRILLING FINAL EPISODE


----------



## Andyx01 (Jun 19, 2015)

mike b said:


> Hi All
> please refer to my otus 55 review comments and also to my comment in the lens rental blog on filter stack thickness problems for what i believe could be causing SOME of the irregularities ........ie that the otus could be generically designed with nominal filter packet size and not specifically for each versions individual size.While controversy reigns this is my somewhat tongue in cheek take on the latest cameras.



I used to wonder why the 5Ds-"r" was more expensive then the 5Ds

I beleived the 'r' stood for 'removed' filter.

And figured - If the filter wasn't installed, wouldn't it cost LESS not MORE!

However!!! After further review, it turns out the "r" does not stand for 'removed'. Instead, they install ANOTHER filter on top of the existing filter, to effectively cancel it out.

Why would they do this? It was suggested it was done for cost reasons -- however all this talk of stack thickness make me wonder if it was instead done for this reason.

So much conjecture


----------



## Andyx01 (Jun 19, 2015)

bdunbar79 said:


> <Wouldn't the lenses run into the sensor limit even sooner with the low MP body?>
> 
> No! Higher resolution sensors will be able to resolve the diffraction sooner (wider apertures, all else equal).



Sorry, it was bugging me.

Yes! - lenses will run into the sensor limit earlier as the sensor resolution falls.

As resolution is increased, the ability of the sensor to record additional information is increased, this also means that diffraction, and other anomalies become more defined where they were otherwise obscured by inadaquate sensor resolution.


----------



## mistaspeedy (Jun 19, 2015)

> However!!! After further review, it turns out the "r" does not stand for 'removed'. Instead, they install ANOTHER filter on top of the existing filter, to effectively cancel it out.



Canon did exactly the same thing with the 5Ds and 5DsR that Nikon did with the D800 and D800E... one with a low pass filter, and the other with a low pass filter, but another filter to cancel it out.


----------



## bdunbar79 (Jun 19, 2015)

Andyx01 said:


> bdunbar79 said:
> 
> 
> > <Wouldn't the lenses run into the sensor limit even sooner with the low MP body?>
> ...



I'm sorry, maybe I'm just misunderstanding, but I haven't a clue what you mean. I'm not at all sure what you mean by "limit" and "sooner." So I'll summarize my statement:

As sensor resolution increases, all else equal, you will be able to resolve the diffraction earlier, at wider apertures. 

400 f/2.8 at f/6.3 on sensor A (where A = pixel density) may not "see" diffraction at f/6.3.

But put the same lens at f/6.3 on camera 2A and you might very well resolve the diffraction because pixel density is twice as much. 

So as pixel density increases, diffraction becomes more apparent at wider apertures, all else equal. 

But it doesn't matter, you always get a NET gain in resolution. You may not hit the theoretical twice the resolution gain due to diffraction, but it will be a gain nonetheless.


----------



## Andyx01 (Jun 19, 2015)

bdunbar79 said:


> Andyx01 said:
> 
> 
> > bdunbar79 said:
> ...



Technically you won't always experiance a gain in image resolution as sensor resolution is increased because of the chance of increased noise with smaller pixels. This is part of the reason for the lower ISO cap on the 5Ds/r, and the reason a 5D Mk IV is also in the works. However you are correct, electronic noise aside: more or less - yes; higher sensor densities will display more detail.

Diffraction is a part of every image at every aperture. In order to actually see / observe / record the effects of it, the following guidline is provided. Please note that guideline is simply when it becomes noticable to an average eye at an average print size with a specific sensor density (size / mp)

subjectively - diffraction typically begins to be noticed to varing degrees as follows:

Crop	MP	FF

f/1.4	364MP	f/2.2
f/2.0	182MP	f/3.2
f/2.8	96MP	f/4.5
f/4.0	48MP	f/6.4
f/5.6	24MP	f/9.0
f/8.0	12MP	f/12.7
f/11.3	6MP	f/18
f/16	3MP	f/25.4
f/22.6	1.5MP	f/36
f/32	750KP	f/50.8
f/45.2	375KP	f/72

For what it's worth: 720P HD TV is 922KP, and 1080P HD is: 2MP


----------



## bdunbar79 (Jun 19, 2015)

Yes I agree about the per pixel noise level and that is why I specifically always leave that out of diffraction discussions. I agree with your FF numbers presented; I haven't studied crop sensors, but that makes sense to me.


----------



## Bdube (Jun 20, 2015)

Hi All,

This is Brandon from OLAF Optical Testing. I wish to shed some light on this discussion and hopefully put it to rest. 

First and foremost I would like to address the suspicion as to why we tested against the D810 but did not post the results. 

The system that produces these results contains many elements which cannot be separated from each other, in order we have the chart, the lens, the operator, the imaging sensor, the camera's internal processing, the raw converter, and the software. 

The chart is hopefully not a limiting factor. Honestly I do not know if it is, Roger may know for sure, but I do not think it is if it is printed at at least 300ppi knowing its size. The lens, operator, sensor, raw converter, and software are all limiting factors and are imperfect. 

Skipping over the lens briefly, consider the sensor which is itself a system. First we have the stack which contains the color filter array, IR and UV cut filters, and the optical low pass filter. Someone here has wondered why canon goes through the effort of canceling the OLPF instead of removing it altogether - I will explain that first. 

An OLPF is traditionally made by using a thin piece of a birefringent material. These materials cannot be coated to reduce reflections and have relatively irregular surfaces which can cause quite a bit of scattering. The most suitable materials are only bifringent in one plane, so you can thick of it as splitting "1" into "2" vertically oriented initially. The IR/UV filter is 'plain' coated glass and can have its reflectance reduced to ideal levels, so it comes next. After it comes the second layer of the OLPF. In a 'functional' OLPF the same trick as the first layer is used, but it is oriented in the other axis, turning vertical "2" into square "4." For a canceled OLPF as in the 5Dsr they essentially install the front filter upside down to cancel the first filter. This is imperfect, but it keeps the optical path length the same which is critical. 

If you have a converging beam, as exists between the lens and its capture device, the path length for the "center" ray is shortest and the path grows longer as you look to the more peripheral sections of the beam. If you look at this lens diagram - http://i.imgur.com/j4wYtY7.png - and specifically the blue rays, the upper and lower ones have a longer path than the middle one. *The OPL of the sensor stack is not constant across manufactures, this is a major source of error when comparing the same lens model across camera brands - we do not know which thickness the lens is designed for.*

If you alter the optical path length (which is essentially a form of "apparent length" to light) of the sensor stack, those upper and lower rays will not form the same common focus they used to form with the middle ray, and you get spherical aberration. Thus, keeping the OPL constant is important. The CFA has a simply impact on MTF from a practical standpoint, from a physical standpoint it doesn't have one.

Ignoring the color filter array, which is cemented to the sensor itself and cannot easily be removed, the sensor itself then has a "sum" MTF. The sensor MTF, like a lens' MTF, may be represented as a fourier transform and we can plot contrast vs frequency. In all imperfect systems the MTF will be curved - not linear. If you look at fig 2.3 here - http://spie.org/x34358.xml - note that the sensor MTF hits 0 at a particular frequency and then bounces off zero for a bit. This false detail is what produces aliasing, etc. An ideal OLPF would "brickwall" filter at the zero point, but this is an impossible goal to realize in terms of manufacturing if not design. 

The system MTF is simply the multiple of its parts, so SensorMTF*LensMTF=SystemMTF. This is essentially what is being sampled by programs like Imatest (though this is a gross oversimplification). 

Worth noting here that films also have their own MTF curves, as sensors do. 

From there you get to the raw converter, which does some form of processing to turn a patterned grid of red-corresponding, green-corresponding, and blue-corresponding dots into full RGB matrix dots. This is an uncontrollable factor without writing your own software - the truth is we don't know how dcraw, embedded in Imatest, Canon's DPP software, Adobe, etc, do this except that they use the "best" method they can come up with. *This is a major source of error, even if the converter is standardized to a single software.*

Moving onto the software - the first stage of using Imatest is to run a software calibration and an alignment calibration. The 300 II is nice here because the camera mounts to it instead, so tilt, etc, usually do not need to be adjusted between cameras. This returns an alignment error quantity - our standard is to have <0.1deg of tilt. We know tilt will damage the result (another source of test error) but we do not know how much, and imatest cannot tell us because they are also unaware of how much difference it makes, is it linear, etc. 

Once we get past this result, the software calculates a contrast ratio based on the spread of lines oriented along various angles w.r.t the sensor normal. There are several algorithms internal to the software which adjust the raw number to calibrate various errors in the calculation method and known phenomena, but again they are not something we can control. *We do not endorse Imatest or claim that its calculation method is bulletproof, the best method available, etc. These calibrations if not done properly are another source of error. Lenses approaching the diffraction limit should require different corrections than those very far from it - this is a very large source of imprecision.* 

----

Returning to the lens - the lens' MTF is calculated via a fourier transform of a known source. In the case of design software, the spot is simulated and gets FFT'd. In this article I wrote for photozone recently you can see a manufacture curve in the internal industry standard format - http://www.photozone.de/aberrationsExample

Essentially MTF vs freq gives more information than sampling MTF at specific frequencies and plotting against image height, so we use a different format. 

Note that the curve - especially the blue curve - bounces off of zero. This is false resolution due to phase error in the lens. All non-diffraction-limited lenses do this. The better the lens, the higher the spatial frequency it happens at. Usually it is higher than the spatial frequency of the sensor being used. More importantly, *the curve is not linear and without knowing it explicitly, how a lens "scales" to different sensors is not something one can even begin to hypothesize accurately.* If the 5D3 and 5Ds were both sampling at a spatial frequency in the large, flat, near-zero region of that graph the lenses would appear to improve linearly as sensor res rises, but even the rather soft EF 50mm f/1.4 flattens out like that at a spatial frequency higher than the 5D3 can sample. 

_With the D810, we basically saw an extremely unnatural scaling between D810:5Dsr with the otus that was not expected. Due to dcraw not being updated and adobe camera raw not being updated, we had to use DPP to process the 5Dsr files and could not guarantee processing integrity. We also do not know which stack thickness the otus is optimized for and therefore which camera has more "lens error" - so the results are invalid anyway._ 

Between the 5D3 and 5Dsr we are also using different converters, dcraw for the 5D3 and DPP for the 5dsr. In terms of operator error we are using focus bracketing, MLU, etc, but thousands of lenses move through the room this test is done in every day and there are lots of vibrations that can muck with things. The 300 measures better with IS on consistently than with it off. Even with focus bracketing we cannot achieve precisely the same focus between bodies, only that we are at the best of several sample focus points. I.e nuanced differences should not be considered. 

This is why we did not post them, nor do we really ever intend to. Speaking frankly and in my own opinion, Imatest and all other camera-based lens analysis tools are crude and poor. We often measure lenses on the MTF bench or OLAF and they look terrible - but they look as good as any "good copy" on camera because the camera is not sensitive enough. Imatest is only useful to us now for comparing cameras, but it is deeply flawed for that purpose. (end opinion).

In terms of Otus vs 300mm comparison, here is the plotted raw data that we used to determine the otus on the 5Dsr was a "good copy" - https://dl.dropboxusercontent.com/u/39289306/TestOtus.pdf - _this is at f/1.4, as are all our MTF measurements for the foreseeable future._ Here is a different otus that tested worse, so we did not use this copy - https://dl.dropboxusercontent.com/u/39289306/ExampleOtus_2.pdf. 

Regarding Amateur Photographer.co.uk - we know little about their testing. We know that:

* if they are backing up between cameras, they are not using a frame-filling chart or their chart isn't high enough res, so already there is a source of error

* refocusing the lens, esp. with asymmetrical (not aspheric, asymmetrical) internal focusing lenses will change performance

* are they using sharpening?

* The non-L 100mm USM is a poor choice of lens for testing cameras with - at the nyquist of the 5Dsr the Zeiss 100mm makro for instance delivers over 2x the resolution on an MTF bench, and has lower sample variation to boot. 

* do they have a "good copy" for this - how do they know?

Our bench measurement are made without any kind of flat in the path, so they do not reflect 1:1 performance on camera, but they do allow us to compare all lenses with "infinite" sampling resolution. This particular 4-rotation chart is mostly internal. They need to be too big to show on the blog, but they allow us to very quickly determine if a lens is tilted, etc, so we can verify that every single copy we average is a "good copy". Soon we are launching a new measurement set that we have recorded 220 different lens copies for as of today and will expand by over 100 copies per week at least through august. We will present graphs in this format - average MTF - https://dl.dropboxusercontent.com/u/39289306/EF%2014mm%20f2.8_avg_.png - and the variance plot w/ score - https://dl.dropboxusercontent.com/u/39289306/EF%2014mm%20f2.8_var_.png. We do not yet have enough measurements of different otii to comment on that lens, but this is a $2,000 canon L lens to give you an idea of how big the range between copies can be. *Bad copies are already removed from this.* I will not explain the charts much - that comes soon during the launch post. Lots of computation goes into the variance plot and it is too much to explain now. 

"Will we retest at aperture xxx, with lens yyy, etc etc" - Roger Aaron and I will not, it is not happening. Imatest is *very* frustrating to work with because there is lots of math done behind the scenes that is not transparent, too many sources of error are not well understood by *anyone* using the software - not even its makers, and setting up the tests takes forever. 

Right now there are over 9,000 items waiting to be inspected during peak season and we have several inspection techs out. We cannot afford the time to test more, and we cannot guarantee result accuracy because of the flaws in camera-based testing. 

As frequently as we can we are testing lenses on the MTF bench. It is a "pure" testing methodology and we have control over everything necessary to guarantee accuracy down to +/-0.005. In the range of 0-1.0, this is astronomically smaller than what is possible via something like imatest.

Please let me know if you have any other questions about the test, future numbers, our methods, etc, and I will answer as best I can.

Warmest regards,
Brandon
OLAF Optical Testing


----------



## mistaspeedy (Jun 20, 2015)

Thanks for your detailed reply. It is always good to hear from someone with knowledge and expertise in the field. As always, I appreciate the time and effort that you and others put into similar tests so that the consumers have an insight into the products and how they are tested.


----------



## meywd (Jun 20, 2015)

Thanks a lot Brandon for taking the time to explain all this.


----------

