# At long last, DXO aggregates all lens testing together



## ahsanford (Dec 22, 2015)

Weekes after DXO reported the 5DS R lens testing, the lens test results are finally now in 'gen pop' with the rest of their lens testing in their tables and rankings:

http://www.dxomark.com/lenses#hideAdvancedOptions=false&viewMode=list&yDataType=rankDxo

Joyous DXO magic of note:


The sharpest 9 lens + body combos in their database are on the 5DS R, yet only 4 of those cracked the DXO top 10 in their absurd lens ranking system.


In the following instances of a 'near identical' lens from Zeiss / Sigma being offered on both the EF or Nikon mount, we have the following wonderful findings:

*Zeiss Otus 55mm*
On Nikon (D800E): Sharpness = 33 P-MPix, Overall Score = 50
On Canon (5DS R): Sharpness = 37 P-MPix, Overall Score = 46

*Zeiss Otus 85mm*
On Nikon (D800E): Sharpness = 35 P-MPix, Overall Score = 49
On Canon (5DS R): Sharpness = 41 P-MPix, Overall Score = 47

*Sigma 35mm Art*
On Nikon (D800E): Sharpness = 30 P-MPix, Overall Score = 43
On Canon (5DS R): Sharpness = 34 P-MPix, Overall Score = 41

So if the lenses are effectively identical, one would look at the sensor/mount as a root cause for a lower score. I'm sure someone more versant with DXO's pentagon-like level of transparency can explain how sensor-stack-y transmission differences might explain that scoring discrepancy. Or it's just DXO being DXO.

In fairness, there were _some_ less high profile Sigma / Zeiss lenses that the Canon so comprehensively outresolved the Nikon that even DXO had to give Canon a higher score. 


- A


----------



## neuroanatomist (Dec 22, 2015)

The Lens Score is based on shooting at base ISO in 150 lux, and clearly a SoNikon sensor means the lens is better at shooting in a dimly lit warehouse.


----------



## Don Haines (Dec 22, 2015)

neuroanatomist said:


> The Lens Score is based on shooting at base ISO in 150 lux, and clearly a SoNikon sensor means the lens is better at shooting in a dimly lit warehouse.



For those readers unfamiliar with how bright 150 lux is, think of a very dark overcast rainy day and that's about the brightness DXO tests under.... sunlight has a brightness from about 50,000 lux to a bit over 100,000 lux.

Another way of putting it is to imagine living your life looking through a pair of glasses with ND-9 filters.... Do that and you can see the world like DXO.....


----------



## rfdesigner (Dec 22, 2015)

They really do need to publish their formula, it's getting them so much ridicule.

I have the 100F2.0 myself, chosen ahead of the 85f1.8 partly due to being sharper wide open, the 85 is known for being just a touch soft wide open. But DxO rates the 85 as sharper and then goes on to score the 100 as better overall... 

http://www.dxomark.com/Lenses/Compare/Side-by-side/Canon-EF-100mm-F2-USM-on-Canon-EOS-5DS-R-versus-Canon-EF85mm-f-1.8-USM-on-Canon-EOS-5DS-R__798_1009_241_1009

Until they publish their magic scoring formula I'll suspect the defining factor for any lens score is how much they had to drink at lunch.


----------



## rfdesigner (Dec 22, 2015)

Don Haines said:


> neuroanatomist said:
> 
> 
> > The Lens Score is based on shooting at base ISO in 150 lux, and clearly a SoNikon sensor means the lens is better at shooting in a dimly lit warehouse.
> ...



Or living in the UK... I did see the sun a few weeks ago mind.


----------



## ajfotofilmagem (Dec 22, 2015)

It is very simple:

A certain lens when mounted on a body with wide dynamic range, magically becomes a better lens.


----------



## zim (Dec 22, 2015)

Maybe Sony sensors are flater


----------



## StudentOfLight (Dec 22, 2015)

Intriguing that the Zeiss 135mm f/2 is the highest scoring Zeiss lens on the 5DsR but on the D800E it is only the third highest Zeiss lens. I would have expected the relative positions to remain the same when moving from one system to another.


----------



## RGF (Dec 22, 2015)

rfdesigner said:


> They really do need to publish their formula, it's getting them so much ridicule.



Why? That would be no fun. Let's consider their test to be unbias and above question.

After all, it is on the web, so it MUST be true ;D


----------



## ahsanford (Dec 23, 2015)

StudentOfLight said:


> Intriguing that the Zeiss 135mm f/2 is the highest scoring Zeiss lens on the 5DsR but on the D800E it is only the third highest Zeiss lens. I would have expected the relative positions to remain the same when moving from one system to another.



Yep, and the 35L II loses in sharpness to the Sigma 35 Art on the 5D3 yet wins on the 5DS R.

N = 1 lens on 1 body = a statistical grab bag. 

- A


----------



## Sporgon (Dec 23, 2015)

dilbert said:


> Don Haines said:
> 
> 
> > ...
> ...



Dear Dilbert, low light doesn't equal high ISO. Have a good Christmas, Regards, Sporgon.


----------



## Rahul (Dec 23, 2015)

dilbert said:


> Don Haines said:
> 
> 
> > ...
> ...



I would assume that for testing sharpness, all testing is done at base ISO setting up the camera on a tripod. No? Or maybe you are right - DxO has lost its marbles and is testing for sharpness shooting handheld :


----------



## neuroanatomist (Dec 23, 2015)

Sporgon said:


> dilbert said:
> 
> 
> > Don Haines said:
> ...



Lol. Maybe he can ask Santa for better reading comprehension skills, which might help him seem less foolish. 



neuroanatomist said:


> The Lens Score is based on shooting *at base ISO* in 150 lux, and clearly a SoNikon sensor means the lens is better at shooting in a dimly lit warehouse.



A Lens Score that's strongly influenced by DR and color depth at base ISO makes perfect sense...to DxO, and almost no one else. DxO = BS.


----------



## StudentOfLight (Dec 23, 2015)

dilbert said:


> Don Haines said:
> 
> 
> > ...
> ...


No, Canon is not behind at high ISO. DXO ISO score does not reflect real world performance. Look at the discrepancy between what is seen in RAW Lab images from DP Review vs DXO ISO score. According to DXO the D800E should be 1/3 stop cleaner than the 5Ds R.


----------



## Don Haines (Dec 23, 2015)

dilbert said:


> Don Haines said:
> 
> 
> > ...
> ...


It's not a question of who works better at what levels, it's about picking a realistic level to use as the standard.

If it were me, I would have picked a level of somewhere around 25,000 Lux...... a condition that we often shoot at. At 150 Lux, most of us are grabbing flashes and turning on the lights.....


----------



## neuroanatomist (Dec 23, 2015)

Don Haines said:


> At 150 Lux, most of us are grabbing flashes and turning on the lights.....



Especially if for some reason you were forced to set your camera to base ISO and a 1/60 s exposure. But somehow DxO believes those are the most relevant conditions to test lenses.


----------



## ahsanford (Dec 23, 2015)

StudentOfLight said:


> dilbert said:
> 
> 
> > Don Haines said:
> ...



It all depends on whether you are talking about noise at a pixel level or at some normalized resolution. 5D3 has less noise than 5DS at a pixel level, but if you downsample the 5DS to 22 MP, the two are quite similar (some argue the 5DS is better in that comparison).

So at a pixel level, the D800E _should_ outperform the 5DS, shouldn't it? 

But your DPR crops that were size-normalized for comparison purposes represent a greater downsampling of the 50 MP image than the 36 MP image, which would reduce noise in that comparison. So those RAW clips you posted can be misleading depending on the context.

Carnathan covers this pretty well.

Comparing a 5D3 to a 5DS R (natively):
http://www.the-digital-picture.com/Reviews/Camera-Noise.aspx?Camera=792&Test=0&ISO=1600&CameraComp=979&TestComp=0&ISOComp=1600

Comparing a 5D3 to a 5DS R (downsampled to 22 MP):
http://www.the-digital-picture.com/Reviews/Camera-Noise.aspx?Camera=792&Test=0&ISO=1600&CameraComp=979&TestComp=1&ISOComp=1600

- A


----------



## StudentOfLight (Dec 23, 2015)

ahsanford said:


> StudentOfLight said:
> 
> 
> > dilbert said:
> ...


DXO score is based on a same-size scaling, so DPReview's "equalized sizes comparison" is valid image comparison to DXO, as in both cases the images are scaled to the same relative sizes.


----------



## StudentOfLight (Dec 26, 2015)

dilbert said:


> StudentOfLight said:
> 
> 
> > ...
> ...


What about the pictures that I posted support what?
What are you seeing that I am not seeing?


----------



## neuroanatomist (Dec 26, 2015)

StudentOfLight said:


> dilbert said:
> 
> 
> > StudentOfLight said:
> ...



Someone looking through dilbert-goggles could be seeing anything from noise where there is none to lenses that are cameras to pink elephants flying around. 

Friends don't let friends dilbert-goggle!


----------



## rfdesigner (Dec 26, 2015)

dilbert said:


> Don Haines said:
> 
> 
> > ...
> ...



Canon loses purely on "colour depth" which is basically Colour SNR. If the debayering process includes strong negative values (i.e. Green = -0.5xR+1.3*G-0.6*B) then the noise is amplified, but only for colour situations not B&W. If you have wider filtering on colours as canon does which seems to yield better skin tones then you lose out to sharper colour cut offs used on the sony sensors.

I'll happily go with more chromatic noise to get better skin tones. On the monochrome high ISO measurements Canon sensors are equal to Sony.


----------



## jthomson (Jan 15, 2016)

Must we keep flogging the DxO horse?
Their score is a joke. 
Their software isn't.


----------



## Fstopkid (Jan 16, 2016)

ahsanford said:


> [*]In the following instances of a 'near identical' lens from Zeiss / Sigma being offered on both the EF or Nikon mount, we have the following wonderful findings:
> 
> *Zeiss Otus 55mm*
> On Nikon (D800E): Sharpness = 33 P-MPix, Overall Score = 50
> ...



Long time reader, first time poster.
I have a question about the above.
What accounts for the loss of P-MPix in Canons vs. Nikons cameras with the same lens?
I mean with the above numbers, for example the Otus 55mm shows 33 P-Mpix on the Nikon D800E and 37 P-Mpix on the Canon 5DSR. Sure 37 P-Mpix is greater than 33 P-Mpix, but thats 13 P-Mpix loss on a 50 MP sensor, vs. only 3 P-Mpix loss on a 36 MP sensor. Thats a 26% loss on a Canon over a 8% loss on a Nikon, if there was a similar 8% loss on a Canon it should show 46MP sharpness no?
Surely there can't be 26% loss between sensor to output???


----------



## Don Haines (Jan 16, 2016)

dilbert said:


> Don Haines said:
> 
> 
> > ...
> ...


Your point is valid, but the point that I was trying to make is that rather than to test lenses and cameras under a very contrived set of conditions, in this case lighting so poor that we photographers are reaching for flashes and the light switch, to test the devices under more typical conditions.

If I am trying to decide which camera or lens to choose, I want to know the relative merits of each under the conditions that they will be used under. For that, DXO fails everyone....


----------



## PureClassA (Jan 21, 2016)

Fstopkid said:


> ahsanford said:
> 
> 
> > [*]In the following instances of a 'near identical' lens from Zeiss / Sigma being offered on both the EF or Nikon mount, we have the following wonderful findings:
> ...



Welcome to the DxO mystery hour!! Let me blow your mind even further. Go look up lens scores for the Same Zeiss glass on the 5D3... Go look up several of Canon's top ranking L glass. The loss ratio is far lower. Somehow the loss ratio magically jumped to what you said on the 5DSR...


----------



## PureClassA (Jan 21, 2016)

Almost immeasurable loss. 1MP. 5% at most on the 5D3.


----------

