# DXO Test on 16-35mm from sony



## xps (Oct 14, 2017)

DXO Tested the Sony 16-35 GM lens:
https://www.dxomark.com/sony-fe-16-35mm-f28-gm-lens-review-highest-rated-wide-angle-zoom/

How can you explain the big gap in sharpness compared to the 16-35 III from Canon?


----------



## Larsskv (Oct 14, 2017)

My 16-35 LIII is way sharper than any of the four 24-70LII lenses I’ve had, yet DXO says the 24-70 scores 32 in sharpness on the 5Dsr, and the 16-35LIII scores 24. 

My 16-35 LIII is as sharp as my 35LII, maybe sharper at f5.6. The DXO score does not correspond with my experience.


----------



## Mt Spokane Photography (Oct 14, 2017)

The first law of lens testing says not to compare tests on one model with tests on a different model camera. That law does not apply to France, of course, they have their own set of rules.

There is also the small matter if using a undefined number to rate sharpness rather than the standard resolution used by the rest of the world. What is Perceptual Mpix? As far as I know, its a rating that is determined by a secret formula that no one else knows or can repeat.


So, wait for other testers to weigh in with their test methods that are repeatable and at least comparible across testers. Roger at Lens Rentals will eventually look at the results using just the lens with his tester, that takes the camera out of the equation, so its another data point.

I have no reason to believe that it is not a super lens, Canon does compromise between producability, repairability, and pricing and leans heavily toward profitability. I think that Sony makes a lens more expensive to produce, very difficult and expensive to repair, and has a fat profit as well, that's a common denominator.

Companies buying equipment often look at a thing called Life Cycle Costs. I suspect that the Sony cost over its life is going to be a huge difference. For Enthusiasts who do not use a product day in and day out, its likely a lesser cost, but, if they drop one, they will wish it were insured.


----------



## AlanF (Oct 14, 2017)

Roger has done proper MTF tests. The Canon looks superior. 
https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/


----------



## raptor3x (Oct 14, 2017)

AlanF said:


> Roger has done proper MTF tests. The Canon looks superior.
> https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/



Could be due to sample variation in DxO's copies or it could be coming from the Sony or Canon performing differently at short focal distances versus infinity.


----------



## Mt Spokane Photography (Oct 14, 2017)

AlanF said:


> Roger has done proper MTF tests. The Canon looks superior.
> https://www.lensrentals.com/blog/2017/08/sony-fe-16-35mm-f2-8-gm-sharpness-tests/



Thanks for the link. Looking at his display of the wide variability of 35mm performance, its possible that a single Sony lens could be better than a single Canon lens.

Its the expected result of issuing a single test value that is done using a secret formula to give a nonsense value.

As other lens testers weigh in, we may see a lot of variability at the 35mm focal length, but they should all look good at 16mm.


----------



## AlanF (Oct 14, 2017)

Yes. Drawing conclusions from testing a single copy of one lens vs a single copy of another is in the realm of fake news.


----------



## BillB (Oct 15, 2017)

raptor3x said:


> AlanF said:
> 
> 
> > Roger has done proper MTF tests. The Canon looks superior.
> ...



Could also have to do with DXO's use of different camera bodies with different sensors for the two lenses, along with secret, and quite possibly unreliable, assessment methods.


----------



## Jopa (Oct 16, 2017)

raptor3x said:


> AlanF said:
> 
> 
> > Roger has done proper MTF tests. The Canon looks superior.
> ...



Also could be due to a few dollars (euros, yens, russian roubles, etc...) coming from Sony


----------



## SecureGSM (Oct 16, 2017)

What are they smoking over there at DXO Labs?

Canon 16-35 III is so sharp it is reduculous!
Ignore them. They are total nuts. 
I ran FoCal calibration on my 16-35 III and was shocked how sharp the lens is.
The Mark II lens is quite soft at 35mm end but Mark III is a stellar performer. 



BillB said:


> Could also have to do with DXO's use of different camera bodies with different sensors for the two lenses, along with secret, and quite possibly unreliable, assessment methods.


----------



## Talys (Oct 16, 2017)

SecureGSM said:


> What are they smoking over there at DXO Labs?
> 
> Canon 16-35 III is so sharp it is reduculous!
> Ignore them. They are total nuts.
> ...



One thing that I dislike about DXO's lens ratings is that they do not consider the autofocus consistency. 

My favorite lenses aren't just really sharp when they're mounted on a tripod and I have the luxury of making sure that it's perfectly focused. My favorite lenses have highly reproducible autofocus, where I can rely on AF being in sharp focus every time. Or, in Reikan FoCal terms, the curve "fit". It doesn't really matter to me on a lens that I'm not manually focusing, if it's really sharp, but it only AFs to really sharp 1 time out of 4.

As an example, my Canon 50/1.8 and 50/1.4 have terrible AF consistency, and my Sigma 150-600 is slightly inconsistent. So I CAN get amazing sharp images out of them. But AF inconsistency makes AFMA much less useful -- sometimes -8 gives me a perfect image; sometimes +3 gives me a perfect image.

In comparison, my 24-70 or 70-200 or 100-400 all have dead-on consistency, so once tuned, I can be very confident about it. But this is never considered in DXO tests.


----------



## ahsanford (Oct 17, 2017)

Talys said:


> One thing that I dislike about DXO's lens ratings is that they do not consider the autofocus consistency.



+1. I've owned/used/rented three f/2.8L zooms and got nothing but flawless AF performance from them. I continue to marvel at how fast/accurate/consistent my 70-200 f/2.8L IS II is.

And Sony has the gall to court L-lens loving Canonites with FBW lenses. : I'm sure they are sharp for the money they are charging for them, but LensTip -- the only show in town for fairly consistent AF testing -- doesn't have this 16-35 G Master or many other recent offerings to report on AF consistency. I'm actually curious to see how they perform.

- A


----------



## SecureGSM (Oct 17, 2017)

Are you saying there is a AF consistency problem with MILCs now? 
There is none in my books 
What about MF lenses? Zeiss Otus glass is second rate now? ;D


ahsanford said:


> Talys said:
> 
> 
> > One thing that I dislike about DXO's lens ratings is that they do not consider the autofocus consistency.
> ...


----------



## Jopa (Oct 17, 2017)

SecureGSM said:


> Are you saying there is a AF consistency problem with MILCs now?
> There is none in my books
> What about MF lenses? Zeiss Otus glass is second rate now? ;D
> 
> ...



It's possible (but very rare) with OSPDAF in the AF-S (one shot) mode. The camera showed a green OSPDAF square on my subject but the focus was somewhere else. I think a pure CDAF or DPAF should not have this problem.

Also it depends on the implementation. Sony is focusing at the chosen aperture, and it's good to prevent any focus shift, but if the lens doesn't have any - the result won't be as sharp as focusing wide open and then narrowing the aperture down. Diglloyd once complained about this issue. There is no perfect solution...


----------



## ahsanford (Oct 17, 2017)

SecureGSM said:


> Are you saying there is a AF consistency problem with MILCs now?
> There is none in my books
> What about MF lenses? Zeiss Otus glass is second rate now? ;D



Have you _seen_ the AF testing on the Otus? Wow. Worst AF LensTip ever tested.

- A


----------



## Ryananthony (Oct 17, 2017)

I was suraised to realise that both the Sony 16-35/4 and 16-35/2.8 both extend going through the zoom range. I've never shot with an ultra wide that did that.


----------



## ahsanford (Oct 17, 2017)

Ryananthony said:


> I was suraised to realise that both the Sony 16-35/4 and 16-35/2.8 both extend going through the zoom range. I've never shot with an ultra wide that did that.



The 16-35 f/4L IS and 16-35 f/2.8L III's front element moves with zooming, but not like a 24-whatever standard zoom does. It moves behind the front rim of the outer lens barrel. So it doesn't actually change physical length as you use it. 

You can see what I mean here:
https://www.the-digital-picture.com/Reviews/Lens-Product-Images.aspx?Lens=1073&LensComp2=0&LensComp=949
(mouseover the non-hooded 'select view' options below the lens shots to see what I mean)

Can't speak for which 16-35 for Sony you are referring to (I get mixed up between A / FE Sony options and the Zeiss made FE options), but I believe one of them actually trombone extends like a 24-something zoom but in reverse -- 16mm is the longer configuration while 35 is the shorter configuration.

- A


----------



## SecureGSM (Oct 17, 2017)

that was a joke of course.



ahsanford said:


> SecureGSM said:
> 
> 
> > Are you saying there is a AF consistency problem with MILCs now?
> ...


----------



## jeffa4444 (Oct 26, 2017)

Working as I do in an exacting industry where lens results are projected onto movie theatre screens in everyday motion picture shoots the testing conducted by DXO is questionable. 

All our lenses are measured against the theoretical specification we applied at the design stage where we manufacture lenses. Copy to copy variance will happen mainly due to mechanical set-up rather than optical but even optically you can have a variance albeit normally quite small. The testing is done on an MTF machine and on optical projectors which are calibrated and checked often in a controlled room (we also test f / T stops). The camera is separately checked for flange focal depth & sensor alignment and then finally the lens is tested on the camera both on test charts and projected images from a live feed. Of the tools I would say the optical projector is just as important if not more so than the MTF machine and simply testing a lens on a camera for a company like DXO is amateur. 

Ive tested my copy of the Canon EF 16-35mm f4L IS USM on the projector and projected in our theatre on a 5DS, yes its one example but the results are extremely good and we test lenses from all the major brands and our results are at odds with DXO.


----------



## 3kramd5 (Nov 16, 2017)

The “right” way to use DXO’s lens numbers is to see how various lenses perform on camera bodies you’re interested in, or vice versa. Comparing across mounts is pointless.


----------



## aceflibble (Nov 22, 2017)

ITT a load of people who don't understand how DxO works, what DxO intends, or have even bothered to _attempt_ to read DxO's explanations of their measurements.

DxO's measurements for the 16-35 mk III line up with my experience using seven different copies of that lens, on three different bodies. Seven across three is a pretty decent sample size in my book—far more than the average consumer will get through—and when that experience matches up with DxO's numbers (which, contrary to what some eejits in this thread have insinuated, DxO _do_ explain), I see little reason to doubt them or give it a second thought.

As for why the 16-35 and 24-70 test so differently, you'll find that is the nature of all wider angles when pitched against longer focal lengths. Longer lenses inherently have finer resolving power as details are larger and not all smashed into a smaller pixel count. Hence all the lenses rated toward the top end of sharpness are >50mm and very few wide-angle lenses manage to resolve detail matching more than about two thirds of any given sensor's pixel count. (With a few scattered exceptions.)

When you zoom in, details get bigger and more easily-defined. Who ever would have thought?


----------

