# DXO tests the 50 f/1.8 STM



## ahsanford (Sep 2, 2015)

FYI if you hadn't seen.

http://www.dxomark.com/Reviews/Canon-EF-50mm-F1.8-STM-lens-review-Thrifty-fifty/Measurements-Very-good-optical-performance

You'll notice the test, like with all DXO lens reviews, was conducted on every body that Canon sells _...Except for the 5DS and 5DS R._ (Which we know they own -- they've already published those reviews.)

Someone at DXO is trying super hard to avoid having a good budget lens leap to the top of their absurd ranking system just because we now have a 50 MP sensor. Method #1 was to withhold all new lens testing until an a7R II review could be rushed into place, and now Method #2 appears to be avoiding lens testing at all on the 5DS rigs. Classy!

#DXO #fairandbalanced

- A


----------



## Deleted member 375103 (Sep 19, 2015)

#Thanks #4 #Posting #This. 

As for the 5Ds/5Dr, I guess the DxO guys are a little frightened with all those mega pixels stuffed inside a Canon.


----------



## ajfotofilmagem (Sep 19, 2015)

dilbert said:


> ahsanford said:
> 
> 
> > Someone at DXO is trying super hard to avoid having a good budget lens leap to the top of their absurd ranking system just because we now have a 50 MP sensor. Method #1 was to withhold all new lens testing until an a7R II review could be rushed into place, and now Method #2 appears to be avoiding lens testing at all on the 5DS rigs. Classy!
> ...


If members of CanonRumors deride DXO Mark is one more reason to prove us wrong. Why delay in showing how bad is the Canon lens in a body 50 megapixel?

Perhaps DXO is having a hard time trying to push dynamic range score on the lens ...
You know, right? There can only be one at the top of the mountain, and EXMOR have to make the performance of *lenses* be better.


----------



## JoFT (Oct 18, 2015)

For me the results are not reasonable: If I look at my 50mm lenses on a 7D2 with similar pixel pitch to the 5Ds - the f1.8 is way better... I compared it with my EF 50mm f1.4: with a clear winner: the cheaper one..


If someone wants to have a look: http://bit.ly/1OCuFi5


unfortunately the admins do not allow me to embed the photos here ;-(


----------



## PhotographyFirst (Oct 18, 2015)

It's pretty obvious DXO has an axe to grind with Canon. Anyone know why that is? 

With their stupid scoring system, the crappy-plasty-fifty would shoot to the top of their MP scores on a 5Ds. I don't personally care about anything DXO does in terms of how I decide on gear, but there are many people who don't know any better and will take them at face value. They have a very legit and professional looking website, which is deceiving. 

Read the title of this lens review for a good laugh, or cry. It get even better as you read the review. 

http://www.dxomark.com/Reviews/Canon-EF-S-55-250mm-f-4-5.6-IS-STM-lens-review-Updated-EF-S-telephoto-zoom-no-improvement-on-predecessor


----------



## beforeEos Camaras (Oct 18, 2015)

PhotographyFirst said:


> It's pretty obvious DXO has an axe to grind with Canon. Anyone know why that is?
> 
> With their stupid scoring system, the crappy-plasty-fifty would shoot to the top of their MP scores on a 5Ds. I don't personally care about anything DXO does in terms of how I decide on gear, but there are many people who don't know any better and will take them at face value. They have a very legit and professional looking website, which is deceiving.
> 
> ...



I guess I have to give up in despair all my l lens are no good and my regular lens need to be in the dust bin how was I so naïve on buying canon Sarcasm

wow I have not used that website and thank god I did not, even kens site is more forgiving.


----------



## 3kramd5 (Oct 19, 2015)

PhotographyFirst said:


> It's pretty obvious DXO has an axe to grind with Canon.



I highly doubt that is the case.


----------



## PhotographyFirst (Oct 19, 2015)

3kramd5 said:


> PhotographyFirst said:
> 
> 
> > It's pretty obvious DXO has an axe to grind with Canon.
> ...


I wonder how long it took them to test lenses on the D800 when it came out? 

It sure didn't take them long to get the sensor score up on their website for the 5Ds. If they had any neutral bias, I can't imagine why they wouldn't be super eager to see how the Canon lenses perform on the 50MP sensors. 

I guess DXO is balancing out the universe, since Bryan at TDP has a stiffy for Canon gear and often neglects the other brands in terms of priority. He doesn't bias his test data, but he sure biases his priorities. DXO seems to be the same but with a Nikon stiffy.


----------



## chromophore (Oct 19, 2015)

PhotographyFirst said:


> 3kramd5 said:
> 
> 
> > PhotographyFirst said:
> ...



Maybe you weren't around long enough to remember, but TDP originally was a Canon-only site. Bryan only had Canon gear to use and test. Later on, he expanded into Nikon products, in part because there was so little information on how the two brands would perform under comparatively similar testing conditions. This is why TDP continues to have a heavy emphasis on Canon products. It isn't that Bryan neglects the other brands--it is that he is predominately a Canon shooter and the site was originally intended as a resource for Canon photographers. It was not positioned as a general photography review site--unlike DXO, which purports to be an unbiased source of testing information across all brands, but is very clearly not.


----------



## ahsanford (Oct 19, 2015)

chromophore said:


> PhotographyFirst said:
> 
> 
> > 3kramd5 said:
> ...



Yep. TDP is a Canon resource first and foremost. Love the detail of his reviews and the content of the site.

I only wish Bryan Carnathan would curve his Canon grades to a 'B' and not the 'A' he always seems to give. He very very very rarely has a bone to pick with a Canon product, so I wish he would be a bit more critical. Stronger criticism of Canon products would would help us understand the purchasing tradeoffs better, and a rarer 'A' grade would really stand out as a sensational product. 

But as it is right now, _everything_ is great from Canon according to him.

Love his site, love that resource -- don't get me wrong.

- A


----------



## chromophore (Oct 19, 2015)

3kramd5 said:


> PhotographyFirst said:
> 
> 
> > It's pretty obvious DXO has an axe to grind with Canon.
> ...



This is complete hearsay, but I seem to remember some time ago that they tried to get Canon to give them some money in exchange for their reviews--i.e., some kind of endorsement. Canon refused. DXO basically retaliated. But again, I emphasize that I have not substantiated this and I make no claims to its veracity.

What is undoubtedly true is that DXO's testing and analysis methods are not statistically sound. I should know: I hold degrees in mathematics and statistics. It's my job. Anyone following my posting history here should take note that I am not mathematically illiterate. I actually have not yet seen any source of information regarding photographic equipment testing that I would regard to be statistically reliable. LensRentals comes very, very close: I would take some issue with the sample sizes, and the way they present some of the charts; I would also recommend a blinded approach to measurement and analysis. DXO, however...picking random numbers out of a hat would be less biased than their results.


----------



## ahsanford (Oct 19, 2015)

chromophore said:


> 3kramd5 said:
> 
> 
> > PhotographyFirst said:
> ...



The industry turns a blind eye to stats with testing. LR is relatively above-board there, but other sites which are considered more honest/transparent than DXO -- say, PhotoZone or LensTip -- just test one lens, and often only on one body. That's an inappropriate sample size for the metrics they are reporting on.

The problem with DXO is that, at _any_ level of scrutiny, you can find all sorts of loose ends to their arguments. Their methods, transparency, rating system and interpretation of results are all highly questionable. This forum is littered with comical examples of their mis-deeds to the fact-craving photography community. In particular, I cannot stand their lens reviews -- on their sensor work, they are slightly less irritating and are (in many cases) the only show in town.

- A


----------



## 3kramd5 (Oct 19, 2015)

PhotographyFirst said:


> 3kramd5 said:
> 
> 
> > PhotographyFirst said:
> ...



I don't know. I do know, however, that they haven't tested any lenses on the Sony A7R ii, and there is little indication DXO has an anti-sony bias.


----------



## K (Oct 23, 2015)

I gave them the benefit of the doubt being a new camera body, and with so many lenses to test and so many samples to collect it isn't something that is done overnight.

However, it has been *several months* and they've had ample time to put out some numbers and results.

At this point, this is* bias by omission*.

Being that they are Sonikon Shills --

Yes, they are waiting for high MP Sony/Nikon body results so that their glass will then be ranked high enough to then be covered over by other factors. They have to get close enough to the margin where their other measures can then pick up the slack and rank either equal to or better than Canon. They do this because Canon has a slight edge on optical quality. 

It's the same way they arbitrarily rank low ISO DR so heavily - this then picks up the slack for areas.

The more one digs into DXO, the more bias, inconsistency and flawed methodology one finds.


Folks, if you want to compare gear -- SEEING IS BELIEVING. Nothing, nothing, nothing beats analyzing RAW files of test shots on your own computer and and coming to your own conclusions. 

For all the hype about competitors to Canon, when I load that RAW file and go 1:1 - I'm sorry, but they are NOT better than Canon. They are all very close and comparable. What I see on the screen does not reflect or equate to the hype and hysteria out there.


----------



## jeffa4444 (Dec 1, 2015)

DXO = questionable testing methods. They would not last a day in the cinematography world.


----------



## zim (Dec 1, 2015)

So it's out
http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009

What does it mean by 'best at 2.8' ?
The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD


----------



## PhotographyFirst (Dec 3, 2015)

zim said:


> So it's out
> http://www.dxomark.com/Lenses/Canon/Canon-EF-50mm-F18-STM-mounted-on-Canon-EOS-5DS-R__1009
> 
> What does it mean by 'best at 2.8' ?
> The 40 pancake gets exactly the same score and comment, I have that lens and 2.8 certainly ain't its optimal ;DD


Also strange about the 50 STM. Why is Transmission tested at 1.9 T-Stop, yet they measure 3 stops of vignetting?


----------



## rfdesigner (Dec 3, 2015)

PhotographyFirst said:


> zim said:
> 
> 
> > So it's out
> ...



I take it you know Tstop is the light transmission figure... not the focal ratio. it accounts for lossy optics, which suggest 0.1 stops of loss which is really very good if true.

The Vignetting is measured as 2.56 stops at photozone, so 3 stops isn't that far off.

Personally I'd drop the vignetting figure, especially for comarative purposes, and change to "speed in corners".. so a f1.2 lens with 2 stops of vignetting (f3.2) will then beat an F4 lens with 0.5 stops of vignetting (f4.5).. which in reality it would do, comfortably.


----------



## JoFT (Dec 3, 2015)

For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:


On the 7D2:

EF 50 STM: 12 MP
EF 50 1.4: 10 MP
Sigma 50 Art: 13MP


On the 5DIII

EF 50 STM: 16 MPEF 50 1.4: 17 MPSigma 50 Art: 21MP

On the 5DsR

EF 50 STM: 29 MPEF 50 1.4: 22 MPSigma 50 Art: 40MP

For me this does not make any sense! Or does someone has a reasonable explanation???


----------



## Maximilian (Dec 3, 2015)

JoFT said:


> For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:
> 
> 
> On the 7D2:
> ...



Long answer:
The reason therefore is that DxO makes you believe they are testing lenses. 
But they are always testing combinations of lens AND camera body/sensor.

So the lens of course is behaving different when you're using an APS-C sensor (smaller image circle) 
or different FF sensors (different MP design).


Short answer:
Forget DxO scores. Go out and shoot photos 8)


----------



## ritholtz (Dec 3, 2015)

Maximilian said:


> JoFT said:
> 
> 
> > For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:
> ...



Are these sharpness measurement in terms of Mpix good indicator of performance even for combination. Does it mean 5DSR with 50mm stm always produces sharper pictures compared to cheaper FF camera with expensive 50mm lens? In that case, every one will be spending 5DSR body and cheap glass than spending big on lenses right.


----------



## ahsanford (Dec 3, 2015)

JoFT said:


> For me the DXO Results are fare away from being consistent. An eexcample just sharpnes:
> 
> 
> On the 7D2:
> ...



If the same single lens is being used in all three tests, the resolution values should roughly scale with the sensor (the 7D2 brings a crop wrinkle to that, however).

So these inconsistent results as you go from crop 20 MP, FF 22 MP, to FF 50 MP could be due to:


A different lens was used on one camera than on the other.
The lenses weren't properly calibrated on each body
The same lens was used, but it's been a few years since the 5D3 test and it took some wear and tear over that time
[Insert myriad reasons here]

The point is: no lens testing site is perfect. But ALL of them would benefit from testing multiple copies of a lens like LR usually does. Why we hang our hat on a single lens' performance is nuts.

- A


----------



## PhotographyFirst (Dec 3, 2015)

rfdesigner said:


> PhotographyFirst said:
> 
> 
> > zim said:
> ...



I know what I am saying. 

Vignetting directly affects the transmission value. A lens that has heavy vignetting, should have a much higher t-stop value compared to aperture value. Unless of course, the center of the image area somehow is much brighter than the f1.8 aperture. 

My beef with DXO is why they list the lens as having nearly perfect transmission when it has not even close to perfect vignetting control.


----------



## old-pr-pix (Dec 4, 2015)

PhotographyFirst said:


> rfdesigner said:
> 
> 
> > PhotographyFirst said:
> ...



I'm not going to defend DxO, but the calculation of T-stop value is based on the transmission of the lens on axis. I.E. through the center of the image area. Hence, vignetting will not impact the T-stop value unless it is so severe as to impinge on the central image area. 

One big problem I have with any DxO measurement is that they never fully disclose methodology, accuracy or measurement error and they only test one copy. I suspect in many cases the differences between two manufacturer's similar items tested may well be less than the +/- error of the measurement. By algorithm, DxO then takes these statistically insignificant differences and magnifies them by various weighting factors resulting in what appears to be significant difference. 

If we knew DxO's actual algorithm we could take some of the information that LensRentals finds in terms of copy-2-copy variation and plug it in to determine how wildly DxO's single number values would change.


----------



## PhotographyFirst (Dec 4, 2015)

old-pr-pix said:


> PhotographyFirst said:
> 
> 
> > rfdesigner said:
> ...



Like I said, I _don't _know what I am saying. 

Thanks for the info, I was not aware that the t-stop wasn't the average transmission of the whole frame.


----------



## Sabaki (Dec 4, 2015)

I have my bodies and lenses. Last thing I think of when I'm out in the field is whether my kit is qualified according to DXO or not.

Canon is a superb brand, do we really need a dubious body to qualify that for us?


----------



## Maximilian (Dec 4, 2015)

Sabaki said:


> I have my bodies and lenses. Last thing I think of when I'm out in the field is whether my kit is qualified according to DXO or not.
> 
> Canon is a superb brand, do we really need a dubious body to qualify that for us?



As I said before:


Maximilian said:


> Short answer:
> Forget DxO scores. Go out and shoot photos 8)


----------

