# Why so long on the DXOMark and DPReviews?



## MarkB (Apr 9, 2012)

I got my 5d III the week before last. I am sure both those organizations got one from at least the first shipped batch. Why is it taking so long to come out with anything other than a preview? Are they running into issues? 

I am pretty happy with the 5d3, although the wife is a little mad about the price. 

I am really curious about how low the DXOMark score will be and if it is causing them to rethink their test. I just think they might worry it would damage what credibility they have if the nikon 800 had a 95 and the 5d3 had a 85 or so. They clearly are not that far apart unless the Nikon lenses are really holding them back. I don't think that is the case given he zeiss comparisons Lensrentals did.


----------



## JR (Apr 9, 2012)

Dpreview has just posted more sample picture from the camera and their test should come shortly. Most of these guys have a pre-production unit first which allows them to post first impression and preview quicklky after launch but they then always wait for to play with a production unit for a few weeks before publishing full reviews.

Dpreview still has not done full review of d4 and d800 either. As for dxo it should come soon since they too have put a overview...


----------



## prestonpalmer (Apr 9, 2012)

I was wondering the same thing.


----------



## JR (Apr 9, 2012)

Given how many potential buyers will pay attention to these type of reviews (Dp review and dxo) I am sure they want to take their time and if (god forbid!) they were to find something bad or odd, they likely would give a chance to Canon to respond or maybe provide another body. Remember these reviews, once completed, will serve as reference for potential buyers for the next 3-4 years. So while we are eager to see them, I can understand why they are not done in a "rush" way...


----------



## [email protected] (Apr 10, 2012)

Mho is that dxo might lean a little to the N ! The mark III has been out since the 22nd and still no review? Seems strange! But they were so quick to put up the d4 and especially the d800 that is still not even out yet? I think Nikon has a hand in their pocket to try to put an overwhelming positive buzz for it! Nikon doesn't want the same thing happening to them as last time when they released the 700 and a couple of months later the mark II was released! I don't really care what they score the mark III , won't sway me a bit as with most people. Fanboys just want bragging rights! Me personally I don't like the way Nikons feel in your hand! But canons IMO feels awesome! All I really wanted was a better focusing ! A 75meg file is pretty freakn huge! You are also going to need to invest in a new rig too!


----------



## XanuFoto (Apr 10, 2012)

[email protected] said:


> Mho is that dxo might lean a little to the N ! The mark III has been out since the 22nd and still no review? Seems strange! But they were so quick to put up the d4 and especially the d800 that is still not even out yet? I think Nikon has a hand in their pocket to try to put an overwhelming positive buzz for it! Nikon doesn't want the same thing happening to them as last time when they released the 700 and a couple of months later the mark II was released! I don't really care what they score the mark III , won't sway me a bit as with most people. Fanboys just want bragging rights! Me personally I don't like the way Nikons feel in your hand! But canons IMO feels awesome! All I really wanted was a better focusing ! A 75meg file is pretty freakn huge! You are also going to need to invest in a new rig too!


Nikon is a client of dxo so I expect them to have Nikon equipment before any body


----------



## XanuFoto (Apr 10, 2012)

JR said:


> Given how many potential buyers will pay attention to these type of reviews (Dp review and dxo) I am sure they want to take their time and if (god forbid!) they were to find something bad or odd, they likely would give a chance to Canon to respond or maybe provide another body. Remember these reviews, once completed, will serve as reference for potential buyers for the next 3-4 years. So while we are eager to see them, I can understand why they are not done in a "rush" way...


Do you think if someone is invested thousands in Canon glass and knows the new AF allows him/her to get more keepers, he/she is gonna wait for the dxo ratings. How many new buyers who are geeks rather then photographers who would care about the dxo ratings. Just wondering. Do you have any market research numbers on this?


----------



## marekjoz (Apr 10, 2012)

XanuFoto said:


> JR said:
> 
> 
> > Given how many potential buyers will pay attention to these type of reviews (Dp review and dxo) I am sure they want to take their time and if (god forbid!) they were to find something bad or odd, they likely would give a chance to Canon to respond or maybe provide another body. Remember these reviews, once completed, will serve as reference for potential buyers for the next 3-4 years. So while we are eager to see them, I can understand why they are not done in a "rush" way...
> ...



I'd say that there is quite a lot of people justifying their buy with statement "this is the best camera in the world" 
Also it may be easier for some to justify their work before a client with the same statement.
And according to previous discussion on DXO ratings, If I were conspiracy theory believer I'd say that we'll wait some more time for their test but only if no1 holder was to change after the test


----------



## [email protected] (Apr 10, 2012)

I did not know that about dxo. Makes since! So why is everyone so concerned with them scoring it? If they score it lower than the 800 most will say they are being bias!


----------



## Mt Spokane Photography (Apr 10, 2012)

[email protected] said:


> I did not know that about dxo. Makes since! So why is everyone so concerned with them scoring it? If they score it lower than the 800 most will say they are being bias!


 
The utility of a camera body depends to a great degree on ow it will be used. DXO does a lot of testing, and reading the test results, you can decide which camera is best for your use.

When they give a overall score, it reflects their values, and is easy to be biased toward a body that matches. Read their test results, and form your own opinion.

The same goes for the DPR final score. They have done some careful and intensive testing, and that takes weeks. It can also be held up by the need for software, DPP is broken, and ADR is a release candidate but not finalized. A good review should be based on final versions of editing software, no matter how long it takes.


----------



## JR (Apr 10, 2012)

I think some of the individual metrics DxO measures might be interesting like DR and ISO score. Yes we need to take them with a grain of salt, but since are not officially published by Canon - allbeit maybe no one might agree on the methodology - still interested to see the result personally.

Will it make me switch? No!


----------



## AnselA (Apr 10, 2012)

scrappydog said:


> [email protected] said:
> 
> 
> > If they score it lower than the 800 most will say they are being bias!
> ...



I may be alone here, but only if the testing design was purposely shaped around what Nikon is strong at, would I agree with you. After that, I have to believe this company just honestly carries out their tests.


----------



## iso79 (Apr 10, 2012)

I didn't wait for either and enjoying my MKIII immensely ;D


----------



## well_dunno (Apr 10, 2012)

Mt Spokane Photography said:


> It can also be held up by the need for software, DPP is broken, and ADR is a release candidate but not finalized. A good review should be based on final versions of editing software, no matter how long it takes.



+1 to that


----------



## Tcapp (Apr 10, 2012)

scrappydog said:


> AnselA said:
> 
> 
> > I may be alone here, but only if the testing design was purposely shaped around what Nikon is strong at, would I agree with you. After that, I have to believe this company just honestly carries out their tests.
> ...



If i owned that site, I would take advertising dollars from whoever wanted to give it to me. If being biased and taking money gives them a bigger bottom line, you can bet your butt thats what they are going to do. If being unbiased and attracting a few more visitors made them more money, then they would turn that advertising down. 

It's all about money. DXO is just like any other business.


----------



## JR (Apr 10, 2012)

I just hope they are beeing impartial in the way they actually perform the tests...


----------



## Tcapp (Apr 10, 2012)

scrappydog said:


> Tcapp said:
> 
> 
> > It's all about money. DXO is just like any other business.
> ...



People are often unable to form their own opinions. Sad state of affairs.


----------



## Viggo (Apr 10, 2012)

I'm out writing my own opinion about the 5d3. It doesn't involve one single written word. It's the fact that the images I got this easter of my kids, couldn't be done the way they were with anything else. That's all I care about. :

Buy one, get used to it, like it or sell it. That is the only thing that will tell you for certain.


----------



## Tcapp (Apr 10, 2012)

Viggo said:


> I'm out writing my own opinion about the 5d3. It doesn't involve one single written word. It's the fact that the images I got this easter of my kids, couldn't be done the way they were with anything else. That's all I care about. :
> 
> Buy one, get used to it, like it or sell it. That is the only thing that will tell you for certain.



+8374


----------



## JR (Apr 10, 2012)

Viggo said:


> I'm out writing my own opinion about the 5d3. It doesn't involve one single written word. It's the fact that the images I got this easter of my kids, couldn't be done the way they were with anything else. That's all I care about. :
> 
> Buy one, get used to it, like it or sell it. That is the only thing that will tell you for certain.



Well said...in the end it is about taking pictures!


----------



## psolberg (Apr 10, 2012)

MarkB said:


> I got my 5d III the week before last. I am sure both those organizations got one from at least the first shipped batch. Why is it taking so long to come out with anything other than a preview? Are they running into issues?
> 
> I am pretty happy with the 5d3, although the wife is a little mad about the price.
> 
> I am really curious about how low the DXOMark score will be and if it is causing them to rethink their test. I just think they might worry it would damage what credibility they have if the nikon 800 had a 95 and the 5d3 had a 85 or so. They clearly are not that far apart unless the Nikon lenses are really holding them back. I don't think that is the case given he zeiss comparisons Lensrentals did.



they would damage their credibility if they adjusted their test to make the 5DmkIII score better 8) so no, that's not the reason. More likely they are just swampped with other things.


----------



## well_dunno (Apr 11, 2012)

LOL apparently the reason for delayed DxO evaluation of the mark III is because they have not been able to get their hands on a production sample 

http://forum.dxomark.com/index.php/topic,720.0.html


----------



## Tracy Pinto (Apr 11, 2012)

Attacking the integrity of the Dx0 testing labs and then wondering if it is a conspiracy that they are delaying their disreputable work is somewhat funny.

My most important resource is myself. I need to know what I intend to use the new camera/lens for and what I hope to gain by making a new purchase. Then I read away only at sources I trust to sort out what I need to be concerned about in making my trade-offs. As an informed buyer I should not be surprised. All the cameras I have bought had much more features at purchase time than I had skill... and they all had limitations. My 5D Mark III is no exception. I hope to master it and improve as a photographer as I do.


----------



## Tcapp (Apr 12, 2012)

If I already have *MY* grubby little hands on a 5d3, why doesn't a website like DXO have one yet?! Weird.


----------



## JR (Apr 12, 2012)

well_dunno said:


> LOL apparently the reason for delayed DxO evaluation of the mark III is because they have not been able to get their hands on a production sample
> 
> http://forum.dxomark.com/index.php/topic,720.0.html



This is really odd indeed! Am sure I can get them one quickly! I guess they are not on Canon "preferred" list, if ever such list exist!


----------



## wickidwombat (Apr 12, 2012)

Tcapp said:


> If I already have *MY* grubby little hands on a 5d3, why doesn't a website like DXO have one yet?! Weird.


Nikon didn't send them one


----------



## psolberg (Apr 12, 2012)

scrappydog said:


> Tracy Pinto said:
> 
> 
> > Attacking the integrity of the Dx0 testing labs and then wondering if it is a conspiracy that they are delaying their disreputable work is somewhat funny.
> ...



I don't think they don't favor a brand per se or orient their results to favor anybody. Their test is always the same and it isn't reformulated on a per camera basis to help or sink a particular body. They simply start with a fixed criteria and methods and carry judgement on those results. Nikon's choices simply score better because they more closely resemble their interpretation of what is important in an imaging sensor. It could have been the case Canon or Sony would have been the best if those companies had chosen different designs and specs on their sensors. DXO simply provides a ruler. If you're not 7 feet tall, changing the ruler doesn't change the fact you're not 7 feet tall.

the bottom line is that no matter who wins, they'll get accused of biased so I see no reason why they should change anything 

nothing stops canon from improving the resolution and dynamic range of their bodies and until they don't, they will simply not do very well on these normalized tests. this is entirely canon's fault, not dxo's.


----------



## marekjoz (Apr 12, 2012)

We're not far from starting a ranking of camera and lenses review sites...

<projected sarcasm quote>
DXO tests description: "they provide tests where Canon doesn't win because of criteria mostly based on sensor and dynamic range. DXO can blame themselves for being so low in our rank" 
</projected sarcasm quote>


----------



## JR (Apr 12, 2012)

marekjoz said:


> We're not far from starting a ranking of camera and lenses review sites...
> 
> DXO tests description: "they provide tests where Canon doesn't win because of criteria mostly based on sensor and dynamic range. DXO can blame themselves for being so low in our rank"



Not a bad idea!


----------



## SambalOelek (Apr 12, 2012)

I lost quite a bit of faith in DXOMark after they tested the 70-200 f/2.8 IS II and found it to be significantly worse than the v1 in terms of resolution, and refused the possibility of a subaverage sample.

Their sensor tests may still be good, though.


----------



## well_dunno (Apr 12, 2012)

JR said:


> marekjoz said:
> 
> 
> > We're not far from starting a ranking of camera and lenses review sites...
> ...



lol +1! 

I think it would be great if Klaus /photozone.de maintained a database and some sort of rating with all the tests he has...


----------



## PhilDrinkwater (Apr 12, 2012)

SambalOelek said:


> I lost quite a bit of faith in DXOMark after they tested the 70-200 f/2.8 IS II and found it to be significantly worse than the v1 in terms of resolution, and refused the possibility of a subaverage sample.
> 
> Their sensor tests may still be good, though.


While I largely agree, I also question the ethics of a company which wouldn't take that possibility seriously.

Personally when it comes to sensors, I prefer my eyes to do the viewing because they will ignore the 1/3 to 1/2 stop differences that look quite significant when you look just at numbers. That said, I think they are "interesting". I just wouldn't base any decision on them.


----------



## Wrathwilde (Apr 12, 2012)

SambalOelek said:


> I lost quite a bit of faith in DXOMark after they tested the 70-200 f/2.8 IS II and found it to be significantly worse than the v1 in terms of resolution, and refused the possibility of a subaverage sample.
> 
> Their sensor tests may still be good, though.



They said the v2 "offers slightly less resolution" but tests better on chromatic aberration. I fail to see how that qualifies as "significantly worse". Also, "These 2 lenses are very similar in term of transmission, distortion and vignetting", again hardly a condemning review.

The tests are what they are, and the results speak for themselves. Just because you're not happy with how the test turned out, or think they may have got a sub-average sample is irrelevant. The test was done with a final production sample. They put the sensor or lens through their specific testing procedures, and those are the results they got... end of story. The v1 production unit they tested was better than the v2. Sure it was probably a fluke, but if you were upgrading, there is also the possibility that your v2 could score worse than your v1. Production samples vary, it's a fact of life.

That said, I think Nikon tests better because Nikon actually uses DXOMark equipment to test their sensors, and probably uses that data to determine what's working and what's not in terms of DR, color depth, and ISO. Not that they tune their sensors to beat the test, but use the test's to determine which of their sensor designs should produce the best real world results. 

In no way do I think DxO favors one brand over another, the fact that Nikon is a customer doesn't influence their DxO scores, as the scores aren't a matter of subjective bias. The scores are based on a rigid set of criteria. Nikon scores well because they use the same equipment and methods to test their sensors... then use that data to improve their sensors.


----------



## marekjoz (Apr 12, 2012)

Wrathwilde said:


> SambalOelek said:
> 
> 
> > I lost quite a bit of faith in DXOMark after they tested the 70-200 f/2.8 IS II and found it to be significantly worse than the v1 in terms of resolution, and refused the possibility of a subaverage sample.
> ...



I defended DXO some time ago just because they publish their testing procedures first and then follow them when testing. But I think you will also admit, that when there is a place for some speculation it will always happen. When only one copy (even if production one) was tested, then someone always can ask questions:
1. Was it the only copy they had?
2. If no didn't they "carefully" selected one "special" copy for testing purposes?
3. If they publish tests of only one copy of gear, doesn't it open space for such speculations as above?
4. Do they have any interest in such behaviour if they are sponsored by one producent?

I don't say they do it like described above. But as far as there is a reason to speculate, people will ask questions and doubt.


----------



## Wrathwilde (Apr 12, 2012)

marekjoz said:


> I defended DXO some time ago just because they publish their testing procedures first and then follow them when testing. But I think you will also admit, that when there is a place for some speculation it will always happen. When only one copy (even if production one) was tested, then someone always can ask questions:
> 1. Was it the only copy they had?
> 2. If no didn't they "carefully" selected one "special" copy for testing purposes?
> 3. If they publish tests of only one copy of gear, doesn't it open space for such speculations as above?
> ...



Personally I'd be suspect of any site that cherry picked to find the best possible sample to test, or tested several and only published the best performer. This is the reason that the most reputable sites test units acquired from the retail channel, and not cherry picked units from the manufacture.

I'd also be suspect of any test site that would bow to pressure from end users to redo their tests. Now, if the manufacture claimed that the results were far different than their own in house testing, and contacted them about the possibility that the lens they tested might be defective... and paid for them to acquire another sample, then sure, they should be willing to retest. That said, they are under no obligation to do so. 

Sure you can speculate they got a bad copy, suggesting that their credibility is shot because a lens didn't test well is a different matter. 

Lastly, no, they are not sponsored by any Camera or Lens company

_*From their website - *_

_*We test commercial products: in other words, we buy or rent lenses and cameras from the very same retailers that consumers use. When we do test pre-production models (when commercial products are not yet available), we clearly indicate this on our site, and we retest those models when they become commercially available.*_ 

_*Finally, DxOMark has no ties to or interests of any sort with camera or lens manufacturers, which means that we are completely independent from them.*_

edit - That and DxOMark has probably figured out that 99.9% of people posting comments have absolutely zero credibility or experience in testing sensors and/or lenses, and are just butt-hurt that the Camera/Lens that they purchased (or planned to purchase) didn't score as high as they believed it should. That and the posters just can't accept that their beloved product isn't the greatest thing since sliced sex on buttered buns.


----------



## marekjoz (Apr 12, 2012)

Wrathwilde said:


> marekjoz said:
> 
> 
> > I defended DXO some time ago just because they publish their testing procedures first and then follow them when testing. But I think you will also admit, that when there is a place for some speculation it will always happen. When only one copy (even if production one) was tested, then someone always can ask questions:
> ...



Wrathwilde - yes, I'm aware of that and I'm not saying they cheat. I'd rather say they have too much too loose to manipulate. But it's strange anyway, when they seem to be the only site saying 70-200 2.8 II is worse than mk1. Everybody else is wrong, they are wrong or sth else is wrong? If this lens is considered to be better than mk1, then maybe they tested a bad copy. And it's ok if it indicates how good or bad the Canon internal quality check was. But most probably it doesn't show how these lenses good in general are. And this is the problem. Such a test should present the potential of design, materials, build etc. of a product in general but not of a single copy. That's my opinion.


----------



## marekjoz (Apr 12, 2012)

Wrathwilde said:


> marekjoz said:
> 
> 
> > I defended DXO some time ago just because they publish their testing procedures first and then follow them when testing. But I think you will also admit, that when there is a place for some speculation it will always happen. When only one copy (even if production one) was tested, then someone always can ask questions:
> ...



Yes, but people believing in conspiracy say, that as far as Nikon uses their products and pays for commercial, then this absolute independence might be disturbed. 
Yes, I believe that fanboys don't believe in independent tests. That's absolutely true. But in this case, anyway I don't find them as the only ones with groundbreaking discovery.

Anyway - I really don't know. I just mention what I'm aware of. I don't have nor had neither 70-200 2.8 mk1 nor mk2 so can't say.


----------



## moreorless (Apr 13, 2012)

If anyone is looking to "make sensors for DxO tests" then I'd say its Sony rather than Nikon, the D4 with Nikons own sensor in afterall follows Canon performance more closely with DR holding up well thoughout the lower ISO settings.

The D800 on the other hand follows the pattern of Sony's recent sensors with the DR declining steadly below ISO 100 which with the DxO criteria does tend to overplay its performance at bit.


----------



## LetTheRightLensIn (Apr 13, 2012)

moreorless said:


> If anyone is looking to "make sensors for DxO tests" then I'd say its Sony rather than Nikon, the D4 with Nikons own sensor in afterall follows Canon performance more closely with DR holding up well thoughout the lower ISO settings.
> 
> The D800 on the other hand follows the pattern of Sony's recent sensors with the DR declining steadly below ISO 100 which with the DxO criteria does tend to overplay its performance at bit.



You got that in reverse it is the Sony Exmors where the DR holds up well in the low ISOs and the Nikon and especially Canon ones where the lines curves over and you don't keep gaining the expected stop better each stop you go below ISO800-1600 or so.


----------



## moreorless (Apr 13, 2012)

LetTheRightLensIn said:


> moreorless said:
> 
> 
> > If anyone is looking to "make sensors for DxO tests" then I'd say its Sony rather than Nikon, the D4 with Nikons own sensor in afterall follows Canon performance more closely with DR holding up well thoughout the lower ISO settings.
> ...



It doesnt really matter how you state it, my point was that the DxO headline scores will tend to overstate the performance of a sensor who's DR declines in a steady fashion over one who's DR holds steady for longer. 

Add to that of course that Sony hasnt produced a FF sensor with the extreme ISO range of the D3s, D4 and 1DX which DxO also do not take into account with their headline scores.

Just seems to me that thus headline marks are rather overused given that they are to some extent subjective, you don't for example see people throwing around photozone marks out of 5 in the same fashion.


----------



## Wrathwilde (Apr 13, 2012)

scrappydog said:


> To quote DxO's site: "Here is just a sample of our current customers:..." with a picture of Nikon's image below it. I think it is fair to say that they have ties to camera and lens manufacturers because they are _expressly _advertising these businesses as their _customers _on their site. Paste the link in a browser and look for yourself. This isn't conjecture nor is it conspiracy theory; it is fact.
> 
> EDIT:
> I created a PDF of the website. See the attached. Opening the page sometimes defaults to their Home page.



Wrong. It means that several large companies find enough value in DxOMark products and services that they are willing to purchase them, and Nikon happens to be one of many. It's not advertising for Nikon, it's advertising for DxOMark. A lot of large companies list who their customers are, so what? It provides a reassurance for other companies who are looking at purchasing their equipment and services, and therefore _further_ establishes their legitimacy. It doesn't make DxOMark beholden to Nikon in any way.

A customer is a customer, either they find value in your products and services, or they don't. Nikon currently finds that DxoMark's equipment and procedures are helpful in their quest to make better sensors. If anything this makes Nikon beholden to DxOMark, not the other way around.


----------



## PhilDrinkwater (Apr 13, 2012)

Wrathwilde said:


> It doesn't make DxOMark beholden to Nikon in any way.


No. But it doesn't discount it either.

I doubt there's anything in the DxO / Nikon "connection" but to assume it is or isn't true (ie. to assume either side is true) is as random as taking the opposite stance.

The simple truth is this:
* Nikon pay DxO for "something".
* We don't know what it is.
* We don't know whether it gives Nikon any advantage.

Past that everything on BOTH sides is just guesswork.


----------



## JR (Apr 13, 2012)

Past that everything on BOTH sides is just guesswork.
[/quote]

Maybe some educated guess  but well said!


----------



## well_dunno (Apr 17, 2012)

Mark III results will be on DxO within the next few days it seems...

http://forum.dxomark.com/index.php?topic=720.15


----------



## Invertalon (Apr 17, 2012)

I can't wait for the flood of D800 vs. 5D3 DXOMark results... (not!)


----------

