# Patent: Quad Pixel autofocus image sensor



## Canon Rumors Guy (Mar 23, 2019)

> Canon News has uncovered a patent application for a quad pixel autofocus image sensor.
> *Canon News explains Japan Patent Application 2019041178:*
> This patent application from Canon deals with a quad pixel autofocus sensor.  Right now Canon is using dual pixel autofocus sensors, but if you ever tried to use an EOS R or an EOS M in landscape orientation to focus on a horizontal line you’ll quickly realize that the phase detect sensors just go in one direction, and have little sensitivity in the other 90 degrees offset direction.
> This patent application indicates that Canon has split the pixel into 4 pieces, and also offsets the microlenses as you go further out from the center.
> The sensor they are describing in this document seems to be a 20.7MP sensor, with 83 million focus detection points!  The pixel size seems to be 4 micrometers, which would make that approximately 22mm on the width...



Continue reading...


----------



## CanonGrunt (Mar 23, 2019)

Would this have any relevancy to super 35 cinema image sensors? Or is this something completely different. Not sure about such technical things...


----------



## addola (Mar 24, 2019)

This isn't the first Quad-Pixel AF patent. This was posted on CanonRumors here a year ago.

It would be interesting if this is a new sensor technology that we would see in a future camera!


----------



## juststeve (Mar 24, 2019)

Yes, quad pixel patents have appeared before. If you multiply the approximately 4 micron pixel size by the 2.56 crop to full frame factor, it comes to just over 50 MP. Sounds something like a 5DS. 

This could be a breakthrough but a holdup in a pro R camera, say a Nikon 850-Z7-Sony A7Riii competitor. The amount of information coming off a sensor like this is a torrent. Could it be four times what comes of a dual pixel sensor? How to process it all? How to exploit it to maximum advantage? It could well be worth waiting for. Or not.


----------



## uri.raz (Mar 24, 2019)

juststeve said:


> Could it be four times what comes of a dual pixel sensor?



Not necessarily. X pixels in DPAF translates into 2X value in the raw files. X pixels in QPAF would translate into 4x values in raw file. So for the same resolution you'd get a raw file twice as large, question is whether Canon would make a QPAF sensor with the same resolution.



juststeve said:


> How to process it all? How to exploit it to maximum advantage?



Apparently it would allow two stops of latitude processing over exposed pixels, and reconstructing 3D info from the sub-pixel phase info. I wouldn't bet on either catching. IIRC, it was noted the former was problematic with DPAF, as the left sub-pixels have a different perspective than the right sub-pixels, which would be worked around with QPAF (average left-top w/ right-bottom and left-bottom w/ right-top), so it might have better success with QPAF.



juststeve said:


> It could well be worth waiting for. Or not.



I'm gonna keep an open eye for the improved AF performance.


----------



## mb66energy (Mar 24, 2019)

juststeve said:


> Yes, quad pixel patents have appeared before. If you multiply the approximately 4 micron pixel size by the 2.56 crop to full frame factor, it comes to just over 50 MP. Sounds something like a 5DS.
> 
> This could be a breakthrough but a holdup in a pro R camera, say a Nikon 850-Z7-Sony A7Riii competitor. The amount of information coming off a sensor like this is a torrent. Could it be four times what comes of a dual pixel sensor? How to process it all? How to exploit it to maximum advantage? It could well be worth waiting for. Or not.



I am with uri.raz: 2 times the data.

IMO QPAF resolves one problem with DPAF - DPAF can only detect vertical structures. Try focusing on distant venetian blinds and sometimes it struggles. Rotate the camera to portrait mode and AF acquisition is VERY fast. That is an example for the "problem" described in the article

I see QPAF as a solution to that problem: Use a 2nd DIGIC processor and secondary readout lanes orthogonal to the existing ones and you have no speed penalty (just theoretically) for to improve AF for horizontal structures.

With QPAF the whole sensor is a cross type sensor resulting in smaller AF points (no need to include a larger sensor region to find some vertical structures) which might find enough detail just in lower light.

Maybe DPAF and its maybe-successor are much more important for the future of very reliable AF than I thought before: I am not shure if it is possible for other sensor designs of e.g. Sony to implement some QPAF analogon with their phase detect pixels (but maybe they have it just now). 

It was a great idea to give imaging pixels both functions: recording image + providing AF information - seems to me that this makes a more homogenous sensor compared to designs with different pixel types!


----------



## TMHKR (Mar 24, 2019)

Wouldn't QPAF also decrease light sensitivity of the sensor, since you split the pixels even further?
It always baffled me how DPAF sensor actually performed better in this regard than sensors with "whole" pixels.


----------



## uri.raz (Mar 24, 2019)

TMHKR said:


> Wouldn't QPAF also decrease light sensitivity of the sensor, since you split the pixels even further?



AFAIK, the pixels don't cover 100% of the sensor area, and the microlenses concentrate the light into the active area. Thus, as long as the four sub pixels have the same total area as the pixel whence they were split, everything should be fine.


----------



## Sharlin (Mar 24, 2019)

TMHKR said:


> Wouldn't QPAF also decrease light sensitivity of the sensor, since you split the pixels even further?
> It always baffled me how DPAF sensor actually performed better in this regard than sensors with "whole" pixels.



Why would it? As long as the total light-sensitive area stays the same, nothing changes. 2*n/2 equals n.


----------



## Mt Spokane Photography (Mar 24, 2019)

uri.raz said:


> Not necessarily. X pixels in DPAF translates into 2X value in the raw files. X pixels in QPAF would translate into 4x values in raw file. So for the same resolution you'd get a raw file twice as large, question is whether Canon would make a QPAF sensor with the same resolution.



Not correct. The outputs from the dual pixel are combined into one pixel before saving to the card as a CR2 / CR3 so files are no larger. Try it with any dual pixel camera. It will be the same for Quad pixel cameras. 

The 5D MK IV and the EOS R have a option to output separate values using Dual Pixel RAW, and that doubles the file size, but few use that. I'm not certain why Canon bothered to put it in the EOS R, perhaps there are future plans to make better use of it.


----------



## djack41 (Mar 24, 2019)

Interesting but Canon has boxes filled with patents. This is as likely to come to market as Canon's years-old patents for stacked sensors or back light illumination. Hard to imagine Canon making the investment to build such a sensor when the CEO is predicting market doom. Hope I'm wrong!


----------



## Kit. (Mar 24, 2019)

djack41 said:


> Hard to imagine Canon making the investment to build such a sensor when the CEO is predicting market doom. Hope I'm wrong!


"Market doom" is for traditional photo cameras, not for video sensors.


----------



## Graphic.Artifacts (Mar 24, 2019)

DPAF and ultimately QPAF are great core technology and I’d expect that the processing will eventually catch up to what seems like a lot of data to crunch between frames. Right now, I think they’ll need to stick to getting DPAF processing up to speed before doubling the data flow. Sounds like very promising future tech though. So far I haven’t seen where anyone else has a better core AF model.


----------



## Ozarker (Mar 24, 2019)

djack41 said:


> CEO is predicting market doom.


----------



## SecureGSM (Mar 25, 2019)

Mt Spokane Photography said:


> Not correct. The outputs from the dual pixel are combined into one pixel before saving to the card as a CR2 / CR3 so files are no larger. Try it with any dual pixel camera. It will be the same for Quad pixel cameras.
> 
> The 5D MK IV and the EOS R have a option to output separate values using Dual Pixel RAW, and that doubles the file size, but few use that. I'm not certain why Canon bothered to put it in the EOS R, perhaps there are future plans to make better use of it.


there is an extra stop of DR that lives in the second sub frame of the DPRAW file. I do use it when multiple exposure are not an option - e.g. slow / fast moving subject.


----------



## crazyrunner33 (Mar 25, 2019)

juststeve said:


> Yes, quad pixel patents have appeared before. If you multiply the approximately 4 micron pixel size by the 2.56 crop to full frame factor, it comes to just over 50 MP. Sounds something like a 5DS.
> 
> This could be a breakthrough but a holdup in a pro R camera, say a Nikon 850-Z7-Sony A7Riii competitor. The amount of information coming off a sensor like this is a torrent. Could it be four times what comes of a dual pixel sensor? How to process it all? How to exploit it to maximum advantage? It could well be worth waiting for. Or not.



The way you're describing it is assuming they'll use quad bayering for the sensor like Sony Semicondocutor is using on the newer products. The A7S III rumors shows a quad bayer sensor, the GH5 S is also using a variation of a quad bayer sensor. Below is an idea of how the concept works on small sensors that are already in the field.









Updated: Huawei Mate 20 Pro camera review - DXOMARK


Please note: In September 2019, we updated the DXOMARK Mobile test protocol to cover ultra-wide-angle performance and have renamed the protocol DXOMARK Camera. We also expanded our low-light testing and created the new Night sub-score, which incorporates the previous Flash score. We have...




www.dxomark.com


----------



## CJudge (Mar 25, 2019)

SecureGSM said:


> there is an extra stop of DR that lives in the second sub frame of the DPRAW file. I do use it when multiple exposure are not an option - e.g. slow / fast moving subject.


For real? That's awesome! Do you need to use Canon's DPP to process? And would this also benefit max usable ISO in low light, or is it just for low ISO DR?


----------



## [email protected] (Mar 25, 2019)

CJudge said:


> For real? That's awesome! Do you need to use Canon's DPP to process? And would this also benefit max usable ISO in low light, or is it just for low ISO DR?



Yeah. I think people underestimate the DPP factor on the lack of uptake for Dual Pixel features. That is precisely and only what has kept me from exploiting the focus-after-exposure feature. Even though its effects are slight, I'd still go out of my way to get the extra millimeter of sharpness if I could do it in Lightroom or, really, anything else. That software is truly dreadful.


----------



## syyeung1 (Mar 26, 2019)

Meaning that processor power (which I think Canon is behind) will be more important than ever!


----------



## Ah-Keong (Mar 26, 2019)

DIGIC 9++ processor is coming !


----------



## Mt Spokane Photography (Mar 26, 2019)

SecureGSM said:


> there is an extra stop of DR that lives in the second sub frame of the DPRAW file. I do use it when multiple exposure are not an option - e.g. slow / fast moving subject.


I've tested it, it appears to be a fraction of a stop, but does help. Not enough for me to bother using it.

I've heard from only a very few who use the DP Raw feature. It will be interesting if they come out with a quad pixel to see if there is a application.


----------



## Mt Spokane Photography (Mar 26, 2019)

CJudge said:


> For real? That's awesome! Do you need to use Canon's DPP to process? And would this also benefit max usable ISO in low light, or is it just for low ISO DR?


You need to separate the DP RAW images into two files as well as the merged file from the camera. There is software called DPR Split that does this. Then, you stack the images to obtain extra DR.

The top image is the original, the bottom is the processed one.


----------



## SecureGSM (Mar 26, 2019)

CJudge said:


> For real? That's awesome! Do you need to use Canon's DPP to process? And would this also benefit max usable ISO in low light, or is it just for low ISO DR?



yup, here:









DPRSplit will help pull more dynamic range from Canon 5D Mark IV Dual Pixel Raw files


DPRSplit can help you pull about +1EV of usable dynamic range from Canon's 5D Mark IV Dual Pixel Raw files—as long as you don't mind a bit of parallax error, that is.




www.dpreview.com





this won't affect max usable iso in low light. quite opposite, it affects ability to recover clipped highlight.
an extra stop of DR (on demand) is a wonderful feature to have.


----------



## CJudge (Mar 26, 2019)

[email protected] said:


> Yeah. I think people underestimate the DPP factor on the lack of uptake for Dual Pixel features. That is precisely and only what has kept me from exploiting the focus-after-exposure feature. Even though its effects are slight, I'd still go out of my way to get the extra millimeter of sharpness if I could do it in Lightroom or, really, anything else. That software is truly dreadful.




Considering that Canon's new c-RAW format is much friendlier on storage space (as well as to my eyes having practically zero effect on image quality compared to standard CR3 files), I would agree with you that I would love to take advantage of what DPRAW offers, but there's no way I'm switching my whole workflow just to do it. If QPRAW is indeed put into use, and Lightroom enables access to these post processing options, I'd be all over that!


----------



## CJudge (Mar 26, 2019)

SecureGSM said:


> yup, here:
> 
> 
> 
> ...



Thanks for sharing that! I wonder though, if you can separate and then merge the two image files (as described by Mt Spokane Photography above) , would you not effectively cut the noise in an image by averaging it?


----------



## Mt Spokane Photography (Mar 26, 2019)

CJudge said:


> Thanks for sharing that! I wonder though, if you can separate and then merge the two image files (as described by Mt Spokane Photography above) , would you not effectively cut the noise in an image by averaging it?


Canon merges the two images in the camera, which presumably reduces the noise. You gain more DR by a small but visible amount, and hopefully reduce noise by merging three images. Its somewhat klunky, the files are .dng. You should read about it at their site.


----------



## uri.raz (Mar 27, 2019)

If Canon makes a QPAF sensor, would it be debayered same as DPAF (= sum four sub pixels of same color, and interpolate colors same as non *PAF sensor), or would it gain by adding four adjacent sub pixels as in the orange squares, and not interpolating color at all?


----------

