# Canon publishes a paper discussing a new 3.4 μm pixel pitch global shutter CMOS image sensor with dual in-pixel charge domain memory



## Canon Rumors Guy (Mar 13, 2019)

> The Japanese Journal of Applied Physics has published a Canon Inc. paper on a global shutter entitled  “A 3.4 μm pixel pitch global shutter CMOS image sensor with dual in-pixel charge domain memory” by Masahiro Kobayashi, Hiroshi Sekine, Takafumi Miki, Takashi Muto, Toshiki Tsuboi, Yusuke Onuki, Yasushi Matsuno, Hidekazu Takahashi, Takeshi Ichikawa, and Shunsuke Inoue.
> *From the paper:*
> In this paper, we describe a newly developed 3.4 μm pixel pitch global shutter CMOS image sensor (CIS) with dual in-pixel charge domain memories (CDMEMs) has about 5.3 M effective pixels and achieves 19 ke− full well capacity, 30 ke−/lxcenterdots sensitivity, 2.8 e- rms temporal noise, and −83 dB parasitic light sensitivity. In particular, we describe the sensor structure for improving the sensitivity and detail of the readout procedure. Furthermore, this image sensor realizes various readout with dual CDMEMs. For example...



Continue reading...


----------



## efmshark (Mar 13, 2019)

3.4u pixel pitch would roughly translate to a 32Mp APS-C sensor. However, this device appears to be mostly targeted at video capture and it is either a test chip, or a prototype of a sensor aimed at a prosumer 4K camcorder.


----------



## Proscribo (Mar 13, 2019)

Considering the downsides, would it be more viable to develop electronic shutter that is as fast or possibly even faster than the current mechanical ones?


----------



## flip314 (Mar 13, 2019)

efmshark said:


> 3.4u pixel pitch would roughly translate to a 32Mp APS-C sensor. However, this device appears to be mostly targeted at video capture and it is either a test chip, or a prototype of a sensor aimed at a prosumer 4K camcorder.



Based on 32MP APS-C, that works out to ~80MP full-frame. That's in the same ballpark as the 70MP rumored sensor.

This could completely solve the rolling-shutter problems that the Canon ILCs suffer from, and also improve silent shutter performance.


----------



## preppyak (Mar 13, 2019)

flip314 said:


> This could completely solve the rolling-shutter problems that the Canon ILCs suffer from, and also improve silent shutter performance.


And look at what the Sony rumor for the a7IIIS is: global shutter. Blackmagic does it in their Ursa cameras, so, it makes sense for a video-oriented mirrorless camera.


----------



## Trey T (Mar 13, 2019)

Proscribo said:


> Considering the downsides, would it be more viable to develop electronic shutter that is as fast or possibly even faster than the current mechanical ones?


One more component to break. Good for Canon, kinda bad for us, I think ... not too sure.

What they're talking about is for video. Majority of photogs shoots @ about 1-2fps, relative to motion pictures (30-60fps), w/ a really fast shutter speed (e.g. 1/3000s). Consider the two shutter types (rolling and global shutter), the image quality is critical as you increase the frame rate, but this is only mattered if you put the images together and turn it into motion pictures. Otherwise, this topic is useless for many of us photogs. I believe canon uses a hybrid technology right now, combining electronic and mechanical shutter curtain to capture image (e.g. canon 5D IV ... and EOS R!?!?)

Canon is behind on the video camera side while small company like BlackMagic have produced more proper camera for fast motion picture.


----------



## rrcphoto (Mar 13, 2019)

preppyak said:


> And look at what the Sony rumor for the a7IIIS is: global shutter. Blackmagic does it in their Ursa cameras, so, it makes sense for a video-oriented mirrorless camera.



Canon already has a global shutter for video. It's in the C700.
This is a video only sensor, it's not going into a stills camera.


----------



## mb66energy (Mar 13, 2019)

Proscribo said:


> Considering the downsides, would it be more viable to develop electronic shutter that is as fast or possibly even faster than the current mechanical ones?



A fast electronic shutter means that you collect light only over a short period - e.g. 1/2000 second. With limited light you need high ISO or wide apertures. And it does not circumvent the problem that you cannot read out the whole (standard) CMOS chips fast enough for the required framerates as far I understand the necessity to read information line by line from top to bottom for the current frame before you start to read the next first line to build up the next frame.

With the new sensor layout you can maybe use 1/60 second @24 fps for exposure and 1/40 second for readout (just an example, 1/60 s + 1/40 s = 5/120 s = 1/24 s, the time for 1 frame at 24 fps). The benefit of the described design is that the charge buffers (CDMEM) store electrical charge proportional to the amount of light which hit the sensor during exposure + maybe a much faster readout/conversion speed. After exposure the information is "freezed" in the sensor (memory cells) and you can read out the "freezed still image" and convert it into storable "computer numbers" in a very short time before you clear the memory cells for the next exposure.

1/60 s is ~32 times longer compared to 1/2000 s which converts to 5 stops - 5 stops lower ISO or 5 stops narrower aperture which is always helpful if you already are at ISO 3200 with f/1.4 at 1/60s in low light


----------



## tmroper (Mar 13, 2019)

Proscribo said:


> Considering the downsides, would it be more viable to develop electronic shutter that is as fast or possibly even faster than the current mechanical ones?


Isn't a "global shutter" the same thing as an "electronic shutter? Global just being the terms video people use?


----------



## Rado (Mar 13, 2019)

Trey T said:


> One more component to break. Good for Canon, kinda bad for us, I think ... not too sure.
> 
> What they're talking about is for video. Majority of photogs shoots @ about 1-2fps, relative to motion pictures (30-60fps), w/ a really fast shutter speed (e.g. 1/3000s). Consider the two shutter types (rolling and global shutter), the image quality is critical as you increase the frame rate, but this is only mattered if you put the images together and turn it into motion pictures. Otherwise, this topic is useless for many of us photogs.



No it's not. As a strobist I consider global shutter to be the next big innovation I'm looking forward to. Flash sync at any shutter speed without workarounds like HSS or HS.


----------



## canonnews (Mar 13, 2019)

't.


SwissFrank said:


> The TLDR is: 1) This article DOES appear to me to discuss technology FULLY applicable to stills as well as motion picture. 2) The point of the technology seems to be achieved, which is HDR images from a single sensor, whereby the highlight exposure will have the same motion blur as the shadow exposure.
> 
> From the Canon News explanation of the technical article, that I feel totally misunderstands that article:
> 
> ...



Actually I didn't. This paper is about the dual memory architecture of a global shutter sensor. a normal global shutter only has one memory cell per pixel.
For stills dual memory for the most part is entirely unnecessary and impractical. As with any motion, you'll have blurr. it simply can't be done on the sensor without some pretty convoluted brains sitting behind the sensor doing motion detection. while the time between shots is relatively fast, you still have the exposure itself. for instance, a 1/20th of a second exposure would still, in this case, require at least a 1/10th + 1/120th of a second dual exposure saved to the memory. For video, it's easy, they just take two fast exposures and combine them, as they have much more finite control over the shutter speed. it's not the case though with stills. this isn't the first time Canon has done interleaved exposures for Global Shutter. for instance, it's already in production on their C700 video camera. however all their patents and all their background work on this stuff is all to do with video, and will stay with video.
What Canon has to do with stills is far more expensive. It needs to go to a stacked artitecture, where the DR is not impacted by having memory cells at each pixel. Also Dual Pixel AF extremely complicates Global shutter even more. This type of arrangement would need FOUR memory cells for each actual pixel. Ugh.
that's not what this paper is about, nor is what this sensor is about.


----------



## Trey T (Mar 13, 2019)

Rado said:


> No it's not. As a strobist I consider global shutter to be the next big innovation I'm looking forward to. Flash sync at any shutter speed without workarounds like HSS or HS.


Well, that I didn't think of. That would be good for capturing the flash.


----------



## canonnews (Mar 13, 2019)

SwissFrank said:


> Unless you want HDR with motion blur and without any additional logic trying to stretch the highlight exposure to match the blur of the shadows exposure. WHICH IS THE ENTIRE POINT OF THE PAPER.
> 
> 
> Sure. And thanks to the interleaved sandwich partial-exposure method outlined here, you'll have the same blur on your highlight exposure that you do on your shadows exposure.
> ...



it's an entirely different shutter process for stills than it is for video. but regardless, see what you want out of it. this technology will not make it into a stills camera for Canon.
what you are not considering is the TIME for each exposure - which is from 30s to 1/8000th of a second for stills. if you have a 1s exposure, you're probably effectively taking a .5s exposure and then a 2s exposure to make up the HDR image for a total of 2.5s.
while doing this while you are doing in video, your shutter speed can be finitely more controlled, because, your shutter speed, well is more controlled and far more predictable.
Then you have the complexity of DPAF which aggravates this by a factor of 2.
but really see into it what you want, but there's a reason canon already has global shutter implemented with the single cell memory version of this technology and it's not available on any ILC.


----------



## Mt Spokane Photography (Mar 13, 2019)

A company does not expose any of their best and latest technology to the public. The information revealed will have been thoroughly vetted to make sure that nothing confidential is revealed. So, I would place zero confidence that this implementation will happen, they may be using some of the ideas revealed in their latest projects, but this is just a snapshot of the past and likely intended to throw off the competition.

We had global shutters 20 years ago using CCD technology, and still have it. The goal is to achieve it with large CMOS sensors. Tiny CMOS sensors are available with a GS, but there are lots of issues whith a large sensor. The dual memory is a trick that can overcome the need for a super computer to read out 50 milliom photosites instantly.


----------



## flip314 (Mar 13, 2019)

Mt Spokane Photography said:


> A company does not expose any of their best and latest technology to the public. The information revealed will have been thoroughly vetted to make sure that nothing confidential is revealed. So, I would place zero confidence that this implementation will happen, they may be using some of the ideas revealed in their latest projects, but this is just a snapshot of the past and likely intended to throw off the competition.



They absolutely *do *expose their cutting edge ideas, when they patent them. Disclosure is part of the price for patent protection. I guarantee anything novel in this paper has already been patented. Companies' trade secrets are usually only things that are non-patentable, because there is no legal protection for a trade secret if it is disclosed in any way. Companies may _defer_ patenting certain things until 1) they have an actual product in development, or 2) they want to publish some aspect of the design. But generally they patent quickly unless they worry the patent length wouldn't be long enough to cover the bulk of the product's market availability.

It's unlikely that Canon is doing anything to purposely confuse their competition. The competitors are more interested in what Canon has already announced or brought to market. Their competitors will reverse-engineer many aspects of Canon's designs 1) to see what Canon is doing, and 2) to see if Canon is infringing their patents. Canon does the same to their competitors. Companies may extrapolate certain performance numbers to have some expectation of certain future products, but I doubt they draw much from patents and journal papers. So much gets patented/published that's never brought to market.

Believe me, I work for a cell-phone manufacturer, and it is _crazy_ what it's possible to learn about other chip-makers' designs just from poking them from the outside. Just one example, by running certain targeted benchmarks and putting the chip under thermal analysis, you can figure out what most of the chip die is dedicated to, and compare how much area they're spending on certain functions to what you are. Then you know what you're leading or trailing on in the design. (Actually, you can guess a lot of the design just by seeing the die photo. Certain things like caches stick out like a sore thumb.)


----------



## Kit. (Mar 13, 2019)

tmroper said:


> Isn't a "global shutter" the same thing as an "electronic shutter? Global just being the terms video people use?


Rolling shutter in video is (usually) electronic too.


----------



## peterzuehlke (Mar 14, 2019)

rrcphoto said:


> Canon already has a global shutter for video. It's in the C700.
> This is a video only sensor, it's not going into a stills camera.


Canon may want to compete with the Sony A9 whose shutter isn't global, but on the way, and does frame rates close to video. If the Canon sports / wildlife camera application and a true merging of video and stills is what they want (for a future mirrorless one series) this might be the direction.


----------



## justaCanonuser (Mar 14, 2019)

SwissFrank said:


> You clearly didn't understand the article. While they're talking about taking an exposure _in the context of_ video, nothing about the method requires it to be a frame of video.


I agree. Such a paper in a physics journal is introducing the principle. The authors use video as main application example because the very fast frame rates are overall a much more demanding task. I am pretty sure that Canon will use such sensors as a platform for different camera models (even more as the already do today), since industrial platform technologies are in general much less expensive than making many special parts for special applications. This is even more probable since any differentiation between stills and video cameras makes less and less sense, at least on the sensor (or electronics) side of life . It may make sense in future to stick with different classic form factors of camera bodies for stills or for video, but the core electronics, i.e. the sensor-electronic shutter unit, stuffed into those bodies will be about the same, simply because of the pressure to cut costs.


----------



## zonoskar (Mar 14, 2019)

For HDR you don't need 2 double exposure if you have 2 memory buffers per cell. Just store the value halfway through the exposure to the first memory buffer, then at the end, store the value in the second memory buffer. If there's motion blur, it would be present in the exposure anyways.


----------



## Timedog (Mar 14, 2019)

God, I so don't care about rolling shutter. I've never noticed it in any videos i've taken on canon. Just work on backside illumination pls Canon.

Also I don't get how this does HDR without changing shutter speed for the 2 exposures. Does it lower ISO on one of the exposures?


----------



## deleteme (Mar 14, 2019)

Trey T said:


> One more component to break. Good for Canon, kinda bad for us, I think ... not too sure.


Actually, one less component to break. The sensor thus has a solid state shutter as opposed to a mechanical shutter that is the subject of shutter count queries when time to sell.


----------



## justaCanonuser (Mar 16, 2019)

SwissFrank said:


> Just my guess, but it's possible they started off with this goal, and someone just happened to figure out that you could use this for HDR as well.


I agree. That's the way science normally works (I know that because I edit a German physics magazine + from my own experience back when I studied physics). 



SwissFrank said:


> Right. Up until now, on-sensor memory may have only had one possible function--avoiding rolling shutter--and thus be something you'd only support on a mainly-video camera. Thx to this paper, they're now dangling 42dB or more extra headroom (basically SEVEN STOPS more highlight detail) and that's something you'd totally pay an extra couple grand for if you really understood it. It's simply staggering.


Yes, I'd really love to see that technology to hit the market soon. But the publication in a primary research journal without any patent claims so far indicates that we'll have to wait a while...


----------

