The Lytro cinema camera appears to be closer to a real thing than I thought. The folks at no film school got a demo with a prototype and their editing software
http://nofilmschool.com/…/lytro-cinema-camera-cinematograph…
It’s overall pretty exciting. Plenoptic cameras make more sense for video because focus is much more of a challenge there.
The sensor size is crazy: more than 0.5m! (I’ll take this as 10 times bigger than a full frame sensor to simplify some of my calculations). This allows them to pack 750MPixels, and probably means pixels of roughly 10 microns, which is comparable to moderate-resolution SLRs. This points to potential for shrinking. It probably has ridiculous low-light performance once you aggregate pixels, and that’s probably why they can claim 16 stops of dynamic range. This size is also bigger than standard silicon wafers, which suggests a tiled sensor.
Since defocus is proportional to sensor size, this means that base depth of field (without light field trickery) is much more shallow than that of a full frame SLR. Defocus is 10 times blurrier for the same field of view, focus distance and f number.
They say that their aperture is 100mm but don’t discuss focal length. Assuming a 35mm wide angle equivalent, which would be 500mm in their case, means a max aperture of f/5. At 70mm equivalent it would be f/10. Depending on the max apertures you’re used to, it’s a factor of 2 to 5, which cancels some of the shallowness due to sensor size. Still, at 35mm equivalent you’d have the depth of field of an f/0.5 lens on a full frame SLR (just based on the full aperture without light field gimmicks). They claim they can get below this, but I assume this involves digital tricks since they are just not capturing light rays that would fall on a lens aperture bigger than 100mm. Update: their metadata panel at 16:54 says focal length 1346, f number 12.5, which does indeed correspond to a physical aperture of roughly 100mm and an equivalent focal length close to 100mm.
Now the big question is what is the microlens resolution. Given that they have 750MPixel, in order to get native 4k (8Mipxels), a reasonable choice would be 10×10. Knowing them, they’re probably pushing it a little more, maybe 20×20. I’ll use this number for my calculations below. I’m curious if they use a grid pattern or a similar hexagon as in their consumer cameras. Craig Kolb pointed out it’s a hexagon https://youtu.be/4qXE4sA-hLQ?t=884
The biggest depth of field they can generate corresponds to just picking the central microlens, which means 20 times smaller than the full aperture (assuming sub-apertures/pixels cover their full area). At 35mm equivalent, this means f/(0.5*20)=f/10. It’s medium depth of field (for reference a Canon 35mm goes to f/22). Probably sufficient for the cinematography needs they are targeting. After making the calculation, I went back to the video and the maximum value for the aperture slider appears to be f/11 (https://youtu.be/4qXE4sA-hLQ?t=7m19s) Elsewhere, Their control panel has the equivalent focal length at 110mm and their max virtual aperture at f/30. The DoF would be the equivalent of an f/1.1 lens at full aperture, and to get to f/30 equivalent you’d need a 27×27 microlens array.
Refocusing follows the same numbers. You can refocus your f/0.5 image within the depth of field of an f/10 image. This won’t enable crazy rack focus, but is well enough to correct moderate misfocus. The example of the kid with the baseball is probably representative of the maximum refocusing you can achieve. It is not surprising that marketing material would show you the best you can achieve!
Some of the features are probably not usable together (at least not in full quality) since you can’t use your full aperture at the same time for stereo and for full-aperture shallow depth of field. The sum of your stereo baseline and your aperture size is bounded by the input aperture.
https://lytro.com/…/lytro-brings-revolutionary-light-field-… mentions active scanning. This video does not discuss it, but the tracking window has both results from a tracking and a lidar and it’s probably at play for the matting.
It’s cool that they can do compositing in 4D, which will handle defocus between layers much better than a simple 2D alpha compositing.
They don’t mention it, but optical quality will probably be excellent even if the lens is not perfect because the light field allows you to correct many optical aberration (spherical, chromatic, coma, etc.). This is one of the unheralded impressive features of the Lytro illum.
I have no idea how they handle the bandwidth: 755Mpixels, 16 bits (a guess based on pixel pitch), 300fps would be almost half a terabyte per second ! They’re talking about 650MB per frame once compressed. One question is whether 300fps is actually possible at full resolution or if one needs to make sacrifices. The Phantom Flex4 can handle 4k at 1000fps but only for 5 seconds, and here we are talking about 100 times 4k at only 1/3 the framerate, for 30 times more than the Phantom. Even if the max framerate is not available at full resolution, 750MPixels 8 bits at 24fps is still 18GBytes/sec!
