Julian Iseringhausen, Bastian Goldlücke, Nina Pesheva, Stanimir Iliev, Alexander Wender, Martin Fuchs, Matthias B. Hullin
ACM Transactions on Graphics 36(4) (Proc. SIGGRAPH), 35:1--35:11, 2017.
AbstractLight fields are a powerful concept in computational imaging and a mainstay in image-based rendering; however, so far their acquisition required either carefully designed and calibrated optical systems (micro-lens arrays), or multi-camera/multi-shot settings. Here, we show that fully calibrated light field data can be obtained from a single ordinary photograph taken through a partially wetted window. Each drop of water produces a distorted view on the scene, and the challenge of recovering the unknown mapping from pixel coordinates to refracted rays in space is a severely underconstrained problem. The key idea behind our solution is to combine ray tracing and low-level image analysis techniques (extraction of 2D drop contours and locations of scene features seen through drops) with state-of-the-art drop shape simulation and an iterative refinement scheme to enforce photo-consistency across features that are seen in multiple views. This novel approach not only recovers a dense pixel-to-ray mapping, but also the refractive geometry through which the scene is observed, to high accuracy. We therefore anticipate that our inherently self-calibrating scheme might also find applications in other fields, for instance in materials science where the wetting properties of liquids on surfaces are investigated.
- Full Paper (PDF): IseringhausenEtAl-SprayOn4D-SIGGRAPH2017.pdf
- Supplemental Document (PDF): IseringhausenEtAl-SprayOn4D-SIGGRAPH2017-supp.pdf
- Supplemental Dataset(s): IseringhausenEtAl-SprayOn4D-SIGGRAPH2017-suppData.zip
- Video (Quicktime MOV): IseringhausenEtAl-SprayOn4D-SIGGRAPH2017-video.mov