This research aims to establish “innovative photogrammetry based on a data-driven approach and co-design of imaging systems” for the purpose of faithful 3D digitization of real-world objects using cameras, and to realize a world where anyone can easily create high-quality 3D digital content. In this project, we will develop technologies to obtain not only the rough objects’ shape (macro shape) but also the visible micro shape of the object surface (meso shape), which is essential for faithful digital reproduction and a finer shape that expresses reflectance (micro shape). In addition, this research will establish a technology for easily digitizing real-world objects in three dimensions using simple photographic equipment through the cooperative design of data-driven and imaging systems.

Grant

• 2023/4 – 2028/3 JSPS KAKENHI (S)
  “Photogrammetry through co-design of data-driven 3D estimation and imaging systems”
  PI: Yasuyuki Matsushita, Co-PIs: Fumio Okura, Hiroaki Santo, Grant: 157,200K JPY

Publications

• Xinpeng Liu, Zeyi Huang, Fumio Okura, Yasuyuki Matsushita. HoGS: Unified near and far object reconstruction via homogeneous gaussian splatting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
• Heng Guo, Jieji Ren, Feishi Wang, Boxin Shi, Mingjun Ren, Yasuyuki Matsushita. DiLiGenRT: A Photometric Stereo Dataset with Quantified Roughness and Translucency. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. [Paper] [Project page]
• Yufei Han, Heng Guo, Koki Fukai, Hiroaki Santo, Boxin Shi, Fumio Okura, Zhanyu Ma, Yunpeng Jia. NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. [arXiv]
• Hiroaki Santo, Fumio Okura, Yasuyuki Matsushita. MVCPS-NeuS: Multi-view Constrained Photometric Stereo for Neural Surface Reconstruction. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. [Paper]
• Siwei Zhou, Youngha Chang, Nobuhiko Mukai, Hiroaki Santo, Fumio Okura, Yasuyuki Matsushita, Shuang Zhao. Path-Space Differentiable Rendering of Implicit Surfaces. Proceedings of ACM SIGGRAPH 2024 Conference, 2024. [Paper] [Project page]
• Takuto Narumoto, Hiroaki Santo, Fumio Okura. Synthesizing time-varying BRDFs via latent space.
  Proceedings of European Conference on Computer Vision (ECCV), 2024. [Paper]
• Kohei Ashida; Hiroaki Santo; Fumio Okura; Yasuyuki Matsushita. Resolving scale ambiguity in multi-view 3D reconstruction using dual-pixel sensors. Proceedings of European Conference on Computer Vision (ECCV), 2024. [Paper]
• Xu Cao, Hiroaki Santo, Fumio Okura, Yasuyuki Matsushita. Multi-View Azimuth Stereo via Tangent Space Consistency. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. [arXiv]
• Kensuke Uchida, Hajime Nagahara, Yasuyuki Matsushita. A Compact BRDF Scanner with Multi-conjugate Optics. Proceedings of the IEEE International Conference on Computational Photography (ICCP), 2023.
• Lilika Makabe, Heng Guo, Hiroaki Santo, Fumio Okura, Yasuyuki Matsushita. Near-light Photometric Stereo with Symmetric Lights Proceedings of the IEEE International Conference on Computational Photography (ICCP), 2023.
•