Probabilistic Plant Modeling via Multi-View Image-to-Image Translation

Presented in CVPR2018

Takahiro Isokane^1,*, Fumio Okura^1,2,*, Ayaka Ide¹, Yasuyuki Matsushita¹, Yasushi Yagi¹

¹ Osaka University, ² JST PRESTO

^* Equally contributed

Abstract

We propose a method for inferring three-dimensional (3D) plant branch structures that are hidden under leaves from multi-view observations. Unlike previous geometric approaches that heavily rely on the visibility of the branches or use parametric branching models, our method makes statistical inferences of branch structures in a probabilistic framework. By inferring the probability of branch existence using a Bayesian extension of image-to-image translation applied to each of multi-view images, our method generates a probabilistic plant 3D model, which represents the 3D branching pattern that cannot be directly observed. Experiments demonstrate the usefulness of the proposed approach in generating convincing branch structures in comparison to prior approaches.

Download

Citation

@InProceedings{CVPR18_isokane_plant,
  author    = {Isokane, Takahiro and Okura, Fumio and Ide, Ayaka and Matsushita, Yasuyuki and Yagi, Yasushi},
  title     = {Probabilistic plant modeling via multi-view image-to-image translation},
  booktitle = {Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR'18)},
  year      = {2018},
}

Press Release

1. 2018/06/14: 朝日新聞「ミチを開く -人工知能を農業に活用へ-」
2. 2018/05/09: Seamless 「大阪大学、植物を複数方向から撮影した画像から、葉などに隠れた枝構造も正確に再現するDeep learningを用いた手法を発表」
3. 2018/05/07: PHYS.org 他3件 「Invisible Structures Exposed!」
4. 2018/05/07: ScienceDaily 「3D reconstruction of hidden branch structures made by using image analysis and AI tech」
5. 2018/05/04: 読売新聞「隠れた枝もAIで「丸見え」に…阪大チーム開発」
6. 2018/05/01: 毎日新聞「「隠れた枝」推定　３Ｄ画像で栽培効率化へ　阪大チーム　／大阪」
7. 2018/04/27: 日刊工業新聞「葉で隠れた部分の3D植物画像　阪大、AI活用で再現」
8. 2018/04/26: 共同通信　他40件 「AIで植物の枝ぶりを立体画像に 大阪大、葉に隠れた部分も」