KIST 시각지능 연구팀

본 강연에서는 영상속 상황인식을 위한 다양한 고수준 장면이해 기법을 소개한다. 구체적인 태스크로는 장면그래프 생성 및 사람 사물 상호작용 탐지 기법을 소개하며, 초거대 인공지능 모델을 활용하여 해당 태스크를 향상시키는 방안에 대하여 논의한다. 또한 추가적으로 2D 영상 인식을 넘어 3D 영상을 위한 데이터 증강기법 및 새로운 네트워크 구조를 소개한다.

In this talk, I will present two pieces of my recent research, which improves 3D-aware image synthesis techniques by leveraging the knowledge of pretrained multimodal embedding model called CLIP. In the first part of the talk, I will introduce a novel approach for text-driven 3D-aware image editing that applies the editing to the local relevant regions by leveraging the pseudo mask information computed from the CLIP model. In the second part, I will introduce FaceCLIPNeRF, which can edit given facial images as well as perform its novel-view synthesis. Finally, I will conclude the talk by discussing future research directions.

3D scanning of a single view of an object seldom suffices. Be it for 3D printing, augmented reality, or virtual reality, scanning of the shape of the entire object in all its complexity-what we refer to as "full-surround 3D"-is critical to have a faithful digital twin. In this talk, I will present a full-surround 3D imaging system that incorporates multiple mirrors to exploit inter-reflections. This system, when paired with structured light or neural rendering, enables us to reconstruct, with high accuracy and full coverage, highly complex objects that have intricate geometric features, including concavities and self-occlusions.

In this talk, I will introduce a general trend in machine learning (ML) for solving physics/simulation problems, especially within the computer graphics community. Since the rise of deep learning and AI, numerous approaches have emerged for fast, large-scale, and high-fidelity physics simulations. Recently, the field has been advancing at an even faster pace due to the power of new AI models, such as generative AI and neural representations. I will present and review some of the recent studies and then discuss how the intersection of ML and physics will evolve.

Text를 입력으로 받아 3차원 모델을 만드는 기술은 최근에 Diffusion Model과 Neural Radiance Fields (NeRF)의 성공에 힘입어 실제적인 적용이 가능한 수준의 결과들이 공개되고 있다. 하지만 이러한 기존의 기술은 2차원 Diffusion Model에만 의존하다보니 생성된 3차원 모델이 View-inconsistent 한 결과를 자주 보여준다. 본 세미나에서는 이러한 View-inconsistency 문제를 해결하기 위한 연구들을 소개하고 향후 방향성에 대해서 논의해보고자 한다.