Title: Enabling the Metaverse with 3D Vision and Generative AI
Date: 2023-05-25 15:30-16:30
Location: CSIE R104
Speaker: Prof. Hao Li, Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)/ CEO and Co-Founder of Pinscreen,
Host: Prof. Shih-Hao Hung
Co-organiser: NTU High Performance and Scientific Computing Center (臺大高效能與科學計算技術研究中心)
In this talk, I will motivate our research ambitions at MBZUAI for enabling the Metaverse. Key capabilities include virtual telepresence using digital humans and enhancing education using new forms of generative content creation.
I will discuss the latest generative AI approachs, from StyleGAN to Diffusion Models, and showcase applications in human digitization. I will present our latest techniques for normalized human synthesis and Pinscreen’s most advanced
Avatar Neo SDK, and how these solutions can enable high-fidelity avatar creation from photos. I will then present various methods for real-time and accessible volumetric capture and rendering for human performances and entire scenes.
Finally, I will also showcase new tools in generative media, where we demonstrated exciting new applications in hollywood such as visual dubbing, digital face replacement, and automated rotoscoping.
Hao Li is an Associate Professor at the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) as well as CEO and Co-Founder of Pinscreen, a startup that builds cutting edge AI-driven virtual avatar technologies. He was previously a Distinguished Fellow of the Computer Vision Group at UC Berkeley and Associate Professor of Computer Science at the University of Southern California, where he was also Director of the USC Institute for Creative Technologies. Hao's work in Computer Vision and Graphics focuses on 3D digitization and neural synthesis for immersive technologies and content creation. His research involves the development of novel deep learning, data-driven, and geometry processing algorithms. He is known for his seminal work in avatar creation, facial animation, hair digitization, dynamic shape processing, as well as his recent efforts in AI media synthesis and deepfake detection. He was also a visiting professor at Weta Digital, a research lead at Industrial Light & Magic / Lucasfilm, and a postdoctoral fellow at Columbia and Princeton Universities. He was named top 35 innovator under 35 by MIT Technology Review in 2013 and was also awarded the Google Faculty Award, the Okawa Foundation Research Grant, as well as the Andrew and Erna Viterbi Early Career Chair. He won the Office of Naval Research (ONR) Young Investigator Award in 2018 and was named named to the DARPA ISAT Study Group in 2019. In 2020, he won the ACM SIGGRAPH Real-Time Live! “Best in Show” award. Hao was speaker at the World Economic Forum in Davos in 2020 and exhibited at SXSW in 2022. His startup, Pinscreen, was recipient of the Epic Megagrants in 2021, and in 2022, Hao was featured in the first season of Amazon's documentary re:MARS Luminaries. Hao obtained his PhD at ETH Zurich and his MSc at the University of Karlsruhe (TH).