Title: Open-Source AI Projects at UC Berkeley & LMSys: Vicuna and Chatbot Arena
Date: 2023-12-22, 14:20-15:30
Location: CSIE R102
Speaker: Wei-Lin Chiang, PhD student, UC Berkeley
Host: Prof. Chih-Jen Lin
Abstract
In this talk, I will share experience learned from the two pioneering open-source AI projects at UC Berkeley & LMSys : Vicuna and Chatbot Arena.
Vicuna is one of the first high-quality open LLM released in March. Building on the foundational work of Meta's Llama, Vicuna is trained by fine-tuning on user-shared conversations collected from ShareGPT, showing ChatGPT-like conversation capability. It has enabled follow-up research in developing vision-language model, AI safety research, etc. Our blog post has garnered 500+ citations and millions of downloads on HuggingFace in a few months of its release.
Next I'll introduce Chatbot Arena, which is an open & crowdsourced platform for evaluating large language models (LLMs) with human feedback. It allows users to interact with two anonymous models side-by-side, and vote for the one they believe provides better responses. The platform uses the Elo rating system, a method commonly used in chess and other competitive games, to rank the performance of the chatbots. Over the past few months, Chatbot Arena has served millions of user requests and collected 150K+ human preference votes to rank 45+ cutting-edge LLMs. We also make the datasets of user conversations and human preferences publicly available for future research.
Demo: https://chat.lmsys.org
Learn more about our work: https://lmsys.org/blog/
Bio
Wei-Lin Chiang is currently a PhD student at UC Berkeley working with Prof. Ion Stoica. He obtained his M.S. and B.S. from National Taiwan University under the supervision of Prof. Chih-Jen Lin.
His research is at the intersection of AI systems and cloud computing. One of his notable projects, SkyPilot, is an intercloud broker system tailored for AI workloads. The open-source system was published at NSDI'23 and adopted by many AI labs and organizations. Wei-Lin is also behind the creation of Vicuna, a high-quality open-source LLM fine-tuned from Llama, and FastChat, an open platform for training, serving, and evaluating Chat LLMs. More details can be found on his personal website. https://infwinston.github.io/