Adaptive Computation in Language Models: Does Every Token Require the Same Effort?

April 11, 2025

10:30 am - 11:30 am

Pomerene Hall, Room 320

Add to Calendar 2025-04-11 10:30:00 2025-04-11 11:30:00 Adaptive Computation in Language Models: Does Every Token Require the Same Effort? TDAI's Foundations of Data Science and AI’s Community of Practice with Co-Directors Huan Sun and Subhadeep Paul are welcoming Xifeng Yan, Professor at the University of California at Santa Barbara, to share insights and introduce some early ideas for tackling these challenges to inspire further research beyond the Transformer architecture. Yan will also briefly discuss their ongoing AI research in materials science and finance. About the ProgramTransformer-based large language models (LLMs) have achieved remarkable success, yet significant challenges remain. In this talk, I will explore two key questions: (1) Does every token require the same level of computational effort? and (2) Why do LLMs sometimes struggle with seemingly simple tasks, such as arithmetic operations? To address these questions, techniques like mixture of experts (MoE), speculative decoding, and early exit strategies have been developed to dynamically adjust computational demands based on task complexity.  However, these approaches are not sufficient. I will share our insights and introduce some early ideas for tackling these challenges, to inspire further research beyond the Transformer architecture. Lastly, I will briefly discuss our ongoing AI research in materials science and finance. About the SpeakerXifeng Yan is a professor at the University of California at Santa Barbara. He holds the Venkatesh Narayanamurti Chair of Computer Science. He received his Ph.D. degree in Computer Science from the University of Illinois at Urbana-Champaign in 2006. He was a research staff member at the IBM T. J. Watson Research Center between 2006 and 2008. His work is centered on knowledge discovery, knowledge bases, and artificial intelligence. He received the NSF CAREER Award, IBM Invention Achievement Award, ACM-SIGMOD Dissertation Runner-Up Award, IEEE ICDM 10-year Highest Impact Paper Award, 2022 PLDI Distinguished Paper Award, 2022 VLDB Test of Time Award, and the first-place prize of Amazon SocialBot Grand Challenge 5. His team is the first leveraging Transformer for time-series forecasting, opening a new area. Pomerene Hall, Room 320 OSU ASC Drupal 8 ascwebservices@osu.edu America/New_York public

2025-04-11 10:30:00 2025-04-11 11:30:00 Adaptive Computation in Language Models: Does Every Token Require the Same Effort? TDAI's Foundations of Data Science and AI’s Community of Practice with Co-Directors Huan Sun and Subhadeep Paul are welcoming Xifeng Yan, Professor at the University of California at Santa Barbara, to share insights and introduce some early ideas for tackling these challenges to inspire further research beyond the Transformer architecture.  Yan will also briefly discuss their ongoing AI research in materials science and finance.   About the ProgramTransformer-based large language models (LLMs) have achieved remarkable success, yet significant challenges remain. In this talk, I will explore two key questions: (1) Does every token require the same level of computational effort? and (2) Why do LLMs sometimes struggle with seemingly simple tasks, such as arithmetic operations? To address these questions, techniques like mixture of experts (MoE), speculative decoding, and early exit strategies have been developed to dynamically adjust computational demands based on task complexity.  However, these approaches are not sufficient. I will share our insights and introduce some early ideas for tackling these challenges, to inspire further research beyond the Transformer architecture. Lastly, I will briefly discuss our ongoing AI research in materials science and finance.  About the SpeakerXifeng Yan is a professor at the University of California at Santa Barbara. He holds the Venkatesh Narayanamurti Chair of Computer Science. He received his Ph.D. degree in Computer Science from the University of Illinois at Urbana-Champaign in 2006. He was a research staff member at the IBM T. J. Watson Research Center between 2006 and 2008. His work is centered on knowledge discovery, knowledge bases, and artificial intelligence. He received the NSF CAREER Award, IBM Invention Achievement Award, ACM-SIGMOD Dissertation Runner-Up Award, IEEE ICDM 10-year Highest Impact Paper Award, 2022 PLDI Distinguished Paper Award, 2022 VLDB Test of Time Award, and the first-place prize of Amazon SocialBot Grand Challenge 5. His team is the first leveraging Transformer for time-series forecasting, opening a new area. Pomerene Hall, Room 320 America/New_York public

TDAI's Foundations of Data Science and AI’s Community of Practice with Co-Directors Huan Sun and Subhadeep Paul are welcoming Xifeng Yan, Professor at the University of California at Santa Barbara, to share insights and introduce some early ideas for tackling these challenges to inspire further research beyond the Transformer architecture. Yan will also briefly discuss their ongoing AI research in materials science and finance.

About the Program

Transformer-based large language models (LLMs) have achieved remarkable success, yet significant challenges remain. In this talk, I will explore two key questions: (1) Does every token require the same level of computational effort? and (2) Why do LLMs sometimes struggle with seemingly simple tasks, such as arithmetic operations? To address these questions, techniques like mixture of experts (MoE), speculative decoding, and early exit strategies have been developed to dynamically adjust computational demands based on task complexity.  However, these approaches are not sufficient. I will share our insights and introduce some early ideas for tackling these challenges, to inspire further research beyond the Transformer architecture. Lastly, I will briefly discuss our ongoing AI research in materials science and finance.

About the Speaker

Xifeng Yan is a professor at the University of California at Santa Barbara. He holds the Venkatesh Narayanamurti Chair of Computer Science. He received his Ph.D. degree in Computer Science from the University of Illinois at Urbana-Champaign in 2006. He was a research staff member at the IBM T. J. Watson Research Center between 2006 and 2008. His work is centered on knowledge discovery, knowledge bases, and artificial intelligence. He received the NSF CAREER Award, IBM Invention Achievement Award, ACM-SIGMOD Dissertation Runner-Up Award, IEEE ICDM 10-year Highest Impact Paper Award, 2022 PLDI Distinguished Paper Award, 2022 VLDB Test of Time Award, and the first-place prize of Amazon SocialBot Grand Challenge 5. His team is the first leveraging Transformer for time-series forecasting, opening a new area.

Events Filters:

TDAI Event

Foundations of Data Science & AI CoP