NLP @ Michigan Day

If you plan to attend, please fill out the registration form at this link by February 28. Registrants will receive more details in a future email.

Join us to exchange thoughts around natural language processing (NLP).The schedule will include keynote talks, a poster sessions, and roundtable mixer. Lunch will also be served. We hope everyone interested in NLP (with or without doing work in this space) will consider joining! It will be a great way to see old friends, meet new people, learn about cool new research in NLP, and get feedback on your own work.

9:00-10:00       Registration/coffee

10:00-11:00      Keynote: Parisa Kordjamshidi, Michigan State University

Compositional Reasoning for Natural Language Comprehension and Grounding Leveraging Neuro-Symbolic AI 

11:00:-12:00     Poster Session

12:00-1:00        Lunch & Roundtable Discussion Mixer

1:00-2:00          Keynote: Mohit Bansal, University of North Carolina @ Chapel Hill

Trustworthy Planning Agents for Collaborative Reasoning and Multimodal Generation

2:00                    Closing remarks/awards

Compositional Reasoning for Natural Language Comprehension and Grounding Leveraging Neuro-Symbolic AI 

Parisa Kordjamshidi, Associate Professor of Computer Science & Engineering at Michigan State University

Recent research indicates that large language models lack consistent reliability in tasks requiring complex reasoning. While they may impress us with fluently written articles prompted by user input, they can easily disappoint by displaying shortcomings in basic reasoning skills, such as the functional understanding of ‘left is the opposite of right’, let alone grounding such concepts in diverse real-world situations involving perception and action. To address real-world problems, computational models often need to involve multiple interdependent learners, along with significant levels of composition and reasoning. In this talk, I will present our findings regarding reasoning challenges of LLMs and discuss how symbolic representations can leverage the capacity of neural models for compositional reasoning over complex linguistic structures, grounding language in visual perception, combining multiple modalities of information and handling uncertainty. I will highlight our efforts in Neurosymbolic modeling and introduce DomiKnowS, our developed library that facilitates such modeling. The DomiKnowS framework exploits both symbolic and sub-symbolic representations to solve complex, AI-complete problems and seamlessly integrates symbolic and logical knowledge into deep models through various underlying algorithms.

Parisa Kordjamshidi is an Associate Professor of Computer Science & Engineering at Michigan State University. Her research is on Natural Language Processing, Combination of Vision and Language and Neurosymbolic AI. She earned her Ph.D. from KU Leuven in 2013 and conducted a post-doctoral research at the University of Illinois at Urbana-Champaign until 2016. Following that, she had a joint appointment as an Assistant Professor at Tulane University and  Research Scientist at Florida Institute for Human and Cognition, before joining MSU in 2019. She is a recipient of NSF CAREER, Amazon Faculty Research and Fulbright Specialist awards. She leads multiple projects supported by the Office of Naval Research, investigating the integration of symbolic domain knowledge into  deep neural modeling and the combination of vision and language modalities. Kordjamshidi is also a member of the Editorial Board of the Journal of Artificial Intelligence Research (JAIR) and Action Editor of Transactions of the ACL journal. She has served as a senior area chair and a senior program committee member for major conferences in the ACL, AI and ML communities. She has contributed to the organization of major conferences such as NAACL, EACL, EMNLP, ECML-PKDD, AAAI.

Trustworthy Planning Agents for Collaborative Reasoning and Multimodal Generation

Mohit Bansal, John R. & Louise S. Parker Distinguished Professor and the Director of the MURGe-Lab (UNC-NLP Group) in the Computer Science department at UNC Chapel Hill

In this talk, I will present our journey of developing diverse, adaptive, uncertainty-calibrated AI planning agents that can robustly communicate and collaborate for multi-agent reasoning (on math, commonsense, coding, etc.) as well as for interpretable, controllable multimodal generation (across text, images, videos, audio, layouts, etc.). In the first part, we will discuss improving reasoning via multi-agent discussion among diverse LLMs and its structured distillation to smaller, open-source models (ReConcile, MAGDi), as well as making LLMs better teammates through confidence calibration (using speaker-listener pragmatic reasoning) and by teaching them to accept/reject persuasion as appropriate. In the second part, we will discuss interpretable and controllable multimodal generation via LLM-agents based planning and programming, such as layout-controllable image generation (and evaluation) via visual programming (VPGen+VPEval), consistent multi-scene video generation via LLM-guided planning (VideoDirectorGPT), interactive and composable any-to-any multimodal generation (CoDi, CoDi-2), as well as multi-agent interaction for adaptive environment/data generation based on discovered weak skills (EnvGen, DataEnvGym).

Dr. Mohit Bansal is the John R. & Louise S. Parker Distinguished Professor and the Director of the MURGe-Lab (UNC-NLP Group) in the Computer Science department at UNC Chapel Hill. He received his PhD from UC Berkeley in 2013 and his BTech from IIT Kanpur in 2008. His research expertise is in natural language processing and multimodal machine learning, with a particular focus on multimodal generative models, grounded and embodied semantics, reasoning and planning agents,  faithful language generation, and interpretable, efficient, and generalizable deep learning. He is a AAAI Fellow and recipient of the Presidential Early Career Award for Scientists and Engineers (PECASE), IIT Kanpur Young Alumnus Award, DARPA Director’s Fellowship, NSF CAREER Award, Google Focused Research Award, Microsoft Investigator Fellowship, Army Young Investigator Award (YIP), DARPA Young Faculty Award (YFA), and outstanding paper awards at ACL, CVPR, EACL, COLING, CoNLL, and TMLR. He has been a keynote speaker for the AACL 2023, CoNLL 2023, and INLG 2022 conferences. His service includes EMNLP and CoNLL Program Co-Chair, and ACL Executive Committee, ACM Doctoral Dissertation Award Committee, ACL Americas Sponsorship Co-Chair, and Associate/Action Editor for TACL, CL, IEEE/ACM TASLP, and CSL journals. Webpage: https://www.cs.unc.edu/~mbansal/