Work Experience [CV], R&D [Google Scholar|DBLP]
- Head Large Language Models, Oct 2020 - present
Facebook AI Research, USA
- Leading & managing Large Language Model efforts and teams
- Read the latest reserch papers published at top tier conferences
- Workshop co-organizer:- Simple and Efficient Natural Language Processing, EMNLP 2022 & 2021
- Structured Prediction for NLP, ACL-IJCNLP 2021
- Board Member (Industry Advisory Board) , Sept 2020 - present
University of California, Santa Cruz
- Program Chair EMNLP 2022, Oct 2021 - Dec 2022
- Senior Leader & Manager, Dec 2017 - Oct 2020
Google, USA
- Our work on On-device Neural Networks for Natural Language Processing and Conversational AI
(featured in: VentureBeat, AndroidHeadlines, GizBot, NextBigWhat, Silicon Angle)
- Read our latest On-device NLP & Conversational AI papers published in EACL 2021, EMNLP-IJCNLP 2019, ACL 2019, NAACL 2019 & EMNLP 2018
- Workshops co-organizer Structured Prediction for NLP (@EMNLP 2020 & @NAACL 2019), On-device Machine Learning and Compact Deep Neural Network Representations @ICML 2019 - Applied Science Manager, June 2016 - Dec 2017
Amazon, AWS Deep Learning, USA
- Amazon Comprehend - Natural Language Processing for Cloud
- Amazon Lex - Conversational Bots for Cloud (watch my TWiML interview)
- Amazon Alexa - Natural Language Understanding - Senior Manager Engineering, Jan 2014 - June 2016
Yahoo!, USA, Query Processing & User Intent
- Mobile Search and Product Ads
- Query Processing and User Intent Understanding
- Knowledge Graph - Research Assistant Professor & Scientist, 2009-2014
University of Southern California & ISI, USA
- Machine Reading, Large Scale Knowledge Extraction from the Web, Information Extraction, Sentiment Analysis, Graph Algorithms for Natural Language Processing
Awards
- EACL 2021 Best Paper Honorable Mntion
- John Atanasoff Award by the President of Republic of Bulgaria 2016
- Yahoo! Choice Award Hack Winner, 2015
- Yahoo! Choice Award Hack Winner, 2014
- LEAP Award (Labs Excellence Award Program) Yahoo! Labs, 2014
- RANLP Young Researcher Award, 2011
Recent Tech Talks [All]
- Robust & Efficient On-device Neural Text Representations via Projections
The 7th Workshop on Noisy User-generated Text, November 2021 - Fast and Small On-device Neural Networks for Natural Language Understanding & Conversational AI
University of California Santa Cruz, October 2020 - Conversational AI on the Edge
Stanford University, October 2019 - Efficient On-device Neural Models for Natural Language Processing
Salesforce Research, April 2019 - German-American Frontiers of Engineering
GAFOE 2019, Germany, March 2019 - On-device Neural Networks for Natural Language Processing
ReWork Deep Learning Summit 2019, San Francisco, January 2019 - Deep Learning Models for Conversational Assistants and Question Answering
RIKEN, Tokyo, Japan, October 2017 - Conversational Assistants with Deep Learning
Data Day, Seattle, October 2017 - Teaching Machines to Communicate with People Using Deep Learning
IEEE Rock Stars of Machine Learning and Deep Learning, Santa Clara, September 2017 - Natural Language Understanding for Amazon Alexa and AWS Lex [Youtube & SoundCloud]
Interview with This Week in Machine Learning & AI, July 2017 - Deep Learning for Natural Language Processing and Dialog Systems
Association of Pattern Recognition and Image Analysis Summer School, Spain, July 2017 - Natural Language Understanding for Conversational Assistants
Machine Learning Innovation Summit, San Fransicos, June 2017 - What Do Users Want? Using Semantics to Predict User Intents at Scale
ReWork Machine Intelligence Summit, San Francisco, March 2017
Organizer [All]
- 2nd Workshop on Simple and Efficient Natural Language Processing (SustaiNLP), EMNLP 2021
- 5th Workshop on Structured Prediction for NLP, ACL-IJCNLP 2021
- 4th Workshop on Structured Prediction for NLP, EMNLP 2020
- Workshop on On-Device Machine Learning & Compact Deep Neural Network Representations (ODML-CDNNR), ICML 2019
- 3rd Workshop on Structured Prediction for NLP, NAACL 2019
Senior Area Chair & Area Chair [Full List]
- Demo Chair, ACL 2022
- Area Chair, AAAI 2022
- Senior Area Chair for Information Extraction, NAACL-HLT 2021
- Area Chair for Question Answering, ACL-IJCNLP 2021
- Area Chair for Multidisciplinary NLP, EMNLP 2021
- Area Chair, AAAI 2021
- Area Chair for Machine Learning, ACL 2020
- Senior Area Chair for Question Answering, ACL 2019
- Area Chair for Dialog and Interactive systems, NAACL 2019
- Area Chair for Question Answering, ACL 2018
- Area Chair for Text Mining, NAACL 2018
- Area Chair for Question Answering, ACL 2017
Recent Publications [Google Scholar, DBLP]
- Efficient Large Scale Language Modeling with Mixtures of Experts
Mikel Artetxe, Shruti Bhosale, Naman Goyal, Todor Mihaylov, Myle Ott, Sam Shleifer, Xi Victoria Lin, Jingfei Du, Srinivasan Iyer, Ramakanth Pasunuru, Giri Anantharaman, Xian Li, Shuohui Chen, Halil Akin, Mandeep Baines, Louis Martin, Xing Zhou, Punit Singh Koura, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Mona Diab, Zornitsa Kozareva, Ves Stoyanov
In Arxiv, 2021. - Few-shot Learning with Multilingual Language Models
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O'Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li.
In Arxiv, 2021. - Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs
Peter Hase, Mona Diab, Asli Celikyilmaz, Xian Li, Zornitsa Kozareva, Veselin Stoyanov, Mohit Bansal, Srinivasan Iyer.
In Arxiv, 2021 - Fixed Support Tree-Sliced Wasserstein Barycenter
Yuki Takezawa, Ryoma Sato, Zornitsa Kozareva, Sujith Ravi and Makoto Yamada
in Arxiv, 2021. - SoDA: On-device Conversational Slot Extraction
Sujith Ravi and Zornitsa Kozareva
in Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL 2021 - ProFormer: Towards On-Device LSH Projection Based Transformers
Chinnadhurai Sankar, Sujith Ravi and Zornitsa Kozareva
in Proceedings of the European Chapter of the Association for Computational Linguistics, EACL 2021
*Best Paper Honorable Mention (Twitter & LinkedIn) - On-Device Text Representations Robust To Misspellings via Projections
Chinnadhurai Sankar, Sujith Ravi and Zornitsa Kozareva
in Proceedings of the European Chapter of the Association for Computational Linguistics, EACL 2021 - ProSeqo: Projection Sequence Networks for On-Device Text Classification
Zornitsa Kozareva and Sujith Ravi
in Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019 - PRADO: Projection Attention Networks for Document Classification On-Device
Prabhu Kaliamoorthi, Sujith Ravi and Zornitsa Kozareva
in Proceedings of the Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019 - On-device Structured and Context Partitioned Projection Networks
Sujith Ravi and Zornitsa Kozareva
in Proceedings of the Association for Computational Linguistics, ACL 2019 - Transferable Neural Projection Representations
Chinnadhurai Sankar, Sujith Ravi and Zornitsa Kozareva
in Proceedings of the North American Chapter of the Association for Computational Linguistics, NAACL 2019 - Self-Governing Neural Networks for On-Device Short Text Classification
Sujith Ravi and Zornitsa Kozareva
in Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2018 - Learning Steady-States of Iterative Algorithms over Graphs
Hanjun Dai, Zornitsa Kozareva, Bo Dai, Alexander Smola and Le Song
in Proceedings of the International Conference on Machine Learning, ICML 2018 - Variational Reasoning for Question Answering with Knowledge Graph
Yuyu Zhang, Hanjun Dai, Zornitsa Kozareva, Alexander Smola and Le Song
in Proceedings of AAAI Conference on Artificial Intelligence, AAAI 2018