Luke Vilnis

Google DeepMind





lvilnis@gmail.com

GitHub | LinkedIn | Google Scholar | CV

I am a research scientist at Google DeepMind. I finished my Ph.D. in 2020 in the Information Extraction and Synthesis Lab at UMass Amherst, where I was advised by Professor Andrew McCallum. Previously I was a software engineer and consultant on business intelligence for the financial sector, working on compilers, query engines, financial math and data visualization. I graduated from Duke University in 2010 with a B.S. in Mathematics and Economics. I received my M.S. in Computer Science from the University of Massachusetts Amherst in 2015, passing the Ph.D. candidacy portfolio With Distinction.

I interned with Paul Mineiro and Nikos Karampatziakis in the Microsoft Cloud Information Services Laboratory in the summer of 2014. I interned with Andrew Dai on the Google Brain team in the summer of 2015. I interned with George Dahl on the Google Brain team in the summer of 2016.

These days I work on Gemini. I am broadly interested in machine learning, natural language processing, and knowledge base construction, with an emphasis on probabilistic modeling. Areas of interest include representation learning, machine reasoning, structured prediction, and common sense knowledge, as well as applications in the biomedical and sustainable energy domains. I have been the fortunate co-author of multiple publications awarded Best or Outstanding Paper, since two is a multiple of one.

I am also an armchair PLT and functional programming enthusiast, and erstwhile chief maintainer of the factorie machine learning toolkit.

Publications

• CodeGemma: Open Code Models Based on Gemma. CodeGemma Team. Technical report, 2024. 
  

• Gemini 1.5: Unlocking Multimodal Understanding Across Millions of Tokens of Context. Gemini Team. Technical report, 2024. 
  

• MEMORY-VQ: Compression for Tractable Internet-Scale Memory.  Yury Zemlyanskiy^*, Michiel de Jong^*, Luke Vilnis, Santiago Ontañón, William W. Cohen, Sumit Sanghai, Joshua Ainslie.  Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), 2024. ^* Equal contribution.
  

• Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models.  Luke Vilnis^*, Yury Zemlyanskiy^*, Patrick Murray, Alexandre Passos, Sumit Sanghai.  International Conference on Machine Learning (ICML), 2023. ^* Equal contribution. Oral presentation.
  

• ImPaKT: A Dataset for Open-Schema Knowledge Base Construction.  Luke Vilnis, Zach Fisher, Bhargav Kanagal, Patrick Murray, Sumit Sanghai.  Preprint, 2022. 

• Capacity and Bias of Learned Geometric Embeddings for Directed Graphs.  Michael Boratko^*, Dongxu Zhang^*, Nicholas Monath, Luke Vilnis, Kenneth L. Clarkson, Andrew McCallum.  Neural Information Processing Systems (NeurIPS), 2021. ^* Equal contribution.
  

• Geometric Representation Learning.  Luke Vilnis.  PhD thesis, 2021. 

• Improving Local Identifiability for Probabilistic Box Embeddings.  Shib Sankar Dasgupta^*, Michael Boratko^*, Dongxu Zhang, Luke Vilnis, Xiang Lorraine Li, Andrew McCallum.  Neural Information Processing Systems (NeurIPS), 2020. ^* Equal contribution.
  

• Representing Joint Hierarchies with Box Embeddings. Dhruvesh Patel^*, Shib Sankar Dasgupta^*, Michael Boratko, Xiang Li, Luke Vilnis, Andrew McCallum. Automated Knowledge Base Construction (AKBC), 2020. ^* Equal contribution.
  

• Smoothing the Geometry of Probabilistic Box Embeddings.  Xiang Li^*, Luke Vilnis^*, Dongxu Zhang, Michael Boratko, and Andrew McCallum.  International Conference on Learning Representations (ICLR), 2019.  ^* Equal contribution. Oral presentation.
  

• Embedded-State Latent Conditional Random Fields for Sequence Labeling.  Dung Thai, Sree Harsha Ramesh, Shikhar Murty, Luke Vilnis and Andrew McCallum.  Conference on Computational Natural Language Learning (CoNLL), 2018.  
  

• Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures.  Luke Vilnis^*, Xiang Li^*, Shikhar Murty, Andrew McCallum.  Annual Meeting of the Association for Computational Linguistics (ACL), 2018.  ^* Equal contribution.
  

• Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking.  Shikhar Murty, Patrick Verga, Luke Vilnis, Irena Radonvanovic, Andrew McCallum.  Annual Meeting of the Association for Computational Linguistics (ACL), 2018.  Oral presentation.

• Learning Conditionally Calibrated Equations of State for Direct Fired sCO₂ Cycles with Deep Neural Networks.  Luke Vilnis, David Freed, Navid Rafati, Joe Camilo, Andrew McCallum.  The 6th International Supercritical CO2 Power Cycles Symposium (sCO2), 2018.
  

• Unsupervised Hypernym Detection by Distributional Inclusion Vector Embedding. Haw-Shiuan Chang, ZiYun Wang, Luke Vilnis, Andrew McCallum.  Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), 2018.
  

• Go for a Walk and Arrive at the Answer: Reasoning Over Knowledge Bases with Reinforcement Learning. Rajarshi Das^*, Shehzaad Dhuliawala^*, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola and Andrew McCallum.  International Conference on Learning Representations (ICLR), 2018.  ^* Equal contribution.
  

• Finer Grained Entity Typing with TypeNet. Shikhar Murty, Patrick Verga, Luke Vilnis, Andrew McCallum. Neural Information Processing Systems Workshop on Automated Knowledge Base Construction (AKBC), 2017.
  

• (Workshop Version) Go for a Walk and Arrive at the Answer: Reasoning Over Knowledge Bases with Reinforcement Learning. Rajarshi Das^*, Shehzaad Dhuliawala^*, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola and Andrew McCallum.  Neural Information Processing Systems Workshop on Automated Knowledge Base Construction (AKBC), 2017.  ^* Equal contribution. Oral presentation. Best paper award.
  

• Improved Representation Learning for Predicting Commonsense Ontologies. Xiang Li, Luke Vilnis, Andrew McCallum.  International Conference on Machine Learning Workshop on Deep Structured Prediction (ICML WS), 2017.  
  

• Low-Rank Hidden State Embeddings for Viterbi Sequence Labeling. Dung Thai, Shikhar Murty, Trapit Bansal, Luke Vilnis, David Belanger, Andrew McCallum.  International Conference on Machine Learning Workshop on Deep Structured Prediction (ICML WS), 2017.  
  

• Generating Sentences from a Continuous Space.  Samuel Bowman^*, Luke Vilnis^*, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, Samy Bengio.  Conference on Computational Natural Language Learning (CoNLL), 2016.   ^* Equal contribution. Oral presentation.
  

• Adding Gradient Noise Improves Learning for Very Deep Networks.  Arvind Neelakantan^*, Luke Vilnis^*, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens.  International Conference on Learning Representations Workshop (ICLR WS), 2016.   ^* Equal contribution.
  

• Bethe Projections for Non-Local Inference.  Luke Vilnis^*, David Belanger^*, Daniel Sheldon, Andrew McCallum.  Uncertainty in Artificial Intelligence (UAI), 2015.   ^* Equal contribution.
  

• Learning Dynamic Feature Selection for Fast Sequential Prediction.  Emma Strubell, Luke Vilnis, Kate Silverstein, Andrew McCallum.  Annual Meeting of the Association for Computational Linguistics (ACL), 2015. Oral presentation. Outstanding paper award.
  

• Word Representations via Gaussian Embedding.  Luke Vilnis, Andrew McCallum.  International Conference on Learning Representations (ICLR), 2015. Oral presentation.
  

• Generalized Eigenvectors for Large Multiclass Problems.  Luke Vilnis, Nikos Karampatziakis, Paul Mineiro.  Neural Information Processing Systems Workshop on Representation and Learning Methods for Complex Outputs (NIPS WS), 2014. Oral presentation.
  

• Training for Fast Sequential Prediction Using Dynamic Feature Selection.  Emma Strubell, Luke Vilnis, Andrew McCallum.  Neural Information Processing Systems Workshop on Modern Machine Learning and Natural Language Processing (NIPS WS), 2014.
  

• Optimization and Learning in Factorie.  Alexandre Passos, Luke Vilnis, Andrew McCallum.  Neural Information Processing Systems Workshop on Optimization for Machine Learning (NIPS WS), 2013.
  

• Dynamic Knowledge Base Alignment for Coreference Resolution.  Jiaping Zheng, Luke Vilnis, Sameer Singh, Jinho Choi, Andrew McCallum.  Conference on Computational Natural Language Learning (CoNLL), 2013.
  

Tutorials/Notes

These are some notes and tutorials I've worked on for class projects and for fun.

• Some assorted tips for people using MacBooks for ML, scientific computing, etc. Advice on how to set up their tooling and workflow, what libraries to use, how to use the shell, lots of things. Assumes basically no knowledge.
  
  
• Markov Chain Monte Carlo, Mixing, and the Spectral Gap.  Luke Vilnis, 5/12/2013.