e-mail: abhshkdz@meta.

com
Abhishek Das webpage: abhishekdas.com

Interests Climate change, computer vision, language, reinforcement learning, reasoning & interpretability.

Employment Fundamental AI Research (FAIR), Meta 2020 - present

Research Scientist

Education Georgia Tech 2016 - 2020

Ph.D. in Computer Science. Thesis: “Building Agents that can See, Talk, and Act".
Committee: Dhruv Batra (advisor), Devi Parikh, James Hays, Joelle Pineau, Jitendra Malik

Indian Institute of Technology, Roorkee 2011 - 2015

B.Tech. in Electrical Engineering

Internships Tesla Autopilot, Palo Alto Summer 2019

With Andrej Karpathy
Worked on automated architecture search for the base neural network powering the vision system.

DeepMind, London Spring 2019

With Laura Rimell, Stephen Clark and Felix Hill
Studied the use of question answering as a general paradigm to probe and evaluate symbolic
knowledge in an embodied predictive agent’s world beliefs. Published at ICML 2020.

Facebook AI Research, Montréal Summer 2018

With Mike Rabbat and Joelle Pineau
Worked on targeted communication architectures in multi-agent RL. Published at ICML 2019.

Facebook AI Research, Menlo Park Summer 2017, Spring 2018

With Georgia Gkioxari, Devi Parikh and Dhruv Batra
Developed Embodied Question Answering – a new task combining active perception, language,
and action (published as an oral at CVPR 2018) – and modular, hierarchical navigation policies for
EmbodiedQA, trained using reinforcement learning (published as a spotlight at CoRL 2018).

Virginia Tech, Blacksburg 2015 - 2016

With Dhruv Batra
Studied role of attentional mechanisms in visual question answering. Published at EMNLP 2016.

Queensland Brain Institute, Brisbane, Australia Winter 2013, 2014

With Geoffrey Goodhill
Studied population neural coding in the zebrafish visual system. Work presented at COSYNE ‘15.

Google Summer of Code Summer 2013, 2014

2014: Open Web Application Security Project (OWASP).
2013: Department of Biomedical Informatics, Emory University.

Fellowships, AAAI/ACM SIGAI Dissertation Award, Runner-up (most significant Ph.D. theses in AI globally) 2022
Awards, & Georgia Tech Sigma Xi Best Ph.D. Thesis Award (1 of 10 awardees from Georgia Tech) 2021
Recognition Georgia Tech College of Computing Dissertation Award (1 of 3 awardees) 2021
Facebook Graduate Fellowship (1 of 20 awardees from 900+ applicants) 2019-21
 Microsoft Research Ph.D. Fellowship (declined) 2019-21
NVIDIA Graduate Fellowship (1 of 10 awardees from 230 applicants; declined) 2019-20
Adobe Research Fellowship (1 of 8 awardees) 2018
Snap Inc. Research Fellowship (1 of 12 awardees from 100+ applicants) 2018
Outstanding GRA Award, College of Computing, Georgia Tech 2019
Among top 30% reviewers, NeurIPS 2018 2018
Outstanding Reviewer Award, CVPR 2017 (Among top 3.6% reviewers) 2017
Best Student Paper Award, ICML 2016 Workshop on Visualization for Deep Learning 2016
The University of Queensland Research Scholarship 2013 & 2014
IIT Roorkee Heritage Foundation Award (For academic and extra-curricular achievements.) 2014
1st, Deloitte Collegiate Cyber Threat Competition 2013 & 2014
1st, Microsoft Code.Fun.Do., IIT Roorkee (blog.sdslabs.co/2014/02/code-fun-do) 2014
1st, Yahoo! HackU!, IIT Delhi (blog.sdslabs.co/2012/09/hacku) 2012
1st, Adobe Express Apps, IIT Roorkee 2012

Invited Talks Open Catalyst Project

& Panels Texas A&M University, College Station 2022
Microsoft Research 2022
Indian Institute of Technology Roorkee 2020

Towards Agents that can See, Talk, and Act

Vision & Language seminar series, Arizona State University 2021
Indian Institute of Technology Roorkee 2020
Microsoft Research 2019
Google Research 2019
Facebook AI Research 2019
Allen Institute for Artificial Intelligence [video] 2018, 2019
Montréal Institute for Learning Algorithms 2018
Indian Institute of Technology Kanpur 2018
Toyota Technological Institute at Chicago 2018

Probing Emergent Semantics in Predictive Agents via Question Answering

Natural Language Understanding workshop by Apple 2020
ICML, Oral [video] 2020

Targeted Multi-Agent Communication

ICML, Oral (Targeted Multi-Agent Communication) [video 1:06:22+] 2019
ICML Workshop: Imitation, Intent, and Interaction [video] 2019

Embodied Question Answering

CoRL, Spotlight (Neural Modular Control for Embodied Question Answering) 2018
SIGDIAL Special Session on Physically Situated Dialogue 2018
CVPR, Oral (Embodied Question Answering) [video] 2018
NVIDIA GTC [video] 2018

Visual Dialog
ICCV, Oral (Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning) [video] 2017
CS 8803 – Vision & Language, Georgia Tech 2017
CVPR Workshop: Language & Vision, Spotlight 2017
CVPR, Spotlight (Visual Dialog) [video] 2017
 CVPR Workshop: Visual Question Answering Challenge (invited talk + panel) [video] 2017

Towards Transparent Visual Question Answering Systems

ICML Workshop: Visualization for Deep Learning [video] 2016

Preprints [44] Generalizing Denoising to Non-Equilibrium Structures Improves Equivariant Force Fields
Y.-L. Liao, T. Smidt, A. Das
arXiv preprint 2403.09549, 2024.

[43] The Open DAC 2023 Dataset and Challenges for Sorbent Discovery in Direct Air Capture
A. Sriram, S. Choi, X. Yu, L. M. Brabson, A. Das, Z. Ulissi, M. Uyttendaele, A. J. Medford,
D. S. Sholl
arXiv preprint 2311.00341, 2023.

[42] Rotation Invariant Graph Neural Networks using Spin Convolutions

M. Shuaibi, A. Kolluru, A. Das, A. Grover, A. Sriram, Z. Ulissi, C. L. Zitnick
arXiv preprint 2106.09575, 2021.

[41] An Intro to Electrocatalyst Design using Machine Learning for Renewable Energy Storage
C. L. Zitnick, L. Chanussot, A. Das, S. Goyal, J. Heras-Domingo, C. Ho, W. Hu, T. Lavril,
A. Palizhati, M. Rivière, M. Shuaibi, A. Sriram, K. Tran, B. Wood, J. Yoon, D. Parikh, Z. Ulissi
opencatalystproject.org, 2020.

Journal [40] AdsorbML: A Leap in Efficiency for Adsorption Energy Calculations Using Generalizable
Articles Machine Learning Potentials
J. Lan*, A. Palizhati*, M. Shuaibi*, B. M. Wood*, B. Wander, A. Das, M. Uyttendaele,
C. L. Zitnick, Z. W. Ulissi
npj Computational Materials, 2023.

[39] The Open Catalyst 2022 (OC22) Dataset and Challenges for Oxide Electrocatalysis
R. Tran*, J. Lan*, M. Shuaibi*, S. Goyal*, B. M. Wood*, A. Das, J. Heras-Domingo, A. Kolluru,
A. Rizvi, N. Shoghi, A. Sriram, Z. Ulissi, C. L. Zitnick
ACS Catalysis, 2023.

[38] GemNet-OC: Developing Graph Neural Networks for Large and Diverse Molecular
Simulation Datasets
J. Gasteiger, M. Shuaibi, A. Sriram, S. Günnemann, Z. Ulissi, C. L. Zitnick, A. Das
Transactions of Machine Learning Research (TMLR), 2022.

[37] Open Challenges in Developing Generalizable Large Scale Machine Learning Models for
Catalyst Discovery
A. Kolluru*, M. Shuaibi*, A. Palizhati, N. Shoghi, A. Das, B. Wood, C. L. Zitnick, J. R. Kitchin,
Z. Ulissi
ACS Catalysis (Perspective), 2022.

[36] Transfer learning using attentions across atomic systems with graph neural networks
(TAAG)
A. Kolluru, N. Shoghi, M. Shuaibi, S. Goyal, A. Das, C. L. Zitnick, Z. Ulissi
Journal of Chemical Physics, 2022.
 [35] The Open Catalyst 2020 (OC20) Dataset and Community Challenges
L. Chanussot*, A. Das*, S. Goyal*, T. Lavril*, M. Shuaibi*, M. Rivière, K. Tran,
J. Heras-Domingo, C. Ho, W. Hu, A. Palizhati, A. Sriram, B. Wood, J. Yoon, D. Parikh,
C. L. Zitnick, Z. Ulissi
ACS Catalysis, 2021.

[34] Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra
International Journal of Computer Vision (IJCV), 2019.

[33] Visual Dialog [visualdialog.org]

A. Das, S. Kottur, K. Gupta, A. Singh, D. Yadav, S. Lee, J. M. F. Moura, D. Parikh, D. Batra
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2018.

[32] Human Attention in Visual Question Answering: Do Humans and Deep Networks
Look at the Same Regions? [abhishekdas.com/vqa-hat]
A. Das*, H. Agrawal*, C. L. Zitnick, D. Parikh, D. Batra
Computer Vision and Image Understanding (CVIU), 2017.

Conference [31] EquiformerV2: Improved Equivariant Transformer for Scaling to Higher-Degrees

Publications Y.-L. Liao, B. Wood, A. Das*, T. Smidt*
International Conference on Learning Representations (ICLR), 2024.

[30] PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav

R. Ramrakhya, D. Batra, E. Wijmans, A. Das
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

[29] Spherical Channels for Modeling Atomic Interactions

C. L. Zitnick, A. Das, A. Kolluru, J. Lan, M. Shuaibi, A. Sriram, Z. Ulissi, B. Wood
Advances in Neural Information Processing Systems (NeurIPS), 2022.

[28] Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations

at Scale [ram81.github.io/projects/habitat-web]
R. Ramrakhya, E. Undersander, D. Batra, A. Das
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

[27] Towards Training Billion Parameter Graph Neural Networks for Atomic Simulations
A. Sriram, A. Das, B. M. Wood, S. Goyal, C. L. Zitnick
International Conference on Learning Representations (ICLR), 2022.

[26] Auxiliary Tasks and Exploration Enable ObjectNav [joel99.github.io/objectnav]

J. Ye, D. Batra, A. Das, E. Wijmans
IEEE International Conference on Computer Vision (ICCV), 2021.

[25] Automated Video Description for Blind and Low Vision Users
A. Bodi, P. Fazli, S. Ihorn, Y.-T. Siu, A. T. Scott, L. Narins, Y. Kant, A. Das, I. Yoon
ACM Conference on Human Factors in Computing Systems (CHI) - Extended Abstract, 2021.
[24] Auxiliary Tasks Speed Up Learning PointGoal Navigation
J. Ye, D. Batra, E. Wijmans*, A. Das*
Conference on Robot Learning (CoRL), 2020.

[23] Large-scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline

V. Murahari, D. Batra, D. Parikh, A. Das
European Conference on Computer Vision (ECCV), 2020.

[22] Feel The Music: Automatically Generating A Dance For An Input Song
P. Tendulkar, A. Das, A. Kembhavi, D. Parikh
International Conference on Computational Creativity (ICCC), 2020.

[21] Probing Emergent Semantics in Predictive Agents via Question Answering

A. Das*, F. Carnevale*, H. Merzic, L. Rimell, R. Schneider, A. Hung, J. Abramson, A. Ahuja,
S. Clark, G. Wayne, F. Hill
International Conference on Machine Learning (ICML), 2020.

[20] IR-VIC: Unsupervised Discovery of Sub-goals for Transfer in RL

N. Modhe, P. Chattopadhyay, M. Sharma, A. Das, D. Parikh, D. Batra, R. Vedantam
International Joint Conference on Artificial Intelligence and the Pacific Rim International Conference
on Artificial Intelligence (IJCAI-PRICAI), 2020.

[19] Improving Generative Visual Dialog by Answering Diverse Questions

V. Murahari, P. Chattopadhyay, D. Batra, D. Parikh, A. Das
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019.

[18] TarMAC: Targeted Multi-Agent Communication

A. Das, T. Gervet, J. Romoff, D. Batra, D. Parikh, M. Rabbat, J. Pineau
International Conference on Machine Learning (ICML), 2019.

[17] Embodied Question Answering in Photorealistic Environments

with Point Cloud Perception
E. Wijmans*, S. Datta*, O. Maksymets*, A. Das, G. Gkioxari, S. Lee, I. Essa, D. Parikh, D. Batra
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, Oral.

[16] Audio-Visual Scene-Aware Dialog

H. Alamri, V. Cartillier, A. Das, J. Wang, S. Lee, P. Anderson, I. Essa, D. Parikh, D. Batra,
A. Cherian, T. K. Marks, C. Hori
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

[15] End-to-end Audio Visual Scene-Aware Dialog

using Multimodal Attention-based Video Features
C. Hori, H. Alamri, J. Wang, G. Wichern, T. Hori, A. Cherian, T. K. Marks, V. Cartillier,
R. Lopes, A. Das, I. Essa, D. Batra, D. Parikh
International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2019.

[14] Neural Modular Control for Embodied Question Answering [embodiedqa.org]

A. Das, G. Gkioxari, S. Lee, D. Parikh, D. Batra
Conference on Robot Learning (CoRL), 2018, Spotlight.

[13] Embodied Question Answering [embodiedqa.org]

A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, D. Batra
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, Oral.
 [12] Evaluating Visual Dialog Agents via Cooperative Human-AI Games [visualdialog.org]
P. Chattopadhyay*, D. Yadav*, V. Prabhu, A. Chandrasekaran, A. Das, S. Lee, D. Batra,
D. Parikh
AAAI Conference on Human Computation and Crowdsourcing (HCOMP), 2017.

[11] Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra
IEEE International Conference on Computer Vision (ICCV), 2017.

[10] Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
A. Das*, S. Kottur*, J. M. F. Moura, S. Lee, D. Batra
IEEE International Conference on Computer Vision (ICCV), 2017, Oral.

[9] Visual Dialog [visualdialog.org]

A. Das, S. Kottur, K. Gupta, A. Singh, D. Yadav, J. M. F. Moura, D. Parikh, D. Batra
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, Spotlight.

[8] Human Attention in Visual Question Answering: Do Humans and Deep Networks
Look at the Same Regions? [abhishekdas.com/vqa-hat]
A. Das*, H. Agrawal*, C. L. Zitnick, D. Parikh, D. Batra
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2016.

Workshop [7] ForceNet: A Graph Neural Network for Large-Scale Quantum Calculations
Papers W. Hu, M. Shuaibi, A. Das, S. Goyal, A. Sriram, J. Leskovec, D. Parikh, C. L. Zitnick
ICLR workshop – Deep Learning for Simulation, 2021, Best Paper Award.

[6] Unsupervised Discovery of Decision States through Intrinsic Control

N. Modhe, M. Sharma, P. Chattopadhyay, A. Das, D. Parikh, D. Batra, R. Vedantam
ICLR Workshop – Task-Agnostic Reinforcement Learning, 2019.

[5] Embodied Question Answering [embodiedqa.org]

A. Das, S. Datta, G. Gkioxari, S. Lee, D. Parikh, D. Batra
NIPS Workshop – Visually-Grounded Interaction and Language, 2017.

[4] Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning
A. Das*, S. Kottur*, J. M. F. Moura, S. Lee, D. Batra
CVPR Workshop – Language & Vision, 2017.

[3] Visual Procedural Content Generation with an Artificial Abstract Artist

M. Guzdial, D. Long, C. Cassion, A. Das
ICCC Workshop – Computational Creativity in Games, 2017.

[2] Grad-CAM: Why did you say that?

R. Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, D. Batra
NIPS Workshop – Interpretable ML for Complex Systems, 2016.

[1] Human Attention in Visual Question Answering: Do Humans and Deep Networks
Look at the Same Regions? [abhishekdas.com/vqa-hat]
A. Das*, H. Agrawal*, C. L. Zitnick, D. Parikh, D. Batra
ICML Workshop – Visualization for Deep Learning, 2016, Best Student Paper Award.
Side ai-paygrades [aipaygrad.es] 2020
Projects Statistics of industry job offers in AI.

neural-vqa-attention [github.com/abhshkdz/neural-vqa-attention] 2017

Torch code for the stacked attention VQA model from Yang et al. (CVPR ‘16).

ai-deadlines [aideadlin.es] 2016-2021

∼3.8k stars on GitHub, ∼20k active users / month.
Transferred ownership to Papers with Code in 2021.

neural-vqa [github.com/abhshkdz/neural-vqa] 2015

Torch code for the VQA model from Ren et al. (NIPS ‘15).

AirMaps [github.com/abhshkdz/airmaps] 2014

Gesture & voice-controlled Google Earth navigation tool.

Erdős [erdos.sdslabs.co] 2013

Competitive learning platform for math geeks. ∼3.5k users, ∼70k submissions.

HackView [github.com/sdslabs/hackview] 2012

Peer-to-peer video conferencing (using WebRTC) + collaborative document editing.

Relevant Computer Vision, Advanced Computer Vision, Introduction to Machine Learning,

Coursework Mathematical Foundations of Machine Learning, Machine Learning Theory,
Deep Learning, Introduction to Cognitive Science, Human-Robot Interaction, Algorithms

Professional Reviewing
Activities IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) 2018-2022
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017-2021
IEEE International Conference on Computer Vision (ICCV) 2017-2021
European Conference on Computer Vision (ECCV) 2018, 2020
Neural Information Processing Systems (NeurIPS) 2017-2020
International Conference on Machine Learning (ICML) 2021-2022
Association for Computational Linguistics (ACL) 2018-2021
Conference on Empirical Methods in Natural Language Processing (EMNLP) 2019

Advising
Ram Ramrakhya, Master’s student, Georgia Tech 2020-2022
Joel Ye, Bachelor’s + Master’s student, Georgia Tech 2019-2021
Sri Vivek Vanga, Master’s student, Georgia Tech 2020
Oluwabukola Adegboro, African Institute For Mathematical Sciences (AIMS) 2020
Vishvak Murahari, Master’s student, Georgia Tech 2018-2020
Black in AI graduate mentoring program 2019
Sandeep Kumar, Bachelor’s student, IIT Kanpur 2018-19

Workshop Organization
NAACL Workshop: Visually-Grounded Interaction and Language 2021
CVPR Workshop: VQA Challenge and Visual Dialog [visualqa.org] 2018-2020
NeurIPS Workshop: Visually-Grounded Interaction and Language 2017, 2019
 Tutorial Organization
ACL Tutorial: Connecting Language and Vision to Actions [lvatutorial.github.io] 2018

Challenge Organization
Open Catalyst Challenge [opencatalystproject.org/challenge] 2021, 2022
Visual Dialog Challenge [visualdialog.org/challenge] 2018, 2019, 2020

Extra Google Student Ambassador, IIT Roorkee

Curricular Joint Secretary, SDSLabs [sdslabs.co], IIT Roorkee
Activities Joint Secretary, Music Section, IIT Roorkee

Teaching CS 7643: Deep Learning, Georgia Tech Fall 2017

Experience Teaching Assistant with Dhruv Batra

ECE 2574: Data Structures and Algorithms, Virginia Tech Fall 2016
Teaching Assistant with Chris Wyatt

ECE 6504: Deep Learning for Perception, Virginia Tech Fall 2015
Teaching Assistant with Dhruv Batra