RESEARCH
OVERVIEW
A call for project proposals in 2019 resulted in 10 projects that encompass a total of 15 research workstreams. The projects engage more than 150 faculty, researchers, and students, who are affiliated with more than 20 different organizational units across MIT Campus and MIT Lincoln Laboratory. All project teams involve Air Force personnel, who are embedded in the research teams and serve as liaisons between projects and Department of Defense stakeholders. The projects started in January 2020 and advance AI research in a broad range of areas, including weather modeling and visualization, optimization of training schedules, and enhancement of autonomy for augmenting and amplifying human decision-making.
The research activities of the AI Accelerator have been successfully expanding, including seed research projects in collaboration with the Naval Postgraduate School, a project with the United States Space Force, an AI Education Research project, a cyber project, and two additional projects that were spun out of our initial round of projects. AI Accelerator publications can be found on our Google Scholar page.
AI Accelerator Projects
Guardian Autonomy for Safe Decision Making
Air Guardian aims to advance AI and autonomy by developing algorithms and tools for augmenting and amplifying human decision making. The AI Guardian assists humans by suggesting actions using data from the past and fusing inputs from sensors and information sources. Support from an AI Guardian system is especially useful in the presence of surprises and complex situations. Guardian’s end-to-end machine learning algorithms learn from experts how to respond with common sense reasoning in highly dynamic and surprising situations. Our goal is to enable an agent to perceive its environment, identify short-term risks, reason about intentions and behaviors of its operator, and other cooperative and adversarial agents to determine the best course of action. This will lead to Guardian autonomy systems capable of anticipating potential hazardous situations in the future.
Team Rus
– Daniela Rus (MIT PI)
– Sanjeev Mohindra (MIT Lincoln Laboratory Lead)
– Morgan Mitchell (DAF Liaison)
Fast AI: Data Center & Edge Computing
The AI revolution has been enabled by the availability of vast amounts of labeled data, novel algorithms, and computer performance. But long computer-in-the-loop development cycles inhibit humans from inventing and deploying creative AI solutions. Moreover, the end of Moore’s has curtailed the historical ability of semiconductor technology to deliver performance. AI performance increasingly relies on hardware architecture, software, and algorithms. The Fast AI project focuses on developing a foundation for quickly building AI solutions, enabling performance and portability on both modern and legacy hardware platforms. We innovate in the areas of programming languages, compiler technologies, comprehensive instrumentation, analytical productivity tools, and parallel algorithms.
– Charles E. Leiserson (MIT PI)
– Tao B. Schardl (MIT)
– Neil Thompson (MIT)
– Vijay Gadepally (MIT Lincoln Laboratory Lead)
– Koley Borchard (DAF Liaison)
ML and Generative AI Enhanced Data Systems, Discovery and Applications
A core requirement for AI techniques to be successful is high quality data. Preparing systems to be “AI ready” involves collecting and parsing raw data for subsequent ingest, scan, query and analysis. This project will develop ML-enhanced database technologies to reduce storing and processing costs while enabling data sharing amongst various database silos. Additionally, we will develop an outlier detection engine to identify temporal anomalies amongst complex event streams from multiple sources.
– Tim Kraska (MIT PI)
– Vijay Gadepally (MIT Lincoln Laboratory Lead)
– Koley Borchard (DAF Liaison)
The AI revolution has been enabled by the availability of vast amounts of labeled data, novel algorithms, and computer performance. But long computer-in-the-loop development cycles inhibit humans from inventing and deploying creative AI solutions. Moreover, the end of Moore’s has curtailed the historical ability of semiconductor technology to deliver performance. AI performance increasingly relies on hardware architecture, software, and algorithms. The Fast AI project focuses on developing a foundation for quickly building AI solutions, enabling performance and portability on both modern and legacy hardware platforms. We innovate in the areas of programming languages, compiler technologies, comprehensive instrumentation, analytical productivity tools, and parallel algorithms.
A core requirement for AI techniques to be successful is high quality data. Preparing systems to be “AI ready” involves collecting and parsing raw data for subsequent ingest, scan, query and analysis. This project will develop ML-enhanced database technologies to reduce storing and processing costs while enabling data sharing amongst various database silos. Additionally, we will develop an outlier detection engine to identify temporal anomalies amongst complex event streams from multiple sources.
– Charles E. Leiserson (MIT PI)
– Tao B. Schardl (MIT)
– Neil Thompson (MIT)
– Vijay Gadepally (MIT Lincoln Laboratory Lead)
– Koley Borchard (DAF Liaison)
– Tim Kraska (MIT PI)
– Vijay Gadepally (MIT Lincoln Laboratory Lead)
– Koley Borchard (DAF Liaison)
Multi-Robot Learning for Personnel Recovery
This project seeks to advance autonomy algorithms tailored for agile and assertive maneuvering of aerial and ground robots, robust perception systems capable of operating in challenging environmental conditions, and strategies for seamless coordination among diverse robotic platforms. We envision the deployment of teams comprised of both UAVs and ground robots, equipped with autonomous capabilities, to swiftly navigate through intricate and hazardous terrains typically encountered in disaster zones. These vehicles will need to operate effectively in close proximity to the ground, contending with adverse conditions such as dense foliage, wind, dust, and precipitation, as well as in potentially contested or GPS-denied environments. These versatile teams must be able to function autonomously and/or collaboratively with ground personnel to expedite the process of locating objects of interest (i.e., survivors) and providing aid accordingly.
– Sertac Karaman (MIT PI)
– Luca Carlone (MIT Co-PI)
– Daniel Griffith (MIT Lincoln Laboratory Lead)
– Yakov Salzberg (MIT Lincoln Laboratory Lead)
– Stephanie Riley (DAF Liaison)
Overcoming the Fundamental Problems of LLMs: Truthful and Secure Multimodal Agents
No matter how useful Large Language Models (LLMs) may be, as long as they have certain fatal weaknesses (hallucinations, security leaks, systematic failures on many inputs), their deployment in the DoD will be limited. The goal of this project is to remove these roadblocks—to create models that are truthful, that cannot be attacked or leak sensitive information, and that correctly process the many demanding types of data the DoD uses. We aim to disseminate these new techniques and tools to the wider community, enabling a broad array of models to transition seamlessly to DoD, academic, and commercial use cases. At the same time, the project team is focused on integrating all these technologies into a useful, truthful, and secure multimodal model.
As a key component of our strategy, we are pioneering a map understanding initiative that establishes a world-leading benchmark for vision-enabled LLMs. We are developing a comprehensive, first-of-its-kind dataset of map-based question-answer pairs that challenges AI systems to interpret spatial, geographic, and symbolic information as adeptly as humans. This dataset, combined with our comprehensive AI testbench that currently benchmarks the world’s best Vision-LLMs, sets a new standard for evaluating and validating AI-driven map interpretation. Complementing these advancements, our SecureLLM method is being extended to handle sensitive information, further ensuring that our models are both secure and reliable for DoD applications.
– Jim Glass (MIT PI)
– Boris Katz (MIT Co-PI)
– Leslie Shing (MIT Lincoln Laboratory Lead)
– Nathaniel Maidel (DAF Liaison)
MultiFM: Multimodal Foundation Models for ISR Decision Making
Remote sensing imagery interpretation is crucial in ISR (Intelligence, Surveillance, and Reconnaissance) decision making. Traditional machine learning models for remote sensing are typically highly specialized and limited to a single modality and task. This specialization necessitates the redevelopment of models as tasks or modalities change, leading to resource and time inefficiencies. In contrast, recent foundation models in computer vision exhibit a wide range of capabilities, hinting at a possibility of similarly versatile models in remote sensing. These foundation models, which are trained on massive web-scale datasets, have proven effective for creative tasks by generating photorealistic images. However, when it comes to scientific and ISR applications that demand accuracy in representing reality – and quantitative estimates of uncertainty – current models may fall short.
Truly intelligent AI for ISR decision making must be capable of accurately representing a scene and identifying relevant objects from a variety of intelligence signals, while being adaptable to various downstream tasks. MultiFM aims to develop an AI agent that functions as an “autonomous decision assistant,” capable of learning scene representations and reasoning to support human decision-makers. The resulting AI will utilize a range of intelligence signals—from textual annotations and physical constraints to multimodal and multiview remote sensing data (e.g., SAR, EO/IR, LiDAR, and ground-level sensors) along with sensor parameters—to develop and refine scene representations. These refined representations will facilitate ISR decision making at speed and scale.
Team Freeman
– Bill Freeman (MIT PI)
– Taylor Perron (MIT Co-PI)
– Sara Beery (MIT Co-PI)
– Miriam Cha (MIT Lincon Laboratory Lead)
– Nathaniel Maidel (DAF Liaison)
The Earth Intelligence Engine
The Earth Intelligence Engine (EIE) includes research and prototype development centered on terrestrial and space weather forecasting, and advanced data visualization that supports rapid, effective decision-making for the DAF. To support these efforts, the EIE creates benchmark, ML-ready datasets designed to accelerate AI advancements that benefit both society and DAF use cases. The EIE aims to create novel algorithms that improve forecast skill and reduce uncertainty. Advancements in these focus areas will enable improved resource (people, property, and critical infrastructure) protection, enhanced mission support, and environmentally informed decision making.
– Dava J. Newman (MIT PI)
– Richard Linares (MIT Co-PI)
– Mark Veillette (MIT Lincoln Laboratory Lead)
– Peter Saunders (DAF Liaison)
AI-Enhanced Spectral Awareness and Interference Rejection
This project seeks to apply AI to enhance the USAF’s ability to detect, identify, and geolocate unknown radiofrequency (RF) signals, while providing tools for adaptive interference mitigation and smart spectrum analysis. These capabilities enhance Air Force Intelligence Surveillance and Reconnaissance (ISR) missions, communications, signals intelligence (SIGINT), and electronic warfare. Results will increase bandwidth utilization efficiency and spectrum sharing, improve Air Force communications performance in high interference environments, produce higher-quality RF signals intelligence, and improve system robustness to adversarial attacks and interference.
Team Wornell
– Gregory W. Wornell (MIT PI)
– Yury Polyanskiy (MIT Co-PI)
– Alexia Schulz (MIT Lincoln Laboratory Lead)
– Binoy Kurien (MIT Lincoln Laboratory Lead)
– Morgan Mitchell (DAF Liaison)
Automation in Space Domain Awareness
As space operations become ever more complex and disaggregated and heterogenous space observation data sources proliferate, it is increasingly difficult to separate significant events from routine spaceflight activities and deliver actionable space domain awareness (SDA) to satellite and other operators. AI-based techniques offer the combination of scalability and near-real-time performance to support human-machine teaming paradigms necessary to enable proactive SDA in this increasingly congested, contested, and competitive space. The SDA project leverages AI-based technologies to improve both space domain representation and understanding. Additionally, new AI approaches will be developed to optimize the behaviors of sensors and translate AI reasoning or recommendations into human-interpretable forms. The team also plans to host public challenges intended to render these problems accessible to the SDA expert community and to AI experts outside the space community to accelerate the infusion of the latest AI developments into the project’s approaches. The goal is to provide a common framework and benchmark scenario that enables the comparison of the performances of various AI methods across a set of key SDA-specific problems. A common challenge task framework is also expected to accelerate steps towards the infusion of best-in-breed AI techniques into operational systems by providing a common reference implementation architecture for adaptation.
– Richard Linares (MIT PI)
– Jonathan How (MIT Co-PI)
– Suvendra Dutta (MIT Lincoln Laboratory Lead)
– Jeffrey Price (DAF Liaison)
Better Networks via AI Enabled Hierarchical Connection Science
The costs of adversarial activity on networks are growing at an alarming rate. 90% of Americans are now concerned about cyber-attacks. The Air Force mission places it at the forefront of growing cyber challenges. In the land, sea, undersea, air, and space operating domains observe-pursue-counter (detect-handoff-intercept) walls-out architectures have proven cost effective. Our recent innovations in high performance privacy-preserving network sensing and analysis offer new opportunities for obtaining the required observations to enable such architectures in the cyber domain. Using these network observations to pursue and counter adversarial activity requires the development of novel privacy-preserving hierarchical AI analytics techniques on heavy-tail distributions that explore connections both within and across the layers of the knowledge pyramid from low-level network traffic to high-level social media. This project will explore AI methods fusing diverse data across layers to create an understandable enriched view of network activities, along with appropriate mitigations, and estimated impacts.
– Alex Pentland (MIT PI)
– Jeremy Kepner (MIT Lincoln Laboratory Lead)
– Hayden Jananthan (MIT Lincoln Laboratory)
– Chasen Milner (DAF Liaison)
Few-Shot and Continual Learning
AI techniques have proven very successful in many critical applications such as object recognition, speech recognition, and others. However, these successes have relied on collecting enormous datasets and careful manual annotations. This process is expensive, time-consuming, and in many scenarios, enough data is not available. Transfer learning offers a solution to these problems by leveraging past data seen by a machine to solve future problems using only few annotated examples. This research focuses on challenges in transfer learning and aims at developing algorithms that can fundamentally learn from multiple heterogeneous tasks, moving beyond low-level task similarity to enable broader transfer across distinct tasks. Such algorithms will find general applicability in several areas, including robotics, computer vision and natural language processing. Furthermore, it will substantially reduce the dependence on large amounts of annotated data and consequently reduce costs and time for deployment and maintenance of AI systems.
– Pulkit Agrawal ( MIT PI)
– Regina Barzilay (MIT Co-PI)
– Marin Soljacic (MIT Co-PI)
– Olga Simek (MIT Lincoln Laboratory Lead)
– Peter Saunders (DAF Liaison)
Agents for Conflict Resolution and Diplomacy
The goal of our project is to develop AI agents that achieve social intelligence, collaborating to resolve conflicts and arrive at diplomatic solutions to multi-agent disputes. In order to achieve collaboration, it is important that agents have the ability to communicate with one another. To this end, we are exploring how large language models (LLMs) can be used to parameterize effective communication policies and other agent behaviors. A primary interest is to understand how collaboration can be incentivized and can emerge even when agents have differing goals, and how a collection of agents can work together to solve problems that are beyond the abilities of any single agent. In addition to studying the basic science of multi-agent collaboration, we are also developing the tools and platforms necessary for efficiently simulating complex and realistic multi-agent scenarios.
– Phillip Isola ( MIT PI)
– Daniela Rus (MIT Co-PI)
– Jacob Andreas (MIT Co-PI)
– Sanjeev Mohindra (MIT Lincoln Laboratory Lead)
– Chasen Milner (DAF Liaison)
Multi-Agent Teaming and Coordination for Congested Aerospace Environments
As congestion increases in both the airspace and orbital domains, there is a need for advanced command and control systems that facilitate coordination between teams containing human and autonomous agents. These systems need to function effectively in dynamic and uncertain environments, with time delays, communication constraints, and potentially noncooperative or competing entities in the environment. Solutions that address these challenges will be vital to applications of interest to the Department of the Air Force, such as search-and-rescue using swarms of drones or ground-/space-based systems for space domain awareness. These applications underscore the need to refine current technologies for the safe coexistence and maneuvering of human-operated, semi-autonomous, and fully autonomous agents. Key directions of research include the development of approaches for interpretable and efficient long-horizon task planning for complex multi-agent missions, enhancing existing navigation and collision avoidance capabilities to optimize team-level objectives, and understanding the tradeoffs between decentralization, scalability, robustness, operational efficiency and information-sharing in these systems. The project seeks to devise operationally-relevant strategies that optimize and enhance these capabilities, thereby enhancing the safety and efficacy of operations within increasingly congested airspace and orbital environments.
– Hamsa Balakrishnan (MIT PI)
– Sanjeev Mohindra (MIT Lincoln Laboratory Lead)
– Jeffrey Price (DAF Liaison)
Large Improved Human-Machine Collaboration through Explainable AI and Transparent Value Alignment
The project aims to address the challenges of effective human-machine collaboration in complex, uncertain environments relevant to the Department of the Air Force (DAF). The goal is to develop shared mental models that allow human-machine teams to anticipate each other’s actions and adapt to dynamic scenarios, enabling calibrated reliance and seamless collaboration. Central to this effort is investigation of principled ways to refine the representations employed by complex models or decision-making systems so as to meet the informational and cognitive needs of human teammates Enabling a human and machine to iteratively refine the representations and abstractions employed in explanation systems is important to calibrate trust between users and AI systems by providing users with the right level of information to assess the system’s performance and limitations, and enable more efficient communication between AI systems and humans. This project will develop and study interactive and iterative processes for refining representations and abstractions employed in explanation systems, and empirically validate that such a process supports the formation of shared mental models that allow human-machine teams to anticipate each other’s actions.
– Julie Shah (MIT PI)
– Chuchu Fan (MIT Co-PI)
– Vincent Mancuso (MIT Lincoln Laboratory Lead)
– Sanjeev Mohindra (MIT Lincoln Laboratory Lead)
– Josh Rountree (DAF Liaison)
High-Speed Flight Vehicle Analysis via Physics-Informed Machine Learning (“HiFlight”)
Accurate and rapid predictions of the flow field surrounding high-speed flight vehicles are crucial for both early design stages and autonomous vehicle control. While data-driven model reduction offers a promising opportunity to leverage the accuracy of high-fidelity simulations for real-time applications, these methods are challenged when applied beyond the bounds of their training data or when data is sparse. Incorporating physical laws directly into the model training objective allows for the development of more robust representations tailored to address ill-posed inverse problems. Despite these advancements, the integration of multi-physics forward Partial Differential Equation (PDE) solvers with physics-informed reduced order models presents a challenge. Progress in physics-informed machine learning offer the potential to enhance early design phase analysis and support the development of lightweight models essential for onboard autonomous vehicle control. HiFlight explores physics-informed surrogate modeling for high-speed flight vehicles to make fast and accurate predictions of flight vehicle aerodynamic performance.
– Alan Edelman (MIT PI)
– Matthew Jones (MIT Lincoln Laboratory Lead)
– Tyler Korenyi-Both (DAF Liaison)
– Stephanie Riley (DAF Liaison)
Research was sponsored by the Department of the Air Force Artificial Intelligence Accelerator and was accomplished under Cooperative Agreement Number FA8750-19-2-1000. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Department of the Air Force or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein.
