The Fabric
Developed by Northeastern University in partnership with the NSF Delta high-performance computing cluster at the NCSA at University of Illinois Urbana-Champaign, NDIF consists of three major components.
The Three Parts of NDIF
A nationwide high-performance computing fabric
Hosting the largest open pretrained machine learning models for transparent deep inference. This National Deep Inference Fabric is a unique combination of GPU hardware and deep network AI inference software that provides a remotely-accessible computing resource for scientists to perform detailed and reproducible experiments on large AI systems on the fabric. The fabric is designed for many scientists to efficiently and simultaneously share the same AI computing capacity to make efficient use of resources.
NDIF is powered by NCSA's Delta — including H200 and A40 GPU nodes — providing free remote access to run experiments on large-scale AI models. Delta is part of the NSF high-performance computing portfolio at the University of Illinois Urbana-Champaign.
Go to NCSA Delta↗FAQ
NDIF is available for you to use today. Get started here: ndif-team.github.io/ndif-web-beta/get-started
Commercial AI inference services such as ChatGPT, Claude, and Gemini only provide black-box access to large AI models—you can send inputs and receive outputs, but you cannot observe or alter any internal computations. In contrast, NDIF provides full transparency for AI inference, allowing users to fully examine and modify every step of the internal computation of large AI models using the NNsight library.
Please cite: Jaden Fried Fiotto-Kaufman et al., "NNsight and NDIF: Democratizing Access to Foundation Model Internals," ICLR 2025. When you publish work using NNsight or NDIF resources, please also email us at info@ndif.us to tell us about your work.
Traditional HPC systems support coarse-grained computing jobs and do not natively support fine-grained sharing of pretrained AI models. NDIF provides a shared deep inference fabric, allowing many users to access shared AI models in a fine-grained manner—submitting specialized deep inference tasks that may run for as briefly as a fraction of a second, sharing preloaded models simultaneously.
NDIF's API, NNsight, is built on PyTorch, so it will be familiar to any PyTorch user. However, NNsight defines Python contexts where models can be run with interventions that are defined locally but executed either locally or remotely. This enables a workflow where you develop methods at small scale locally and then deploy the same code at large scale on NDIF.
NNsight, the open-source software underlying NDIF, is available worldwide and can be used with your own hardware. The NSF-funded computing resources will be available to educational and research users with a U.S. affiliation or collaborator after account creation via CILogin.
If you'd prefer to access NDIF resources without coding, check out Workbench, our web app! You can run experiments on NDIF models remotely, all from a browser. Try it today: workbench.ndif.us
For information on open positions, including full-time, part-time, co-op, and volunteer roles, see the Jobs section of our Community page.
Citing NDIF
If you use NNsight or NDIF resources in your research, please cite the following:
Citation
Jaden Fried Fiotto-Kaufman, Alexander Russell Loftus, Eric Todd, Jannik Brinkmann, Koyena Pal, Dmitrii Troitskii, Michael Ripa, Adam Belfki, Can Rager, Caden Juang, Aaron Mueller, Samuel Marks, Arnab Sen Sharma, Francesca Lucchetti, Nikhil Prakash, Carla E. Brodley, Arjun Guha, Jonathan Bell, Byron C Wallace, and David Bau. "NNsight and NDIF: Democratizing Access to Foundation Model Internals," ICLR 2025. Available at https://openreview.net/forum?id=MxbEiFRf39.
BibTeX
@inproceedings{fiotto-kaufman2025nnsight,
title={{NNsight} and {NDIF}: Democratizing Access to Foundation Model Internals},
author={Jaden Fried Fiotto-Kaufman and Alexander Russell Loftus and Eric Todd and Jannik Brinkmann and Koyena Pal and Dmitrii Troitskii and Michael Ripa and Adam Belfki and Can Rager and Caden Juang and Aaron Mueller and Samuel Marks and Arnab Sen Sharma and Francesca Lucchetti and Nikhil Prakash and Carla E. Brodley and Arjun Guha and Jonathan Bell and Byron C Wallace and David Bau},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=MxbEiFRf39}
}In addition, when you publish work using NNsight or NDIF resources, we'd love you to email us directly at info@ndif.us to tell us about your work. This helps us track our impact and supports our continued efforts to provide open-source resources for reproducible and transparent research on large-scale AI systems.