How is information retrieved and generated in masked diffusion language models?

Understanding Bidirectional Information Retrieval in Masked Diffusion Language Models with ROME

Introduction

In this post, I will discuss the intermediate progress on using ROME to study information retrieval in masked (discrete) diffusion language models.

Page under construction!

Analysis in-progress.

A primer on masked discrete diffusion

Background and Related Work

Dataset

Methodology

Current Progress

Future Work

Acknowledgements

References

If you would like to cite this work, please use the following BibTeX entry:

@article{rai2025discrete-diffusion-rome,
  title={How is information retrieved and generated in masked diffusion language models?},
  author={Rai, Ashish},
  year={2025},
  month={July},
  url={https://raishish.github.io/blog/2025/discrete-diffusion-rome/}
}