Rohit Swami
India Resume ↗

Rohit Swami · 6 long reads

Writing

Long reads on the kind of work I do, from fast pipelines and real-time systems to retrieval and open data. Each has figures you can push around.

Engineering notes

  1. 01Ten times faster, mostly by doing lessData engineering · 8 minI rebuilt a graph engine from R to Python and Polars, and it got ten times faster on 80% less memory. Very little of that came from changing languages.
  2. 02Push, don't pollReal-time systems · 9 minGetting a notification from a background job to a browser in under 100 milliseconds is easy once. The hard parts arrive when it has to happen for everyone at once.
  3. 03Spot instances without the 3am pageCloud infrastructure · 8 minSpare cloud capacity is cheap until it's taken back. Interruptions, checkpoints, retries and the spot/on-demand mix behind a batch pipeline bill that came down by 28%.
  4. 04Drawing a graph that's too big to drawVisualisation · 8 minSpatial indexes, label placement, Barnes–Hut, culling, level of detail and a single event listener: the unglamorous techniques that make a huge biological network feel instant.
  5. 05Retrieval is the hard partGenAI · 8 minWhen a RAG system gives a bad answer, the model usually takes the blame. More often it was handed the wrong context. Embeddings, chunking, approximate search, hybrid ranking and reranking, seen from the retrieval side.
  6. 06A dataset is a productKaggle · 8 minFive datasets and six notebooks on Kaggle, downloaded nearly 18,000 times and built on in 63 public notebooks. What it takes to make data that strangers can trust, from every NeurIPS paper to two million Indian companies, and the notebooks I built on top of my own datasets.

Earlier, on Medium

And four peer-reviewed papers, listed on the front page.