Publications

You can also find my articles on my Google Scholar profile.

Conference Papers

Intelligent router for llm workloads: Improving performance through workload-aware scheduling

Published in The 5th Workshop on Machine Learning and Systems (EuroMLSys) co-located with EuroSys'25, 2025

Load balancing amid multiple instances of the same LLM

Recommended citation: Kunal Jain, Anjaly Parayil, Ankur Mallick, Esha Choukse, Xiaoting Qin, Jue Zhang, Íñigo Goiri, Rujia Wang, Chetan Bansal, Victor Rühle, Anoop Kulkarni, Steve Kofsky, Saravan Rajmohan, Microsoft . 2025. Performance Aware LLM Load Balancer for Mixed Workloads. In The 5th Workshop on Machine Learning and Systems (EuroMLSys ’25), March 30-April 3, 2025, Rotterdam, Netherlands. ACM, New York, NY, USA, 12 pages. https://doi. org/10.1145/3721146.3721947
Download Paper

Serving Models, Fast and Slow: Optimizing Heterogeneous LLM Inferencing Workloads at Scale

Published in Under Review, 2025

Auto scaling for LLM datacenters

Recommended citation: Shashwat Jaiswal*, Kunal Jain*, Yogesh Simmhan, Anjaly Parayil, Ankur Mallick, Rujia Wang, Renee St Amant, et al. ‘Serving Models, Fast and Slow:Optimizing Heterogeneous LLM Inferencing Workloads at Scale’. arXiv [Cs.DC], 2025. https://doi.org/10.48550/ARXIV.2502.14617.
Download Paper

Seer: A Framework for Optimizing Traffic Camera Placement and Deep Learning Inference at the Edge for Vehicle Path Reconstruction

Published in 2024 IEEE/ACM Symposium on Edge Computing (SEC), 2024

Placement and deployment of edge cameras in cities.

Recommended citation: Siddhant Jain, Kunal Jain, Arun Ravindran and Suresh Purini, "Seer: A Framework for Optimizing Traffic Camera Placement and Deep Learning Inference at the Edge for Vehicle Path Reconstruction," 2024 IEEE/ACM Symposium on Edge Computing (SEC), Rome, Italy, 2024, pp. 333-345, doi: 10.1109/SEC62691.2024.00033.
Download Paper

Ensuring Fair LLM Serving Amid Diverse Applications

Published in Under Review, 2024

Fairness in serving multiple users and applications.

Recommended citation: Redwan Ibne Seraj Khan*, Kunal Jain*, Haiying Shen, Ankur Mallick, Anjaly Parayil, Anoop Kulkarni, Steve Kofsky, et al. ‘Ensuring Fair LLM Serving amid Diverse Applications’. arXiv [Cs.LG], 2024. https://doi.org/10.48550/ARXIV.2411.15997.
Download Paper

A cloud-fog architecture for video analytics on large scale camera networks using semantic scene analysis

Published in 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid), 2023

Distributed infrastructure for managing large scale camera networks.

Recommended citation: Kunal Jain, Kishan Sairam Adapa, Kunwar Grover, Ravi Kiran Sarvadevabhatla and Suresh Purini, "A Cloud-Fog Architecture for Video Analytics on Large Scale Camera Networks Using Semantic Scene Analysis," 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid), Bangalore, India, 2023, pp. 513-523, doi: 10.1109/CCGrid57682.2023.00054.
Download Paper