About

Hi, I’m Tushar Garg.

I’m a GPU performance architect. I spent four years at Intel predicting how AI workloads would run on GPUs that didn’t exist yet. Now I measure how LLMs actually run on the ones that do.

My work sits where models meet hardware: why a kernel is slow, which limit it hits (compute, memory bandwidth, latency, occupancy), and what would change if the hardware or the software changed. I studied Computer Science at IIT Hyderabad (M.Tech), then joined Intel, where I built analytical performance models for pre-silicon Xe GPUs and helped architects decide things like cache sizes and matrix-engine shapes.

AI on Silicon is where I explain what I find. Every episode predicts a result from first principles, measures it on real hardware, explains every millisecond, and ships a notebook so you can check my numbers.

Experience & education

1. Oct 2026 – now

   Independent research: LLM inference profiling on 1x NVIDIA H100 GPU

   SillyBench

   • Profiling vLLM serving Qwen3.8-27B (hybrid Gated DeltaNet + full attention) on 1x NVIDIA H100 GPU with Nsight Systems and Nsight Compute, from scheduling and host overhead down to individual kernels.
   • Built a predict → measure → explain harness: Spot-VM orchestration, NVTX-ranged runs, sha256-verified artifacts. A fresh-VM re-run matched decode latency within 0.6% at all six batch sizes.
   • The findings become Series 1 on this site.

2. 2022 – 2026

   GPU Performance Architect (AI)

   Intel

   • Built analytical AI performance models for Intel’s pre-silicon Xe GPUs, from single kernels up to end-to-end workloads, and correlated them against Intel’s internal GPU profiler.
   • Worked with product architects on mapping workloads to hardware: Xe-core counts, cache and memory hierarchy, matrix-engine dimensions and execution width. Modeled FlashAttention tiling to see how it loads the memory hierarchy.
   • Analyzed bottlenecks across Transformers, MLA, ResNet and Stable Diffusion. Built an analysis pipeline that cut a two-day job to two hours and was adopted by 10–12 engineers. Mentored two graduate interns.

3. 2021 – 2022

   AI Research Intern

   Mercedes-Benz R&D

   • Deep-learning lane-marking detection for driver assistance, under automotive accuracy and compute limits. Two IEEE conference papers (INSITA 2022, ICPRS 2022).

4. 2020 – 2022

   M.Tech, Computer Science & Engineering

   IIT Hyderabad

   • GPA 9.0/10. Led the student AI research team at TiHAN, IIT Hyderabad’s autonomous-navigation hub.

5. 2014 – 2018

   B.Tech, Computer Science

   Dr. A.P.J. Abdul Kalam Technical University

What I work with

• vLLM
• CUDA graphs
• torch.compile
• FlashAttention
• KV cache
• Speculative decoding
• Quantization
• Nsight Systems
• Nsight Compute
• Roofline & analytical models
• Pre-silicon GPU modeling
• NVIDIA Hopper
• Intel Xe GPUs
• Python
• PyTorch
• Triton

Say hello

Spotted a wrong number, or know the answer to one of the open questions in an episode? Tell me on LinkedIn. Corrections get credited in the episode.