PyTorch x GPU MODE Hackathon: No gradient descent. Only ascent.

by Verda

In person

When: Thursday 9 April 2026
Where: Paris

Join us to wrap up the PyTorch Conference with a GPU MODE x PyTorch IRL hackathon.

You can expect an intimate yet advanced day-long hackathon alongside researchers and engineers working at the bleeding edge of ML systems.

Here's what you can expect:

Two tracks on distributed training and inference optimization
Access to a B300 cluster from Verda and H200 instances from Sesterce
Cloud credits as prizes, including 48-hour access to a GB300 NVL72 rack
Talks from PyTorch (Helion), vLLM, Prime Intellect, ISTA, Sesterce, and SemiAnalysis
Food and refreshments

More details on the tracks, teams, and prizes below ⬇️

Schedule

9:30 - Doors open
10:00 - Opening words + tasks revealed
10:10 - Tech talks10:10 - Will Feng, PyTorch, Helion: A High-level DSL for Kernel Authoring10:25 - Tyler Michael Smith, vLLM: Large-scale inference 10110:40 - Matej Sirovatka, Prime Intellect: Reinforcement learning at scale10:55 - Erik Schultheis, ISTA: Low-precision formats
11:10 - Team formation
11:30 - Tasks start
13:30LunchFireside chat: Power is the new computeYoussef El Manssouri, SesterceJeremie Eliahou Ontiveros, SemiAnalysis
19:30 - Tasks end & dinner
20:00 - Closing ceremony
23:00 - Doors close

Tracks

1st track: LLM training on B300 cluster by Verda

Task: Pre-train an LLM from scratch in a limited time on a B300 cluster. You will participate as a team, trying to make optimal use of 360 PFLOP/s (BF16). Will this require asynchronous optimizer steps, large-batch size training, models much larger than chinchilla, or something else entirely?

Compute: Your team will get a B300 node for development and a set time on a B300 cluster.

Teams:

Come with a pre-formed team or get matched to a team on-site
The expected team sizes are 2-4

Prizes:

🥇 1st place: 48 hours on GB300 NVL72 + 2,500 EUR in cloud credits
🥈 2nd place: 2,000 EUR in cloud credits
🥉 3rd place: 1,500 EUR in cloud credits

2nd track: LLM inference on H200 GPUs by Sesterce

Task: Compete as a team on a leaderboard for the fastest inference for a fixed model.

Compute: Your team will get access to 4x H200s for the duration of the hackathon.

Teams: Similarly, you can come with a pre-formed team or get matched on-site.

Prizes:

🥇 1st place: 5,000 EUR in cloud credits
🥈 2nd place: 2,000 EUR in cloud credits
🥉 3rd place: 1,500 EUR in cloud credits

Lightning talks

Speakers, topics, and times TBA soon!