Classical

A brief history of LLM Scaling Laws from compute-optimal training and inference to scaling test-time compute and whether Scaling Laws are coming to an end.

A comparison of Scaling Laws for LLM Pretraining, from Kaplan, to Chinchilla, the Chinchilla Trap, covering compute-optimal training and inference.

An overview of the motivations and techniques used for generating synthetic data for LLM post-training, as seen in the Llama 3.1, AFM, Qwen2 and Hunyuan-Large papers.

An overview of recent research on scaling test-time compute in large language models (LLMs) including CoT, STaR, ReST, RISE, ORMs, PRMs and more.

Comparing the performance of a simple web server implemented in Python (FastAPI & SQLAlchemy) and Rust (Rocket, Axum and Diesel).

A simple web server implementation in Python using FastAPI and SQLAlchemy.

Two simple web server implementations in Rust using Rocket, axum and Diesel.

Learn how to manage python virtual environments with pyenv on macOS. Optional: create a jupyter notebook kernel for your venv using ipykernel.