Tag: Medusa architecture
Speculative Decoding with Compressed Draft Models for LLMs
Accelerate LLM inference by up to 3x using speculative decoding. Learn how draft models, Medusa, and EAGLE reduce latency without compromising output quality.
Read More
Speculative Decoding Explained: Draft-and-Verify for Faster LLMs
Learn how speculative decoding speeds up LLMs using a draft-and-verify pipeline. Discover the math behind rejection sampling, Medusa architecture, and implementation tips for production.
Read More