Shypd.aiThe largest open index of AI toolsAwesome-LLM-Inference is an open-source research and education tool that curates a list of papers and code for Large Language Model (LLM) inference. It covers techniques like Flash-Attention, Paged-Attention, and various optimization methods.
Trending
About
Awesome-LLM-Inference is a comprehensive, curated list of research papers and associated code focused on Large Language Model (LLM) and Vision-Language Model (VLM) inference. This resource is designed for researchers and engineers looking to explore advanced techniques for optimizing LLM performance, including Flash-Attention, Paged-Attention, WINT8/4 quantization, and various parallelism strategies. The repository provides an organized collection of topics such as multi-GPU/multi-node parallelism, disaggregating prefill and decoding, KV cache scheduling, and long-context attention optimization. It also features sections on LLM algorithmic/evaluation surveys, inference frameworks, and specific topics like Mixture-of-Experts (MoE) LLM inference. Users can download all listed PDFs via a Python script, making it a valuable hub for staying updated on the latest advancements in efficient LLM inference.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
