🧠 Awesome AI
A curated list of Enterprise AI, LLMs, GenAI, AI Agents, MCP, governance, security, MLOps, and production-ready AI tools.
RAG, vector databases, orchestration frameworks, and enterprise AI resources.
This list is opinionated and production-first . Entries are selected for teams that have to ship, operate, secure, and govern AI systems — not for demos. Where a category has an obvious default, it is marked with ⭐.
Legend: ⭐ widely adopted default · 🔓 open source · 💰 commercial / paid tier · ☁️ managed service · 🆓 free
Contributing · Author · License
🤖 Foundation Models (LLMs, VLMs, SLMs)
Frontier & Proprietary Models
Model Family
Provider
Highlights
Access
Claude ⭐
Anthropic
Claude 5 family (Fable, Opus, Sonnet) + Haiku — strong reasoning, coding, long context, agentic tool use
API · Console
GPT
OpenAI
Broad general-purpose family with reasoning variants
API
Gemini
Google DeepMind
Natively multimodal, very long context, tight GCP integration
API
Grok
xAI
Real-time-leaning general models
API
Command
Cohere
Enterprise/RAG-oriented, strong multilingual + citations
API
Nova
Amazon
Cost-tiered family, native to Bedrock
Bedrock
Mistral Large
Mistral AI
EU-based provider, open + commercial mix
API
Model Family
Org
Notes
License
Llama ⭐
Meta
The de-facto open baseline; huge fine-tune/tooling ecosystem
Llama Community
Qwen ⭐
Alibaba
Dense + MoE, very strong multilingual & coding, many sizes
Apache-2.0 (most)
DeepSeek
DeepSeek
MoE + reasoning models with strong price/performance
MIT (most)
Mistral / Mixtral
Mistral AI
Efficient dense and MoE models
Apache-2.0
Gemma
Google
Lightweight open siblings of Gemini
Gemma Terms
Phi
Microsoft
Small models trained on curated "textbook" data
MIT
GLM
Z.ai (Zhipu)
Strong bilingual + agentic/coding variants
MIT / Apache-2.0
Kimi
Moonshot AI
Very large MoE, long-context and agentic focus
Modified MIT
Granite
IBM
Enterprise-governed, indemnified, watsonx-native
Apache-2.0
OLMo
Ai2
Fully open: weights, data, code, logs — best for research
Apache-2.0
Falcon
TII
Multilingual open family
Apache-2.0
SmolLM
Hugging Face
Genuinely small (135M–1.7B), on-device oriented
Apache-2.0
Vision-Language & Multimodal (VLMs)
Project
Description
Qwen-VL
Leading open VLM family — documents, charts, video, GUI grounding
InternVL
Open multimodal models competitive with proprietary VLMs
Pixtral
Mistral's open multimodal model
Molmo
Ai2 open VLM with pointing/grounding capability
Whisper
Robust open speech-to-text across many languages
CLIP / OpenCLIP
Foundational image-text embedding models
Model Access, Routing & Local Runtimes
Tool
Description
Hugging Face Hub ⭐ 🔓
The model registry of record — weights, datasets, Spaces, leaderboards
OpenRouter ☁️
One API across hundreds of models with routing, fallbacks and price transparency
Ollama ⭐ 🔓
Dead-simple local model runner with a familiar CLI/API
LM Studio 🆓
Desktop GUI for running and serving local models
llama.cpp 🔓
The C/C++ inference engine underpinning much of local AI (GGUF)
MLX 🔓
Apple-silicon array framework for fast on-device training/inference
Together AI · Fireworks · Groq · Cerebras ☁️
High-throughput / low-latency hosted open-model inference
Replicate ☁️
Run and fine-tune open models via API, pay-per-second
Artificial Analysis 🆓
Independent benchmarks of quality, speed and price across providers
LMArena 🆓
Crowd-sourced pairwise model comparison leaderboard
⬆ back to top
Resource
Description
Anthropic Prompt Engineering Guide ⭐
The most practical vendor guide — clarity, examples, XML structure, thinking, chaining
Anthropic Prompt Library
Ready-made, well-structured prompts across dozens of tasks
OpenAI Prompt Engineering
Vendor guidance and reasoning-model prompting patterns
Google Prompting Guidance
Gemini-specific strategies and multimodal prompting
Prompt Engineering Guide 🆓
Comprehensive, paper-backed technique catalogue
Learn Prompting 🆓
Structured course from basics to adversarial prompting
Anthropic Prompt Engineering Interactive Tutorial
Hands-on notebook course, chapter by chapter
Anthropic Cookbook ⭐
Runnable recipes: tool use, RAG, vision, classification, evals
Technique
What it does
Reference
Few-shot prompting
Demonstrations steer format and behavior more reliably than instructions alone
Paper
Chain-of-Thought
Ask for intermediate reasoning before the answer
Paper
Self-consistency
Sample multiple reasoning paths, take the majority answer
Paper
ReAct
Interleave reasoning with tool calls — the basis of most agents
Paper
Reflexion / self-critique
Let the model review and revise its own output
Paper
Tree of Thoughts
Explore and score branching reasoning paths
Paper
Prompt caching
Cache long, stable prefixes to cut cost and latency dramatically
Docs
Structured output
Constrain generation to a schema instead of parsing prose
Guide
Tool
Description
DSPy ⭐ 🔓
Program — don't prompt. Declarative modules with automatic prompt/weight optimization
promptfoo ⭐ 🔓
Test, compare and red-team prompts with assertions in CI
Instructor 🔓
Pydantic-typed structured outputs with validation and retries
BAML 🔓
A dedicated language for prompts as typed functions, with tests
Outlines 🔓
Guaranteed-valid structured generation (JSON, regex, grammars)
Guidance 🔓
Constrained generation with interleaved control flow
TextGrad 🔓
Backpropagate natural-language "gradients" to optimize prompts
Priompt 🔓
Priority-based JSX prompt composition for context budgeting
⬆ back to top
Rule of thumb: retrieval quality dominates generation quality. Invest in parsing, chunking and reranking before swapping models.
Tool
Description
LlamaIndex ⭐ 🔓
The most complete data framework for RAG — connectors, indexes, query engines, workflows
LangChain ⭐ 🔓
Ubiquitous building blocks and integrations across the entire stack
Haystack 🔓
Production-minded, explicitly-wired pipelines from deepset
RAGFlow 🔓
Deep document understanding RAG engine with a full UI
R2R 🔓
RAG-as-a-service with ingestion, graphs, auth and observability built in
txtai 🔓
All-in-one embeddings database for semantic search and RAG
Cognita 🔓
Modular, config-driven RAG framework for production teams
Verba 🔓
Open-source RAG application built on Weaviate
Dify ⭐ 🔓
Visual LLM app platform with RAG pipeline, agents and observability
Document Parsing & Ingestion
Tool
Description
Docling ⭐ 🔓
IBM's high-fidelity document converter — layout, tables, formulas → structured output
Unstructured 🔓
Preprocess 25+ file types into clean, chunk-ready elements
MinerU 🔓
High-quality PDF → Markdown/JSON extraction, strong on scientific docs
Marker 🔓
Fast, accurate PDF-to-Markdown with equation and table support
LlamaParse 💰 ☁️
Managed parser tuned for complex/tabular documents
Firecrawl ⭐ 🔓
Crawl and scrape sites into clean LLM-ready markdown
Crawl4AI 🔓
Async, LLM-friendly web crawler and scraper
Jina Reader 🆓
Prefix any URL with r.jina.ai to get markdown
Chonkie 🔓
Fast, no-nonsense chunking library with many strategies
Tool
Description
Cohere Rerank ⭐ ☁️
The default cross-encoder reranker — often the single biggest quality win
BGE Reranker 🔓
Strong open reranker family from BAAI
Jina Reranker ☁️
Multilingual and long-context reranking models
RAGatouille 🔓
Late-interaction (ColBERT) retrieval made usable
rerankers 🔓
One tiny API over every reranking method
rank_bm25 🔓
Classic lexical retrieval — still essential in hybrid search
Pattern
Why it matters
Reference
Hybrid search (BM25 + dense)
Lexical recall covers what embeddings miss (IDs, names, rare terms)
Guide
Contextual Retrieval
Prepend chunk-specific context before embedding; large recall gains
Anthropic
GraphRAG
Build a knowledge graph for global, corpus-wide questions
Microsoft
LightRAG
Simpler, cheaper graph-augmented retrieval
HKUDS
RAPTOR
Recursive clustering + summarization into a retrieval tree
Stanford
Self-RAG
Model decides when to retrieve and critiques what it retrieved
Paper
Corrective RAG
Grade retrieved docs, fall back to web search when weak
Paper
Neo4j GraphRAG
Official graph + vector retrieval package
Neo4j
Tool
Description
Ragas ⭐ 🔓
The standard RAG metrics suite — faithfulness, relevancy, context precision/recall
DeepEval 🔓
Pytest-style LLM evaluation with RAG-specific metrics
TruLens 🔓
Feedback functions and the "RAG triad" for tracking quality
FlashRAG 🔓
Research toolkit to reproduce and compare RAG methods
⬆ back to top
🔍 Embeddings & Vector Databases
Model
Provider
Notes
Voyage AI ⭐ ☁️
Voyage
Anthropic-recommended; strong general + domain-specific (code, finance, legal) models
OpenAI Embeddings ☁️
OpenAI
Solid general-purpose default with adjustable dimensions
Cohere Embed ☁️
Cohere
Excellent multilingual and compressed (int8/binary) embeddings
BGE ⭐ 🔓
BAAI
The leading open embedding family, many sizes and languages
Jina Embeddings 🔓 ☁️
Jina AI
Long-context and multimodal open embeddings
Nomic Embed 🔓
Nomic
Fully open (weights + data + code) long-context embeddings
E5 / multilingual-e5 🔓
Microsoft
Strong, well-studied open baselines
Sentence-Transformers ⭐ 🔓
UKP Lab
The library for using and fine-tuning embedding models
Model2Vec 🔓
Minish Lab
Distill encoders into static embeddings — orders of magnitude faster
MTEB Leaderboard 🆓
HF
Compare embedding models before you commit
Database
Model
Best for
License
pgvector ⭐ 🔓
Postgres extension
Teams already on Postgres — start here before adding infrastructure
PostgreSQL
Qdrant ⭐ 🔓 ☁️
Standalone / cloud
Rich filtering, quantization, excellent performance-per-euro
Apache-2.0
Weaviate 🔓 ☁️
Standalone / cloud
Built-in hybrid search, modules and multi-tenancy
BSD-3
Milvus 🔓 ☁️
Distributed
Billion-scale workloads and GPU indexing
Apache-2.0
Chroma 🔓
Embedded / server
Prototyping and local-first apps
Apache-2.0
LanceDB 🔓 ☁️
Embedded / serverless
Multimodal, object-storage-native, zero-copy versioning
Apache-2.0
Pinecone 💰 ☁️
Managed
Fully managed serverless vector search with minimal ops
Proprietary
Turbopuffer 💰 ☁️
Managed
Object-storage-backed search at very low cost per vector
Proprietary
Vespa 🔓 ☁️
Distributed
Serious hybrid ranking and real-time recommendation at scale
Apache-2.0
Elasticsearch / OpenSearch
Search engine
Adding vectors to an existing search deployment
Elastic v2 / Apache-2.0
Redis Vector 🔓 ☁️
In-memory
Lowest-latency lookups, semantic caching
RSALv2/SSPL
MongoDB Atlas Vector Search 💰 ☁️
Document DB
Vectors next to operational documents
Proprietary
Supabase Vector 🔓 ☁️
Postgres platform
pgvector with auth, storage and edge functions attached
Apache-2.0
ClickHouse 🔓 ☁️
OLAP
Analytics + vector filtering over huge tables
Apache-2.0
ANN Libraries (bring your own store)
Library
Description
FAISS ⭐ 🔓
Meta's similarity-search library — the reference implementation for ANN
hnswlib 🔓
Fast, minimal header-only HNSW
USearch 🔓
Compact, multi-language engine with many metrics and quantization
ScaNN 🔓
Google's high-recall, high-throughput ANN
Annoy 🔓
Memory-mapped, read-only indexes from Spotify
DiskANN 🔓
SSD-resident indexes for datasets far larger than RAM
⬆ back to top
Agent
Description
Claude Code ⭐ 💰
Anthropic's agentic coding tool — terminal, IDE, desktop and web, with hooks, subagents, skills and MCP
OpenAI Codex 💰
OpenAI's coding agent CLI and cloud environment
Gemini CLI 🔓
Google's open-source terminal agent
Cursor 💰
AI-native editor with deep codebase context and background agents
Cline 🔓
Autonomous coding agent inside VS Code, bring-your-own-model
Aider 🔓
Pair programming in the terminal, git-commit-native
OpenHands 🔓
Full agent platform that codes, browses and runs commands in a sandbox
Goose 🔓
Block's extensible on-machine agent, MCP-first
Continue 🔓
Build and run custom autocomplete/chat/agent flows in your IDE
SWE-agent 🔓
The research agent that popularized agent-computer interfaces
Plandex 🔓
Terminal agent designed for large, multi-file, long-running tasks
Devin 💰
Cognition's autonomous software engineer
Browser & Computer-Use Agents
Tool
Description
browser-use ⭐ 🔓
Let agents drive a real browser — the most widely used option
Stagehand 🔓
Playwright plus act/extract/observe primitives; deterministic where it matters
Playwright MCP 🔓
Accessibility-tree browser control exposed over MCP
Skyvern 🔓
LLM + vision workflow automation for browser tasks
Computer Use (Claude) 💰
Screen-, mouse- and keyboard-level control reference implementation
E2B 🔓 ☁️
Secure cloud sandboxes for running agent-generated code
Daytona 🔓 ☁️
Fast, isolated runtimes for agent workloads
Autonomous & Multi-Agent Systems
Project
Description
MetaGPT 🔓
Multi-agent software company simulation with SOP-driven roles
AutoGPT 🔓
The original autonomous agent, now a workflow platform
Open Interpreter 🔓
Natural-language interface to run code locally
Letta 🔓
Stateful agents with long-term memory (formerly MemGPT)
CAMEL 🔓
Research framework for multi-agent communication at scale
Suna 🔓
Open-source generalist assistant with browser, files and shell
See Benchmarks → Agents & Tool Use for SWE-bench, Terminal-Bench, τ-bench, BFCL, GAIA, OSWorld and WebArena, and Metrics → Agent Metrics for what to measure on your own traffic.
⬆ back to top
🔌 MCP (Model Context Protocol)
MCP is the open standard for connecting AI applications to tools, data and prompts — "USB-C for AI apps". It is now the default integration layer across Anthropic, OpenAI, Google and Microsoft tooling.
Specification & Official Resources
Frameworks & Infrastructure
Tool
Description
FastMCP ⭐ 🔓
The fast, Pythonic way to build MCP servers and clients
mcp-use 🔓
Connect any LLM to any MCP server in a few lines
mcp-agent 🔓
Build agents using MCP with composable workflow patterns
Cloudflare Remote MCP ☁️
Deploy authenticated remote MCP servers on Workers
Docker MCP Toolkit 🔓
Containerized, signed MCP servers with a managed gateway
MCP Gateway (IBM) 🔓
Federate, secure and observe many MCP servers behind one endpoint
Directory
Description
Smithery
Large registry with one-click install and hosted deployment
Glama MCP
Indexed directory with security and quality scoring
PulseMCP
Servers, clients and a weekly ecosystem newsletter
mcpservers.org
Curated, human-reviewed server listing
Awesome MCP Servers
The community's own curated list
Enterprise Considerations
Topic
Guidance
Authorization
Remote servers should implement the OAuth 2.1 authorization spec — never ship static shared secrets
Transport
Prefer Streamable HTTP for remote servers; stdio for local-only tools
Tool poisoning
Treat third-party tool descriptions as untrusted input — they enter the model's context
Least privilege
Scope each server to one system with the narrowest credentials that work
Auditability
Log every tool invocation with arguments, identity and outcome
Vetting
Pin versions and review source before installing community servers
⬆ back to top
If MCP gives an agent tools and data , Agent Skills give it procedural knowledge — how this team does code review, ships a release, or formats a report. A skill is a folder with a SKILL.md file; agents load the name and description at startup and pull in the full instructions only when a task matches.
This section deliberately favors skills written and used by the teams that maintain the product they describe , over bulk-generated skill dumps. A skill encoding a real team's hard-won conventions is worth a hundred generated from a framework's README.
Skills Shipped by the Teams That Own the Domain
The strongest signal that a skill is real: the people who maintain the framework, platform or product also maintain its skills, and ship them in the repo.
Skills
Maintained by
What they encode
Laravel Boost ⭐ 🔓
Laravel core team
Laravel conventions and best practices, so agents write idiomatic Laravel rather than generic PHP
Anthropic document skills ⭐
Anthropic
The pdf, docx, pptx and xlsx skills that power Claude's real document editing
Spring AI 🔓
Spring team
Bringing skills to Spring AI agents in the Java ecosystem
Google AI Edge Gallery 🔓
Google AI Edge
Skills shipped alongside an on-device LLM application
Goose 🔓
Block
Context engineering with skills in an open agent runtime
Pulumi Neo
Pulumi
Infrastructure workflows constrained by org policy and approvals
Databricks Genie Code
Databricks
Data engineering and analytics procedures inside the lakehouse
Snowflake Cortex Code
Snowflake
Extending an in-platform data agent with team procedures
Qodo
Qodo
A written-up account of using skills to auto-fix PR issues
Letta 🔓
Letta
Skills combined with persistent agent memory
OpenHands 🔓
OpenHands
Skills for cloud coding agents at scale
Firebender
Firebender
Android-specific build, emulator and test workflows
Skills that encode a whole way of working, not a single task.
Collection
Author
What it encodes
Superpowers ⭐ 🔓
Jesse Vincent
A full development methodology — TDD red/green/refactor, systematic debugging, brainstorming before building, verification before completion, subagent-driven development, worktree management
Netresearch marketplace 🔓
Netresearch
An agency's working skills for TYPO3, PHP, Go, Docker, Jira, security and docs
skill-creator ⭐
Anthropic
The skill for writing skills — start here before authoring your own
mcp-builder
Anthropic
Generating well-formed MCP servers, itself packaged as a skill
Directories & Curated Lists
Vetting a skill before you install it: a skill is instructions your agent will follow and scripts it may execute. Read SKILL.md in full, check what scripts/ actually does, prefer skills maintained by the owners of the domain, and pin a commit rather than tracking a moving branch.
The format is portable; the same SKILL.md works across a growing set of agents.
Client
Skills documentation
Claude Code ⭐
Terminal, IDE, desktop and web
OpenAI Codex
Codex CLI and cloud
GitHub Copilot
Across Copilot agents
VS Code
Editor-native customization
Cursor
Skills as codebase context
Gemini CLI 🔓
Google's open terminal agent
Goose 🔓
Block's extensible agent
OpenCode 🔓
Terminal, IDE and desktop
Amp
Sourcegraph's coding agent
Roo Code 🔓
Multi-mode agent team in your editor
Factory
Droids across IDE and CI/CD
Kiro
Spec-driven development
Junie
JetBrains IDE agent
Tabnine
Enterprise-controlled deployments
Writing Skills That Actually Get Used
Practice
Why
The description is the trigger ⭐
It is all the agent sees until activation. Write when to use it, not what it is — vague descriptions never fire
Keep SKILL.md short
It loads into context wholesale. Push detail into references/ and link to it
Use progressive disclosure ⭐
Name/description at startup, instructions on activation, referenced files on demand. Design for all three stages
Prefer scripts over prose
Deterministic steps belong in scripts/, not in instructions the model may paraphrase
Encode judgment, not just steps
The valuable part is why your team does it this way and when to deviate
Version-control with the code
A skill describing your repo's conventions belongs in your repo, reviewed like any other change
Test it like code
Run real tasks against it. A skill that never activates is worse than none — it creates false confidence
One skill, one job
Composable skills beat a monolith; the agent loads only what the task needs
⬆ back to top
General-Purpose Orchestration
Framework
Language
Distinctive strength
LangGraph ⭐ 🔓
Python, JS
Graph-based, durable, stateful agents with human-in-the-loop and time travel
CrewAI ⭐ 🔓
Python
Role-playing crews and deterministic flows; standalone of LangChain
AutoGen 🔓
Python, .NET
Event-driven multi-agent conversations from Microsoft Research
AG2 🔓
Python
Community continuation of the original AutoGen line
Microsoft Agent Framework 🔓
Python, .NET
Convergence of AutoGen + Semantic Kernel for production agents
Semantic Kernel 🔓
.NET, Python, Java
Enterprise .NET-first orchestration with planners and plugins
PydanticAI ⭐ 🔓
Python
Type-safe agents that feel like FastAPI; excellent validation and testing
Mastra ⭐ 🔓
TypeScript
Batteries-included TS framework — agents, workflows, RAG, evals, memory
OpenAI Agents SDK 🔓
Python, JS
Minimal primitives: agents, handoffs, guardrails, sessions, tracing
Claude Agent SDK ⭐ 🔓
Python, TS
Build agents on the same harness that powers Claude Code
Google ADK 🔓
Python, Java
Code-first agent development, Vertex AI Agent Engine deployment
Strands Agents 🔓
Python
AWS's model-driven SDK, integrates with Bedrock AgentCore
Agno 🔓
Python
High-performance runtime with built-in memory, knowledge and UI
smolagents 🔓
Python
Minimal agents that think in code; ~1k lines of core logic
LlamaIndex Workflows 🔓
Python, TS
Event-driven, async-first orchestration tightly coupled to retrieval
Atomic Agents 🔓
Python
Small, composable, strictly-typed agent building blocks
Cloudflare Agents SDK 🔓
TypeScript
Durable, stateful agents on Workers with WebSockets and scheduling
Visual & Low-Code Builders
Tool
Description
Dify ⭐ 🔓
Full LLMOps platform: visual workflows, RAG, agents, observability
n8n ⭐ 🔓
Workflow automation with native AI/agent nodes and 400+ integrations
Flowise 🔓
Drag-and-drop LLM app builder
Langflow 🔓
Visual builder for agents and RAG flows, deployable as an API
Rivet 🔓
Visual graph IDE for debugging complex prompt chains
Durable Execution (the unglamorous half of production agents)
Tool
Description
Temporal ⭐ 🔓 ☁️
Durable workflow engine — the reference answer for long-running, retryable agents
Inngest 🔓 ☁️
Event-driven durable functions with steps, retries and concurrency control
Restate 🔓 ☁️
Durable execution with low-latency journaling
Cloudflare Workflows ☁️
Durable multi-step execution at the edge
Prefect 🔓 ☁️
Pythonic orchestration that adapts well to AI pipelines
⬆ back to top
Agent frameworks orchestrate models. These frameworks are the layer models are built, trained and optimized in.
Framework
Org
Notes
PyTorch ⭐ 🔓
PyTorch Foundation
The framework nearly all modern AI research and open models are written in
JAX 🔓
Google
Composable function transforms + XLA; dominant for large-scale TPU training
TensorFlow 🔓
Google
Mature production ecosystem; still widespread in established deployments
Keras 🔓
Keras team
High-level API that now runs on JAX, PyTorch or TensorFlow
Flax NNX 🔓
Google
The neural network library most JAX research builds on
MLX 🔓
Apple
Array framework designed for Apple silicon unified memory
tinygrad 🔓
tiny corp
Radically minimal framework — excellent for understanding the stack
Candle 🔓
Hugging Face
Minimalist Rust ML framework for serverless and embedded inference
Burn 🔓
Tracel AI
Rust deep learning framework with pluggable backends
ONNX 🔓
LF AI
Open interchange format for moving models between frameworks
Model & Training Libraries
Library
Description
Transformers ⭐ 🔓
The model-definition library of record for text, vision, audio and multimodal
Diffusers ⭐ 🔓
State-of-the-art diffusion models for image, audio and video generation
Sentence-Transformers 🔓
Train and serve embedding and reranking models
Accelerate 🔓
Run the same training script on any distributed configuration
Lightning ⭐ 🔓
Remove training boilerplate without giving up PyTorch control
Composer 🔓
MosaicML's speed-focused training library with algorithmic optimizations
timm 🔓
The reference collection of vision backbones and training recipes
Distributed & Large-Scale Training
Framework
Description
DeepSpeed ⭐ 🔓
ZeRO sharding, offload and pipeline parallelism for very large models
Megatron-LM ⭐ 🔓
NVIDIA's tensor/pipeline/sequence parallelism reference implementation
PyTorch FSDP 🔓
Native fully-sharded data parallel training
torchtitan 🔓
PyTorch-native reference for pretraining at scale with 4D parallelism
Ray Train 🔓
Distributed training orchestration across heterogeneous clusters
Colossal-AI 🔓
Unified parallelism toolkit for large model training
Nanotron 🔓
Minimal, readable 3D-parallel pretraining library
MaxText 🔓
High-performance JAX/TPU LLM training reference
Levanter 🔓
Legible, scalable, bitwise-reproducible JAX training
Reinforcement Learning & Post-Training
Library
Description
TRL ⭐ 🔓
SFT, DPO, GRPO and PPO trainers that integrate with Transformers
verl ⭐ 🔓
Production-grade RL library for LLM post-training (HybridFlow)
OpenRLHF 🔓
Scalable RLHF built on Ray, vLLM and DeepSpeed
TorchRL 🔓
PyTorch-native RL primitives and environments
Gymnasium ⭐ 🔓
The standard RL environment API (maintained fork of OpenAI Gym)
PettingZoo 🔓
Gymnasium for multi-agent environments
Stable-Baselines3 🔓
Reliable, well-tested implementations of standard RL algorithms
CleanRL 🔓
Single-file RL implementations built for readability
Ray RLlib 🔓
Scalable RL for production workloads
Classical ML & Structured Data
Library
Description
scikit-learn ⭐ 🔓
The baseline that still solves most tabular problems — try it before an LLM
XGBoost ⭐ 🔓
Gradient boosting that remains state of the art on tabular data
LightGBM 🔓
Fast, memory-efficient gradient boosting
CatBoost 🔓
Gradient boosting with excellent categorical feature handling
statsmodels 🔓
Statistical models and hypothesis testing with proper inference
PyTorch Geometric 🔓
Graph neural networks on PyTorch
DGL 🔓
Deep graph library with multi-backend support
Prophet 🔓
Interpretable forecasting for business time series
Darts 🔓
Unified time series forecasting and anomaly detection
⬆ back to top
Cross-Provider & Application SDKs
SDK
Description
Vercel AI SDK ⭐ 🔓
The TypeScript standard for AI apps — unified provider API, streaming, tools, generative UI
LiteLLM ⭐ 🔓
Call 100+ LLMs with the OpenAI format; proxy adds keys, budgets, routing and logs
aisuite 🔓
Thin, uniform interface across providers for easy comparison
Instructor 🔓
Structured, validated outputs across providers
Mirascope 🔓
LLM calls as typed Python functions
Portkey Gateway 🔓 ☁️
Fast AI gateway with routing, fallbacks, caching and guardrails
Chainlit 🔓
Build and ship conversational AI UIs in pure Python
Gradio ⭐ 🔓
Turn any model into a shareable web demo in a few lines
Streamlit 🔓
Data and AI apps in Python, no frontend required
assistant-ui 🔓
Composable React primitives for chat and agent UIs
CopilotKit 🔓
Drop in-app copilots and agentic UX into React apps
⬆ back to top
Tokenization & Context Budgeting
Tool
Description
tiktoken ⭐ 🔓
Fast BPE tokenizer — the standard way to count tokens before you send them
Tokenizers 🔓
Rust-backed tokenization used across the open ecosystem
Token counting API 🆓
Count tokens exactly as the provider will, including tools and images
tokencost 🔓
Estimate USD cost of prompts and completions across providers
ttok 🔓
Count and truncate text to a token budget from the shell
Playgrounds & Prompt IDEs
Tool
Description
Anthropic Console ⭐
Workbench with prompt generator, improver, test cases and evaluations
Google AI Studio 🆓
Fast Gemini prototyping with multimodal input and code export
OpenAI Playground
Model and parameter experimentation with function calling
Langfuse Prompt Management 🔓
Version, label and deploy prompts without a code release
Latitude 🔓
Open-source prompt engineering platform with evals and versioning
Agenta 🔓
Prompt playground, versioning and evaluation for teams
PromptLayer 💰
Prompt registry and observability aimed at non-engineer collaborators
Tool
Description
Argilla ⭐ 🔓
Collaboration tool for AI engineers who care about data quality
Label Studio ⭐ 🔓
Multi-type labeling for text, image, audio, video and time series
doccano 🔓
Lightweight open-source text annotation
Cleanlab 🔓
Automatically find label errors and data issues in your datasets
Lilac 🔓
Explore, cluster and clean unstructured datasets before training
Nomic Atlas 🔓
Interactively visualize and inspect millions of embeddings
Synthetic Data Generation
Tool
Description
distilabel ⭐ 🔓
Synthetic data and AI feedback pipelines for reliable dataset creation
Curator 🔓
Bespoke Labs' library for post-training and structured synthetic data
DataDreamer 🔓
Reproducible synthetic data and prompting workflows for research
NeMo Curator 🔓
GPU-accelerat
10BC8
ed curation, deduplication and filtering at scale
SDV 🔓
Synthetic tabular and relational data with privacy-preserving evaluation
Quantization & Model Optimization
Tool
Description
bitsandbytes ⭐ 🔓
8-bit and 4-bit quantization primitives used throughout fine-tuning
GGUF ⭐
The dominant quantization format for local inference (produced by llama.cpp)
llm-compressor 🔓
Produce quantized, sparse checkpoints ready to serve on vLLM
AutoAWQ 🔓
Activation-aware weight quantization for 4-bit inference
GPTQModel 🔓
Maintained GPTQ quantization toolkit
torchao 🔓
Native PyTorch quantization and sparsity for training and inference
Optimum 🔓
Export and optimize Transformers for ONNX, OpenVINO and accelerators
ExLlamaV2 🔓
Fast quantized inference on consumer GPUs
Benchmarking & Load Testing
Tool
Description
GuideLLM ⭐ 🔓
Evaluate real-world latency, throughput and capacity of an LLM deployment
vLLM benchmarks 🔓
Reference throughput and serving benchmark scripts
LLMPerf 🔓
Load-test and compare LLM API providers on latency and consistency
k6 🔓
General-purpose load testing that adapts well to streaming endpoints
nvitop 🔓
Interactive GPU process monitoring — the htop of NVIDIA devices
Terminal & Agent Utilities
Tool
Description
llm ⭐ 🔓
Simon Willison's CLI for prompting, logging and templating across models
files-to-prompt 🔓
Concatenate a directory into a single, well-structured prompt
Repomix ⭐ 🔓
Pack an entire repository into one AI-friendly file
gitingest 🔓
Turn any Git repo into a digestible text digest via URL
aichat 🔓
All-in-one CLI with shell assistant, RAG and function calling
Context7 🔓
Up-to-date library documentation served to coding agents over MCP
strip-tags 🔓
Strip HTML down to the text an LLM actually needs
⬆ back to top
🧪 Evaluation & Observability
If you ship without evals, you don't have a product — you have a demo. Instrument first, then iterate.
Tracing & Observability Platforms
Platform
Description
Langfuse ⭐ 🔓 ☁️
Open-source LLM engineering platform — tracing, evals, prompt management, self-hostable
LangSmith 💰 ☁️
Deep tracing, datasets and evaluation; best-in-class for LangChain/LangGraph
Arize Phoenix 🔓
Open-source, OpenTelemetry-native tracing and evaluation notebook-first
Braintrust 💰 ☁️
Eval-centric workflow with playgrounds and CI integration
W&B Weave 🔓 ☁️
Tracing and evaluation inside the Weights & Biases ecosystem
Opik 🔓 ☁️
Comet's open-source tracing, evaluation and guardrails
Helicone 🔓 ☁️
One-line proxy for logging, caching, rate limiting and cost tracking
OpenLLMetry 🔓
OpenTelemetry extensions for LLM apps — vendor-neutral by design
OpenTelemetry GenAI 🆓
The emerging standard semantic conventions for AI telemetry
Datadog LLM Observability 💰 ☁️
LLM traces alongside your existing APM and infrastructure monitoring
Tool
Description
promptfoo ⭐ 🔓
Declarative evals and red-teaming that run in CI
DeepEval ⭐ 🔓
Pytest-like unit testing for LLM outputs with 40+ metrics
Ragas 🔓
Purpose-built RAG and agent metrics
Inspect AI ⭐ 🔓
The UK AI Security Institute's rigorous evaluation framework
lm-evaluation-harness 🔓
The standard harness for academic benchmark reporting
HELM 🔓
Stanford's holistic, multi-metric model evaluation
OpenAI Evals 🔓
Framework and registry of benchmark evals
Giskard 🔓
Automated vulnerability and quality scanning for ML and LLM apps
Evidently 🔓
Data drift, quality and LLM evaluation with rich reports
MLflow LLM Evaluate 🔓
Evaluation integrated with experiment tracking and the model registry
⬆ back to top
Public benchmarks tell you which models to shortlist. They do not tell you whether your system works — only a golden dataset drawn from your own traffic does that. Treat leaderboards as a filter, never as an acceptance test.
Benchmark
Measures
MMLU-Pro ⭐
Harder, less saturated successor to MMLU across 14 domains
GPQA ⭐
Graduate-level physics, chemistry and biology questions that resist search
Humanity's Last Exam
Expert-written questions at the frontier of human knowledge
ARC-AGI ⭐
Abstract visual reasoning designed to resist memorization
BIG-bench Hard
The subset of BIG-bench where models historically underperformed humans
MuSR
Multi-step soft reasoning over natural language narratives
DROP
Discrete reasoning over paragraphs — arithmetic, counting, sorting
Benchmark
Measures
AIME ⭐
Competition math; the current default frontier reasoning check
MATH
12.5k competition problems with step-by-step solutions
GSM8K
Grade-school word problems — largely saturated, still a smoke test
FrontierMath
Unpublished research-level problems designed to stay unsaturated
MathArena
Evaluation on competitions held after model training cutoffs
Code & Software Engineering
Benchmark
Measures
SWE-bench ⭐
Resolving real GitHub issues in real repositories — the industry reference
SWE-bench Verified ⭐
Human-validated 500-problem subset; the number most vendors report
Terminal-Bench ⭐
End-to-end task completion in a real terminal environment
LiveCodeBench
Contamination-free competitive programming, continuously refreshed
BigCodeBench
Realistic tasks requiring compositional use of many libraries
HumanEval
The classic function-synthesis benchmark — saturated, but ubiquitous
MBPP
Entry-level Python programming problems
Aider Polyglot
Editing existing code correctly across six languages
Commit0
Writing entire libraries from scratch against a test suite
Benchmark
Measures
τ-bench / τ²-bench ⭐
Tool use plus policy adherence in realistic customer dialogues
BFCL ⭐
Berkeley Function Calling Leaderboard — the standard tool-calling test
GAIA
General assistant tasks needing tools, browsing and multi-step reasoning
OSWorld
Open-ended computer-use tasks in a real operating system
WebArena
Autonomous agents on realistic self-hosted websites
AgentBench
Agent capability across eight distinct environments
MLE-bench
Agents competing on real Kaggle machine learning tasks
Benchmark
Measures
RULER ⭐
What a model's effective context length actually is, beyond the spec sheet
LongBench
Bilingual, multi-task long-context understanding
Needle in a Haystack
Retrieval of a single fact at varying depth and context length
BEIR ⭐
Zero-shot retrieval generalization across 18 datasets
MTEB ⭐
Massive text embedding benchmark across tasks and languages
LoCoMo
Very long-term conversational memory
Benchmark
Measures
MMMU ⭐
College-level multimodal understanding across 30 subjects
MathVista
Mathematical reasoning in visual contexts
DocVQA
Question answering over document images — core to enterprise RAG
ChartQA
Reasoning over charts and plots
Video-MME
Comprehensive video understanding evaluation
ScreenSpot
GUI element grounding — the basis of computer-use agents
Instruction Following, Safety & Truthfulness
Benchmark
Measures
IFEval ⭐
Verifiable instruction following ("write exactly 3 bullets in JSON")
SimpleQA ⭐
Short-form factuality and, crucially, calibrated abstention
TruthfulQA
Resistance to reproducing common human misconceptions
HaluEval
Hallucination recognition across QA, dialogue and summarization
HarmBench
Standardized red-teaming and refusal robustness
AgentHarm
Harmfulness of agentic behavior, not just generated text
AIR-Bench 2024
Safety evaluated against 314 risk categories drawn from real regulations
Benchmark
Measures
Global-MMLU
Culturally-aware MMLU across 42 languages
MGSM
Grade-school math reasoning in 10+ languages
FLORES-200
Machine translation across 200 languages
XTREME
Cross-lingual transfer across 40 languages and 9 tasks
Belebele
Reading comprehension parallel across 122 language variants
Reading Benchmark Results Honestly
Pitfall
What to do
Contamination
Test sets leak into training data. Prefer continuously-refreshed benchmarks (LiveBench, LiveCodeBench, MathArena)
Saturation
A benchmark above ~90% has stopped discriminating. Move up a tier
Prompt sensitivity
Scores swing several points on formatting alone. Compare only same-harness numbers
Self-reported numbers
Vendors pick favorable configurations. Prefer independent runs
Pass@k inflation
pass@10 is not pass@1. Check which is quoted
Averaging across subsets
A strong average can hide a domain that matters to you. Read the breakdown
Cost and latency omitted
A 2-point gain for 10× the cost is usually a bad trade
⬆ back to top
Pick metrics before you build. A metric you added after seeing results is a story, not a measurement.
Metric
Definition
Use when
Recall@k
Fraction of all relevant documents that appear in the top k
The ceiling on your RAG quality — measure this first
Precision@k
Fraction of the top k that are relevant
Context window is tight and noise hurts
MRR
Mean of 1/rank of the first relevant result
Exactly one right answer exists (lookup, navigation)
nDCG@k
Gain discounted by log rank, normalized to the ideal ordering
Graded relevance and ranking order both matter
MAP
Mean of average precision across queries
Multiple relevant documents, order-sensitive
Hit Rate@k
Share of queries with ≥1 relevant result in top k
A blunt but readable executive-facing number
In practice: optimize retrieval for Recall@k with a generous k , then let a reranker fix precision. Improving the generator cannot recover a document retrieval never returned.
Metric
Question it answers
Typical source
Faithfulness / Groundedness ⭐
Is every claim supported by retrieved context?
Ragas
Answer Relevancy
Does the answer actually address the question asked?
Ragas, DeepEval
Context Precision
Are the retrieved chunks relevant and well-ranked?
Ragas
Context Recall
Was all information needed for the ground truth retrieved?
Ragas
Noise Sensitivity
Does irrelevant retrieved context degrade the answer?
Ragas
Citation Accuracy
Do the cited sources genuinely support the claims?
Custom / LLM judge
Refusal Correctness ⭐
Does it decline when the context truly lacks the answer?
Custom
Generation Quality Metrics
Metric
Type
Notes
LLM-as-Judge ⭐
Model-graded
The workhorse for open-ended output. Requires its own validation against human labels
Pairwise preference ⭐
Model/human
More reliable than absolute scoring — comparison beats calibration
BERTScore
Embedding
Semantic similarity to a reference; better than n-gram overlap
ROUGE
Lexical
Summarization overlap. Weak signal, but cheap and stable
BLEU / chrF
Lexical
Translation. Still standard in MT, poor for open generation
Exact Match / F1
Lexical
Extractive QA and structured field extraction
Perplexity
Probabilistic
Model fit during training — not a product quality metric
Schema validity ⭐
Deterministic
Percentage of outputs that parse and validate. Cheap and unambiguous
On LLM judges: validate the judge before you trust it. Measure judge-vs-human agreement (Cohen's κ) on a labeled sample, control position and verbosity bias, and never judge with the same model and prompt that produced the output.
Classification & Extraction Metrics
Metric
Formula
Use when
Precision
TP / (TP + FP)
False positives are expensive (spam filters, auto-actions)
Recall
TP / (TP + FN)
Misses are expensive (safety, compliance, fraud)
F1
2PR / (P + R)
You need one balanced number
Fβ
Weighted harmonic mean
Recall matters β× more than precision
PR-AUC ⭐
Area under precision-recall
Imbalanced classes — preferred over ROC-AUC here
Cohen's κ
Agreement above chance
Comparing model to human, or judge to human
ECE
Expected calibration error
Confidence scores drive downstream routing or escalation
Metric
What it captures
Task Success Rate ⭐
Did the agent achieve the goal, verified by an outcome check — not self-report
Tool Selection Accuracy
Right tool chosen for the step
Tool Call Validity
Arguments parse and satisfy the schema
Steps to Completion
Efficiency; sudden growth signals looping or confusion
Cost per Task ⭐
Total tokens across every turn and subagent — the number finance asks about
Loop / Stall Rate
Runs hitting the step ceiling without terminating
Recovery Rate
Runs that hit an error and still finished successfully
Human Intervention Rate ⭐
How often a person had to step in — the real autonomy measure
Trajectory Match
Did it follow an acceptable path, not just reach an acceptable end
Serving & Production Metrics
Metric
Definition
Why it matters
TTFT ⭐
Time to first token
Dominates perceived latency in streaming UIs
TPOT / ITL
Time per output token (inter-token latency)
Determines whether text renders faster than reading speed
E2E Latency
Total request time
The number that matters for non-streaming and batch
Throughput
Output tokens/sec, requests/sec
Capacity planning and cost per unit of work
Goodput ⭐
Throughput of requests that met their SLO
Prevents "fast but useless" scaling wins
Queue Time
Wait before scheduling
Early warning of saturation
Cache Hit Rate ⭐
Prompt-cache hits / total
Usually the biggest single cost lever
Cost per Request/User ⭐
Spend attributed by tenant and feature
Unit economics; measure before scaling
Error / Refusal Rate
4xx, 5xx, overloads, refusals
Split these — a refusal is a product signal, not an outage
Report percentiles, not averages. p50 hides the experience; p95 and p99 are what users complain about. Track TTFT and E2E separately — they fail for different reasons.
Metric Selection by Use Case
Use case
Primary metric
Guardrail metrics
RAG question answering
Faithfulness
Context Recall, refusal correctness, cost/query
Search & retrieval
nDCG@10
Recall@50, p95 latency
Summarization
LLM-judge quality
Faithfulness, length adherence
Structured extraction
Field-level F1
Schema validity rate, cost/document
Classification
PR-AUC or F1
Calibration, per-class recall
Coding agent
Task success rate
Test pass rate, cost/task, human intervention
Customer support agent
Resolution rate
Escalation rate, policy violations, CSAT
Content generation
Pairwise preference
Safety violation rate, brand adherence
⬆ back to top
🛡️ AI Security & Guardrails
Standards & Threat Models
Guardrails & Runtime Defense
Tool
Description
Guardrails AI ⭐ 🔓
Input/output validators ("guards") with a shared hub of checks
NeMo Guardrails 🔓
Programmable rails for topic, safety and execution control
LLM Guard 🔓
Comprehensive scanner suite — injection, PII, toxicity, secrets
Llama Guard / Purple Llama 🔓
Meta's safety classifiers and cybersecurity evals for LLM I/O
Granite Guardian 🔓
IBM's risk detection models for prompts, responses and RAG groundedness
ShieldGemma 🔓
Google's open content-safety classifiers
Presidio ⭐ 🔓
PII detection, anonymization and redaction for text and images
Rebuff 🔓
Multi-layered prompt-injection detection with canary tokens
Bedrock Guardrails ☁️
Managed policy, PII and contextual-grounding filters
Azure AI Content Safety ☁️
Content filtering plus prompt-shield detection
Tool
Description
garak ⭐ 🔓
LLM vulnerability scanner — the nmap of prompt attacks
PyRIT ⭐ 🔓
Microsoft's Python Risk Identification Toolkit for generative AI
promptfoo red team 🔓
Automated adversarial test generation wired into CI
Adversarial Robustness Toolbox 🔓
Attacks and defenses across the full ML stack
Counterfit 🔓
CLI for automating AI security assessments
HarmBench 🔓
Standardized evaluation of red-teaming and refusal robustness
Pattern
Practice
Untrusted content isolation
Anything retrieved, scraped or returned by a tool is untrusted input — never let it grant new authority
Sandboxed execution
Run generated code in E2B , Daytona , gVisor or Firecracker — never on the host
Human-in-the-loop gates
Require approval for irreversible, outward-facing or high-blast-radius actions
Least-privilege credentials
Scoped, short-lived tokens per tool; no ambient production credentials
Egress control
Allowlist network destinations to contain exfiltration via tool calls
Dual LLM / CaMeL
Separate the planner from the data-handling model so injected text can't change the plan (paper )
Full audit trail
Log prompts, tool calls, arguments, identities and outcomes for replay
⬆ back to top
Enterprise Platforms & Assistants
Stage
Key questions
Use case selection
Is there a measurable baseline? What is the cost of being wrong? Is a human already reviewing this output?
Data readiness
Where does ground truth live? Are permissions enforced at retrieval time, not just in the UI?
Build vs. buy
Does this differentiate us? Total cost including evals, on-call and model upgrades?
Security review
Threat-modelled against OWASP LLM Top 10 and MITRE ATLAS? Injection paths enumerated?
Evaluation
Golden dataset committed to version control? Regression gate in CI before deploy?
Observability
Traces, cost per request/user, and a feedback signal captured in production?
Governance
Model inventory, owner, risk tier, documented human oversight, retention policy?
Change management
Who is accountable when the model changes? Is there a rollback and a pinned version?
⬆ back to top
☁️ Cloud AI (Azure, AWS, GCP)
Cross-Cloud & Alternatives
⬆ back to top
Experiment Tracking & Registries
Tool
Description
MLflow ⭐ 🔓
Tracking, projects, models and registry — with first-class GenAI support
Weights & Biases 💰 ☁️
Experiment tracking, sweeps, artifacts and model management
ClearML 🔓
End-to-end MLOps: experiments, orchestration, data and serving
Neptune 💰 ☁️
Tracking built for very long, large-scale training runs
DVC 🔓
Git-native data and model versioning
LakeFS 🔓
Git-like branching and commits over object storage
Pipelines & Orchestration
Tool
Description
ZenML ⭐ 🔓
Portable MLOps + LLMOps pipelines that run on any stack
Kubeflow 🔓
The Kubernetes-native ML toolkit
Metaflow 🔓
Netflix's human-centric framework for real-life ML/AI projects
Flyte 🔓
Strongly-typed, reproducible, Kubernetes-native workflows
Dagster 🔓
Asset-oriented orchestration with strong data-quality semantics
Airflow 🔓
The ubiquitous scheduler, now with AI/LLM provider packages
Ray ⭐ 🔓
Distributed compute underpinning much of modern training and serving
SkyPilot 🔓
Run jobs on the cheapest available GPUs across clouds
Feast 🔓
Open-source feature store for online/offline consistency
Fine-Tuning & Post-Training
Tool
Description
Unsloth ⭐ 🔓
Dramatically faster, lower-memory fine-tuning for popular open models
LLaMA-Factory ⭐ 🔓
Unified fine-tuning for 100+ models with a web UI
Axolotl 🔓
Config-driven fine-tuning covering the full method matrix
TRL ⭐ 🔓
SFT, DPO, GRPO and PPO from Hugging Face
PEFT 🔓
LoRA/QLoRA and other parameter-efficient methods
torchtune 🔓
Native PyTorch library for LLM post-training
DeepSpeed 🔓
ZeRO optimization for training very large models
Liger Kernel 🔓
Triton kernels that cut training memory substantially
verl 🔓
Production RL library for LLM post-training
Cost & Gateway Management
Tool
Description
LiteLLM Proxy ⭐ 🔓
Virtual keys, budgets, rate limits, fallbacks and spend tracking
Portkey 🔓 ☁️
AI gateway with caching, retries, guardrails and analytics
Cloudflare AI Gateway ☁️
Caching, rate limiting and observability in front of any provider
Kong AI Gateway 🔓 ☁️
Enterprise API gateway extended with AI-specific plugins
Envoy AI Gateway 🔓
CNCF-aligned gateway for LLM/AI traffic
Prompt caching
Often the single largest cost lever — cache stable prefixes
Batch APIs
~50% discounts for asynchronous, non-latency-sensitive work
⬆ back to top
High-Throughput Inference Servers
Engine
Description
vLLM ⭐ 🔓
The default open serving engine — PagedAttention, continuous batching, OpenAI-compatible API
SGLang ⭐ 🔓
Fast serving with RadixAttention prefix caching; excellent for agentic/structured workloads
TensorRT-LLM 🔓
NVIDIA's maximum-performance compiled inference
Text Generation Inference 🔓
Hugging Face's production serving stack
LMDeploy 🔓
Efficient compression, deployment and serving toolkit
Triton Inference Server 🔓
Multi-framework, multi-model serving with dynamic batching
Ray Serve 🔓
Compose multi-model, multi-stage inference graphs
BentoML 🔓
Package and ship any model as a production service
KServe 🔓
Kubernetes-native, standards-based model serving
Tool
Description
llama.cpp ⭐ 🔓
CPU/GPU inference anywhere, from laptops to Raspberry Pi
Ollama ⭐ 🔓
The friendliest way to run and serve local models
ONNX Runtime 🔓
Cross-platform accelerated inference including browsers
ExecuTorch 🔓
PyTorch on mobile and embedded devices
MLC LLM 🔓
Compile and deploy LLMs to almost any hardware backend
OpenVINO 🔓
Intel-optimized inference for CPU/iGPU/NPU
LocalAI 🔓
Drop-in OpenAI-compatible API you can self-host
Xinference 🔓
Serve LLMs, embeddings and rerankers with one command
Serverless & Managed Inference
⬆ back to top
⬆ back to top
⬆ back to top
Engineering & Research Blogs
Newsletter
Cadence
Focus
The Batch ⭐
Weekly
Andrew Ng's balanced industry roundup
Import AI ⭐
Weekly
Jack Clark on research, policy and capability trends
TLDR AI
Daily
Five-minute daily digest
Ben's Bites
Daily
Product- and startup-oriented AI news
Last Week in AI
Weekly
Summaries plus a podcast
AI Snake Oil
Occasional
Princeton researchers on separating hype from evidence
Latent Space
Weekly
AI engineering practice and interviews
PulseMCP
Weekly
MCP ecosystem news and new servers
Data Machina
Weekly
Dense technical link roundup
⬆ back to top
Core libraries — PyTorch, JAX, Transformers, scikit-learn and friends — live in AI & ML Frameworks . This section covers the applications and utilities built on top of them.
Chat UIs & Self-Hosted Apps
Project
Description
Open WebUI ⭐
Feature-rich, offline-capable self-hosted AI interface
LibreChat ⭐
Multi-provider ChatGPT-style app with agents, RAG and auth
AnythingLLM
All-in-one desktop/docker app for documents and agents
Jan
Offline-first, open-source ChatGPT alternative
Lobe Chat
Modern multi-modal chat framework with a plugin ecosystem
Khoj
Self-hostable AI second brain over your own documents
Perplexica
Open-source AI answer engine
SearXNG
Privacy-respecting metasearch — the retrieval backbone for many OSS agents
Speech, Vision & Creative
Project
Description
Whisper ⭐
Robust multilingual speech recognition
faster-whisper
CTranslate2 reimplementation, several times faster
WhisperX
Word-level timestamps and speaker diarization
Kokoro
Small, high-quality open TTS
F5-TTS
Fast, natural zero-shot voice cloning
Piper
Fast local neural TTS for edge devices
ComfyUI ⭐
Node-based generative pipeline engine with a huge ecosystem
Ultralytics YOLO
Real-time detection, segmentation and tracking
Segment Anything
Promptable segmentation foundation model
Project
Description
Polars ⭐
Blazing-fast DataFrames for AI data preparation
DuckDB ⭐
In-process analytical SQL — ideal for eval and dataset work
Pydantic ⭐
Validation layer underpinning most Python AI tooling
Datasets
Load, stream and process datasets of any size efficiently
datatrove
Large-scale text data processing pipelines for pretraining corpora
Apache Arrow
The columnar memory format most AI data tooling interchanges through
⬆ back to top
💼 Real-world Case Studies
Notable Engineering Writeups
Company
Use case
Takeaway
LinkedIn ⭐
Generative AI product
Honest account of the last 20% — evals, latency and quality plateaus
Uber
QueryGPT: natural language → SQL
Multi-agent decomposition beat a single large prompt
Uber
Platform evolution
Migrating an established ML platform to serve GenAI
Airbnb
Conversational AI platform
Constrain agents with workflows rather than trusting free-form autonomy
Airbnb
Migrating 3.5k test files with LLMs
Retry loops with expanded context beat prompt perfection
DoorDash
Support automation
RAG plus an LLM Judge to control hallucination in customer-facing replies
Discord
Shipping GenAI features
A pragmatic prototype → eval → productionize loop
GitHub
Copilot Workspace
Designing agentic developer UX around review and correction
Grab
Data classification at scale
LLMs for governance and metadata, not just chat
Klarna
Customer service assistant
Widely-cited deployment numbers — and a useful lesson in later rebalancing toward humans
Bloomberg
BloombergGPT
Domain-specific pretraining, and why most teams should not repeat it
Replit
Agent self-testing at scale
REPL-base
8CCE
d verification instead of expensive computer-use checking
Anthropic ⭐
Multi-agent research system
Orchestrator–worker patterns, and where multi-agent genuinely pays off
Anthropic ⭐
Building effective agents
Start simple: most "agent" problems are solved by workflows
⬆ back to top
Contributions are very welcome — this list is only as good as the people maintaining it.
Before opening a PR, please check that your entry:
✅ Is production-relevant — something a team would actually deploy, not a weekend demo
✅ Is actively maintained — recent commits, releases or documentation updates
✅ Is not already listed under another section
✅ Uses the canonical link — the project's repository or official documentation
✅ Follows the existing table format and has a one-line, non-marketing description
✅ Is placed in the most specific applicable section
How to contribute:
# 1. Fork and clone
git clone https://github.com/< your-username> /awesome-ai.git
cd awesome-ai
# 2. Create a branch
git checkout -b add-awesome-tool
# 3. Edit README.md, then commit
git commit -am " Add <Tool> to <Section>"
# 4. Push and open a pull request
git push origin add-awesome-tool
Found a dead link, a project that has been archived, or a description that is no longer accurate? Open an issue — corrections are as valuable as additions.
⬆ back to top
If this list is useful to you, please consider starring it — it helps other engineers find it.
Released under the MIT License .
Copyright © 2026 Seyhun Akyürek
All trademarks, product names and logos belong to their respective owners.
Listing here does not imply endorsement by, or affiliation with, any listed project or vendor.
⬆ back to top
Made with ❤️ for the AI engineering community