Agent & Prompt Frameworks
Developer frameworks that turn prompts into programs: structured output libraries, agent orchestration toolkits, prompt compilers and automatic prompt optimizers.
LiteLLM
A gateway and Python SDK that exposes more than a hundred model APIs in the OpenAI request format, with budgets, fallbacks and logging so one prompt can be tested across providers.
AICI
A Microsoft research project that runs prompt controllers as WebAssembly modules inside the inference loop, letting custom logic constrain and steer token generation in real time.
Trace
A Microsoft Research framework for end-to-end optimization of AI workflows, propagating execution feedback back through a computation graph to update prompts, code and parameters.
Agno
A Python framework and runtime for building agents with instructions, tools, memory and knowledge, then serving them behind an API. It emphasises low overhead per agent instance.
Fabric
An open-source framework built around a large library of reusable prompt patterns invoked from the command line, each pattern being a markdown system prompt for a specific task.
Google Agent Development Kit
Google's open-source Python toolkit for building and deploying agents, with agent instructions, tools, sessions and a built-in evaluation runner for checking agent trajectories.
AG2
The community-maintained continuation of the original AutoGen codebase, providing conversational multi-agent patterns with configurable system messages and group chat managers.
Rig
A Rust library for building LLM-powered applications with typed agents, embeddings and vector store integrations, aimed at teams that want prompt logic in a compiled language.
12-Factor Agents
A written set of principles for building reliable LLM software, including factors on owning your prompts, owning context construction and treating tools as structured output.
Chainlit
An open framework and hosted service for building conversational AI interfaces with built-in observability, feedback collection and evaluation of the conversations produced.
Strands Agents SDK
An open-source SDK, originally from AWS, for building production agents in Python and TypeScript with an explicit model-tools-prompt loop that developers control end to end.
FastChat
The LMSYS platform for serving and evaluating chat models, including the MT-Bench multi-turn judge harness and the infrastructure used for crowd-sourced model comparisons.
Bifrost
An open-source AI gateway that fronts many model providers behind one API, adding routing, load balancing, guardrails and governance for prompts sent from applications.
ContextGem
A Python framework for extracting structured data from documents where the extraction prompts and schemas are generated from declarative aspect and concept definitions.
Promptify
A Python library with prompt templates for common NLP tasks such as named entity recognition, classification and question answering, returning parsed structured output.
Portkey AI Gateway
The open-source gateway behind Portkey, routing to a large catalogue of models with built-in guardrails, retries and conditional routing rules configured per request.
Ragbits
A set of building blocks from deepsense.ai for generative AI applications, including typed prompt classes with input and output models plus a CLI for testing prompts.
Swarms
A multi-agent orchestration framework offering several collaboration topologies such as sequential, concurrent and hierarchical swarms, with per-agent system prompts.
Dynamiq
An orchestration framework for agentic and retrieval workflows, providing nodes for prompts, tools, retrievers and validators that are wired into a runnable graph.
Graph of Thoughts
The official implementation of the Graph of Thoughts paper, which models a prompt-solving process as a graph of thoughts that can be merged, refined and scored.
Guardrails AI
A Python framework that wraps model calls with input and output validators from a shared hub, re-asking the model or failing closed when a check does not pass.
Tree of Thoughts
A plug-and-play implementation of tree-of-thoughts prompting, where a model explores and evaluates multiple reasoning branches before committing to an answer.
OpenAI Agents SDK
OpenAI's lightweight framework for multi-agent workflows, built around agents with instructions, handoffs between agents, guardrails and tracing of each run.
Relevance AI
A no-code platform for building specialist AI agents and multi-step workflows, where each agent's instructions and tools are configured in a visual builder.
Agently
A development framework for generative AI applications with a chained request syntax that builds structured prompts and enforces an expected output schema.
DataDreamer
A Python library for reproducible synthetic data and prompting workflows, with caching, resumable sessions and export of the full prompt history for a run.
Taskade
A workspace that combines project management with configurable AI agents and prompt-driven automations that operate on the team's own documents and tasks.
LLM Gateway
An open-source gateway that routes requests across multiple model providers through one unified API and records analytics on cost and usage per request.
Dust
A workspace where teams build and share custom assistants connected to company tools and documents, with prompt instructions managed per assistant.
Swiftide
A Rust framework for streaming indexing, querying and agent workflows, with prompt templates rendered through Tera and an async pipeline model.