Back

#LlamaStack: Standardizing #GenerativeAI Development

Defines open API specs for #AI application building blocks

Covers full lifecycle: model training, evaluation, production deployment

Includes APIs for inference, safety, memory, agents, and more

Supports multiple environments: local, hosted, and on-device

Features:

#OpenSource API providers and distributions

Mix-and-match capabilities (e.g., local small models, cloud-based large models)

Consistent APIs across platforms (server, mobile, etc.)

Supported implementations:

API Providers: #Meta Reference, #Fireworks, #AWS Bedrock, #Together, #Ollama, TGI, #Chroma, PG Vector, #PyTorch ExecuTorch

Distributions: Meta Reference, Dell-TGI

Easy installation via pip or from source Includes 'llama' CLI for managing distributions, models, and more

Learn more: https://github.com/meta-llama/llama-stack