A programming framework for agentic AI 🤖

The first real AI developer

Create Customized Software using Natural Language Idea (through LLM-powered Multi-Agent Collaboration)

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)

Distribute and run LLMs with a single file.

Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama for WhatsApp & Messenger.

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

🚀 MaxKB 是一款基于大语言模型和 RAG 的开源知识库问答系统，广泛应用于智能客服、企业内部知识库、学术研究与教育等场景。

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

Official inference library for Mistral models

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

vits2 backbone with multilingual-bert

Retrieval and Retrieval-augmented LLMs

🚀 Next Generation AI One-Stop Internationalization Solution. 🚀 下一代 AI 一站式 B/C 端解决方案，支持 OpenAI，Midjourney，Claude，讯飞星火，Stable Diffusion，DALL·E，ChatGLM，通义千问，腾讯混元，360 智脑，百川 AI，火山方舟，新必应，Gemini，Moonshot 等模型，支持对话分享，自定义预设，云端同步，模型市场，支持弹性计费和订阅计划模式，支持图片解析，支持联网搜索，支持模型缓存，丰富美观的后台管理与仪表盘数据统计。

Unified framework for building enterprise RAG pipelines with small, specialized models

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

Multi agent system for AI-driven software development. Combine LLM with DevOps tools to convert natural language requirements into working software. Supports any development language and extends the existing code.

Official release of InternLM2.5 base and chat models. 1M context support

A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

a state-of-the-art-level open visual language model | 多模态预训练模型

A code-first agent framework for seamlessly planning and executing data analytics tasks.

An Open-source Framework for Data-centric, Self-evolving Autonomous Language Agents

🤖 🔥 Language-to-actions engine

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

Tools for merging pretrained large language models.

Robust recipes to align language models with human and AI preferences

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs. (LLM: Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, Gemma2, ...; MLLM: Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL, Phi3.5-Vision, ...)

A series of large language models developed by Baichuan Intelligent Technology

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

The LLM Evaluation Framework

Open-source observability for your LLM application, based on OpenTelemetry

Universal and Transferable Attacks on Aligned Language Models

Various AI scripts. Mostly Stable Diffusion stuff.

SoTA LLM for converting natural language questions to SQL queries

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

SDG is a specialized framework designed to generate high-quality structured tabular data.

The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but completely free and 100% private.

大模型基础: 一文了解大模型基础知识

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

An Open-source Toolkit for LLM Development

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

适配轻小说/Galgame的日中翻译大模型

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

收集和梳理垂直领域的开源模型、数据集及评测基准。

开源社区第一个能下载、能运行的中文 LLaMA2 模型！

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

Local AI API Platform

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

Developer AI Persona Search Agent

Automatable GenAI Scripting

BionicGPT is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality

Curated tutorials and resources for Large Language Models, Text2SQL, Text2DSL、Text2API、Text2Vis and more.

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

A lightweight framework for building LLM-based agents

FinGLM: 致力于构建一个开放的、公益的、持久的金融大模型项目，利用开源开放来促进「AI+金融」。

Hackers' Guide to Language Models

A comprehensive guide to building RAG-based LLM applications for production.

[TMLR] A curated list of language modeling researches for code and related datasets.

No-code multi-agent framework to build LLM Agents, workflows and applications with your data

RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

Fast, Accurate, Lightweight Python library to make State of the Art Embedding

🤖 Deploy a private ChatGPT alternative hosted within your VPC. 🔮 Connect it to your organization's knowledge base and use it as a corporate oracle. Supports open-source LLMs like Llama 2, Falcon, and GPT4All.

FlashInfer: Kernel Library for LLM Serving

Chat language model that can use tools and interpret the results

LLaMA v2 Chatbot

This repo includes Claude prompt curation to use Claude better.

Traditional Mandarin LLMs for Taiwan

The Security Toolkit for LLM Interactions

[IJCAI 2024] Generate different roles for GPTs to form a collaborative entity for complex tasks.

The TypeScript framework for AI development

Xwin-LM: Powerful, Stable, and Reproducible LLM Alignment

日本語LLMまとめ - Overview of Japanese LLMs

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

LLMs as Copilots for Theorem Proving in Lean

Serving multiple LoRA finetuned LLM as one

Ship RAG based LLM web apps in seconds.

Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone

Code for 3D-LLM: Injecting the 3D World into Large Language Models

A curation of awesome tools, documents and projects about LLM Security.

One API to connect your data sources to your LLMs

Reliable LLM Memory for AI Applications and AI Agents

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

Epsilla is a high performance Vector Database Management System. Try out hosted Epsilla at https://cloud.epsilla.com/

AWS-native chatbot using Bedrock + Claude (+Mistral)

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

LLPhant - A comprehensive PHP Generative AI Framework using OpenAI GPT 4. Inspired by Langchain

[ECCV 2024 Oral] DriveLM: Driving with Graph Visual Question Answering

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

A boilerplate for creating a Llama 3 chat app

Council is an open-source platform for the rapid development and robust deployment of customized generative AI applications