A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

The first real AI developer

Create Customized Software using Natural Language Idea (through LLM-powered Multi-Agent Collaboration)

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)

Distribute and run LLMs with a single file.

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

Official inference library for Mistral models

Bisheng is an open LLM devops platform for next generation AI applications.

🚀 基于 LLM 大语言模型的知识库问答系统。开箱即用，支持快速嵌入到第三方业务系统，1Panel 官方出品。

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

Multi agent system for AI-driven software development. Combine LLM with DevOps tools to convert natural language requirements into working software. Supports any development language and extends the existing code.

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

Retrieval and Retrieval-augmented LLMs

a state-of-the-art-level open visual language model | 多模态预训练模型

Official release of InternLM2 7B and 20B base and chat models. 200K context support

A Blazing Fast AI Gateway. Route to 200+ LLMs with 1 fast & friendly API.

🤖 🔥 Language-to-actions engine

A code-first agent framework for seamlessly planning and executing data analytics tasks.

An Open-source Framework for Autonomous Language Agents

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

Unified framework for building enterprise RAG pipelines with small, specialized models

Robust recipes to align language models with human and AI preferences

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

A series of large language models developed by Baichuan Intelligent Technology

Tools for merging pretrained large language models.

LLM App templates for Dynamic RAG. Ready to run with Docker,⚡in sync with your data sources.

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

Universal and Transferable Attacks on Aligned Language Models

SoTA LLM for converting natural language questions to SQL queries

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

🚀 Next Generation AI One-Stop Internationalization Solution. 🚀 下一代 AI 一站式 B/C 端解决方案，支持 OpenAI，Midjourney，Claude，讯飞星火，Stable Diffusion，DALL·E，ChatGLM，通义千问，腾讯混元，360 智脑，百川 AI，火山方舟，新必应，Gemini，Moonshot 等模型，支持对话分享，自定义预设，云端同步，模型市场，支持弹性计费和订阅计划模式，支持图片解析，支持联网搜索，支持模型缓存，丰富美观的后台管理与仪表盘数据统计。

An Open-source Toolkit for LLM Development

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but completely free and 100% private.

The LLM Evaluation Framework

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

开源社区第一个能下载、能运行的中文 LLaMA2 模型！

ms-swift: Use PEFT or Full-parameter to finetune 250+ LLMs or 35+ MLLMs. (Qwen2, GLM4, Internlm2, Yi, Llama3, Llava, MiniCPM-V, Deepseek, Baichuan2, Gemma2, Phi3-Vision, ...)

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

收集和梳理垂直领域的开源模型、数据集及评测基准。

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

Developer AI Persona Search Agent

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

InternLM-XComposer2 is a groundbreaking vision-language large model (VLLM) excelling in free-form text-image composition and comprehension.

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM, ONNX). Powers 👋 Jan

Hackers' Guide to Language Models

BionicGPT is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality

适配轻小说/Galgame的日中翻译大模型

大模型基础: 一文了解大模型基础知识

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

FinGLM: 致力于构建一个开放的、公益的、持久的金融大模型项目，利用开源开放来促进「AI+金融」。

A comprehensive guide to building RAG-based LLM applications for production.

Open-source observability for your LLM application, based on OpenTelemetry

🤖 Deploy a private ChatGPT alternative hosted within your VPC. 🔮 Connect it to your organization's knowledge base and use it as a corporate oracle. Supports open-source LLMs like Llama 2, Falcon, and GPT4All.

LLaMA v2 Chatbot

RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.

No-code multi-agent framework to build LLM Agents, workflows and applications with your data

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

Fast, Accurate, Lightweight Python library to make State of the Art Embedding

Python SDK for agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

The TypeScript framework for AI development

[IJCAI 2024] Generate different roles for GPTs to form a collaborative entity for complex tasks.

Various AI scripts. Mostly Stable Diffusion stuff.

Xwin-LM: Powerful, Stable, and Reproducible LLM Alignment

A curated list of language modeling researches for code and related datasets.

A lightweight framework for building LLM-based agents

The Security Toolkit for LLM Interactions

Ship RAG based LLM web apps in seconds.

Traditional Mandarin LLMs for Taiwan

Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone

Epsilla is a high performance Vector Database Management System. Try out hosted Epsilla at https://cloud.epsilla.com/

Serving multiple LoRA finetuned LLM as one

LLMs as Copilots for Theorem Proving in Lean

日本語LLMまとめ - Overview of Japanese LLMs

Code for 3D-LLM: Injecting the 3D World into Large Language Models

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training

A boilerplate for creating a Llama 3 chat app

Scale LLM Engine public repository

A curation of awesome tools, documents and projects about LLM Security.

One API for all LLMs either Private or Public (Anthropic, Llama V2, GPT 3.5/4, Vertex, GPT4ALL, HuggingFace ...) 🌈🐂 Replace OpenAI GPT with any LLMs in your app with one line.

LLPhant - A comprehensive PHP Generative AI Framework using OpenAI GPT 4. Inspired by Langchain

FlashInfer: Kernel Library for LLM Serving

Deterministic LLMs Outputs for AI Applications and AI Agents

Run AI models locally on your machine with node.js bindings for llama.cpp. Force a JSON schema on the model output on the generation level

Toolkit for fine-tuning, ablating and unit-testing open-source LLMs.

AWS-native chatbot using Bedrock + Claude (+Mistral)

This repo includes Claude prompt curation to use Claude better.

DriveLM: Driving with Graph Visual Question Answering

The simplest way to build AI workloads on Postgres

[AAAI 2024 Oral] AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models

Industrial-first evaluation benchmark for LLMs in the DevOps/AIOps domain.

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.