A programming framework for agentic AI 🤖

The first real AI developer

Create Customized Software using Natural Language Idea (through LLM-powered Multi-Agent Collaboration)

Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. Multiple engine support (llama.cpp, TensorRT-LLM)

Distribute and run LLMs with a single file.

Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama for WhatsApp & Messenger.

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

🚀 MaxKB 是一款基于大语言模型和 RAG 的开源知识库问答系统，广泛应用于智能客服、企业内部知识库、学术研究与教育等场景。

A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!

Official inference library for Mistral models

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include: GenAI workflow, RAG, Agent, Unified model management, Evaluation, SFT, Dataset Management, Enterprise-level System Management, Observability and more.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

vits2 backbone with multilingual-bert

Retrieval and Retrieval-augmented LLMs

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

Multi agent system for AI-driven software development. Combine LLM with DevOps tools to convert natural language requirements into working software. Supports any development language and extends the existing code.

Unified framework for building enterprise RAG pipelines with small, specialized models

Official release of InternLM2.5 base and chat models. 1M context support

A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.

a state-of-the-art-level open visual language model | 多模态预训练模型

A code-first agent framework for seamlessly planning and executing data analytics tasks.

An Open-source Framework for Data-centric, Self-evolving Autonomous Language Agents

🤖 🔥 Language-to-actions engine

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

Tools for merging pretrained large language models.

Robust recipes to align language models with human and AI preferences

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

Use PEFT or Full-parameter to finetune 400+ LLMs or 100+ MLLMs. (LLM: Qwen2.5, Llama3.2, GLM4, Internlm2.5, Yi1.5, Mistral, Baichuan2, DeepSeek, Gemma2, ...; MLLM: Qwen2-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL, Phi3.5-Vision, ...)

A series of large language models developed by Baichuan Intelligent Technology

An efficient, flexible and full-featured toolkit for fine-tuning LLM (InternLM2, Llama3, Phi3, Qwen, Mistral, ...)

The LLM Evaluation Framework

Universal and Transferable Attacks on Aligned Language Models

Open-source observability for your LLM application, based on OpenTelemetry

SoTA LLM for converting natural language questions to SQL queries

Various AI scripts. Mostly Stable Diffusion stuff.

SDG is a specialized framework designed to generate high-quality structured tabular data.

RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry

The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but completely free and 100% private.

大模型基础: 一文了解大模型基础知识

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

An Open-source Toolkit for LLM Development

ModelScope-Agent: An agent framework connecting models in ModelScope with the world

Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output

适配轻小说/Galgame的日中翻译大模型

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

开源社区第一个能下载、能运行的中文 LLaMA2 模型！

收集和梳理垂直领域的开源模型、数据集及评测基准。

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

Local AI API Platform

Developer AI Persona Search Agent

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen

BionicGPT is an on-premise replacement for ChatGPT, offering the advantages of Generative AI while maintaining strict data confidentiality

A lightweight framework for building LLM-based agents

Project Page for "LISA: Reasoning Segmentation via Large Language Model"

Curated tutorials and resources for Large Language Models, Text2SQL, Text2DSL、Text2API、Text2Vis and more.

Automatable GenAI Scripting

Hackers' Guide to Language Models

FinGLM: 致力于构建一个开放的、公益的、持久的金融大模型项目，利用开源开放来促进「AI+金融」。

A comprehensive guide to building RAG-based LLM applications for production.

[TMLR] A curated list of language modeling researches for code and related datasets.

No-code multi-agent framework to build LLM Agents, workflows and applications with your data

RAG architecture: index and query any data using LLM and natural language, track sources, show citations, asynchronous memory patterns.

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

Fast, Accurate, Lightweight Python library to make State of the Art Embedding

🤖 Deploy a private ChatGPT alternative hosted within your VPC. 🔮 Connect it to your organization's knowledge base and use it as a corporate oracle. Supports open-source LLMs like Llama 2, Falcon, and GPT4All.

Chat language model that can use tools and interpret the results

FlashInfer: Kernel Library for LLM Serving

LLaMA v2 Chatbot

Traditional Mandarin LLMs for Taiwan

The Security Toolkit for LLM Interactions

This repo includes Claude prompt curation to use Claude better.

[IJCAI 2024] Generate different roles for GPTs to form a collaborative entity for complex tasks.

The TypeScript framework for AI development

Xwin-LM: Powerful, Stable, and Reproducible LLM Alignment

日本語LLMまとめ - Overview of Japanese LLMs

LLMs as Copilots for Theorem Proving in Lean

Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level

Serving multiple LoRA finetuned LLM as one

Ship RAG based LLM web apps in seconds.

Retrieval Augmented Generation (RAG) framework and context engine powered by Pinecone

Code for 3D-LLM: Injecting the 3D World into Large Language Models

A curation of awesome tools, documents and projects about LLM Security.

One API to connect your data sources to your LLMs

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.

Epsilla is a high performance Vector Database Management System. Try out hosted Epsilla at https://cloud.epsilla.com/

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

AWS-native chatbot using Bedrock + Claude (+Mistral)

Reliable LLM Memory for AI Applications and AI Agents

LLPhant - A comprehensive PHP Generative AI Framework using OpenAI GPT 4. Inspired by Langchain

[ECCV 2024 Oral] DriveLM: Driving with Graph Visual Question Answering

A boilerplate for creating a Llama 3 chat app

Neum AI is a best-in-class framework to manage the creation and synchronization of vector embeddings at large scale.

Council is an open-source platform for the rapid development and robust deployment of customized generative AI applications

[AAAI 2024 Oral] AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models