Top Open Source AI Organization Influencers in 2024Q3

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

Official inference framework for 1-bit LLMs

A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations

The TypeScript AI agent framework. ⚡ Assistants, RAG, observability. Supports any LLM: GPT-4, Claude, Gemini, Llama.

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

PDF to Markdown with vision models

an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM

Let AI be your browser operator.

🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)

Efficient Triton Kernels for LLM Training

Multilingual Voice Understanding Model

Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

Open-source Next.js template for building apps that are fully generated by AI. By E2B.

TensorZero creates a feedback loop for optimizing LLM applications — turning production data into smarter, faster, and cheaper models.

AI Agent Builder in Python

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

In-browser Postgres sandbox with AI assistance (formerly postgres.new)

Conquer Any Code in VSCode: One-Click Comments, Conversions, UI-to-Code, and AI Batch Processing of Files! 在 VSCode 中征服任何代码：一键注释、转换、UI 图生成代码、AI 批量处理文件！💪

Build multimodal language agents for fast prototype and production

Local realtime voice AI

✨✨VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.

A system for agentic LLM-powered data processing and ETL

AI-native (edge and LLM) proxy for agents. Engineered with fast ⚡️ LLMs for task (query) routing, rich observability, and the seamless integration of prompts with your APIs for agentic tasks. Built by the contributors of Envoy proxy.

Kodu is an autonomous coding agent that lives in your IDE. It is a VSCode extension that can help you build your dream project step by step by leveraging the latest technologies in automated coding agents

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.

The goal of RamaLama is to make working with AI boring.

Chat with any codebase in under two minutes | Fully local or via third-party APIs

LOTUS: A semantic query engine for fast and easy LLM-powered data processing

Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.

ROSA 🤖 is an AI Agent designed to interact with ROS1- and ROS2-based robotics systems using natural language queries. ROSA helps robot developers inspect, diagnose, understand, and operate robots.

BaseAI — The Web AI Framework. The easiest way to build serverless autonomous AI agents with memory. Start building local-first, agentic pipes, tools, and memory. Deploy serverless with one command.

E2M converts various file types (doc, docx, epub, html, htm, url, pdf, ppt, pptx, mp3, m4a) into Markdown. It’s easy to install, with dedicated parsers and converters, supporting custom configs. E2M offers an all-in-one, flexible, and open-source solution.

Inference code for the paper "Spirit-LM Interleaved Spoken and Written Language Model".

Use LLMs in Excel formulas

Dynamiq is an orchestration framework for agentic AI and LLM applications

Hexabot is an open-source AI chatbot / agent builder. It allows you to create and manage multi-channel and multilingual chatbots / agents with ease.

HumanLayer enables AI agents to communicate with humans in tool-based and async workflows. Guarantee human oversight of high-stakes function calls with approval workflows across slack, email and more. Bring your LLM and Framework of choice and start giving your AI agents safe access to the world. Agentic Workflows, human in the loop, tool calling

Prompt chains maximize intelligence and results when using LLMs

A conversational, AI device + software framework for companionship, entertainment, education, healthcare, IoT applications, and DIY robotics. Built with Python, NextJS, Arduino, ESP32, LLMs (GPT-4o), Deepgram STT and Azure TTS 🤖

AI-powered assistant to help you with your daily tasks, powered by Llama 3, DeepSeek R1, and many more models on HuggingFace.

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

LLM2CLIP makes SOTA pretrained CLIP model more SOTA ever.

An example of multi-agent orchestration with llama-index

The fastest way to bring multi-agent workflows to production.

Claude Memory: Long-term memory for Claude

A simple example implementation of the VoiceRAG pattern to power interactive voice generative AI experiences using RAG with Azure AI Search and Azure OpenAI's gpt-4o-realtime-preview model.

High-quality and streaming Speech-to-Speech interactive agent in a single file. 只用一个文件实现的流式全双工语音交互原型智能体！

Call LLM APIs from R

Get up and running with the Gemini API in under 5 minutes (with Python)

An awesome repository & A comprehensive survey on interpretability of LLM attention heads.

Deploy agentic reasoning in a scalable and reliable platform in minutes. Become an on demand subject matter expert by loading portable cognitive cores for the most complex knowledge work. 🧠

End-to-End Local-First Text-to-SQL Pipelines

Chat with your data, modify it, visualize it, create and test machine learning models all in plain English. DataHorse makes data analysis and data science conversational using LLMs.

开箱即用的基于企业私有知识库的LLM大语言模型的智能客服机器人问答系统，支持私有化部署，代码免费开源且可商用，由芝麻小客服官方推出。

APO is a one-stop observability platform combining OpenTelemetry with eBPF. Leveraging LLM capabilities to enable auto-pilot analyzing and troubleshooting 🚀.

Conversational voice AI agents

LlamaVoice is a llama-based large voice generation model, providing inference and training ability.

SwarmZero's SDK for building AI agents, swarms of agents and much more.

The Dawn of Video Generation: Preliminary Explorations with SORA-like Models

Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"

Easily generate multilingual translations for your project with a single command, powered by Azure AI Services.

TapeAgents is a framework that facilitates all stages of the LLM Agent development lifecycle

参考GraphRag使用 Semantic Kernel 来实现的dotnet版本，可以使用NuGet开箱即用集成到项目中

MooER: Moore-threads Open Omni model for speech-to-speech intERaction. MooER-omni includes a series of end-to-end speech interaction models along with training and inference code, covering but not limited to end-to-end speech interaction, end-to-end speech translation and speech recognition.

AUITestAgent is the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification.

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

The repository of TrafficLLM, a universal LLM adaptation framework to learn robust traffic representation for all open-sourced LLM in real-world scenarios and enhance the generalization across diverse traffic analysis tasks.

Context-aware structured outputs. Search your documents or the web for specific data and get it back in JSON or Markdown.

Gateway API Inference Extension

Python SDK for Llama Stack

DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception

A survey on harmful fine-tuning attack for large language model

Higher performance OpenAI LLM service than vLLM serve: A pure C++ high-performance OpenAI LLM service implemented with GPRS+TensorRT-LLM+Tokenizers.cpp, supporting chat and function call, AI agents, distributed multi-GPU inference, multimodal capabilities, and a Gradio chat interface.

Efficiently find the best-suited language model (LM) for your NLP task

LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuning

Effective LLM Alignment Toolkit

LLM-powered AI guide that leads and drives intelligent conversations

Knowledge Base Builder，是一款基于LLM大语言模型的开源知识库文本处理系统，是「滨电智言」的一款开源工具，旨在成为企业的知识库构建中枢。

A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.

Build multimodal data processing pipelines with Azure AI Services + LLMs

Finetune Llama-3-8b on the MathInstruct dataset

Burn through tech debt with AI agents!

FlockMTL: DuckDB extension to seamlessly combine analytics and semantic analysis using language models (LMs)

Framework for testing vulnerabilities of large language models (LLM).

探索 LLM 在法律行业的应用潜力

A proxy sidecar to access Gemini models via OpenAI and Ollama APIs

Typescript library for Llama Stack

LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale

Paper collections of multi-modal LLM for Math/STEM/Code.

multispy is a lsp client library in Python intended to be used to build applications around language servers.

An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT

multi-agents banking assistant with Java and Semantic Kernel

A framework offers an OS simulator within a Python Code Interface for AI Agents