Top Open Source AI Organization Influencers in 2024Q3

Official inference framework for 1-bit LLMs

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view

PDF to Markdown with vision models

an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM

Let AI be your browser operator.

🔍 An LLM-based Multi-agent Framework of Web Search Engine (like Perplexity.ai Pro and SearchGPT)

Efficient Triton Kernels for LLM Training

Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

Multilingual Voice Understanding Model

Open-source Next.js template for building apps that are fully generated by AI. By E2B.

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

In-browser Postgres sandbox with AI assistance (formerly postgres.new)

TensorZero creates a feedback loop for optimizing LLM applications — turning production data into smarter, faster, and cheaper models.

Conquer Any Code in VSCode: One-Click Comments, Conversions, UI-to-Code, and AI Batch Processing of Files! 在 VSCode 中征服任何代码：一键注释、转换、UI 图生成代码、AI 批量处理文件！💪

Local realtime voice AI

Build multimodal language agents for fast prototype and production

A system for agentic LLM-powered data processing and ETL

Laminar - open-source all-in-one platform for engineering AI products. Crate data flywheel for you AI app. Traces, Evals, Datasets, Labels. YC S24.

AI-native (edge and LLM) proxy for agents. Engineered with fast ⚡️ LLMs for task (query) routing, rich observability, and the seamless integration of prompts with your APIs for agentic tasks. Built by the contributors of Envoy proxy.

The TypeScript AI framework.

Graph UI for AI Agents in Python

Kodu is an autonomous coding agent that lives in your IDE. It is a VSCode extension that can help you build your dream project step by step by leveraging the latest technologies in automated coding agents

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.

Chat with any codebase in under two minutes | Fully local or via third-party APIs

LOTUS: A semantic query engine for fast and easy LLM-powered data processing

Streamlines and simplifies prompt design for both developers and non-technical users with a low code approach.

ROSA 🤖 is an AI Agent designed to interact with ROS1- and ROS2-based robotics systems using natural language queries. ROSA helps robot developers inspect, diagnose, understand, and operate robots.

E2M converts various file types (doc, docx, epub, html, htm, url, pdf, ppt, pptx, mp3, m4a) into Markdown. It’s easy to install, with dedicated parsers and converters, supporting custom configs. E2M offers an all-in-one, flexible, and open-source solution.

BaseAI — The Web AI Framework. The easiest way to build serverless autonomous AI agents with memory. Start building local-first, agentic pipes, tools, and memory. Deploy serverless with one command.

Inference code for the paper "Spirit-LM Interleaved Spoken and Written Language Model".

Dynamiq is an orchestration framework for agentic AI and LLM applications

Hexabot is an open-source AI chatbot / agent builder. It allows you to create and manage multi-channel and multilingual chatbots / agents with ease.

HumanLayer enables AI agents to communicate with humans in tool-based and async workflows. Guarantee human oversight of high-stakes function calls with approval workflows across slack, email and more. Bring your LLM and Framework of choice and start giving your AI agents safe access to the world. Agentic Workflows, human in the loop, tool calling

Prompt chains maximize intelligence and results when using LLMs

An open source voice-enabled, compact, empathic AI hardware + software 🤖 framework for companionship, entertainment, education, healthcare, IoT applications, AI-enhanced robotics application services, and DIY robotics. Built with Python, NextJS, Arduino, ESP32, LLMs (GPT-4o), STT, TTS, AI agent.

AI-powered assistant to help you with your daily tasks, powered by Llama 3.2. It can recognize your voice, process natural language, and perform various actions based on your commands: summarizing text, rephasing sentences, answering questions, writing emails, and more.

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

LLM2CLIP makes SOTA pretrained CLIP model more SOTA ever.

An example of multi-agent orchestration with llama-index

The fastest way to bring multi-agent workflows to production.

Claude Memory: Long-term memory for Claude

A simple example implementation of the VoiceRAG pattern to power interactive voice generative AI experiences using RAG with Azure AI Search and Azure OpenAI's gpt-4o-realtime-preview model.

Call LLM APIs from R

High-quality and streaming Speech-to-Speech interactive agent in a single file. 只用一个文件实现的流式全双工语音交互原型智能体！

Get up and running with the Gemini API in under 5 minutes (with Python)

An awesome repository & A comprehensive survey on interpretability of LLM attention heads.

Deploy agentic reasoning in a scalable and reliable platform in minutes. Become an on demand subject matter expert by loading portable cognitive cores for the most complex knowledge work. 🧠

End-to-End Local-First Text-to-SQL Pipelines

Chat with your data, modify it, visualize it, create and test machine learning models all in plain English. DataHorse makes data analysis and data science conversational using LLMs.

开箱即用的基于企业私有知识库的LLM大语言模型的智能客服机器人问答系统，支持私有化部署，代码免费开源且可商用，由芝麻小客服官方推出。

APO is a one-stop observability platform combining OpenTelemetry with eBPF. Leveraging LLM capabilities to enable auto-pilot analyzing and troubleshooting 🚀.

LlamaVoice is a llama-based large voice generation model, providing inference and training ability.

SwarmZero's SDK for building AI agents, swarms of agents and much more.

Offical Repo for "Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale"

Easily generate multilingual translations for your project with a single command, powered by Azure AI Services.

The Dawn of Video Generation: Preliminary Explorations with SORA-like Models

TapeAgents is a framework that facilitates all stages of the LLM Agent development lifecycle

参考GraphRag使用 Semantic Kernel 来实现的dotnet版本，可以使用NuGet开箱即用集成到项目中

MooER: Moore-threads Open Omni model for speech-to-speech intERaction. MooER-omni includes a series of end-to-end speech interaction models along with training and inference code, covering but not limited to end-to-end speech interaction, end-to-end speech translation and speech recognition.

AUITestAgent is the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification.

Conversational voice AI agents

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

The repository of TrafficLLM, a universal LLM adaptation framework to learn robust traffic representation for all open-sourced LLM in real-world scenarios and enhance the generalization across diverse traffic analysis tasks.

Context-aware structured outputs. Search your documents or the web for specific data and get it back in JSON or Markdown.

Gateway API Inference Extension

A survey on harmful fine-tuning attack for large language model

DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception

Python SDK for Llama Stack

Efficiently find the best-suited language model (LM) for your NLP task

Effective LLM Alignment Toolkit

LLM-powered AI guide that leads and drives intelligent conversations

Knowledge Base Builder，是一款基于LLM大语言模型的开源知识库文本处理系统，是「滨电智言」的一款开源工具，旨在成为企业的知识库构建中枢。

A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.

Build multimodal data processing pipelines with Azure AI Services + LLMs

Finetune Llama-3-8b on the MathInstruct dataset

LLaVA-MORE: Enhancing Visual Instruction Tuning with LLaMA 3.1

FlockMTL: DuckDB extension to seamlessly combine analytics and semantic analysis using language models (LMs)

【高性能OpenAI LLM服务】通过GPRS+TensorRT-LLM+Tokenizers.cpp实现纯C++版高性能OpenAI LLM服务，支持chat和function call模式，支持ai agent，支持分布式多卡推理，支持多模态，支持gradio聊天界面。

探索 LLM 在法律行业的应用潜力

A proxy sidecar to access Gemini models via OpenAI and Ollama APIs

LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale

Paper collections of multi-modal LLM for Math/STEM/Code.

multispy is a lsp client library in Python intended to be used to build applications around language servers.

An Efficient LLM Fine-Tuning Factory Optimized for MoE PEFT

A framework offers an OS simulator within a Python Code Interface for AI Agents

Code for evaluating with Flow-Judge-v0.1 - an open-source, lightweight (3.8B) language model optimized for LLM system evaluations. Crafted for accuracy, speed, and customization.

Dabarqus is incredibly fast RAG that runs everywhere.

A modern learning management system to place learning in the hands of the students

[ECCV 2024] - Improving Zero-shot Generalization of Learned Prompts via Unsupervised Knowledge Distillation

A demonstration of chatting with uploaded images using OpenAI vision models like gpt-4o.

The Library for LLM-based web-agent applications