Discover The Top AI hackers & Projects

31155 AI hackers and 36904 AI repos(projects) in the AI listing. Data updated from github daily.

Trending New LLM Diffusion GPT RAG Multi-modality DeepSeek

Multi-modality Project

om-ai-lab

VLM-R1

Solve Visual Understanding with Reinforced VLMs

NexaAI

United States of America

nexa-sdk

Nexa SDK is a comprehensive toolkit for supporting GGML and ONNX models. It supports text generation, image generation, vision-language models (VLM), Audio Language Model, auto-speech-recognition (ASR), and text-to-speech (TTS) capabilities.

bytedance

UI-TARS-desktop

A GUI Agent application based on UI-TARS(Vision-Lanuage Model) that allows you to control your computer using natural language.

manycore-research

Hangzhou, China

SpatialLM

SpatialLM: Large Language Model for Spatial Understanding

QiuYannnn

Local-File-Organizer

An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.

cambrian-mllm

cambrian

Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

jingyaogong

HangZhou, China

minimind-v

🚀 「大模型」1小时从0训练26M参数的视觉多模态VLM！🌏 Train a 26M-parameter VLM from scratch in just 1 hours!

illuin-tech

colpali

The code used to train and run inference with the ColPali architecture.

SkyworkAI

Skywork-R1V

Pioneering Multimodal Reasoning with CoT

SkalskiP

vlms-zero-to-hero

This series will take you on a journey from the fundamentals of NLP and Computer Vision to the cutting edge of Vision-Language Models.

zubair-irshad

Silicon Valley, CA, USA

Awesome-Robotics-3D

A curated list of 3D Vision papers relating to Robotics domain in the era of large models i.e. LLMs/VLMs, inspired by awesome-computer-vision, including papers, codes, and related websites

OpenBMB

VisRAG

Parsing-free RAG supported by VLMs

AIDC-AI

Ovis

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

NVlabs

EAGLE

Eagle Family: Exploring Model Designs, Data Recipes and Training Strategies for Frontier-Class Multimodal LLMs

yueliu1999

Awesome-Jailbreak-on-LLMs

Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, exciting jailbreak methods on LLMs. It contains papers, codes, datasets, evaluations, and analyses.

nrl-ai

llama-assistant

AI-powered assistant to help you with your daily tasks, powered by Llama 3, DeepSeek R1, and many more models on HuggingFace.

vlm-run

United States of America

vlmrun-hub

A hub for various industry-specific schemas to be used with VLMs.

awwaiid

ghostwriter

Use the reMarkable2 as an interface to vision-LLMs (ChatGPT, Claude, Gemini). Ghost in the machine!

Flame-Code-VLM

Flame-Code-VLM

Flame is an open-source multimodal AI system designed to translate UI design mockups into high-quality React code. It leverages vision-language modeling, automated data synthesis, and structured training workflows to bridge the gap between design and front-end development.

fpgaminer

joycaption

JoyCaption is an image captioning Visual Language Model (VLM) being built from the ground up as a free, open, and uncensored model for the community to use in training Diffusion models.

baaivision

EVE

EVE Series: Encoder-Free Vision-Language Models from BAAI

Aident-AI

United States of America

open-cuak

Reliable Automation Agents at Scale

zjysteven

lmms-finetune

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

mbzuai-oryx

VideoGPT-plus

Official Repository of paper VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding

CircleRadon

TokenPacker

The code for "TokenPacker: Efficient Visual Projector for Multimodal LLM".

bz-lab

AUITestAgent

AUITestAgent is the first automatic, natural language-driven GUI testing tool for mobile apps, capable of fully automating the entire process of GUI interaction and function verification.

IDEA-Research

ChatRex

Code for ChatRex: Taming Multimodal LLM for Joint Perception and Understanding

TIGER-AI-Lab

VLM2Vec

This repo contains the code for "VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks" [ICLR25]

lucasjinreal

Namo-R1

A CPU Realtime VLM in 500M. Surpassed Moondream2 and SmolVLM. Training from scratch with ease.

2U1

Llama3.2-Vision-Finetune

An open-source implementaion for fine-tuning Llama3.2-Vision series by Meta.

thu-ml

FIT Building, Tsinghua University, Beijing, China

MMTrustEval

A toolbox for benchmarking trustworthiness of multimodal large language models (MultiTrust, NeurIPS 2024 Track Datasets and Benchmarks)

aimagelab

LLaVA-MORE

LLaVA-MORE: A Comparative Study of LLMs and Visual Backbones for Enhanced Visual Instruction Tuning

balrog-ai

BALROG

Benchmarking Agentic LLM and VLM Reasoning On Games

microsoft

88.6K followers

eureka-ml-insights

A framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings.

Ravi-Teja-konda

Surveillance_Video_Summarizer

VLM driven tool that processes surveillance videos, extracts frames, and generates insightful annotations using a fine-tuned Florence-2 Vision-Language Model. Includes a Gradio-based interface for querying and analyzing video footage.

FennelFetish

qapyq

An image viewer and AI-assisted editing/captioning/masking tool that helps with curating datasets for generative AI models, finetunes and LoRA.

MDGrey33

pyvisionai

The PyVisionAI Official Repo

shikiw

Modality-Integration-Rate

The official code of the paper "Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate".

Yxxxb

VoCo-LLaMA

VoCo-LLaMA: This repo is the official implementation of "VoCo-LLaMA: Towards Vision Compression with Large Language Models".

WooQi57

Helpful-Doggybot

Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Models

fangyuan-ksgk

Mini-LLaVA

A minimal implementation of LLaVA-style VLM with interleaved image & text & video processing ability.

OpenRobotLab

VLM-Grounder

[CoRL 2024] VLM-Grounder: A VLM Agent for Zero-Shot 3D Visual Grounding

WisconsinAIVision

YoLLaVA

🌋👵🏻 Yo'LLaVA: Your Personalized Language and Vision Assistant

TrustGen

United States of America

TrustEval-toolkit

TrustEval: A modular and extensible toolkit for comprehensive trust evaluation of generative foundation models (GenFMs)

Gumpest

SparseVLMs

Official implementation of paper "SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference".