inferflow
Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).
baichuan2bloomdeepseekfalcongemmainternlmllama2llamacppllm-inferencem2m100minicpmmistralmixtralmixture-of-expertsmodel-quantizationmoemulti-gpu-inferencephi-2qwen
Github Data
Star 236
Fork 24
Subscriber 8
Language C++
Size 2.0K
owner
created at 2023-12-26
updated at 2024-10-31
pushed at 2024-03-15
