CompBench
CompBench evaluates the comparative reasoning of multimodal large language models (MLLMs) with 40K image pairs and questions across 8 dimensions of relative comparison: visual attribute, existence, state, emotion, temporality, spatiality, quantity, and quality. CompBench covers diverse visual domains, including animals, fashion, sports, and scenes.
benchmarkevaluation-llmsfoundation-modelshuman-annotationlarge-language-modelsllmsllms-benchmarkingmultimodal-deep-learningmultimodal-large-language-modelsreasoningvision-and-languagevision-language-model
Github Data
Star 31
Fork 1
Subscriber 1
Language Jupyter Notebook
Size 11.2K
owner
created at 2024-07-23
updated at 2024-10-23
pushed at 2024-08-06
