JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Paper
• 2310.17631 • Published
• 35
LLM as a Judge
Curated resources that support the use of LLMs to serve as automatic evaluators of other LLM outputs.
-
-
Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
Paper • 2310.08491 • Published • 57 -
Generative Judge for Evaluating Alignment
Paper • 2310.05470 • Published • 1 -
Calibrating LLM-Based Evaluator
Paper • 2309.13308 • Published • 12 -
Judging LLM-as-a-judge with MT-Bench and Chatbot Arena
Paper • 2306.05685 • Published • 40 -
BAAI/JudgeLM-33B-v1.0
Text Generation • Updated • 84 • 27 -
BAAI/JudgeLM-13B-v1.0
Text Generation • Updated • 159 • 9 -
BAAI/JudgeLM-7B-v1.0
Text Generation • Updated • 146 • 18 -
prometheus-eval/prometheus-13b-v1.0
Text Generation • Updated • 106 • 143 -
prometheus-eval/prometheus-7b-v1.0
Text Generation • Updated • 33 • 31 -
Benchmarking Cognitive Biases in Large Language Models as Evaluators
Paper • 2309.17012 • Published • 3 -
Evaluating Large Language Models: A Comprehensive Survey
Paper • 2310.19736 • Published • 2 -
LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models
Paper • 2305.13711 • Published • 2 -
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
Paper • 2303.16634 • Published • 3 -
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
Paper • 2405.01535 • Published • 124 -
flowaicom/Flow-Judge-v0.1
Text Generation • 4B • Updated • 568 • 71 -
opencompass/CompassJudger-1-32B-Instruct
Text Generation • 33B • Updated • 49 • 18 -
JudgeBench: A Benchmark for Evaluating LLM-based Judges
Paper • 2410.12784 • Published • 47 -
Judge Arena
💻109View and compare open‑source AI model rankings with ELO scores
-
Agent-as-a-Judge: Evaluate Agents with Agents
Paper • 2410.10934 • Published • 23