benchmark results

#1
by eramax - opened

Please checkout again your benchmark results, I see most of the results are less than the base.

Best

rrchat org

Thanks for checking, you're right: in several benchmarks Mercury-Acad is below the base model.
It's an experimental model (teaching the base to reason), not a general upgrade. I've updated the
card with a side-by-side table (base vs. Mercury-Acad, same settings) and a note on where it
helps and where it doesn't. If you see something specific that looks off, let me know.
By the way, Mercury-1.7B beats its base on our benchmarks (Mercury-Acad is a separate, experimental model).

Sign up or log in to comment