can this run with vLLM ?

by Adenialzz - opened Jan 11

Jan 11

Hi, thanks for open-sourcing. Can this run with vLLM ?

taobao-mnn org Jan 12

I have only tested it on SGLang. However, since this is a standard EAGLE-3 format safetensors weight, it should work on vLLM.

zhaode changed discussion status to closed Jan 12

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment