Instructions to use inclusionAI/SingVL_S1_4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use inclusionAI/SingVL_S1_4B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="inclusionAI/SingVL_S1_4B") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("inclusionAI/SingVL_S1_4B") model = AutoModelForMultimodalLM.from_pretrained("inclusionAI/SingVL_S1_4B", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use inclusionAI/SingVL_S1_4B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "inclusionAI/SingVL_S1_4B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inclusionAI/SingVL_S1_4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/inclusionAI/SingVL_S1_4B
- SGLang
How to use inclusionAI/SingVL_S1_4B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "inclusionAI/SingVL_S1_4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inclusionAI/SingVL_S1_4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "inclusionAI/SingVL_S1_4B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "inclusionAI/SingVL_S1_4B", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use inclusionAI/SingVL_S1_4B with Docker Model Runner:
docker model run hf.co/inclusionAI/SingVL_S1_4B
Sing VL S1
Sing VL S1 is a multimodal decision model built on Qwen3.5. It answers named questions over text or JSON and up to four images. Each question is encoded as a complete prompt. In text requests with multiple questions, the engine batches those prompts by length with a budget of 65,536 padded tokens per forward pass.
- Choice selects an option and returns a probability for each candidate.
- Noul returns the probability that a yes/no proposition is true.
- Score rates the input against 2โ10 levels ordered from low to high. It returns the probability of each level and the expected level, computed as the sum of each zero-based level index times its probability.
Public Decision Index 0.3
The 65,536-token batch engine scored 69.75 Text (140,178 requests) and 73.74949871316771 Vision (12,210 requests) on the complete public suites. The table and per-dataset figures below document the original serial submission.
| Panel | Sing VL S1 (original submission) | TypeSafe Jev 1.14 | StartLux-Decision v1.1 4B | Perplexity Decider Flash v1.2 (4B) |
|---|---|---|---|---|
| Text | 69.76 | 60.70 | 67.8 | 60.1 |
| Vision | 73.75 | โ | 67.90 | 72.5 |
The submitted Vision result uses the 1,638,400-pixel cap. The older 73.94 result used a larger image budget and is retained below as historical context.
Per-dataset results
Show the Vision and Text breakdown
Historical Public Vision (11 datasets; larger image budget)
The breakdown below belongs to the historical 73.94 run. The submitted 1.6 MP-capped run scores 73.7495; its per-benchmark values are in the pinned vision-scores.json.
| Dataset | Non-Skill (Accuracy) | Skill |
|---|---|---|
| CV-Bench | 87.34 | 79.12 |
| BLINK | 72.11 | 56.83 |
| RealWorldQA | 82.17 | 70.87 |
| CharXiv | 86.65 | 82.21 |
| InfographicVQA | 96.31 | 95.08 |
| Mind2Web | 87.67 | 83.56 |
| Winoground | 93.12 | 86.25 |
| KIE (CORD+FUNSD) | 95.82 | 93.12 |
| Moderation (Hateful Memes) | 71.64 | 43.28 |
| R-Bench-M | 49.77 | 39.72 |
| MMMU-Pro vision | 50.17 | 42.85 |
| Public Vision Index | โ | 73.94 |
Public Text (43 datasets)
| Dataset | Original metric | Non-Skill | Skill |
|---|---|---|---|
| BFCL | Case exact accuracy | 96.81 | 95.70 |
| ToolRet | nDCG@10 | 67.96 | 63.00 |
| API-Bank | Accuracy | 84.45 | 84.15 |
| BANKING77 | Macro-F1 | 94.00 | 93.92 |
| CLINC150+OOS | Macro-F1 | 97.03 | 97.01 |
| RouterBench โ | Selected quality | 79.83 | 52.71 |
| Home appliance simulator | Case exact accuracy | 100.00 | 100.00 |
| SGD/SGD-X โ | Macro-F1 | 8.33 | 0.00 |
| ContractNLI | Macro-F1 | 84.05 | 76.94 |
| ANLI | Macro-F1 | 78.29 | 67.48 |
| BPoMP * | Accuracy | 95.18 | 90.70 |
| Humicroedit | Accuracy | 67.16 | 34.32 |
| POP909-CL * | Accuracy | 80.60 | 79.74 |
| cfcolor * | Accuracy | 70.14 | 44.04 |
| MMLU โ | Accuracy | 82.93 | 77.24 |
| GPQA Diamond | Accuracy | 53.06 | 37.41 |
| ARC-Easy โ | Accuracy | 99.03 | 98.71 |
| ARC-Challenge โ | Accuracy | 96.25 | 95.00 |
| WinoGrande | Accuracy | 89.82 | 79.64 |
| HellaSwag | Accuracy | 96.48 | 95.31 |
| GSM8K | Accuracy | 73.84 | 68.68 |
| ChessBench | Accuracy | 26.88 | 20.36 |
| MuSR | Accuracy | 65.82 | 45.66 |
| SATA-Bench | Case exact accuracy | 53.88 | 53.27 |
| SimpleBench โ | Accuracy | 50.00 | 40.00 |
| BRIGHT | nDCG@10 | 47.51 | 40.62 |
| Amazon ESCI | Macro-F1 | 60.74 | 50.75 |
| ACOS | Per-review F1 | 64.56 | 63.43 |
| FinEntity | Macro-F1 | 95.19 | 92.93 |
| iSarcasmEval | Sarcasm F1 (A ยท English) | 64.98 | 54.95 |
| VAST | Macro-F1 | 79.69 | 69.53 |
| NLI4CT | Macro-F1 | 86.41 | 73.56 |
| CRUXEval | Accuracy | 78.07 | 65.21 |
| CLadder | Accuracy | 94.72 | 89.44 |
| HLE | Accuracy | 12.77 | 0.00 |
| Habermas Machine | Accuracy | 63.60 | 47.18 |
| PhishNChips | Accuracy | 85.55 | 71.10 |
| MMLU-Pro | Accuracy | 68.40 | 64.46 |
| BBH | Accuracy | 88.61 | 83.49 |
| RAGTruth | Hallucinated-class F1 | 83.49 | 65.77 |
| HoVer | Accuracy | 89.25 | 78.50 |
| When2Call MCQ | Accuracy | 90.61 | 87.48 |
| New Yorker caption matching | Accuracy | 80.49 | 75.61 |
| Public Text Index | Weighted aggregate | 77.26 | 69.76 |
โ Decision Index reports these datasets separately and excludes them from the Text Index.
Usage
Use Python 3.12 and hardware-compatible PyTorch and torchvision builds. From the model repository root, enter inference/ before running the examples. If you have a standalone copy of the inference code, run the examples from that directory and replace .. with the checkpoint path.
cd inference
python -m pip install -r requirements.txt
Python
from PIL import Image
from model import SingVLS1
model = SingVLS1("..", device="cuda:0")
# Text
text, _ = model.predict(
state="The color is red. ",
questions={
"color": {
"type": "choice",
"instructions": "Which color is named?",
"criteria": {"red": "Red", "blue": "Blue"},
},
"yesno": {"type": "noul", "instructions": "Is the named color red?"},
"redness": {
"type": "score",
"instructions": "How clearly does the text identify the color as red?",
"criteria": ["Not red", "Possibly red", "Clearly red"],
},
},
)
print(text["answers"])
# Images in blue-then-red order. Both state values work.
questions = {
"which": {
"type": "choice",
"instructions": "Which image is red?",
"criteria": {"first": "The first image", "second": "The second image"},
}
}
with Image.open("examples/blue.png") as blue, Image.open("examples/red.png") as red:
for state in ({}, "<image>\n<image>\n"):
vision, _ = model.predict(state=state, questions=questions, images=[blue, red])
print(vision["answers"])
With no <image> markers, the model inserts them before the state text. With explicit markers, use one per image. The markers match the images in list order.
Use the same score question format with images. The answer includes probabilities and a legend, both keyed by the zero-based level number as a string. For example, probabilities of 0.1, 0.3, and 0.6 on three levels give 0 ร 0.1 + 1 ร 0.3 + 2 ร 0.6 = 1.5.
HTTP API
Start the FastAPI server. Send requests to POST http://127.0.0.1:8173/v1/systemone from another terminal on the same machine. Interactive API docs are at http://127.0.0.1:8173/docs.
python serve.py --model .. --port 8173 --image-root ./examples
Send a text request with an inline JSON body.
curl -fsS http://127.0.0.1:8173/v1/systemone \
-H 'Content-Type: application/json' -d '{
"model": "sing-vl-s1",
"state": "The color is red. ",
"questions": {
"color": {
"type": "choice",
"instructions": "Which color is named?",
"criteria": {"red": "Red", "blue": "Blue"}
},
"yesno": {"type": "noul", "instructions": "Is the named color red?"},
"redness": {
"type": "score",
"instructions": "How clearly does the text identify the color as red?",
"criteria": ["Not red", "Possibly red", "Clearly red"]
}
}
}'
For a Vision request, add an ordered images array. Its entries can be Base64 data URLs (data_url), raw Base64 strings (base64), or server-side file paths (path). Paths are relative to --image-root or absolute paths inside that directory. With the server command above, send the example filenames directly.
curl -fsS http://127.0.0.1:8173/v1/systemone \
-H 'Content-Type: application/json' -d '{
"model": "sing-vl-s1",
"state": {},
"images": ["blue.png", "red.png"],
"questions": {
"which": {
"type": "choice",
"instructions": "Which image is red?",
"criteria": {"first": "The first image", "second": "The second image"}
}
}
}'
For the same request with explicit image markers, use "state": "<image>\n<image>\n".
The response puts results under answers using the supplied question IDs. The text example selects red and returns answers.redness.score on a 0โ2 scale. The Vision example selects second.
Decision Index compatibility and submission
Both the Public and private suites use inference.engine:D3Engine with the same model weights and the same default request-level batching. Download the model and DI adapter together.
python -m pip install "huggingface_hub>=0.34,<2"
hf download inclusionAI/SingVL_S1_4B --revision dd9fcbe2412068def01666b1d1a443e412f11214 --local-dir ./singvl-s1-4b
python -m pip install -r ./singvl-s1-4b/inference/requirements-di.txt
# With already-built and verified Public 0.3 suites:
python ./singvl-s1-4b/inference/run_decision_index.py \
--panel text --suite-dir ./suite-0.3 --out ./runs/singvl-65k-text
python ./singvl-s1-4b/inference/run_decision_index.py \
--panel vision --suite-dir ./work/vision --out ./runs/singvl-65k-vision
Use fresh output directories for the batch engine so that prior serial predictions are not resumed.
The launcher resolves its own code and checkpoint paths, so it can be called from any directory. The model argument is a downloaded local checkpoint path. The adapter applies the submitted prefix-bidirectional attention and decision readout.
See the complete reproduction guide for suite
preparation, direct DI commands, HTTP serving, capacity limits and exact revisions.
The original public results and code snapshot
are pinned to 90ec7e159da6fa21d762f1f75876ea79049d0f4a. For the batch release, see the
Public predictions and scores and
inference/release.json for hashes and provenance.
The original serial submission scored Text 69.76 and Vision 73.7495. Both releases use a 1,638,400-pixel image cap. Private Text/Vision evaluation and official single-RTX-PRO-6000 latency qualification remain maintainer-run.
- Downloads last month
- 1,588