Nexa-cli failure with model
% nexa infer NexaAI/qwen3vl-30B-A3B-mlx
Hello
Hello! How can I help you today?
— 2.8 tok/s • 9 tok • 3.1 s first token -
What can you do?
Traceback (most recent call last):
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/interface.py", line 308, in generate_stream
for result in stream_generate_impl(
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/generate_qwen3_vl_moe.py", line 202, in stream_generate_qwen3_vl
messages = json.loads(prompt)
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/python_runtime/lib/python3.10/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/python_runtime/lib/python3.10/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/python_runtime/lib/python3.10/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Error: SDKError(Unknown error)
Follow up: % nexa infer NexaAI/qwen3vl-30B-A3B-mlx
/?
Commands:
/?, /h, /help Show this help message
/exit Exit the REPL
/clear Clear the screen and conversation history
/load Load conversation history from a file
/save Save conversation history to a file
/mic Record audio for transcription
What do you see in this picture: /Users/javi/ComfyUI/input/2020-09-27_photopoem1.jpg
Traceback (most recent call last):
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/interface.py", line 308, in generate_stream
for result in stream_generate_impl(
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/generate_qwen3_vl_moe.py", line 241, in stream_generate_qwen3_vl
for token, logprobs in nexa_generate_step(
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/modeling/models/qwen3_vl/llm_common/generate.py", line 382, in nexa_generate_step
next_y, next_logprobs = _step(y)
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/modeling/models/qwen3_vl/llm_common/generate.py", line 346, in _step
logits = _model_call(y[None])
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/modeling/models/qwen3_vl/llm_common/generate.py", line 340, in _model_call
return model(y, cache=prompt_cache, rope_deltas=rope_deltas)
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/modeling/models/qwen3vl_moe/qwen3vl_moe.py", line 986, in call
out = self.language_model(
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/modeling/models/qwen3vl_moe/qwen3vl_moe.py", line 801, in call
hidden_states = decoder_layer(
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/modeling/models/qwen3vl_moe/qwen3vl_moe.py", line 728, in call
hidden_states, _ = self.self_attn(
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/py-lib/vlm/modeling/models/qwen3vl_moe/qwen3vl_moe.py", line 613, in call
queries = self.rope(queries, offset=cache.offset+rope_deltas)
File "/Applications/NexaCLI.app/Contents/Resources/nexa_mlx/python_runtime/lib/python3.10/site-packages/mlx/nn/layers/positional_encoding.py", line 47, in call
return mx.fast.rope(
ValueError: [rope] offset must have at most one dimension but has shape (1,1).
Error: SDKError(Unknown error)