Issues running your model in LM Studio
Hello.
I couldn’t use the model; no matter what settings I choose, it only outputs fragments of sentences, code, and Chinese characters.
I’m using LM Studio on a MacBook M1 16 GB Pro.
Could you please advise how I can launch and use your model correctly?
Thank you.
Hello.
I couldn’t use the model; no matter what settings I choose, it only outputs fragments of sentences, code, and Chinese characters.
I’m using LM Studio on a MacBook M1 16 GB Pro.
Could you please advise how I can launch and use your model correctly?
Thank you.
Thank you for your feedback and for trying out our model.
While the model architecture may appear similar to Llama, the Stable-DiffCoder model family requires a specialized inference logic adapted for Diffusion Language Models. It cannot use standard autoregressive inference directly.
To run the model correctly, please ensure you are using the dedicated inference code we provide. You can find the core implementation here:
https://huggingface.co/ByteDance-Seed/Stable-DiffCoder-8B-Instruct/blob/main/modeling_seed_diffcoder.py
If you are using a quantized version of the model, you will need to adapt this specific inference logic into your quantization framework/tool. Using standard autoregressive sampling with a quantized DiffCoder model will lead to incorrect outputs (such as fragments, code snippets, or garbled text).
Could you please first confirm that your current setup is using the correct diffusion-based inference logic as linked above? If the issue persists after adapting the code, please provide more detailed information about your environment and the exact steps you are taking, and I will be glad to help you resolve it.
Best regards
Thank you. I am not programmer and only use a LM Studio. I will wait when model can be used in LM Studio )
Xie-xie dear Developers!