Instructions to use thibaud/controlnet-sd21 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use thibaud/controlnet-sd21 with Diffusers:
pip install -U diffusers transformers accelerate
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline controlnet = ControlNetModel.from_pretrained("thibaud/controlnet-sd21") pipe = StableDiffusionControlNetPipeline.from_pretrained( "fill-in-base-model", controlnet=controlnet ) - Notebooks
- Google Colab
- Kaggle
"No diffusion_pytorch_model.bin"
Is there any way to run this with a fine-tuned SD-2.1 model?
I tried the following the example here (https://huggingface.co/blog/controlnet) and use this code:
controlNet = ControlNetModel.from_pretrained(
"thibaud/controlnet-canny-sd21",
torch_dtype=torch.float16
)
pipe = StableDiffusionControlNetPipeline.from_pretrained(
"my-model-dir",
controlnet=controlNet,
torch_dtype=torch.float16
).to("cuda")
but it gives me
OSError: thibaud/controlnet-canny-sd21 does not appear to have a file named diffusion_pytorch_model.bin.
What's the best way to download this? I thought from_pretrained did it automatically..?
I really don't know. The models works great with controlnet extension for Automatic1111.
If you find a way, let us know.
Yeah, the model is not yet ready for diffusers format.
We are trying to convert it, you can follow progress over here --> https://github.com/huggingface/diffusers/issues/2581
Thanks for the the link!
If a solution is found, let me know in case I didn’t see it!