XLabs AI - ControlNets, IPAdapter & LoRAs
Added 2024-08-23 13:54:13 +0000 UTCVideo - https://youtu.be/g8W3xe5kRBQ
A whole different set of nodes for Flux.1 are available from XLabs to give a new sampler along with a whole bunch of different models for ControlNets, LoRAs and an initial ipadapter.
Canny & HED models allow you to control your generations using image outlines, whilst their depth model will work with depth maps. Use your own, or pre-process standard images as required.
It's still early days, but their IPAdapter option is another way to easily change the style of your images without needing to train a style LoRA. Simply provide a style image and prompt away as usual!
IPAdapter isn't just about styles though, as it also "sees" the content / composition too, giving all sorts of options when prompting.
Links
https://github.com/XLabs-AI/x-flux-comfyui
https://github.com/chrisgoringe/cg-use-everywhere
https://github.com/city96/ComfyUI_ExtraModels
Low end systems, can use https://github.com/city96/ComfyUI-GGUF with the sampler, but it doesn't look like controlnet or ipadapter works well with those models as yet.
Each of the attached workflows also has links for the various custom nodes in the notes, all of which I installed using ComfyUI Manager.
Comments
That fixed it! Thanks for the timely reply, you are the best Dr R
Alpha Romeo
2024-09-15 18:40:16 +0000 UTCAt a guess, you could be trying to use a different vae maybe?
Nerdy Rodent
2024-09-15 11:40:28 +0000 UTCI must have done something wrong haha: "Given groups=1, weight of size [4, 4, 1, 1], expected input[1, 16, 128, 128] to have 4 channels, but got 16 channels instead"
Alpha Romeo
2024-09-15 03:20:12 +0000 UTCIdentical speeds for me
Nerdy Rodent
2024-08-24 11:51:25 +0000 UTCdid you fiind significant slowdown with the XLabs Sampler? I'm seeing the xlabs sampler ticking at 116s/it where Ksampler was running a 3.2s/it.
cosmogonaut
2024-08-24 05:07:58 +0000 UTC
