Stable WarpFusion v0.28 - IP Adapters

Local install guides:
Windows: https://github.com/Sxela/WarpFusion#local-installation-guide-for-windows-venv
Linux: https://github.com/Sxela/WarpFusion/blob/main/README.md#local-installation-guide-for-linux-ubuntu-2204-venv

If something's not working, make a new env and grab a fresh install.bat from that repo first, ask questions later :D

This notebook needs an updated transformers 4.31.0 package. Make a new env with a fresh install.bat or update the package by running !pip install transformers==4.31.0 in a new code cell.

Changelog:

• add ipadapter support for sd15/sdxl modes from https://github.com/Mikubill/sd-webui-controlnet
• add ipadapter clip vit-h/-g preprocessors
• add ipadapter sd15/sdxl model urls to autodownloader
• add clip vit-h/-g model urls to autodownloader
• add ipadapters to controlnet gui
• add clip vision embedding cache for ipadapters
• add vae encoder cache
• remove unnecessary vae runs for inactive guidance functions
• removed unused gui options for ipadapters
• fix reappearing unused options for ipadapters, inpaint, and other controlnets that do not have preprocessors or resolution setting
• increase source textbox width for ipadapters

v0.28.2, 2.12.2023:

• add ipadapter for animatediff
• add v1 motion module support for animatediff
• fix some loras not working with animatediff thanks to #runshouse

v0.28.3, 3.12.2023:

• fix ldm.modules.encoders.modules error for colab thanks to #runshouse

v0.28.4, 5.12.2023:

• add safetensors support for aniimatediff motion modules

23.12.2023:

• move to L tier

12.1.2024, v0.28.5:

• fix dwpose annotator weights access denied error

18.3.2024:

• move to M tier

IP Adapters

Added numerous versions of IP Adapters for sd 1.5 and sdxl models.
They are available in the GUI -> Controlnet tab.

You should provide source image for them otherwise previously stylized frames will be used.

(GUI->Controlnet-> Source on the right side of a given adapter)

I haven't tested multiple adapters at the same time. For better and more consistent results I suggest picking source image with content similar to your video if you want to keep the content recognizable (i.e. a stylized human source if you want to keep human in the output)

You can provide different source images for each frame, if you specify a video of a folder with images as the source. Mind the file naming convention to have numbers with leading zeros to be sure you get source-frame pairs sorted correctly.

IP Adapter embeddings are cached, so if you only have 1 source image, it will be only preprocessed once per video.
IP Adapters and their CLIP Vision model counterparts are downloaded automatically and placed into your ControlNet folder.

Local install guide:
https://github.com/Sxela/WarpFusion/blob/main/README.md

Guides made by users:

• 05.05.2023, v0.10 Video to AI Animation Tutorial For Beginners: Stable WarpFusion + Controlnet | MDMZ
• 11.05.2023, v0.11 How to use Stable Warp Fusion
• 13.05.2023, v0.8 Warp Fusion Local Install Guide (v0.8.6) with Diffusion Demonstration
• 14.05.2023, v0.12 Warp Fusion Alpha Masking Tutorial | Covers Both Auto-Masking and Custom Masking
• 23.05.2023, v0.12 STABLE WARPFUSION TUTORIAL - Colab Pro & Local Install
• 15.06.2023, v0.13 AI Animation out of Your Video: Stable Warpfusion Guide (Google Colab & Local Intallation)
• 17.06.2023, v0.14 Stable Warpfusion Tutorial: Turn Your Video to an AI Animation
• 21.06.2023, v0.14 Avoiding Common Problems with Stable Warpfusion
• 21.06.2023, v0.15 Warp Fusion: Step by Step Tutorial
• 04.07.2023, v0.15 Intense AI Video Maker (Stable WarpFusion Tutorial)
• 15.08.2023, v0.17 BEST Laptop for AI ( SDXL & Stable Warpfusion ) ft. RTX 4090 - Make AI Art FREE and FAST!
• 25.08.2023, ComfyWarp v0.1 WarpFusion: Warp and Consistency explanation in ComfyUI
• 2.09.2023, ComfyWarp v0.2 WarpFusion: ComfyWarp iteration 2.
• 3.09.2023, v0.16+ WarpFusion - Multiple Masked Prompts Guide
• 10.10.2023, ComfyWarp v0.4 WarpFusion: ComfyWarp v0.4.2 (schedulers, flow_blend)
• 4.11.2023, ComfyWarp v0.5 WarpFusion: ComfyWarp v0.5 - FixedQueue
• 13.11.2023, v0.27 Stable WarpFusion v0.27 - Changelog

YouTube playlist with settings:
https://www.youtube.com/watch?v=wvvcWm4Snmc&list=PL2cEnissQhlCUgjnGrdvYMwUaDkGemLGq

For tech support and other questions please join our discord server:
https://discord.gg/58Rd73uAe7

Discord is the preferred method because it is nearly impossible to provide any decent help or tech support via Patreon due to its limited text formatting and inability to add screenshots or videos to comments or DMs.
Due to the recent Patreon comments update it's impossible to reply to comments from notifications anymore, so if your comment hasn't been replied to for a while, DM me.