FreeInit In AnimateDiff Setting Workflow
Added 2024-02-28 13:00:11 +0000 UTCVideo Tutorial : https://youtu.be/f3cchKVfPT8
We discuss the revolutionary FreeInit technology in video diffusion models. In this video, we dive deep into the importance of overcoming the initialization gap and how FreeInit technology bridges that gap to unlock the full potential of video diffusion models. Traditional methods often struggle with the initialization process in video diffusion models, resulting in suboptimal performance and slow convergence.
But with FreeInit technology, a combination of pretraining and fine-tuning techniques is utilized to achieve an optimal initialization, leading to improved performance and faster convergence.
Attached Workflow for Patreon Supporters
Comments
https://github.com/Kosinkadink/ComfyUI-AnimateDiff-Evolved Go to Comfyui manager, update this node.
Benjamin Law
2024-03-01 03:21:33 +0000 UTCHey, I'm sorry, can you explain what I need to do to get this new node? I looked for it in the usual places and couldn't find it. Maybe a link? Thanks again.
GT
2024-03-01 01:53:10 +0000 UTCGreat thanks!
GT
2024-02-29 23:04:58 +0000 UTCYou have to get the latest AnimateDiff Eovled. That was the Gen 2 node from it, it comes with the schduler.
Benjamin Law
2024-02-29 19:07:09 +0000 UTCYes it does, other other workflow are stable one. The workflow in this post is just a lightweight version. Just see how many nodes and how organised is it you can tell.
Benjamin Law
2024-02-29 19:06:20 +0000 UTCUpdated everything. Making progress, I've got half of it working, but it's stalling on the Sample Settings node with this error: Error occurred when executing ADE_AnimateDiffSamplingSettings: SampleSettingsNode.create_settings() got an unexpected keyword argument 'sigma_schedule' File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Also, the IPAappterApply is throwing this error: I am having trouble finding an upstream LoRA called add_detail.safetensors. I have add_detail-xl.safetensors, will that work? Error occurred when executing IPAdapterApply: 'NoneType' object has no attribute 'patcher' File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 751, in apply_ipadapter clip_embed = encode_image_masked(clip_vision, image, clip_vision_mask) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py", line 270, in encode_image_masked comfy.model_management.load_model_gpu(clip_vision.patcher) ^^^^^^^^^^^^^^^^^^^
GT
2024-02-29 19:03:56 +0000 UTCGreat thanks, I'll give that a shot. Oh, and one other thing. I've been using another one of your video2video workflows and while it's producing some nice results, the video itself is rather soft, almost like it's a low-res clip that's been upscaled but without any enhancements. If I just make a still without pumping it through the AnimDiff nodes it looks nice and detailed. Is that just an artifact produced by the AnimDiff process? Still learning this stuff. Thanks again for your help.
GT
2024-02-29 17:28:28 +0000 UTCTry update your Comfy and all other custom nodes
Benjamin Law
2024-02-29 13:35:24 +0000 UTCSorry, here's another one: Error occurred when executing ADE_AnimateDiffSamplingSettings: SampleSettingsNode.create_settings() got an unexpected keyword argument 'sigma_schedule' File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
GT
2024-02-29 00:38:22 +0000 UTCHi Benji, just joined today to try out this workflow. I keep getting this error: Any ideas? I believe I have all the loaders set correctly. Error occurred when executing KSampler: 'ModuleList' object has no attribute '1' File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1368, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1338, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-diffusion-cg_old\recenter.py", line 29, in sample_center return SAMPLE(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateLCM\animatediff\sampling.py", line 334, in motion_sample latents = wrap_function_to_inject_xformers_bug_info(orig_comfy_sample)(model, noise, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateLCM\animatediff\utils_model.py", line 272, in wrapped_function return function_to_wrap(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 248, in motion_sample return orig_comfy_sample(model, noise, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\comfy\sample.py", line 100, in sample samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_smZNodes\__init__.py", line 130, in KSampler_sample return _KSampler_sample(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 706, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_smZNodes\__init__.py", line 149, in sample return _sample(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 604, in sample pre_run_control(model, negative + positive) File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\comfy\samplers.py", line 449, in pre_run_control x['control'].pre_run(model, percent_to_timestep_function) File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 442, in pre_run_inject self.base.pre_run(model, percent_to_timestep_function) File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\comfy\controlnet.py", line 296, in pre_run comfy.utils.set_attr(self.control_model, k, self.control_weights[k].to(dtype).to(comfy.model_management.get_torch_device())) File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\ComfyUI\comfy\utils.py", line 295, in set_attr obj = getattr(obj, name) ^^^^^^^^^^^^^^^^^^ File "C:\ComfyUI\ComfyUI_windows_portable_nvidia_cu121_or_cpu_Upscaler\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1688, in __getattr__ raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
GT
2024-02-29 00:23:19 +0000 UTCOMG
KEVIN
2024-02-28 14:29:26 +0000 UTC
