RAVE_AnimateDiff_V2V LCMIPA Version 4_9_1 Workflow
Added 2024-03-15 15:00:14 +0000 UTCVideo Tutorial : https://youtu.be/CJzByQ2Z4TU
Download the Comfy UI impact pack https://github.com/ltdrdata/ComfyUI-Impact-Pack
Comfy UI inspired pack https://github.com/ltdrdata/ComfyUI-Inspire-Pack
Workflow available for Patreon Supporters only
Comments
Ok this issue is caused by person_yolov8m-seg.pt, no issue with hand_yolo8m.pt
Patryk Bartkowiak
2024-03-20 11:13:35 +0000 UTCHave you ever seen this error? I am getting it only for videos with frames>8 Error: ``` Error occurred when executing SEGSDetailerForAnimateDiff: The size of tensor a (270) must match the size of tensor b (256) at non-singleton dimension 1 File "/nas/people/patryk.bartkowiak/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/nas/people/patryk.bartkowiak/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/nas/people/patryk.bartkowiak/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/animatediff_nodes.py", line 92, in doit segs, cnet_images = SEGSDetailerForAnimateDiff.do_detail(image_frames, segs, guide_size, guide_size_for, max_size, seed, steps, cfg, sampler_name, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/animatediff_nodes.py", line 69, in do_detail enhanced_image_tensor, cnet_images = core.enhance_detail_for_animatediff(cropped_image_frames, model, clip, vae, guide_size, guide_size_for, max_size, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/core.py", line 407, in enhance_detail_for_animatediff refined_latent = impact_sampling.ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/impact_sampling.py", line 125, in ksampler_wrapper refined_latent = nodes.KSampler().sample(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise * sigma_factor)[0] File "/nas/people/patryk.bartkowiak/ComfyUI/nodes.py", line 1369, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) File "/nas/people/patryk.bartkowiak/ComfyUI/nodes.py", line 1339, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/sampling.py", line 365, in motion_sample latents = wrap_function_to_inject_xformers_bug_info(orig_comfy_sample)(model, noise, *args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/utils_model.py", line 377, in wrapped_function return function_to_wrap(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 22, in informative_sample raise e File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 9, in informative_sample return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/sample.py", line 100, in sample samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 704, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 609, in sample samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 548, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/k_diffusion/sampling.py", line 745, in sample_lcm denoised = model(x, sigmas[i] * s_in, **extra_args) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 286, in forward out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 273, in forward return self.apply_model(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 270, in apply_model out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/sampling.py", line 405, in evolved_sampling_function cond_pred, uncond_pred = comfy.samplers.calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 224, in calc_cond_uncond_batch output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/model_base.py", line 96, in apply_model model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float() File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 850, in forward h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/sampling.py", line 104, in forward_timestep_embed x = layer(x, context, transformer_options) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/attention.py", line 633, in forward x = block(x, context=context[i], transformer_options=transformer_options) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/attention.py", line 460, in forward return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/diffusionmodules/util.py", line 191, in checkpoint return func(*inputs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/attention.py", line 557, in _forward n = attn2_replace_patch[block_attn2](n, context_attn2, value_attn2, extra_options) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/IPAdapterPlus.py", line 575, in __call__ out_ip = out_ip * mask_downsample ```
Patryk Bartkowiak
2024-03-20 09:46:49 +0000 UTCcheck this out : https://www.patreon.com/posts/98757420 Thanks :)
Benjamin Law
2024-03-18 15:50:51 +0000 UTCHey I am currently missing two models to run the pipeline and can't really find them online, could you please help? - Animate_controlnet_checkpoint.ckpt - SD1-5\add_detail.safetensors
Patryk Bartkowiak
2024-03-18 14:01:36 +0000 UTCHaven't face this error message before in DWpose preprocessor. Do you have a large video and load that in?
Benjamin Law
2024-03-17 11:48:42 +0000 UTChello,my comfyui don't workοΌThe process stops at the dwpose estimator, and here is the error message DWPose: Pose 390.29ms on 1 people Prompt executed in 874.86 seconds Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: Traceback (most recent call last): File "asyncio\events.py", line 80, in _run File "asyncio\proactor_events.py", line 165, in _call_connection_lost ConnectionResetError: [WinError 10054] θΏη¨δΈ»ζΊεΌΊθΏ«ε ³ιδΊδΈδΈͺη°ζηθΏζ₯γ
Sky
2024-03-17 11:44:44 +0000 UTCNo, it just like the previous RAVE, so no new models need. Just add more Segmentation and Mask features
Benjamin Law
2024-03-16 15:50:46 +0000 UTCqq, is there a list of the models that i need to download? Thanks for all the great videos!!
Seth Chapin
2024-03-16 15:45:29 +0000 UTCThe earlier post is for public view only.i post it again here for supporters like you guysπ and this workflow is the update version of the previous RAVE AnimateDiff workflow , and keep enhancing it.
Benjamin Law
2024-03-16 09:06:37 +0000 UTCIs this the right video tutorial linked? It seems to point to the SEGS and AnimateDiff tutorial posted earlier. I didn't see anything about RAVE in that linked video.
Peter C
2024-03-16 00:26:55 +0000 UTChi, you might want to take a look at this one : https://www.youtube.com/watch?v=HxlZHsd6xAk About the AnimateLCM. basically, it's faster in process.
Benjamin Law
2024-03-15 16:56:52 +0000 UTCI found two new file at AnimateDiff "AnimateLCM_sd15_lora_beta.safetensors", "AnimateLCM_ssd15_t2v_beta.ckpt" where get download it? and I want to know why you changed it.
Yu Mu
2024-03-15 16:05:30 +0000 UTC
