RAVE_AnimateDiff_V2V LCMIPA Version 4_9_1 Workflow
Added 2024-03-15 15:00:14 +0000 UTCVideo Tutorial : https://youtu.be/CJzByQ2Z4TU
Download the Comfy UI impact pack https://github.com/ltdrdata/ComfyUI-Impact-Pack
Comfy UI inspired pack https://github.com/ltdrdata/ComfyUI-Inspire-Pack
Workflow available for Patreon Supporters only
Comments
Ok this issue is caused by person_yolov8m-seg.pt, no issue with hand_yolo8m.pt
Patryk Bartkowiak
2024-03-20 11:13:35 +0000 UTCHave you ever seen this error? I am getting it only for videos with frames>8 Error: ``` Error occurred when executing SEGSDetailerForAnimateDiff: The size of tensor a (270) must match the size of tensor b (256) at non-singleton dimension 1 File "/nas/people/patryk.bartkowiak/ComfyUI/execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) File "/nas/people/patryk.bartkowiak/ComfyUI/execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) File "/nas/people/patryk.bartkowiak/ComfyUI/execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/animatediff_nodes.py", line 92, in doit segs, cnet_images = SEGSDetailerForAnimateDiff.do_detail(image_frames, segs, guide_size, guide_size_for, max_size, seed, steps, cfg, sampler_name, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/animatediff_nodes.py", line 69, in do_detail enhanced_image_tensor, cnet_images = core.enhance_detail_for_animatediff(cropped_image_frames, model, clip, vae, guide_size, guide_size_for, max_size, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/core.py", line 407, in enhance_detail_for_animatediff refined_latent = impact_sampling.ksampler_wrapper(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/impact_sampling.py", line 125, in ksampler_wrapper refined_latent = nodes.KSampler().sample(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise * sigma_factor)[0] File "/nas/people/patryk.bartkowiak/ComfyUI/nodes.py", line 1369, in sample return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise) File "/nas/people/patryk.bartkowiak/ComfyUI/nodes.py", line 1339, in common_ksampler samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/sampling.py", line 365, in motion_sample latents = wrap_function_to_inject_xformers_bug_info(orig_comfy_sample)(model, noise, *args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/utils_model.py", line 377, in wrapped_function return function_to_wrap(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 22, in informative_sample raise e File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 9, in informative_sample return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations. File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/sample.py", line 100, in sample samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 704, in sample return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 609, in sample samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 548, in sample samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/k_diffusion/sampling.py", line 745, in sample_lcm denoised = model(x, sigmas[i] * s_in, **extra_args) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 286, in forward out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 273, in forward return self.apply_model(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 270, in apply_model out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/sampling.py", line 405, in evolved_sampling_function cond_pred, uncond_pred = comfy.samplers.calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/samplers.py", line 224, in calc_cond_uncond_batch output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/model_base.py", line 96, in apply_model model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float() File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 850, in forward h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI-AnimateDiff-Evolved/animatediff/sampling.py", line 104, in forward_timestep_embed x = layer(x, context, transformer_options) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/attention.py", line 633, in forward x = block(x, context=context[i], transformer_options=transformer_options) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/nas/people/patryk.bartkowiak/comfy-conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/attention.py", line 460, in forward return checkpoint(self._forward, (x, context, transformer_options), self.parameters(), self.checkpoint) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/diffusionmodules/util.py", line 191, in checkpoint return func(*inputs) File "/nas/people/patryk.bartkowiak/ComfyUI/comfy/ldm/modules/attention.py", line 557, in _forward n = attn2_replace_patch[block_attn2](n, context_attn2, value_attn2, extra_options) File "/nas/people/patryk.bartkowiak/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/IPAdapterPlus.py", line 575, in __call__ out_ip = out_ip * mask_downsample ```
Patryk Bartkowiak
2024-03-20 09:46:49 +0000 UTCcheck this out : https://www.patreon.com/posts/98757420 Thanks :)
Benjamin Law
2024-03-18 15:50:51 +0000 UTCHey I am currently missing two models to run the pipeline and can't really find them online, could you please help? - Animate_controlnet_checkpoint.ckpt - SD1-5\add_detail.safetensors
Patryk Bartkowiak
2024-03-18 14:01:36 +0000 UTCHaven't face this error message before in DWpose preprocessor. Do you have a large video and load that in?
Benjamin Law
2024-03-17 11:48:42 +0000 UTChello,my comfyui don't work，The process stops at the dwpose estimator, and here is the error message DWPose: Pose 390.29ms on 1 people Prompt executed in 874.86 seconds Exception in callback _ProactorBasePipeTransport._call_connection_lost(None) handle: Traceback (most recent call last): File "asyncio\events.py", line 80, in _run File "asyncio\proactor_events.py", line 165, in _call_connection_lost ConnectionResetError: [WinError 10054] 远程主机强迫关闭了一个现有的连接。
Sky
2024-03-17 11:44:44 +0000 UTCNo, it just like the previous RAVE, so no new models need. Just add more Segmentation and Mask features
Benjamin Law
2024-03-16 15:50:46 +0000 UTCqq, is there a list of the models that i need to download? Thanks for all the great videos!!
Seth Chapin
2024-03-16 15:45:29 +0000 UTCThe earlier post is for public view only.i post it again here for supporters like you guys😊 and this workflow is the update version of the previous RAVE AnimateDiff workflow , and keep enhancing it.
Benjamin Law
2024-03-16 09:06:37 +0000 UTCIs this the right video tutorial linked? It seems to point to the SEGS and AnimateDiff tutorial posted earlier. I didn't see anything about RAVE in that linked video.
Peter C
2024-03-16 00:26:55 +0000 UTChi, you might want to take a look at this one : https://www.youtube.com/watch?v=HxlZHsd6xAk About the AnimateLCM. basically, it's faster in process.
Benjamin Law
2024-03-15 16:56:52 +0000 UTCI found two new file at AnimateDiff "AnimateLCM_sd15_lora_beta.safetensors", "AnimateLCM_ssd15_t2v_beta.ckpt" where get download it? and I want to know why you changed it.
Yu Mu
2024-03-15 16:05:30 +0000 UTC
