Simple LLM Enhanced Flux!

Video Guide: https://youtu.be/qMNJ0dCoT_I

This is one for those who don't like noodles, tweaking things - or even moving around in the ComfyUI interface!

Everything you need for generating is all in once place, with shortcuts keys for the first-time use settings. This leaves you to focus (foooocus?) on prompting, with just the required options for:

• Prompting

• Image size

• Seed

• LLM prompting mode

• HighRes mode

• USD mode + resize factor

Example interface screenshot:

Press 1 = Model Settings

Press 2 = LLM Settings (optional)

The custom scheduler is a sort of mix between beta & simple which seems to gives the fewest stripes whilst also having reasonable detail. I've tested up to 2048x2048, so you can get fairly large outputs. Note that Ultimate Upscale takes a while!

Update Notes!

13/Oct/24 - The updated version (Flux-Simple-LLM-Ultimate_v2.json) has better switch logic, so now turning multiple things on or off at the same time should work properly :)

18/Oct/24 - Added Flux-Simple-LLM_v1.json, which is much the same as v0 but has a power lora loader and the LLM party node has been updated.

NB. The LLM party node was updated recently such that you'll need to:

• Copy the existing system prompt text in the LLM Party node

• right-click and select "Fix Node (recreate)" on any older workflows with this node.

• Paste the system prompt text back in

 The node has already been re-created in v1.

The same fix has been pre-applied to Flux-Simple-LLM-Ultimate_v2a.json

LLM Prompting Mode

The prompting mode is configured slightly differently here to enable "conversations". You can chat with Llama3.2 making prompts such as these possible:

• Add more flowers

• Give everyone a silly hat

• What is to the left of them?

• What is the main character thinking about?

Ollama

For the LLM Enhanced workflows you'll need to have ollama installed & running:

https://github.com/ollama/ollama

The workflow is configured to use llama3.2 by default, which you can download by running:

ollama pull llama3.2

Workflow Notes

Three different workflow versions are attached to this post:

• Flux-Simple_v0.json = Simple interface, no LLM

• Flux-Simple-LLM_v0.json = Simple interface + LLM

• Flux-Simple-LLM-Ultimate_v0.json = Simple interface + LLM + Ultimate Upscale

If "Install Missing Custom Nodes" fails, they can be installed by using the search option instead:

• Impact Pack

• Extra Models for ComfyUI

• ComfyUI-GGUF

• ComfyUI_SUNoise

• UltimateSDUpscale

• Comfyroll Studio

• rgthree's ComfyUI Nodes

• ComfyUI Extra Samplers

• Use Everywhere (UE Nodes)

• KJNodes for ComfyUI

• ComfyUI-ppm

• pythongosssss/ComfyUI-Custom-Scripts

• comfyui_LLM_party

Models

If you are missing any models, download them from here:

Flux = https://huggingface.co/black-forest-labs/FLUX.1-dev

Flux GGUF = https://huggingface.co/city96/FLUX.1-dev-gguf/tree/main

CLIP = https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14

Hyper-SD = https://huggingface.co/ByteDance/Hyper-SD

Shakker Labs = https://huggingface.co/Shakker-Labs

Upscalers = https://openmodeldb.info/

Low VRAM

A GGUF loader (bypassed by default) is also available for those with low VRAM. Simply delete the standard loader then un-bypass and connect the GGUF one. The default configuration works best with 24GB+

Files to download for the regular version

From https://comfyanonymous.github.io/ComfyUI_examples/flux/ -

"If you don’t have t5xxl_fp16.safetensors or clip_l.safetensors already in your ComfyUI/models/clip/ directory you can find them on: this link. You can use t5xxl_fp8_e4m3fn.safetensors instead for lower memory usage but the fp16 one is recommended if you have more than 32GB ram.

The VAE can be found here and should go in your ComfyUI/models/vae/ folder."