Flux.1 LoRA Training Tests - Llava Captioning
Added 2024-08-17 18:54:35 +0000 UTCFigured I'd try a different model, so this time it's llava!
Much the same idea as with using Florence to do the captioning, only this time I'm using LLM Party and Llava:
ollama pull llava-llama3
As with the Florence captioning you'll need to set 3 paths:
1. The directory with all your images
2. The full path to where you want to save the text files (e.g. /home/myuser/comfy/output/my-lora)
3. The path relative to your ComfyUI output directory to save the image files (e.g. my-lora)
Is it any better for training? Who knows! Give it a try and see... I know I am :)
Comments
I finally managed to re-create the issue :) ComfyUI recently changed up to CUDA 12.4 and I think it's llama_index using that old URL. llama-cpp-python doesn't look to be needed anyway, so it will run fine without it. They've also just made a few updates - https://github.com/heshengtao/comfyui_LLM_party/issues/82
Nerdy Rodent
2024-09-10 11:49:05 +0000 UTCIs this solved? I run into the same problem. Installing llama-cpp-python... Looking in indexes: https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu124 ERROR: Could not find a version that satisfies the requirement llama-cpp-python (from versions: none) ERROR: No matching distribution found for llama-cpp-python
Zenith Wu
2024-09-07 13:32:19 +0000 UTCYes. Very useful π
Guillaume C.
2024-09-07 08:35:29 +0000 UTCIt's Comfyui_yanc - https://github.com/ALatentPlace/ComfyUI_yanc
Nerdy Rodent
2024-08-18 17:35:07 +0000 UTCRobert CODRON
2024-08-18 17:31:58 +0000 UTCBefore it says "failed to import" the errors will be shown. Something like "can't do this", or "that thing failed" type messages. Another option is to make another comfy environment, install manager, then install llm party
Nerdy Rodent
2024-08-18 13:41:30 +0000 UTCI have checked the github but nothing there was helpful. No errors, just says βfailed to importβ Let me get back to the pc and see if there was anything else.
SouthbayJay
2024-08-17 23:53:11 +0000 UTCHave you checked the github issues? Also, any error messages at startup will provide information as to what's wrong with your environment
Nerdy Rodent
2024-08-17 23:50:16 +0000 UTCI wish I could use all these workflows with the LLM Part nodes but for the life of me I can't get them to install. It's a huge bummer..
SouthbayJay
2024-08-17 23:47:31 +0000 UTCMay have to wait a while for whatever configuration you have, perhaps, if it's related to this one - https://github.com/abetlen/llama-cpp-python/issues/1682 ?
Nerdy Rodent
2024-08-17 23:14:45 +0000 UTCNot tried those, nope. Something to look at then - thanks! :)
Nerdy Rodent
2024-08-17 23:10:51 +0000 UTCLittle off topics but have you tried Griptape nodes? They're a great set of custom nodes for working with different tools like LLMs, a little more streamlined for getting image captions IMO and linking multi prompt functions. And they support, OpenAI, Claude, Ollama, Florence2, and OpenAI endpoint style models. Thanks for the workflows, keep up the good work.
Daniel Aleshinloye
2024-08-17 23:07:27 +0000 UTCI have trouble installing LLM Party, hope someone here happens to know the solution. Installing llama-cpp-python... Looking in indexes: https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu124 ERROR: Could not find a version that satisfies the requirement llama-cpp-python (from versions: none) ERROR: No matching distribution found for llama-cpp-python The version seems to only go up to cu122.
Arjan Speelman
2024-08-17 22:28:55 +0000 UTCI've been pretty impressed with llava captioning so far. 512 tokens, 1000 steps, 8e-5 and it's already got the details!
Nerdy Rodent
2024-08-17 21:05:50 +0000 UTCI've been playing with use kijai's Florence2 node and using cogflorence 2.1 - large with good results.
Andro.Meta
2024-08-17 20:10:14 +0000 UTC
