ComfyUI: Flux with LLM, 5x Upscale (Workflow Tutorial)

Seth introduces Flux, a tool with a unique learning curve compared to Stable Diffusion. He discusses the creation of a Flux Sampler node and other custom nodes to simplify the process. Flux operates on pixels rather than resolutions, and Seth presents the Flux Resolution Calculator and Get Image Size node. He highlights Flux's ability to handle image-to-image tasks without adapters or control nets, showcasing varied creative results from slight prompt modifications. The workflow is designed for upscaling up to 5.4x the input resolution on consumer-grade hardware. Seth also mentions the Flux One models' hybrid architecture, combining multimodal capabilities with advanced processing techniques.

The paragraph details the initial setup for the workflow, including the use of custom nodes from Impact Pack for logic switches and flow execution, and nodes from other packs for various tasks like text handling and image processing. Seth explains the process of setting up the nodes for image conditioning, including the use of set nodes for width and height, and the organization of nodes into groups for better workflow management. He also discusses the dynamic nature of the Flux resolution calculator node and its importance in maintaining image coherence across different resolutions.

Seth describes the process of text conditioning, where user inputs are structured and expanded by an LLM model to enhance image generation. He outlines the creation of a group for text input and the use of an Olama Vision node for image reference. The integration of the LLM model within Comfy UI is emphasized, as it can drastically change image outputs. The paragraph also covers the logic for enabling and disabling the LLM chain based on input, using switches and Boolean nodes to control the workflow direction.

This section delves into the logic control for image and text processing, using the Bridge control node for workflow management. Seth explains the use of Boolean logic to enable or disable the LLM node based on user input. He also discusses the creation of custom conditioning for disabling both LLMs and the importance of maintaining consistency in image generation. The paragraph covers the setup for text and image LLM conditioning, ensuring the correct flow of logic and data through the nodes.

Seth provides an in-depth look at the text LLM conditioning, explaining how to generate detailed prompts for the T5 XL model. He emphasizes the importance of structured prompts for quality outputs and demonstrates how to fine-tune prompts using custom conditioning. The paragraph also covers the process of summarizing prompts for the clip L model, ensuring the summary is concise and relevant.

The focus shifts to image LLM conditioning, where Seth instructs on how to analyze and describe images using the LLM. He discusses the challenges of inconsistent responses from the vision node and the solution of saving the initial prompt. The paragraph also includes instructions for modifying image prompts while maintaining context, showcasing the flexibility of LLM in workflow integration.

Seth explains the process of modifying image prompts through LLM, demonstrating how to change subjects while keeping the overall context intact. He addresses the complexities of the switch logic and how to ensure the correct flow of logic based on user input. The paragraph also covers the creation of a separate modify image logic to handle different user modifications and the importance of testing each part of the workflow.

This section discusses the settings and controls for image-to-image processes in Flux. Seth talks about the use of switches for model and latent selection, and the importance of denoise values in maintaining image consistency. He introduces the concept of max shift for creative control, explaining how different values affect the output. The paragraph also covers the organization of nodes into groups for better workflow management.

Seth demonstrates the inpainting process in Flux, detailing the use of switches, Florence 2 run nodes, and segmentation for masking. He emphasizes the importance of mask quality and the sensitivity of the output to mask shape. The paragraph covers the process of fine-tuning masks and the use of inpainting for fixing artifacts or changing subjects while maintaining consistency.

The paragraph discusses the upscale process in Flux, including the use of different upscale models and the importance of maintaining original image consistency. Seth explains the process of calculating new width and height for upscales, the use of flux samplers, and the significance of denoise values. He also covers post-processing techniques like auto adjust and levels node adjustments for final image output.

Seth concludes the tutorial by reflecting on the upscale quality and consistency maintained throughout the workflow. He mentions the recent addition of control net support for Flux in Comfy UI and his plans to update the workflow after testing. The paragraph also hints at the potential coverage of Guff, a format for lower VRAM usage, in future tutorials.