Dev Log: Playing around with Comfy UI

Based on our recent pivot, I’m working toward the SpriteDX prototype.

During my vacation, I’ve been doing some preliminary research on using Flux pipelines to build an image-to-image system for generating consistent retro-style pixel sprite sheets.

Here’s what I’ve explored so far:

• Using Retro Pixel Flux LoRA for Pixel Art

• Using Tile Control Net on Flux to lay out sprite sheet

• Using Flux Fill Model for sprite sheet population

TL;DR: The Flux1-Pro/Dev model, when combined with Retro Pixel LoRA and Tile ControlNet, can generate consistent character sheets through image-to-image workflows.

But… it falls short when it comes to animation-ready sprite-sheets. I think the root issue is this: the models were trained on image types you’d find on Instagram or Pinterest—images optimized for human aesthetics, not machine-readable character animation. These reference styles just aren’t structured enough for animating frame-by-frame sprites.

So, I’m experimenting with using walk cycle templates sprite sheets as ControlNet input to guide the layout better. I’m not overly optimistic, but I want to test it out before ruling it out.

Today, I installed ComfyUI and started playing around with the Flux1-Dev models inside it. Mostly just prompt tweaking for now—nothing groundbreaking to share yet.

Tomorrow, I plan to plug in IP Adapters and LoRAs (as described here) to try generating sprite sheets with walk cycles that follow a provided template. Fingers crossed.

— Sprited Dev