SpriteDX - Stage 3 Integration

Yesterday, we integrated Stage 2 into SpriteDX’s web UI. Things are still a bit rough, but the priority now is moving forward with Stage 3.

Why not go back and fix Stage 1 and Stage 2 first? Right now, Stage 1 is hard-coded to only support 4×2 character templates. That’s not ideal—but what we’ve learned is that every new stage reveals new limitations and better approaches.

“Perfect is the enemy of the good.” — Voltaire

Our strategy is to complete one full end-to-end pass first, even if parts aren’t perfect. Only then we will circle back and refine earlier stages. This way, we avoid re-doing work prematurely and keep momentum toward a complete pipeline.

---

Stage 3 Recap

What’s Stage 3? Stage 3 is our “Post Production Editor.”

In Stage 2, SpriteDX generates an animated WebP: a single image sequence that contains multiple shots strung together. Stage 3 is where we break that sequence back down into meaningful chunks. We’re using PySceneDetect to automatically cut the animation into shots, so each segment can be edited, rearranged, or enhanced independently.

In addition to shot detection, Stage 3 also handles loop detection. Once a shot is isolated, we trim it down to the cleanest repeating cycle — whether it’s a walk, idle, or attack motion. These loops are what make sprite sheets possible.

With this step, SpriteDX doesn’t just generate animations; it structures them in to reusable, production-ready building blocks. Stage 3 is the bridge between raw generative output and polished loops that can be meaningfully reused and composed.

Stage 3 in ComfyUI:

Integration

Let’s export it and see if it will work out of the box! Actually, I’m almost certain that it won’t work out of the box.

I’ve already installed sprited-comfyui-nodes into the docker image, so I think that part I can more or less expect to just work. However, current workflow uses video files, and RunPod’s worker-comfyui is only able to specify image files as input/outputs.

We have two options.

1. Make worker-comfyui able to interface video inputs and outputs.

2. Convert input video into animated WEBP and conver output video as animated WEBP.

First option will require me to take a journey into the worker-comfyui codebase and figure things out there. So it will probably take time. However, it is one-time investment that will solve many situations that require video in/outs.

The thing about option 1 is that runpod/worker-comfyui has since required bit of updates so far.

• Fixing legacy unet folder path.

• CUDA 12.6 to CUDA 12.4 downgrade to make it run fine in all nodes in RunPod Serverless.

• Installing custom nodes into it.

We have already forked the repo for the first two problems. Now, is the fourth time we are patching things up here… So, my thinking is that we should double down on this fork and work on adding video IO support there.

Pixel vetos it and asking me to do Option 2 first to move fast. True hacker, we’ve got here… Okay, let’s go with Option 2.

To replace the Load Video node, I added Load Image and Create Video node, and now I’m able to take animated WEBP file and run it through Video Shot Splitter and Loop Trim node. Great!

Next step is to change the output to animated webp file. I’m using Get Video Components and Save Animated WEBP nodes.

Wow, that was simpler than I thought. Let’s export the workflow and put it on SpriteDX.

Quick Demo: Clicking on Generate button successfully generate a Run Cycle that fully loops around.

Animated WEBP file:

---

Next Steps

That’s it for today. I will spend rest of the day on course works. Tomorrow, I will look into integrating Stage 4—Automatic Background Removal.

— Sprited Dev 🌱