FLUX Kontext Pro for Pixel Art Character Costume Replacement

Just a little experimentation Saturday morning.

Model: Flux Kontext Pro

Input Image:

Prompt: change EVERYONE's outfit to construction worker while keeping EVERYTHING the same (same bg, same layout, same color scheme)

This works better than expected—style and scale survives while updating only the costumes—but some character’s have their hair color swapped.

It appears that there is limit to the number of attention heads. Seems like it is difficult for the model to try to replicate the hair style of more than 4 characters.

---

How this could be useful for SpriteDX?

Well for one, Costume replacement could be a feature in SpriteDX. However, more importantly, I think this will help with mutating the template image before Stage 1 runs.

The achilles hill of “fill-in-the-blank” approach used in Stage 1 is that text guidance has limited impact on generated sprites. It will always prioritize overall image consistency over the text prompt. For instance, even if you type in “boy,“ if all characters in the template images are girls, it will produce a female character.

Also, even if you try to change the costume of the character, if all characters in the template image is wearing a dress, it almost certainly uses the same costume.

In the below workflow, we explicitly type in “gender: boy,“ but it has no impact because all the other characters in the template are female characters. Even if you yank up guidance, no luck.

So, things like gender swap and costume change, we can have a Stage 0.5 where we mutate the template image itself before moving on to Stage 1.

— Sprited Dev