Midjourney's 🖌️ Vary (Region) method through API

This article comments on the use of the 🖌️ Vary (Region) - called Inpaint in the documentation

This method is available after calling Upscale.

That is, the sequence of actions is as follows:

1. Imagine -> get hash1

2. Upscale (pass hash1 + choice (position of the image we plan to work with next)) -> get hash2

3. Inpaint (pass hash2) and additional data, about which below -> get hash3 and wait for the final result.

The Inpaint method implies that the user specifies an area in the upscale image and enters a new prompt for that particular area. For example, if your upscale image shows a dog lying on the couch, then by selecting that dog and passing the new prompt "cat", Midjourney will repaint the area with the dog and draw the cat.

How do I pass mask?

mask is base64 from a black and white image with dimensions matching the Upscale image. Where the white area is the area for which the new prompt will be applied

Now let's take a concrete example using curl

1. Create an image through Imagine:

curl -X POST https://api.userapi.ai/midjourney/v2/imagine \
-H "Content-Type: application/json" \
-H "api-key: 3fb0b6d9-6e1b-4bd7-993f-f79e9f1717e8" \
-d '{ "prompt": "a dog lying on a cozy couch" }'

In response, we get hash:

{
  "hash":"323b5f0a-9b62-4982-ab25-658b6004e6d3"
}

We wait for the image to be generated, and finally call the Status method, passing the hash above to it:

curl -X GET \
-H "Content-Type: application/json" \
-H "api-key: 3fb0b6d9-6e1b-4bd7-993f-f79e9f1717e8" \
"https://api.userapi.ai/midjourney/v2/status?hash=323b5f0a-9b62-4982-ab25-658b6004e6d3"

In the response we get a link to 4 images (part of the fields are removed because they are completely unnecessary in the context of the example):

{
  "hash":"323b5f0a-9b62-4982-ab25-658b6004e6d3",
  "prompt":"a dog lying on a cozy couch",
  "type":"imagine",
  "progress":100,
  "status":"done",
  "result":{
    "url":"https://cdn.discordapp.com/attachments/example.png",
    "proxy_url":"https://media.discordapp.net/attachments/example.png",
    "width":2048,
    "height":2048
  },
  "created_at":"2024-03-25T14:42:57Z"
}

Here it is:

2. We got the image, choose the one we like the most (for example, the second one) and enlarge it using the Upscale method:

curl -X POST https://api.userapi.ai/midjourney/v2/upscale \
-H "Content-Type: application/json" \
-H "api-key: 3fb0b6d9-6e1b-4bd7-993f-f79e9f1717e8" \
-d '{ "hash": "323b5f0a-9b62-4982-ab25-658b6004e6d3", "choice": 2}'

Calling Upscale in response we get hash:

{
  "hash":"2e324ddc-f6e4-4c2f-8511-81cc10214904"
}

3. After Upscale is executed, using the Status method by hash above - we get a reference to image number two and the size of this image (unnecessary fields are removed):

{
  "hash":"2e324ddc-f6e4-4c2f-8511-81cc10214904",
  "choice":"2",
  "type":"upscale",
  "status":"done",
  "result":{
    "url":"https://cdn.discordapp.com/attachments/example.png",
    "width":1024,
    "height":1024
  },
  "created_at":"2024-03-25T15:06:35Z"
}

4. At this step it is necessary to provide the user with a tool with which he will select the area to be redrawn. The size of the mask must match the width and height of the upscaled image (in our case 1024x1024). As a result, for our image, we get this area (the dog is highlighted):

In the same step ask the user to specify a new prompt for this particular white area, in my example it would be "a cat"

4. We make a request to Inpaint, with mask being the base64 representation of the png image we got above by converting the black and white image

curl -X POST https://api.userapi.ai/midjourney/v2/inpaint \
-H "Content-Type: application/json" \
-H "api-key: 3fb0b6d9-6e1b-4bd7-993f-f79e9f1717e8" \
-d '{
    "hash": "6794f6ef-866a-4bc3-b0bc-7b28ec010d1b",
    "mask": "UklGRgQoAABXR...",
    "prompt": "a cat"
}'

After waiting for rendering through the same Status method we get the final image:

Great, instead of a dog, it's a cat now! Let's summarize! In this article we've figured out how you can redraw a certain area on an image by specifying with text what exactly should be drawn instead through a new prompt with API in Midjourney!