In this article we are going to learn more about inpainting, so the first question that comes is “what is inpainting with DALL-E”?
Inpainting with DALL-E refers to the process of modifying an image by replacing or editing specific parts of the generated image using the capabilities of the DALL-E model. In this article we are going to remove an object from the image.
(Btw, slightly off topic, nevertheless, an important observation – a few days ago I uploaded an image to DALL-E and I expected the model to edit the image. Obviously DALL-E is not an image editing tool, but an image generating model, so my expectations were just too high (we do have high expectations of AI, don’t we? Not to mention the fact I sometimes expect it to read my mind 🙂 ). Nevertheless, DALL-E did generate a new image – one very similar to my upload – and the model did follow the instructions provided in the prompt.)
So, back to topic, as you already know, DALL-E, developed by OpenAI, is a powerful AI model that can generate images from textual descriptions. I asked DALL-E to generate an image where it would combine three tarot cards – the devil, lovers and six of pentacles. This is what the model came up with:
Unfortunately, I noticed DALL-E isn’t that good with counting objects, for example, in case of generating tarot cards, when it generates the card “6 of pentacles”, the model often generates 8 or 10 pentacles on the card instead of just six, so this is the moment where inpainting feature comes very handy.
How does inpainting on DALL-E works?
If you open the image DALL-E generates, you will notice in the upper right corner above the image a few tools, and the one with the brush (the option on the left) is the inpaint option:
So, if you click on that inpaint feature you will get the following:
On the upper left, there will be a selection tool – you can change its size with a slider – I usually decrease the size because when smaller it is easier to select a specific part of an image; and right next to the slider you will also see a “clear selection” option, in case you need to clear the selection and improve the selection process.
How to remove objects in DALL-E with inpaint feature?
In my case, I selected the two pentacles because I wanted my image to have 6 pentacles and not 8, so I selected the 2 pentacles that the children kneeling at the side were holding:
Once you select the inpaint option, you will notice DALL-E will open the chat option on the right side of the screen. And once you select a specific object or part of the image with a selection tool, “edit selection” will appear in the chat box and this is where you prompt what you want to change. In may case it was: “remove the selected objects and replace them with the existing background”. I noticed it is better to use more generic and technical language, for example, I used “remove selected objects” instead of “remove the pentacles”, and yes, the DALL-E did what I asked – the model removed the pentacles and used the background instead. This was the result:
All in one, the process of inpainting with DALL-E demonstrates a significant advancement in image manipulation technology, offering users the flexibility to modify and perfect generated images. In this article we showcased how inpainting can effectively address issues such as inaccuracies in object counts within images, using a practical example where excessive elements were seamlessly removed to align with the desired depiction. Such capabilities underscore the potential of AI-driven tools in enhancing creative outputs and solving specific visual challenges, making them invaluable resources for designers, artists, and anyone involved in digital content creation. The intuitive interface of DALL-E’s inpainting feature also highlights the accessibility of sophisticated AI technologies, ensuring that even users with minimal technical background can achieve professional-level results with ease. This blend of accessibility and power makes DALL-E not just a tool for image generation but a comprehensive solution for personalized image editing and improvement.