We had already a chat with an AI and the upscaling of adventure game backgrounds. Now I stumbled over this interesting (and long!) blog post where the AI DALL-E2 is generating pixel graphic backgrounds for adventures games:
6 Likes
Nice article, but unfortunately the AI didn’t create the pixel graphic.
It was simply asked to create images in the style of oil paintings. Then the author simply downsampled everything with the software “magick”.
Currently, the contribution of AI to pixel graphic is substantial in style transfer: you give the AI a drawing or photo and the AI can draw a pixel art version of it. With very good results, like those shown here.
When it comes to generation of pixel art, though, DALL-E 2 isn’t so good at it. Some people tried and got this, but the quality is still inferior to what you get when you ask the AI to transform an existing image to pixel art.
2 Likes
That could be an interesting way of generating at least some inspiration for an artist
2 Likes
guybrush picking his human nose
3 Likes
This one is great. Its just Guybrush, with a normal human nose. They are hanging out.
1 Like
And I can’t complain, because it’s technically exactly what I asked for.
2 Likes
I kind of wish it had given you Guybrush picking a human nose from a list of noses.
1 Like
I asked it to show me “The secret of Monkey Island” and it’s given me some intriguing options…
The secret is that Guybrush finds his son - this strange gopher boy:
The secret is that Guybrush finds this giant spoon and then marries a handsome cowboy:
The secret is that Guybrush and his monkey friend find a cool cactus wearing sunglasses and sideways baseball cap:
The secret is that Guybrush finds a shovel and throttles a small giraffe:
3 Likes
If this isn’t the actual secret I’m going to be mad for life.
1 Like
From asking for pixel art:
To asking for modernist art and applying a pixel filter:
2 Likes
Isn’t that the exact article that started this thread?
At any rate, I wouldn’t really consider images like those to be pixel art, but rather pixelated art 
. I also fail to see the advantage of this approach over, say, taking a photo or collage of the desired scene and then applying the same filters. Maybe having the AI create an “original” scene alleviates copyright concerns? Though where did the AI get its “inspiration” from?
Though would be interesting to see what an AI might come up with when trained on actual pixel art.
2 Likes
Exactly. I’m pretty sure that every pixel artist wouldn’t consider downsampling an image a form of pixel art. Pixel art requires drawing with pixels, juxtaposing them manually or at least with the aid of tools that don’t make the process entirely automatic.
I don’t know why the author of the article chose this path, but there are a lot of public domain images (including images of paintings) that could be “pixelated” without any legal risk.
DALL-E 2 was trained on about 650 million images of every kind and style, including images of copyrighted works.
DALL-E 2 was trained on actual pixel art images, meaning that those 650 millions images included also pixel art. The issue is that results are not good enough.
Here are some results for the prompt “An astronaut lounging in a tropical resort in space as pixel art.”:
[Source]
As you can see, pixels don’t even have the same size. While DALL-E 2 has understood the concept of “a drawing made of little squares”, it has not grasped the notion of a grid composed of squares of the same size.
This AI is very good at creating images of many styles but pixel art is not one of them.
There are also attempts at generating pixel art game assets, here is the first tweet of a list with many examples; you can visit Twitter to see the other images:
So, it’s not that DALL-e 2 can’t generate nice looking pixel art images, it’s just that they contain many artifacts.
3 Likes
Even the first pixel graphic generated by the AI? (The “hotel entrance in mexico, in the style of pixel art”). This created the AI as pixel art.
Is this necessary for pixel art?
Let me rephrase then: would be interesting to see what an AI might come up with when trained exclusively on actual pixel art.
You could easily argue that there are solid colored squares comprised of multiple pixels instead of different sized pixels. And I think in case of the astronauts it’s not too jarring either.
If the hotel scene was considered pixel art, it’s either done in a very extravagant style or by a not so talented artist 
.
As for actual pixels of different size, I would argue that if a pixel is defined as
the smallest controllable element of a picture represented on the screen
you’d just have to construct a screen with different sized pixels, et voila! Any pixel art displayed on that screen would still remain pixel art.
1 Like