AI Neural Net upscaling of old pixel backgrounds

Found this on my Twitter feed. Those AI-powered upscales for old adventure games look really impressive! Can’t wait for such a functionality to be included in ScummVM… :open_mouth:

5 Likes

Here’s an imgur album with a lot of upscaled adventure game images:

2 Likes

*cough*

Impressive! :slight_smile:

Edit: also Wadjet Eye games. Shardlight looks good, Blackwell Epiphany looks, um, different. (Quite good, but also very different.)

This is something… where you can download it or how this filter works?

1 Like

It’s probably a bit too slow for the moment.

Unfortunately it looks like I can’t run it:

$ python3 test.py models/RRDB_ESRGAN_x4.pth
Traceback (most recent call last):
  File "test.py", line 21, in <module>
    model = model.to(device)
  File "/home/frans/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 381, in to
    return self._apply(convert)
  File "/home/frans/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 187, in _apply
    module._apply(fn)
  File "/home/frans/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 187, in _apply
    module._apply(fn)
  File "/home/frans/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 193, in _apply
    param.data = fn(param.data)
  File "/home/frans/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 379, in convert
    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
  File "/home/frans/.local/lib/python3.7/site-packages/torch/cuda/__init__.py", line 161, in _lazy_init
    _check_driver()
  File "/home/frans/.local/lib/python3.7/site-packages/torch/cuda/__init__.py", line 82, in _check_driver
    http://www.nvidia.com/Download/index.aspx""")
AssertionError: 
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from
http://www.nvidia.com/Download/index.aspx

I could temporarily install an Nvidia 7600GT, but something tells me it still wouldn’t work… :stuck_out_tongue:

I mean, I have OpenCL, don’t I? What with this Nvidia nonsense. >_<

Ah, good point. It probably needs a lot of computing power and might not be possible on the spot. So I guess before it could become a ScummVM feature, there will be pre-rendered remakes of those old games at some point. :slight_smile:

I was hoping to see just how long it’d take my GPU (or CPU). The GH page doesn’t mention anything about Cuda/Nvidia so I stupidly assumed it’d Just Work™. Grmbl, I wanted to try a couple of screens for TWP and LucasArts stuff.

GH issue:

Edit: I might get it going yet… not sure I want to spend the time though. :slight_smile:

I found this: " you don’t need an NVIDIA card for this just go into test.pyand change “device = torch.device(‘cuda’)“ to “ device = torch.device(‘cpu’)”."

We did some quick tests with a basic/standard dataset models and here are the results:

and final resolution from original 320 pixels to 1280 pixels

this method is used a lot in FPS games 3D remakes

NOW! We wish to create a brand new model that will work much better than this one.

1 Like

Thanks for the tip! Pity I already removed the whole thing in the meantime lest I forgot (and then I’d be mysteriously out of 2 GB or so :smiley: ).

Maybe I’ll play around with it tonight, maybe not. First I’ve got an Android compilation issue to sort out. (Wholly unrelated to this topic.)

You mean for upscaling the textures? Interesting, I didn’t know that!

ok, we got the new model set up and here is the satisfying result… excellent result. I will try more images.

6 Likes

I think this is a “filter” that works very well for the hand drawn backgrounds which were scanned into a pixel matrix (sorry if I do not use the correct terms, I hope you will get me).
The results on Monkey Island 2 are in fact great! (With the exception of the jungle that needs a little tweak maybe). I disliked even back in the nineties the drawings put into pixels.

While for the games whose graphics were originally drawn from scratch pixel by pixel, well, in that case I don’t particularly like the results (Monkey Island 1 for example).

4 Likes

I agree with @Gffp: Especially the last Monkey Island 1 picture looks ugly IMHO.

3 Likes

Less colors in original image, worse results… but thats normal. In most cases, 75% images will be fine.
here is one from TWP 4K

P.S. I would always do a quick retouch on top of it.

The results aren’t that bad … especially on the left side with the rocks. But the edges are too “sharp” (especially the bushes and the grass).

I don’t think it’s a matter of number of colours… or not only that.
Have a look at the palms in the landscape of the monkey head area, or look at the barrels in the Melee shop… you can see some really giant pixels which were poorly upscaled.
Anyway, it seems to me you used the 256 color art from MI… so the number of colors should be the same. Am I wrong? I think it’s more a matter of shades…
pixel art has sharper edges, while on a scanned drawing you always have some interpolation shades.
Looking at the TWP sample, it seems it works great for round shapes (see the cliff in the background) and poorly with oblique lines (see the structure of the bridge).

BTW, sorry if I used the wrong terms, too. I’m not into graphics

1 Like

Look at it this way:

its greatly generated by AI code, thats amazing. Any version that comes off it, is a great help to any other paint overs later. Its actually big step up, its very talented code.
Pastels environments from old games have best results, because of the smoothness from scanned images.
Also, keep on mind, that those generating models can have a bunch of different AI trainings to achieve even better quality. Thats cool.

2 Likes

Actually guys, Monkey Island 3 brought up the best results and looks amazing 2K 4K, it can be used as beta version for a test. :smiley::smile:

check this out :open_mouth:
https://drive.google.com/open?id=1twcONP_MwCqKwgW4i_IKcqkptthfpWKN

2 Likes

That greatly exceeded my wildest expectations. Sure, it’s higher res than MI2 and all, but damn. :slight_smile:

1 Like