Improve downscale quality based on window size

[ruanformigoni]

Greetings,

Would it be possible to implement something like RSR or DLDSR to improve image quality on down-scaling? I've tested gamescope with a virtual resolution in xorg, and it has better quality than the downscaling done by gamescope xrandr --output "HDMI-A-0" --mode 2560x1080 --panning 5120x2160 --scale 2x2. Here is an example with Need for speed pro street.

Take the image below as an example, this was running at 10240x4320 (both window and internal resolution). The aliasing seems to not see much improvement over the 2560x1080 counterpart, in the power lines, light rails and the tower to the right.

However, when zoomed in, one can see that the improvement was incredible, but the down-scaling process causes additional aliasing.

Edit:

I've done some research, and played around with the code. Here are some resources:

[1] https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkSamplerCreateInfo.html
[2] https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VkFilter.html
[3] https://vulkan-tutorial.com/Texture_mapping/Image_view_and_sampler
[4] https://towardsdatascience.com/image-processing-image-scaling-algorithms-ae29aaa6b36c
[5] https://stackoverflow.com/questions/13501081/efficient-bicubic-filtering-code-in-glsl

Maybe it is possible to enable bilinear or bicubic interpolation with the samplerInfo.minFilter setting.

Also, is there a way to do something like xrandr --output "HDMI-A-0" --mode 2560x1080 --panning 5120x2160 --scale 2x2 in the nested X session? That would improve the final quality as shown in the first comparison.

---

I got close to make it compile today with the changes in panning:

auto const& ctx = this->ctx;
Window wind = ctx->ourWindow;
Display* disp = ctx->dpy;
XRRScreenResources* res = XRRGetScreenResourcesCurrent(disp,wind);
RRCrtc* crtcs = res->crtcs;
XRRPanning* panning = XRRGetPanning(disp, res, *crtcs);
panning->width = 10240;
panning->height = 4320;
Status st = XRRSetPanning(disp, res, *crtcs, panning);

But there are linking issues with the xrandr header #include <X11/extensions/Xrandr.h>:

Compile output

FAILED: gamescope c++ -o gamescope gamescope.p/meson-generated_.._protocol_gamescope-xwayland-protocol.c.o gamescope.p/meson-generated_.._protocol_gamescope-pipewire-protocol.c.o gamescope.p/meson-generated_.._protocol_gamescope-input-method-protocol.c.o gamescope.p/meson-generated_.._protocol_gamescope-tearing-control-unstable-v1-protocol.c.o gamescope.p/src_steamcompmgr.cpp.o gamescope.p/src_main.cpp.o gamescope.p/src_wlserver.cpp.o gamescope.p/src_drm.cpp.o gamescope.p/src_modegen.cpp.o gamescope.p/src_sdlwindow.cpp.o gamescope.p/src_vblankmanager.cpp.o gamescope.p/src_rendervulkan.cpp.o gamescope.p/src_log.cpp.o gamescope.p/src_ime.cpp.o gamescope.p/src_mangoapp.cpp.o gamescope.p/src_pipewire.cpp.o -Wl,--as-needed -Wl,--no-undefined -Wl,--start-group subprojects/wlroots/libwlroots.a subprojects/libliftoff/libliftoff.a /usr/lib/libX11.so /usr/lib/libXdamage.so /usr/lib/libXfixes.so /usr/lib/libXcomposite.so /usr/lib/libXrender.so /usr/lib/libXext.so /usr/lib/libXxf86vm.so /usr/lib/libXRes.so /usr/lib/libdrm.so /usr/lib/libwayland-server.so /usr/lib/libxkbcommon.so -pthread /usr/lib/libSDL2.so /usr/lib/libudev.so /usr/lib/libpixman-1.so -lm -lrt /usr/lib/libinput.so /usr/lib/libwayland-client.so /usr/lib/libseat.so /usr/lib/libxcb.so /usr/lib/libxcb-composite.so /usr/lib/libxcb-icccm.so /usr/lib/libxcb-render.so /usr/lib/libxcb-res.so /usr/lib/libxcb-xfixes.so /usr/lib/libxcb-errors.so /usr/lib/libvulkan.so /usr/lib/libXtst.so /usr/lib/libcap.so /usr/lib/libpipewire-0.3.so -Wl,--end-group /sbin/ld: gamescope.p/src_wlserver.cpp.o: in function `gamescope_xwayland_server_t::gamescope_xwayland_server_t(wl_display*)': /home/ruan/Repositories/gamescope/build/../src/wlserver.cpp:766: undefined reference to `XRRGetScreenResourcesCurrent' /sbin/ld: /home/ruan/Repositories/gamescope/build/../src/wlserver.cpp:768: undefined reference to `XRRGetPanning' /sbin/ld: /home/ruan/Repositories/gamescope/build/../src/wlserver.cpp:771: undefined reference to `XRRSetPanning' collect2: error: ld returned 1 exit status ninja: build stopped: subcommand failed.

---

Managed to make it compile with dep_xrandr = dependency('xrandr'), however it segfaults.

[rKsanu2MMYvypWePtQWM]

Hi, you seem to understand this better than me - I've tried to downscale from 4K to my native resolution, however it seems like currently it doesn't supersample at all - the pixelation remains the same if I run it at 1080p or at 2160p.

Your screenshots indicate the same.

So would it really need to use RSR, or would simple supersampling be sufficient? I remember back in 2016 on Windows I used to use AMD Virutal Super resolution, it made the image super smooth, and back then RSR definitely didn't exist.

[ruanformigoni]

Hi @rKsanu2MMYvypWePtQWM , I don't have extensive knowledge on this, I'm just enthusiastic :). Something like AMD virtual resolution would be great.

[rKsanu2MMYvypWePtQWM]

Perhaps the entire downscaling issue could be fixed by making sure it's getting downscaled properly? Because currently it doesn't seem to do it very well...

[specfreq]

I've done similar testing and came to the same conclusion: Gamescope simply does not have downscaling/supersampling support yet.

RetroArch is a perfect example of what kind downscaling options should be available for Gamescope, that's just due to the nature of the games being played over there. Essentially, RetroArch users want to run an era-appropriate resolution (320x240) with integer scaling but also render internally 2x, 3x, 4x, 8x, ect. This is not possible with Xrandr or Gamescope but absolutely could be added in.

An example in Gamescope: Halo Master Chief Collection, I'll load up Halo 2 Aniversary (classic). I want the render resolution set to 1920x1440 (3x) and then that should be scaled back down to my target of 640x480. There's no way to properly do this as of right now.

In Windows, AMD's Virtual Super Resolution (VSR) can do downscaling/supersampling but only to your monitor's native resolution, eg. 3200x1800 (VSR) downscales to 1920x1080 (native). I don't want to downscale only to native and Windows now hard caps the desktop resolution to 800x600 at the lowest.

Here's a video that shows exactly what this looks like and will give you an idea of why this feature is desirable:

https://youtu.be/FyiYgZkBi04

https://youtu.be/85lJcSlOi0g

[ruanformigoni]

Hi, I've implemented a bicubic downscaling algorithm (from this source) in PR #740. To test it:

git clone https://github.com/ruanformigoni/gamescope.git
cd gamescope
git checkout issue-692-bicubic

And then follow the instructions on the building section.

You can use the downscaler with the -D flag:

gamescope -D -W 2560 -H 1080 -w 5120 -h 2160 game-executable

[DisplayTalk]

I've tested your downscaling, it works pretty well.

couple example images taken on my 1920x1440p display downscaling from 3840x2880 with integer overscale on in retroarch.

original(outside gamescope, no scaling):

your bicubic:

gamescope's original nearest neighbor downscaling:

Pixel interpolation on the downscale indeed does reduce aliasing. There might be even better quality(zoom in on mario, you can see some uneven edges on bicubic) with lanzcos/sinc. I think integer downscaling is probably less bad for 3D graphics the higher the starting resolution on your display is. For 2d graphics it's fine(just so long as you don't use an integer scaling scanline shader, that won't work).

[specfreq]

I tried it out and it works pretty well!

I found a scaler that may be more appropriate than bicubic. Sharp-Bilinear, there's a link to it's GitHub in here and it appears to have integer scaling built in as well.

[DisplayTalk]

Sharp-Bilinear is for upscaling. Works by going to the nearest integer then finishing up with bilinear interpolation. It won't help here as the default for gamescope is already to do nearest-neighbor downscaling, which has almost identical effects to using sharp-bilinear(fractionally scaled output without pixel wobble, scaling artifacts in general, perfectly sharp). What the OP wants is a downscaling method that is softer than the current one(to hide aliasing further), not sharper than it. It's already as sharp as it can possibly get on the default -i mode(which is on without the -i flag unless your default config is using -Y or -U). This all said having Sharp-Bilinear added to gamescope would be pretty awesome.

[specfreq]

Thanks for correcting me. I don't know a whole lot about this stuff yet.

[DisplayTalk]

I think all that's needed here is an optional smoothing filter(0-100 scale with 100 being smoothest/blurriest) like the one included with Nvidia's DSR and DLDSR on windows.

[specfreq]

There's an issue with downscaling while using fullscreen with a target resolution that's below the monitor's native resolution.

Here's my launch options:
gamemoderun gamescope -D -f -W 640 -H 480 -w 2560 -h 1920 -- %command%

When you are not fullscreen this seems to be working correctly. However, when you go into fullscreen, it seems to be scaling 2560x1920 to 1920x1080(native) instead of my target of 640x480.

I also tried the launch options -D and -i and it does not do downscaling with integer scaling together.

Here's some images:
https://imgsli.com/MTUyODA0/0/1
https://imgsli.com/MTUyODQ5

There is a lot of what looks like texture aliasing or bright/dark pixels that don't seem to be an average of the samples from the higher res image. We may have a better result with box sampling or somthing other than bicubic. You can go through the extra trouble of getting all these samples but some of them are getting ignored because now there are too many samples for bicubic sampling in some cases.

https://en.wikipedia.org/wiki/Image_scaling#Box_sampling

https://community.topazlabs.com/t/downscale-using-perceptually-based-downscaling-of-images/28675

[DisplayTalk]

Doesn't happen to me. Make sure your monitor is set to 640x480 from display settings before running that.