Hi,
I developped Upscaler for Windows, iOS, Android and Xbox, and I am wondering if you would be interested into adding this feature for your application which seems to target Switch (but not only?).
Here is an exemple of how it looks like:
moonlight-stream/moonlight-qt#1557
The feature makes sense if you stream at a lower resolution than the TV and you want to get an upscaled and clear picture. For instead streaming a 1080p on a 4K TV.
I don't have a switch, and I don't know if there is a proper way to emulate this code on PC.
My guess is that Switch would probaly use shader like FSR1, NIS or SGSR1 which are available as GLSL.
I see 3 different paths to investigate:
-
SGSR1: Not the best quality, but still a lot better than a classic bilinear or bi-cubic scale up. It is a 1-Pass RGB shader, shaped for mobile, very fast to compute thus. Probably the easiest way to implement. The custom version I made sgsr1_shader_mobile_edge_direction.frag is 1-Pass RGB and doesn't require any parameter from the code. Made by QUALCOMM, it is a cross-platform shader.
-
NIS: Decent Quality, still 1-Pass RGB but with few parameters to get via NIS functions. Made by NVIDIA, it is a cross-platform shader.
-
FSR1: The best quality of the three (just don't expect something like DLSS4...). A bit more tricky to get it done as it uses 2-Pass RGB (Upscaler EASU then Sharpener RCAS). I only use HLSL version (d3d12va_shaders.cpp#L1375), but it probably works as GLSL using this FSR1 Mod. Made by AMD, it is a cross-platform shader.
By reading the code, it looks like Moonlight-Switch is not only for Switch but also for other platforms like iOS, Android, Mac.
If my assumption is correct, here is my suggestion:
-
deko3d => Use NIS as a Switch is relying on NVIDIA Hardware, NIS is from NVIDIA, it has few performance optimizations. NIS Sample code d3d12va_shaders.cpp#L1086. Swith to SGSR1 if NIS adds 5ms latency. SGSR1 Sample code GlesPassthroughBridge.java#L279.
-
OpenGL => Use NIS. Sample code: d3d12va_shaders.cpp#L1086.
-
Metal => I upscale with MetalFX (seems to be an AMD AMF technology on the background, better than Shader as it seems to us Machine Learning Model). MetalFX works for Mac and iOS. It can be implemented in 2 different ways, Upscale YUV with Luma via MetalFX and Chroma via Bilinear (this is the method I use for Mac), or upscale RGB via MetalFX. Both are fast with similar result, the YUV method shall be slightly faster but it makes the code more conplex to maintain. I advise to use RGB method, which is the one I use for iPhone, I tested on a iPhone 13 Pro and it is very fast, I actually could not see any additional latency).
Regards,
Hi,
I developped Upscaler for Windows, iOS, Android and Xbox, and I am wondering if you would be interested into adding this feature for your application which seems to target Switch (but not only?).
Here is an exemple of how it looks like:
moonlight-stream/moonlight-qt#1557
The feature makes sense if you stream at a lower resolution than the TV and you want to get an upscaled and clear picture. For instead streaming a 1080p on a 4K TV.
I don't have a switch, and I don't know if there is a proper way to emulate this code on PC.
My guess is that Switch would probaly use shader like FSR1, NIS or SGSR1 which are available as GLSL.
I see 3 different paths to investigate:
SGSR1: Not the best quality, but still a lot better than a classic bilinear or bi-cubic scale up. It is a 1-Pass RGB shader, shaped for mobile, very fast to compute thus. Probably the easiest way to implement. The custom version I made sgsr1_shader_mobile_edge_direction.frag is 1-Pass RGB and doesn't require any parameter from the code. Made by QUALCOMM, it is a cross-platform shader.
NIS: Decent Quality, still 1-Pass RGB but with few parameters to get via NIS functions. Made by NVIDIA, it is a cross-platform shader.
FSR1: The best quality of the three (just don't expect something like DLSS4...). A bit more tricky to get it done as it uses 2-Pass RGB (Upscaler EASU then Sharpener RCAS). I only use HLSL version (d3d12va_shaders.cpp#L1375), but it probably works as GLSL using this FSR1 Mod. Made by AMD, it is a cross-platform shader.
By reading the code, it looks like Moonlight-Switch is not only for Switch but also for other platforms like iOS, Android, Mac.
If my assumption is correct, here is my suggestion:
deko3d => Use NIS as a Switch is relying on NVIDIA Hardware, NIS is from NVIDIA, it has few performance optimizations. NIS Sample code d3d12va_shaders.cpp#L1086. Swith to SGSR1 if NIS adds 5ms latency. SGSR1 Sample code GlesPassthroughBridge.java#L279.
OpenGL => Use NIS. Sample code: d3d12va_shaders.cpp#L1086.
Metal => I upscale with MetalFX (seems to be an AMD AMF technology on the background, better than Shader as it seems to us Machine Learning Model). MetalFX works for Mac and iOS. It can be implemented in 2 different ways, Upscale YUV with Luma via MetalFX and Chroma via Bilinear (this is the method I use for Mac), or upscale RGB via MetalFX. Both are fast with similar result, the YUV method shall be slightly faster but it makes the code more conplex to maintain. I advise to use RGB method, which is the one I use for iPhone, I tested on a iPhone 13 Pro and it is very fast, I actually could not see any additional latency).
Regards,