Skip to content

Commit 9dd96d0

Browse files
committed
Port Video Super Resolution (MetalFX) from upstream PR moonlight-stream#704
Adds an optional MetalFX spatial upscaler to the tvOS video pipeline, ported from upstream PR moonlight-stream#704 with tvOS-specific fixes and hardening. Off by default; when enabled it decodes on a dedicated queue, converts YUV->RGB via Core Image, and MetalFX-upscales the stream to the display size before enqueuing to the display layer. Only engages when the display is larger than the stream (e.g. 1080p -> 4K); at native 4K it is bypassed. tvOS integration the PR omitted / got wrong: - Add the "Video Super Resolution (MetalFX)" toggle to Moonlight TV Settings.bundle (key videoSuperResolution, default off). Without this the feature was permanently off on Apple TV as shipped. - Skip the PR's iOS-only SettingsViewController + storyboard changes. Hardening over the raw PR: - +[VideoSuperResolution isDeviceSupported] gates the feature on actual MetalFX MTLFXSpatialScaler support so unsupported GPUs fall back to the normal pipeline instead of dropping every frame (black screen). - Disable VSR for the session when the display is not larger than the stream, or when scaler configuration fails, instead of running a pointless second decode/convert or dropping frames. - Runtime per-frame fallback: after repeated failures, revert to direct enqueue and request a fresh IDR (avoids mixing decoded-RGB and mid-GOP compressed samples on one layer). - Wait for GPU completion before handing the upscaled buffer to the display layer (runs on the VSR queue, not main, so no input-latency cost) to fix a tearing / pixel-buffer-pool recycle race. - Make the HDR reconfigure path serialize on the VSR queue without a queue->main->queue deadlock against stop(). MetalFX auto weak-links (introduced tvOS 16 > deployment target), so no extra linker flags are needed. Private-API-free; fork/self-signed only is not a concern here (this is standard framework usage). Claude-Session: https://claude.ai/code/session_01QiVtyqxowFKZc8gKnNjevy
1 parent a1c78dd commit 9dd96d0

8 files changed

Lines changed: 1143 additions & 11 deletions

File tree

Limelight/Database/TemporarySettings.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@
3131
@property (nonatomic) BOOL playAudioOnPC;
3232
@property (nonatomic) BOOL optimizeGames;
3333
@property (nonatomic) BOOL enableHdr;
34+
@property (nonatomic) BOOL videoSuperResolution;
3435
@property (nonatomic) BOOL btMouseSupport;
3536
@property (nonatomic) BOOL absoluteTouchMode;
3637
@property (nonatomic) BOOL statsOverlay;

Limelight/Database/TemporarySettings.m

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ - (id) initFromSettings:(Settings*)settings {
4040
self.useFramePacing = [[NSUserDefaults standardUserDefaults] integerForKey:@"useFramePacing"] != 0;
4141
self.playAudioOnPC = [[NSUserDefaults standardUserDefaults] boolForKey:@"audioOnPC"];
4242
self.enableHdr = [[NSUserDefaults standardUserDefaults] boolForKey:@"enableHdr"];
43+
self.videoSuperResolution = [[NSUserDefaults standardUserDefaults] boolForKey:@"videoSuperResolution"];
4344
self.optimizeGames = [[NSUserDefaults standardUserDefaults] boolForKey:@"optimizeGames"];
4445
self.multiController = [[NSUserDefaults standardUserDefaults] boolForKey:@"multipleControllers"];
4546
self.swapABXYButtons = [[NSUserDefaults standardUserDefaults] boolForKey:@"swapABXYButtons"];
@@ -80,6 +81,7 @@ - (id) initFromSettings:(Settings*)settings {
8081
self.useFramePacing = settings.useFramePacing;
8182
self.playAudioOnPC = settings.playAudioOnPC;
8283
self.enableHdr = settings.enableHdr;
84+
self.videoSuperResolution = [[NSUserDefaults standardUserDefaults] boolForKey:@"videoSuperResolution"];
8385
self.optimizeGames = settings.optimizeGames;
8486
self.multiController = settings.multiController;
8587
self.swapABXYButtons = settings.swapABXYButtons;

Limelight/Stream/StreamManager.m

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@ - (BOOL) resumeApp:(HttpManager*)hMan receiveSessionUrl:(NSString**)sessionUrl {
149149

150150
- (NSString*) getStatsOverlayText {
151151
video_stats_t stats;
152+
BOOL videoSuperResolutionEnabled = [[NSUserDefaults standardUserDefaults] boolForKey:@"videoSuperResolution"];
152153

153154
if (!_connection) {
154155
return nil;
@@ -179,11 +180,13 @@ - (NSString*) getStatsOverlayText {
179180
}
180181

181182
float interval = stats.endTime - stats.startTime;
182-
return [NSString stringWithFormat:@"Video stream: %dx%d %.2f FPS (Codec: %@)\nFrames dropped by your network connection: %.2f%%\nAverage network latency: %@%@",
183+
NSString* videoEnhancementString = videoSuperResolutionEnabled ? @"\nVideo Enhancement: MetalFX" : @"";
184+
return [NSString stringWithFormat:@"Video stream: %dx%d %.2f FPS (Codec: %@)%@\nFrames dropped by your network connection: %.2f%%\nAverage network latency: %@%@",
183185
_config.width,
184186
_config.height,
185187
stats.totalFrames / interval,
186188
[_connection getActiveCodecName],
189+
videoEnhancementString,
187190
stats.networkDroppedFrames / interval,
188191
latencyString,
189192
hostProcessingString];

0 commit comments

Comments
 (0)