Compare commits

..
Author SHA1 Message Date
spoopyghosty0 8167cb86c6 Merge the shared hardware video decoder (PR #128, adapted: per game, off switch, VP9 fixes)
# Conflicts:
#	agent/frameport_agent.py
2026-10-10 02:25:41 -04:00
spoopyghosty0 44bc689348 Shared video decoder (codec revision 8): hidden VP9 frames no longer end the EOS drain early
Iris returns an empty capture buffer (bytesused 0, no LAST flag, time 0) for every VP9 frame that isn't shown
(alt-ref frames split from superframes). FFmpeg's V4L2 wrapper takes any empty capture buffer during a drain
for the end of the stream, so the first hidden frame decoded after EOS ended it and the pictures still in the
driver were lost (two-pass VP9 4K: 573/600 and 617/640, always the last frames); outside a drain the empty
buffers were returned as pictures, which the component then guessed away by counting hidden frames.
build.py now patches v4l2_context.c to requeue empty capture buffers without LAST/ERROR; the drain ends at the
LAST buffer, or after a skipped empty buffer at one silent second, so it can't block. The component's
hidden-frame counting is removed. Dev Frame (Batman's container): two-pass VP9 4K 600/600 and 640/640 in
hardware with the same Y hashes as OMX.google.vp9.decoder (seek phase too), VP9 without hidden frames,
H.264 4K and HEVC 4K 600/600, VP9 surface output 600/600. Two builds (different dirs and NDK copies) give the
same libstagefrighthw.so 141de01f.
2026-10-10 02:22:48 -04:00
spoopyghosty0 007db344d7 Shared video decoder: remove a game's old per-game codec folder once its launcher is converted, also when the recipe (not deployment.json) asks for hardware decoding (seen with Batman on the dev Frame) 2026-10-10 01:01:33 -04:00
spoopyghosty0 3fb2621030 Shared video decoder: per game (frame.hw_video_decode) with a global off switch; 8K retry at start and VP9 fixes from the hw-video-decode branch
- Opt-in per game: only launchers of games whose recipe has frame.hw_video_decode (install stage, no APK change;
  suggested from analysis media_codec, ANALYSIS_VERSION 8) get the codec line (deployment.json hw_video_decode,
  finalize + upgrade_launchers). Games that had agent 70's per-game codec (Batman) keep it; their old
  <base>/frameport-codec folders are removed afterwards. Batman: catalog + migration batman_video_patches
  (frame.hw_video_decode + the hidden adapter setting surface_native, no more package check); 4XVR catalog.
- Off switch: Settings -> Installing "Hardware video decoding" (library video.hw_decode, default on) -> agent
  video_codec_switch (video-codec/disabled, checked by the launcher line and the wrapper); FRAMEPORT_NO_HW_VIDEO=1
  for one game.
- Connection: the codec is checked once per connection, also after a failed install; a Frame keeps the same or a
  newer revision built elsewhere (logged) instead of flip-flopping between PCs.
- Wrapper: the merged media_codecs.xml goes to $XDG_RUNTIME_DIR/frameport-video, never into the version directory.
- Decoder: refused (busy) Iris sessions retry until 20 s after the plugin loaded (SteamVR's link holds a session
  ~12 s at game start), else 2 s, before the software fallback; VP9 capped at 4096x2304 (XML + component, larger
  decodes in software); hidden VP9 frames' pictures dropped. Manifest revision 7.
- build.py builds from paths with spaces and from freshly extracted FFmpeg sources. Artifacts rebuilt here, identical
  from two trees (ext4 and the NTFS repo path, two NDK copies); PR #128's own sources rebuild to its binary.
- AGENT_VERSION 71 (matches the connection's gate); triage hw-video-decoder-busy, PLAYBOOK row, docs.
2026-10-10 00:58:24 -04:00
spoopyghosty0 39ca9a5f0d Catalog: Audica works (shared working config)
Closes #126
2026-10-10 00:28:39 -04:00
spoopyghosty0 e64e8a95e0 Uninstall dialog: option to also delete the game's saves and everything it stored on the Frame (mods, downloads); they survived every uninstall before (GitHub #130) 2026-10-10 00:28:37 -04:00
spoopyghosty0 69e888c7fe Install on this PC works on Windows again: the agent (loaded for its VDF code) imported the POSIX-only fcntl at the top (ModuleNotFoundError for every PC VR install; GitHub #131) 2026-10-10 00:28:36 -04:00
spoopyghosty0 ce809c432d Merge PR #128: shared hardware video decoding (Lucas-Mathieu) 2026-10-10 00:23:42 -04:00
Lucas-Mathieu 26cee1551e Deploy shared hardware video decoding for compatible Lepton apps 2026-10-10 03:36:56 +02:00
spoopyghosty0 f812ea8716 Codec wrapper (PR #96): find the host's podman when Lepton calls it with the Android guest's PATH (its boot-wait, app-pid and logcat-mirror podman exec calls failed, so launch.log lost logcat and the container could be stopped early) 2026-10-09 17:54:44 -04:00
spoopyghosty0 3dc0a95379 Catalog: Doom3Quest uses frame.gl_multiview_fbo (HUD/PDA shown, no frame-rate drop: Xandrix1987's headset test, GitHub #77) 2026-10-09 16:30:33 -04:00
spoopyghosty0 f8244f582d Catalog: cubism works (shared working config)
Closes #121
2026-10-09 16:29:36 -04:00
spoopyghosty0 69fbdfdc4a SUPERHOT VR (Quest): create its cloud save folder at install, so the launcher gives it group write permission before the very first start (the game creates and checks it in the same millisecond; GitHub #120); triage text says to start again 2026-10-09 16:29:34 -04:00
spoopyghosty0 ef3272a31b Catalog: Vader Immortal III's campaign intro starts black by design (owner) 2026-10-09 15:04:18 -04:00
spoopyghosty0 b0f3ef2698 Catalog: Vader Immortal Episodes II and III work (owner's headset test) with Episode I's Unreal fixes (quest precompile, key map, thumb touch, pose_time_fix), all found in their code by the heuristics 2026-10-09 14:37:11 -04:00
spoopyghosty0 1711e7f7d9 Docs: session triage (PLAYBOOK rows for the session findings, CLAUDE.md note incl. the pac_hints survey) 2026-10-09 14:22:22 -04:00
spoopyghosty0 ff6cdf3b6a Triage real play sessions: the connection refresh fetches a finished session's log (agent v70) and triages it (launch-test signatures + new space-warp-used question and gpu-hang, find_rp_state for unreal-msrtt-crash, slow-frames from FrameBridge pacing, focus-dips), stored as last_session; game page "Last session" callout; value suggestions (adapter.key=value); FrameBridge-only fixes applied live via set_settings; CLI frameport session [--apply] 2026-10-09 14:22:20 -04:00
spoopyghosty0 28517a1999 Agent v70: session_log (the newest play session's launch.log sliced to 4 MB + its crash logcat + kernel GPU hang lines), last_play in list_installed, launch tests mark their session in plays.log 2026-10-09 14:22:17 -04:00
spoopyghosty0 af9c207ddf FrameBridge: always log the runtime's focus losses and returns with their length (focus: lost / back after N ms), before focus_hold hides a dip; for session triage 2026-10-09 14:03:23 -04:00
spoopyghosty0 88045bcf4e Merge remote-tracking branch 'origin/main' into merge/pr96 2026-10-09 13:43:24 -04:00
spoopyghosty0 5147710fad Merge Batman cutscene playback (hardware HEVC decoding, stereo composition) by Lucas-Mathieu (GitHub #96)
Adapter rebuilt here; the codec plugin rebuilt here with native/hevc/build.py (NDK r27c, Lepton SoftOMX fingerprint 456e912c) is byte-identical to the contributed one (c1c2a73e). Agent version 70 -> 69 (main was at 68).
2026-10-09 13:43:22 -04:00
spoopyghosty0 9c45cff09a Heuristics: suggest pose_time_fix for Unreal 4 games with Vader Immortal's Oculus input (the unreal_thumb_touch code match). Catalog: Phantom, Robo Recall and Time Stall get frame.unreal_thumb_touch + pose_time_fix (their code matches Vader's; not yet checked in these games), Star Wars: Tales gets frame.unreal_thumb_touch instead of proximity_emul (as Vader) 2026-10-09 13:43:07 -04:00
Lucas-Mathieu b4848c0424 Harden codec deployment and isolate native video hooks 2026-10-09 19:24:51 +02:00
spoopyghosty0 3a46b78361 Catalog: Clockwork (Quest), Beat Saber, Eleven: Table Tennis VR and Pistol Whip (PC VR, run directly without Revive) from shared working configs
Closes #116, closes #117, closes #118, closes #119
2026-10-09 13:21:50 -04:00
spoopyghosty0 75bbc6eb5f Heuristics: suggest scene_emul for every game that declares Meta's USE_SCENE permission, not only mixed-reality-only ones (VR HOT's room setup retried forever: LoadSceneModel failed / no XR_FB_spatial_entity_query; with scene_emul the room setup returned without errors). Catalog recipes still win (6 verified games declare USE_SCENE and run without it) 2026-10-09 13:06:44 -04:00
Lucas-Mathieu da638aea99 Merge remote-tracking branch 'origin/main' into fix/batman-cutscene-playback
# Conflicts:
#	agent/frameport_agent.py
#	artifacts/SHA256SUMS
#	artifacts/arm64-v8a/libopenxr_loader_generic.so
#	artifacts/armeabi-v7a/libopenxr_loader_generic.so
#	native/adapter/frame_adapter.c
2026-10-09 18:29:12 +02:00
spoopyghosty0 7c37b479db Agent v68: launch tests count their window from Lepton's 'Waiting for app' (up to 240 s more for boot + install). The first start after an APK change installs the app first; a 45 s window from the launcher stopped the container mid-install and left a broken installed APK ('base.apk is not zip') that never started again (VR HOT) 2026-10-09 11:58:54 -04:00
spoopyghosty0 fa473c5ada frame.unreal_thumb_touch: UE4 OculusInput's ThumbUp from the capacitive touches instead of near-touch, which the Frame never reports (thumbs always pointed up). Three instructions in SendControllerEvents (masks 0x2/0x8 -> 0x0f00/0x000f, NearTouches load -> Touches), matched exactly; suggested where the code matches (analysis unreal_thumb_touch, ANALYSIS_VERSION 7): Vader Immortal Ep. I, Robo Recall, Phantom: Covert Ops, Time Stall, Star Wars: Tales. Vader's recipe uses it instead of proximity_emul (the binding didn't animate the thumbs). Found by Klownicle, GitHub #49 2026-10-09 11:35:36 -04:00
spoopyghosty0 1e6ce04e4f pose_time_fix: never move a recent predicted display time; monotonic times only told apart when the clocks are more than 4 display periods apart and the time is clearly nearer the monotonic now (GitHub #49: on a Frame with XrTime only ~68 ms ahead of CLOCK_MONOTONIC, a request at the frame's display time after a hitch was taken for a monotonic time and moved +67.7 ms). Found by Klownicle, GitHub #49 2026-10-09 11:35:35 -04:00
spoopyghosty0 5cd5b119e7 Lint: extraneous parentheses (ruff UP034) 2026-10-09 11:33:00 -04:00
spoopyghosty0 186138cc3c Game page: a patch the build left out because the OVRPort runtime already has the fix (build.superseded, e.g. haptic_fix with runtime 3.4.3-aa54c3f) no longer counts as a settings change: 'Update on Frame' stayed after every update (owner's VR HOT, Klownicle's Vader, GitHub #49) 2026-10-09 11:32:40 -04:00
spoopyghosty0 98c7ca980a Triage: Lepton's own short lines survive the game-process filter. With the package known, game_lines dropped every line of 3 words or fewer, so Lepton's 'Boot complete!' vanished and the transient 'is not a running context' failed launch tests in the GUI although the games ran (owner's runs: Come Closer, AntiZeroGames CH, SKYBOX; Klownicle's Vader, GitHub #49). Only logcat lines are filtered by pid now 2026-10-09 11:29:12 -04:00
spoopyghosty0 b2470b962a Catalog: Max Mustard is listed by its name, not its package name Matilda (GitHub #110) 2026-10-09 11:10:09 -04:00
spoopyghosty0 2a705bc6d1 frame.gl_multiview_fbo: single-view twins for multiview programs drawn into flat framebuffers (GitHub #77)
Mesa enforces OVR_multiview's rule that a draw's program declares as many views as the draw framebuffer has and drops the draw silently; Doom3Quest compiles every vertex shader with layout(num_views=2) and draws its HUD/PDA into 2D-texture FBOs, which stay black on the Frame.

native/glmv (libfpglmv.so, same length as libGLESv3.so): the engine library's dlopen string is rewritten to it and it becomes the first DT_NEEDED. It records stage sources at link time and, when a multiview program draws into a framebuffer without views, draws with a lazily built single-view twin (num_views layout blanked, gl_ViewID_OVR -> 0u; attribute locations, block bindings and uniform values copied), then rebinds the original. Opt-in and experimental (analysis gl_multiview_libs, ANALYSIS_VERSION 6), setting gl_mv_debug, triage gl-multiview-twin-failed.

Host tests: the rewriter on Doom3Quest's 19 shaders (GPL-3.0 fixtures) and glmv.c against a stand-in GL that applies Mesa's rule. Untested on the device.
2026-10-09 10:54:40 -04:00
spoopyghosty0 5d44831757 Agent v67: keep launch.log filling when Lepton's logcat mirror dies ('logcat: Unexpected EOF!' right after the game starts, about 1 launch in 50: Vader Immortal on Lepton 3.0.5, Under Cover on 2.8.14). launch.sh starts _logcat_keeper, which then reads the container's logcat itself (podman exec ... logcat -T 2000, up to 5 restarts) so the dashboard auto-hide and launch tests see the game; upgrade_launchers adds it to existing launchers 2026-10-09 10:14:57 -04:00
spoopyghosty0 456880501b PC VR: a catalog recipe verified with another build (different VR APIs) keeps the build's own launch arguments; Electron launchers rank below the game; agent v67 collects Unity's Player.log / output_log.txt / crash error.log for launch tests and diagnostics; triage unity-vr-init and unity-crash (GitHub #105) 2026-10-09 10:10:33 -04:00
spoopyghosty0 63f7ae12e1 Catalog: Vader Immortal: Episode I works (owner's headset test of the GitHub #49 fixes by Klownicle) 2026-10-09 10:06:44 -04:00
spoopyghosty0 fa69ffefd2 Vader Immortal: loading card, controls, lightspeed shaders, thumbs (GitHub #49)
Klownicle's five fixes for Vader Immortal: Episode I, reimplemented as FramePort patches (found by Klownicle,
GitHub #49; their prebuilt APK is not used):

- frame.unreal_quest_precompile: UVRUtils::GetQuestShaderPrecompilePercent returned 0.0 on its non-Quest branch, so
  the menu waited forever on the loading card; that branch now returns 1.0. Located by symbol, every instruction
  around it checked (bl IsRunningOnSantaCruz; tbz w0,#0; fmov s0,wzr; ldp; ret); other builds stay untouched.
- frame.unreal_quest_keymap: the RPOC key selector (shared by AddAxisMapping and AddActionMapping, so both mapping
  kinds were affected) picked the empty Gear VR key set for an Oculus HMD; the tbz to it becomes a nop. Located as
  the common callee of both functions, pattern-checked.
- frame.zink_shader_fix + native/zinkfix (libVkLayer_fp_shaderfix.so): a Vulkan layer under Zink for GLES games
  that applies zink_shader_fix entries (vk_shader_fix format) to the SPIR-V Zink generates; it adds itself to
  GraphicsEnv's debug layer list from a constructor (the engine library loads it first). zink_shader_dump=1
  captures modules. Vader's recipe carries the two lightspeed-shader fixes (12 OpStores each).
- adapter proximity_emul (FrameBridge): OVRPlugin's thumb/index proximity actions get bindings to the capacitive
  touch inputs when the runtime lacks XR_FB_touch_controller_proximity (OVRPort offers the extension anyway):
  generic replacement for the libUE4 ThumbUp patch.
- Vader's recipe: pose_time_fix + proximity_emul, status unknown (to test in the headset), min_app 0.12.1.

Analysis records unreal_quest_gates (ANALYSIS_VERSION 4). Star Wars: Tales from the Galaxy's Edge (same studio)
has neither gate (different engine build) and gets a catalog entry with pose_time_fix + proximity_emul.

Headless on the dev Frame (Lepton 3.0.5): the layer loads into Zink's instances, the proximity bindings are
accepted, and the game passes the loading card to the "guardian smaller than recommended, press any button" screen.
2026-10-09 09:20:15 -04:00
spoopyghosty0 f78bc82cc4 frame.vivox_audio_route: Vivox voice chat without Android 12 audio routing (GitHub #101)
Newer Vivox builds (Green Hell VR) call AudioManager communication-device methods (API 31) from com.vivox.sdk.AudioChangeListener without a version check; Lepton is Android 11 -> NoSuchMethodError. The methods that call them now return at once (in-place dex edit, Dex.return_early). Detected by analysis vivox_api31 (ANALYSIS_VERSION 4), triage vivox-api31. Verified headless: Green Hell starts, Vivox initialises, ~65-70 fps.
2026-10-09 09:09:24 -04:00
spoopyghosty0 dfacf6a206 FrameBridge: serve cube swapchains the runtime refuses (GitHub #107)
Budget Cuts Ultimate asks for a 2048x2048 cube swapchain (faces=6); the Frame's runtime has no cube layers and refuses it (-2), OVRPlugin carries on with no images and crashes in ovrp_EndFrame4 (memset). cube_standin (default on, GLES) serves it as a GL cube map in the game's context and drops its layers. Triage cube-swapchain-refused (supersedes unity-render-crash/native-crash). Verified headless: the game runs on at ~70 fps.
2026-10-09 09:09:23 -04:00
spoopyghosty0 b2f14c1e2e Catalog: SUPERHOT VR (Rift) keeps the build's own launch arguments (GitHub #105: the older Oculus + SteamVR build needs -vrmode OpenVR; the catalog recipe from the OpenXR build dropped it) 2026-10-09 08:49:53 -04:00
spoopyghosty0 03d7a311bf Catalog: Batman: Arkham Shadow gets sync_guard to test (GitHub #102: quit to Steam's waiting screen after the Frame runtime crashed in xrSyncActions, the input race Myst had) 2026-10-09 08:32:50 -04:00
spoopyghosty0 cd5119278b Catalog: The Light Brigade (works with issues)
Closes #114
2026-10-09 07:56:14 -04:00
spoopyghosty0 6ab3608a44 Catalog: Gravity Lab (works)
Closes #113
2026-10-09 07:56:06 -04:00
spoopyghosty0 c0539af368 Catalog: Does it Stack? (works with issues)
Closes #112
2026-10-09 07:56:02 -04:00
spoopyghosty0 c6363140f4 Catalog: Walkabout Mini Golf (works)
Closes #111
2026-10-09 07:55:58 -04:00
spoopyghosty0 187105b729 Catalog: Matilda (works)
Closes #110
2026-10-09 07:55:55 -04:00
spoopyghosty0 b983c0f11f Catalog: The Tale of Onogoro (works)
Closes #108
2026-10-09 07:55:52 -04:00
spoopyghosty0 c7cee33889 Catalog: Please Don't Touch Anything (works)
Closes #106
2026-10-09 07:55:49 -04:00
spoopyghosty0 d9d73b8aec Triage: Lepton 3.0.5's transient 'is not a running context' (printed while its container is still starting, then 'Boot complete!') no longer fails a launch test (container-not-started unless Boot complete!; VR4 ran at 72 fps on the dev Frame but was marked failed) 2026-10-09 07:55:24 -04:00
spoopyghosty0 48455b9b72 pose_time_fix: robust clock offset (the largest of the last ~2 s of xrWaitFrame samples; hitches made single samples dip by up to 2.5 s and pushed fixed times up to 2 s into the future), far-past requests located at the frame's display time, nothing moved before 8 samples. Headless survey of 54 games: no other game asks for head/views on the monotonic clock; UE4 OVRPlugin 1.89 games and The Room VR ask for XrTime ~0 every frame and work today, so the setting stays suggested for Unity built-in Oculus games only. BattleSisters on the Frame: every moved request now at 0.0 ms from the display time 2026-10-09 01:35:15 -04:00
spoopyghosty0 af798c0bf3 Catalog: BattleSisters entry parses again (my previous edit put colons into a plain YAML text block); written with catalog.to_yaml 2026-10-08 22:55:47 -04:00
spoopyghosty0 ed714c9d65 Catalog: BattleSisters plays with hands following the controllers (adapter pose_time_fix, owner's headset test) 2026-10-08 22:51:26 -04:00
spoopyghosty0 832b401630 Pose time fix (adapter pose_time_fix, pose_debug): OVRPlugin locates its "now" poses at the monotonic clock
BattleSisters' hands lagged behind the controllers (also with ovrp_hold_physics on or off). pose_debug=1 (new,
FrameBridge: per 5 s and per located space, the requested time minus the predicted display time) showed on the dev
Frame (SteamOS 0.4.5): XrTime runs 2.56 s ahead of CLOCK_MONOTONIC, the hand spaces were located at -2564 ms (the
monotonic "now" passed on as an XrTime: OVRPort's dispatcher converts XR_KHR_convert_timespec_time 1:1 and
FrameBridge's emulation is never asked) and the head at XrTime 0.1 s (~1450 xrLocateViews/xrLocateSpace per 5 s).

pose_time_fix (default off; suggested for Unity built-in OVRPlugin games): in xrLocateSpace(s)/xrLocateViews a time
nearer the monotonic clock than XrTime's "now" is moved by the offset measured at xrWaitFrame, one more than 0.5 s
before the display time goes to "now". Headless with it on: every far-past request moved, located times -9.8..0 ms
from the display time. Host unit test for the time rules; PLAYBOOK row.
2026-10-08 22:41:34 -04:00
spoopyghosty0 114b7cb4d6 Catalog: BONELAB works on 1.2068 and 1.2974 (owner's headset test of frame.slz_vulkan_hooks) 2026-10-08 22:08:29 -04:00
spoopyghosty0 a4d3ce5e00 Merge BONELAB 1.2974 fix by UnicronNL (GitHub #82)
frame.slz_vulkan_hooks: Stress Level Zero's plugin no longer intercepts Unity's Vulkan start-up and samplers (it called an invalid function right after OVRPlugin's pre-init instance was destroyed). Test fixture rebuilt by us, byte-identical. Owner's headset test: BONELAB 1.2974 plays well.

Closes #82

# Conflicts:
#	docs/GAMES.md
#	tests/test_patches.py
#	tests/test_triage.py
2026-10-08 22:07:54 -04:00
spoopyghosty0 b3095b110a Merge Sniper Elite VR fixes by UnicronNL (GitHub #81)
ovrpshim frame-begin gate and held-back physics update (per-game settings ovrp_begin_gate / ovrp_hold_physics, on in Sniper Elite VR's recipe), xrshim PCM vibrations, unity_oculus_check revision 5; artifacts rebuilt by us. Owner's headset test: Sniper Elite VR plays, Accounting+ unchanged, BattleSisters unaffected (settings off).

Closes #81

# Conflicts:
#	artifacts/SHA256SUMS
#	docs/GAMES.md
2026-10-08 22:06:18 -04:00
spoopyghosty0 fb284b6fc4 Catalog: Star Wars Pinball VR (works; frame.unreal_gl_shim + frame.unreal_ovrp_entrypoints + frame.pac_hints, needs FramePort 0.12.1)
Closes #83
2026-10-08 21:56:46 -04:00
spoopyghosty0 c0df52b981 Unpaired pointer-authentication hints (frame.pac_hints, GitHub #83): with VR running, Star Wars Pinball VR died with SIGILL ILL_ILLOPN at an autiasp (*pc=0xd50323bf) on Unreal's HttpManager thread: its libUE4.so has 38 paciasp but 40 autiasp (OpenSSL's Poly1305 NEON assembly). Quest CPUs run PAC hints as NOPs, the Frame's CPU checks them. In a library whose counts differ, every paciasp/autiasp becomes a NOP in place (libraries using retaa/pacibsp forms are left alone). Suggested by the new triage signature pac-unpaired 2026-10-08 21:50:01 -04:00
spoopyghosty0 a1b0a3f350 Unreal OVRPlugin stand-ins (frame.unreal_ovrp_entrypoints, GitHub #83): Unreal's Oculus module dlsyms every ovrp_* function of the OVRPlugin it was built against and ANDs the results; OVRPort's OpenXR OVRPlugin lacks ovrp_GetPTWNear (Star Wars Pinball VR, UE 4.25 / OVRPlugin 1.44), so the wrapper failed, OculusHMD never pre-initialised and the game crashed without an HMD. Analysis records the engine library's ovrp_* names (extra.unreal_ovrp_lookups, ANALYSIS_VERSION 3); the patch compares them with the shipped libOVRPlugin.so and adds a generated libfp_ovrpstubs.so (DT_NEEDED of the plugin: dlsym on its handle searches its dependencies) whose stand-ins return ovrpFailure (stubgen result=). libUE4.so stays byte-identical. Suggested when the newest OVRPort runtime's plugin (tools.overport.runtime_lib) lacks a looked-up name (bundled fallback before a runtime is downloaded) 2026-10-08 21:33:06 -04:00
spoopyghosty0 df864e9fed GL shim keeps Unreal's multiview (GitHub #83): Unreal 4.25 only enables mobile multiview when GL_OVR_multiview, GL_OVR_multiview2 and GL_OVR_multiview_multisampled_render_to_texture are all listed (FOpenGLES::ProcessExtensions), so hiding the last one with gl_hide_msrtt switched multiview off. For Unreal (libUE4.so/libUnreal.so loaded, MSRTT hidden, multiview kept) the shim keeps that extension visible and maps glFramebufferTextureMultisampleMultiviewOVR to the single-sampled glFramebufferTextureMultiviewOVR (log 'multiview multisampled render-to-texture (N samples) drawn single-sampled'). Unity/other engines unchanged. Previous source rebuilt byte-identical before the change 2026-10-08 21:26:30 -04:00
spoopyghosty0 6595a31048 ovrpshim: frame-begin gate and held-back physics update only with the per-game settings ovrp_begin_gate / ovrp_hold_physics (default off; Sniper Elite VR's recipe turns both on). With them BattleSisters' hands lagged and its loading screen stuttered (owner's headset test); Accounting+ was fine. libfp_ovrp.so rebuilt 2026-10-08 21:13:28 -04:00
spoopyghosty0 f4bcb658e1 Unreal GL shim (frame.unreal_gl_shim, GitHub #83): Star Wars Pinball VR crashes ~3 s in with fault addr 0x10000 on the RHIThread in libgallium_dri.so: Unreal's mobile MSAA renders through multisampled render-to-texture and Zink's find_rp_state indexes rendering_state_cache[6] one past its end. The GL shim hides MSRTT (gl_hide_msrtt); it now also keeps multiview for Unreal (libUE4.so/libUnreal.so, like Unity). Offered for Unreal GLES games, suggested by the new triage signature unreal-msrtt-crash, not from the APK alone. Old glshim source rebuilt byte-identical before the change 2026-10-08 20:11:40 -04:00
spoopyghosty0 e4f70144e9 Unity split-binary builds expect an OBB too (GitHub #92): analysis extra.unity_split → expects_obb, ANALYSIS_VERSION 2
analysis/unity_split.py: an XR-plugin build (Oculus XR Plugin / Unity OpenXR library) without assets/bin/Data/UnitySubsystems/ in the APK (zip names only), or BuildSettings listing more scenes than the APK has levelN files (globalgamemanagers loose, or its node read from the head of data.unity3d: only the blocks up to it). Exact on the 48 Unity APKs of both dump folders: all 13 split builds flagged, none of the 17 full builds (whose OBBs are asset bundles or sound banks). The missing-OBB warning and launch-test finding now cover them; existing entries are re-analysed at the next start.
2026-10-08 16:39:12 -04:00
spoopyghosty0 020334b37b OBBs found by file name: (main|patch).<versionCode>.<package>.obb next to the APK, in an obb folder beside an apk folder, or in a SideQuest backup (GitHub #85, #91)
quest_dump.find_data: after the folder-layout checks, search the APK's folder, its parent and up to 3 levels below each (folders with their own APKs are other games) for the package's expansion files; the folder with the APK's own versionCode wins, per kind the newest file not newer than it, else any version. When that folder holds other things too (the APK, other games' OBBs), the entry gets data_files and only those files are uploaded, counted and deleted with the game; no language-pack/content-file lookup there. Both dump folders scan identically (62/47 games).
2026-10-08 16:39:11 -04:00
spoopyghosty0 24a98ab263 Linux apps use their own icon (agent v64): Desktop Mode entry, Steam shortcut, library
The agent copies an AppImage's .DirIcon / the bundled .desktop file's Icon= (hicolor, pixmaps; symlinks resolved
and kept inside the app folder) to <anchor>/artwork/app-icon.{png,svg} at finalize_linux and when menu entries are
refreshed, and uses it for the menu entry's Icon= and the Steam shortcut (PNG only) unless the user chose an icon
(artwork/.icon-source = custom, sent by the PC with every art upload). StartupWMClass comes from the app's .desktop
file. A PNG goes back to the PC and becomes the library icon when there is no pick or store icon (no .picked;
.app-icon marker); folder apps get it at add time.

Closes #99
2026-10-08 16:33:16 -04:00
spoopyghosty0 58563b99a5 Re-analyse library entries an older FramePort analysed (GitHub #104)
detect.ANALYSIS_VERSION (stored as analysis.extra.analysis_version, bumped
whenever analyze() gains a field): Quest/Android entries below it whose APK
still exists are analysed again in a background thread at the GUI's start
and before a build (pipeline.refresh_analyses). Only analysis and suggested
change; recipes the user didn't edit follow the new suggestion (e.g. an SDL
app gets frame.sdl_clipboard once sdl_java exists), user recipes, tags, art,
titles and builds stay. An unreadable APK keeps its entry and is marked
analysis_failed (not retried until the next bump); a missing one is tried at
a later start. Analyze again sets the version too. Rift/Linux entries skipped.
PLAYBOOK row + CLAUDE.md note.
2026-10-08 16:25:35 -04:00
spoopyghosty0 9013a7d13c Catalog: AgeOfJoy (works)
Closes #103
2026-10-08 16:14:38 -04:00
spoopyghosty0 f36a17b46a Catalog: Pistol Whip (works)
Closes #100
2026-10-08 16:14:36 -04:00
spoopyghosty0 7335ff2ce5 Catalog: VR HOT Quest (works)
Closes #98
2026-10-08 16:14:33 -04:00
spoopyghosty0 21c039f9b3 Catalog: Retronika (works)
Closes #97
2026-10-08 16:14:31 -04:00
Lucas-Mathieu 8814b1ce56 Merge remote-tracking branch 'origin/main' into fix/batman-cutscene-playback
# Conflicts:
#	src/frameport/pipeline.py
2026-10-08 17:49:12 +02:00
spoopyghosty0 3e8a480ea6 Android version / web wrapper: warnings only, never unsupported from the manifest alone (owner's call): the recipe notes, game page (warning callout) and CLI say what the manifest asks for; a launch test's triage (android-too-new, web-wrapper) decides 2026-10-07 23:08:36 -04:00
spoopyghosty0 ea2cd893b4 Linux apps in Desktop Mode's menu: per-app switch on the game page, docs
The game page's "What FramePort will do" for a Linux app gets a Desktop Mode switch ("In the menu and on the
desktop", default on): kept in the library entry (desktop_entry; patches don't apply to Linux apps), passed to
finalize_linux at install and applied at once on the Frame through the agent's desktop_entry command when the app is
installed. The agent side (menu + desktop .desktop files, launcher with FRAMEPORT_DESKTOP=1, refresh for existing
installs, removal on uninstall/purge) came with agent v63. Help text desktop_entry, INSTALL/FRAME_RUNTIME/PLAYBOOK,
CLAUDE.md. Game Mode's Steam Input problem from the report is separate.

Closes #84
2026-10-07 22:46:38 -04:00
spoopyghosty0 a90a8c27dc Install to and move games to a microSD card: Frame page Storage section, game menu Move to…
The Steam Frame page lists the Frame's drives (filesystem, free space, games on each; exFAT/NTFS cards say why they
can't hold games) and sets where new games go (library setting install.drive, default internal storage). Game menu
→ "Move to…" (Frame-installed games, agent v63) picks a drive in a dialog and queues a Frame job that follows the
agent's copy with progress (cancellable until the copy starts). The game page and the Frame page's installed list
show "on SD Card" / "SD Card not inserted"; a game whose card is out counts as installed, not "Not installed".
Help texts install_drive / move_game, ui_smoke --fake-frame drives + the Move dialog, docs (FRAME_RUNTIME "Where
FramePort keeps games", INSTALL, PLAYBOOK row, CLAUDE.md).

Closes #90
2026-10-07 22:46:37 -04:00
spoopyghosty0 9d0d72c40a Agent v63: games on another drive (microSD) and Desktop Mode entries for Linux apps
#90: `drives` lists internal storage, the drives under /run/media (from /proc/mounts) and the drives of Steam
library folders, with free space and whether they can hold games (vfat/exfat/ntfs can't: Lepton data and Proton
prefixes need Unix permissions and symlinks; read-only mounts neither). Games on a drive live in
<mount>/FramePort/<pkg>; the anchor stays internal, so the Steam shortcut never changes. prepare/prepare_pcvr/
prepare_linux take dest (free space from it); a drive that isn't mounted is an error, never a silent fallback to
internal storage, and an installed game keeps its folder. `move` (refused while the game runs, free-space check)
renames on the same drive, else copies with cp -a (in podman's user namespace when podman is there: container-owned
files), checks file count + bytes, retargets symlinks into the old folder (LibOVRRT redirect), points launch.sh at
the new folder (Quest: the app_dir line; Linux/PC VR: written again from the install record, which now keeps env and
game args) and rewrites deployment.json before removing the old copy; detached via systemd-run with a status file
(move_status). list_installed adds drive + drive_missing; launch refuses a game whose card is out; cleanup may remove
leftovers in a drive's FramePort folder (never an installed game); purge/uninstall remove an empty FramePort folder.
PC: install/drives.py (list, setting install.drive, move with progress, picks a running move up again), targets
pass dest for Quest/PC VR/Linux installs, CLI `frameport frame drives`, `frameport frame move <pkg> --to`,
`frameport install --dest`.

#84: finalize_linux writes ~/.local/share/applications/frameport-<slug>.desktop (+ an executable copy on ~/Desktop
when it exists; Exec quoted per the Desktop Entry spec, X-FramePort-Package marks ours) unless desktop_entry is off;
`desktop_entry` command switches it per app; ensure_host_fixes adds entries for existing Linux installs and drops
stale ones; uninstall/purge remove them. The Linux launcher with FRAMEPORT_DESKTOP=1 skips the Steam-parent
watchdog and Steam's display (upgrade_launchers updates existing launchers).
2026-10-07 22:46:36 -04:00
spoopyghosty0 56d2ac117f Missing OBB: analysis extra.expects_obb (Unreal's GameActivity.bHasOBBFiles, UE4 and UE5 names); a game that expects one but has no data folder gets an error callout on the game page, a question before install (default: skip) and a warning in frameport scan/install, and a launch test without frames gets the missing-obb finding from the library (GitHub #85: TRIANGLE STRATEGY was installed without its OBB and hung silently after OVRPlugin's JNI_OnLoad). In the dumps every Unreal game with the flag has its data folder 2026-10-07 22:42:41 -04:00
spoopyghosty0 cb8202d891 Scan: OBBs in more backup layouts (quest_dump.find_data_dir): a <package> folder with .obb files up to 3 levels below the APK's folder (obb/<package>/, OBB/, Android/obb/<package>/) or 2 below its parent, skipping neighbouring folders with APKs of their own (other games/versions); <game>/apk(s)/ + <game>/obb/<package>/ (SideQuest-style backups) is one game named after <game>. The data folder is always the one holding the .obb files (obb/<package>/ used to be sent as obb/ and land one level too deep). Both dump folders scan exactly as before 2026-10-07 22:42:40 -04:00
spoopyghosty0 029b0674d1 Web wrappers: a Trusted Web Activity (manifest meta-data android.support.customtabs.trusted.DEFAULT_URL, resolved through resources.arsc, or androidbrowserhelper's LauncherActivity) is analysis extra.web_wrapper with its URL; suggested status unsupported, the game page/CLI say to open the site in a browser (GitHub #86 Mahjong Table VR opened Meta's com.oculus.browser, which the Frame doesn't have). Triage web-wrapper 2026-10-07 22:42:39 -04:00
spoopyghosty0 c852473a43 Android version gate: analysis records the APK's minSdk (extra.min_sdk, read from the original: OVRPort lowers it to 29); above 32 the suggested status is unsupported and the game page/CLI say which Android it needs (GitHub #71/#72: minSdk 34 apps crash at start on android.window / VarHandle fences, Lepton runs Android 11). minSdk 32 stays allowed: 16 working catalog games declare it. Triage android-too-new 2026-10-07 22:42:38 -04:00
spoopyghosty0 acf1ef1da2 haptic_fix: keep revision 2 (owner's call, drop this commit to keep #81's bump)
The PCM conversion is preventive (Sniper Elite VR and Hyper Clinic send envelopes), and revision 3 would show
'Update on Frame' for every build with adapter.haptic_fix (suggested for every OVRPlugin game), including builds
where upstream.py already left the workaround out. Same reasoning as rect_clamp. New builds get the PCM conversion
anyway (the artifact is rebuilt).
2026-10-07 22:27:16 -04:00
spoopyghosty0 5fe120e897 Sniper Elite VR follow-up: merge the new haptic OOM PLAYBOOK row into the existing one (same OVRPort bug, now with the upstream-fix note); wrap unity_oculus_check's description to 120 columns (ruff); regenerate docs/GAMES.md 2026-10-07 22:25:43 -04:00
spoopyghosty0 41624c8dbd Rebuild libfp_ovrp.so and libframe_xrshim.so from the reviewed sources (native/build.py --only ovrpshim,xrshim, NDK r27c; the previous sources reproduce the committed artifacts byte for byte) 2026-10-07 22:23:19 -04:00
UnicronNL e8ec39db86 Sniper Elite VR: works (GitHub #81)
- native/ovrpshim: libunity.so's ovrp_BeginFrame/ovrp_EndFrame go through the shim; the frame wait waits <= 50 ms
  for the last waited frame to begin and skips otherwise (the game deadlocked at its Init scene when Unity skipped a
  begin); a begin/end of another index while a wait is outstanding gets the waited index. The physics-step
  ovrp_Update2 is held back (prediction 0 located the controllers in the past on the Frame: the hands lagged).
  frame.unity_oculus_check revision 5.
- native/xrshim: PCM vibrations (XrHapticPcmVibrationFB) become one plain vibration too; haptic_fix revision 3.
- Catalog: works with frame.unity_oculus_check + haptic_fix, min_app 0.12.1. Triage unity-frame-not-begun (info),
  PLAYBOOK rows and how to get all-thread backtraces with gdbserver64 inside the container, test.
2026-10-07 22:21:08 -04:00
spoopyghosty0 936d4b188b BONELAB: catalog note leads with the per-build status (GAMES.md showed only 'One recipe for both builds.'); regenerate docs/GAMES.md (also adds the missing Doom3Quest row) 2026-10-07 22:20:24 -04:00
UnicronNL e9ab1489cc BONELAB 1.2974: frame.slz_vulkan_hooks (GitHub #82)
Stress Level Zero's graphics plugin (libSLZQuestNative.so) hooks Unity's Vulkan start-up and vkCreateSampler; on
the Frame its vkCreateInstance wrapper calls an invalid pointer right after OVRPlugin's pre-init instance is
destroyed (pc 0, a jump into an unloaded library, or a hang). The new patch turns the plugin's two registrations into
no-ops (found through its exported functions, 4 bytes each), so Unity starts Vulkan itself; ~35 s of shader
prewarming without the plugin's pipeline cache, then 72 fps (headless). One BONELAB recipe serves 1.2068 and 1.2974.
Triage slz-vulkan-hook-crash, PLAYBOOK row, tests with a fixture built from tests/fixtures/src/fakeslz.S.
2026-10-07 22:15:57 -04:00
spoopyghosty0 1043eb1d1b Catalog: Richie's Plank Experience (works)
Closes #95
2026-10-07 22:13:33 -04:00
spoopyghosty0 0f444da595 Catalog: RUINSMAGUS (works)
Closes #89
2026-10-07 22:13:27 -04:00
spoopyghosty0 fda644d036 Catalog: GORN2 (works)
Closes #88
2026-10-07 22:13:19 -04:00
spoopyghosty0 fa0fbbd29a Catalog: Time Crisis VR (Experimental) (works)
Closes #80
2026-10-07 22:13:14 -04:00
spoopyghosty0 3446999109 Upstream fixes replace our workarounds per build (patches/upstream.py)
Each build probes OVRPort's output for registered upstream fixes and leaves out the workarounds they make
unnecessary (recipes unchanged: builds with an older runtime keep them). Logged ("not needed: ..."), recorded in
build.superseded, left out of settings.conf at install, shown on the game page and in `frameport show`;
FRAMEPORT_KEEP_WORKAROUNDS=1 turns it off. Registered: ovrport.haptic_envelope (-> adapter.haptic_fix) and
ovrport.microphone_stream (-> frame.ovr_microphone), both fixed in OVRPort runtime 3.4.3-aa54c3f (ovrport/app#73;
haptics verified in the headset with Lucky's Tale). Probes checked against both real runtimes; fixtures in
tests/fixtures/src/fakeruntime.S.
2026-10-07 21:02:12 -04:00
Lucas-Mathieu e9eedb2241 Merge remote-tracking branch 'origin/main' into fix/batman-cutscene-playback
# Conflicts:
#	artifacts/SHA256SUMS
2026-10-08 01:57:46 +02:00
Lucas-Mathieu d3a35c307a Fix Batman native cutscene decoding and stereo composition 2026-10-08 01:57:29 +02:00
spoopyghosty0 d6f87f79a6 Live view: status updates from the ticker/start/stream-end threads go through components.LoopUpdater (the GUI logged 'dropped a patch for unknown control' during the owner's test, the same cause the Monitor tab fixed) 2026-10-07 19:52:09 -04:00
spoopyghosty0 24a1ef9f1d Live view: less delay with sound. The -1 s video input offset (1080p fix) made ffmpeg release video in clumps (output gaps up to 550 ms: a longer, stuttering browser delay, worse with sound); -0.25 s keeps 1080p from stalling and the output steady (measured on the Frame with sound playing). Player: no speed-up while sound is on (Chrome keeps ~0.6 s of audio ahead and stalled 8x/30 s at 1.1x, 1-2x without; same ~1 s lag; muted ~0.3 s); the Sound on tooltip says so 2026-10-07 19:44:46 -04:00
spoopyghosty0 ae4904c7d9 Live view: 1080p no longer drops frames with sound on (ffmpeg paced its inputs against pulse's late audio and stopped reading the video for up to 0.7 s, blocking fp_venc's writes; -itsoffset -1 on the video input keeps it from counting as ahead). Found with fp_venc's new timing stats (write_ms 17-38 avg, 260 max at 1080p) and a busy 1080p replay into the real pipeline: 20 s took 34-47 s with sound, exactly 20 s with the fix; output A/V spans unchanged 2026-10-07 19:13:45 -04:00
spoopyghosty0 b142f7c0ac Live view: Stop ends a backed-up stream (fp_venc blocked in a write never saw stdin EOF and ffmpeg's output wasn't drained after the channel closed: both stayed on the Frame holding the headset view open, found after the owner's 1080p run). The script reports its process group; close() waits 2 s and then kills the group. Helper uploads use their own temp name (two starts raced on fp_venc.new). Verified on the Frame: forced kill path and the normal stop both leave nothing running 2026-10-07 18:37:11 -04:00
spoopyghosty0 934f356927 Live view: the panel warns when the Frame drops frames (encoder stats per 10 s: late + skipped slots >= 5 %, 'choose a lower quality'); fp_venc stats add per-window convert/write timing and no-free-buffer counts to find why 1080p stalled (owner: 1080p choppy, 360p smooth; log: 208 of 320 slots, 172 late, 104 skipped) 2026-10-07 18:09:15 -04:00
spoopyghosty0 a61764960d Live view: even frame timestamps and VBR after the first headset test
Owner's headset test (720p): heavy compression artifacts and many dropped-looking frames.
- Frames stamped on arrival (wall clock) bunched when ffmpeg read several at once (gaps of 0-10 ms and 45+ ms). The
  input stays on the wall clock (read in step with pulse's audio; setts alone held output ~7 s), and setts now
  numbers the output frames on fp_venc's fps grid: every gap exactly 1/fps on the Frame, audio aligned.
- fp_venc --peak: VBR capped at the peak (else CBR as before), level picked for the peak. The hardware path uses
  1.5x peaks and higher targets (360p 1.5, 480p 2.5, 720p 5, 1080p 8, Full 10 Mbit/s); 3 Mbit/s CBR blurred every
  head turn. The x264 fallback keeps its rates.
- fp_venc's info/stats lines go to the app log (late/skipped frames next time); its errors still explain a stop.

Verified on the Frame: probe with VBR (no refused control); end to end through livestream.start at 720p32: 403
frames all 31 ms apart, audio 0-12.57 s next to video 0-12.58 s, late viewer on its keyframe at once, nothing left
after stop. Picture quality with motion still needs the headset.
2026-10-07 18:00:59 -04:00
spoopyghosty0 55a376cdfd fp_venc: exact NEON fast paths for 3:2 (720p) and 3:1 (360p) scaling
The box boxes repeat every 3 source pixels at these ratios, so the averages become rounding shifts (3:2: block
sizes 1, 2, 4) or an exact multiply (3:1: x7282 >> 16 = /9 for every sum), with no per-pixel scalar loop. Same
bytes as the generic path; 480p (852 px wide, irregular) keeps the generic path. Self-test adds 1158x648 -> 432/216
(columns left over after the 48-pixel steps) and times all four qualities.

Frame (2026-10-07): selftest ok (184 cases); 1920x1080 -> 720: 1.3-1.65 ms (was 5.6), -> 360: 0.7 ms,
-> 1080: 0.6-1.0 ms, -> 480: 2.8 ms.
2026-10-07 17:50:59 -04:00
spoopyghosty0 da7655ee25 Live view hardware encoder: fixes from the first device run
- fp_venc: accept the source's buffer count (the Frame's v4l2loopback has max_buffers=2; asking for 3 failed the
  probe); OUTPUT buffers remember which picture they hold, so repeats (standby, still scenes) skip the conversion
  (720p: 35% -> 2.5% of a core with the headset asleep).
- Pipeline: raw H.264 stamped on arrival (-use_wallclock_as_timestamps, same clock as pulse) with a 32-byte probe:
  -framerate was ignored, -fflags nobuffer lost the first seconds, frames numbered from 0 held all output ~7 s next
  to the sound. frag_keyframe so a requested keyframe starts a fragment (a joining viewer starts there).

Verified on the Frame (headset asleep): selftest ok (104 cases), probe at 360p-1080p, end to end through
livestream.start: 720p36 + AAC in 0.1 s fragments, audio and video spans match, a second viewer starts on its
requested keyframe at once, nothing left running after stop; FP_VENC_DISABLE=1 falls back to x264.
2026-10-07 17:50:57 -04:00
spoopyghosty0 0ae754fa5d Live view: hardware H.264 encoder (fp_venc), x264 stays as fallback
native/venc/fp_venc: own clean-room driver of the Frame's iris encoder (V4L2 stateful encoder interface, written from
native/venc/SPEC.md = kernel docs + values measured on the Frame). Reads SteamVR's headset-view webcam, box-scales and
converts RGB24 to NV12 (NEON, bit-identical scalar reference), CBR H.264 with SPS/PPS on every IDR, one frame per slot
of an even fraction of the panel rate read through DRM (96 Hz -> 32 fps). stdin "k" = keyframe, EOF = stop;
--probe/--selftest. Static freestanding aarch64 binary in artifacts/linux-arm64-bin (reproducible).

livestream: the script probes fp_venc and pipes it into ffmpeg (-c:v copy + AAC), else the old x264 30 fps path; the
helper is synced to ~/.local/share/frameport/bin by sha256 at stream start (no agent change); a new viewer asks for a
keyframe and waits for it; the status line shows the encoder and fps. Tests: hardware/fallback script branches,
keyframe-on-join, helper sync; tests/test_venc.py compares convert.c with a Python reference (native marker).
Not yet run on the device.
2026-10-07 17:50:55 -04:00
spoopyghosty0 ecfd2408cf Monitor: the detail boxes' first update goes through the event loop like the ticks (sent from the connect thread it could arrive after a tick's patch: 'dropped a patch for unknown control'); README screenshot with the new layout 2026-10-07 17:33:21 -04:00
spoopyghosty0 44dfd43692 Monitor feedback: updates from the stream's thread go through Flet's event loop (components.LoopUpdater: the window showed nothing until a click and the charts moved in bursts); intervals 0.1/0.25/0.5 s (0.5 s default; processes still scanned every 2 s, their GPU % averaged over it); charts in 1-second buckets (2 minutes at any interval); same-name processes with one parent collapse into a group row (click to open, end or force-kill all); tile headers line up; detail cards share one height; temperature boxes with mini charts and a Fan box; 'Monitor is using … of the Frame's CPU' 2026-10-07 17:26:20 -04:00
spoopyghosty0 29876ec046 README: Monitor section + screenshot (docs/images/monitor.png, ui_smoke --fake-frame --game: the fake monitor follows --game and sends two minutes at once so the charts are full). Monitor game card: the game's artwork is an art_fill control inside the box (copying its image onto the box showed nothing) 2026-10-07 17:05:15 -04:00
spoopyghosty0 3667f504ec Monitor agent after device tests (4XVR on the dev Frame): full names for processes whose comm the kernel truncated (Android's last 15 characters), power rails per cluster every 5 s and battery current/voltage every 2 s (I2C reads), temperatures every 2 s, fast render-fd rescans only for busy young game processes, end_game reports a game that wasn't running 2026-10-07 16:55:11 -04:00
spoopyghosty0 375b976dcd Monitor tab: a live view of the Frame while the tab is shown (running game with fps, CPU/GPU/memory/temperature/power/battery tiles with 2-minute sparklines, details: per-core load, sensors, power split, network) and its processes (filter Game / Steam & SteamVR / All, search, sort; right-click: end, force kill, end game; critical programs ask again). frame/monitor.py = session + pure helpers, C.Sparkline/MeterBar on Flet's canvas (no charts extension), ui_smoke FakeMonitorSession + --only, docs 2026-10-07 16:55:09 -04:00
spoopyghosty0 2a74e5ff64 Agent v62: _monitor stream for the Monitor tab (CPU/GPU/memory/pressure/temperatures/fan/power rails/battery/network, running FramePort games with fps from FrameBridge pacing and container cgroup usage, processes grouped by game/Steam/SteamVR/desktop with per-process GPU % from DRM fdinfo; end a process or a game). Kernel threads, command lines and render fds are cached so a sample stays cheap on the Frame 2026-10-07 16:55:06 -04:00
spoopyghosty0 0c623c2efc Install question: each file's name on its own line (long names end in an ellipsis, full name on hover), its type, size and checksum below; the narrow label column wrapped long file names 2026-10-07 16:51:32 -04:00
spoopyghosty0 d49b3ae884 Install links: FramePort-only manifest fields (frameport.description / frameport.icon) shown in the install question and used as the game's icon and description; readable titles for bare file links; handler scripts wait for the open window to take a link (a just-closed window's heartbeat swallowed clicks on Windows), the heartbeat is removed on close, and the Windows handler runs under conhost --headless (no console flash) 2026-10-07 16:35:21 -04:00
spoopyghosty0 f811261507 Lint: line lengths, import order, unused loop variable (ruff) 2026-10-07 16:09:03 -04:00
spoopyghosty0 ec701e6e56 Install links (FrameDrop's "Install with FrameDrop" button protocol), drop files on the Library, add a lone Windows .exe, VR / flat window choice, x86_64 Linux apps through FEX
- deeplink.py: framedrop:// and frameport:// links and the pasted https://framedropvr.com/install?manifest=|url= link; manifest framedrop.install/v1; https only (http on loopback), no credentials, no LAN addresses (also after DNS and redirects), sha256 checked, cancel/mismatch leave no .part
- urlhandler.py: per-scheme registration (Settings → Install links, links.framedrop / links.frameport, default on; another program's scheme only taken on request) through a handler script that queues the link and starts FramePort (a flet build bundle can't take a URL argument); the GUI's heartbeat + link inbox
- GUI: Add games → Install from a link… and Add a Windows program (.exe)…; confirmation before any download; files dropped on the Library (packaged app); CLI frameport open-link
- patch device.display_mode (Automatic / VR / Flat window), verified on the Frame with 4XVR
- x86_64 Linux apps run through FEX (agent v61: FEX installed unattended, STEAM_COMPAT_DATA_PATH set, RootFS from SteamOS's /usr/share/guestos/fex-mesa); verified on the Frame with an x86_64 test program; triage linux-x86-no-fex
2026-10-07 16:06:40 -04:00
345 changed files with 48994 additions and 1327 deletions

No files matched your search

+134 -20
View File
@@ -10,19 +10,7 @@ Read `docs/PLAYBOOK.md` (symptom → fix) before debugging a game, and `docs/FRA
## Layout
- `src/frameport/` — Python package. `pipeline.py` is the API the CLI (`cli.py`) and GUI (`ui/`, Flet 1.0) share.
- `ui/` — `app.py` shell (sidebar with Frame connection + activity cards, routing, actions, 30 s connection poll),
`theme.py` (design tokens; change colours/spacing only there. Dark-only colour themes (owner: no light mode):
built-ins `portal` (default), `portal_oled` (true black) and `original` (the old violet; alias `classic`). Portal =
the logo's dichotomy: orange ACCENT = main action + selection, blue SECONDARY = secondary (outlined) buttons,
progress, switch tracks, PC; `dual` themes add the blue→orange sidebar edge, two-colour wordmark (`C.wordmark`) and
selected-tab fade (`C.portal_gradient`). Installable theme files `<data>/themes/*.json` (docs/THEMES.md: name,
base, dual, any subset of `T.TOKENS`; refused if light/unreadable, `T.ThemeError`), Settings → Appearance: cards
switch live via `app.restyle()` (rebuilds the shell, drops cached views; setting `ui.theme`), "Install theme
file…", "Copy this theme as a file", bin icon removes. Read tokens as `T.NAME` when building — never as a default argument or module-level constant (they
keep the import-time theme; module maps refill through `T.on_change`, `tests/test_theme.py` checks defaults)),
`glyphs.py` + `icons/` (the logo `logo.svg`/`logo-solid.svg` = a monitor that comes out of a tilted portal as the
Frame, and 24 px line glyphs `fp:frame|port|patch|container|test|pc|sync|live|shot|keys|recipe|link`, copied into
the assets dir and shown as tinted `ft.Image`s: pass `G.FRAME` … wherever an icon goes, `C.as_icon` / the button
helpers handle both kinds; `src/assets/icon.png` = the bundles' app icon), `components.py` (pill, card, callout,
`theme.py` (dark design tokens; change colours/spacing only there), `components.py` (pill, card, callout,
status_row, art_fill, confirm, `update()` = safe update: in Flet 1.0 reading `.page` of an unmounted control
raises), `jobs.py` (background FIFO job queue, one at a time, cancel via Reporter; no Flet), `views/`
(library: search/filters/tags/sort as pure tested helpers; game: hero + one-click install, patches under
@@ -40,13 +28,24 @@ Read `docs/PLAYBOOK.md` (symptom → fix) before debugging a game, and `docs/FRA
delete leaves screenshots.vdf alone (Steam rewrites it at exit); game menu → "Screenshots" = `go("screenshots",
pkg)`); live view (`views/live.py` + `install/livestream.py`: the Frame's built-in SteamVR "headset view" webcam
(`steamvr-v4l2cam.service` → v4l2loopback "SteamVR" /dev/video99, see docs/FRAME_RUNTIME.md) + the default
output's pulse monitor (sound) → ffmpeg/x264 + AAC (qualities scale down only; "full" = SteamVR's size) →
fragmented MP4 on an SSH exec channel's stdout (the script stops ffmpeg on stdin EOF) → relay on 127.0.0.1 (keeps
init + fragments since the last keyframe of the *video* track for late viewers) → player page
output's pulse monitor (sound) → `fp_venc` (`native/venc`, own clean-room V4L2 driver of the Frame's iris
hardware encoder: RGB24→NV12 box downscale with NEON, H.264 CBR, one frame per slot of an even fraction of the
panel rate read from DRM, e.g. 96 Hz → 32 fps; stdin `k` = keyframe, EOF = stop; `--probe` / `--selftest`;
artifact `linux-arm64-bin/`, synced to `~/.local/share/frameport/bin` by sha256 at stream start, no agent change)
piped into ffmpeg (`-c:v copy` + AAC), else the old ffmpeg/x264 30 fps path (qualities scale down only; "full" =
SteamVR's size; stderr `live: encoder=… fps=…` → status line) →
fragmented MP4 on an SSH exec channel's stdout (stdin EOF stops it) → relay on 127.0.0.1 (a new viewer asks
the hardware encoder for a keyframe and waits for it, else gets init + fragments since the last keyframe of the
*video* track) → player page
`install/live_player.py` (MSE; starts muted as browsers require, "Sound on" button; 0.3 s cushion, catches up at
1.1x, seeks only when >2 s behind: seeking to the very edge starved it) opened
in the user's default browser: Flet can't show video outside `flet build` bundles; the stream outlives the tab and
stops on disconnect/window close); settings; welcome; activity panel). Files, Screenshots and the Library share right-click menus
stops on disconnect/window close); monitor (`views/monitor.py` + `frame/monitor.py` + agent v62 `_monitor`: one
JSON sample per tick over an SSH exec channel while the tab is shown (stopped in go/disconnect/on_close, the
agent ends at EOF); sources and costs in docs/FRAME_RUNTIME.md "Monitoring sources"; game card (fps from
FrameBridge pacing; End game = Steam's Exit game, then cmd_stop), tiles with `C.Sparkline` (Flet canvas, no charts
extension), details, a pooled process table (Game / Steam & SteamVR / All; right-click: end / force kill / end
game; MON_CRITICAL needs force, MON_NEVER is refused)); settings; welcome; activity panel). Files, Screenshots and the Library share right-click menus
(one `ft.ContextMenu` per view, filled on right-click; on one of several selected items they act on the whole
selection, `C.menu_targets`) and click-and-drag multi-select (`C.DragSelect`: pan start/end on the area + item
hover events, which Flutter also sends with the button held; Flet can't report item positions, so no rubber band).
@@ -72,6 +71,14 @@ Read `docs/PLAYBOOK.md` (symptom → fix) before debugging a game, and `docs/FRA
uploads are interruptible (Cancel checked per MiB) and resumable (big files via SFTP `.part` append, small files
streamed in tar batches; the agent counts files already in `incoming/`). Multi-select in the Library queues
installs after asking every needed question up front. Failures end in one pop-up (Resume / Uninstall / log).
- Other drives (GitHub #90, agent v63): anchors stay in ~/Applications/quest-frame, a game's files may live in
`<mount>/FramePort/<pkg>` (`deployment.json` base). Agent `drives` (/proc/mounts, /run/media + Steam library
drives; vfat/exfat/ntfs/read-only refused), prepare*/finalize_linux `dest` (unmounted = error, never a fallback;
installed games keep their base), `move` (detached systemd-run + `move_status`; cp -a under podman unshare,
count+bytes check, symlink retarget, launch.sh: Quest app_dir line / Linux+PC VR rewritten from the record's
`launcher` field, else text swap), list_installed `drive`/`drive_missing` (install_state keeps such games
"installed"). PC: `install/drives.py`, library setting `install.drive` (Frame page → Storage), game menu
"Move to…" (job kind tool-frame), CLI `frame drives`/`frame move`/`install --dest`. Untested on the device.
- **PC VR repacks are pre-patched to run directly** (proven: Rick and Morty, Vader Immortal run when the exe is
launched directly; Revive breaks them). So Rift games default to `as_is` = install the copy unchanged and launch
the exe directly (`pcvr.xr_timefix` for the Frame OpenXR-1.1→1.0 fix, `pcvr.no_crash_reporter` for Unreal).
@@ -84,7 +91,11 @@ Read `docs/PLAYBOOK.md` (symptom → fix) before debugging a game, and `docs/FRA
doesn't do). `VD.bat` is Virtual Desktop's launcher: ignore it except as an exe-location hint. Library migration
`rift_run_direct` resets existing recipes.
Auto launch-test is skipped on PC installs (it would start the game on the user's desktop). Launch tests collect the Unreal
game log + crash summaries from the Proton prefix; triage `unreal-crash`. Lies Beneath via Proton without Revive
game log + crash summaries from the Proton prefix; triage `unreal-crash`. Agent v67 adds Unity's logs (LocalLow/<Company>/<Product>/Player(-prev).log via `<Name>_Data/app.info`,
`output_log.txt`, Temp/…/Crashes/*/error.log; `unity_logs`) to launch tests and diagnostics; triage `unity-vr-init` /
`unity-crash`. A catalog recipe verified with another build (catalog `xr` ≠ the build's) keeps the build's own
`pcvr.launch_args` (`engine._other_build`; GitHub #105: SUPERHOT VR's Oculus+OpenVR build needs `-vrmode OpenVR`,
the catalog's is the OpenXR build); Electron launchers next to a game rank −30 (`rift.is_electron`). Lies Beneath via Proton without Revive
crashed (UE 4.23 "Unhandled exception").
- Rift scanning: `sources/rift_dump.scan` = the scanned folder's subfolders are games (one per folder; a folder is a
game if all candidate exes sit under one child), recursing into collections; `analysis/rift.py` walks once,
@@ -123,11 +134,20 @@ Rick and Morty runs on the Frame via its catalog recipe (OpenVR, no Revive),
set: the name on a colour from the title hash + the APK icon, `steam_set_for`; 2D Android apps get no store lookups) and tags: how it runs, the original platform
(Meta Quest / Oculus Rift), genres, user tags — merged with tags set in Steam (non-Steam shortcuts can't hold a
description).
- Linux apps (GitHub #31, library kind `linux`, `linux.<slug>`): GUI = Add games → "Add a Linux app (arm64)…" /
- Linux apps (GitHub #31, library kind `linux`, `linux.<slug>`; x86_64 builds run through FEX (app 3127680, no SLR:
RootFS /usr/share/guestos/fex-mesa from the OS image; needs `STEAM_COMPAT_DATA_PATH`, see docs/FRAME_RUNTIME.md):
agent v61 `linux_x86_tools`/`pick_tool`, `install_proton`/`proton_status` `kind: linux_x86`, `finalize_linux
x86_64`, `installer.ensure_proton(kind=)`; an arm64 program wins over an x86_64 one): GUI = Add
games → "Add a Linux app…" /
"…folder…" (`app.add_linux` job → `pipeline.add_linux_app`); game page `linux_summary` (program + Change…, AppImage,
OpenXR, source) instead of recipe/patches, `C.missing_libraries` callout (Frame deployment, else last install);
no Analyze/Rebuild/recipe/share/Game settings actions, Frame only; platform "Linux" badge + library filter; Steam
tags "Linux app on Frame"/"Linux"; `_follow_catalog` skips them; local files of a lone AppImage = the file only.
Desktop Mode entries (GitHub #84, agent v63): finalize_linux writes `frameport-<slug>.desktop` to
~/.local/share/applications (+ ~/Desktop if it exists; `X-FramePort-Package` marks ours), launch.sh with
`FRAMEPORT_DESKTOP=1` skips the Steam-parent watchdog and Steam's display; ensure_host_fixes refreshes entries
(older installs, stale ones removed), uninstall/purge remove them. Per app: library entry field `desktop_entry`
(default on; patches don't apply to Linux apps) → game page switch → agent `desktop_entry`. Untested on device.
- Quest/Rift twins stay separate entries, shown and named in Steam "Title (Quest)"/"(Rift)" (`core/titles.py`).
- `Recipe.as_is` = install unchanged (pre-patched libraries): `pipeline.prepare_as_is`; auto for APKs that already
contain FrameBridge (`frame_patched`). For Rift it changes nothing (the dump is never modified; the Frame copy
@@ -143,6 +163,10 @@ Rick and Morty runs on the Frame via its catalog recipe (OpenVR, no Revive),
- `patches/` — **the unit of modularity**. `base.py` (Patch interface, registry), `overport.py` (overport CLI patch ids,
discovered dynamically via `overport patches`), `frame/*.py` (one module per Frame fix), `settings.py` (FrameBridge
adapter keys + device files as patches). Add a patch = add a module that calls `register(...)`.
`upstream.py`: upstream fixes that replace a workaround per build (a probe finds the fix in OVRPort's output →
the build leaves the workaround out, `build.superseded`; recipes unchanged). Registered: `ovrport.haptic_envelope`
(→ `adapter.haptic_fix`, `frame/haptic_envelope.py`) and `ovrport.microphone_stream` (→ `frame.ovr_microphone`),
both fixed in OVRPort runtime 3.4.3-aa54c3f (ovrport/app#73; haptics owner-verified with Lucky's Tale 2026-10-07).
- `analysis/` — APK/ELF inspection (`detect.py`), `elf.py` (pyelftools reads; own DT_NEEDED writer), `stubgen.py`
(generates the ovr_* stub .so without a compiler).
- `apk/` — `axml.py` (binary manifest editor), `workspace.py` (staged zip edits), `sign.py` (apksigner; it aligns too).
@@ -517,7 +541,7 @@ recipe keeps it on). I Am Cat works with issues (judder; scale 0.8 no help). Rob
libroblox, also with the Vulkan shim). Installs skip the Steam restart when the shortcut is unchanged and wait while a
game runs (agent v37). New default fixes in FramePort's code reach games already in a library: after every app update
`library._follow_catalog` re-derives each non-user recipe once (setting `recipes.app_version`; tests switch it off via
`library.REFRESH_ON_UPDATE`); analysis fields added later still need a re-analysis. Troubleshooting techniques: docs/PLAYBOOK.md "Debugging techniques". Unity `boot.config` "vulkan" substring mislabels GLES games as Vulkan
`library.REFRESH_ON_UPDATE`). Analysis fields added later: **bump `analysis/detect.ANALYSIS_VERSION`** (stored as `analysis.extra.analysis_version`); older entries whose APK is still there are analysed again at the GUI's start (background thread, not a job: ~8 s per 900 MB APK) and before a build (`pipeline.refresh_analyses`, GitHub #104); only `analysis`/`suggested` change, user recipes stay; an unreadable APK gets `analysis_failed` = the version (not retried until the next bump). Troubleshooting techniques: docs/PLAYBOOK.md "Debugging techniques". Unity `boot.config` "vulkan" substring mislabels GLES games as Vulkan
(I Am Cat ran GLES); OVRPlugin's "Unavailable OpenXR extension: XR_FB_scene" is routine (no longer triaged).
**Round 4 (2026-10-04):** XR_KHR_android_surface_swapchain is listed by the Frame's runtime but returns
FUNCTION_UNSUPPORTED → adapter `surface_emul` (default on, `native/adapter/surface_swapchain.c`): an ordinary runtime
@@ -594,6 +618,7 @@ FrameBridge `snapshot=N` (`snapshot_gl.c`, GLES): every N s the left-eye image t
own context and saved as `files/fb_snap_0-7.ppm` (quarter size) — headless launches show a black headset view, this
shows what the game draws. Vader: Lucasfilm logo (an OBB mp4: video works), then its loading card (portrait + segmented
bar) that never advances; the async loader thread sleeps and OBB reads stop (~168 MB of a 2.7 GB pak).
**Lepton's logcat mirror dies (agent v67, 2026-10-09):** occasionally launch.log ends with `logcat: Unexpected EOF!` right after the game starts (about 1 launch in 50, Lepton 2.8.14 and 3.0.5): no dashboard auto-hide, no launch-test result. launch.sh's `_logcat_keeper` then reads `podman exec lepton-steamlaunch-<appid> logcat` itself into launch.log (up to 5 restarts while the game runs).
**Double launch (agent v57, 2026-10-06):** a second Play while Lepton still boots (~10 s with nothing to see) made
the second Lepton stop the first one's container ("Waiting for steamlaunch-<appid> (PID …) to exit", exit 137
"(starting)", "Clearing baked app data due to early exit") and both died (Vader, BattleSisters). launch.sh now takes
@@ -621,10 +646,99 @@ head-pose deadline ends VR mode without a worn headset (not a game bug).
**Language packs (merged from PR #20, 2026-10-04):** overport's `libovrplatformloader.so` is a dispatcher that `dlopen`s Meta's own loader (`libovrplatformloader_meta.so` / `_meta_q1.so`, also `libpxrplatformloader.so`) and keeps its own message queue; `ovr_LanguagePack_GetCurrent/SetCurrent` are 8-byte `return 0` stubs in it, `ovr_AssetFile_GetList` forwards to Meta's loader. `frame.langpacks` (opt-in, `native/langpack`) serves `<tag>.lang` files from the game's data; `elf.hide_exports` marks the loader's exports STB_LOCAL (bionic and glibc only match GLOBAL/WEAK; verified for glibc, bionic's `is_symbol_global_and_defined` is from memory). `libfp_langpack.so` is built here (`python native/build.py --only langpack`) and committed. Owner-verified 2026-10-04: Deadpool VR with only `en.lang` and the patch on plays English dialogue and runs normally (headless launch tests show 2-6 fps while it loads: not a regression sign). A dispatcher answer that arrives after we answered the timed-out GetList is dropped (one answer per request); games already in a library get `lang_packs` filled after an app update (`library._refresh_data_fields`). The library logs to logcat under the tag `fp_langpack` (dirs looked in, packs found, every language-pack call), so it shows up in the game's `launch.log`; an `ovr_AssetFile_GetList` the dispatcher leaves unanswered for 1.5 s is answered with our packs alone. Deadpool VR (Unreal) accepts a pack only when its `Metadata` equals the game's own version string (`ULanguagePacksSubsystem` compares it with `%s.%s.%s.%s.%s` built from the build info, e.g. `1.0.40.356975.Quest` = versionName; found by disassembling `libUE4.so`): the patch writes the APK's versionName into the library (`@FPMETA@` slot, `with_metadata`), `FRAMEPORT_LANGPACK_META` overrides it. Deadpool VR (2026-10-04, headset): German became selectable with that Metadata, but dialogue stayed silent (even English once reported as a pack) while the path was spelled `/sdcard/Android/obb/<pkg>/x.lang`; with the `/storage/emulated/0/Android/obb/<pkg>/x.lang` spelling (Unreal's own, now listed first in `scan_dirs`) German dialogue plays. `FRAMEPORT_LANGPACK_SKIP=<tags>` or a file `fp_langpack_skip` in the obb folder leaves packs out of the list (experiments).
**Cube swapchains (GitHub #107, 2026-10-09):** the Frame's runtime refuses `faceCount=6` swapchains (-2,
XR_ERROR_RUNTIME_FAILURE); OVRPlugin ignores that ("CreateSwapchain for eye 0: 0x0, 0 stages") and crashes in
ovrp_EndFrame4 (memset). FrameBridge `cube_standin` (default on, `native/adapter/cube_standin.c`) serves a refused cube
swapchain as one GL cube-map texture in the app's context (GLES only) and drops its layers. Verified headless with
Budget Cuts Ultimate (2048² sRGB, 12 mips; runs on at ~70 fps); what the cube layer showed is simply missing.
**Hardware video decoding (PR #96, PR #128 by Lucas-Mathieu, adapted 2026-10-10, agent v71):** `native/hevc` = one
OMX plugin `OMX.frameport.{avc,hevc,vp9}.decoder` (FFmpeg v4l2m2m on Iris /dev/video-dec0, FFmpeg software decoders as
fallback inside the component, Vulkan copy for big native surfaces). The connection installs it once per Frame
(`ensure_video_codec` → agent `video_codec_status`/`install_video_codec`: `~/.local/share/frameport/video-codec/versions/
<manifest sha>` + `current` symlink). A Frame keeps the same or a newer `revision` (no flip-flop between PCs) → **bump
`revision` in native/hevc/build.py with every artifact change**; a failed install isn't retried during the connection.
Per game: only launchers of games whose recipe has `frame.hw_video_decode` (install stage, no APK change; suggested
from analysis `media_codec`, ANALYSIS_VERSION 8; deployment.json `hw_video_decode`) get the codec line → the Podman
wrapper (native/hevc/podman.py) mounts the plugin into that container (merged media_codecs.xml in
$XDG_RUNTIME_DIR/frameport-video). Off for every game: Settings → Installing (library `video.hw_decode` → agent
`video_codec_switch` = video-codec/disabled); one game: `FRAMEPORT_NO_HW_VIDEO=1 %command%`. `upgrade_launchers`
converts agent ≤70 launchers (Batman's `<base>/frameport-codec` → keeps the line, folder then removed). Batman: catalog
+ migration `batman_video_patches` (`frame.hw_video_decode` + hidden adapter setting `surface_native`). Iris: 8K
refused (ENOMEM) while any other decoder session is open; SteamVR's vrlinkrunthread holds one ~12 s at game start →
refused opens retry until 20 s after the plugin loaded (else 2 s), then software; VP9 7680x3840 never returned a
picture → VP9 ≤4096x2304; hidden VP9 frames come back as empty capture buffers (bytesused 0, no LAST):
FFmpeg's wrapper ended the EOS drain at the first one (two-pass VP9 lost its last ~25 frames) → build.py requeues
them (codec revision 8; drain ends at LAST, or 1 s after a skipped empty buffer; dev Frame 2026-10-10: 600/600 =
OMX.google.vp9 Y hashes). Rebuilds are deterministic (ext4 + NTFS path
with spaces, two NDK copies → same; rev 8 141de01f…; PR #128's own sources → its 27a2d749…). Not yet run on the device.
**Vivox API 31 (GitHub #101, 2026-10-09):** newer Vivox builds (Green Hell VR) call Android 12 AudioManager
communication-device methods from `com.vivox.sdk.AudioChangeListener` with no SDK check → NoSuchMethodError on Lepton's
Android 11. `frame.vivox_audio_route` (analysis `vivox_api31`, ANALYSIS_VERSION 4) makes every such method return at
once (`Dex.return_early`: return-void / `const/4 v0,0; return v0`; nopping the invoke would leave a move-result the
verifier rejects). Older Vivox (Eleven Table Tennis, BattleSisters) lacks that code and isn't matched. Verified
headless: Vivox initialises, 150 s at ~65-70 fps.
**Multiview programs on flat framebuffers (GitHub #77, Doom3Quest, 2026-10-09, prototype):** Mesa enforces OVR_multiview's
"program num_views == draw framebuffer views" rule (`draw_validate.c`, the draw is dropped silently); Qualcomm doesn't.
Doom3Quest compiles every VS with `layout(num_views=2) in;` and draws its HUD/PDA into 2D-texture FBOs → black. Opt-in
`frame.gl_multiview_fbo` (analysis `gl_multiview_libs`, ANALYSIS_VERSION 6; own-engine GLES only): `native/glmv` =
`libfpglmv.so` (12 chars = "libGLESv3.so": libdoom3.so's one `.rodata` dlopen string is rewritten in place, its qgl*
table comes from dlsym on that handle; also first DT_NEEDED for its direct gl*/egl* imports). Such draws use a lazily
built single-view twin (view 0, uniforms copied per draw); eglMakeCurrent resets the per-thread cache. Host-tested only
(rewriter on all 19 Doom3Quest shaders + glmv.c against a stand-in GL, `tests/test_gl_multiview_fbo.py`); the host's
Mesa 23.2 llvmpipe has no OVR_multiview. Headset-verified by Xandrix1987 (2026-10-09: HUD/PDA shown, no fps drop) -> in the Doom3Quest catalog recipe.
**Session triage (agent v70, 2026-10-09):** launch tests never reach FOCUSED, so real play sessions are triaged
too. list_installed gives each game `last_play` {start, end, test} from `<anchor>/plays.log` (launch tests write a
`test <unix>` line first → test sessions are only marked); `session_log` returns the newest session's launch.log
(`<base>/session.log`, ≤4 MB: head + tail + FrameBridge/crash lines between) + that session's logcat-crash.log +
`journalctl -k` GPU lines ("kernel: …"). The GUI's connection refresh (`app._check_sessions` → `pipeline.sessions_due`,
background thread; sessions >7 days old are skipped) runs `pipeline.triage_session` = `validate/session.analyze`
(triage.yaml signatures incl. `space-warp-used` (info, `question:`, never auto-applied), `gpu-hang` (`report: true`)
+ computed `slow-frames` (pacing windows vs the nearest refresh rate) and `focus-dips` (FrameBridge always logs
`focus: lost` / `focus: back after N ms`, session_fixes.c)) → library `last_session` / `last_session_checked` → game
page "Last session" callout (Try this fix / Rebuild with this fix / Yes-No question / Report / Dismiss). Suggestions may
be values (`adapter.scale=0.85`, `triage.split_suggestion`); adapter-only fixes are pushed live (`apply_suggestions_live`
→ agent set_settings, like the Game settings dialog). CLI `frameport session <pkg> [--apply]`. PC VR (Proton) launchers
log no session end, so they aren't triaged yet. pac_hints stays triage-only: a survey of all 68 dump APKs found unpaired
PAC hints in 21 libraries of 18 games (OpenSSL's 38/40 in most UE4 libUE4.so and libEOSSDK.so, UE5 libUnreal.so ~400
vs +2-3, libass, libopencv, …), most of them games that work, so default-on would rewrite many working builds.
**Unresolved (as of 2026-09-28):** Arcsmith (right-eye distortion) and Time Stall (both eyes) — swap, tracking, Valve
layers, depth, pacing ruled out. Sniper Elite VR (DEVICE LOST), Espire 1 (Mesa GL upload crash), HITMAN 3 (freedreno
crash): use PC versions.
**Install links / FrameDrop button protocol (2026-10-07, not yet clicked end to end from a browser):** FrameDrop
(framedropvr.com, a closed-source Windows sideloader) defines "Install with FrameDrop" buttons:
`https://framedropvr.com/install?manifest=<url>|url=<file>` → that page opens `framedrop://install?…` (1.6 s, else its
home page); manifest `{"schema":"framedrop.install/v1","name","files":[{"url","sha256"}]}` (name = Steam title; .apk
or Linux .zip). `deeplink.py` (no Flet) parses framedrop://, frameport:// and the pasted https link, enforces its
rules (https; http only on loopback; no credentials; no LAN/loopback/link-local IPs, also after DNS and redirects; URL
ends in a file name), caps manifests at 256 KiB, ignores non-hex sha256 (FrameDrop's own example has a placeholder),
downloads into `<data>/downloads/<slug>-<hash>/` (OBBs → `obb/` next to the APK, `.part` removed on cancel/mismatch);
`pipeline.add_from_link` routes APK / Linux / exe and sets `title` + `title_locked` + `link` (a bare file link keeps
FramePort's title). GUI: `views/link_dialog.py` (always asks first), Add games → "Install from a link…", CLI
`frameport open-link [--yes --no-install --gui]`. **A `flet build` bundle can't take a URL argument** (the Flutter
host treats any argv as a developer page URL), so `urlhandler.py` registers a script, not FramePort.exe: Windows
HKCU `Software\Classes\{framedrop,frameport}` → hidden PowerShell `frameport-link-handler.ps1`; WSL (source runs)
the same keys → `wsl.exe -d <distro> -e sh frameport-link-handler.sh`; Linux `frameport-links.desktop` +
`xdg-mime`; macOS unsupported (Apple Events, paste instead). The script drops the link into `<data>/links/*.link`
and starts FramePort unless `<data>/gui.alive` is < 10 s old (`gui.starting` stops double starts); the GUI's
`_watch_links` thread touches the heartbeat and opens links (newest window session). Settings → Install links: one
switch per scheme (`links.framedrop`, `links.frameport`, default on); a scheme another program owns (FrameDrop) is
only taken with "Use FramePort for these links" (`register([s], force=True)`); off removes only FramePort's own
registration (`MARK` in the command). Never registered with FRAMEPORT_HOME/FRAMEPORT_NO_LINK_HANDLER (tests,
screenshots). Same round: files dropped on the Library (`ui/dropped.py`, bundles only like the Files tab), "Add a
Windows program (.exe)…" (`pipeline.add_windows_exe`: exe in Downloads/home/drive root copied alone into
`<data>/windows-apps/<slug>/`), patch `device.display_mode` (Automatic / VR / Flat window → `InstallContext.display`,
`installer.show_window`). FramePort-only manifest extension `"frameport": {"description", "icon"}` (bad values ignored; icon: same URL rules,
≤2 MiB, Pillow-checked, ≥32 px, saved as PNG in `<data>/downloads/icons/`, shown in the question by asset URL, then
`sources.apply_custom(pkg, "icon")` unless `.picked` exists; description fills `details.description` only when
empty). Bare file links get a title guessed from the file name (`title_from_filename`: version/arch dropped, package
names → last part). Windows test of 0.12.1.dev191 (2026-10-07): a button click opened the dialog; but a click right
after closing FramePort did nothing (the closed window's heartbeat was < 10 s old) → the handler scripts now wait
up to 4 s for the link file to be taken before trusting the heartbeat, the window's CLOSE event / atexit delete
`gui.alive`, and the Windows command runs under `conhost.exe --headless` (plain `-WindowStyle Hidden` flashed a
console). Screens: `scripts/ui_smoke.py --links --fake-frame --game <pkg>` (tall pages: `--viewport
1280x7000`; Flutter's popup menu ignores Escape).
## Releases, CI, GitHub
Maintainer-only notes (accounts, credentials, key locations) live in the git-ignored `CLAUDE.local.md`.
Public repo `github.com/spoopyghosty0/frameport` (branch `main`). Push a `v*` tag → CI (`.github/workflows/build.yml`)
+17 -2
View File
@@ -28,11 +28,15 @@ Install games that target the Meta Quest, Android, or general PCVR onto your **V
curved and 360° layers); game settings as simple switches.
- **Beyond Quest:** Android apps as windows, PC VR via Proton or Revive, Windows (non-VR) games via Proton, a Files
tab with drag and drop.
- **Linux apps:** install arm64 Linux apps (AppImage, a folder, or a zip/tar archive) on the Frame with a Steam
shortcut; they run natively on SteamOS.
- **Linux apps:** install Linux apps (AppImage, a folder, or a zip/tar archive) on the Frame with a Steam
shortcut; arm64 builds run natively on SteamOS, x86_64 builds through Valve's FEX translator.
- **Install links:** "Install with FrameDrop" buttons on web pages (the one-click protocol of the FrameDrop
sideloader) and pasted links open in FramePort, which asks, downloads, adds and installs the build.
- **Screenshots tab:** the screenshots you took in the headset, sorted by game (matched by play time) and day;
view them and download them to your computer.
- **Live view tab:** watch what the headset shows, with sound, in a browser window on your computer.
- **Monitor tab:** the running game's frame rate, the Frame's load, temperatures, power and battery live, and its
processes, which you can end. [More](#monitor).
- **Self-updating** releases, redacted diagnostics, one-click problem reports and working-config sharing.
## Quick start
@@ -87,6 +91,17 @@ videos muted). Pick 360p to 1080p, or the headset view's full size. The picture
view on the Frame and is encoded there while you watch (about one CPU core), so stop it when you're done. It's black
while the headset sleeps.
## Monitor
The **Monitor** tab shows what the Frame is doing while it's open: the running game with its frame rate against the
display's refresh rate, CPU, graphics chip, memory, the hottest temperature with the fan speed, power draw and battery
time left, each with a 2-minute chart. **Show details** adds every CPU core, all temperature sensors, where the power
goes and the network. Below, the game's processes (or Steam's, or all of them) with their CPU, GPU and memory use:
right-click one to end it, or end the whole game. Programs that Steam, SteamVR or the desktop need are marked and ask
again. The Frame sends the numbers itself (about 1 % of one CPU core) and stops when you leave the tab.
![Monitor](docs/images/monitor.png)
## Compatibility
If a game has already been tested with FramePort, it will automatically use the optimal game config. Otherwise, FramePort
+2
View File
@@ -11,6 +11,8 @@ FramePort is GPL-3.0-only (see `LICENSE`). It includes or downloads the followin
| Android NDK runtime (statically linked libc++) | native Android libraries | Apache-2.0 with LLVM exception, plus legacy notices (`native/vrapi-bridge/licenses/ANDROID-NDK.txt`) |
| Flet and Flutter (desktop app runtime) | release bundles | Apache-2.0 / BSD-3-Clause |
| flet-dropzone / desktop_drop (drag-and-drop) | release bundles | Apache-2.0 / MIT |
| FFmpeg 7.1.1 hardware HEVC wrapper (LGPL configuration, without GPL/nonfree components) | `artifacts/hevc/libstagefrighthw.so`; source/rebuild instructions in `native/hevc/build.py` and `native/hevc/README.md` | LGPL-2.1-or-later (`artifacts/hevc/COPYING.FFmpeg`); unmodified source: https://ffmpeg.org/releases/ffmpeg-7.1.1.tar.xz |
| AOSP Android 11 native media/utility headers | `native/hevc/platform/` | Apache-2.0; copyright/license notices retained in the headers |
| Python packages (paramiko, zeroconf, psutil, pyelftools, capstone, UnityPy, PyYAML, requests, typer, pyaxmlparser, Pillow, cryptography, …) | release bundles | their own licenses (see each package's metadata) |
FramePort's own native code (the FrameBridge adapter, GL/Vulkan/OpenXR shims, the Windows helpers) is GPL-3.0-only.
+2244 -54
View File
File diff suppressed because it is too large. Load diff
+13 -5
View File
@@ -1,16 +1,24 @@
5becb96ee86fbda0ce98c0e0fd064f8ba092314aceaa400e3dcc785705dc3c23 ./arm64-v8a/libVkLayer_fp_shaderfix.so
e7554c6343ee2989b0a273ee6230e65c25bfe6499eefd00de19f5f0fee58754f ./arm64-v8a/libfp_langpack.so
98a99cefe93079268d9158b23fa8ad5c7ade8e54c0d31526892635625eb3ef04 ./arm64-v8a/libfp_ovrp.so
321da502e0f8f46f0880aad39fe6b84bdef88f925164568814cf704a1b31a577 ./arm64-v8a/libfp_ovrp.so
40defdaddcda53bd2649eb48076fae1622bfc2bfc88e9e03c94fc8b45af2a4f2 ./arm64-v8a/libfp_ovrtrace.so
7522a7cfb236e48d3442d418c4820cd37760a9627c6bdef80fb0df82df5fd826 ./arm64-v8a/libfp_vk.so
383054f8b3b41dde76d062c71856cd3163655e50009c08454bc1793f393ead65 ./arm64-v8a/libfpg.so
06c8a17b37cb1513fdfeba87ece457a02097bb22dd0a3a3a8719fb8fea16b52a ./arm64-v8a/libframe_xrshim.so
9f5ba793d1804728394a8fe9e83028e13f1bc939b558892e7beb5c3662317259 ./arm64-v8a/libglshim.so
cc5b493cadefe76ad13fd096d3f8494a140ba593e0e1692db12d284bc548c1cf ./arm64-v8a/libopenxr_loader_generic.so
5432674a59d02f411cd853a5dc57e3fa547d1ac518f25bf433fc54c43677177a ./arm64-v8a/libfpglmv.so
8a8f6b1da8952cb3933a5b53558424a5fc56043b2527a16bb6dd62a00d2de01a ./arm64-v8a/libframe_xrshim.so
fc559dbe0b87eecc3641256837293260cefb7c2ce7b21d808d10edd2a2dba9fc ./arm64-v8a/libglshim.so
1655921f4c91f2f336edf0aac66d8df1b2b649a272aafe365c1f2fa7867129de ./arm64-v8a/libopenxr_loader_generic.so
1feaeafad467c4cafdf2b018a4d84b0bee200c3f711697b4ce97e66a3ba256ca ./arm64-v8a/libovrplatformcompat.so
aa4dc0020c77e12d41ef6ce80b23ad90c9cb7e882338eddf2261efe8645cf38d ./arm64-v8a/libvrapi.so
c10412dd76a65a1d587b7ed226367d1a6f2bec238c733c97ee67eb8870c868e8 ./armeabi-v7a/libopenxr_loader_generic.so
7943c6825e1c2841e54627f96acbd0754a3c06bb608c47f7aa15007aec2191a5 ./armeabi-v7a/libopenxr_loader_generic.so
1871eae093432d277da4bc751bf5f3269dfcf11f9168260b0c3caadb0dedb19d ./dex/oculusos-stubs.dex
b634ab5640e258563c536e658cad87080553df6f34f62269a21d554844e58bfe ./hevc/COPYING.FFmpeg
141de01f0d40b97d4f9a720db5ab8ea442f6aa35db5272048899292884994a1c ./hevc/libstagefrighthw.so
4516971bbb2b635e19b14d2d37b9353962876f7de02043e7457327052c1fcbc0 ./hevc/manifest.json
46c3dcad2c43cc30bea3b3680b362ed84a99c15d60714bbbbb422c80ecf90405 ./hevc/media_codecs_frameport.xml
529ea35d5d82474afe9e09ad1216d269317f56dd24a05fb100fced52b7b4dbca ./hevc/podman.py.txt
1aa733117cf57ccff7cf23f0425dbfd73b0332a3ef1c905ceeca0ccd0449c4e0 ./linux-arm64/XR_APILAYER_FRAMEPORT_timefix.json
28c2430a02bbd8902c5bfb9562c6fd0e318654f9e05c095bfeb94b0450ab1b07 ./linux-arm64/libxr_frameport_timefix.so
b10b3a5c10c3339fc63fd9f6cb4d01ea29be11a863d0ebad1389ea69fd471940 ./linux-arm64-bin/fp_venc
c2c70f4af1a5c3e6089dff1130161dd417f8044f99ec639eeecf1083930aab16 ./win-x64/fp_oculushmd.exe
60ebc5fea05b8945082e2ca8b9f2eb74616012a26eb50133af273dff0fcddc42 ./win-x64/fp_vrsettings.exe
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+502
View File
@@ -0,0 +1,502 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 2.1, February 1999
Copyright (C) 1991, 1999 Free Software Foundation, Inc.
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
[This is the first released version of the Lesser GPL. It also counts
as the successor of the GNU Library Public License, version 2, hence
the version number 2.1.]
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
Licenses are intended to guarantee your freedom to share and change
free software--to make sure the software is free for all its users.
This license, the Lesser General Public License, applies to some
specially designated software packages--typically libraries--of the
Free Software Foundation and other authors who decide to use it. You
can use it too, but we suggest you first think carefully about whether
this license or the ordinary General Public License is the better
strategy to use in any particular case, based on the explanations below.
When we speak of free software, we are referring to freedom of use,
not price. Our General Public Licenses are designed to make sure that
you have the freedom to distribute copies of free software (and charge
for this service if you wish); that you receive source code or can get
it if you want it; that you can change the software and use pieces of
it in new free programs; and that you are informed that you can do
these things.
To protect your rights, we need to make restrictions that forbid
distributors to deny you these rights or to ask you to surrender these
rights. These restrictions translate to certain responsibilities for
you if you distribute copies of the library or if you modify it.
For example, if you distribute copies of the library, whether gratis
or for a fee, you must give the recipients all the rights that we gave
you. You must make sure that they, too, receive or can get the source
code. If you link other code with the library, you must provide
complete object files to the recipients, so that they can relink them
with the library after making changes to the library and recompiling
it. And you must show them these terms so they know their rights.
We protect your rights with a two-step method: (1) we copyright the
library, and (2) we offer you this license, which gives you legal
permission to copy, distribute and/or modify the library.
To protect each distributor, we want to make it very clear that
there is no warranty for the free library. Also, if the library is
modified by someone else and passed on, the recipients should know
that what they have is not the original version, so that the original
author's reputation will not be affected by problems that might be
introduced by others.
Finally, software patents pose a constant threat to the existence of
any free program. We wish to make sure that a company cannot
effectively restrict the users of a free program by obtaining a
restrictive license from a patent holder. Therefore, we insist that
any patent license obtained for a version of the library must be
consistent with the full freedom of use specified in this license.
Most GNU software, including some libraries, is covered by the
ordinary GNU General Public License. This license, the GNU Lesser
General Public License, applies to certain designated libraries, and
is quite different from the ordinary General Public License. We use
this license for certain libraries in order to permit linking those
libraries into non-free programs.
When a program is linked with a library, whether statically or using
a shared library, the combination of the two is legally speaking a
combined work, a derivative of the original library. The ordinary
General Public License therefore permits such linking only if the
entire combination fits its criteria of freedom. The Lesser General
Public License permits more lax criteria for linking other code with
the library.
We call this license the "Lesser" General Public License because it
does Less to protect the user's freedom than the ordinary General
Public License. It also provides other free software developers Less
of an advantage over competing non-free programs. These disadvantages
are the reason we use the ordinary General Public License for many
libraries. However, the Lesser license provides advantages in certain
special circumstances.
For example, on rare occasions, there may be a special need to
encourage the widest possible use of a certain library, so that it becomes
a de-facto standard. To achieve this, non-free programs must be
allowed to use the library. A more frequent case is that a free
library does the same job as widely used non-free libraries. In this
case, there is little to gain by limiting the free library to free
software only, so we use the Lesser General Public License.
In other cases, permission to use a particular library in non-free
programs enables a greater number of people to use a large body of
free software. For example, permission to use the GNU C Library in
non-free programs enables many more people to use the whole GNU
operating system, as well as its variant, the GNU/Linux operating
system.
Although the Lesser General Public License is Less protective of the
users' freedom, it does ensure that the user of a program that is
linked with the Library has the freedom and the wherewithal to run
that program using a modified version of the Library.
The precise terms and conditions for copying, distribution and
modification follow. Pay close attention to the difference between a
"work based on the library" and a "work that uses the library". The
former contains code derived from the library, whereas the latter must
be combined with the library in order to run.
GNU LESSER GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License Agreement applies to any software library or other
program which contains a notice placed by the copyright holder or
other authorized party saying it may be distributed under the terms of
this Lesser General Public License (also called "this License").
Each licensee is addressed as "you".
A "library" means a collection of software functions and/or data
prepared so as to be conveniently linked with application programs
(which use some of those functions and data) to form executables.
The "Library", below, refers to any such software library or work
which has been distributed under these terms. A "work based on the
Library" means either the Library or any derivative work under
copyright law: that is to say, a work containing the Library or a
portion of it, either verbatim or with modifications and/or translated
straightforwardly into another language. (Hereinafter, translation is
included without limitation in the term "modification".)
"Source code" for a work means the preferred form of the work for
making modifications to it. For a library, complete source code means
all the source code for all modules it contains, plus any associated
interface definition files, plus the scripts used to control compilation
and installation of the library.
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running a program using the Library is not restricted, and output from
such a program is covered only if its contents constitute a work based
on the Library (independent of the use of the Library in a tool for
writing it). Whether that is true depends on what the Library does
and what the program that uses the Library does.
1. You may copy and distribute verbatim copies of the Library's
complete source code as you receive it, in any medium, provided that
you conspicuously and appropriately publish on each copy an
appropriate copyright notice and disclaimer of warranty; keep intact
all the notices that refer to this License and to the absence of any
warranty; and distribute a copy of this License along with the
Library.
You may charge a fee for the physical act of transferring a copy,
and you may at your option offer warranty protection in exchange for a
fee.
2. You may modify your copy or copies of the Library or any portion
of it, thus forming a work based on the Library, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) The modified work must itself be a software library.
b) You must cause the files modified to carry prominent notices
stating that you changed the files and the date of any change.
c) You must cause the whole of the work to be licensed at no
charge to all third parties under the terms of this License.
d) If a facility in the modified Library refers to a function or a
table of data to be supplied by an application program that uses
the facility, other than as an argument passed when the facility
is invoked, then you must make a good faith effort to ensure that,
in the event an application does not supply such function or
table, the facility still operates, and performs whatever part of
its purpose remains meaningful.
(For example, a function in a library to compute square roots has
a purpose that is entirely well-defined independent of the
application. Therefore, Subsection 2d requires that any
application-supplied function or table used by this function must
be optional: if the application does not supply it, the square
root function must still compute square roots.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Library,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Library, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote
it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Library.
In addition, mere aggregation of another work not based on the Library
with the Library (or with a work based on the Library) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may opt to apply the terms of the ordinary GNU General Public
License instead of this License to a given copy of the Library. To do
this, you must alter all the notices that refer to this License, so
that they refer to the ordinary GNU General Public License, version 2,
instead of to this License. (If a newer version than version 2 of the
ordinary GNU General Public License has appeared, then you can specify
that version instead if you wish.) Do not make any other change in
these notices.
Once this change is made in a given copy, it is irreversible for
that copy, so the ordinary GNU General Public License applies to all
subsequent copies and derivative works made from that copy.
This option is useful when you wish to copy part of the code of
the Library into a program that is not a library.
4. You may copy and distribute the Library (or a portion or
derivative of it, under Section 2) in object code or executable form
under the terms of Sections 1 and 2 above provided that you accompany
it with the complete corresponding machine-readable source code, which
must be distributed under the terms of Sections 1 and 2 above on a
medium customarily used for software interchange.
If distribution of object code is made by offering access to copy
from a designated place, then offering equivalent access to copy the
source code from the same place satisfies the requirement to
distribute the source code, even though third parties are not
compelled to copy the source along with the object code.
5. A program that contains no derivative of any portion of the
Library, but is designed to work with the Library by being compiled or
linked with it, is called a "work that uses the Library". Such a
work, in isolation, is not a derivative work of the Library, and
therefore falls outside the scope of this License.
However, linking a "work that uses the Library" with the Library
creates an executable that is a derivative of the Library (because it
contains portions of the Library), rather than a "work that uses the
library". The executable is therefore covered by this License.
Section 6 states terms for distribution of such executables.
When a "work that uses the Library" uses material from a header file
that is part of the Library, the object code for the work may be a
derivative work of the Library even though the source code is not.
Whether this is true is especially significant if the work can be
linked without the Library, or if the work is itself a library. The
threshold for this to be true is not precisely defined by law.
If such an object file uses only numerical parameters, data
structure layouts and accessors, and small macros and small inline
functions (ten lines or less in length), then the use of the object
file is unrestricted, regardless of whether it is legally a derivative
work. (Executables containing this object code plus portions of the
Library will still fall under Section 6.)
Otherwise, if the work is a derivative of the Library, you may
distribute the object code for the work under the terms of Section 6.
Any executables containing that work also fall under Section 6,
whether or not they are linked directly with the Library itself.
6. As an exception to the Sections above, you may also combine or
link a "work that uses the Library" with the Library to produce a
work containing portions of the Library, and distribute that work
under terms of your choice, provided that the terms permit
modification of the work for the customer's own use and reverse
engineering for debugging such modifications.
You must give prominent notice with each copy of the work that the
Library is used in it and that the Library and its use are covered by
this License. You must supply a copy of this License. If the work
during execution displays copyright notices, you must include the
copyright notice for the Library among them, as well as a reference
directing the user to the copy of this License. Also, you must do one
of these things:
a) Accompany the work with the complete corresponding
machine-readable source code for the Library including whatever
changes were used in the work (which must be distributed under
Sections 1 and 2 above); and, if the work is an executable linked
with the Library, with the complete machine-readable "work that
uses the Library", as object code and/or source code, so that the
user can modify the Library and then relink to produce a modified
executable containing the modified Library. (It is understood
that the user who changes the contents of definitions files in the
Library will not necessarily be able to recompile the application
to use the modified definitions.)
b) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (1) uses at run time a
copy of the library already present on the user's computer system,
rather than copying library functions into the executable, and (2)
will operate properly with a modified version of the library, if
the user installs one, as long as the modified version is
interface-compatible with the version that the work was made with.
c) Accompany the work with a written offer, valid for at
least three years, to give the same user the materials
specified in Subsection 6a, above, for a charge no more
than the cost of performing this distribution.
d) If distribution of the work is made by offering access to copy
from a designated place, offer equivalent access to copy the above
specified materials from the same place.
e) Verify that the user has already received a copy of these
materials or that you have already sent this user a copy.
For an executable, the required form of the "work that uses the
Library" must include any data and utility programs needed for
reproducing the executable from it. However, as a special exception,
the materials to be distributed need not include anything that is
normally distributed (in either source or binary form) with the major
components (compiler, kernel, and so on) of the operating system on
which the executable runs, unless that component itself accompanies
the executable.
It may happen that this requirement contradicts the license
restrictions of other proprietary libraries that do not normally
accompany the operating system. Such a contradiction means you cannot
use both them and the Library together in an executable that you
distribute.
7. You may place library facilities that are a work based on the
Library side-by-side in a single library together with other library
facilities not covered by this License, and distribute such a combined
library, provided that the separate distribution of the work based on
the Library and of the other library facilities is otherwise
permitted, and provided that you do these two things:
a) Accompany the combined library with a copy of the same work
based on the Library, uncombined with any other library
facilities. This must be distributed under the terms of the
Sections above.
b) Give prominent notice with the combined library of the fact
that part of it is a work based on the Library, and explaining
where to find the accompanying uncombined form of the same work.
8. You may not copy, modify, sublicense, link with, or distribute
the Library except as expressly provided under this License. Any
attempt otherwise to copy, modify, sublicense, link with, or
distribute the Library is void, and will automatically terminate your
rights under this License. However, parties who have received copies,
or rights, from you under this License will not have their licenses
terminated so long as such parties remain in full compliance.
9. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Library or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Library (or any work based on the
Library), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Library or works based on it.
10. Each time you redistribute the Library (or any work based on the
Library), the recipient automatically receives a license from the
original licensor to copy, distribute, link with or modify the Library
subject to these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties with
this License.
11. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Library at all. For example, if a patent
license would not permit royalty-free redistribution of the Library by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Library.
If any portion of this section is held invalid or unenforceable under any
particular circumstance, the balance of the section is intended to apply,
and the section as a whole is intended to apply in other circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
12. If the distribution and/or use of the Library is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Library under this License may add
an explicit geographical distribution limitation excluding those countries,
so that distribution is permitted only in or among countries not thus
excluded. In such case, this License incorporates the limitation as if
written in the body of this License.
13. The Free Software Foundation may publish revised and/or new
versions of the Lesser General Public License from time to time.
Such new versions will be similar in spirit to the present version,
but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Library
specifies a version number of this License which applies to it and
"any later version", you have the option of following the terms and
conditions either of that version or of any later version published by
the Free Software Foundation. If the Library does not specify a
license version number, you may choose any version ever published by
the Free Software Foundation.
14. If you wish to incorporate parts of the Library into other free
programs whose distribution conditions are incompatible with these,
write to the author to ask for permission. For software which is
copyrighted by the Free Software Foundation, write to the Free
Software Foundation; we sometimes make exceptions for this. Our
decision will be guided by the two goals of preserving the free status
of all derivatives of our free software and of promoting the sharing
and reuse of software generally.
NO WARRANTY
15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO
WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW.
EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR
OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE
LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME
THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN
WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY
AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU
FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR
CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE
LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING
RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A
FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF
SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH
DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Libraries
If you develop a new library, and you want it to be of the greatest
possible use to the public, we recommend making it free software that
everyone can redistribute and change. You can do so by permitting
redistribution under these terms (or, alternatively, under the terms of the
ordinary General Public License).
To apply these terms, attach the following notices to the library. It is
safest to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least the
"copyright" line and a pointer to where the full notice is found.
<one line to give the library's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with this library; if not, write to the Free Software
Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Also add information on how to contact you by electronic and paper mail.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the library, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the
library `Frob' (a library for tweaking knobs) written by James Random Hacker.
<signature of Ty Coon>, 1 April 1990
Ty Coon, President of Vice
That's all there is to it!
Binary file not shown.
+19
View File
@@ -0,0 +1,19 @@
{
"revision": 8,
"runtime_sha256": "456e912c75cd389abcf6a63bc80e2a53bdc334371d00b200c93680388ae955e2",
"files": {
"libstagefrighthw.so": "141de01f0d40b97d4f9a720db5ab8ea442f6aa35db5272048899292884994a1c",
"podman.py": "529ea35d5d82474afe9e09ad1216d269317f56dd24a05fb100fced52b7b4dbca",
"media_codecs_frameport.xml": "46c3dcad2c43cc30bea3b3680b362ed84a99c15d60714bbbbb422c80ecf90405",
"COPYING.FFmpeg": "b634ab5640e258563c536e658cad87080553df6f34f62269a21d554844e58bfe"
},
"codecs": [
"video/hevc",
"video/avc",
"video/x-vnd.on2.vp9"
],
"build": {
"ndk_revision": "27.2.12479018",
"ffmpeg_source_sha256": "733984395e0dbbe5c046abda2dc49a5544e7e0e1e2366bba849222ae9e3a03b1"
}
}
+32
View File
@@ -0,0 +1,32 @@
<?xml version="1.0" encoding="utf-8"?>
<MediaCodecs>
<Decoders>
<MediaCodec name="OMX.frameport.hevc.decoder" type="video/hevc" rank="64">
<Limit name="size" min="128x128" max="8192x8192" />
<Limit name="alignment" value="2x2" />
<Limit name="block-size" value="16x16" />
<Limit name="block-count" range="1-138240" />
<Limit name="blocks-per-second" range="1-7864320" />
<Limit name="bitrate" range="1-245000000" />
<Limit name="concurrent-instances" max="1" />
</MediaCodec>
<MediaCodec name="OMX.frameport.avc.decoder" type="video/avc" rank="64">
<Limit name="size" min="128x128" max="8192x8192" />
<Limit name="alignment" value="2x2" />
<Limit name="block-size" value="16x16" />
<Limit name="block-count" range="1-138240" />
<Limit name="blocks-per-second" range="1-7864320" />
<Limit name="bitrate" range="1-245000000" />
<Limit name="concurrent-instances" max="1" />
</MediaCodec>
<MediaCodec name="OMX.frameport.vp9.decoder" type="video/x-vnd.on2.vp9" rank="64">
<Limit name="size" min="128x128" max="4096x2304" />
<Limit name="alignment" value="2x2" />
<Limit name="block-size" value="16x16" />
<Limit name="block-count" range="1-36864" />
<Limit name="blocks-per-second" range="1-2211840" />
<Limit name="bitrate" range="1-245000000" />
<Limit name="concurrent-instances" max="1" />
</MediaCodec>
</Decoders>
</MediaCodecs>
+146
View File
@@ -0,0 +1,146 @@
#!/usr/bin/python3
"""Add shared codecs to Lepton containers without editing their shared rootfs."""
import hashlib
import json
import os
import re
import shutil
import sys
from pathlib import Path
from xml.etree import ElementTree as ET
def mounts(directory, config, args):
if not args or args[0] != "run":
return []
name = None
for i, arg in enumerate(args):
if arg == "--name" and i + 1 < len(args):
name = args[i + 1]
elif arg.startswith("--name="):
name = arg.partition("=")[2]
if config.get("scope") == "shared":
# The common launcher supplies these values. Do not intercept arbitrary
# Podman containers or depend on a game's package/modified APK.
appid = os.environ.get("SteamAppId", "")
if not re.fullmatch(r"[0-9]+", appid) or name != f"lepton-steamlaunch-{appid}":
return []
app = Path(os.environ.get("STEAM_COMPAT_INSTALL_PATH", ""))
if app.name != "lepton-app" or not app.is_dir():
return []
root_arg = None
for i, arg in enumerate(args):
if arg == "--rootfs" and i + 1 < len(args):
root_arg = args[i + 1]
elif arg.startswith("--rootfs="):
root_arg = arg.partition("=")[2]
if not root_arg or not root_arg.endswith(":O"):
return []
root = Path(root_arg[:-2]).resolve()
else: # old per-game deployments remain compatible during migration
if name != f"lepton-steamlaunch-{config['appid']}":
return []
root = Path(config["lepton"]).resolve().parent / "images" / "rootfs"
expected_root = str(root) + ":O"
if not any(arg in (expected_root, "--rootfs=" + expected_root) for arg in args):
return []
device = Path("/dev/video-dec0")
runtime = root / "vendor/lib64/libstagefright_softomx.so"
upstream_plugin = (root / "vendor/lib64/libstagefrighthw.so").exists()
for i, arg in enumerate(args):
if arg == "--mount" and i + 1 < len(args):
fields = dict(item.split("=", 1) for item in args[i + 1].split(",") if "=" in item)
target = fields.get("destination", fields.get("target"))
if target == "/vendor/lib64/libstagefrighthw.so":
upstream_plugin = True
if target == "/vendor/lib64/libstagefright_softomx.so" and fields.get("source"):
runtime = Path(fields["source"])
if upstream_plugin:
print("FramePort video: using the runtime's hardware codec plugin", file=sys.stderr)
return []
if not device.exists() or hashlib.sha256(runtime.read_bytes()).hexdigest() != config["runtime_sha256"]:
print("FramePort video: device or runtime ABI differs; retaining the stock codecs", file=sys.stderr)
return []
xml = ET.parse(root / "vendor/etc/media_codecs.xml")
if not any(node.get("href") == "media_codecs_frameport.xml" for node in xml.getroot().findall("Include")):
ET.SubElement(xml.getroot(), "Include", href="media_codecs_frameport.xml")
# Immutable plugin versions can serve simultaneous app launches and
# different Lepton installations. Never share a temporary XML filename.
# The version directory stays as the agent verified it: the merged list goes to the user's runtime dir.
root_key = hashlib.sha256(str(root).encode()).hexdigest()[:16]
merged = merged_dir() / f"media_codecs.{root_key}.xml"
temporary = merged.with_suffix(f".{os.getpid()}.tmp")
xml.write(temporary, encoding="utf-8", xml_declaration=True)
temporary.replace(merged)
# Lepton supplies its own /dev tmpfs. A Podman --device node disappears
# beneath it; a bind mount matches Lepton's existing GPU/sound device setup.
result = ["--mount", f"type=bind,source={device},destination=/dev/video-dec0,rw"]
for path, target in (
(directory / "libstagefrighthw.so", "/vendor/lib64/libstagefrighthw.so"),
(merged, "/vendor/etc/media_codecs.xml"),
(directory / "media_codecs_frameport.xml", "/vendor/etc/media_codecs_frameport.xml"),
):
if not path.is_file():
raise FileNotFoundError(f"missing video codec mount: {path}")
result += ["--mount", f"type=bind,source={path},destination={target},ro"]
print("FramePort video: loading the Iris hardware codec plugin for this container", file=sys.stderr)
return result
def merged_dir():
runtime = os.environ.get("XDG_RUNTIME_DIR", "")
base = Path(runtime) if runtime and Path(runtime).is_dir() else Path.home() / ".cache"
path = base / "frameport-video"
path.mkdir(mode=0o700, parents=True, exist_ok=True)
return path
def switched_off(directory):
"""FRAMEPORT_NO_HW_VIDEO=1 (e.g. in a game's Steam launch options) or the Frame-wide switch (FramePort's
setting; the agent writes video-codec/disabled) leave every container with Android's stock codecs."""
if os.environ.get("FRAMEPORT_NO_HW_VIDEO", "") not in ("", "0"):
return True
return directory.parent.name == "versions" and (directory.parent.parent / "disabled").exists()
def real_podman(directory):
"""Find Podman independently of deployment.json, without recursing into this wrapper."""
own_bin = (directory / "bin").resolve()
own_script = Path(__file__).resolve()
# Lepton runs its `podman exec` calls (boot wait, app pid, logcat mirror) with the Android guest's PATH
# (/product/bin:/system/bin:...), which has no host Podman: the system folders come after PATH. Failing there
# broke Lepton's logcat mirror and app-pid checks, and the container was stopped early.
entries = os.environ.get("PATH", os.defpath).split(os.pathsep) + ["/usr/local/bin", "/usr/bin", "/bin"]
for entry in entries:
folder = Path(entry or os.curdir).resolve()
if folder == own_bin:
continue
found = shutil.which("podman", path=str(folder))
if found:
executable = Path(found).resolve()
if executable.parent != own_bin and executable != own_script:
return str(executable)
raise RuntimeError("no real Podman executable found outside the codec wrapper directory")
def main():
directory = Path(__file__).resolve().parent.parent
args = sys.argv[1:]
fallback = real_podman(directory)
if switched_off(directory):
os.execv(fallback, [fallback, *args])
try:
config = json.loads((directory / "deployment.json").read_text())
podman = Path(config.get("podman", fallback)).resolve()
if podman.parent == (directory / "bin").resolve() or podman == Path(__file__).resolve():
raise ValueError("configured Podman points to the codec wrapper")
extra = mounts(directory, config, args)
launch_args = [args[0], *extra, *args[1:]] if extra else args
os.execv(str(podman), [str(podman), *launch_args])
except Exception as exc: # a codec/configuration failure must never prevent the stock container from starting
print(f"FramePort video: retaining stock codecs: {exc}", file=sys.stderr)
os.execv(fallback, [fallback, *args])
if __name__ == "__main__":
main()
Binary file not shown.
@@ -6,10 +6,14 @@ notes: 'Native OpenXR GLES video player. 360 theatres/videos (equirect2 layers,
at 72 fps (the runtime asks for 4536 px per eye). Confirmed in the headset by the owner (2026-10-01). Its "Internal
Storage" list is /sdcard/4XPlayer: send videos with the game page''s "Add videos".'
details: 'Settings: equirect_emul (GLES worker draws 360 layers into a projection layer), stable_local, focus_hold,
scale 0.75; frame.swapchain_limit for its 7680x3840 theatre swapchains.'
scale 0.75; frame.swapchain_limit for its 7680x3840 theatre swapchains. Its hardware decoding mode (Java MediaCodec,
Vr4pMediaPlayer; its own FFmpeg is the software path) gets the Frame''s video hardware with frame.hw_video_decode
(added 2026-10-09, not yet checked in the headset).'
tested_version: 2.0.22
engine: Other
xr: OpenXR
frame:
- frame.hw_video_decode
adapter:
equirect_emul: 1
stable_local: 1
@@ -17,4 +21,6 @@ adapter:
scale: 0.75
verified:
date: '2026-10-01'
updated: '2026-10-10'
min_app: 0.12.1
source_hint: 4XVR Video Player
@@ -0,0 +1,19 @@
package: com.CortopiaStudios.Graves
title: GORN2
status: works
tested_version: 1.10.1
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-07'
known_good_sha256: d8a7949edc90c19902eb89f888ad6a350475387d630991767b3097667ad16d4c
app: 0.12.0
overport_cli: 1.2.5
issue: 88
source_hint: GORN 2
@@ -0,0 +1,24 @@
package: com.CyberneticWalrus.DoesitStackMeta
title: Does it Stack?
status: issues
notes: Mixed reality mode does not work (works on Demeo), everything else seems to be OK
tested_version: 1.02 (3683)
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
- frame.oculusos
device:
- device.text_input_window
adapter:
haptic_fix: 1
refresh_rate: 90.0
verified:
date: '2026-10-09'
known_good_sha256: e9fce6ac812f1bb317a7a6959be89c5ff54e87f56af9ad4c42dff5fecad4efe8
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261006.6173745'
agent: 59
issue: 112
source_hint: Does It Stack
+20
View File
@@ -0,0 +1,20 @@
package: com.ForwardXP.nuke
title: Please Don't Touch Anything
status: works
notes: Seems functional
tested_version: '2.2'
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
verified:
date: '2026-10-08'
known_good_sha256: 17460c8691b9ff82f6c7ca5597afd0aacf1a94aa30ebb6a6108802991b60f51b
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261006.6173745'
agent: 59
issue: 106
source_hint: apks
@@ -0,0 +1,21 @@
package: com.FourPlayersStudio.Retronika
title: Retronika
status: works
tested_version: 3.2.601
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-07'
known_good_sha256: 290bfda3969547d5f1ff621ab5a8be9dd322fc9eee6b243c863576d4ff5b4b81
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261006.6173745'
agent: 59
issue: 97
source_hint: Retronika
@@ -0,0 +1,21 @@
package: com.FunktronicLabs.TheLightBrigade
title: The Light Brigade
status: issues
notes: Judder in weapons
tested_version: '746'
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-09'
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 59
issue: 114
source_hint: The Light Brigade
@@ -1,20 +1,36 @@
package: com.ILMxLAB.VaderImmortal.ep1
title: 'Vader Immortal: Episode I'
status: issues
notes: Starts in VR and plays the intro, then stays on the loading card (Vader's portrait with a progress bar).
details: Unreal Engine 4 (GLES). After the Lucasfilm intro the game shows its loading card and never loads the next
scene; it keeps rendering at 72 fps and ignores input. Ruled out so far - Meta platform requests (all answered),
video playback (the intro is a video and plays), the no-ForceQuit build, frame and swapchain handling. The Frame's
SteamVR runtime also leaks about 20 MB of memory a second while the game runs, which slows it down after a few
minutes. Next steps - find what the loading code waits for in the game's engine library (no symbols) and report
the runtime leak to Valve. GitHub issue 49.
status: works
notes: 'Plays (owner''s headset test): loading card, grip/trigger, hands and the lightspeed sequence fixed. Thumbs
follow the touches with frame.unreal_thumb_touch (Klownicle''s engine fix).'
details: Unreal Engine 4 (GLES). Found and verified in gameplay on a Frame by Klownicle (GitHub issue 49); FramePort
reimplements them as patches that match the game's code exactly and change nothing when it differs. The engine's
Quest checks (IsRunningOnSantaCruz) answer "not a Quest" on the Frame - the menu waited for a Quest-only shader
precompile that never starts (frame.unreal_quest_precompile makes the non-Quest branch report 100 %), and the
game's key map picked the empty Gear VR controls (frame.unreal_quest_keymap keeps the Quest set). Two lightspeed
shaders, as the Frame's GL driver (Zink) compiles them, read loop counters and accumulators before setting them
and hang the GPU; a Vulkan layer under Zink inserts the stores that zero them (frame.zink_shader_fix, zink_shader_fix;
the captured modules are from Lepton 2.8.14 and stop matching if a Frame update changes the driver's output -
the layer logs that). OVRPlugin asks for poses at its own clock (pose_time_fix). Unreal's Oculus input animates the
thumbs from near-touch, which the Frame never reports (frame.unreal_thumb_touch reads the touches instead; binding
the proximity to the touch inputs in FrameBridge didn't reach the game). The Frame's SteamVR runtime
leaks about 20 MB a second while the game runs (reported to Valve).
tested_version: 1.1.1+667256.cl.387770
engine: Unreal
xr: VrApi
alt_overport:
- patch_remove_unreal_force_quit
frame:
- frame.unreal_quest_precompile
- frame.unreal_quest_keymap
- frame.unreal_thumb_touch
- frame.zink_shader_fix
adapter:
pose_time_fix: 1
zink_shader_fix: 12016:b2919629761ad268e0b21c72ac8520f81b76e72c3adc26897f97eae9a3e70a66:3092:0x0003003e,166,82,0x0003003e,177,82,0x0003003e,183,82,0x0003003e,189,82,0x0003003e,256,82,0x0003003e,266,82,0x0003003e,272,82,0x0003003e,278,82,0x0003003e,318,82,0x0003003e,328,82,0x0003003e,334,82,0x0003003e,340,82;11436:6f18be49f2aaa2452f962eb16f7af744c5c22a7b25f37ce681e0d6399e35a869:2932:0x0003003e,164,80,0x0003003e,175,80,0x0003003e,181,80,0x0003003e,187,80,0x0003003e,254,80,0x0003003e,264,80,0x0003003e,270,80,0x0003003e,276,80,0x0003003e,316,80,0x0003003e,326,80,0x0003003e,332,80,0x0003003e,338,80
verified:
date: '2026-10-06'
date: '2026-10-09'
issue: 49
updated: '2026-10-06'
updated: '2026-10-09'
min_app: 0.12.1
source_hint: Vader Immortal- Episode I
@@ -0,0 +1,24 @@
package: com.ILMxLAB.VaderImmortal.ep2
title: 'Vader Immortal: Episode II'
status: works
notes: Plays (owner's headset test, 2026-10-09) with Episode I's fixes, which FramePort finds in this episode's
code too.
details: 'Unreal Engine 4 (GLES), the same engine build as Episode I: the Quest-only shader precompile and key map
(frame.unreal_quest_precompile, frame.unreal_quest_keymap), thumbs from the capacitive touches (frame.unreal_thumb_touch)
and poses at OVRPlugin''s clock (pose_time_fix). Episode I''s lightspeed shader fix doesn''t apply. The campaign
intro once stayed black with its dialogue playing; it rendered on the next run.'
tested_version: 2.0.3+667261.cl.387778
engine: Unreal
xr: VrApi
alt_overport:
- patch_remove_unreal_force_quit
frame:
- frame.unreal_quest_precompile
- frame.unreal_quest_keymap
- frame.unreal_thumb_touch
adapter:
pose_time_fix: 1
verified:
date: '2026-10-09'
min_app: 0.12.1
source_hint: Vader Immortal- Episode II
@@ -0,0 +1,24 @@
package: com.ILMxLAB.VaderImmortal.ep3
title: 'Vader Immortal: Episode III'
status: works
notes: Plays (owner's headset test, 2026-10-09) with Episode I's fixes, which FramePort finds in this episode's
code too.
details: 'Unreal Engine 4 (GLES), the same engine build as Episode I: the Quest-only shader precompile and key map
(frame.unreal_quest_precompile, frame.unreal_quest_keymap), thumbs from the capacitive touches (frame.unreal_thumb_touch)
and poses at OVRPlugin''s clock (pose_time_fix). Episode I''s lightspeed shader fix doesn''t apply. The campaign
intro starts on a black screen with only dialogue: that is the game, not a bug.'
tested_version: 3.0.3+667263.cl.387932
engine: Unreal
xr: VrApi
alt_overport:
- patch_remove_unreal_force_quit
frame:
- frame.unreal_quest_precompile
- frame.unreal_quest_keymap
- frame.unreal_thumb_touch
adapter:
pose_time_fix: 1
verified:
date: '2026-10-09'
min_app: 0.12.1
source_hint: Vader Immortal- Episode III
@@ -4,14 +4,25 @@ tested_version: '1.03'
source_hint: Sniper Elite VR
engine: Unity
xr: VrApi
status: unsupported
notes: 'GPU hang (zink: DEVICE LOST) even with MSAA off.'
details: Legacy VrApi via OVRPlugin.
status: works
notes: Plays with controllers that follow the hands (owner's headset test, Quest 1 build 35713).
details: 'Unity 2019.4 on its built-in Oculus VR (no Oculus XR Plugin). Its legacy frame loop never waited for frames
(GPU hang, zink: DEVICE LOST) and Unity reported only Go controllers: frame.unity_oculus_check. Its frame wait then
deadlocked at the Init scene (Unity skipped beginning a frame) and the physics-step pose update located the
controllers in the past (hands lagged): both fixed in that patch (revision 5), switched on for this game by ovrp_begin_gate and ovrp_hold_physics. Grabbing an object in the tutorial
vibrates the controller; OVRPort''s envelope conversion then allocated ~1 GB/s until the memory cap ended the game:
haptic_fix. The Quest 1 build has no arraysources_assets.hd, which the game loads on the Frame (it detects a Quest
2) and then skips the .sd bundle: textures are missing unless the .sd file is also present under the .hd name in the
game''s obb folder.'
frame:
- frame.unity_no_msaa
- frame.unity_oculus_check
adapter:
haptic_fix: 1
ovrp_begin_gate: 1
ovrp_hold_physics: 1
pcvr_alternative: Sniper Elite VR (Steam PC VR). Winter Warrior (Quest) works on the Frame.
verified:
date: '2026-09-28'
overport_cli: 1.2.3
overport_runtime: 3.4.3-23204ea
known_good_sha256: e80883eb6945e267a375edf0ffd789268686440819add9d4dee9fdbdf466cbe4
date: '2026-10-07'
updated: '2026-10-07'
min_app: 0.12.1
@@ -0,0 +1,24 @@
package: com.MightyCoconut.WalkaboutMiniGolf
title: Walkabout Mini Golf
status: works
tested_version: '6.7'
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
- frame.oculusos
device:
- device.text_input_window
adapter:
controller_models: 1
haptic_fix: 1
scale: 1.5
verified:
date: '2026-10-08'
known_good_sha256: cf3b5cbe131af8a2baaa3f5c665b1c7001c8b299473abfcf98d90e3c02b08a2f
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 59
issue: 111
source_hint: Walkabout Mini Golf
+12 -9
View File
@@ -1,12 +1,14 @@
package: com.PixelToys.BattleSisters
title: BattleSisters
status: works
notes: Starts in VR; controller buttons and vibration work.
details: Unity 2019.4 with Unity's built-in Oculus support. frame.unity_oculus_check starts VR (Meta's system-app check),
adds the frame wait its legacy loop never makes (without it the GPU hung, black screen) and lets Unity's Oculus
input accept Lepton's device model (it only reported controllers on a device named "Oculus ...", so buttons were
dead). haptic_fix stops the first controller vibration from freezing the Frame (OVRPort's loader read its duration
in nanoseconds as seconds and the game ran out of memory).
notes: Plays; controller buttons and vibration work, hands follow the controllers (pose_time_fix).
details: Unity 2019.4 with Unity's built-in Oculus support. frame.unity_oculus_check starts VR (Meta's system-app
check), adds the frame wait its legacy loop never makes (without it the GPU hung, black screen) and lets Unity's
Oculus input accept Lepton's device model (it only reported controllers on a device named "Oculus ...", so buttons
were dead). haptic_fix stops the first controller vibration from freezing the Frame (OVRPort's loader read its
duration in nanoseconds as seconds and the game ran out of memory). With pose_time_fix the hands follow the controllers
- OVRPlugin asked for hand poses at Android's monotonic "now", which on SteamOS 0.4.5 is 2.56 s behind the runtime's
XrTime, so the hands lagged far behind.
tested_version: 1.2.4
engine: Unity
xr: VrApi
@@ -18,9 +20,10 @@ frame:
- frame.unity_gl_shim
adapter:
haptic_fix: 1
pose_time_fix: 1
verified:
date: '2026-10-06'
date: '2026-10-08'
issue: 48
updated: '2026-10-06'
min_app: 0.11.1
updated: '2026-10-08'
min_app: 0.12.1
source_hint: Battle Sister
+14 -9
View File
@@ -1,20 +1,25 @@
package: com.StressLevelZero.BONELAB
title: BONELAB
status: issues
notes: Build 1.2068 plays (head tracking and controls work with frame.unity_user_presence); the newer build
1.2974 crashes at start (Vulkan), no fix yet.
details: The game runs (72 fps), but its player body stayed frozen - no head tracking, the controllers stuck to the
status: works
notes: Builds 1.2068 and 1.2974 play (owner's headset tests). One recipe serves both builds - 1.2068 needs frame.unity_user_presence (head tracking and controls); 1.2974 also needs
frame.slz_vulkan_hooks (72 fps after about 35 s of shader prewarming), which changes nothing on 1.2068.
details: Both builds - the game runs (72 fps), but its player body stayed frozen - no head tracking, the controllers stuck to the
model, no buttons - although OpenXR input and OVRPlugin's poses were fine. A few seconds after start (when Unity
switches XR loaders) OVRPlugin reports the worn headset as not worn (ovrp_GetUserPresent2 -> 0), and BONELAB's
Marrow rig only follows the player while Unity's HMD device reports UserPresence (OpenControllerRig checks
XRHMD.IsUserPresent every frame). frame.unity_user_presence reports the headset as worn. Build 1.2974 crashed
earlier in vkCreateInstance from the game's own Vulkan hooks (libSLZQuestNative.so); not retested.
tested_version: 1.2068.34555
XRHMD.IsUserPresent every frame). frame.unity_user_presence reports the headset as worn. Build 1.2974 only - it adds Stress
Level Zero's graphics plugin (libSLZQuestNative.so), which hooks Unity's Vulkan start-up (vkCreateInstance/
vkCreateDevice) and vkCreateSampler; on the Frame its vkCreateInstance wrapper calls an invalid pointer right after
OVRPlugin's pre-init instance is destroyed (a jump to 0, into an unloaded library, or a hang). frame.slz_vulkan_hooks
switches its two registrations off, so Unity starts Vulkan itself; the plugin's pipeline cache isn't used.
tested_version: 1.2974.57485
engine: Unity
xr: OpenXR
frame:
- frame.unity_user_presence
- frame.slz_vulkan_hooks
verified:
date: '2026-10-06'
updated: '2026-10-06'
date: '2026-10-08'
updated: '2026-10-08'
source_hint: BONELAB
min_app: 0.12.1
+22
View File
@@ -0,0 +1,22 @@
package: com.ToastVR.Matilda
title: Max Mustard
status: works
tested_version: 1.1.0
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-08'
known_good_sha256: 9a67459fa36c3b9ac0d035e6c43b1809b39772699333e8cfd7d2a4d7f17926a7
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 59
issue: 110
updated: '2026-10-09'
source_hint: com.ToastVR.Matilda
@@ -0,0 +1,22 @@
package: com.ToastVR.RichiesPlankExperience
title: Richie's Plank Experience
status: works
tested_version: 2.3.585
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
- frame.oculusos
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-07'
known_good_sha256: 7eb1366c2398e554624a5d66aefb0c731c64b1b6944bc1726d9187f123c37fb7
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20260922.6101926'
agent: 59
issue: 95
source_hint: apks
@@ -0,0 +1,23 @@
package: com.VR_HOT.VR_HOT_Quest
title: VR HOT Quest
status: works
tested_version: 0.8.8.1
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
- frame.oculusos
device:
- device.text_input_window
adapter:
controller_models: 1
haptic_fix: 1
verified:
date: '2026-10-08'
known_good_sha256: c321606c536eba1673572a9f44fd9e20d2cb5bd9609d536189e6ad7057b9cd67
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20260922.6101926'
agent: 59
issue: 98
source_hint: VR-HOT-Quest-0.8.8.1
+12 -5
View File
@@ -1,16 +1,23 @@
package: com.YourCompany.RoboRecall
title: Robo Recall
tested_version: '1.0'
source_hint: 'Robo Recall- Unplugged'
engine: Unreal
xr: VrApi
status: works
notes: Needed the LAUNCHER category fix.
details: Source APK is a community 'patch+savefix+90Hz' build.
details: Source APK is a community 'patch+savefix+90Hz' build. Shares Vader Immortal's Unreal 4 Oculus input (GitHub
49) - thumbs follow the touches (frame.unreal_thumb_touch) and controller poses are located at the right time
(pose_time_fix); added 2026-10-09 from Vader's headset results, not yet checked in this game.
tested_version: '1.0'
engine: Unreal
xr: VrApi
alt_overport:
- patch_remove_unreal_force_quit
frame:
- frame.unreal_thumb_touch
adapter:
pose_time_fix: 1
verified:
date: '2026-09-28'
overport_cli: 1.2.3
overport_runtime: 3.4.3-23204ea
known_good_sha256: 4ce6e3563da131e288645f4b62a634ac98043e55bc42b7c0ad5f461207d5f53c
updated: '2026-10-09'
source_hint: Robo Recall- Unplugged
+25
View File
@@ -0,0 +1,25 @@
package: com.YourCompany.SWP_VR
title: Star Wars Pinball VR
tested_version: '1.4'
source_hint: Star Wars Pinball VR
engine: Unreal
xr: VrApi
status: works
notes: Plays (owner's headset test); one short audio hiccup seen.
details: 'Unreal 4.25 (GLES) built against OVRPlugin 1.44. Four Frame problems in a row: Zink crashed on Unreal''s
multisampled render-to-texture (fault addr 0x10000 on the RHIThread): frame.unreal_gl_shim, which keeps Unreal''s
multiview and draws those passes single-sampled; OVRPort''s OVRPlugin lacks ovrp_GetPTWNear, so Unreal''s Oculus
module never started VR (then a null pointer in GetMultiViewSceneColor): frame.unreal_ovrp_entrypoints; the engine''s
OpenSSL assembly has unpaired return-address checks the Frame''s CPU enforces (SIGILL on the HttpManager thread):
frame.pac_hints. Needs its OBB (main.10765).'
frame:
- frame.unreal_gl_shim
- frame.unreal_ovrp_entrypoints
- frame.pac_hints
adapter:
haptic_fix: 1
verified:
date: '2026-10-08'
issue: 83
updated: '2026-10-08'
min_app: 0.12.1
+23
View File
@@ -0,0 +1,23 @@
package: com.asg.clockworkdev
title: Clockwork
status: works
tested_version: '1.29'
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
- frame.unity_runtime_msaa_off
- frame.unity_gl_shim
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-09'
known_good_sha256: 99a14eed320ba267153963c2d87d4e11e943b9920d48dfd5352c681de28ba95e
app: 0.12.1.dev203
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 67
issue: 116
source_hint: com.asg.clockworkdev
+15 -5
View File
@@ -1,16 +1,26 @@
package: com.camouflaj.manta
title: 'Batman: Arkham Shadow'
tested_version: 1.4.1-350961
source_hint: 'Batman- Arkham Shadow'
engine: Unity
xr: OpenXR
status: works
notes: Large data set (~27 GiB) incl. all language packs.
details: Uses Meta XR Audio (Wwise), handled by patch_meta_xr_audio. Includes all language packs (~28 GB).
details: 'Uses Meta XR Audio (Wwise), handled by patch_meta_xr_audio. Cutscenes are 8K HEVC panoramas: frame.hw_video_decode
decodes them on the Frame''s hardware, adapter.surface_native shows them in stereo (PRs #96, #128). Includes all
language packs (~28 GB). sync_guard (to test, GitHub #102) - a reporter''s game quit to Steam''s waiting screen
after the Frame runtime crashed in xrSyncActions (CVRInputLatest::UpdateActionState), the input race Myst had.'
tested_version: 1.4.1-350961
engine: Unity
xr: OpenXR
overport_extra:
- patch_disable_space_warp
frame:
- frame.hw_video_decode
adapter:
sync_guard: 1
surface_native: 1
verified:
date: '2026-09-28'
overport_cli: 1.2.3
overport_runtime: 3.4.3-23204ea
known_good_sha256: 796d1cab08f614d72c50d9702974c719c0908f5de91183cfe02d1fd5eb09551a
updated: '2026-10-10'
min_app: 0.12.1
source_hint: Batman- Arkham Shadow
@@ -0,0 +1,21 @@
package: com.characterBank.ruinsmagus
title: RUINSMAGUS
status: works
tested_version: 1.2.3
engine: Unity
xr: VrApi
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-07'
known_good_sha256: 230e5625fbdfebb47c4d96aeb87f45270e4e6c79a33c08315c90626ab7465fe5
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261006.6173745'
agent: 59
issue: 89
source_hint: RUINSMAGUS
@@ -0,0 +1,22 @@
package: com.cloudheadgames.pistolwhip
title: Pistol Whip
status: works
notes: 120 HZ
tested_version: 1.6.0.2
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-08'
known_good_sha256: ebd97af54c59d59d473c98492c1bb26ce6059b7e59f27a08acfc5ada23d0b774
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261006.6173745'
agent: 59
issue: 100
source_hint: Pistol Whip
+22
View File
@@ -0,0 +1,22 @@
package: com.curif.AgeOfJoy
title: AgeOfJoy
status: works
tested_version: '1.0'
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
- frame.oculusos
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-08'
known_good_sha256: 9780e71e417b86290147c67c83755305801d8957909734c717a519034edefdf9
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261006.6173745'
agent: 59
issue: 103
source_hint: AgeOfJoy 05 rc30
+10 -1
View File
@@ -1,14 +1,23 @@
package: com.drbeef.doom3quest
title: Doom3Quest
status: issues
notes: PDA shows black screen.
notes: 'HUD and PDA show with frame.gl_multiview_fbo (Xandrix1987''s headset test, GitHub 77, no frame-rate drop
with the PDA open). Open: the player sometimes switches between walking and running.'
details: Doom3Quest compiles every vertex shader for two views (OVR_multiview) and draws its HUD/PDA into single-view
framebuffers. Mesa (the Frame's GL) rejects that combination as the spec requires and drops the draw, so they
stayed black; frame.gl_multiview_fbo draws those passes with a single-view copy of the shader. Xandrix1987's framebuffer.cpp
fix went upstream as a PR to Team-Beef-Studios/Doom3Quest.
tested_version: 1.4.8
engine: Other
xr: OpenXR
frame:
- frame.gl_multiview_fbo
verified:
date: '2026-10-06'
known_good_sha256: 3c733f5cd08b6fa5d9e2775b1c1aee36b5943518b5d8451ba87e3c59a702afdf
app: 0.12.0
overport_cli: 1.2.3
issue: 77
updated: '2026-10-09'
min_app: 0.12.1
source_hint: doom3quest148
+11 -5
View File
@@ -1,18 +1,24 @@
package: com.forcefieldxr.timestall
title: Time Stall
tested_version: '1.0'
source_hint: Time Stall
engine: Unreal
xr: VrApi
status: issues
notes: Both eyes distort during movement (unresolved).
details: Legacy VrApi via OVRPlugin.
details: Legacy VrApi via OVRPlugin. Shares Vader Immortal's Unreal 4 Oculus input (GitHub 49) - thumbs follow the
touches (frame.unreal_thumb_touch) and controller poses are located at the right time (pose_time_fix); added 2026-10-09
from Vader's headset results, not yet checked in this game.
tested_version: '1.0'
engine: Unreal
xr: VrApi
alt_overport:
- patch_remove_unreal_force_quit
frame:
- frame.unreal_thumb_touch
- frame.nodebug
adapter:
pose_time_fix: 1
verified:
date: '2026-09-28'
overport_cli: 1.2.3
overport_runtime: 3.4.3-23204ea
known_good_sha256: 50c4d05b663b2f1851f5a7d650a89fd34587d65fafba13cc58671c96f7016db6
updated: '2026-10-09'
source_hint: Time Stall
+24
View File
@@ -0,0 +1,24 @@
package: com.harmonixmusic.kata
title: Audica
status: works
tested_version: 1.0.3.4
engine: Unity
xr: VrApi
frame:
- frame.unity_no_msaa
- frame.unity_oculus_check
- frame.unity_text_input
- frame.unity_runtime_msaa_off
- frame.unity_gl_shim
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-09'
known_good_sha256: bc2c00fe049fd97643a296e622e3a90e08f0bd796ee454b0278525e9d72f8139
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 59
issue: 126
+27
View File
@@ -0,0 +1,27 @@
package: com.ilmxlab.tales
title: 'Star Wars: Tales from the Galaxy''s Edge'
status: unknown
notes: Not working yet - after the intro logos and a loading animation the picture goes black (GitHub issue 61).
Gets the controller fixes found for Vader Immortal (same studio) to test.
details: Unreal Engine 4 (GLES) by ILMxLAB, like Vader Immortal, but a different engine build - it has none of Vader's
Quest-only branches (no GetQuestShaderPrecompilePercent or RPOC key map; it precompiles shaders through Unreal's
own pipeline cache), so frame.unreal_quest_precompile/_keymap don't apply. Its seasons and Wwise banks are Meta
platform asset files (frame.asset_files). Vader's OVRPlugin findings apply to it as well - poses asked for at
OVRPlugin's own clock (pose_time_fix) and thumbs from near-touch, which the Frame never reports (frame.unreal_thumb_touch).
Ruled out for the black picture so far (see the diagnostics in issue 61) - asset-file paks, Valve's foveation,
GL errors; the game's own eye image reads back black.
tested_version: 1.1.9+734762.cl.439699
engine: Unreal
xr: VrApi
alt_overport:
- patch_remove_unreal_force_quit
frame:
- frame.asset_files
- frame.unreal_thumb_touch
adapter:
pose_time_fix: 1
verified:
issue: 61
updated: '2026-10-09'
min_app: 0.12.1
source_hint: Star Wars- Tales from the Galaxys Edge
@@ -0,0 +1,24 @@
package: com.markschramm.gravitylab
title: Gravity Lab
status: works
notes: Works great, along with passthrough mode
tested_version: '1.221'
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
- frame.unity_runtime_msaa_off
- frame.unity_gl_shim
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-09'
known_good_sha256: 82c95ee7487a130b6e37b967dd97c8362bf4d7e873fcff823f2c7877a62ff99f
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 59
issue: 113
source_hint: Gravity Lab
+12 -5
View File
@@ -1,18 +1,25 @@
package: com.nDreams.PhantomQuest
title: 'Phantom: Covert Ops'
tested_version: '1.2'
source_hint: 'Phantom- Covert Ops'
engine: Unreal
xr: VrApi
status: issues
notes: DLC/store button crashes (no Meta store). Installs the no-ForceQuit build.
details: The regular build quits itself on the Frame (System.exit after a failed platform check), so the no-ForceQuit
build is installed.
build is installed. Shares Vader Immortal's Unreal 4 Oculus input (GitHub 49) - thumbs follow the touches (frame.unreal_thumb_touch)
and controller poses are located at the right time (pose_time_fix); added 2026-10-09 from Vader's headset results,
not yet checked in this game.
tested_version: '1.2'
engine: Unreal
xr: VrApi
alt_overport:
- patch_remove_unreal_force_quit
use_alt: true
frame:
- frame.unreal_thumb_touch
adapter:
pose_time_fix: 1
verified:
date: '2026-09-28'
overport_cli: 1.2.3
overport_runtime: 3.4.3-23204ea
known_good_sha256: bb02b20a23fbe5e7a580ecde15a3cd015a4183421a51aab4a4dbcddbe7a09d1d
updated: '2026-10-09'
source_hint: Phantom- Covert Ops
+22
View File
@@ -0,0 +1,22 @@
package: com.tvb.cubism
title: cubism
status: works
tested_version: 1.8.0
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
scale: 2.0
verified:
date: '2026-10-09'
known_good_sha256: e4e000392e53111a991c1c78653c9241b8007f81dcdb126a4f4b2003033f5a6e
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 59
issue: 121
source_hint: com.tvb.cubism
+21
View File
@@ -0,0 +1,21 @@
package: jp.co.amata.nvr
title: The Tale of Onogoro
status: works
tested_version: 1.1.0248
engine: Unity
xr: OpenXR
frame:
- frame.unity_text_input
device:
- device.text_input_window
adapter:
haptic_fix: 1
verified:
date: '2026-10-08'
known_good_sha256: d6258a22a0a6e4588bbec36170f126bffc44ed64784f99e37cf4b03ad212fb79
app: 0.12.0
overport_cli: 1.2.5
frame_build: '20261006.6173745'
agent: 59
issue: 108
source_hint: The Tale of Onogoro
+18
View File
@@ -0,0 +1,18 @@
package: org.timecrisis.quest
title: Time Crisis VR (Experimental)
status: works
notes: Fully playable at 120hz
tested_version: 0.8.3
engine: Other
xr: OpenXR
frame:
- frame.sdl_clipboard
verified:
date: '2026-10-06'
known_good_sha256: 4e67411a8e9c0b5fbb19ca7908c4222aa5ad1b2d47a8f4b5018b4fe9c2f5324e
app: 0.12.0
overport_cli: 1.2.3
frame_build: '20261005.6135832'
agent: 59
issue: 80
source_hint: TimeCrisisVR-v0.8.3-quest
+23
View File
@@ -0,0 +1,23 @@
package: rift.beat_saber
title: Beat Saber
status: works
notes: 'Supports SteamVR/OpenXR itself: runs directly, without Revive.'
engine: Unity
xr: OpenXR
kind: rift
quest_package: com.beatgames.beatsaber.lj369vr
pcvr:
- pcvr.xr_timefix
- pcvr.steamvr_tuning
pcvr_remove:
- pcvr.revive
as_is: true
verified:
date: '2026-10-09'
known_good_sha256: dc80bd1f8e46e7b7cdc4fbf51a84e68f2918d7481581bee9c21f65e269c6bf8b
app: 0.12.1.dev232
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 67
issue: 117
source_hint: Beat Saber
@@ -0,0 +1,23 @@
package: rift.eleven_table_tennis_vr
title: 'Eleven: Table Tennis VR'
status: works
notes: 'Supports SteamVR/OpenXR itself: runs directly, without Revive (arguments -vrmode OpenVR).'
engine: Unity
xr: LibOVR+OpenVR
kind: rift
pcvr:
- pcvr.launch_args
- pcvr.xr_timefix
- pcvr.steamvr_tuning
pcvr_remove:
- pcvr.revive
as_is: true
verified:
date: '2026-10-09'
known_good_sha256: 2cdfe3b096aef0bf76726156699381f7847ca4d17e7222359521982778470705
app: 0.12.1.dev232
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 67
issue: 118
source_hint: Eleven Table Tennis VR
+22
View File
@@ -0,0 +1,22 @@
package: rift.pistol_whip
title: Pistol Whip
status: works
notes: 'Supports SteamVR/OpenXR itself: runs directly, without Revive.'
engine: Unity
xr: OpenVR
kind: rift
pcvr:
- pcvr.xr_timefix
- pcvr.steamvr_tuning
pcvr_remove:
- pcvr.revive
as_is: true
verified:
date: '2026-10-09'
known_good_sha256: a01eaf73b292eb8ad21e316f1f99ad258d41b2cfed8436858a8caf2f5c0a4d28
app: 0.12.1.dev232
overport_cli: 1.2.5
frame_build: '20261007.6125817'
agent: 67
issue: 119
source_hint: Pistol Whip
+4 -1
View File
@@ -1,11 +1,13 @@
package: rift.superhot_vr
title: SUPERHOT VR
status: works
notes: 'Supports SteamVR/OpenXR itself: runs directly, without Revive.'
notes: 'Supports SteamVR/OpenXR itself: runs directly, without Revive. Older builds with both Oculus and SteamVR support
(SUPERHOTVR.exe) start with -vrmode OpenVR.'
engine: Unity
xr: OpenXR
kind: rift
pcvr:
- pcvr.launch_args
- pcvr.xr_timefix
- pcvr.steamvr_tuning
pcvr_remove:
@@ -17,4 +19,5 @@ verified:
app: 0.6.3
overport_cli: 1.2.5
issue: 28
updated: '2026-10-09'
source_hint: SUPERHOT VR
@@ -3,9 +3,11 @@ title: SUPERHOT VR
status: works
notes: Quest version. Needs FramePort's launcher from agent 41 (reinstall once) so the game can write its saves.
details: Unity 2018.4 with built-in Oculus support (GLES). Its eye swapchains (GL_RGBA8, then a 16-bit format) are
created as sRGB by the adapter's swapchain_fix. The game creates its cloud save folder with mode 1700 and quits at
start when it can't write there ("Something failed to initialize. Quitting!"); the launcher now gives every folder
in the game's storage owner and group write permission.
created as sRGB by the adapter's swapchain_fix. The game creates its cloud save folder with mode 1700 and quits
at start when it can't write there ("Something failed to initialize. Quitting!"); the launcher now gives every
folder in the game's storage owner and group write permission. On the very first start it creates that folder
and checks it in the same moment (GitHub 120), so the recipe creates the folder at install (device_files) and
the launcher fixes it before the game starts.
tested_version: '1.161'
engine: Unity
xr: VrApi
@@ -16,7 +18,9 @@ frame:
- frame.unity_runtime_msaa_off
device:
- device.text_input_window
device_files:
cloud/data/.frameport: ''
verified:
date: '2026-10-04'
updated: '2026-10-04'
updated: '2026-10-09'
source_hint: SUPERHOT VR
+120 -3
View File
@@ -1,6 +1,9 @@
# Log signature -> diagnosis -> suggested fix. Matched against the game's launch.log (Lepton logcat mirror).
# `pattern` is a Python regex (case-sensitive). `suggest` lists patch ids (+ optional params) the UI offers as
# "apply and rebuild". `severity`: fatal (the game can't run) | error | warning | info.
# "apply and rebuild"; an entry `adapter.<key>=<value>` sets that value (else 1). `severity`: fatal (the game can't
# run) | error | warning | info. `question`: a symptom only the player sees (play sessions, GUI "Last session"): its
# fix is offered as that question, never applied on its own. `report: true` = the useful next step is a problem report
# with diagnostics. Launch tests and real play sessions (agent session_log, `frameport session`) are both matched.
# Sources: every failure we hit porting 34 games to the Steam Frame (see docs/PLAYBOOK.md).
signatures:
- id: no-launcher
@@ -8,16 +11,48 @@ signatures:
severity: fatal
diagnosis: Lepton found no activity with category LAUNCHER.
suggest: [frame.launcher]
- id: linux-x86-no-fex
pattern: '(cannot execute binary file: Exec format error|exec format error)'
severity: fatal
diagnosis: An x86_64 Linux program was started directly; the Frame's arm64 CPU needs FEX (x86 translation) for it. Reinstall the app with FramePort 0.12 or newer, which installs FEX and starts the program through it.
suggest: []
- id: no-arm64
pattern: 'INSTALL_FAILED_NO_MATCHING_ABIS'
severity: fatal
diagnosis: 32-bit-only APK. The Steam Frame has no AArch32; this cannot run. Use the PC (Rift) version via Revive.
suggest: []
- id: android-too-new
pattern: '(NoClassDefFoundError|ClassNotFoundException|NoSuchMethodError)\b.*(Landroid/window/|android\.window\.|(storeStoreFence|loadLoadFence)\(\)V in class Ljava/lang/invoke/VarHandle)'
severity: fatal
diagnosis: The app calls Android 12/13+ classes (android.window.*, VarHandle fences) that the Frame's Android 11 (API 30) container doesn't have; it needs a newer Android (e.g. minSdk 34 apps). Nothing to patch - it can't run until Valve updates the Frame's Android.
suggest: []
- id: web-wrapper
pattern: '(Creating TwaLauncher for com\.oculus\.browser|NameNotFoundException:? com\.oculus\.browser)'
severity: fatal
diagnosis: The app is a website in an Android wrapper (Trusted Web Activity) that opens Meta's browser (com.oculus.browser), which the Frame doesn't have, so nothing opens. There's no game to port - open the website (TWALauncherActivity's "Using URL" line) in a browser instead.
suggest: []
- id: unity-render-crash
pattern: 'CRASH\s*:.*#\d+\s+pc [0-9a-f]+\s+\S*libgallium_dri\.so'
severity: fatal
diagnosis: Unity's render thread crashed inside the Frame's GL driver (Mesa/Zink); the picture freezes or stays grey. Known triggers - an OVROverlay copy (try "Unity - skip OVROverlay layers", e.g. The Room VR), MSAA switched on at runtime, a GPU hang (zink DEVICE LOST).
suggest: [frame.unity_no_overlay_copy, frame.unity_runtime_msaa_off]
- id: unreal-msrtt-crash
pattern: 'fault addr 0x10000 in tid \d+ \((RHIThread|RenderThread)|libgallium_dri\.so \(find_rp_state'
severity: fatal
diagnosis: The game's render thread jumped to address 0x10000 inside the Frame's GL driver (Mesa/Zink) a few seconds after start. Zink looks up a render-pass cache slot past its end when a game renders through multisampled render-to-texture (Unreal's mobile MSAA, e.g. Star Wars Pinball VR). Load the GL shim, which hides that extension.
suggest: [frame.unreal_gl_shim, frame.unity_gl_shim]
- id: pac-unpaired
pattern: 'ILL_ILLOPN\), fault addr 0x[0-9a-f]+ \(\*pc=0xd50323bf\)'
severity: fatal
diagnosis: The game crashed on a return-address check (autiasp) its code never signed. The Quest's CPU ignores these checks, the Steam Frame's enforces them; some OpenSSL assembly in game engines has such unpaired checks (e.g. Star Wars Pinball VR on its first network connection, thread HttpManager).
suggest: [frame.pac_hints]
- id: cube-swapchain-refused
pattern: 'xrCreateSwapchain \d+x\d+ .*faces=6 .*result=-\d+'
unless: 'cube_standin: runtime refused'
severity: fatal
diagnosis: The game asked for a cube-map swapchain (e.g. an OVROverlay skybox or loading cube), which the Frame's runtime doesn't have. OVRPlugin goes on without images and crashes in ovrp_EndFrame4 (Unity's render thread SIGSEGV in memset, e.g. Budget Cuts Ultimate). FrameBridge from FramePort 0.12 serves such swapchains itself and drops their cube layers (cube_standin, GLES games) - rebuild and reinstall the game.
suggest: [frame.adapter]
supersedes: [unity-render-crash, native-crash]
- id: swapchain-rect-invalid
pattern: 'xrEndFrame failed -25\b'
severity: fatal
@@ -50,11 +85,45 @@ signatures:
severity: error
diagnosis: The Frame rejected the swapchain format (GLES games must use sRGB formats, no MSAA).
suggest: [adapter.swapchain_fix]
- id: zink-shader-fix-mismatch
pattern: 'shader fix layer: a \d+-byte module differs from fix'
severity: warning
diagnosis: The game's OpenGL ES shader fix (zink_shader_fix) no longer matches the shader the Frame's GL driver (Zink)
builds - a Frame or Lepton update changed the driver's output - so the shader it fixes (e.g. Vader Immortal's
lightspeed jump) may hang the GPU again. Capture the new modules with zink_shader_dump=1 (files/fp_spirv/ in the
game's storage) and update the fix.
suggest: [adapter.zink_shader_dump]
- id: zink-shader-layer-inactive
pattern: 'shader fix layer: NOT active'
severity: warning
diagnosis: FramePort's shader-fix Vulkan layer couldn't add itself to the game's Vulkan layers (Android's
GraphicsEnv functions it uses were not found or behaved differently, e.g. after a Lepton update), so the game's
OpenGL ES shader fixes (zink_shader_fix) don't apply.
suggest: []
- id: zink-device-lost
pattern: 'zink.*DEVICE[_ ]LOST|VK_ERROR_DEVICE_LOST'
severity: fatal
diagnosis: The GPU hung (Zink/Mesa). For GLES Unity games MSAA render-to-texture is the usual cause.
diagnosis: The GPU hung (Zink/Mesa). For GLES Unity games MSAA render-to-texture is the usual cause; at one effect
only, a shader reading undefined loop counters (e.g. Vader Immortal's lightspeed jump - frame.zink_shader_fix).
suggest: [frame.unity_no_msaa, frame.unity_runtime_msaa_off]
- id: gpu-hang
pattern: 'kernel: .*(?i:hangcheck|gpu fault|msm_drm.*(hang|recover)|kgsl.*(hang|fault)|adreno.*(hang|fault))'
severity: fatal
diagnosis: The Frame's GPU hung while the game ran (the kernel reset it; the game froze or closed). Usually one
shader or effect the Frame's driver can't handle (e.g. Vader Immortal's lightspeed jump). FramePort can't fix
that on its own; capture the game's shaders (OpenGL ES games) and send a problem report with diagnostics, so a
shader fix can be added for this game.
suggest: [adapter.zink_shader_dump]
report: true
- id: space-warp-used
pattern: 'xrCreateSwapchain \d+x\d+ format=(97|129) '
unless: 'per-game: hide_space_warp=1'
severity: info
diagnosis: The game uses Meta's space warp (it renders half of its frames plus motion vectors, and the runtime makes
up the rest). On the Frame that can make textures flicker, jump or smear while you move (e.g. Into The Radius 2,
Metro Awakening). Turning it off makes the game render every frame itself.
question: Did textures flicker, jump or smear while you moved?
suggest: [adapter.hide_space_warp]
- id: unity-runtime-msaa
pattern: 'recommended MSAA level is [248]\. Switching to the recommended level'
severity: info
@@ -79,6 +148,12 @@ signatures:
diagnosis: Unity didn't start its Oculus VR device (older Unity checks for Meta's system apps first), so the game
runs as a hidden 2D app.
suggest: [frame.unity_oculus_check]
- id: unity-frame-not-begun
pattern: 'frame loop shim: frame \d+: the last waited frame wasn.t begun'
severity: info
diagnosis: Unity skipped beginning a frame (typically while it activates a scene); FramePort's frame wait didn't
wait for that frame instead of blocking the game. A few at each scene load are normal.
suggest: []
- id: sdl-no-clipboard
pattern: 'ClipboardManager\.addPrimaryClipChangedListener|SDLClipboardHandler\.<init>'
severity: fatal
@@ -102,6 +177,13 @@ signatures:
severity: error
diagnosis: Shaders written for Quest drivers fail on Mesa (GLSL strictness).
suggest: [frame.gl_shim]
- id: gl-multiview-twin-failed
pattern: 'GLMV\s*: twin of program \d+(: stage| failed)'
severity: warning
diagnosis: The multiview interposer (frame.gl_multiview_fbo) couldn't build a single-view copy of one of the
game's shader programs (the line has the compiler's message); draws of that program into flat panels (HUD,
menus) still stay black. Please report it with the log; turning the patch off changes nothing else.
suggest: []
- id: vrapi-unsupported-layer
pattern: 'OVRPortVrApi.*Unsupported VrApi layer type (\d+)'
severity: error
@@ -214,6 +296,18 @@ signatures:
voice chat a few seconds after the logo).
suggest: [frame.ovr_microphone]
supersedes: [native-crash]
- id: slz-vulkan-hook-crash
pattern: 'SLZ Graphics plugin loading![\s\S]{0,20000}?E CRASH : .*pc 0000000000000000'
severity: fatal
diagnosis: Stress Level Zero's graphics plugin (libSLZQuestNative.so) hooks Unity's Vulkan start-up and calls an invalid function there on the Frame (e.g. BONELAB 1.2974); the game crashes or hangs before its first frame.
suggest: [frame.slz_vulkan_hooks]
supersedes: [java-crash, native-crash]
- id: vivox-api31
pattern: 'NoSuchMethodError: No virtual method \w*CommunicationDevice\w*\(.*Landroid/media/AudioManager'
severity: fatal
diagnosis: The game's Vivox voice-chat SDK calls Android 12 audio-routing methods (AudioManager communication devices) without checking the Android version; Lepton runs Android 11, so the app crashes (e.g. Green Hell VR). The patch makes Vivox's audio-route check return early (voice chat keeps the default route).
suggest: [frame.vivox_audio_route]
supersedes: [java-crash]
- id: native-crash
pattern: 'Fatal signal (\d+) \(SIG[A-Z]+\)'
severity: fatal
@@ -238,6 +332,7 @@ signatures:
suggest: []
- id: container-not-started
pattern: "is not a running context|OCI runtime error"
unless: 'Boot complete!' # Lepton 3.x prints the error while its container is still starting, then boots fine
severity: fatal
diagnosis: The Lepton container failed to start (see the error above it in launch.log).
suggest: []
@@ -321,6 +416,19 @@ signatures:
severity: fatal
diagnosis: The game crashed (Unreal's crash reporter started). See the game log and crash summary in the log. Common causes are that no VR runtime reached the game (Revive off) or a GPU or driver problem under Proton.
suggest: [pcvr.revive, pcvr.proton_log, pcvr.no_crash_reporter]
- id: unity-vr-init
kind: pcvr
pattern: '(?i)OpenVR failed initialization|VRInitError_(?!None\b)\w+|Initialization of device \w+ failed|Failed to (load|initialize|start) (VR|XR|OpenVR|Oculus)\b'
severity: error
diagnosis: The game's Unity log shows that a VR device failed to start. Unity games built for both the Oculus and the SteamVR (OpenVR) runtime choose one with an argument; without it they may try Oculus, which the Frame doesn't have, and run without VR or quit. Start the game with -vrmode OpenVR (Game arguments).
suggest: [pcvr.launch_args]
- id: unity-crash
kind: pcvr
pattern: 'Crash!!!|caused an Access Violation|caused an Unhandled Exception'
severity: fatal
diagnosis: The game crashed (Unity's crash handler wrote a report; the Unity log in the launch log shows the stack). Common causes are that no VR runtime reached the game or a GPU or driver problem under Proton.
suggest: [pcvr.proton_log]
supersedes: [wine-crash]
- id: wine-crash
kind: pcvr
pattern: 'Unhandled exception: page fault|wine: Unhandled|Backtrace:'
@@ -333,7 +441,8 @@ signatures:
severity: error
diagnosis: The game can't write to a folder it created in its storage (e.g. SUPERHOT's cloud/data, mode 1700, which
makes it quit at start). FramePort's launcher (agent 41 or newer) gives such folders owner and group write
permission before and during every launch; update the game on the Frame (reinstall) to get the new launcher.
permission before and during every launch (agent 41 or newer; else reinstall the game). A folder the game creates
and checks in the same moment on its very first start can't be repaired in time, so start the game again.
suggest: []
- id: surface-swapchain-emulated
pattern: 'surface_emul: emulated Android surface swapchain'
@@ -347,6 +456,14 @@ signatures:
diagnosis: FrameBridge couldn't emulate an Android video panel (XR_KHR_android_surface_swapchain); the panel stays
empty, and a game waiting for its intro video may stay black (e.g. I Am Monkey's intro).
suggest: []
- id: hw-video-decoder-busy
pattern: 'Iris \S+ unavailable; keeping Android|Iris \S+ initialization failed \(-1[26]\); using software'
severity: info
diagnosis: FramePort's hardware video decoder (frame.hw_video_decode) couldn't open a session, because other
decoder sessions held the hardware (Steam's own hardware video decoding, SteamVR's link while a game starts);
the video was decoded in software instead, which is slow for 4K/8K. Turn off hardware video decoding in Steam's
settings, restart Steam, then start the game again.
suggest: []
milestones: # progress markers, in order; the furthest one reached is reported
- {id: pcvr-launch, kind: pcvr, pattern: 'FramePort: launching|Launched injector with', label: Launcher started}
- {id: pcvr-oculus-hmd, kind: pcvr, pattern: 'FramePort oculushmd: OculusHMDConnected event ready', label: Oculus headset event provided}
+5
View File
@@ -35,6 +35,11 @@
## Extending
- **New fix**: `patches/frame/<name>.py` with a `Patch` subclass (`detect`, `apply`, optional `validate`), plus a
signature in `catalog/triage.yaml` and a PLAYBOOK row. Stages: `overport` | `apk` | `install`.
- **Upstream fixed a bug we work around**: register an `UpstreamFix` (`patches/upstream.py`) in the workaround's
module: a probe that finds the fix in OVRPort's output (True / False / None = can't tell). Each build runs the probes
on the converted APK and leaves out the workarounds whose fix is there (log "not needed: …", `build.superseded`,
left out of settings.conf at install; game page and `frameport show` say so). The recipe keeps asking for the fix,
so builds made with an older runtime keep the workaround. `FRAMEPORT_KEEP_WORKAROUNDS=1` turns it off.
- **New heuristic**: put it in the patch's `detect()` (and `applies()` for visibility), with the evidence in the
reason text; check `scripts/eval_heuristics.py` still reproduces the catalog and add a test in
`tests/test_heuristics.py`.
+2
View File
@@ -38,6 +38,8 @@ games/<pkg>/package/ stand-in for the game files (no content):
games/<pkg>/target/ from the Frame (or the PC): launch.sh, settings.conf, deployment.json, launch.log,
launch-test.log (PC VR), lepton-steamlaunch-<appid>.log, logcat-{main,crash,system,...}.log,
ReviveInjector.txt / steam-<appid>.log / game-log-N.txt (PC VR), files.json (+ missing)
(game-log-N.txt: Unreal Saved/Logs + crash summaries, Unity Player.log / Player-prev.log /
output_log.txt + crash error.log from the Proton prefix, agent v67)
```
Each log keeps at most its last 4 MB (2 MB per file from the Frame).
+130 -1
View File
@@ -28,6 +28,23 @@
work on Android 11: `policy_control` is gone, `cmd statusbar send-disable-flag home recents` has no `back` and
moves the back button onto the app's controls, the `sysui_nav_bar` layout and disabling SystemUI had no effect.
## Where FramePort keeps games (internal storage, microSD)
- Every game has an **anchor** on internal storage: `~/Applications/quest-frame/<pkg>/` with `launch.sh`,
`deployment.json`, `artwork/` and `plays.log`. Steam's shortcut points at the anchor's `launch.sh`, so it never
changes when the files move.
- The game's files (`lepton-app/`, `lepton-data/` = saves, `lepton-shaders/`, `settings.conf`, `launch.log`; PC VR:
`game/`, `revive/`, `compatdata/` = Proton prefix; Linux: `app/`) live in `deployment.json["base"]`: the anchor
itself on internal storage, or `<mount>/FramePort/<pkg>` on another drive (GitHub #90, agent v63).
- SteamOS mounts removable drives (microSD) under `/run/media/<user>/<label or uuid>`; the agent's `drives` reads
`/proc/mounts` (plus the drives of Steam library folders) and refuses vfat/exfat/ntfs (Lepton's data and Proton
prefixes need Unix owners, permissions and symlinks) and read-only mounts. A drive that isn't mounted is an error for
new installs (never a silent fallback to internal storage); `list_installed` marks games on it `drive_missing` and
their launchers stop with "storage not mounted?".
- `move` copies with `cp -a` inside `podman unshare` (files Lepton's containers own belong to subordinate user ids),
compares file count + bytes, retargets absolute symlinks into the old folder (the LibOVRRT → Revive redirect), points
`launch.sh` at the new folder and only then deletes the old copy; on the same filesystem it renames instead.
Moving a game whose Lepton data holds absolute paths, or a PC VR prefix, is not yet verified on the device.
## OpenXR runtime (as seen by games through overport's loader)
- Instance extensions present include KHR_android_create_instance (must be enabled; the adapter adds it),
KHR_vulkan_enable(2), KHR_opengl_es_enable, KHR_composition_layer_depth, FB_display_refresh_rate, EXT_hand_tracking
@@ -59,6 +76,13 @@
`device.foveation`: `fixed` = `FDM_DEBUG=disable_offsets`, `off` = `VK_INSTANCE_LAYERS=""`. Lepton passes `FDM`,
`FDM_DEBUG`, `FOVE_LEVEL` and `FDM_SWAPCHAIN_SIZE` through to the container (liblepton/mounting.sh PASSTHROUGH_VARS);
the layers are chosen on the host, so setting them inside the game does nothing.
- How Lepton loads them: `liblepton/vulkan_layers.sh` mounts the chosen layers (only those in the OS image's
`/usr/share/guestos/android/vendor/vulkan_layers`) into the app's lib dir and writes their names to Android's
`settings global gpu_debug_layers` for `gpu_debug_app` = the game; Android's loader reads that list from GraphicsEnv
at each vkCreateInstance and searches the app's lib dir (`/data/app/…/lib/arm64`). A layer bundled in the APK is
found there too; FramePort's shader-fix layer (`frame.zink_shader_fix`) adds its own name to GraphicsEnv's list from
inside the process (`android::GraphicsEnv::setDebugLayers`, exported by the guest's libgraphicsenv.so, Lepton 3.0.5).
That is the only way to reach the Vulkan side of OpenGL ES games (Zink creates the instance inside Mesa).
- GL ES: Zink (Mesa GL on Vulkan). Strict GLSL (see PLAYBOOK) and occasional `DEVICE LOST` with MSAA render-to-texture.
## Proton / Windows games (surveyed 2026-09-29; running a Rift game under it not yet verified)
@@ -69,6 +93,17 @@
SteamLinuxRuntime_4). None are installed by default.
- `steam -ifrunning steam://install/<appid>` only opens a confirmation dialog in the headset. The agent's unattended
mode writes an appmanifest stub (StateFlags 1026, installdir from appinfo) and restarts Steam, which then downloads it.
- **x86_64 Linux apps (agent v61; verified on the device 2026-10-07: an x86_64 glibc test program installed + launch test RUNNING, "machine=x86_64 glibc=2.41"):**
FEX is Steam app 3127680 (`fex`, installs as `common/FEX-Emu`, ~6 MB, 21 s via the appmanifest-stub path:
`install_proton` / `proton_status` with `kind: linux_x86`). Its toolmanifest commandline is
`/fex-compat-tool %verb% --`, **no** require_tool_appid: it does not use the Steam Linux Runtime. fex-compat-tool
(Python) runs `<FEX-Emu>/usr/bin/FEX` with RootFS `/usr/share/guestos/fex-mesa` (part of the SteamOS image: an
x86 Arch-style root with glibc 2.41, Mesa, graphics_provider.json for x86_64 + i386), emulates x86_64 and i386
(emulator.json), honours `STEAM_FEX_TSOENABLED`, `STEAM_FEX_MULTIBLOCK`, `STEAM_COMPAT_FEX_CONFIG`, sets
`tu_override_uncached_as_cache_coherent=true` and logs to `/tmp/fex-compat-tool-<pid>.log`. It **exits 1 ("No compat
data path?") without `STEAM_COMPAT_DATA_PATH`** (keeps Config.json/AppConfig/Server/Telemetry in `<it>/fex-emu/`):
the Linux launcher exports `<base>/compatdata`. Version seen: FEX-2607-76-g37265b1. ldd can't read x86 programs,
so the missing-library check is skipped for them; an x86_64 AppImage is extracted through the same chain.
- The command Steam runs is built from each tool's `toolmanifest.vdf` (`commandline`, `require_tool_appid`):
`<SLR4-arm64>/_v2-entry-point --verb=waitforexitandrun -- <Proton>/proton waitforexitandrun <exe>`.
- Proton sets up VR (vrclient/wineopenxr registry) only when `SteamGameId` is set (steam_helper `setup_vr_registry`).
@@ -109,6 +144,24 @@
- Processes started from Steam (Konsole, SSH sessions?) share steam.service's cgroup: use `systemd-run --user`.
- SSH: `sshd` must be enabled (`sudo systemctl enable --now sshd`), which needs a user password (`passwd`).
- mDNS: avahi-daemon runs by default; hostname `frame` → `frame.local`.
- Desktop Mode menu entries (GitHub #84, agent v63): Linux apps get `~/.local/share/applications/frameport-<slug>.desktop`
(+ an executable copy in `~/Desktop` when that folder exists; Plasma starts executable `.desktop` files there
without a trust prompt), `Exec=env FRAMEPORT_DESKTOP=1 "<anchor>/launch.sh"`, marked `X-FramePort-Package=<pkg>`.
Plasma's launcher exits right after starting the program, so with `FRAMEPORT_DESKTOP=1` the Linux launcher skips
its "Steam parent gone → end the app" watchdog and keeps the desktop's DISPLAY/WAYLAND_DISPLAY instead of taking
gamescope's from Steam. Not yet tried from the Frame's Desktop Mode.
- Linux apps' own icons (GitHub #99, agent v64): finalize_linux (and every refresh of the menu entries) copies the
app's icon to `<anchor>/artwork/app-icon.{png,svg}`: an AppImage's `squashfs-root/.DirIcon` (usually a symlink),
else the `Icon=` of its top-level `.desktop` file (a folder app: the first `.desktop` within 3 levels) looked up
next to it, in `(usr/)share/icons/hicolor/*/apps/` and `(usr/)share/pixmaps/`; the biggest PNG ≥128 px, else an
SVG, else the biggest PNG. Symlinks are resolved and must stay inside the app folder; absolute and `../` names
are ignored. Precedence for the menu entry's `Icon=` and the Steam shortcut's icon: the user's chosen/store icon
(`artwork/.icon-source` = `custom`, written by the PC with every art upload) > the app's own (Steam: PNG only) >
FramePort's placeholder (`artwork/icon.*`). `StartupWMClass=` is copied from the app's `.desktop` file (Plasma's
task bar matches the window to the entry). A PNG ≤1 MiB goes back to the PC (finalize result `app_icon.png`,
base64) and becomes the library's icon when the game has none and nothing was picked (`.app-icon` marker = its
sha256, no `.picked`); folder apps get it on the PC at add time (`analysis/linux.find_icon`). Not yet seen in
Desktop Mode on the device.
## Video of the headset view (surveyed 2026-10-05; used by the Live view tab)
- `steamvr-v4l2cam.service` (user unit, part of gamescope-session.target, `Restart=always`) runs SteamVR's
@@ -131,7 +184,60 @@
`h264_v4l2m2m` hangs, gst `v4l2h264enc` not-negotiated). ffmpeg + libx264 works: FramePort's live view
(`install/livestream.py`: `fps=30` before the scale, ultrafast/zerolatency, 3 threads, nice 10, fragmented MP4 on
stdout, + AAC 128k) measured 0.34 core at 720p / 0.56 at 1080p with a quiet picture (video only); expect ~1-1.4
cores with a busy scene.
cores with a busy scene. That is now only the fallback.
- Hardware encoding (2026-10-07): the iris encoder works when driven directly through the V4L2 stateful encoder
interface. That is FramePort's `fp_venc` (`native/venc`, spec + every measured value in `native/venc/SPEC.md`).
Facts:
- Device: `/dev/video23` (`/dev/video-enc0` links to it), driver `iris_driver`, M2M multiplanar.
- Formats: input NV12/NV21/AB24(RGBA)/QC24/Q08C; output H264/HEVC; sizes 128..8192.
- NV12 layout (S_FMT answers):
- stride is a multiple of 128;
- the returned height is padded to a multiple of 32;
- CbCr starts at stride × padded height;
- sizeimage is rounded up to 4 KiB;
- the default crop is the requested size.
- Controls: CBR, FORCE_KEY_FRAME, PREPEND_SPSPPS_TO_IDR, HEADER_MODE joined, FRAME_SKIP_MODE, H.264 profiles
Baseline..Constrained High, levels up to 6.0.
- The `steamos` user can open it (group video).
- The panel's current refresh rate can be read without privileges through DRM: `/dev/dri/card0` is mode 0666, and
GETCRTC reports e.g. `2*2160x2160_96` (clock 1402720 kHz / 4448 × 3285 = 96 Hz) even while the headset sleeps.
The panel offers 72/80/90/96/108/120/144 Hz.
- `/dev/video99` (v4l2loopback) has `max_buffers=2`: a reader asking for more gets 2.
- Mid-stream keyframe requests (FORCE_KEY_FRAME) take effect on the next frame, and the GOP restarts from there.
- Measured 2026-10-07 with the headset asleep (still picture):
- `fp_venc` alone at 32/36 fps: 1% of a core at 1080p, 2.5% at 720p.
- Live view end to end: `fp_venc` 2.6% + ffmpeg 8.4% (AAC encoding + muxing).
- Conversion cost per new picture (self-test, NEON, 2026-10-07):
- 1080p (no scaling): 0.6–1.0 ms.
- 720p (exact 3:2 fast path): 1.3–1.65 ms, down from 5.6 ms with the generic box loop (~11 ms at idle clocks).
- 360p (3:1 fast path): 0.7 ms.
- 480p: 2.8 ms; its 852-px width doesn't repeat cleanly, so it uses the generic path.
- At 36 fps with live content that is roughly 2–6% of a core.
- ffmpeg with raw H.264 on a pipe:
- `-framerate` is ignored.
- `-fflags nobuffer` loses the first seconds of tiny frames.
- Numbering frames from 0 (`setts`) with no input timestamps holds all output ~7 s next to pulse's audio.
- Arrival stamps alone (`-use_wallclock_as_timestamps 1`) bunch frames that are read together: gaps of 0–10 ms
and 45+ ms, seen in the headset test as dropped frames.
- What works: `-probesize 32 -analyzeduration 0 -use_wallclock_as_timestamps 1` on the input (ffmpeg reads it in
step with the audio), then `-bsf:v setts=ts=N*(1/fps)/TB` on the output (exactly even frames; audio still
0–N s alongside), plus `frag_keyframe` so that a requested keyframe starts a fragment.
- With sound, ffmpeg paces its inputs against each other and pulse's audio arrives later than the wall clock
(more while something plays). The video input then counts as ahead, and ffmpeg stops reading it for up to
0.7 s, so the writer blocks. That was the cause of 1080p dropping frames: busy 1080p replayed with sound took
34–47 s for 20 s of video. Video alone was fine; `nice`, `-raw_packet_size` and the input queue size didn't
matter. Shifting the video input back fixes it, but too far makes ffmpeg hold the video for interleaving and
release it in clumps: -1 s gave output gaps of up to 550 ms, a longer and stuttering delay in the browser.
-0.25 s is the measured sweet spot (no stalled writes at busy 1080p with sound, steady 50–150 ms output; -0.5 s
already clumps). Output audio/video spans stay equal because setts sets the output times.
- Browser delay (headless Chromium, 720p, 2026-10-07): muted about 0.2–0.35 s behind the newest data; with
sound about 1 s. Chrome keeps about 0.6 s of audio ahead and stalls below that, whatever the player does:
1.1× catch-up gave 8 stalls per 30 s, no speed-up 1–2, same average lag; 50 ms fragments didn't help. So the
player doesn't speed up while sound is on.
- Quality (owner's headset test, 2026-10-07): 3 Mbit/s CBR at 720p36 showed heavy compression artifacts. The
hardware path now uses VBR with a 1.5× peak (the encoder accepts BITRATE_MODE VBR + BITRATE_PEAK) and higher
targets: 360p 1.5, 480p 2.5, 720p 5, 1080p 8, Full 10 Mbit/s. The x264 fallback keeps its rates.
- The default sink is SUSPENDED while nothing plays; its monitor still delivers (silent) audio.
## Text input
@@ -143,3 +249,26 @@
one everywhere (FramePort's "Type on Frame", agent `_keyboard`).
- Unity text fields close without a system keyboard; see `frame.unity_text_input` in PLAYBOOK.md.
## Monitoring sources (probed 2026-10-07, SteamOS 0.4.3, kernel 6.18; used by the Monitor tab, agent `_monitor`)
All readable by the steamos user without root; the agent reads them directly (no programs started per sample).
| Metric | Source | Notes |
|---|---|---|
| CPU | `/proc/stat` deltas; `cpufreq/policy{0,2,5,7}` | 8 cores in 4 clusters: 0-1 / 2-4 / 5-6 / 7 (max 2.27 / 3.15 / 2.96 / 3.05 GHz) |
| GPU busy | Sum of `drm-engine-gpu` (ns) deltas in `/proc/<pid>/fdinfo/<fd>` over every render-node fd (msm DRM) | Gives GPU % per process too. A process can hold several render fds: add them all. Lepton games see the node as `/dev/kgsl-3d0` (bind mount of `/dev/dri/renderD128`). `drm-total-memory` reads 0 |
| GPU clock | `/sys/class/devfreq/3d00000.gpu/{cur,max}_freq` | 231–903 MHz in 12 steps |
| Memory / pressure | `/proc/meminfo`; `/proc/pressure/{cpu,memory,io}` (`some avg10`) | 16 GB RAM, 8 GB swap |
| Temperatures | `/sys/class/thermal/thermal_zone*/{type,temp}` | 48 zones: `cpu*`/`cpuss*`, `gpuss-*`, `ddr`, `nsph*` (NPU), `pm8550*`/`pm8010*` (power ICs), `modem*`, `camera*`, `video`, `max1720x_bat*` |
| Fan | hwmon `slg4ax46073v` `fan1_input` | ~8200 rpm idle |
| Power | hwmon `max34417_*` `power{1-4}_{label,input}` (µW) | `vph` = whole system (~3.6 W idle), `gfx` = GPU, `apc0/1/2` = CPU clusters, `nsp1/2` = NPU; each read is an I2C transfer (~0.7 ms wall), so only these are read |
| Battery | `power_supply/max1720x_bat_7-36` | `current_now` (µA, negative = draining) × `voltage_now` (µV) = watts; `time_to_empty_now`/`time_to_full_now` (s), `cycle_count`, `health`, `temp` (0.1 °C) |
| Game fps | `<base>/launch.log` lines `FrameBridge: pacing: N fps …` (every ~5 s) | Quest games only; SteamVR writes PC VR frame stats only as an end-of-session summary in `vrcompositor.txt` |
| Game container | conmon `-n lepton-steamlaunch-<appid>`; its child's `/proc/<pid>/cgroup` → `cpu.stat`, `memory.current` | The container's ~90 Android processes show as uid 1000 on the host and can be signalled (4XVR, 2026-10-07); Android names them after the package's last 15 characters (`lus4xvrplayerov`), the full name is in `cmdline` |
Cost: a naive sample (fds of ~520 processes scanned) took 43 ms CPU. With kernel threads skipped after their first
sighting, command lines checked once per process, render fds cached (rescanned every 60 s, every 4 s for busy young
game processes without one), temperatures every 2 s, the CPU-cluster power rails every 5 s and the battery gauge every
2-10 s, the stream measured ~10-13 ms CPU per 1 s tick on the device (idle clocks; idle or with 4XVR running) = about
1 % of one core, ~0.15 % of the whole CPU. The I2C sensors (power monitors, battery gauge) are the slowest reads.
+20 -3
View File
@@ -8,6 +8,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
|---|---|---|---|
| 4XVR Video Player | Quest | ✅ Works | |
| Accounting+ | Quest | ✅ Works | |
| AgeOfJoy | Quest | ✅ Works | |
| AllInOneSports | Quest | ✅ Works | |
| Asgard's Wrath 2 | Quest | ✅ Works | |
| BAM | Quest | ✅ Works | |
@@ -20,6 +21,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Beat Saber (co-existence build) | Quest | ✅ Works | |
| Blade & Sorcery: Nomad | Quest | ✅ Works | |
| BodyCombat | Quest | ✅ Works | |
| BONELAB | Quest | ✅ Works | |
| Carve Snowboarding | Quest | ✅ Works | |
| Cook-Out | Quest | ✅ Works | |
| Creed | Quest | ✅ Works | |
@@ -27,6 +29,8 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Dinosaur Island | Quest | ✅ Works | |
| Espire 2 | Quest | ✅ Works | |
| Genotype | Quest | ✅ Works | |
| GORN2 | Quest | ✅ Works | |
| Gravity Lab | Quest | ✅ Works | |
| H.U.N.T | Quest | ✅ Works | |
| I Am Cat | Quest | ✅ Works | |
| I Am Monkey | Quest | ✅ Works | |
@@ -40,6 +44,7 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Lucky's Tale | Quest | ✅ Works | |
| Marvel's Deadpool VR | Quest | ✅ Works | |
| Marvel's Iron Man VR | Quest | ✅ Works | |
| Max Mustard | Quest | ✅ Works | |
| Medieval Dynasty New Settlement | Quest | ✅ Works | |
| Metro Awakening | Quest | ✅ Works | |
| Mobile Suit Gundam: Silver Phantom | Quest | ✅ Works | |
@@ -48,13 +53,20 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| NOPE CHALLENGE | Quest | ✅ Works | |
| palazzo_santacruz | Quest | ✅ Works | |
| Path of the Warrior | Quest | ✅ Works | |
| Pistol Whip | Quest | ✅ Works | |
| Please Don't Touch Anything | Quest | ✅ Works | |
| PowerWash Simulator VR | Quest | ✅ Works | |
| QuestCraft | Quest | ✅ Works | |
| Retronika | Quest | ✅ Works | |
| Richie's Plank Experience | Quest | ✅ Works | |
| Rick and Morty: Virtual Rick-ality | PC VR | ✅ Works | |
| Riven | Quest | ✅ Works | |
| Robo Recall | Quest | ✅ Works | |
| RUINSMAGUS | Quest | ✅ Works | |
| Sniper Elite VR | Quest | ✅ Works | |
| Sniper Elite VR: Winter Warrior | Quest | ✅ Works | |
| Space Pirate Trainer Quest | Quest | ✅ Works | |
| Star Wars Pinball VR | Quest | ✅ Works | |
| Stremio | Quest | ✅ Works | |
| SUPERHOT VR | PC VR | ✅ Works | |
| SUPERHOT VR | Quest | ✅ Works | |
@@ -62,21 +74,27 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| The Boys VR | Quest | ✅ Works | |
| The Climb 2 | Quest | ✅ Works | |
| The Room VR | Quest | ✅ Works | |
| The Tale of Onogoro | Quest | ✅ Works | |
| Time Crisis VR (Experimental) | Quest | ✅ Works | |
| Toy Master | Quest | ✅ Works | |
| Under Cover | Quest | ✅ Works | |
| Vader Immortal: Episode I | Quest | ✅ Works | |
| VR HOT Quest | Quest | ✅ Works | |
| VR4 | Quest | ✅ Works | |
| Walkabout Mini Golf | Quest | ✅ Works | |
| Wallace & Gromit in The Grand Getaway | Quest | ✅ Works | |
| Waltz of the Wizard: Extended Edition | Quest | ✅ Works | |
| Wander | Quest | ✅ Works | |
| Arcsmith | Quest | ⚠️ Works with issues | Right eye distorts during movement (unresolved; swap, tracking, Valve layers, depth and pacing ruled out). |
| Assassin's Creed Nexus | Quest | ⚠️ Works with issues | Some launch warning text is still upside down; the rest of the UI is fixed by flip emulation. |
| BONELAB | Quest | ⚠️ Works with issues | Build 1.2068 plays (head tracking and controls work with frame.unity_user_presence); the newer build 1.2974 crashes at start (Vulkan), no fix yet. |
| Does it Stack? | Quest | ⚠️ Works with issues | Mixed reality mode does not work (works on Demeo), everything else seems to be OK |
| Doom3Quest | Quest | ⚠️ Works with issues | PDA shows black screen. |
| Myst | Quest | ⚠️ Works with issues | Minor graphical glitches on some objects. |
| Phantom: Covert Ops | Quest | ⚠️ Works with issues | DLC/store button crashes (no Meta store). |
| Pinball FX VR | Quest | ⚠️ Works with issues | Plays; mixed reality mode not working yet. |
| Silhouette | Quest | ⚠️ Works with issues | Hand-tracking game; the Frame synthesizes hands from controllers, so it is janky. |
| The Light Brigade | Quest | ⚠️ Works with issues | Judder in weapons |
| Time Stall | Quest | ⚠️ Works with issues | Both eyes distort during movement (unresolved). |
| Vader Immortal: Episode I | Quest | ⚠️ Works with issues | Starts in VR and plays the intro, then stays on the loading card (Vader's portrait with a progress bar). |
| WiiCompiled VR | Quest | ⚠️ Works with issues | To add a game: in FramePort's Files tab, upload your .wcgame file to this game's storage, folder Android/data/org.wiicompiled.quest/files/WiiCompiledOpenXRVR… |
| BlazeRush | Quest | ❌ Doesn't run | Starts and reaches the menu room, but the room shows no controllers and ignores all input (it all reaches the game); no fix yet. |
| Espire 1: VR Operative (Quest Edition) | Quest | ❌ Doesn't run | Mesa GL driver crash during texture upload. |
@@ -84,5 +102,4 @@ Generated from [catalog/games](../catalog/games) by `scripts/compat_list.py`.
| Journey of the Gods | Quest | ❌ Doesn't run | 32-bit only; the Frame has no AArch32. |
| Roblox | Quest | ❌ Doesn't run | Crashes on its first VR frame on the Frame. |
| Shadow Point | Quest | ❌ Doesn't run | 32-bit only; the Frame has no AArch32. |
| Sniper Elite VR | Quest | ❌ Doesn't run | GPU hang (zink: DEVICE LOST) even with MSAA off. |
| Sports Scramble (Santa Cruz) | Quest | ❌ Doesn't run | 32-bit only; the Frame has no AArch32. |
+65 -6
View File
@@ -85,13 +85,50 @@ which would end Desktop Mode). This is new: please report anything odd with **Re
- If the Frame goes to sleep, turns off or leaves the Wi-Fi, the queue **waits** and continues once it's back; uploads
pick up where they stopped. While installs run, FramePort keeps the Frame from going to sleep. Before a large batch
it checks the Frame has enough free space.
- **microSD card / other drives:** the **Steam Frame** page's **Storage** section lists the Frame's drives and sets
where new games go (**Install new games to**). Games go into a `FramePort` folder on the card. To move a game that's
installed already, right-click it → **Move to…** (the game must be closed; saves, settings and the Steam entry stay).
A game on the card only starts while the card is inserted (FramePort then says "SD Card not inserted"). Cards
formatted as FAT, exFAT or NTFS can't hold games: format the card in SteamOS first.
- Your own game files are never changed. The converted copy is temporary: it's removed once the game is on the Frame
(Settings → Installing: keep them, or remove all now).
- **Game settings…** (game menu or the Steam Frame page): sharpness, refresh rate, controllers, menus, 360° video and
mixed-reality options in plain words, only those that matter for the game. Changes are kept with the game and,
when it's installed, used the next time it starts.
- Ordinary Android apps (no VR) are installed unchanged and shown as a flat window in the headset. Android's
back/home/recents buttons are hidden by default (patch **Hide Android's navigation bar**).
back/home/recents buttons are hidden by default (patch **Hide Android's navigation bar**). If FramePort guesses
wrong (a phone app shows nothing in the headset, or a VR app opens as a flat window), choose **VR** or **Flat
window** under **Show as VR or as a flat window** in the game's **Customize** section, then **Update on Frame**.
- **Add games → Add a Windows program (.exe)…** adds a single Windows program; the Frame runs it through Proton (as a
window unless it is a VR game). A program sitting in Downloads, the home folder or a drive root is copied on its
own first, so the install doesn't upload everything next to it.
- In the packaged app you can also **drag files onto the Library**: APKs, Linux apps (AppImage, `.zip`/`.tar.gz`),
Windows programs (`.exe`), folders, or a FrameDrop manifest (`.json`).
### Install links ("Install with FrameDrop" buttons)
Some developers put an **Install with FrameDrop** button on their site (the one-click protocol of the FrameDrop
sideloader, documented at framedropvr.com/docs). FramePort understands the same links:
- **Clicking a button** opens FramePort (or the window that's already open) on Windows and Linux. FramePort shows
the title, the files, their size and whether a checksum is given, and asks before it downloads anything. Then it
downloads the build, adds it to your library and starts the usual install on the Frame. If no Frame is connected,
the game is added now and installs once the Frame is back.
- **Add games → Install from a link…** takes the button's address (right-click → Copy link), a `framedrop://` or
`frameport://` link, a manifest (`.json`) or a direct link to an APK, a Linux build or a Windows program. Use it on
macOS, where web pages can't hand links to FramePort yet.
- **Settings → Install links** has one switch for `framedrop://` links (the buttons) and one for FramePort's own
`frameport://` links. Both are on by default. If FrameDrop is installed too and already opens `framedrop://`
links, FramePort leaves them to it; **Use FramePort for these links** takes them over (turn the switch off to give
them back).
- **For developers:** a FrameDrop manifest works as it is
(`{"schema": "framedrop.install/v1", "name": "…", "files": [{"url": "https://…", "sha256": "…"}]}`). FramePort
also reads an optional `"frameport": {"description": "…", "icon": "https://….png"}` object (FrameDrop ignores
it): the install question then shows the icon and description, and they become the game's icon and "About this
game" text when no store has them. Without a manifest (a bare file link) FramePort guesses the title from the
file name and replaces it with the app's own name once it's downloaded.
- Only `https://` links to public servers are used (plain `http://` only on this PC, for testing); links with a
user name or password, or pointing into your local network, are refused. Only install from sites you trust.
![Game page](images/game.png)
@@ -113,6 +150,19 @@ reality), showing only what matters for that game. Changes are kept with the gam
**Make Unity text fields work** for them. The first time, it downloads Cpp2IL (a tool that finds the right spot in the
game's code, ~17 MB). Games added before this version: open the game's menu → **Analyze again**, then reinstall.
## Watching the Frame (Monitor)
- **Monitor** (its own tab in the sidebar) shows live what the Frame is doing while the tab is open: the running game
with its frame rate (Quest games), CPU, graphics chip, memory, the hottest temperature with the fan speed, power
draw and battery time left, each with a 2-minute chart. **Show details** adds every CPU core, all temperature
sensors, where the power goes and the network.
- **Processes**: **Game** (default) lists the running game's processes, **Steam & SteamVR** and **All** show more.
Right-click a process (or use **⋯**) to end it, force-kill it or end its whole game; **End game** on the game card
closes the game the way Steam's Exit game does. A lock marks programs whose end would close Steam, SteamVR or the
desktop: FramePort asks again before ending those.
- The numbers come every second (or every 2/5 s) and cost the Frame about 1 % of one CPU core; nothing keeps
running on the Frame after you leave the tab.
## Updating
**Game configs** (the tested recipes in the catalog) update by themselves: FramePort checks GitHub every 6 hours for
@@ -156,18 +206,25 @@ starts it if asked). FramePort installs it on the Frame and Proton runs it as a
it shows up in the Frame's Steam library tagged "Windows game on Frame". Whether a game runs depends on Proton on ARM
(x86 games run through emulation).
## Linux apps (arm64)
## Linux apps
The Frame runs SteamOS on an arm64 CPU, so native Linux apps built for **aarch64/arm64** run on it directly (no
Android container, no Proton). **Add games → Add a Linux app (arm64)…** takes an AppImage or a `.zip`/`.tar.gz`/
Android container, no Proton). **Add games → Add a Linux app…** takes an AppImage or a `.zip`/`.tar.gz`/
`.tar.xz` archive; **Add a Linux app folder…** takes an unpacked app. FramePort finds the program that starts it (the
game page's **Change…** picks another one) and whether it's a VR (OpenXR) app. **Install on Frame** uploads it
unchanged and adds it to the Frame's Steam library, tagged "Linux app on Frame"; Play, launch tests and Uninstall
work like for other games.
- x86_64 builds can't run on the Frame: FramePort says so when you add one. Look for an aarch64/arm64 download.
- The app must bring the libraries SteamOS doesn't have. If some are missing, the install reports them and the game
- **x86_64 builds** run through **FEX**, Valve's x86 translator, with the x86 system libraries SteamOS ships for it
(the way Steam on the Frame runs x86 Linux games). The first install of one installs FEX on the Frame (Steam
restarts once and downloads it, a few MB). They run slower than arm64 builds: when an app offers both, FramePort picks the arm64 one. The game
page shows which CPU a build is for.
- The app must bring the libraries SteamOS doesn't have (checked for arm64 builds; x86_64 builds use FEX's x86
system, which has glibc and Mesa, and aren't checked ahead). If some are missing, the install reports them and the game
page lists them: look for a build that includes them.
- **Desktop Mode:** Linux apps also appear in Desktop Mode's application menu and as an icon on its desktop. Some
apps work better there, with a mouse and keyboard, than in Gaming Mode (where Steam Input turns the controllers into
a gamepad). Switch it off per app on the game page (**Desktop Mode**).
- From the command line: `frameport add-linux <AppImage, folder or archive> [--exe <program>]`.
## Files on the Frame (videos, documents, mods, saves)
@@ -207,12 +264,14 @@ and `frameport <command> --help` describe every option. The main ones:
| Command | What it does |
|---|---|
| `scan <folder>` / `list` / `show <game>` | add games, list the library, show a game's analysis and patches |
| `add-linux <path>` | add an arm64 Linux app (AppImage, folder or archive) |
| `add-linux <path>` | add a Linux app, arm64 or x86_64 (AppImage, folder or archive) |
| `recipe <game> --enable/--disable <patch>` | change a game's patches (`patches` lists them all) |
| `build <game>` / `install <game>` / `test <game>` | build, install on the Frame (`--to pc` for PC VR on this PC), launch test |
| `frame discover` / `frame connect` / `frame info` | find, pair with and describe the Frame |
| `frame send` / `frame storage` / `frame cleanup` | copy files to the Frame, show where they go, free space |
| `frame drives` / `frame move <game> --to <drive>` / `install --dest <drive>` | the Frame's drives (microSD), move a game, install to a drive |
| `tools status` / `tools install` | the tools FramePort downloads |
| `open-link "<link>"` | install from an "Install with FrameDrop" button's address or a manifest/APK/zip link (`--yes`, `--no-install`) |
| `diag report <game>` / `share-recipe <game>` | report a problem / share a working recipe |
| `update` | update FramePort |
+56 -2
View File
@@ -13,6 +13,7 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
## Startup failures (visible in launch.log)
| Symptom | Cause | Fix |
|---|---|---|
| A launch test suggests a fix (e.g. triage `sdl-no-clipboard` → `frame.sdl_clipboard`) but the game page doesn't show it / lists it as not applicable; or a game added long ago lacks warnings newer ones get (Android version, web wrapper, missing OBB) (GitHub #104, Dramatic Shape) | the library entry was analysed by an older FramePort: the analysis field the patch reads (`sdl_java`, `min_sdk`, `web_wrapper`, `expects_obb`, …) is missing, so `applies()` is false | entries whose `analysis.extra.analysis_version` is below `detect.ANALYSIS_VERSION` are analysed again at the GUI's start (background) and before a build (`pipeline.refresh_analyses`); by hand: game menu → "Analyze again", then Update on Frame. An unreadable APK is marked `analysis_failed` (retried after the next version bump). Adding an analysis field: bump `ANALYSIS_VERSION` |
| One eye (usually the right) shimmers or jitters in the 3D view while menus and panels look steady; pacing is clean (GitHub #69, e.g. RTCWQuest, Moss) | Valve's eye-tracked foveation layer (`VALVE_fdm_injection`) moves its low-density regions with each eye's gaze, and the right eye's gaze filter restarts often (`CEyePoseUKF R: Large dt … Bootstrapped` in `~/.local/share/Steam/logs/eyetracking.txt`) | Customize → "Eye-tracked foveation (Valve)" (`device.foveation`): **Fixed** (`FDM_DEBUG=disable_offsets`, keeps the GPU saving) or **Off** (`VK_INSTANCE_LAYERS=""`); catalog field `foveation:` |
| A Unity OpenXR game closes right after the OpenXR instance is created: `F VrApiLoader: vrapi_SetPropertyInt was called before vrapi_Initialize()!`, SIGABRT on UnityMain (GitHub #62, Jurassic World Aftermath) | the APK still ships Meta's VrApi loader; OVRPlugin (1.89, on OpenXR) calls it without starting VrApi and Meta's loader aborts. The VrApi bridge can't replace it (39 of the loader's 114 functions: OVRPlugin wouldn't link) | `frame.vrapi_stub` (same exports, each returns 0); triage `vrapi-before-init` |
| A VrApi game exits at once: `UnsatisfiedLinkError … dlopen failed: cannot locate symbol "vrapi_PollEvent"` (GitHub #57, BlazeRush) | the engine imports a VrApi function the VrApi bridge didn't implement | the bridge has `vrapi_PollEvent`, `RecenterPose`, `SetDisplayRefreshRate`, `GetSystemPropertyFloatArray` (rebuild the game); the static check "VrApi functions resolvable" names any other missing one; triage `vrapi-symbol-missing` |
@@ -21,7 +22,11 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| `cannot locate symbol "ovrPeerConnectionState_ToString"` (or another `ovr<Enum>_ToString`) at start (BlazeRush, GitHub #57) | OVRPort's platform loader lacks several Platform SDK enum helpers | `frame.ovrstubs` now stubs them too; `*_ToString` stubs return "" (not NULL) |
| Exits right after the OpenXR instance is created: `Failed to create XR session: -50` (`XR_ERROR_GRAPHICS_REQUIREMENTS_CALL_MISSING`, Lambda1VR) | the game skips `xrGet*GraphicsRequirementsKHR`; Meta's runtime tolerates it, SteamVR's doesn't. Lambda1VR's TBXR doesn't even enable `XR_KHR_opengl_es_enable` (only `XR_EXT_local_floor`) | FrameBridge adds `XR_KHR_opengl_es_enable` when an app enables no graphics API extension, asks for the requirements on the game's behalf and retries xrCreateSession once (logs `added XR_KHR_opengl_es_enable …`, `the app skipped …; asked for it`; every launch logs the final extension list `xrCreateInstance with N extension(s)`); triage `graphics-requirements-missing`. Verified headless: Lambda1VR runs at 72 fps |
| Exits at start: `OVRAvatar-Loader: DisplayErrorAndExit: Failed to launch SystemActivities` after `ovrAvatar_Initialize: Failed to load AvatarSDK driver` (BlazeRush, GitHub #57) | Meta's avatar loader needs Horizon; it then tries Meta's error screen and aborts | `frame.avatar_stub` (same exports, do nothing); triage `avatar-driver-missing`. Games that ship the library but don't start avatars at launch (Lucky's Tale, BattleSisters, Arcsmith) don't need it |
| Frame freezes or the game is killed (OOM) on the first controller vibration (e.g. Lucky's Tale's save slots, BattleSisters, Sniper Elite VR's tutorial grab: memory grows ~1 GB/s until the kill) | OVRPort runtimes up to 3.4.3-23204ea read the haptic envelope's duration (ns) as seconds and allocate gigabytes of samples (GitHub #9, ovrport/app#73); also reached from Unity's legacy `ovrp_SetControllerHaptics`; perf page faults all in `libopenxr_loader.so` `xrApplyHapticFeedback` | `adapter.haptic_fix` (envelope → one plain vibration in `native/xrshim`; PCM vibrations (`XrHapticPcmVibrationFB`) are converted the same way). Fixed in runtime 3.4.3-aa54c3f: builds made with it leave the workaround out by themselves (log "not needed: adapter.haptic_fix", `frameport show` "last build left out"); rebuild an older build to switch |
| Controller vibrations much stronger than on a Quest (The Boys VR, Jurassic World, BONELAB, …) | OVRPlugin games send amplitude envelopes; `haptic_fix` turned each into one vibration at the envelope's *peak* for its whole duration (a short fading pulse became a long full-strength buzz) | `haptic_fix` now uses the envelope's RMS; Game settings → "Vibration strength" (`haptic_scale`, 0-1) scales every vibration in FrameBridge; launch.log shows the first requests (`haptic: …`). OVRPlugin stops a vibration with amplitude 0 (2 s duration): FrameBridge turns that into xrStopHapticFeedback (without it the last buzz ran its full 2 s) |
| Play or a launch test fails at once: `Game files missing at <path> (storage not mounted?)` in launch.log, or "… files are on a drive that isn't inserted" (GitHub #90) | the game was installed or moved to a microSD card (`<mount>/FramePort/<pkg>`) that isn't inserted or mounted now | insert the card; or move the game back (game menu → Move to…, `frameport frame move <pkg> --to internal`) once the card is back. `frameport frame drives` shows what the agent sees (exFAT/NTFS cards are refused: format in SteamOS) |
| A Linux app started from Desktop Mode's menu closes after ~2 s, or opens on the wrong display (GitHub #84) | its launcher predates agent v63: the "Steam parent gone" watchdog ended it when Plasma's launcher exited, and it took gamescope's DISPLAY from Steam | connect FramePort once (agent v63's `upgrade_launchers` adds the `FRAMEPORT_DESKTOP` checks; the menu entry runs `env FRAMEPORT_DESKTOP=1 <anchor>/launch.sh`). Game Mode controller problems in such apps are Steam Input's (a separate issue) |
| A Linux app (AppImage) shows no/FramePort's placeholder icon in Desktop Mode or Steam (GitHub #99) | before agent v64 only FramePort's art set's icon was used, and a lone AppImage has no store art | connect FramePort once (agent v64 copies the app's `.DirIcon` / `.desktop` `Icon=` to `<anchor>/artwork/app-icon.*` and refreshes the menu entry); reinstall to also get it into the library + Steam shortcut. Still the placeholder: check `<base>/app/squashfs-root/.DirIcon` and the `.desktop` file's `Icon=` (an icon outside the app folder or only an XPM isn't used); a chosen icon (`artwork/.icon-source` = `custom`) always wins |
| Every game suddenly fails to start: `crun: create keyring …: Disk quota exceeded`, `is not a running context` | rootless podman leaked a kernel keyring per launch; 200-key quota exhausted | `keyring = false` in `~/.config/containers/containers.conf` (FramePort agent does it), then reboot the Frame once |
| A Unity app's text field shows a caret for a moment and loses focus; no keyboard appears (e.g. Stremio VR login) | Unity's TMP_InputField/InputField wait for Android's on-screen keyboard and close themselves without one (Lepton has none); headless VR apps also have no focused Android window, so no key press reaches them | `frame.unity_text_input` (Cpp2IL finds `TouchScreenKeyboardShouldBeUsed`/`isKeyboardUsingEvents`, rewritten to false/true) + `device.text_input_window` (lepton-show-flatscreen: Steam's keyboard and Type on Frame work) |
| Play gives Steam's "Game configuration unavailable"; Steam's console_log: `GameAction [AppID <id>] … RequestingLicense → UpdatingAppInfo → LaunchApp failed with AppError_9` | this Frame's Steam never loaded FramePort's shortcuts.vdf entry (it treats the id as a store app); cause unknown (GitHub #21/#30) | automatic since agent v43: Play registers the game through Steam's devkit interface ("Devkit Game: …", `devkit_register`) and starts that; uninstall removes it. Agent v58: a devkit entry Steam forgot after a restart is added again live on Play (no NOT_IN_LIBRARY → no Steam restart), and installs/art updates of devkit games only copy art (no Steam restart, GitHub #41) |
@@ -29,6 +34,7 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| Black screen at start while audio/ExoPlayer runs; log `xrEndFrame: dropped N unusable layer(s)`, a quad layer with swapchain 0x0 (e.g. I Am Monkey's intro video) | The game plays video into an Android-surface swapchain (XR_KHR_android_surface_swapchain); the Frame's runtime lists the extension but returns FUNCTION_UNSUPPORTED | adapter `surface_emul` (default on): FrameBridge gives the player a SurfaceTexture-backed Surface and copies each frame into an ordinary swapchain (log `surface_emul: …`) |
| The game quits right after start: `Something failed to initialize. Quitting!` / `We don't have write permission to …/files/cloud/data` (e.g. SUPERHOT) | The game created a save folder with mode 1700; inside Lepton the app writes through the folder's group | launch.sh `fix_perms` adds owner + group write to every folder in the game's storage (agent 41); reinstall to get the new launcher |
| The whole Frame slows to a halt, then the game is killed (`Out of memory: Killed process … anon-rss:10+ GB`), e.g. Lucky's Tale | The game's own memory grows ~1 GB/s on Unity's `Loading.PreloadManager` thread (perf page-fault profile); not the flat window, not the text-input patches | none yet: mark unsupported; profile with `perf record -p <pid> -e page-faults --call-graph fp` (works as steamos, `perf_event_paranoid` 2) |
| A Linux app exits at once: `cannot execute binary file: Exec format error` in launch.log | an x86_64 program started directly (installed before FramePort ran x86 builds through FEX, or by hand) | reinstall it: x86_64 Linux apps install FEX on the Frame and start through it (agent v61, `kind: linux_x86`; FEX also needs `STEAM_COMPAT_DATA_PATH`: "No compat data path?" in launch.log = an agent v60 launcher, reinstall); triage `linux-x86-no-fex`. Prefer an arm64 build when the app offers one |
| `APP_ACTIVITY is empty`, nothing starts | Manifest has category INFO only; Lepton needs LAUNCHER | `frame.launcher` (automatic) |
| PC VR game on the Frame shows as a flat window / Revive: `Unable to load LibOVRRT DLL` / `LoaderInstance::CreateInstance chained CreateInstance call failed` | Frame SteamVR runtime rejects OpenXR apiVersion 1.1 (`XR_ERROR_API_VERSION_UNSUPPORTED`), which Proton 11's VR helper requests | `pcvr.xr_timefix` (Frame OpenXR layer, default on): retries xrCreateInstance as 1.0 |
| Unreal PC VR game on the Frame runs as a flat window although OpenXR works (no `LogHMD` OVRPlugin lines; Unreal logs nothing when it skips the Oculus plugin) / launch.log: `FramePort oculushmd: could not create the OculusHMDConnected event` | UE's OculusHMD (and LibOVR's `ovr_Detect`) only start when the Windows event `OculusHMDConnected` exists and is signalled; on a PC the Oculus service creates it. Revive hooks `OpenEventW` for it, but that relies on Detours patching Wine's (ARM64EC) kernelbase | `pcvr.oculus_unreal` (default for Unreal Rift games; PC VR counterpart of overport's `patch_oculus_unreal`): launch.sh runs the injector through `fp_oculushmd.exe`, which provides the real event until the game exits |
@@ -40,15 +46,21 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| PC VR game: `Failed to initialize Oculus API (-3001)` / `Unable to load LibOVRRT DLL` | Revive's LoadLibrary hook doesn't work under Proton arm64 (ARM64EC kernelbase); the game's LibOVR shim finds no runtime DLL | Symlink `LibOVRRT64_1.dll` → Revive's DLL in the prefix's system32 (manual so far) → then -3021 (runtime signature check; open) |
| Uploads to the Frame are slow (~15 MB/s) | PC and Frame both on Wi-Fi through the home router | Connect the PC to the Frame's own hotspot (or a USB cable): FramePort uses the direct link automatically (~80-100 MB/s); job log line "Transfer link" |
| PC VR game exits right away: `Unhandled Exception: 0xc06d007e` (after `FOnlineSubsystemOculus::InitWithWindowsPlatform`) | Delay-loaded `LibOVRPlatform64_1.dll` (Oculus Platform SDK, installed with the Oculus app) is missing | None on the Frame (entitlement check; FramePort doesn't replace it). PC mode with the Oculus app |
| Unity PC VR game (Oculus + SteamVR build: `OVRPlugin.dll` and `openvr_api.dll` in `<Name>_Data/Plugins`) opens a window and quits after ~20 s; no VR Vulkan instance after the game's DXVK device in launch.log; Unity log (`unity log … Player.log`, agent v67): `OpenVR failed initialization` / `Initialization of device … failed` | Unity's built-in VR tries its SDKs in list order (Oculus first) unless `-vrmode` picks one; a catalog recipe verified with another build of the game (e.g. SUPERHOT VR's OpenXR build, GitHub #105) used to drop the build's own arguments | `pcvr.launch_args` = `-vrmode OpenVR` (detected for such builds; kept when the catalog's `xr` differs from the build's). A game's Electron launcher next to it (SHVR.exe) is ranked below the game |
| Unreal PC VR game opens the crash reporter instead of closing | UE starts CrashReportClient.exe on a crash | `pcvr.no_crash_reporter` (default for Unreal Rift games): `-nocrashreports` + CrashReportClient.exe renamed `.disabled` in the Frame copy |
| A backup's game was added without its data (library `data_dir` null, `data_bytes` 0) although the folder has .obb files | The OBBs sit in a layout the scan didn't know: `<game>/obb/<package>/`, `Android/obb/<package>/`, or SideQuest-style `<game>/apk/x.apk` + `<game>/obb/<package>/` | `sources/quest_dump.find_data_dir` finds a `<package>` folder with .obb files ≤3 levels below the APK's folder or ≤2 below its parent (skipping neighbouring folders with APKs: other games); an `apk`/`apks` folder counts as part of its game folder. Add the folder again (Analyze again doesn't look for data) |
| Game hangs at start / never loads although its .obb files are right there: next to the APK, in an `obb` folder beside an `apk` folder inside a `<package>` folder, or a SideQuest backup (`<timestamp>_<versionCode>.apk`) (GitHub #85, #91) | No folder named after the package, so the folder-layout checks found nothing and no OBB was uploaded | `quest_dump.find_data` then looks for the files by name, `(main\|patch).<versionCode>.<package>.obb` (any case), in the APK's folder, its parent and up to 3 levels below each (folders with their own APKs are other games); prefers the folder and files of the APK's own versionCode, else the newest older, else any version. When that folder also holds other things (the APK itself, other games' OBBs), the library entry gets `data_files` and only those files are uploaded, counted and deleted with the game. Add the folder again |
| Unreal game: `JNI_OnLoad` (OVRPlugin), then silence: no OpenXR instance, no crash, no frames (GitHub #85 TRIANGLE STRATEGY; launch-test finding `missing-obb`) | Installed without its OBB: the manifest says `com.epicgames.ue4.GameActivity.bHasOBBFiles` = true (UE5: `com.epicgames.unreal.…`) but the library has no data folder (`data_dir` null, `data_bytes` 0) | Put the .obb files in a `<package>` (or `obb/`) folder next to the APK, add the folder again, reinstall. Analysis `extra.expects_obb`; the game page, the install question and `frameport scan`/`install` warn; launch tests that sent no frames get the `missing-obb` finding from the library (the game logs nothing). Unity split-binary builds count too (`extra.unity_split`, `analysis/unity_split.py`): an XR-plugin build (Oculus XR Plugin / Unity OpenXR library) without `assets/bin/Data/UnitySubsystems/` in the APK, or BuildSettings listing more scenes than the APK has `levelN` files (globalgamemanagers loose or read from the head of data.unity3d); exact on the 48 Unity APKs of the dumps (13 split, 17 full games; OBBs of full builds are asset bundles or sound banks) |
| `INSTALL_FAILED_NO_MATCHING_ABIS` | 32-bit-only APK; Frame has no AArch32 | None. PC/Rift version via Revive |
| Crash at start: `NoClassDefFoundError: … Landroid/window/OnBackInvokedCallback;` or `NoSuchMethodError: No static method storeStoreFence()V in class Ljava/lang/invoke/VarHandle;` (GitHub #71/#72, triage `android-too-new`) | The APK needs Android 13+ (minSdk 33 or more; those were 34) and calls its classes at start; Lepton's Android is 11 (API 30). OVRPort lowers minSdk to 29, so it installs anyway. minSdk ≤ 32 (Quest's Android 12L) is fine: 16 working catalog games declare 32 | None (analysis `extra.min_sdk` > 32 → a warning on the game page/CLI and in the recipe notes; never unsupported from the manifest alone, the launch test's triage decides). Waits for Valve to update Lepton's Android |
| Nothing opens; `TWALauncherActivity: Using URL from Manifest (https://…)`, `Creating TwaLauncher for com.oculus.browser`, `NameNotFoundException: com.oculus.browser` (GitHub #86 Mahjong Table VR, triage `web-wrapper`) | The APK is a Trusted Web Activity (Bubblewrap / Meta's PWA packaging): a website that opens in Meta's browser, which Lepton doesn't have | None: open the URL in a browser. Analysis `extra.web_wrapper` (manifest meta-data `android.support.customtabs.trusted.DEFAULT_URL`, resolved through resources.arsc, or androidbrowserhelper's LauncherActivity) → a warning with the URL (not unsupported: the manifest alone never decides) |
| `UnsatisfiedLinkError` / `cannot locate symbol "ovr_…"` | overport's platform loader lacks Meta platform functions | `frame.ovrstubs` (automatic, generated stubs) |
| missing `ovrMessageType_ToString` | same, but the game needs a real string | `frame.ovrplatformcompat` (automatic) |
| The game stays in its built-in language or waits at start although its OBB / files hold language packs (`<tag>.lang`, e.g. `de.lang`) | overport's platform loader answers `ovr_LanguagePack_GetCurrent/SetCurrent` with request id 0, so the game never gets the pack's path | `frame.langpacks` (opt-in, experimental, shown for games whose data has `*.lang`; `native/langpack`, build it with `python native/build.py --only langpack`). Default pack: the one the game applies, else env `FRAMEPORT_LANGPACK=<tag>`, else the only pack; search path override `FRAMEPORT_LANGPACK_DIRS`. Unreal games (Deadpool VR) grey out a pack whose asset `Metadata` differs from the game's version string: the patch writes the APK's versionName as Metadata (env `FRAMEPORT_LANGPACK_META` overrides) and reports the pack under the `/storage/emulated/0/Android/obb/<pkg>/` spelling of its path; with `/sdcard/…` the text switched but the dialogue stayed silent (verified in the headset with Deadpool VR) |
| `ClassNotFoundException com.oculus.os.AnalyticsEvent` → abort | Quest telemetry lookup in Meta XR Audio (Unreal build) or native code | `frame.metaxr_telemetry` + `frame.oculusos` (automatic) |
| `JNI DETECTED ERROR`, `GetStringUTFChars … NULL` | CheckJNI is on because overport marks the app debuggable | `frame.nodebug` |
| Unreal game quits a few seconds after start (`System.exit`) | ForceQuit after a failed Quest platform check | alternate build with `patch_remove_unreal_force_quit` (`use_alt`) |
| Unreal game crashes a few seconds after the logo with no backtrace (Unreal's own signal handler hides it: logcat-crash.log empty, Zygote `exited due to signal 11`); the crashing thread is `GameThread`, pc in `libaaudio.so`, lr `libovrplatformloader.so (ovr_Microphone_GetOutputBufferMaxSize+0x10)`; the last platform call is `ovr_Microphone_Create` (e.g. The Walking Dead: Saints & Sinners Ch. 2) | OVRPort's platform loader opens the microphone's AAudio stream only in `ovr_Microphone_Start`, but `GetOutputBufferMaxSize` reads that stream unchecked; Unreal's Oculus voice asks for the size right after `Create` | `frame.ovr_microphone` (NULL check rewritten in place; size 0 until the microphone starts) |
| Unreal game crashes a few seconds after the logo with no backtrace (Unreal's own signal handler hides it: logcat-crash.log empty, Zygote `exited due to signal 11`); the crashing thread is `GameThread`, pc in `libaaudio.so`, lr `libovrplatformloader.so (ovr_Microphone_GetOutputBufferMaxSize+0x10)`; the last platform call is `ovr_Microphone_Create` (e.g. The Walking Dead: Saints & Sinners Ch. 2) | OVRPort's platform loader opens the microphone's AAudio stream only in `ovr_Microphone_Start`, but `GetOutputBufferMaxSize` reads that stream unchecked; Unreal's Oculus voice asks for the size right after `Create` | `frame.ovr_microphone` (NULL check rewritten in place; size 0 until the microphone starts). OVRPort runtime 3.4.3-aa54c3f has the check itself: builds made with it leave the patch out (log "not needed: frame.ovr_microphone", `patches/upstream.py`) |
| `xrCreateSwapchain` -26 / format unsupported (GLES) | Frame takes only sRGB formats, no MSAA | adapter `swapchain_fix` (default on) |
| Frames rejected, "Waiting…" forever | a layer uses a failed swapchain or an extension that isn't enabled (e.g. equirect2) | adapter `layer_fix` (default on) |
| `xrConvert…TimeKHR` FUNCTION_UNSUPPORTED spam; VrApi bridge stalls before recenter | runtime lacks timespec conversion | current adapter emulates it |
@@ -67,29 +79,52 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| Video player / app finds no local videos; MediaProvider "Requested path /home/steamos/... doesn't appear under ..." | Lepton's /sdcard is a symlink to a host path, so Android's media index rejects every file | upload in the Files tab (Videos = /sdcard/Movies) and browse folders in the app |
| Batman: Arkham Shadow closes during smoke-bomb effects; later input-tree corruption | Meta XR Audio Wwise deletes queued metadata still referenced by the current audio frame, then writes through the stale pointer | FrameBridge automatically retires metadata after current-frame references disappear, for the verified AArch64 SDK build only; see [AUDIO_METADATA.md](AUDIO_METADATA.md) |
| Game stays on a loading or "Waiting" box although it reached FOCUSED; log `FrameBridge: xrEndFrame failed -25` then no more `pacing:` lines (e.g. PowerWash Simulator on some Frames, GitHub #39) | The eye image rect ends a few pixels past its swapchain (Unity rounds the swapchain width, the runtime's recommended size differs per Frame); SteamVR rejects every frame with XR_ERROR_SWAPCHAIN_RECT_INVALID | FrameBridge clamps every projection/quad image rect to its swapchain (`rect_clamp`, default on since 0.11.0; log `rect_clamp: view N image rect clamped`): rebuild + reinstall the game |
| Unity game (OVRPlugin, GLES) crashes ~10 s in: `FrameBridge: xrCreateSwapchain … faces=6 … result=-2`, OVRPlugin `CreateSwapchain for eye 0: 0x0, 0 stages`, then render-thread SIGSEGV in memset ← libOVRPlugin ← `ovrp_EndFrame4` (libgallium/ANR follow; e.g. Budget Cuts Ultimate, GitHub #107); triage `cube-swapchain-refused` | The game shows a cube-map layer (OVROverlay cubemap); the Frame's runtime has no cube layers and refuses cube swapchains (XR_ERROR_RUNTIME_FAILURE); OVRPlugin doesn't check and writes into the image list it never got | FrameBridge `cube_standin` (default on): a refused cube swapchain is served as a GL cube map in the game's context and its layers are dropped (log `cube_standin: runtime refused …`): rebuild + reinstall. GLES only; a Vulkan game keeps the error |
| Game (Unity, GLES) freezes, `zink: DEVICE LOST` | multisampled render-to-texture hangs the GPU | `frame.unity_no_msaa`; if it persists: unfixable → PC version |
| Unreal GLES game crashes ~3 s after start: `Fatal signal 11 … fault addr 0x10000 in tid … (RHIThread)`, `#01 … libgallium_dri.so` (GitHub #83 Star Wars Pinball VR, triage `unreal-msrtt-crash`) | Unreal's mobile MSAA renders through multisampled render-to-texture; Zink's `find_rp_state` then indexes `rendering_state_cache[6]` one past its end (sample count ≥ 32) and calls a junk hash function (0x10000). Mesa bug (no bounds check) | `frame.unreal_gl_shim` (GL shim with `gl_hide_msrtt`, multiview kept for Unreal: Unreal 4.25 only enables multiview with GL_OVR_multiview, GL_OVR_multiview2 **and** GL_OVR_multiview_multisampled_render_to_texture, so the shim keeps the last one visible for Unreal and maps glFramebufferTextureMultisampleMultiviewOVR to the single-sampled glFramebufferTextureMultiviewOVR; log `multiview multisampled render-to-texture (N samples) drawn single-sampled`) |
| Crash with `SIGILL (ILL_ILLOPN)` and `*pc=0xd50323bf` (autiasp), e.g. on Unreal's HttpManager thread (GitHub #83 Star Wars Pinball VR, triage `pac-unpaired`) | The engine's OpenSSL ARMv8 assembly has autiasp without a matching paciasp (libUE4.so: 38 paciasp, 40 autiasp; Poly1305 NEON). Quest CPUs treat PAC hints as NOPs, the Frame's CPU checks them | `frame.pac_hints`: in a library with unpaired counts every paciasp/autiasp becomes a NOP (in place) |
| launch.log ends with `logcat: Unexpected EOF!` right after `Waiting for app …`; the game runs, but Steam's "Resume game" menu isn't closed and the launch test shows nothing (Vader Immortal on Lepton 3.0.5, Under Cover on 2.8.14; about 1 launch in 50) | Lepton's logcat mirror died | Agent v67 `_logcat_keeper` (started by launch.sh, added to older launchers by `upgrade_launchers`) reads the container's logcat itself (`podman exec lepton-steamlaunch-<appid> logcat -v threadtime -T 2000`) and appends it to launch.log; log `<base>/logcat-keeper.log` |
| A game no longer starts after a reinstall that was launch-tested right away: logcat `Zip: EOCD not found, …/base.apk is not zip` / `Failed to parse …base.apk`, no `Start proc` for the game (VR HOT, 2026-10-09) | The launch test's 45 s ran from the launcher, but the first start after an APK change spends ~1–2 min booting Lepton and installing the app: stopping the container mid-install left a broken installed copy | Agent v68: the test window counts from Lepton's `Waiting for app` (at most 240 s extra for boot + install). Repair a broken one: `touch <base>/lepton-app/game.apk` (Lepton re-installs it on the next start) |
| Unreal game (OVRPlugin) never starts VR: launch.log has `OVRPlugin: JNI_OnLoad` but never `CompositorOpenXR::PreInitialize`, no OpenXR session, then a crash a few seconds in (e.g. Star Wars Pinball VR, UE 4.25: null pointer in `FSceneRenderer::GetMultiViewSceneColor`, GitHub #83) | Unreal's Oculus module looks up every `ovrp_*` function of the OVRPlugin it was built against (`InitializeOculusPluginWrapper`, all ANDed); OVRPort's OpenXR OVRPlugin lacks a few old ones (UE 4.25 / OVRPlugin 1.44: `ovrp_GetPTWNear`), so the wrapper fails and OculusHMD is never pre-initialised | `frame.unreal_ovrp_entrypoints` (suggested when analysis `unreal_ovrp_lookups` name functions the shipped OVRPlugin lacks): generated `libfp_ovrpstubs.so` (DT_NEEDED of libOVRPlugin.so; dlsym on the plugin's handle searches its dependencies) with a stand-in per missing name returning ovrpFailure (-1000); build note lists the names |
| Only the Android home screen shows; the log has Unity's VR device as `None` (e.g. Accounting+, Unity 2017) | Unity 2017–2018 built-in Oculus support starts VR only when `com.oculus.systemactivities` is installed; Lepton has no Meta system apps, so Unity falls back to non-VR | `frame.unity_oculus_check`: that package name in libunity.so → `android`, plus `native/ovrpshim` (libfp_ovrp.so): Unity 2017's legacy frame loop (`ovrp_Update2`/`ovrp_BeginFrame`) never calls `ovrp_WaitToBeginFrame`, so without the shim no `xrWaitFrame` happens, `CompositorOpenXR::Update … outside of frame bounds` floods the log and the dashboard freezes. With it: log `ovrp frame loop shim: waited for frame N`, ~71 fps. Accounting+ then stops at "press any button" although its OVRInput gets clean input (`INPUT_PROBE` in the patch logs connected controllers/buttons/input focus per call) — unresolved |
| Black flat window, then a GPU hang: kernel `hangcheck detected gpu lockup` (offending task the game), `zink: DEVICE LOST`, Unity render-thread crash; `outside of frame bounds` thousands of times a second (e.g. BattleSisters, Unity 2019.4) | Unity 2019 on its built-in VR (no `libOculusXRPlugin.so`) uses the same legacy frame loop as Unity 2017: no `xrWaitFrame`, the render thread floods the GPU | `frame.unity_oculus_check` adds the ovrpshim frame wait for these too (revision 2) |
| Unity built-in VR game (with `frame.unity_oculus_check`) freezes at a scene switch: the log stops after `Boot: Activating Scene: …`, the process stays alive, every thread sleeps (e.g. Sniper Elite VR, Unity 2019.4) | The shim's `xrWaitFrame` blocks until the previously waited frame is begun; Unity skipped beginning it at the scene switch and its render thread then waited for the main thread (gdb: UnityMain in vrclient `CSxrSession::StartNextFrame` ← `ovrp_WaitToBeginFrame` ← `fpov_Update2`). OVRPlugin also only begins the frame index it waited for | `frame.unity_oculus_check` revision 5: libunity.so's `ovrp_BeginFrame`/`ovrp_EndFrame` go through the shim, which waits ≤ 50 ms for the last waited frame to begin, else skips the wait (log `the last waited frame wasn't begun`), and gives a begin/end of another index the waited one |
| Unity built-in VR game: the hands/weapons trail the controllers by centimetres when moving (headset fine, 72 fps), e.g. Sniper Elite VR | Unity's physics-step `ovrp_Update2` (prediction 0) makes OVRPlugin locate the nodes at the monotonic clock's "now", the XrTime base on a Quest but 0.05–0.9 s behind the Frame's XrTime; the runtime extrapolates backwards and the render-step reads after it get those poses too. A larger prediction is capped by OVRPlugin (~0.07 s) | `frame.unity_oculus_check` revision 5 holds the physics-step update back: both steps read the render update's display-time poses |
| OVRPlugin game: the hands lag behind the controllers (also with `ovrp_hold_physics` on or off; rendering fine), e.g. BattleSisters | OVRPlugin passes its monotonic-clock "now" on as the OpenXR time (OVRPort's dispatcher converts XR_KHR_convert_timespec_time 1:1: the Frame's runtime lacks it, and FrameBridge's emulation is never asked), but the Frame's XrTime runs ahead of CLOCK_MONOTONIC (2.56 s on SteamOS 0.4.5): with `pose_debug=1` the hand spaces show `time - predicted display time -2564 ms`; BattleSisters also asks for its head at XrTime 0.1 s every frame | `adapter.pose_time_fix` (FrameBridge, suggested for Unity built-in OVRPlugin games): a located time nearer the monotonic clock than XrTime's "now" is moved by the clock offset measured at xrWaitFrame, one more than 0.5 s before the display time goes to "now" (log `pose_time_fix: …`; `pose_debug=1` counts both per 5 s). Once a build has this FrameBridge it is toggled in settings.conf (no rebuild) The offset is the largest of the last ~2 s of xrWaitFrame samples (hitches made single samples dip by up to 2.5 s and pushed fixed times into the future); far-past requests get the frame's display time. Headless survey of 54 games (2026-10-09): no head/view request on the monotonic clock in any other game; UE4 OVRPlugin 1.89 games (Robo Recall, Vader, Phantom, Time Stall) and The Room VR ask for head/views at XrTime ≈ 0 every frame, which works today; hands only show in the headset. Suggested for Unity built-in Oculus games only. A time equal (±1 ms) to one of the last 6 predicted display times is never moved, and monotonic times are only told apart when the clocks are more than 4 display periods apart and the time is clearly nearer the monotonic "now" (GitHub #49: on a Frame with XrTime only ~68 ms ahead, a display-time request after a hitch was moved +67.7 ms) |
| Unreal game stuck on a loading image after the intro, frames keep coming (72 fps), no crash (e.g. Vader Immortal) | suspect: a Meta Platform SDK request no message ever answers | diagnostics: `frame.ovr_trace` (opt-in) logs every `ovr_*` call, the answers `ovr_PopMessage` returns and every 10 s the unanswered requests (`fp_ovrtrace` in launch.log). Never fake an entitlement answer |
| Own-engine game crashes in `libVkLayer_fossilize.so` (Fossilize's recording thread, e.g. Roblox) | same as Deadpool VR: uninitialised pointers in what Fossilize records | `frame.vk_sanitize` now also covers own-engine libraries that load `libvulkan.so` by name |
| Crash right after the game gets focus, in `vrclient.so … UpdateActionStateInternal` / `sxr_xrSyncActions` (e.g. Myst) | a race in the Frame runtime's input code on the first sync after focus (not every launch) | adapter `sync_guard` (input syncs one at a time with event polling, paused 250 ms after FOCUSED) |
| View jumps sideways / game pauses or freezes for a moment every few seconds (e.g. Blade & Sorcery, BattleGlide) | the Frame drops focus for ~0.5 s (FOCUSED→VISIBLE→SYNCHRONIZED); games pause or re-align the player (`OnVRPresence … Teleport`) | adapter `focus_hold` |
| SDL / LÖVE app crashes at start: `NullPointerException … ClipboardManager.addPrimaryClipChangedListener` in `SDLClipboardHandler.<init>` (e.g. Dramatic Shape) | Lepton's Android has no clipboard service (`service check clipboard`: not found) | `frame.sdl_clipboard` (that call → nops in classes.dex, in place, `apk/dex.py`); reproduced and verified with LÖVE for Android 11.5 |
| Game crashes at start: `java.lang.NoSuchMethodError: No virtual method getAvailableCommunicationDevices()… in class Landroid/media/AudioManager` at `com.vivox.sdk.AudioChangeListener.checkAudioRouteAndApplyChanges` (e.g. Green Hell VR, GitHub #101); triage `vivox-api31` | Newer Vivox (voice chat) builds call Android 12 audio-routing methods without a version check; Lepton is Android 11. Older Vivox builds (Eleven Table Tennis, BattleSisters) don't have that code | `frame.vivox_audio_route` (suggested by analysis `vivox_api31`): every AudioChangeListener method that calls one of them returns at once (`Dex.return_early`: return-void / `const/4 v0, 0; return v0`, in place); voice chat keeps the default audio route |
| Controllers do nothing (e.g. stuck on a setup screen); log: `xrSuggestInteractionProfileBindings … XR_ERROR_PATH_UNSUPPORTED` | the game only suggests bindings for Meta's newer profiles (Touch Plus / Touch Pro); the Frame's Android runtime knows `oculus/touch_controller`, not those | adapter `profile_remap` (default on): the bindings are suggested again for Touch (log: `controller profile … -> oculus/touch_controller`) |
| Some buttons do nothing or the wrong thing (e.g. a Quest game's X/Y on the Frame's left controller) | the game binds only Quest Touch and SteamVR maps Touch onto the Frame controllers with its own remap (Touch left X/Y land on the d-pad, right X and Y repeat B), or the runtime rejected a profile or an input call | turn on adapter `input_diag` (diagnostics, off by default; `frameport settings <pkg> input_diag=1`) and look for `input_diag:` lines: `bindings: <profile> accepted` or `unsupported: interaction profile <profile> -> <result>` with the rejected paths, `bindings: /user/hand/<hand> uses <profile>`, `unsupported: function <name>`, `unsupported: xrCreateAction -> <result> (<action>)` |
| Unity game hangs or the whole Frame restarts (e.g. in a menu); log: `The current MSAA level is 0, but the recommended MSAA level is 4. Switching to the recommended level.` (e.g. Lucky's Tale) | Meta's OVRManager (`useRecommendedMSAALevel`) turns 4x MSAA on at runtime, past QualitySettings; multisampled render-to-texture on GLES/Zink hangs the GPU | `frame.unity_runtime_msaa_off` (Cpp2IL: `OVRDisplay.get_recommendedMSAALevel` → 0) |
| Unity game: one eye shows only effects or grey, the other is fine (e.g. I Am Cat, Oculus XR Plugin on GLES) | the game's single-pass multiview rendering goes wrong for array slice 1 (the adapter submits both slices correctly) | `frame.unity_multipass` (Cpp2IL: `OculusSettings.GetStereoRenderingMode` → MultiPass); `swap_eyes=1` tells whether the game or the adapter is at fault |
| Setup/intro loops every launch (Espire 2) | save folders created without write permission | launcher repairs permissions every 2 s (built in) |
| VrApi bridge: never enters VR | Frame reaches FOCUSED later than Quest; tracking only when worn | bridge waits 30 s; test in the headset |
| Unreal game (ILMxLAB, e.g. Vader Immortal) plays its intro, then stays on its loading card (portrait + progress bar) at 72 fps and ignores input (GitHub #49) | the menu waits for `UVRUtils::GetQuestShaderPrecompilePercent()` to reach 100 %; the precompile only starts when `IsRunningOnSantaCruz()` (a Quest), the other branch returns 0.0 | `frame.unreal_quest_precompile` (non-Quest branch returns 1.0; found by Klownicle); suggested when the engine exports that function (analysis `unreal_quest_gates`) |
| Video in a game or player stutters, falls behind or drops to a few fps (4K/8K, e.g. a video player's 360° video, Batman's cutscenes); `OMX.google.*` / `c2.android.*` decoders in the log | Lepton's Android has only software decoders | `frame.hw_video_decode` (suggested when the APK uses MediaCodec/ExoPlayer/VLC; no APK change, reinstall): `OMX.frameport.{avc,hevc,vp9}.decoder` on the Frame's Iris hardware (H.264 High, HEVC Main, VP9 profile 0 up to 4096x2304; 8-bit; up to 8192 px), mounted into this game's container only (log `Iris hardware video/… decoder active`; launch.log `FramePort video: loading the Iris hardware codec plugin`). `Iris … unavailable` / `initialization failed (-12)` (triage `hw-video-decoder-busy`): another decoder holds the hardware, usually Steam's own hardware video decoding: turn it off in Steam, restart Steam. Video broken with it: Settings → Installing → Hardware video decoding off (all games) or `FRAMEPORT_NO_HW_VIDEO=1 %command%` (one game) |
## Picture problems (headset on)
| Symptom | Cause | Fix |
|---|---|---|
| Found after playing: the game page's "Last session" (or `frameport session <pkg>`) lists a finding (agent v70 `session_log`: the newest `plays.log` session's launch.log, ≤4 MB, + its crash logcat + kernel GPU lines) | launch tests never reach FOCUSED or gameplay; the same `catalog/triage.yaml` signatures (sync_guard, pac_hints, unreal_gl_shim, vk_spec_fixes, …) now also run on real sessions, plus the frame-rate and focus-dip findings below | "Try this fix" (FrameBridge settings: written on the Frame at once via `set_settings`, used from the next start) / "Rebuild with this fix" (APK patches); launch-test sessions (`test <unix>` in plays.log) are skipped |
| Textures flicker, jump or smear while moving in an OVRPlugin game that renders space-warp swapchains (`xrCreateSwapchain 376x376 format=97` / `format=129`; triage `space-warp-used`, info + question) | Application SpaceWarp: half the frames plus motion vectors; the Frame's reprojection of them misbehaves (Into The Radius 2, Metro Awakening) | adapter `hide_space_warp=1`; offered as "Did textures flicker…?" after a play session, never applied on its own |
| The game froze or closed and the kernel log has `hangcheck` / GPU fault / `msm_drm … recover` lines during the session (triage `gpu-hang`, play sessions only) | a shader or effect hangs the Frame's GPU (e.g. Vader Immortal's lightspeed jump) | capture the shaders (`adapter.zink_shader_dump=1`, GLES) and send a problem report; the maintainer adds a `zink_shader_fix` / `vk_shader_fix` catalog entry. `zink-device-lost` keeps its MSAA suggestions |
| The game keeps pausing or recentring for a moment while worn (session finding `focus-dips`: ≥3 `focus: back after N ms` lines longer than the game's focus_hold_ms, up to 5 s) | the Frame's wear sensor flickers "HMD off"; FrameBridge logs every runtime focus loss/return (always, before focus_hold) | `adapter.focus_hold_ms=<longest dip + 250 ms, in 500 ms steps, ≤5000>` (or `adapter.focus_hold=1` when off) |
| Judder or smearing on head turns; FrameBridge pacing below the display rate (session finding `slow-frames`: >30 % of the 5 s windows under 90 % of the nearest refresh rate ≥ the session's 90th-percentile fps, after the first two windows, ≥1 min) | the game renders below the refresh rate; the runtime reprojects | `adapter.scale=<current × 0.85>` first, then `frame.unity_runtime_msaa_off` (Unity) / `adapter.hide_space_warp` (Unreal 5 + OVRPlugin) |
| Unreal game (ILMxLAB, e.g. Vader Immortal): controllers track but grip/trigger do nothing, grabbing never works (Unreal's own input values move, the game's stay 0) | `URPOCKeyMapManagerComponent` picks the Quest or the Gear VR key set by `IsRunningOnSantaCruz()`; on the Frame it takes Gear VR, empty in a Quest build | `frame.unreal_quest_keymap` (the Oculus case keeps the Quest set, for action and axis mappings; found by Klownicle, GitHub #49) |
| The hands' thumbs never move when you touch the thumbstick or buttons (Unreal `ThumbUp` stays 1, e.g. Vader Immortal) | OVRPlugin reports thumb proximity from XR_FB_touch_controller_proximity, which the Frame's runtime lacks; OVRPort's loader offers it anyway and the proximity actions never get a binding | adapter `proximity_emul=1` (thumb proximity from the touch inputs: thumbstick, face buttons, thumb rest; 2 = also the index finger from trigger touch); log `proximity_emul: finger proximity bound to touch`. **For UE4 Oculus games this didn't animate the thumbs** (Vader, GitHub #49): use `frame.unreal_thumb_touch` (found by Klownicle): `OculusInput::FOculusInput::SendControllerEvents` computes ThumbUp from NearTouches (masks 0x2/0x8); the patch makes it read Touches (the slot 4 bytes before) with masks 0x0f00 (X/Y/stick/thumb rest) / 0x000f (A/B/stick/thumb rest), keeping the inversion; three instructions, matched exactly (build note `OculusInput ThumbUp: from the capacitive touches`). Matches Vader Immortal Ep. I, Robo Recall, Phantom: Covert Ops, Time Stall, Star Wars: Tales from the Galaxy's Edge; newer UE4 builds (e.g. Asgard's Wrath 2, RE4, In Death) compile it differently (shared mask registers) and aren't matched |
| OpenGL ES game: the GPU hangs at one effect (freeze/exit, compositor watchdog, e.g. Vader Immortal's lightspeed jump) | a shader (as Zink compiles it) reads loop counters/accumulators before setting them | capture the SPIR-V (adapter `zink_shader_dump=1` writes every module to `files/fp_spirv/`, or a GPU capture), then a `zink_shader_fix` entry (same format as `vk_shader_fix`) + `frame.zink_shader_fix` (Vulkan layer under Zink, activated through GraphicsEnv's debug layer list); log `shader fix layer: fixed shader module N`; triage `zink-shader-fix-mismatch` when a Frame update changed the driver's output |
| 2D Android app: Steam says it's running but nothing shows | Lepton runs apps headless unless the app folder has `lepton-show-flatscreen` | FramePort adds it for apps without VR (agent v28+); reinstall apps installed before |
| A phone/tablet app shows nothing in the headset, or a VR app opens as a flat window | FramePort's VR/2D guess (`vr_kind`) is wrong for this app | patch `device.display_mode` → **Flat window** or **VR** (Customize → "Show as VR or as a flat window"), then Update on Frame |
| "Install with FrameDrop" button opens FrameDrop (or nothing) instead of FramePort | another program owns `framedrop://`, or the switch is off, or macOS (Flet can't receive links there) | Settings → Install links → **Use FramePort for these links**; else Add games → Install from a link… (paste the button's address) |
| 2D Android app: back/home/recents buttons cover the app's own buttons | Android's navigation bar | patch `device.hide_navbar` (on by default for apps without VR) sets `qemu.hw.mainkeys=1`; reinstall to apply |
| Black screen, audio works, GLES engine with direct VrApi | Mesa rejects Quest-style GLSL | `frame.gl_shim` (logs `GLShim: SHADER COMPILE FAILED` + source lines) |
| Black screen, audio works, log `Unsupported VrApi layer type N` | bridge drops frames containing that layer | extend the bridge (cylinder=3 is converted to a quad already) |
| Only some draws visible (e.g. controllers) + `glGetError 0x502` | multiview shaders used on single-view FBOs | GL shim `gl_hide_multiview=1` (default) |
| Own GLES engine (OpenXR, not VrApi): eye view fine, but HUD / menu / PDA panels black (e.g. Doom3Quest, GitHub #77; no error in the log) | every shader declares `layout(num_views=2) in;`, also the ones drawn into the engine's single-layer offscreen framebuffers (2D textures/renderbuffers); OVR_multiview makes such draws INVALID_OPERATION and Mesa drops them (`draw_validate.c`), Quest's driver doesn't | `frame.gl_multiview_fbo` (opt-in, experimental: `libfpglmv.so` interposes GLES and draws those cases with a single-view twin of the program; logs `GLMV: twin built: program N -> M`, per-5-s `single-view draws` counters; `gl_mv_debug=1` adds glGetError checks; triage `gl-multiview-twin-failed`). Alternative: the game's own fix (two-layer multiview pool, Xandrix's patch in #77) |
| Upside-down image in a GL bridge game | GL images start at the bottom row | fixed in the bridge (swap angleUp/Down for GL chains) |
| UI panels upside down (AC Nexus) | XrCompositionLayerImageLayoutFB VERTICAL_FLIP unsupported | adapter `flip_emul=1` (default); rotating quads makes them vanish |
| Passthrough black (BAM) | XR_FB_passthrough missing | adapter `passthrough_emul=1` (default) + `patch_force_passthrough` for MR-only games |
@@ -102,16 +137,27 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
| View vibrates even when holding still after `frame.unity_multipass` (I Am Cat) | poses/times are consistent, but two passes on Zink make every frame one period late, and the game clamps its physics step to 10 ms | not fixed (scale 0.8 didn't help); next ideas: fix multiview instead, or the game's fixed timestep |
| Game freezes / pauses for 0.5-2 s now and then, no flicker (e.g. Blade & Sorcery, Lucky's Tale) | the headset's wear sensor flickers off while worn (`logs/eyetracking.txt` `HMD off, stopping eye tracking` … `HMD on`), the Frame turns it into a focus loss and Quest games that pause on focus loss pause | adapter `focus_hold` (on by default since 2026-10-05, dips up to `focus_hold_ms` = 5000 ms are hidden; longer ones still pause) |
| Minor glitches on some objects (Myst, Unreal Vulkan) | unknown; not application space warp (turning it off changed nothing and ran worse) | unresolved |
| Graphics crash in vkCreateInstance with pc == fault address, from the game's own Vulkan hooks (BONELAB 1.2974 `libSLZQuestNative`) | a pointer into memory freed with OVRPlugin's pre-init OpenXR instance; not the runtime (vrclient.so) or Fossilize being unloaded (keeping both loaded with RTLD_NODELETE didn't help) | unresolved |
| Unity game crashes or hangs right as Vulkan starts, after `SLZ Graphics plugin loading!` and OVRPlugin's pre-init `xrDestroyInstance`: SIGSEGV with pc 0 (or pc == fault address in an unloaded library), x16/x23 in `libSLZQuestNative.so` (BONELAB 1.2974) | Stress Level Zero's graphics plugin hooks Unity's Vulkan start-up (IUnityGraphicsVulkanV2 interception: vkCreateInstance/vkCreateDevice) and vkCreateSampler; its vkCreateInstance wrapper calls an invalid pointer on the Frame | `frame.slz_vulkan_hooks` (its two registrations become no-ops; Unity starts Vulkan itself; ~35 s of shader prewarming without its pipeline cache) |
| Unity game shows a picture but the player body is frozen: no head tracking, the controllers stay on the model, no buttons, although OpenXR input works (BONELAB 1.2068, Oculus XR Plugin) | OVRPlugin reports the headset as not worn (`ovrp_GetUserPresent2` → 0) a few seconds after start, and the game's rig only follows a present user (Marrow `XRHMD.IsUserPresent` = Unity's HMD `UserPresence`) | `frame.unity_user_presence` (the Oculus XR Plugin's lookup → `libfp_ovrp.so`, reports present; log `user presence: OVRPlugin 0 …`) |
| Unreal 5 game crashes at its first render pass, backtrace `vulkan.freedreno.so` ← `libVkLayer_fossilize.so` ← `libfp_vk.so (create_render_pass2)` (Metro Awakening) | a subpass without a depth attachment names a depth/stencil resolve attachment; the Frame's Turnip reads the missing depth attachment (NULL) | the Vulkan shim drops that resolve (always, FramePort after 0.11.0; log `dropped the depth resolve of subpass N (no depth attachment)`); triage `unreal-vulkan-driver-crash` |
| Unreal 5 game crashes in `vulkan.freedreno.so` called straight from `libUnreal.so` right after its swapchains, or a few seconds into the first frames (Metro Awakening) | Unreal turns on fragment-density-map foveation because Turnip offers VK_EXT_fragment_density_map, then records barriers (to `FRAGMENT_DENSITY_MAP_OPTIMAL`) and image views (2D array R8G8) for a density map the Frame never provides (VK_NULL_HANDLE); Turnip dereferences it | adapter `vk_hide_fdm=1` (the game doesn't see the extension) + `vk_spec_fixes=1` (leaves out null-image barriers, refuses null-image views); triage `unreal-fdm-missing`. Found by disassembling the driver at the crash pc: the struct offsets (0x58 = VkSubpassDescription2, 0x48/0x60 = image barriers, viewType/format in registers) name the call |
| Own-engine game crashes in `je_free` from its own library on the first frames (Roblox) | the game frees memory it doesn't own; not the Vulkan shim or the format fallback | unsupported |
## Live view (Live view tab, `install/livestream.py`)
| Symptom | Cause | Fix |
|---|---|---|
| Status line says "software encoder" (30 fps, the Frame's CPU busier) | `fp_venc --probe` failed (missing upload, encoder busy or refused a format: exit 4) or the helper couldn't be put on the Frame | on the Frame: `~/.local/share/frameport/bin/fp_venc --probe; echo $?` and `--selftest`; the app log names upload failures. `FP_VENC_DISABLE=1` forces the fallback for tests |
| Picture drifts out of sync with the sound | the hardware path relies on one H.264 frame per fps slot (ffmpeg `-framerate`); `fp_venc: stats … skipped=` > 0 on stderr means slots were skipped | see the stats line (every 10 s); a stopped/suspended Frame skips slots on purpose |
| Viewer joins only after several seconds | the keyframe request (`k` on the channel) didn't reach the encoder; the viewer then waits for the next regular keyframe (4 s) | check `live: encoder=hardware` came first (requests are sent only then) |
## Debugging techniques that worked
- A user's problem report: `frameport diag inspect <FramePort-diag-*.zip>` re-triages its launch log with the current
signatures; the zip has the recipe, analysis, ELF/PE imports, Frame logs and versions (docs/DIAGNOSTICS.md).
- Read `<base>/launch.log` (logcat mirror). Filter the game's pid: `Start proc <pid>:<package>`.
- **Monitor tab** (GUI, while a game runs): fps from FrameBridge's pacing lines next to GPU busy %, the hottest
sensor, memory pressure and per-process CPU/GPU/memory. A memory runaway (Lucky's Tale, Vader Immortal) shows as
rising memory + "waiting" before the OOM kill; a GPU-bound game as GPU near 100 % with fps under target; heat as the
temperature tile turning amber/red. Sources and costs: FRAME_RUNTIME.md "Monitoring sources".
- The adapter logs as `FrameBridge` (settings, xrCreateInstance result, swapchain retries, pacing fps).
- For GLES/GLAD engines, wrap `eglGetProcAddress` to see shader compile errors (build the shim with `-DGLSHIM_TRACE`
for per-FBO draw counts and draw-call errors; the bridge has `-DOVP_GL_DIAG` for eye-image readback).
@@ -127,6 +173,14 @@ version is `catalog/triage.yaml` (used by `frameport test` / the Job screen); ke
touching new memory (`perf script -F comm,tid,ip,dso`); `-t <tid> -F 499` samples one thread. Map IL2CPP
addresses with a full Cpp2IL diffable-cs dump (method `Offset` = address − the r-xp mapping of libil2cpp.so).
Sample memory per mapping name from `/proc/<pid>/smaps` (`[anon:libc_malloc]` = native heap).
- Hangs (every thread asleep): backtraces of all threads with gdb. Root inside the Lepton container may ptrace the
game (the host's `ptrace_scope` 1 doesn't apply there) and the guest has `gdbserver64`; `debuggerd -b` fails (no
tombstoned). Copy every mapped library to a local sysroot (`podman exec <container> cat /proc/<pid>/maps`, then
`tar -chf - -T <list>` through `podman exec -i`; the app's own libs from the installed `lepton-app/game.apk` at their
`/data/app/…/lib/arm64` path), then on the PC: `gdb -batch -ex "set sysroot <dir>" -ex "target remote | ssh
steamos@<frame> podman exec -i lepton-steamlaunch-<appid> gdbserver64 - --attach <pid in the container>" -ex
"thread apply all bt 25" -ex detach` (`set sysroot target:` makes gdbserver assert in linux_mntns_readlink).
`/proc/<pid>/task/*/syscall` (read inside the container) shows each thread's syscall and pc without a debugger.
- Unity catches native crashes itself: no debuggerd backtrace in Lepton's logcat-crash.log, but a
`tombstone_00` in the game's `Android/data/<pkg>/files/` with the memory map (map pc/lr/registers to libraries).
- Focus losses: the game sees FOCUSED→VISIBLE→SYNCHRONIZED (OVRPlugin `[XR_SESSION]`, FrameBridge `focus_hold:`
+140
View File
@@ -0,0 +1,140 @@
# Batman cutscene playback on Steam Frame
Batman: Arkham Shadow (`com.camouflaj.manta`) submits its prerecorded cutscenes
through Android video surfaces and stereoscopic 180-degree OpenXR equirectangular
layers. Lepton/SteamVR's missing surface/layer support leaves those layers black,
while the separately played audio continues. Uploading software-decoded panorama
pixels also does not provide practical playback of the original 8192x4096 HEVC
assets.
## Scope and installation
FrameBridge enables the native-video path through the recipe-only adapter
setting `surface_native` (Batman's catalog recipe; a one-time library migration,
`batman_video_patches`, adds it and `frame.hw_video_decode` to existing Batman
recipes). Hardware decoding is deployed independently into FramePort's shared,
versioned codec store, without embedding decoder assets; games that have
`frame.hw_video_decode` in their recipe get the launcher line that places the
shared wrapper on the launcher's child PATH. Matching containers
receive read-only plugin/XML mounts and the Iris decoder device. Shared Lepton
files and original MP4/OBB assets are
never replaced, transcoded, resized, or rewritten.
The native projection, view recording and Vulkan-enable hooks are gated by
`surface_native`. With it off, Vulkan function lookup and directly exported
entry points pass through to the original loader; default `surface_emul`
alone does not enable the new Vulkan projection path. The fixture
`tests/fixtures/src/surface_hooks_test.c` checks function identity with the
setting off/on and direct-export forwarding with it off. Compile it with
`FRAMEPORT_FAKE_RUNTIME` as `libopenxr_loader_original.so`, then without that
define as the client; run each setting in a separate process next to the
production adapter and fake loader.
The wrapper verifies the tested Android 11 SoftOMX ABI. An existing runtime
hardware plugin takes precedence; incompatible runtimes keep their stock codecs.
Hardware decoder capacity is probed before the private component is advertised.
If the driver's session limit is exhausted, Android can fall back to software
decoding; that preserves functionality but not full-resolution performance.
FramePort does not change Steam's global hardware-decoding settings.
Other games don't get `surface_native`. The decoder is separate from that
setting, and runs in Lepton's arm64 media service. Rebuilding an older APK
removes the previously bundled codec files; launcher migration replaces
Batman's old per-game wrapper with the shared one. MP4 presence alone is not
evidence of compatible surface/overlay semantics.
The recipe change marks Batman's installed build "Update on Frame"; the shared
adapter revision is unchanged, so other games stay installed as they are.
## Decode and render path
`native/hevc/frameport_hevc.cpp` exposes `OMX.frameport.hevc.decoder` through the
tested SoftOMX ABI, using FFmpeg's LGPL Iris V4L2 hardware decoder wrapper.
Native surface clients receive full-resolution YUV hardware buffers with fences.
Compatible large native surfaces use fenced Vulkan transfers between imported
decoder and Android DMA buffers, retaining the decoded frame until GPU completion.
Other surfaces retain parallel NV12 row copies without CPU color conversion.
Input timestamps, EOS, dynamic dimensions, seek and flush
are preserved.
An AImageReader retains the decoder's actual image. The surface worker samples
that original YUV buffer and fuses conversion with visible-view projection on
the GPU. Vulkan imports the finished shared view with explicit foreign ownership
transfers. There is no full-panorama RGB intermediate or CPU picture readback.
The default native output is 2560x2560 per eye with a 12% tangent-space guard;
an explicit `equirect_res` takes precedence.
Every finished view carries the camera that actually generated its pixels.
Submitted game projection views are preferred over another tracking query.
The worker coalesces video/view arrivals and reuses held pixels only while the
visible frustum remains covered. Color/source changes redraw content without
unnecessarily changing the covered panorama-local sampling camera. Infinite
panoramas retain their layer rotation and have no translational parallax.
## Stable composition
A separately submitted projected video layer can alternate with the main game
projection as opaque coverage changes during head movement. That presentation
path exhibited snap-back/doubled video in headset tests even when rendering
and submitted camera metadata agreed.
The native path instead snapshots the game's successfully waited stereo color
image **before forwarding its release**, restoring its color-attachment layout
after the copy. Acquisition indices are tracked in FIFO order. The application
image is never sampled or modified after its OpenXR release, and release is not
deferred. Snapshot generations must match the last successful release.
The final GPU pass reprojects the cached video into the current game camera,
blends it with the scene in linear light, and draws directly into a runtime sRGB
attachment. One primary projection contains both scene and video, including
fades and uncovered hemisphere edges. Later subtitles/menu layers keep their
order. Once this output is ready, movie updates copy only to its private cache,
avoiding the additional standalone-output copy.
Batman attaches color-scale/bias and image-layout structures even when their
transforms are identity. The compositor applies RGB/alpha scale and bias using
OpenXR's straight-color transform and restored premultiplication, and respects
vertical orientation. Unknown scene/view chains, unsupported layout flags, and
intervening layers keep the ordinary fallback; settings are not silently lost.
The push constants fit Vulkan's guaranteed 128-byte minimum.
Each output image has separate commands, descriptors and fences. Positive wait
timeouts retain acquisition for retry. Missing snapshots retain the previous
combined output with the exact camera that produced it; old pixels are never
relabelled with a newer pose. Teardown retires snapshot writers and compositor
readers. Depth/motion-vector images from another render are not attached.
## Building and validation
Rebuild FrameBridge with Android NDK r27c:
```sh
python native/build.py --only adapter --ndk /path/to/android-ndk-r27c
```
The hardware plugin builder and its pinned source/runtime requirements are in
`native/hevc/README.md`. Shader sources and their generated SPIR-V are included.
Compile them for Vulkan 1.0 and validate with `spirv-val` when changing them.
The host tests cover package-scoped settings/assets/freshness, existing-wrapper
updates, codec extraction checksums, runtime/container isolation, and preservation
of original video assets. Android fixtures in `tests/fixtures/src/` exercise the
production Vulkan shaders, hardware worker, camera/coverage helpers and YUV copies.
The original-8K worker test requires a lawfully supplied cutscene file and the
private codec in an isolated Android container; no game assets are distributed.
Validation on Steam Frame included stereo/fade/transparent/uncovered pixels,
asymmetric vertical orientation, color/alpha transforms, unsupported-chain
fallback, timeout retry, held camera metadata, reader teardown, hardware playback,
pause/resume and seek. One isolated run presented 594/600 and 118/120 distinct
scheduled original pictures, none early; 120 moving-head composition frames
averaged 1.16 ms for submission plus GPU completion. These are isolated fixture
results, not a guarantee of full-game frame rate.
The final build was also tested in Batman in-headset: cutscenes displayed and
head-movement snap-back was reported resolved. Logs confirmed the combined path,
full-resolution hardware decoding, over 12,500 scene frames with only two startup
holds, and no subsequent old promotion messages or composition setup failures.
Some frame-rate dips and existing Unity render-pass warnings remain. The patch
does not claim universal game compatibility, exact Quest 3 pixel equivalence,
or perfect frame timing.
-61
View File
@@ -1,61 +0,0 @@
# Themes
FramePort comes with three colour themes, all dark:
| Theme | Look |
|---|---|
| **Portal** (default) | The logo's two portals: orange for actions and the current selection, blue for secondary actions, progress and the PC side. |
| **Portal (OLED)** | Portal on true black, for OLED screens. |
| **Original** | FramePort's first look, with a violet accent. |
Switch in **Settings → Appearance**. The change applies at once.
## Your own theme
A theme is a small JSON file. To make one:
1. In **Settings → Appearance**, pick the theme closest to what you want and click **Copy this theme as a file**.
2. Paste it into a text editor and save it as `something.json`.
3. Change the name and the colours you want different. You can delete every colour you keep: missing colours come
from the `base` theme.
4. Back in **Settings → Appearance**, click **Install theme file…** and choose the file. FramePort checks it,
switches to it and keeps a copy in its data folder (`themes/`), so you can delete the original.
To remove an installed theme, click the bin icon on its card.
### Format
```json
{
"name": "Ember & Ice",
"base": "portal",
"dual": true,
"colors": {
"ACCENT": "#FF5A36",
"SECONDARY": "#5CE1E6"
}
}
```
| Field | Meaning |
|---|---|
| `name` | Shown on the theme's card (up to 40 characters). |
| `base` | The built-in theme that fills in every colour the file leaves out: `portal` (default), `portal_oled` or `original`. |
| `dual` | `true`: two-colour touches like Portal's (blue-to-orange sidebar edge, two-colour "FramePort", the selected tab's fade, blue secondary buttons). `false`: one accent, like Original. Default: the base's. |
| `colors` | Any of the colours below, as `#RRGGBB` or `#RGB`. Names may be upper or lower case. |
| Colour | Used for |
|---|---|
| `BG` | Window background |
| `SIDEBAR` | Sidebar and activity panel |
| `SURFACE`, `SURFACE_2`, `SURFACE_3` | Cards; raised and hovered parts; inputs and chips |
| `BORDER`, `BORDER_STRONG` | Outlines |
| `TEXT`, `TEXT_2`, `TEXT_3` | Main text; secondary text; captions and disabled text |
| `ACCENT`, `ACCENT_SOFT`, `ON_ACCENT` | Main buttons and the current selection; its tint; text on it |
| `SECONDARY`, `SECONDARY_SOFT` | Secondary buttons, progress, switch tracks; its tint |
| `OK`, `WARN`, `ERROR`, `INFO` | Status: works, needs attention, failed, information |
| `PC` | PC VR games and "on this PC" |
FramePort refuses a theme file it can't show well, and says why: a light window background (FramePort is dark only),
text that's hard to read on cards, an unknown colour name or a value that isn't a colour. An example is in
[`themes/example-theme.json`](themes/example-theme.json).
Binary file not shown.

After

Width:  |  Height:  |  Size: 188 KiB

-13
View File
@@ -1,13 +0,0 @@
{
"name": "Ember & Ice",
"base": "portal",
"dual": true,
"colors": {
"ACCENT": "#FF5A36",
"ACCENT_SOFT": "#3A1710",
"SECONDARY": "#5CE1E6",
"SECONDARY_SOFT": "#0E3033",
"PC": "#5CE1E6",
"INFO": "#5CE1E6"
}
}
+4 -1
View File
@@ -11,14 +11,17 @@ embeds debug paths (`-g`) and the dex depends on the d8 version, so those differ
| Dir | Artifact | What / why |
|---|---|---|
| `adapter/` | `{arm64-v8a,armeabi-v7a}/libopenxr_loader_generic.so` | **FrameBridge**: replaces overport's generic loader (which is renamed `libopenxr_loader_original.so`) and fixes Steam Frame runtime gaps. Hooks xrCreateInstance, swapchains, xrEndFrame, spaces, xrPollEvent, … everything else is forwarded by `gen_forwarders.py`-generated tail calls. `scene_emu.c` = Meta scene/spatial-entity emulation; `flip_vk.c` = Vulkan blit for VERTICAL_FLIP quads. `session_fixes.c` = per-game session fixes, all off by default and hooked only when on (`layer_debug` diagnostics, `stable_local`, `focus_hold`, aim correction `aim_pitch/aim_yaw/aim_forward`, `refresh_rate`); `input_diag.c` = controller-input diagnostics (setting `input_diag`, off by default and hooked only when on; logs suggested interaction profiles and the runtime's answers, each hand's current profile, functions the runtime lacks and failing input/haptics/perf calls, each line once; tested on the host by `tests/test_input_diag.py`); `layer_emul_gl.c` = `equirect_emul` (GLES sessions only): 360° equirect/equirect2 layers become one adapter projection layer, drawn by a worker thread with its own EGL context shared with the app's: each 360° image is converted to a cube map only when it changes, and the view is redrawn only after a 4° head turn (GL/EGL resolved with dlopen, so Vulkan games never load them); `layer_math.h` = their pose/face math, tested on the host by `tests/test_layer_emul.py`. `render_model.c` = XR_FB_render_model emulation (setting `controller_models`, off by default): serves `files/framebridge/controller_{left,right}.glb`, which the agent converts on the Frame from SteamVR's own Frame controller render models (never shipped by FramePort); tested on the host by `tests/test_render_model.py`. Settings: `libframe_settings.so` in the APK + `settings.conf`/`framebridge.conf` on the Frame. |
| `adapter/` | `{arm64-v8a,armeabi-v7a}/libopenxr_loader_generic.so` | **FrameBridge**: replaces overport's generic loader (which is renamed `libopenxr_loader_original.so`) and fixes Steam Frame runtime gaps. Hooks xrCreateInstance, swapchains, xrEndFrame, spaces, xrPollEvent, … everything else is forwarded by `gen_forwarders.py`-generated tail calls. `scene_emu.c` = Meta scene/spatial-entity emulation; `flip_vk.c` = Vulkan blit for VERTICAL_FLIP quads. `session_fixes.c` = per-game session fixes, all off by default and hooked only when on (`layer_debug` diagnostics, `stable_local`, `focus_hold`, aim correction `aim_pitch/aim_yaw/aim_forward`, `refresh_rate`, `proximity_emul` = finger proximity (XR_FB_touch_controller_proximity, which the Frame lacks) from the capacitive touch inputs: OVRPlugin's `hand_thumb_proximity`/`hand_trigger_proximity` actions get extra Touch bindings); `input_diag.c` = controller-input diagnostics (setting `input_diag`, off by default and hooked only when on; logs suggested interaction profiles and the runtime's answers, each hand's current profile, functions the runtime lacks and failing input/haptics/perf calls, each line once; tested on the host by `tests/test_input_diag.py`); `layer_emul_gl.c` = `equirect_emul` (GLES sessions only): 360° equirect/equirect2 layers become one adapter projection layer, drawn by a worker thread with its own EGL context shared with the app's: each 360° image is converted to a cube map only when it changes, and the view is redrawn only after a 4° head turn (GL/EGL resolved with dlopen, so Vulkan games never load them); `layer_math.h` = their pose/face math, tested on the host by `tests/test_layer_emul.py`. `render_model.c` = XR_FB_render_model emulation (setting `controller_models`, off by default): serves `files/framebridge/controller_{left,right}.glb`, which the agent converts on the Frame from SteamVR's own Frame controller render models (never shipped by FramePort); tested on the host by `tests/test_render_model.py`. Settings: `libframe_settings.so` in the APK + `settings.conf`/`framebridge.conf` on the Frame. |
| `vrapi-bridge/` | `arm64-v8a/libvrapi.so` | VrApi → OpenXR bridge from [Android-XR-Bridge/OVRPort](https://github.com/Android-XR-Bridge/OVRPort) `native/vrapi` at 5e7df52 (GPL-3.0, `LICENSE.upstream`; unchanged upstream as of OVRPort v1.2.5, which ships the unpatched code as its opt-in `patch_vrapi_openxr` in experimental CLI builds), with our changes in `upstream-patches/`: GLES sessions + GL texture swapchains, cylinder→quad layers, GL vertical flip, UNORM↔sRGB format twins, 30 s VR-mode deadline, VALID-only recenter, loading-icon layers skipped, `vrapi_PollEvent`/`RecenterPose`/`SetDisplayRefreshRate`/`GetSystemPropertyFloatArray` (BlazeRush imports them, GitHub #57), diagnostics behind `-DOVP_GL_DIAG`. `upstream-patches/` reproduces these sources from the fork's `native/vrapi` (checked 2026-10-06 against its latest commit); upstream PRs Android-XR-Bridge/OVRPort#3-#8, tracked in GitHub #75. |
| `platformcompat/` | `arm64-v8a/libovrplatformcompat.so` | Real `ovrMessageType_ToString` (same fork, `native/platform`). |
| `langpack/` | `arm64-v8a/libfp_langpack.so` | Language packs for the Meta Platform SDK (patch `frame.langpacks`, opt-in). overport's platform loader is a dispatcher in front of Meta's own loader and answers `ovr_LanguagePack_GetCurrent/SetCurrent` with `return 0` (no reply ever). This library becomes the loader's first DT_NEEDED; the loader's own exports of the 31 functions it defines are made STB_LOCAL in `.dynsym` (`elf.hide_exports`, nothing moves), so symbol lookups reach the library first. It serves `<tag>.lang` files found in the game's OBB / app files (`$FRAMEPORT_LANGPACK_DIRS` overrides the search path) through `ovr_LanguagePack_*`, `ovr_AssetFile_GetList` (the loader's list + the packs) and `ovr_AssetFile_StatusById`; its own message queue is merged into `ovr_PopMessage`, and every handle that is not ours (an address-range check) goes to the loader's original function, whose address is read at run time from the loader's in-memory `.dynsym`. With patch `frame.asset_files` (marker byte `@FPASSETS@`, or env `FRAMEPORT_ASSET_FILES=1`) it also lists the data's `*.pak` content files as installed `default` asset files and answers `ovr_AssetFile_DownloadById` for them at once (a completed `DownloadUpdate`, then the result with the path). Host-tested against a stand-in loader (`tests/test_langpack.py`, plain `cc`, no NDK); the bionic/headset side is unverified. |
| `glshim/` | `arm64-v8a/libglshim.so` | Mesa GLSL compatibility for GLAD engines (hooks eglGetProcAddress): comments out `#pragma` before `#extension`, enables `GL_EXT_shader_implicit_conversions`, hides GL_OVR_multiview (`gl_hide_multiview`, default 1) and logs failed shaders. `-DGLSHIM_TRACE` adds per-FBO draw/error tracing. |
| `glmv/` | `arm64-v8a/libfpglmv.so` | Single-view draws of OVR_multiview programs (patch `frame.gl_multiview_fbo`, opt-in, GitHub #77). Mesa enforces OVR_multiview's rule that a draw's program declares as many views as the draw framebuffer has; engines that compile every shader with `num_views=2` and also draw into ordinary 2D framebuffers (Doom3Quest's HUD/PDA pool) lose those draws. The engine library gets it as first DT_NEEDED (direct `gl*` imports) and its `dlopen("libGLESv3.so")` string is rewritten to `libfpglmv.so` (same length; its `qgl*` dlsym table); linked `--no-as-needed` to libGLESv3 so dlsym on its handle finds everything it doesn't wrap. It records stage sources at glLinkProgram, caches the draw framebuffer's color-attachment view count per bind, and when a multiview program draws into a 0-view framebuffer it binds a lazily built twin (`glmv_rewrite.h`: num_views layout blanked, `gl_ViewID_OVR` -> `(0u)`; attribute locations + uniform block bindings copied, default-block uniforms copied before every such draw with a shadow to skip unchanged ones), draws, rebinds the original. Logcat tag `GLMV`; `gl_mv_debug=1`. The rewriter is host-tested on Doom3Quest's 19 shaders (`tests/test_gl_multiview_fbo.py`); untested on the device. |
| `zinkfix/` | `arm64-v8a/libVkLayer_fp_shaderfix.so` | Shader-fix Vulkan layer `VK_LAYER_FP_shader_fix` for OpenGL ES games (patch `frame.zink_shader_fix`): sits between Zink (Mesa's GL on Vulkan) and the driver and inserts the words of `zink_shader_fix` (same format as the Vulkan shim's `vk_shader_fix`: size + SHA-256 match) into SPIR-V modules; `zink_shader_dump=1` writes every distinct module to `files/fp_spirv/`. The engine library loads it first (DT_NEEDED); its constructor puts the layer in front of Android's GraphicsEnv debug layer list (private Android 11 `getDebugLayers`/`setDebugLayers`, found through libvulkan.so's dependencies) before EGL starts, so the loader picks it up from the app's lib dir. Exports only the enumeration functions and `VK_LAYER_FP_shader_fixGet*ProcAddr` (never plain `vk*` entry points). Vader Immortal's lightspeed shaders (Klownicle, GitHub #49). |
| `xrlayer/` | `linux-arm64/libxr_frameport_timefix.so` + `XR_APILAYER_FRAMEPORT_timefix.json` | OpenXR API layer for Windows PC VR games under Proton on the Frame (glibc aarch64, built freestanding with the NDK's clang), on by default (patch `pcvr.xr_timefix`). Retries `xrCreateInstance` as an OpenXR 1.0 app when the runtime rejects 1.1 (the Frame's SteamVR runtime does, and Proton 11's VR helper asks for 1.1). Also emulates `xrConvertTimespecTimeToTimeKHR`/`xrConvertTimeToTimespecTimeKHR` if the runtime refuses them (the Frame's Android runtime does; its Linux runtime, used by Proton, supports them as of 2026-09-29 — so this part is only a fallback; Proton's wineopenxr needs them for `XR_KHR_win32_convert_performance_counter_time`, which Revive requires) with the adapter's xrWaitFrame-calibrated offset, and drops the extension from xrCreateInstance if the runtime rejects it. Enabled per game by the Proton launch.sh (`XR_API_LAYER_PATH`, `XR_ENABLE_API_LAYERS`). `tests/test_xrlayer.py` builds it for x86_64 and drives it via ctypes. |
| `oculushmd/` | `win-x64/fp_oculushmd.exe` | PC VR counterpart of overport's `patch_oculus_unreal` (patch `pcvr.oculus_unreal`, default for Unreal Rift games on the Frame). Unreal's OculusHMD and LibOVR's `ovr_Detect` only start when the Windows named event `OculusHMDConnected` exists and is signalled (the Oculus service creates it). `fp_oculushmd.exe <command line>` creates it (manual reset, signalled), runs the command (Revive's injector) in a job object and keeps the event until every process in the job — the game and its children — has exited (fallback without job support: 3 minutes); returns the command's exit code and logs `FramePort oculushmd: …` to stderr (launch.log). Freestanding Windows x64 console exe (no CRT, no Windows SDK): the NDK's clang + `lld-link /Brepro` with a kernel32 import library generated by `llvm-dlltool`, so it's byte-reproducible. Tested on Windows from WSL with a build whose event name is `-DEVENT_NAME=L"…"` (the Oculus service owns the real event on a PC with the Oculus app). |
| `xrshim/` | `arm64-v8a/libframe_xrshim.so` | FrameBridge extension shim, only in builds with `controller_models=1`. overport's dispatcher (`libopenxr_loader.so`) answers `xrGetInstanceProcAddr` from a fixed table and refuses everything else (`overportOXR: Unknown proc addr: xrLoadRenderModelFB`), so the adapter never sees XR_FB_render_model lookups. Meta's OVRPlugin gets the function with `dlopen("libopenxr_loader.so")` + `dlsym`, so FramePort rewrites that `.rodata` string in `libOVRPlugin.so` (same length, in place) to `libframe_xrshim.so`. The shim returns the adapter's emulated functions (`framebridge_extension_proc`) and forwards every other name to overport. A DT_NEEDED interposer does not work here (dlsym on a handle searches that library first); tried and dropped. |
| `venc/` | `linux-arm64-bin/fp_venc` | Hardware H.264 encoder for the live view, run on the Frame (spec: `venc/SPEC.md`). Reads SteamVR's "headset view" webcam (v4l2loopback `/dev/video99`, RGB24), box-scales and converts it to NV12 (BT.709 limited range; `convert.c`, NEON with a bit-identical scalar reference, also built for x86_64 by the host test), and feeds the SoC's V4L2 stateful encoder (`dev-encoder.rst`, Qualcomm iris `/dev/video-enc0`): CBR, no B-frames, SPS/PPS on every IDR, one frame per slot of a fixed grid (fps = a divisor of the panel refresh read through DRM KMS, default cap 45), Annex B on stdout. stdin `k` = keyframe, `q`/EOF = drain and stop; `--probe` prints the setup as JSON, `--selftest` compares NEON and scalar output. Fully static and freestanding (raw syscalls, own `_start`; only the NDK's kernel UAPI headers, staged in `.cache/uapi-arm64`), byte-reproducible. Lives in `linux-arm64-bin/` because `linux-arm64/` is uploaded whole as the OpenXR layer. |
| `java-stubs/` | `dex/oculusos-stubs.dex` | No-op `com.oculus.os.AnalyticsEvent` / `UnifiedTelemetryLogger` for native code that looks them up. |
The ovr_* stub library is *not* prebuilt: `src/frameport/analysis/stubgen.py` generates it per game from the exact
+130
View File
@@ -0,0 +1,130 @@
// SPDX-License-Identifier: GPL-3.0-only
// Included by frame_adapter.c (after layer_emul_gl.c, whose GL entry points it shares). cube_standin (default on):
// the Frame's runtime has no cube layers (XR_KHR_composition_layer_cube) and refuses cube swapchains (faceCount 6)
// with XR_ERROR_RUNTIME_FAILURE. OVRPlugin doesn't check: it goes on with "CreateSwapchain for eye 0: 0x0, 0 stages"
// and writes past its empty image list in the next ovrp_EndFrame4 (Unity's render thread SIGSEGV in memset, e.g.
// Budget Cuts Ultimate's cube-map overlay, GitHub #107). Only when the runtime refused one, a cube swapchain is served
// by the adapter instead: one GL cube-map texture (the format, size and mip levels asked for) in the app's current
// GLES context, acquire/wait/release succeed at once. The app renders or copies into it as usual; layers that use it
// are dropped in xrEndFrame (the runtime couldn't show them anyway). Vulkan sessions (no current EGL context) keep
// the runtime's error.
#define STANDIN_MAX 8
static struct { XrSwapchain handle; GLuint tex; } standins[STANDIN_MAX];
static pthread_mutex_t standin_lock = PTHREAD_MUTEX_INITIALIZER;
static int standin_find(XrSwapchain handle) {
if (!handle) return -1;
int slot = -1;
pthread_mutex_lock(&standin_lock);
for (int i = 0; i < STANDIN_MAX && slot < 0; ++i)
if (standins[i].handle == handle) slot = i;
pthread_mutex_unlock(&standin_lock);
return slot;
}
static int is_standin(XrSwapchain handle) { return cube_standin && standin_find(handle) >= 0; }
// After the runtime refused a swapchain: serve a cube stand-in if it was a cube swapchain in a GLES context.
static XrResult standin_after_failure(const XrSwapchainCreateInfo *info, XrSwapchain *out, XrResult result) {
if (XR_SUCCEEDED(result) || !cube_standin || !info || !out || info->faceCount != 6) return result;
if (info->width != info->height || !info->width || info->arraySize > 1) return result;
if (!p_eglCreateContext && !emul_load_gl()) { LOG("cube_standin: no GLES, the runtime's error stays"); return result; }
if (p_eglGetCurrentContext() == EGL_NO_CONTEXT) {
LOG("cube_standin: no current GLES context (Vulkan?), the runtime's error stays");
return result;
}
int slot = -1;
pthread_mutex_lock(&standin_lock);
for (int i = 0; i < STANDIN_MAX && slot < 0; ++i)
if (!standins[i].handle) slot = i;
if (slot >= 0) standins[slot].handle = (XrSwapchain)1; // reserved
pthread_mutex_unlock(&standin_lock);
if (slot < 0) { LOG("cube_standin: too many cube swapchains, the runtime's error stays"); return result; }
GLsizei levels = info->mipCount > 1 ? (GLsizei)info->mipCount : 1;
for (int i = 0; i < 8 && p_glGetError() != GL_NO_ERROR; ++i) {}
GLint previous = 0;
p_glGetIntegerv(GL_TEXTURE_BINDING_CUBE_MAP, &previous);
GLuint tex = 0;
p_glGenTextures(1, &tex);
p_glBindTexture(GL_TEXTURE_CUBE_MAP, tex);
p_glTexStorage2D(GL_TEXTURE_CUBE_MAP, levels, (GLenum)info->format, (GLsizei)info->width, (GLsizei)info->height);
GLenum err = p_glGetError();
p_glBindTexture(GL_TEXTURE_CUBE_MAP, (GLuint)previous); // the app's binding, untouched
if (err != GL_NO_ERROR) {
p_glDeleteTextures(1, &tex);
pthread_mutex_lock(&standin_lock);
standins[slot].handle = XR_NULL_HANDLE;
pthread_mutex_unlock(&standin_lock);
LOG("cube_standin: couldn't make a %ux%u cube map (GL 0x%x), the runtime's error stays", info->width,
info->height, err);
return result;
}
XrSwapchain handle = FAKE_HANDLE(XrSwapchain);
pthread_mutex_lock(&standin_lock);
standins[slot].handle = handle;
standins[slot].tex = tex;
pthread_mutex_unlock(&standin_lock);
*out = handle;
LOG("cube_standin: runtime refused a %ux%u cube swapchain (result=%d): served by FrameBridge (GL cube map %u, "
"%d levels); its cube layers are dropped", info->width, info->height, result, tex, (int)levels);
return XR_SUCCESS;
}
static XrResult standin_enumerate(XrSwapchain handle, uint32_t capacity, uint32_t *count, XrSwapchainImageBaseHeader *images) {
if (!count) return XR_ERROR_VALIDATION_FAILURE;
*count = 1;
if (!capacity) return XR_SUCCESS;
if (!images) return XR_ERROR_VALIDATION_FAILURE;
if (images->type != XR_TYPE_SWAPCHAIN_IMAGE_OPENGL_ES_KHR) return XR_ERROR_VALIDATION_FAILURE;
int slot = standin_find(handle);
((emul_gles_image *)images)[0].image = slot >= 0 ? standins[slot].tex : 0;
return XR_SUCCESS;
}
static void standin_destroy(XrSwapchain handle) {
pthread_mutex_lock(&standin_lock);
for (int i = 0; i < STANDIN_MAX; ++i)
if (standins[i].handle == handle) {
if (p_eglGetCurrentContext && p_eglGetCurrentContext() != EGL_NO_CONTEXT) p_glDeleteTextures(1, &standins[i].tex);
standins[i].handle = XR_NULL_HANDLE;
standins[i].tex = 0;
}
pthread_mutex_unlock(&standin_lock);
}
// A layer that shows a stand-in: never passed to the runtime (whatever layer_fix says).
static int layer_uses_standin(const XrCompositionLayerBaseHeader *layer) {
if (!cube_standin || !layer) return 0;
switch (layer->type) {
case XR_TYPE_COMPOSITION_LAYER_CUBE_KHR:
return is_standin(((const XrCompositionLayerCubeKHR *)layer)->swapchain);
case XR_TYPE_COMPOSITION_LAYER_QUAD:
return is_standin(((const XrCompositionLayerQuad *)layer)->subImage.swapchain);
case XR_TYPE_COMPOSITION_LAYER_CYLINDER_KHR:
return is_standin(((const XrCompositionLayerCylinderKHR *)layer)->subImage.swapchain);
case XR_TYPE_COMPOSITION_LAYER_EQUIRECT_KHR:
return is_standin(((const XrCompositionLayerEquirectKHR *)layer)->subImage.swapchain);
case XR_TYPE_COMPOSITION_LAYER_EQUIRECT2_KHR:
return is_standin(((const XrCompositionLayerEquirect2KHR *)layer)->subImage.swapchain);
case XR_TYPE_COMPOSITION_LAYER_PROJECTION: {
const XrCompositionLayerProjection *p = (const XrCompositionLayerProjection *)layer;
for (uint32_t v = 0; v < p->viewCount; ++v)
if (is_standin(p->views[v].subImage.swapchain)) return 1;
return 0;
}
default:
return 0;
}
}
// XR_FB_swapchain_update_state on a stand-in: nothing to update (only hooked when the runtime has the function).
static PFN_xrUpdateSwapchainFB real_update_swapchain;
static PFN_xrGetSwapchainStateFB real_get_swapchain_state;
static XRAPI_ATTR XrResult XRAPI_CALL standin_update_swapchain(XrSwapchain swapchain, const XrSwapchainStateBaseHeaderFB *state) {
if (is_standin(swapchain)) return XR_SUCCESS;
return real_update_swapchain ? real_update_swapchain(swapchain, state) : XR_ERROR_FUNCTION_UNSUPPORTED;
}
static XRAPI_ATTR XrResult XRAPI_CALL standin_get_swapchain_state(XrSwapchain swapchain, XrSwapchainStateBaseHeaderFB *state) {
if (is_standin(swapchain)) return XR_ERROR_VALIDATION_FAILURE; // no state to report (apps treat it as "unsupported")
return real_get_swapchain_state ? real_get_swapchain_state(swapchain, state) : XR_ERROR_FUNCTION_UNSUPPORTED;
}
+1
View File
@@ -40,6 +40,7 @@ static XRAPI_ATTR XrResult XRAPI_CALL eye_hook_xrReleaseSwapchainImage(XrSwapcha
const XrSwapchainImageReleaseInfo *info) {
PFN_xrReleaseSwapchainImage fn = (PFN_xrReleaseSwapchainImage)lookup(active_instance, "xrReleaseSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) return XR_SUCCESS;
if (release_wait_active()) eye_wait_for_gpu();
XrResult result = fn(swapchain, info);
if (eye_debug) {
+5 -1
View File
@@ -47,6 +47,7 @@ typedef struct {
VkImage images[MAX_IMAGES];
uint32_t image_count;
uint32_t last_acquired;
uint32_t acquired[MAX_IMAGES], acquired_count, waited_count;
XrSwapchain shadow;
XrRect2Di shadow_rect;
VkImage shadow_images[MAX_IMAGES];
@@ -105,7 +106,10 @@ static void flip_on_enumerate_images(XrSwapchain handle, uint32_t count, const X
static void flip_on_acquire(XrSwapchain handle, uint32_t index) {
pthread_mutex_lock(&flip_lock);
tracked_swapchain *t = find_tracked(handle, 0);
if (t) t->last_acquired = index;
if (t) {
t->last_acquired = index;
if(t->acquired_count<MAX_IMAGES)t->acquired[t->acquired_count++]=index;
}
pthread_mutex_unlock(&flip_lock);
}
+272 -8
View File
@@ -15,6 +15,7 @@
// * drops requested instance extensions the runtime lacks (e.g. XR_FB_passthrough);
// * retries rejected swapchains with 1 sample / a supported format (swapchain_fix, overport #71);
// * drops composition layers that reference swapchains that failed to create (layer_fix);
// * serves cube swapchains the runtime refuses as GL cube maps and drops their layers (cube_standin, cube_standin.c);
// * optionally requests mutable-format swapchain images (mutable_fix, off by default);
// * optionally swaps left/right images of stereo projection layers (swap_eyes, off by default);
// * emulates XR_FB_passthrough with XR_ENVIRONMENT_BLEND_MODE_ALPHA_BLEND when the runtime lacks it
@@ -37,6 +38,7 @@
// /sdcard/Android/data/<package>/files/framebridge.conf
// $FRAMEBRIDGE_CONFIG
#define _GNU_SOURCE
#define VK_USE_PLATFORM_ANDROID_KHR
#define XR_EXTENSION_PROTOTYPES
#include <openxr/openxr.h>
#include <android/log.h>
@@ -73,6 +75,7 @@ static void fb_log(const char *fmt, ...) {
}
#define LOG(...) fb_log(__VA_ARGS__)
#include "audio_metadata.h"
#include "surface_views.h"
static void *loader;
static PFN_xrGetInstanceProcAddr next_gipa;
@@ -80,8 +83,10 @@ static pthread_once_t init_once = PTHREAD_ONCE_INIT;
static XrInstance active_instance = XR_NULL_HANDLE;
static void *android_vm; // JavaVM from XrInstanceCreateInfoAndroidKHR (surface_swapchain.c)
static int surface_emul = 1; // setting: emulate Android surface swapchains
static int surface_native; // internal: GPU shared video buffers (panoramic video clients)
static int eye_debug; // setting: per-eye diagnostics + file log (eye_debug.c)
static int release_wait; // setting: wait for the app's GPU work before releasing images (2 = after 60 s: A/B)
static void surf_scene_destroy(XrSwapchain handle);
static void surf_on_destroy(XrSwapchain handle);
static float scale = 1.0f;
@@ -103,8 +108,11 @@ static int frame_open;
static int sc_acquires, sc_waits, sc_releases, sc_wait_fails; // per pacing period (layer_debug) // drop XrCompositionLayerColorScaleBiasKHR (the runtime's color pass)
static int controller_fix = 1;
static int swapchain_fix = 1;
static int cube_standin = 1; // cube swapchains the runtime refuses: served by the adapter (cube_standin.c)
static int rect_clamp = 1;
static int pose_consistency = 0; // repeat xrLocateViews queries for one display time get the same poses (GitHub #8) // clamp submitted image rects to their swapchain (SteamVR rejects a 1 px overrun)
static int pose_time_fix; // poses asked for at CLOCK_MONOTONIC "now" or far in the past: at XrTime "now" (pose_time.c)
static int pose_debug; // diagnostics: requested pose times vs the predicted display time (pose_time.c)
static int layer_fix = 1;
static int mutable_fix = 0;
static int swap_eyes = 0;
@@ -127,6 +135,8 @@ static int controller_models; // serve Frame controller models via XR_FB_render
// Per-game settings, all off by default (session_fixes.c, layer_emul_gl.c).
static int sync_guard; // xrSyncActions one at a time with xrPollEvent, paused briefly after focus returns
static int profile_remap = 1; // Meta's newer controller profiles (rejected by the Frame) -> oculus/touch_controller
static int proximity_emul; // finger proximity from capacitive touch: 1 thumb, 2 thumb + index (session_fixes.c)
static int runtime_has_proximity; // the runtime has XR_FB_touch_controller_proximity itself
static int layer_debug; // diagnostics: layers, swapchains, session states, spaces, aim/grip, refresh rates
static int input_diag; // diagnostics: controller profiles, bindings, missing functions, failing input calls
static int stable_local; // keep every LOCAL space the app creates on the session-start origin
@@ -137,7 +147,7 @@ static float aim_pitch, aim_yaw, aim_forward; // aim pose correction (degrees,
static float refresh_rate; // requested display refresh rate (Hz), 0 = the app's choice
static int equirect_emul; // show 360 layers as cube faces (GLES)
static int equirect_face = 1536; // max cube map face size (px)
static int equirect_res = 1536; // projection image size per eye (px)
static int equirect_res = 1536, equirect_res_set; // projection image size per eye (px)
static int equirect_flip; // source orientation fix: 1 upside down, 2 mirrored, 4 turned 180 degrees
static float equirect_fps = 60; // max redraws per second per 360 layer (0 = no limit)
static int equirect_stereo; // 0 auto (mono if the layer limit can't hold both eyes), 1 stereo, 2 mono
@@ -158,8 +168,11 @@ static void read_settings(const char *path) {
if (sscanf(line, "snapshot=%f", &value) == 1) snapshot = value > 0 ? (int)value : 0;
if (sscanf(line, "controller_fix=%f", &value) == 1) controller_fix = value != 0;
if (sscanf(line, "swapchain_fix=%f", &value) == 1) swapchain_fix = value != 0;
if (sscanf(line, "cube_standin=%f", &value) == 1) cube_standin = value != 0;
if (sscanf(line, "rect_clamp=%f", &value) == 1) rect_clamp = value != 0;
if (sscanf(line, "pose_consistency=%f", &value) == 1) pose_consistency = value != 0;
if (sscanf(line, "pose_debug=%f", &value) == 1) pose_debug = value != 0;
if (sscanf(line, "pose_time_fix=%f", &value) == 1) pose_time_fix = value != 0;
if (sscanf(line, "layer_fix=%f", &value) == 1) layer_fix = value != 0;
if (sscanf(line, "mutable_fix=%f", &value) == 1) mutable_fix = value != 0;
if (sscanf(line, "swap_eyes=%f", &value) == 1) swap_eyes = value != 0;
@@ -177,9 +190,11 @@ static void read_settings(const char *path) {
if (sscanf(line, "layer_debug=%f", &value) == 1) layer_debug = value != 0;
if (sscanf(line, "input_diag=%f", &value) == 1) input_diag = value != 0;
if (sscanf(line, "surface_emul=%f", &value) == 1) surface_emul = value != 0;
if (sscanf(line, "surface_native=%f", &value) == 1) surface_native = value != 0;
if (sscanf(line, "eye_debug=%f", &value) == 1) eye_debug = value != 0;
if (sscanf(line, "release_wait=%f", &value) == 1) release_wait = (int)value;
if (sscanf(line, "profile_remap=%f", &value) == 1) profile_remap = value != 0;
if (sscanf(line, "proximity_emul=%f", &value) == 1 && value >= 0 && value <= 2) proximity_emul = (int)value;
if (sscanf(line, "sync_guard=%f", &value) == 1) sync_guard = value != 0;
if (sscanf(line, "stable_local=%f", &value) == 1) stable_local = value != 0;
if (sscanf(line, "focus_hold=%f", &value) == 1) focus_hold = value != 0;
@@ -191,7 +206,7 @@ static void read_settings(const char *path) {
if (sscanf(line, "refresh_rate=%f", &value) == 1 && (value == 0 || (value >= 60 && value <= 144))) refresh_rate = value;
if (sscanf(line, "equirect_emul=%f", &value) == 1) equirect_emul = value != 0;
if (sscanf(line, "equirect_face=%f", &value) == 1 && value >= 256 && value <= 2730) equirect_face = (int)value;
if (sscanf(line, "equirect_res=%f", &value) == 1 && value >= 512 && value <= 4096) equirect_res = (int)value;
if (sscanf(line, "equirect_res=%f", &value) == 1 && value >= 512 && value <= 4096) {equirect_res = (int)value;equirect_res_set=1;}
if (sscanf(line, "equirect_flip=%f", &value) == 1 && value >= 0 && value <= 7) equirect_flip = (int)value;
if (sscanf(line, "equirect_fps=%f", &value) == 1 && value >= 0 && value <= 144) equirect_fps = value;
if (sscanf(line, "equirect_stereo=%f", &value) == 1 && value >= 0 && value <= 2) equirect_stereo = (int)value;
@@ -240,6 +255,8 @@ static void initialize(void) {
log_file = fopen(path, "a");
}
if (input_diag) LOG("per-game: input_diag=1 (controller-input diagnostics)");
if (proximity_emul) LOG("per-game: proximity_emul=%d (finger proximity from capacitive touch: %s)", proximity_emul,
proximity_emul > 1 ? "thumb + index" : "thumb");
if (hide_space_warp) LOG("per-game: hide_space_warp=1 (XR_FB_space_warp hidden, space warp info removed)");
if (strip_color_bias) LOG("per-game: strip_color_bias=%d (layer color scale/bias removed)", strip_color_bias);
if (frame_balance) LOG("per-game: frame_balance=1 (an open frame is ended before the next one begins)");
@@ -359,6 +376,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateInstance(const XrInstanceCreateInfo *info
}
for (uint32_t j = 0; available && j < available_count; ++j)
if (!strcmp(available[j].extensionName, "XR_FB_composition_layer_image_layout")) runtime_has_image_layout = 1;
else if (!strcmp(available[j].extensionName, "XR_FB_touch_controller_proximity")) runtime_has_proximity = 1;
// Layer types are only valid if their extension ends up enabled; some apps submit them regardless.
int has_equirect = 0, has_equirect2 = 0, has_cylinder = 0, has_cube = 0;
for (uint32_t i = 0; i < kept; ++i) {
@@ -499,6 +517,10 @@ static int emul_is_virtual(XrSwapchain handle);
static void emul_virtual_destroy(XrSwapchain handle);
static XrResult emul_virtual_enumerate(XrSwapchain handle, uint32_t capacity, uint32_t *count, XrSwapchainImageBaseHeader *images);
static void emul_on_acquire(XrSwapchain handle, uint32_t index);
static XrResult standin_after_failure(const XrSwapchainCreateInfo *info, XrSwapchain *out, XrResult result);
static int is_standin(XrSwapchain handle);
static void standin_destroy(XrSwapchain handle);
static XrResult standin_enumerate(XrSwapchain handle, uint32_t capacity, uint32_t *count, XrSwapchainImageBaseHeader *images);
static int known_swapchain(XrSwapchain handle) {
int found = 0;
@@ -521,6 +543,11 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSwapchain(XrSession session, const XrSwap
fixed.next = p;
}
if (mutable_fix) fixed.usageFlags |= XR_SWAPCHAIN_USAGE_MUTABLE_FORMAT_BIT;
// Internal native-video composition snapshots the owned stereo scene before
// release. Other games and UI/depth/MSAA swapchains keep their original usage.
if(surface_native && fixed.arraySize==2 && fixed.sampleCount==1 &&
(fixed.usageFlags&XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT))
fixed.usageFlags |= XR_SWAPCHAIN_USAGE_TRANSFER_SRC_BIT;
if (equirect_emul && swapchain && emul_virtual_create(session, &fixed, swapchain)) {
remember_swapchain(*swapchain);
emul_on_create_swapchain(*swapchain, &fixed);
@@ -539,7 +566,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSwapchain(XrSession session, const XrSwap
snap_on_create(*swapchain, &fixed);
return result;
}
if (!swapchain_fix) return result;
if (!swapchain_fix) return standin_after_failure(&fixed, swapchain, result);
// Frame's runtime rejects some GLES formats (GL_RGBA8) and MSAA swapchains (overport #71).
if (fixed.sampleCount > 1) {
@@ -562,7 +589,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrCreateSwapchain(XrSession session, const XrSwap
emul_on_create_swapchain(*swapchain, &fixed);
snap_on_create(*swapchain, &fixed);
}
return result;
return standin_after_failure(&fixed, swapchain, result);
}
XRAPI_ATTR XrResult XRAPI_CALL xrDestroySwapchain(XrSwapchain swapchain) {
@@ -572,6 +599,8 @@ XRAPI_ATTR XrResult XRAPI_CALL xrDestroySwapchain(XrSwapchain swapchain) {
static int logged;
if (logged++ < 200) LOG("layer_debug: xrDestroySwapchain %p", (void *)swapchain);
}
if (is_standin(swapchain)) { standin_destroy(swapchain); return XR_SUCCESS; }
surf_scene_destroy(swapchain);
forget_swapchain(swapchain);
surf_on_destroy(swapchain);
flip_on_destroy(swapchain);
@@ -753,6 +782,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrGetSystemProperties(XrInstance instance, XrSyst
}
#include "flip_vk.c"
#include "pose_time.c"
// XR_KHR_convert_timespec_time for runtimes that lack it (Frame): XrTime is derived from the monotonic clock
// using the offset measured at the last xrWaitFrame. The runtime's own implementation is preferred.
@@ -762,10 +792,14 @@ static PFN_xrConvertTimeToTimespecTimeKHR runtime_time_to_timespec;
static XRAPI_ATTR XrResult XRAPI_CALL emu_timespec_to_time(XrInstance instance, const struct timespec *ts, XrTime *time) {
if (runtime_timespec_to_time) {
XrResult r = runtime_timespec_to_time(instance, ts, time);
if (r != XR_ERROR_FUNCTION_UNSUPPORTED) return r;
if (r != XR_ERROR_FUNCTION_UNSUPPORTED) {
if (XR_SUCCEEDED(r) && time) pose_time_note("xrConvertTimespecTimeToTimeKHR (runtime)", 0, 0, *time);
return r;
}
}
if (!ts || !time) return XR_ERROR_VALIDATION_FAILURE;
*time = (XrTime)((int64_t)ts->tv_sec * 1000000000ll + ts->tv_nsec + (xr_time_calibrated ? xr_time_offset : 0));
pose_time_note("xrConvertTimespecTimeToTimeKHR", 0, 0, *time);
return XR_SUCCESS;
}
static XRAPI_ATTR XrResult XRAPI_CALL emu_time_to_timespec(XrInstance instance, XrTime time, struct timespec *ts) {
@@ -781,6 +815,8 @@ static XRAPI_ATTR XrResult XRAPI_CALL emu_time_to_timespec(XrInstance instance,
}
XRAPI_ATTR XrResult XRAPI_CALL xrLocateSpace(XrSpace space, XrSpace baseSpace, XrTime time, XrSpaceLocation *location) {
time = pose_time_fixed(time);
pose_time_note("xrLocateSpace", (uint64_t)(uintptr_t)space, (uint64_t)(uintptr_t)baseSpace, time);
if (emulate_scene && location) {
XrPosef pose; XrSpaceLocationFlags flags;
if (locate_fake(space, baseSpace, time, &pose, &flags)) {
@@ -796,6 +832,14 @@ XRAPI_ATTR XrResult XRAPI_CALL xrLocateSpace(XrSpace space, XrSpace baseSpace, X
static XrResult locate_spaces_common(const char *name, XrSession session, const XrSpacesLocateInfo *info,
XrSpaceLocations *locations) {
int any_fake = 0;
XrSpacesLocateInfo moved;
if (info && pose_time_fix) {
moved = *info;
moved.time = pose_time_fixed(info->time);
info = &moved;
}
if (info) pose_time_note(name, info->spaceCount ? (uint64_t)(uintptr_t)info->spaces[0] : 0,
(uint64_t)(uintptr_t)info->baseSpace, info->time);
if (emulate_scene && info && locations) {
any_fake = fake_space_index(info->baseSpace) >= 0;
for (uint32_t i = 0; i < info->spaceCount && !any_fake; ++i) any_fake = fake_space_index(info->spaces[i]) >= 0;
@@ -829,8 +873,108 @@ XRAPI_ATTR XrResult XRAPI_CALL xrDestroySpace(XrSpace space) {
#include "input_diag.c"
#include "session_fixes.c"
#include "layer_emul_gl.c"
#include "cube_standin.c"
#include "snapshot_gl.c"
#include "surface_swapchain.c"
// GPU video buffers need explicit external-memory support on the app's device.
// Preserve the runtime's extension list and both Vulkan-enable entry points.
XRAPI_ATTR XrResult XRAPI_CALL xrCreateVulkanInstanceKHR(XrInstance instance,const XrVulkanInstanceCreateInfoKHR *info,
VkInstance *created,VkResult *vulkanResult) {
PFN_xrCreateVulkanInstanceKHR fn=(PFN_xrCreateVulkanInstanceKHR)lookup(instance,"xrCreateVulkanInstanceKHR");
if(!fn)return XR_ERROR_FUNCTION_UNSUPPORTED;
if(!surface_native)return fn(instance,info,created,vulkanResult);
XrResult result=fn(instance,info,created,vulkanResult);
if(XR_SUCCEEDED(result) && created && vulkanResult && *vulkanResult==VK_SUCCESS)vk.instance=*created;
return result;
}
static const char *surf_device_exts[] = {"VK_ANDROID_external_memory_android_hardware_buffer",
"VK_KHR_external_memory", "VK_KHR_get_memory_requirements2", "VK_KHR_dedicated_allocation",
"VK_EXT_queue_family_foreign", "VK_KHR_sampler_ycbcr_conversion", "VK_KHR_bind_memory2", "VK_KHR_maintenance1"};
static int surf_extension_present(const char *text,const char *name) {
size_t length=strlen(name);
for(const char *p=text;(p=strstr(p,name));p++)
if((p==text || p[-1]==' ') && (p[length]==' ' || p[length]=='\0'))return 1;
return 0;
}
static int surf_device_supported(PFN_vkGetInstanceProcAddr gipa, VkInstance instance, VkPhysicalDevice physical) {
if (!surface_native || !gipa) return 0;
if(!physical){
// Unity can ask for extension strings before it creates its instance.
// Query a temporary instance instead of depending on that call order.
PFN_vkCreateInstance create=(PFN_vkCreateInstance)gipa(VK_NULL_HANDLE,"vkCreateInstance");
VkApplicationInfo app={VK_STRUCTURE_TYPE_APPLICATION_INFO,NULL,"FrameBridge video capabilities",0,NULL,0,VK_API_VERSION_1_1};
VkInstanceCreateInfo ci={VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO,NULL,0,&app};
VkInstance probe=VK_NULL_HANDLE;if(!create || create(&ci,NULL,&probe)!=VK_SUCCESS)return 0;
PFN_vkEnumeratePhysicalDevices devices=(PFN_vkEnumeratePhysicalDevices)gipa(probe,"vkEnumeratePhysicalDevices");
PFN_vkDestroyInstance destroy=(PFN_vkDestroyInstance)gipa(probe,"vkDestroyInstance");
uint32_t count=0;VkPhysicalDevice gpu=VK_NULL_HANDLE;
int ok=devices && devices(probe,&count,NULL)==VK_SUCCESS && count==1 &&
devices(probe,&count,&gpu)==VK_SUCCESS && surf_device_supported(gipa,probe,gpu);
if(destroy)destroy(probe,NULL);return ok;
}
PFN_vkEnumerateDeviceExtensionProperties enumerate=(PFN_vkEnumerateDeviceExtensionProperties)
gipa(instance,"vkEnumerateDeviceExtensionProperties");
uint32_t count=0;
if(!enumerate || enumerate(physical,NULL,&count,NULL)!=VK_SUCCESS || count>2048)return 0;
VkExtensionProperties *exts=calloc(count,sizeof(*exts));
if(!exts)return 0;
int ok=enumerate(physical,NULL,&count,exts)==VK_SUCCESS;
for(unsigned j=0;j<sizeof(surf_device_exts)/sizeof(*surf_device_exts);j++){
int found=0;for(uint32_t i=0;i<count;i++)found|=!strcmp(exts[i].extensionName,surf_device_exts[j]);
ok &= found;
}
free(exts);return ok;
}
XRAPI_ATTR XrResult XRAPI_CALL xrGetVulkanGraphicsDeviceKHR(XrInstance instance,XrSystemId system,
VkInstance vulkanInstance,VkPhysicalDevice *physical) {
PFN_xrGetVulkanGraphicsDeviceKHR fn=(PFN_xrGetVulkanGraphicsDeviceKHR)lookup(instance,"xrGetVulkanGraphicsDeviceKHR");
if(!surface_native)return fn?fn(instance,system,vulkanInstance,physical):XR_ERROR_FUNCTION_UNSUPPORTED;
XrResult result=fn?fn(instance,system,vulkanInstance,physical):XR_ERROR_FUNCTION_UNSUPPORTED;
if(XR_SUCCEEDED(result) && physical){vk.instance=vulkanInstance;vk.physical=*physical;}
return result;
}
XRAPI_ATTR XrResult XRAPI_CALL xrGetVulkanDeviceExtensionsKHR(XrInstance instance,XrSystemId system,
uint32_t capacity,uint32_t *count,char *buffer) {
PFN_xrGetVulkanDeviceExtensionsKHR fn=(PFN_xrGetVulkanDeviceExtensionsKHR)lookup(instance,"xrGetVulkanDeviceExtensionsKHR");
if(!fn)return XR_ERROR_FUNCTION_UNSUPPORTED;
if(!surface_native)return fn(instance,system,capacity,count,buffer);
void *lib=dlopen("libvulkan.so",RTLD_NOW|RTLD_LOCAL);
PFN_vkGetInstanceProcAddr gipa=lib?(PFN_vkGetInstanceProcAddr)dlsym(lib,"vkGetInstanceProcAddr"):NULL;
if(!surf_device_supported(gipa,vk.instance,vk.physical))return fn(instance,system,capacity,count,buffer);
uint32_t original=0;XrResult result=fn(instance,system,0,&original,NULL);
if(XR_FAILED(result) || !count || original>65536)return result;
char *text=calloc(original+1024,1);if(!text)return XR_ERROR_OUT_OF_MEMORY;
result=fn(instance,system,original,&original,text);
if(XR_SUCCEEDED(result)){
for(unsigned i=0;i<sizeof(surf_device_exts)/sizeof(*surf_device_exts);i++){
if(!surf_extension_present(text,surf_device_exts[i])){if(*text)strcat(text," ");strcat(text,surf_device_exts[i]);}
}
*count=(uint32_t)strlen(text)+1;
if(capacity && (capacity<*count || !buffer))result=XR_ERROR_SIZE_INSUFFICIENT;
else if(capacity)memcpy(buffer,text,*count);
}
free(text);return result;
}
XRAPI_ATTR XrResult XRAPI_CALL xrCreateVulkanDeviceKHR(XrInstance instance,const XrVulkanDeviceCreateInfoKHR *info,
VkDevice *device,VkResult *vulkanResult) {
PFN_xrCreateVulkanDeviceKHR fn=(PFN_xrCreateVulkanDeviceKHR)lookup(instance,"xrCreateVulkanDeviceKHR");
if(!fn)return XR_ERROR_FUNCTION_UNSUPPORTED;
if(!surface_native)return fn(instance,info,device,vulkanResult);
if(!info || !info->vulkanCreateInfo || !surf_device_supported(info->pfnGetInstanceProcAddr,
vk.instance,info->vulkanPhysicalDevice))return fn(instance,info,device,vulkanResult);
VkDeviceCreateInfo ci=*info->vulkanCreateInfo;
const char **names=calloc(ci.enabledExtensionCount+8,sizeof(*names));
if(!names)return XR_ERROR_OUT_OF_MEMORY;
for(uint32_t i=0;i<ci.enabledExtensionCount;i++)names[i]=ci.ppEnabledExtensionNames[i];
for(unsigned j=0;j<8;j++){
int found=0;for(uint32_t i=0;i<ci.enabledExtensionCount;i++)found|=!strcmp(names[i],surf_device_exts[j]);
if(!found)names[ci.enabledExtensionCount++]=surf_device_exts[j];
}
ci.ppEnabledExtensionNames=names;
XrVulkanDeviceCreateInfoKHR fixed=*info;fixed.vulkanCreateInfo=&ci;
XrResult result=fn(instance,&fixed,device,vulkanResult);free(names);return result;
}
#include "eye_debug.c"
// OpenXR requires xrGet*GraphicsRequirementsKHR before xrCreateSession; Meta's runtime doesn't enforce it, the Frame's
@@ -938,6 +1082,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEnumerateSwapchainImages(XrSwapchain swapchain,
XrSwapchainImageBaseHeader *images) {
PFN_xrEnumerateSwapchainImages fn = (PFN_xrEnumerateSwapchainImages)lookup(active_instance, "xrEnumerateSwapchainImages");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) return standin_enumerate(swapchain, capacity, count, images);
XrResult result = equirect_emul && emul_is_virtual(swapchain) ? emul_virtual_enumerate(swapchain, capacity, count, images)
: fn(swapchain, capacity, count, images);
if (XR_SUCCEEDED(result) && images && capacity && count) {
@@ -952,6 +1097,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEnumerateSwapchainImages(XrSwapchain swapchain,
XRAPI_ATTR XrResult XRAPI_CALL xrWaitSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageWaitInfo *info) {
PFN_xrWaitSwapchainImage fn = (PFN_xrWaitSwapchainImage)lookup(active_instance, "xrWaitSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) return XR_SUCCESS;
XrResult result = fn(swapchain, info);
__atomic_add_fetch(&sc_waits, 1, __ATOMIC_RELAXED);
if (result != XR_SUCCESS) __atomic_add_fetch(&sc_wait_fails, 1, __ATOMIC_RELAXED);
@@ -961,6 +1107,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrWaitSwapchainImage(XrSwapchain swapchain, const
XRAPI_ATTR XrResult XRAPI_CALL xrReleaseSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageReleaseInfo *info) {
PFN_xrReleaseSwapchainImage fn = (PFN_xrReleaseSwapchainImage)lookup(active_instance, "xrReleaseSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) return XR_SUCCESS;
__atomic_add_fetch(&sc_releases, 1, __ATOMIC_RELAXED);
return fn(swapchain, info);
}
@@ -969,6 +1116,11 @@ XRAPI_ATTR XrResult XRAPI_CALL xrAcquireSwapchainImage(XrSwapchain swapchain, co
uint32_t *index) {
PFN_xrAcquireSwapchainImage fn = (PFN_xrAcquireSwapchainImage)lookup(active_instance, "xrAcquireSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) { // adapter-served cube stand-in: one image
if (!index) return XR_ERROR_VALIDATION_FAILURE;
*index = 0;
return XR_SUCCESS;
}
if (equirect_emul && emul_is_virtual(swapchain)) { // adapter-served write-once picture: one image
if (!index) return XR_ERROR_VALIDATION_FAILURE;
*index = 0;
@@ -985,6 +1137,37 @@ XRAPI_ATTR XrResult XRAPI_CALL xrAcquireSwapchainImage(XrSwapchain swapchain, co
return result;
}
// Copy scene content only while the oldest acquired image is successfully waited
// and still application-owned. Release itself is forwarded immediately.
static XRAPI_ATTR XrResult XRAPI_CALL surf_composite_release(XrSwapchain handle,const XrSwapchainImageReleaseInfo *info) {
pthread_mutex_lock(&flip_lock);
tracked_swapchain *t=find_tracked(handle,0);
if(t && t->acquired_count && t->waited_count)surf_scene_capture(t,t->acquired[0]);
pthread_mutex_unlock(&flip_lock);
XrResult result=equirect_emul?hook_xrReleaseSwapchainImage(handle,info):
((PFN_xrReleaseSwapchainImage)lookup(active_instance,"xrReleaseSwapchainImage"))(handle,info);
if(XR_SUCCEEDED(result)) {
surf_scene_release_commit(handle);
pthread_mutex_lock(&flip_lock);t=find_tracked(handle,0);
if(t && t->acquired_count) {
memmove(t->acquired,t->acquired+1,(--t->acquired_count)*sizeof(t->acquired[0]));
if(t->waited_count)t->waited_count--;
}
pthread_mutex_unlock(&flip_lock);
}
return result;
}
static XRAPI_ATTR XrResult XRAPI_CALL surf_composite_wait(XrSwapchain handle,const XrSwapchainImageWaitInfo *info) {
XrResult result=equirect_emul?hook_xrWaitSwapchainImage(handle,info):
((PFN_xrWaitSwapchainImage)lookup(active_instance,"xrWaitSwapchainImage"))(handle,info);
if(result==XR_SUCCESS) {
pthread_mutex_lock(&flip_lock);tracked_swapchain *t=find_tracked(handle,0);
if(t && t->waited_count<t->acquired_count)t->waited_count++;
pthread_mutex_unlock(&flip_lock);
}
return result;
}
// ---------------------------------------------------------------- tracking diagnostics
static XrSpace view_space_handle = XR_NULL_HANDLE;
@@ -1005,7 +1188,9 @@ XRAPI_ATTR XrResult XRAPI_CALL xrPollEvent(XrInstance instance, XrEventDataBuffe
if (got) return XR_SUCCESS;
}
if (sync_guard) pthread_mutex_lock(&sync_lock);
XrResult result = focus_hold && event ? focus_hold_poll(fn, instance, event) : fn(instance, event);
focus_log_next = fn; // the runtime's own focus changes are logged, before focus_hold hides any
XrResult result = focus_hold && event ? focus_hold_poll(focus_log_poll, instance, event)
: focus_log_poll(instance, event);
if (sync_guard) {
if (result == XR_SUCCESS && event && session_state_of(event) == XR_SESSION_STATE_FOCUSED)
sync_resume_at = monotonic_ns() + SYNC_GUARD_PAUSE_NS;
@@ -1079,6 +1264,10 @@ XRAPI_ATTR XrResult XRAPI_CALL xrWaitFrame(XrSession session, const XrFrameWaitI
// predictedDisplayTime is about one display period ahead of "now".
xr_time_offset = (int64_t)(state->predictedDisplayTime - state->predictedDisplayPeriod) - mono;
xr_time_calibrated = 1;
last_display_period = state->predictedDisplayPeriod;
pose_time_sample(xr_time_offset);
pose_time_display(state->predictedDisplayTime);
pose_time_note_offset((long long)(state->predictedDisplayTime - mono));
if (layer_debug) debug_aim_vs_grip(state->predictedDisplayTime);
}
return result;
@@ -1143,10 +1332,19 @@ XRAPI_ATTR XrResult XRAPI_CALL xrLocateViews(XrSession session, const XrViewLoca
uint32_t capacity, uint32_t *count, XrView *views) {
PFN_xrLocateViews fn = (PFN_xrLocateViews)lookup(active_instance, "xrLocateViews");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
XrViewLocateInfo moved;
if (info && pose_time_fix) {
moved = *info;
moved.displayTime = pose_time_fixed(info->displayTime);
info = &moved;
}
if (info) pose_time_note("xrLocateViews", (uint64_t)(uintptr_t)info->space, 0, info->displayTime);
int cacheable = pose_consistency && info && state && count && views && capacity > 0;
if (cacheable && pose_cache_lookup(session, info, state, capacity, count, views)) return XR_SUCCESS;
XrResult result = fn(session, info, state, capacity, count, views);
if (cacheable && result == XR_SUCCESS) pose_cache_store(session, info, state, *count, views);
if(surface_native && surface_emul && XR_SUCCEEDED(result) && count)
surf_views_record(session,info,state,*count<capacity?*count:capacity,views);
if (equirect_emul && XR_SUCCEEDED(result) && info && count && views && state &&
(state->viewStateFlags & XR_VIEW_STATE_ORIENTATION_VALID_BIT))
emul_on_locate_views(info->space, info->displayTime, *count < capacity ? *count : capacity, views);
@@ -1223,6 +1421,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
frame_open = 0;
__atomic_add_fetch(&frame_ends, 1, __ATOMIC_RELAXED);
snap_end_frame(info);
if(surface_native)++sc_serial;
if (eye_debug) eye_debug_end_frame(session, info);
{ // frame pacing statistics every ~5 s: fps and submitted-vs-predicted display time
static struct timespec start;
@@ -1257,6 +1456,7 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
__atomic_exchange_n(&input_sync_ok, 0, __ATOMIC_RELAXED), input_last_sync_result,
__atomic_exchange_n(&input_bool_reads, 0, __ATOMIC_RELAXED),
__atomic_exchange_n(&input_bool_true, 0, __ATOMIC_RELAXED));
pose_time_report();
frames = 0; drift_sum = drift_max = 0;
}
}
@@ -1300,9 +1500,38 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
XrCompositionLayerProjectionView views[64][2];
XrCompositionLayerQuad quads[64];
uint32_t count = 0, dropped = 0, swapped = 0, passthrough = 0;
const XrCompositionLayerBaseHeader *surface_projections[SURF_MAX];
int surface_projection_count = 0;
for (uint32_t i = 0; i < info->layerCount; ++i) {
const XrCompositionLayerBaseHeader *layer = info->layers[i];
if (!layer) { ++dropped; continue; }
if (emul_is_equirect(layer)) {
const XrCompositionLayerBaseHeader *replacement = surface_native ? surf_projection_frame(session, info, layer) : NULL;
if (replacement) {
int already = 0;
for (int k = 0; k < surface_projection_count; ++k) already |= surface_projections[k] == replacement;
if (!already && surface_projection_count < SURF_MAX) {
surface_projections[surface_projection_count++] = replacement;
// An opaque native hemisphere covering the guarded stereo
// view hides every earlier layer. Cull only that hidden
// prefix: video becomes the primary projection, while
// later subtitles/menus keep their original order. Fades
// and uncovered views use the ordinary layered path.
const XrCompositionLayerBaseHeader *combined=NULL;
if(surface_native && count==1 && kept[0]->type==XR_TYPE_COMPOSITION_LAYER_PROJECTION && surf_composite_candidate(info,layer))
combined=surf_composite_frame(session,info,(const void*)kept[0],replacement);
if(combined) {kept[0]=combined;++swapped;}
else if (surf_projection_occludes(replacement)) {
dropped += count;
count = 0;
passthrough = 0;
}
if(!combined)kept[count++] = replacement;
}
++swapped;
continue;
}
}
if (emulate_passthrough && layer->type == XR_TYPE_COMPOSITION_LAYER_PASSTHROUGH_FB) {
++passthrough; // replaced by the runtime's camera environment below
continue;
@@ -1348,6 +1577,12 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEndFrame(XrSession session, const XrFrameEndInf
++dropped; // an emulated video surface without a frame yet
continue;
}
if (layer_uses_standin(layer)) { // a cube layer whose swapchain the runtime refused (cube_standin.c)
static int logged;
if (logged++ < 3) LOG("cube_standin: dropped a layer (type=%d) that shows a stand-in swapchain", layer->type);
++dropped;
continue;
}
if (!layer || (layer_fix && !layer_usable(layer))) { ++dropped; continue; }
if (layer->type == XR_TYPE_COMPOSITION_LAYER_QUAD) {
const XrCompositionLayerQuad *flipped = flip_quad(session, (const XrCompositionLayerQuad *)layer, &quads[count]);
@@ -1678,6 +1913,14 @@ XRAPI_ATTR XrResult XRAPI_CALL xrEnumerateApiLayerProperties(uint32_t capacity,
// For native/xrshim: emulated functions that overport's dispatcher doesn't know (it never asks us for them).
__attribute__((visibility("default"))) PFN_xrVoidFunction framebridge_extension_proc(const char *name) {
pthread_once(&init_once, initialize);
if(surface_native && name){
#define NATIVE_PROC(fn) if(!strcmp(name,#fn))return (PFN_xrVoidFunction)fn;
NATIVE_PROC(xrGetVulkanGraphicsDeviceKHR)
NATIVE_PROC(xrGetVulkanDeviceExtensionsKHR)
NATIVE_PROC(xrCreateVulkanDeviceKHR)
NATIVE_PROC(xrCreateVulkanInstanceKHR)
#undef NATIVE_PROC
}
return name ? render_model_emulation(name) : NULL;
}
@@ -1727,6 +1970,14 @@ XRAPI_ATTR XrResult XRAPI_CALL xrGetInstanceProcAddr(XrInstance instance, const
HOOK(xrEnumerateInstanceExtensionProperties)
// Per-game hooks: only installed when their setting is on, so other games run through exactly the same calls.
#define HOOK_AS(fn, impl) if (!strcmp(name, #fn)) { *function = (PFN_xrVoidFunction)impl; return XR_SUCCESS; }
if (surface_native) {
HOOK(xrGetVulkanGraphicsDeviceKHR)
HOOK(xrGetVulkanDeviceExtensionsKHR)
HOOK(xrCreateVulkanDeviceKHR)
HOOK(xrCreateVulkanInstanceKHR)
HOOK_AS(xrReleaseSwapchainImage, surf_composite_release)
HOOK_AS(xrWaitSwapchainImage, surf_composite_wait)
}
if (equirect_emul) {
HOOK_AS(xrReleaseSwapchainImage, hook_xrReleaseSwapchainImage)
HOOK_AS(xrWaitSwapchainImage, hook_xrWaitSwapchainImage)
@@ -1738,14 +1989,27 @@ XRAPI_ATTR XrResult XRAPI_CALL xrGetInstanceProcAddr(XrInstance instance, const
}
if (surface_emul) HOOK_AS(xrCreateSwapchainAndroidSurfaceKHR, hook_xrCreateSwapchainAndroidSurfaceKHR)
if (eye_debug || release_wait) HOOK_AS(xrReleaseSwapchainImage, eye_hook_xrReleaseSwapchainImage)
if (layer_debug) { // counters only; every other wait/release hook above takes precedence
if (layer_debug || cube_standin) { // counters + cube stand-ins; every other wait/release hook above takes precedence
HOOK(xrWaitSwapchainImage)
HOOK(xrReleaseSwapchainImage)
}
if (cube_standin && (!strcmp(name, "xrUpdateSwapchainFB") || !strcmp(name, "xrGetSwapchainStateFB"))) {
XrResult result = next_gipa(instance, name, function); // only wrapped where the runtime has it
if (XR_FAILED(result) || !*function) return result;
if (!strcmp(name, "xrUpdateSwapchainFB")) {
real_update_swapchain = (PFN_xrUpdateSwapchainFB)*function;
*function = (PFN_xrVoidFunction)standin_update_swapchain;
} else {
real_get_swapchain_state = (PFN_xrGetSwapchainStateFB)*function;
*function = (PFN_xrVoidFunction)standin_get_swapchain_state;
}
return result;
}
if (sync_guard || layer_debug || input_diag) HOOK_AS(xrSyncActions, hook_xrSyncActions)
if (layer_debug) HOOK_AS(xrGetActionStateBoolean, hook_xrGetActionStateBoolean)
if (layer_debug || aim_correction_on() || profile_remap || input_diag)
if (layer_debug || aim_correction_on() || profile_remap || input_diag || proximity_emul)
HOOK_AS(xrSuggestInteractionProfileBindings, hook_xrSuggestInteractionProfileBindings)
if (proximity_emul) HOOK_AS(xrCreateAction, hook_xrCreateAction)
if (layer_debug || aim_correction_on()) {
HOOK_AS(xrCreateActionSpace, hook_xrCreateActionSpace)
}
+3 -1
View File
@@ -42,7 +42,7 @@ typedef struct { XrStructureType type; void *next; uint32_t image; } emul_gles_i
X(eglChooseConfig) X(eglCreatePbufferSurface) X(eglDestroySurface) X(eglGetCurrentContext) X(eglGetError) \
X(eglQueryString)
#define EMUL_GL_FUNCS(X) X(glFenceSync) X(glDeleteSync) X(glWaitSync) X(glClientWaitSync) X(glFlush) \
X(glGenFramebuffers) X(glBindFramebuffer) X(glFramebufferTexture2D) X(glCheckFramebufferStatus) X(glViewport) \
X(glGenFramebuffers) X(glDeleteFramebuffers) X(glBindFramebuffer) X(glFramebufferTexture2D) X(glCheckFramebufferStatus) X(glViewport) \
X(glUseProgram) X(glCreateShader) X(glShaderSource) X(glCompileShader) X(glGetShaderiv) X(glGetShaderInfoLog) \
X(glDeleteShader) X(glCreateProgram) X(glAttachShader) X(glLinkProgram) X(glGetProgramiv) X(glGetProgramInfoLog) \
X(glGetUniformLocation) X(glUniform1i) X(glUniform1f) X(glUniform3f) X(glUniform4f) X(glUniformMatrix3fv) \
@@ -304,6 +304,7 @@ static void emul_on_destroy_swapchain(XrSwapchain handle) {
static XRAPI_ATTR XrResult XRAPI_CALL hook_xrReleaseSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageReleaseInfo *info) {
PFN_xrReleaseSwapchainImage fn = (PFN_xrReleaseSwapchainImage)lookup(active_instance, "xrReleaseSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) return XR_SUCCESS;
pthread_mutex_lock(&emul_lock);
emul_source *s = emul_active() ? emul_find_source(swapchain, 0) : NULL;
int index = s && s->acquired_count ? (int)s->acquired[0] : -1;
@@ -335,6 +336,7 @@ static XRAPI_ATTR XrResult XRAPI_CALL hook_xrReleaseSwapchainImage(XrSwapchain s
static XRAPI_ATTR XrResult XRAPI_CALL hook_xrWaitSwapchainImage(XrSwapchain swapchain, const XrSwapchainImageWaitInfo *info) {
PFN_xrWaitSwapchainImage fn = (PFN_xrWaitSwapchainImage)lookup(active_instance, "xrWaitSwapchainImage");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
if (is_standin(swapchain)) return XR_SUCCESS;
XrResult result = emul_is_virtual(swapchain) ? XR_SUCCESS : fn(swapchain, info);
pthread_mutex_lock(&emul_lock);
emul_source *s = emul_active() ? emul_find_source(swapchain, 0) : NULL;
+174
View File
@@ -0,0 +1,174 @@
// Included by frame_adapter.c: the times games locate poses at (pose_time_fix, pose_debug).
//
// pose_debug (diagnostics): every ~5 s, per located space (xrLocateSpace/xrLocateSpaces) and for xrLocateViews, the
// number of calls and the min/max of (requested time - the last xrWaitFrame's predictedDisplayTime); for
// xrConvertTimespecTimeToTimeKHR the same for its answers, plus the spread of the XrTime-vs-CLOCK_MONOTONIC offset
// measured at xrWaitFrame.
static XrDuration last_display_period;
#define PT_SLOTS 12
typedef struct {
const char *kind;
uint64_t space, base;
int count;
long long min, max; // ns relative to the last predicted display time
} pt_stat;
static pt_stat pt_stats[PT_SLOTS];
static int pt_overflow;
static long long pt_offset_min, pt_offset_max;
static int pt_offset_count;
static pthread_mutex_t pt_lock = PTHREAD_MUTEX_INITIALIZER;
static void pose_time_note(const char *kind, uint64_t space, uint64_t base, XrTime time) {
if (!pose_debug || !last_predicted_time) return;
long long d = (long long)(time - last_predicted_time);
pthread_mutex_lock(&pt_lock);
int i = 0;
while (i < PT_SLOTS && pt_stats[i].count &&
!(pt_stats[i].kind == kind && pt_stats[i].space == space && pt_stats[i].base == base))
++i;
if (i == PT_SLOTS) {
++pt_overflow;
} else {
pt_stat *s = &pt_stats[i];
if (!s->count) s->kind = kind, s->space = space, s->base = base, s->min = s->max = d;
if (d < s->min) s->min = d;
if (d > s->max) s->max = d;
++s->count;
}
pthread_mutex_unlock(&pt_lock);
}
// The offset sampled at xrWaitFrame dips (by up to 2.5 s) whenever xrWaitFrame returns late after a hitch, which put
// corrected times up to 2 s in the future (headless survey, 2026-10-09). The steady value is the largest recent sample:
// pose_time_fix uses the maximum of the last PT_RING samples (~2 s at 72 Hz; the offset drifts only ~0.14 s per hour).
#define PT_RING 144
static int64_t pt_ring[PT_RING];
static int pt_ring_n, pt_ring_i;
static void pose_time_sample(int64_t offset) { // XrTime "now" minus CLOCK_MONOTONIC, from xrWaitFrame
pthread_mutex_lock(&pt_lock);
pt_ring[pt_ring_i] = offset;
pt_ring_i = (pt_ring_i + 1) % PT_RING;
if (pt_ring_n < PT_RING) ++pt_ring_n;
pthread_mutex_unlock(&pt_lock);
}
// The last few predicted display times (xrWaitFrame): a pose asked for at one of them is the game rendering a frame
// and is never moved. On a Frame whose XrTime runs only ~70 ms ahead of the monotonic clock (GitHub #49), such a
// request could look nearer the monotonic "now" than XrTime's and was moved 68 ms into the future.
#define PT_DISPLAY_RING 6
#define PT_DISPLAY_MATCH 1000000ll // 1 ms
static XrTime pt_display[PT_DISPLAY_RING];
static int pt_display_i;
static void pose_time_display(XrTime predicted) { // from xrWaitFrame
pthread_mutex_lock(&pt_lock);
pt_display[pt_display_i] = predicted;
pt_display_i = (pt_display_i + 1) % PT_DISPLAY_RING;
pthread_mutex_unlock(&pt_lock);
}
static int pose_time_is_display(XrTime time) {
int found = 0;
pthread_mutex_lock(&pt_lock);
for (int i = 0; i < PT_DISPLAY_RING && !found; i++) {
long long d = (long long)(time - pt_display[i]);
found = pt_display[i] && d <= PT_DISPLAY_MATCH && d >= -PT_DISPLAY_MATCH;
}
pthread_mutex_unlock(&pt_lock);
return found;
}
static int64_t pose_time_offset(void) {
pthread_mutex_lock(&pt_lock);
int64_t best = pt_ring_n ? pt_ring[0] : 0;
for (int i = 1; i < pt_ring_n; i++)
if (pt_ring[i] > best) best = pt_ring[i];
pthread_mutex_unlock(&pt_lock);
return best;
}
static void pose_time_note_offset(long long offset) {
if (!pose_debug) return;
pthread_mutex_lock(&pt_lock);
if (!pt_offset_count || offset < pt_offset_min) pt_offset_min = offset;
if (!pt_offset_count || offset > pt_offset_max) pt_offset_max = offset;
++pt_offset_count;
pthread_mutex_unlock(&pt_lock);
}
// pose_time_fix: Meta's OVRPlugin takes "now" from CLOCK_MONOTONIC and passes it on as an XrTime unchanged (no
// conversion reaches FrameBridge: OVRPort's dispatcher answers XR_KHR_convert_timespec_time itself, 1:1, since the
// Frame's runtime lacks it). On a Quest XrTime is the monotonic clock; on the Frame it runs ahead (2.56 s measured on
// SteamOS 0.4.5, 0.05-0.9 s seen on older builds), so every "now" pose (Unity's physics step, OVRInput's controller
// poses) was located that far in the past and the hands trailed the controllers (BattleSisters, Sniper Elite VR).
// A located time nearer the monotonic clock than XrTime's "now" is taken as a monotonic timestamp and moved to the
// same moment in XrTime (offset measured at xrWaitFrame); a time more than PT_PAST_LIMIT before the predicted display
// time that isn't one (BattleSisters asks for its head at XrTime 0.1 s; Robo Recall, Phantom, Vader, Time Stall and
// The Room VR at ~0 every frame) is located at the predicted display time, the time the game renders with.
// A monotonic timestamp is only told apart when the clocks are more than PT_MONO_PERIODS display periods apart, and the
// time must be clearly nearer the monotonic "now" (at most a third of its distance to XrTime's "now"); a time equal to
// a recent predicted display time is never moved (GitHub #49: XrTime only ~68 ms ahead, display-time requests moved).
#define PT_PAST_LIMIT 500000000ll // 0.5 s: no runtime keeps a longer pose history; real past queries are far shorter
#define PT_MIN_OFFSET 5000000ll // XrTime and the monotonic clock this close: nothing to tell apart or to fix
#define PT_MONO_PERIODS 4
#define PT_DEFAULT_PERIOD 13888889ll // 72 Hz, before xrWaitFrame reported a period
static int pt_fixed_mono, pt_fixed_past;
static long long pt_fixed_min;
static XrTime pose_time_fixed(XrTime time) {
if (!pose_time_fix || !last_predicted_time || pt_ring_n < 8) return time; // a few frames of offset samples first
int64_t offset = pose_time_offset(); // XrTime "now" minus the monotonic clock (robust, see above)
if (offset < PT_MIN_OFFSET && offset > -PT_MIN_OFFSET) return time;
if (pose_time_is_display(time)) return time; // the frame (or a recent one) the game renders
long long period = last_display_period > 0 ? (long long)last_display_period : PT_DEFAULT_PERIOD;
long long abs_offset = offset < 0 ? -offset : offset;
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
long long mono = (long long)ts.tv_sec * 1000000000ll + ts.tv_nsec;
long long d_mono = (long long)time - mono, d_xr = (long long)time - (mono + offset);
long long abs_mono = d_mono < 0 ? -d_mono : d_mono, abs_xr = d_xr < 0 ? -d_xr : d_xr;
XrTime fixed;
int kind;
if (abs_offset > PT_MONO_PERIODS * period && abs_mono < PT_PAST_LIMIT && 3 * abs_mono < abs_xr) {
fixed = (XrTime)(time + offset), kind = 0; // a monotonic timestamp
} else if ((long long)(time - last_predicted_time) < -PT_PAST_LIMIT) {
fixed = last_predicted_time, kind = 1; // far in the past: the frame's display time
} else {
return time;
}
if (pose_debug) {
long long d = (long long)(time - last_predicted_time);
pthread_mutex_lock(&pt_lock);
if (!(pt_fixed_mono + pt_fixed_past) || d < pt_fixed_min) pt_fixed_min = d;
++*(kind ? &pt_fixed_past : &pt_fixed_mono);
pthread_mutex_unlock(&pt_lock);
}
static int logged[2];
if (logged[kind]++ < 3)
LOG("pose_time_fix: pose asked for at %.1f ms from the predicted display time (%s) located at %.1f ms",
(long long)(time - last_predicted_time) / 1e6, kind ? "far in the past" : "a monotonic-clock time",
(long long)(fixed - last_predicted_time) / 1e6);
return fixed;
}
// called from the pacing statistics in xrEndFrame
static void pose_time_report(void) {
if (!pose_debug) return;
pthread_mutex_lock(&pt_lock);
LOG("pose_debug: period %.2f ms; XrTime - monotonic at xrWaitFrame (%d): %.2f..%.2f ms", last_display_period / 1e6,
pt_offset_count, pt_offset_min / 1e6, pt_offset_max / 1e6);
for (int i = 0; i < PT_SLOTS && pt_stats[i].count; ++i)
LOG("pose_debug: %s space=0x%llx base=0x%llx: %d calls, time - predicted display time %.2f..%.2f ms",
pt_stats[i].kind, (unsigned long long)pt_stats[i].space, (unsigned long long)pt_stats[i].base,
pt_stats[i].count, pt_stats[i].min / 1e6, pt_stats[i].max / 1e6);
if (pt_overflow) LOG("pose_debug: %d calls not counted (more than %d spaces)", pt_overflow, PT_SLOTS);
if (pt_fixed_mono + pt_fixed_past)
LOG("pose_debug: pose_time_fix moved %d monotonic-clock times and %d far-past times (earliest asked %.2f ms)",
pt_fixed_mono, pt_fixed_past, pt_fixed_min / 1e6);
memset(pt_stats, 0, sizeof(pt_stats));
pt_overflow = pt_offset_count = pt_fixed_mono = pt_fixed_past = 0;
pthread_mutex_unlock(&pt_lock);
}
+124 -5
View File
@@ -163,6 +163,39 @@ static void focus_hold_note(int state) {
else if (state >= 0 && state != XR_SESSION_STATE_FOCUSED) is_focused = 0;
}
// ---------------------------------------------------------------- focus log (always on: rare events, cheap)
// Every time the runtime takes focus away and gives it back, as the runtime reports it (before focus_hold hides a dip
// from the app): FramePort's session triage counts these dips after a play session and suggests a longer
// focus_hold_ms when they keep reaching the game.
static PFN_xrPollEvent focus_log_next;
static int focus_log_focused;
static int64_t focus_log_lost_at;
static int focus_log_lost_state;
static XRAPI_ATTR XrResult XRAPI_CALL focus_log_poll(XrInstance instance, XrEventDataBuffer *event) {
XrResult r = focus_log_next(instance, event);
if (r != XR_SUCCESS || !event) return r;
int state = session_state_of(event);
if (state < 0) return r;
if (state == XR_SESSION_STATE_FOCUSED) {
if (focus_log_lost_at)
LOG("focus: back after %.0f ms (lost to state %d)", (monotonic_ns() - focus_log_lost_at) / 1e6,
focus_log_lost_state);
focus_log_focused = 1;
focus_log_lost_at = 0;
} else if (focus_log_focused && (state == XR_SESSION_STATE_VISIBLE || state == XR_SESSION_STATE_SYNCHRONIZED)) {
focus_log_focused = 0;
focus_log_lost_at = monotonic_ns();
focus_log_lost_state = state;
LOG("focus: lost (state %d)", state);
} else if (state != XR_SESSION_STATE_VISIBLE && state != XR_SESSION_STATE_SYNCHRONIZED) {
focus_log_focused = 0; // stopping, loss pending, exiting: the session ends, not a dip
if (focus_log_lost_at) LOG("focus: session ending (state %d)", state);
focus_log_lost_at = 0;
}
return r;
}
static XrResult focus_hold_poll(PFN_xrPollEvent fn, XrInstance instance, XrEventDataBuffer *event) {
for (;;) {
if (held.delivering) { // hand held events to the app, oldest first
@@ -285,6 +318,90 @@ static int touch_has(const char *path) {
return 0;
}
// ---------------------------------------------------------------- finger proximity from touch (proximity_emul)
// Meta's OVRPlugin reports a thumb or index finger resting near the controller through XR_FB_touch_controller_proximity
// (.../thumb_fb/proximity_fb, .../trigger/proximity_fb). The Frame's runtime doesn't have it; OVRPort's loader offers
// it to the game anyway, and the proximity actions never get a working binding: "near" stays false, so games that
// animate the hands from it show a thumb that never moves (Unreal's ThumbUp axis is "no thumb proximity", e.g. Vader
// Immortal; found by Klownicle, GitHub #49). With proximity_emul those actions are bound to the capacitive touch inputs
// instead (OpenXR combines several bindings of one boolean action with OR): thumb = thumbstick, face buttons or thumb
// rest touched; index (proximity_emul=2) = trigger touched. Only when the runtime lacks the extension itself.
static XrAction thumb_proximity_action, trigger_proximity_action;
static XRAPI_ATTR XrResult XRAPI_CALL hook_xrCreateAction(XrActionSet set, const XrActionCreateInfo *info,
XrAction *action) {
PFN_xrCreateAction fn = (PFN_xrCreateAction)lookup(active_instance, "xrCreateAction");
if (!fn) return XR_ERROR_FUNCTION_UNSUPPORTED;
XrResult r = fn(set, info, action);
if (XR_SUCCEEDED(r) && info && action) { // OVRPlugin's action names
if (!strcmp(info->actionName, "hand_thumb_proximity")) thumb_proximity_action = *action;
else if (!strcmp(info->actionName, "hand_trigger_proximity")) trigger_proximity_action = *action;
else return r;
LOG("proximity_emul: game action %s found", info->actionName);
}
return r;
}
static const char *const THUMB_TOUCH_LEFT[] = {"x/touch", "y/touch", "thumbstick/touch", "thumbrest/touch"};
static const char *const THUMB_TOUCH_RIGHT[] = {"a/touch", "b/touch", "thumbstick/touch", "thumbrest/touch"};
static int add_binding(XrInstance instance, PFN_xrStringToPath to_path, XrActionSuggestedBinding *b, uint32_t *n,
XrAction action, const char *hand, const char *input) {
char path[XR_MAX_PATH_LENGTH];
snprintf(path, sizeof(path), "/user/hand/%s/input/%s", hand, input);
XrPath p;
if (XR_FAILED(to_path(instance, path, &p))) return 0;
for (uint32_t i = 0; i < *n; ++i)
if (b[i].action == action && b[i].binding == p) return 0;
b[*n].action = action;
b[(*n)++].binding = p;
return 1;
}
// Suggests Touch bindings with finger proximity bound to the touch inputs (proximity_emul), falling back to the
// bindings as given when the runtime refuses them.
static XrResult suggest_touch(XrInstance instance, PFN_xrSuggestInteractionProfileBindings fn,
PFN_xrPathToString to_string, const XrInteractionProfileSuggestedBinding *suggested) {
PFN_xrStringToPath to_path = (PFN_xrStringToPath)lookup(instance, "xrStringToPath");
if (!proximity_emul || runtime_has_proximity || !to_path || !to_string) return fn(instance, suggested);
XrAction thumb = thumb_proximity_action, index = proximity_emul > 1 ? trigger_proximity_action : XR_NULL_HANDLE;
uint32_t n = suggested->countSuggestedBindings, kept = 0, dropped = 0, added = 0;
XrActionSuggestedBinding *b = calloc(n + 2 * 5, sizeof(*b));
if (!b) return fn(instance, suggested);
for (uint32_t i = 0; i < n; ++i) {
char path[XR_MAX_PATH_LENGTH];
uint32_t size = 0;
const XrActionSuggestedBinding *s = &suggested->suggestedBindings[i];
if (XR_SUCCEEDED(to_string(instance, s->binding, sizeof(path), &size, path)) && strstr(path, "/proximity_")) {
// the game's own proximity binding, which the runtime doesn't know: its action is bound to touch below
if (strstr(path, "/thumb")) { if (!thumb) thumb = s->action; }
else if (proximity_emul > 1 && !index) index = s->action;
++dropped;
continue;
}
b[kept++] = *s;
}
for (int hand = 0; hand < 2; ++hand) {
const char *name = hand ? "right" : "left";
const char *const *thumb_inputs = hand ? THUMB_TOUCH_RIGHT : THUMB_TOUCH_LEFT;
for (int c = 0; thumb && c < 4; ++c) added += add_binding(instance, to_path, b, &kept, thumb, name, thumb_inputs[c]);
if (index) added += add_binding(instance, to_path, b, &kept, index, name, "trigger/touch");
}
XrResult r;
if (!added && !dropped) r = fn(instance, suggested);
else {
XrInteractionProfileSuggestedBinding copy = *suggested;
copy.countSuggestedBindings = kept;
copy.suggestedBindings = b;
r = fn(instance, &copy);
LOG("proximity_emul: finger proximity bound to touch (%u binding(s) added, %u proximity path(s) replaced): %d",
added, dropped, r);
if (XR_FAILED(r)) r = fn(instance, suggested);
}
free(b);
return r;
}
static XrResult remap_to_touch(XrInstance instance, PFN_xrSuggestInteractionProfileBindings fn,
PFN_xrPathToString to_string, const XrInteractionProfileSuggestedBinding *suggested,
const char *profile) {
@@ -305,7 +422,7 @@ static XrResult remap_to_touch(XrInstance instance, PFN_xrSuggestInteractionProf
copy.interactionProfile = touch;
copy.countSuggestedBindings = kept;
copy.suggestedBindings = b;
XrResult r = kept ? fn(instance, &copy) : XR_ERROR_PATH_UNSUPPORTED;
XrResult r = kept ? suggest_touch(instance, fn, to_string, &copy) : XR_ERROR_PATH_UNSUPPORTED;
free(b);
LOG("controller profile %s -> oculus/touch_controller (%u bindings, %u dropped): %d", profile + 22, kept,
n - kept, r);
@@ -325,12 +442,14 @@ static XRAPI_ATTR XrResult XRAPI_CALL hook_xrSuggestInteractionProfileBindings(X
if (XR_SUCCEEDED(to_string(instance, suggested->suggestedBindings[i].binding, sizeof(path), &size, path)))
note_pose_binding(suggested->suggestedBindings[i].action, path);
}
XrResult r = fn(instance, suggested);
input_diag_suggested(instance, suggested, r, to_string); // the runtime's answer to the game's own suggestion
if (!profile_remap || !suggested || !to_string) return r;
char profile[XR_MAX_PATH_LENGTH] = {0};
uint32_t size = 0;
if (XR_FAILED(to_string(instance, suggested->interactionProfile, sizeof(profile), &size, profile))) return r;
int known = suggested && to_string &&
XR_SUCCEEDED(to_string(instance, suggested->interactionProfile, sizeof(profile), &size, profile));
XrResult r = known && !strcmp(profile, TOUCH_PROFILE) ? suggest_touch(instance, fn, to_string, suggested)
: fn(instance, suggested);
input_diag_suggested(instance, suggested, r, to_string); // the runtime's answer to the game's own suggestion
if (!profile_remap || !known) return r;
if (!strcmp(profile, TOUCH_PROFILE)) {
if (XR_SUCCEEDED(r)) app_suggested_touch = 1; // the app's own Touch bindings win over a remap
return r;
+383
View File
@@ -0,0 +1,383 @@
// SPDX-License-Identifier: GPL-3.0-only
// Included after surface_projection.c. Snapshots scene images BEFORE release,
// then composes decoded video + scene into one primary projection on the GPU.
#include "surface_composite.h"
#include "surface_composite_spv.h"
#define SC_SCENES 4
typedef struct {
XrSwapchain handle;
VkImage image;
VkImageView view;
VkDeviceMemory memory;
VkCommandBuffer commands[MAX_IMAGES];
VkFence fences[MAX_IMAGES];
int pending[MAX_IMAGES], ready;
uint64_t serial,captured_release,releases;
uint32_t width,height,layers;
} surf_scene_snapshot;
static surf_scene_snapshot sc_scenes[SC_SCENES];
static uint64_t sc_serial;
static int64_t sc_capture_until;
// Unity attaches both structures even when their transforms are identity.
// Apply supported transforms in our shader; reject other chains rather than
// silently discarding semantics when flattening the original scene.
static int sc_scene_supported(const XrCompositionLayerProjection *scene) {
int reason=0;
if(!scene || scene->viewCount!=2 || !scene->views)return 0;
int count=0;
for(const XrBaseInStructure *n=scene->next;n;n=n->next) {
if(++count>16){reason=-1;break;}
if(n->type==XR_TYPE_COMPOSITION_LAYER_COLOR_SCALE_BIAS_KHR)continue;
if(n->type==XR_TYPE_COMPOSITION_LAYER_IMAGE_LAYOUT_FB) {
const XrCompositionLayerImageLayoutFB *layout=(const void*)n;
if(!(layout->flags&~XR_COMPOSITION_LAYER_IMAGE_LAYOUT_VERTICAL_FLIP_BIT_FB))continue;
}
reason=n->type;break;
}
for(int e=0;e<2 && !reason;e++)if(scene->views[e].next)
reason=((const XrBaseInStructure*)scene->views[e].next)->type;
static int previous;
if(reason && reason!=previous)LOG("surface_composite: unsupported scene extension %d; retaining ordinary composition",reason);
previous=reason;return !reason;
}
typedef struct {
XrSwapchain output;
VkImage images[MAX_IMAGES];
VkImageView views[MAX_IMAGES];
VkFramebuffer framebuffers[MAX_IMAGES];
VkRenderPass render_pass;
VkSampler sampler;
VkDescriptorPool pool;
VkDescriptorSetLayout set_layout;
VkDescriptorSet sets[MAX_IMAGES][2];
VkPipelineLayout layout;
VkPipeline pipeline;
VkCommandBuffer commands[MAX_IMAGES];
VkFence fences[MAX_IMAGES];
int pending[MAX_IMAGES], ready, acquired, failed;
uint32_t count,index;
XrTime time;
XrCompositionLayerProjection layer;
XrCompositionLayerProjectionView eyes[2];
int64_t log_time;
uint64_t frames,holds;
} surf_composite;
static surf_composite sc_outputs[SURF_MAX];
static void sc_barrier(VkCommandBuffer cmd,VkImage image,uint32_t layers,VkImageLayout from,VkImageLayout to,
VkAccessFlags src,VkAccessFlags dst) {
VkImageMemoryBarrier b={VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,NULL,src,dst,from,to,
VK_QUEUE_FAMILY_IGNORED,VK_QUEUE_FAMILY_IGNORED,image,{VK_IMAGE_ASPECT_COLOR_BIT,0,1,0,layers}};
vk.CmdPipelineBarrier(cmd,VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
0,0,NULL,0,NULL,1,&b);
}
static int sc_command(VkCommandBuffer *cmd,VkFence *fence) {
VkCommandBufferAllocateInfo ai={VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO,NULL,vk.pool,VK_COMMAND_BUFFER_LEVEL_PRIMARY,1};
VkFenceCreateInfo fi={VK_STRUCTURE_TYPE_FENCE_CREATE_INFO};
if(vk.AllocateCommandBuffers(vk.device,&ai,cmd)!=VK_SUCCESS)return 0;
if(vk.CreateFence(vk.device,&fi,NULL,fence)==VK_SUCCESS)return 1;
PFN_vkFreeCommandBuffers free_cmd=(PFN_vkFreeCommandBuffers)vk.GetDeviceProcAddr(vk.device,"vkFreeCommandBuffers");
free_cmd(vk.device,vk.pool,1,cmd);*cmd=VK_NULL_HANDLE;return 0;
}
static int sc_reuse(VkFence fence,int *pending) {
if(!*pending)return 1;
if(svk.GetFenceStatus(vk.device,fence)!=VK_SUCCESS)return 0;
if(vk.ResetFences(vk.device,1,&fence)!=VK_SUCCESS)return 0;
*pending=0;return 1;
}
static void sc_retire(VkCommandBuffer *commands,VkFence *fences,int *pending,uint32_t count) {
PFN_vkDestroyFence destroy=(PFN_vkDestroyFence)vk.GetDeviceProcAddr(vk.device,"vkDestroyFence");
PFN_vkFreeCommandBuffers free_cmd=(PFN_vkFreeCommandBuffers)vk.GetDeviceProcAddr(vk.device,"vkFreeCommandBuffers");
for(uint32_t i=0;i<count;i++) {
if(pending[i])vk.WaitForFences(vk.device,1,&fences[i],VK_TRUE,UINT64_MAX);
if(fences[i])destroy(vk.device,fences[i],NULL);
if(commands[i])free_cmd(vk.device,vk.pool,1,&commands[i]);
}
}
static void surf_scene_destroy(XrSwapchain handle) {
if(!handle || spvk.ready<=0)return;
// A compositor command can still be sampling this mirror after its
// snapshot copy completes. Retire those readers before destroying a view.
for(int k=0;k<SURF_MAX;k++)for(uint32_t j=0;j<sc_outputs[k].count;j++)
if(sc_outputs[k].pending[j])vk.WaitForFences(vk.device,1,&sc_outputs[k].fences[j],VK_TRUE,UINT64_MAX);
for(int i=0;i<SC_SCENES;i++)if(sc_scenes[i].handle==handle) {
surf_scene_snapshot *s=&sc_scenes[i];
sc_retire(s->commands,s->fences,s->pending,MAX_IMAGES);
if(s->view)spvk.DestroyImageView(vk.device,s->view,NULL);
if(s->image)spvk.DestroyImage(vk.device,s->image,NULL);
if(s->memory)spvk.FreeMemory(vk.device,s->memory,NULL);
memset(s,0,sizeof(*s));
}
}
static void surf_composite_destroy(surf_projection *p) {
surf_composite *c=&sc_outputs[p-surf_projections];
sc_retire(c->commands,c->fences,c->pending,c->count);
if(c->pipeline)spvk.DestroyPipeline(vk.device,c->pipeline,NULL);
if(c->layout)spvk.DestroyPipelineLayout(vk.device,c->layout,NULL);
if(c->pool)spvk.DestroyDescriptorPool(vk.device,c->pool,NULL);
if(c->set_layout)spvk.DestroyDescriptorSetLayout(vk.device,c->set_layout,NULL);
if(c->sampler)spvk.DestroySampler(vk.device,c->sampler,NULL);
for(uint32_t i=0;i<c->count;i++) {
if(c->framebuffers[i])spvk.DestroyFramebuffer(vk.device,c->framebuffers[i],NULL);
if(c->views[i])spvk.DestroyImageView(vk.device,c->views[i],NULL);
}
if(c->render_pass)spvk.DestroyRenderPass(vk.device,c->render_pass,NULL);
if(c->output) {
PFN_xrDestroySwapchain fn=(PFN_xrDestroySwapchain)lookup(active_instance,"xrDestroySwapchain");
if(c->acquired) {
PFN_xrWaitSwapchainImage wait=(PFN_xrWaitSwapchainImage)lookup(active_instance,"xrWaitSwapchainImage");
PFN_xrReleaseSwapchainImage release=(PFN_xrReleaseSwapchainImage)lookup(active_instance,"xrReleaseSwapchainImage");
XrSwapchainImageWaitInfo wi={XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO,NULL,XR_INFINITE_DURATION};
XrSwapchainImageReleaseInfo ri={XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
if(wait(c->output,&wi)==XR_SUCCESS)release(c->output,&ri);
}
if(fn)fn(c->output);
}
memset(c,0,sizeof(*c));
}
// Called while the application still owns this successfully waited image.
// Queue ordering includes the application's rendering and this snapshot copy;
// no CPU readback, post-release access, or deferred OpenXR release is needed.
static void surf_scene_capture(tracked_swapchain *t,uint32_t index) {
if(!surface_native || monotonic_ns()>sc_capture_until || !t || index>=t->image_count ||
spvk.ready<=0 || t->info.sampleCount!=1 || t->info.arraySize!=2 ||
!(t->info.usageFlags&XR_SWAPCHAIN_USAGE_TRANSFER_SRC_BIT) ||
!(t->info.usageFlags&XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT))return;
surf_scene_snapshot *s=NULL;
for(int i=0;i<SC_SCENES;i++)if(sc_scenes[i].handle==t->handle){s=&sc_scenes[i];break;}
if(!s)for(int i=0;i<SC_SCENES;i++)if(!sc_scenes[i].handle){s=&sc_scenes[i];s->handle=t->handle;break;}
if(!s)return;
if(!s->view) {
s->width=t->info.width;s->height=t->info.height;s->layers=t->info.arraySize;
if(!surf_private_image(s->width,s->height,s->layers,(VkFormat)t->info.format,
VK_IMAGE_USAGE_TRANSFER_DST_BIT|VK_IMAGE_USAGE_SAMPLED_BIT,&s->image,&s->memory,&s->view)) {
surf_scene_destroy(t->handle);return;
}
}
if(!s->commands[index] && !sc_command(&s->commands[index],&s->fences[index]))return;
if(!sc_reuse(s->fences[index],&s->pending[index]))return;
VkCommandBuffer cmd=s->commands[index];
VkCommandBufferBeginInfo begin={VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,NULL,VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
if(vk.ResetCommandBuffer(cmd,0)!=VK_SUCCESS || vk.BeginCommandBuffer(cmd,&begin)!=VK_SUCCESS)return;
sc_barrier(cmd,t->images[index],s->layers,VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT,VK_ACCESS_TRANSFER_READ_BIT);
sc_barrier(cmd,s->image,s->layers,s->ready?VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL:VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,s->ready?VK_ACCESS_SHADER_READ_BIT:0,VK_ACCESS_TRANSFER_WRITE_BIT);
VkImageCopy copy={{VK_IMAGE_ASPECT_COLOR_BIT,0,0,s->layers},{0,0,0},
{VK_IMAGE_ASPECT_COLOR_BIT,0,0,s->layers},{0,0,0},{s->width,s->height,1}};
spvk.CmdCopyImage(cmd,t->images[index],VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,s->image,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,1,&copy);
sc_barrier(cmd,s->image,s->layers,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
VK_ACCESS_TRANSFER_WRITE_BIT,VK_ACCESS_SHADER_READ_BIT);
sc_barrier(cmd,t->images[index],s->layers,VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ACCESS_TRANSFER_READ_BIT,VK_ACCESS_COLOR_ATTACHMENT_READ_BIT|VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT);
if(vk.EndCommandBuffer(cmd)!=VK_SUCCESS)return;
VkSubmitInfo submit={VK_STRUCTURE_TYPE_SUBMIT_INFO,NULL,0,NULL,NULL,1,&cmd};
if(vk.QueueSubmit(vk.queue,1,&submit,s->fences[index])!=VK_SUCCESS)return;
s->pending[index]=1;s->ready=1;s->serial=sc_serial+1;s->captured_release=s->releases+1;
}
static int sc_setup(XrSession session,surf_projection *p,surf_composite *c) {
if(c->pipeline)return 1;
if(c->failed)return 0;
PFN_xrCreateSwapchain create=(PFN_xrCreateSwapchain)lookup(active_instance,"xrCreateSwapchain");
PFN_xrEnumerateSwapchainImages enumerate=(PFN_xrEnumerateSwapchainImages)lookup(active_instance,"xrEnumerateSwapchainImages");
XrSwapchainCreateInfo ci={XR_TYPE_SWAPCHAIN_CREATE_INFO,NULL,0,
XR_SWAPCHAIN_USAGE_TRANSFER_DST_BIT|XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT,
VK_FORMAT_R8G8B8A8_SRGB,1,2*p->size,p->size,1,1,1};
if(!create || !enumerate || XR_FAILED(create(session,&ci,&c->output)))goto fail;
XrSwapchainImageVulkanKHR images[MAX_IMAGES];
for(int i=0;i<MAX_IMAGES;i++)images[i]=(XrSwapchainImageVulkanKHR){XR_TYPE_SWAPCHAIN_IMAGE_VULKAN_KHR};
if(XR_FAILED(enumerate(c->output,MAX_IMAGES,&c->count,(void*)images)) || !c->count || c->count>MAX_IMAGES) {
c->count=0;goto fail;
}
for(uint32_t i=0;i<c->count;i++) {
c->images[i]=images[i].image;
if(!sc_command(&c->commands[i],&c->fences[i]))goto fail;
}
VkAttachmentDescription attachment={0,VK_FORMAT_R8G8B8A8_SRGB,VK_SAMPLE_COUNT_1_BIT,
VK_ATTACHMENT_LOAD_OP_DONT_CARE,VK_ATTACHMENT_STORE_OP_STORE,VK_ATTACHMENT_LOAD_OP_DONT_CARE,
VK_ATTACHMENT_STORE_OP_DONT_CARE,VK_IMAGE_LAYOUT_UNDEFINED,VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL};
VkAttachmentReference reference={0,VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL};
VkSubpassDescription subpass={0,VK_PIPELINE_BIND_POINT_GRAPHICS,0,NULL,1,&reference};
VkSubpassDependency dependency={VK_SUBPASS_EXTERNAL,0,VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT|VK_PIPELINE_STAGE_COLOR_ATTACHMENT_OUTPUT_BIT,
VK_ACCESS_MEMORY_WRITE_BIT|VK_ACCESS_MEMORY_READ_BIT,VK_ACCESS_SHADER_READ_BIT|VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT};
VkRenderPassCreateInfo rpi={VK_STRUCTURE_TYPE_RENDER_PASS_CREATE_INFO,NULL,0,1,&attachment,1,&subpass,1,&dependency};
if(spvk.CreateRenderPass(vk.device,&rpi,NULL,&c->render_pass)!=VK_SUCCESS)goto fail;
for(uint32_t i=0;i<c->count;i++) {
VkImageViewCreateInfo vi={VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,NULL,0,c->images[i],VK_IMAGE_VIEW_TYPE_2D,
VK_FORMAT_R8G8B8A8_SRGB,{0,0,0,0},{VK_IMAGE_ASPECT_COLOR_BIT,0,1,0,1}};
if(spvk.CreateImageView(vk.device,&vi,NULL,&c->views[i])!=VK_SUCCESS)goto fail;
VkFramebufferCreateInfo fi={VK_STRUCTURE_TYPE_FRAMEBUFFER_CREATE_INFO,NULL,0,c->render_pass,1,&c->views[i],2*p->size,p->size,1};
if(spvk.CreateFramebuffer(vk.device,&fi,NULL,&c->framebuffers[i])!=VK_SUCCESS)goto fail;
}
VkSamplerCreateInfo si={VK_STRUCTURE_TYPE_SAMPLER_CREATE_INFO,NULL,0,VK_FILTER_LINEAR,VK_FILTER_LINEAR,
VK_SAMPLER_MIPMAP_MODE_NEAREST,VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE,VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE,
VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE};
if(spvk.CreateSampler(vk.device,&si,NULL,&c->sampler)!=VK_SUCCESS)goto fail;
VkDescriptorSetLayoutBinding bindings[2]={{0,VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,1,VK_SHADER_STAGE_FRAGMENT_BIT},
{1,VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,1,VK_SHADER_STAGE_FRAGMENT_BIT}};
VkDescriptorSetLayoutCreateInfo li={VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO,NULL,0,2,bindings};
if(spvk.CreateDescriptorSetLayout(vk.device,&li,NULL,&c->set_layout)!=VK_SUCCESS)goto fail;
VkDescriptorPoolSize sizes[1]={{VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,4*c->count}};
VkDescriptorPoolCreateInfo pi={VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO,NULL,0,2*c->count,1,sizes};
if(spvk.CreateDescriptorPool(vk.device,&pi,NULL,&c->pool)!=VK_SUCCESS)goto fail;
for(uint32_t i=0;i<c->count;i++)for(int e=0;e<2;e++) {
VkDescriptorSetAllocateInfo ai={VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO,NULL,c->pool,1,&c->set_layout};
if(spvk.AllocateDescriptorSets(vk.device,&ai,&c->sets[i][e])!=VK_SUCCESS)goto fail;
}
VkPushConstantRange range={VK_SHADER_STAGE_FRAGMENT_BIT,0,sizeof(surf_composite_params)};
VkPipelineLayoutCreateInfo pli={VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO,NULL,0,1,&c->set_layout,1,&range};
if(spvk.CreatePipelineLayout(vk.device,&pli,NULL,&c->layout)!=VK_SUCCESS)goto fail;
VkShaderModule modules[2]={0};
VkShaderModuleCreateInfo mi={VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO,NULL,0,sizeof(surface_composite_vert_spv),surface_composite_vert_spv};
if(spvk.CreateShaderModule(vk.device,&mi,NULL,&modules[0])!=VK_SUCCESS)goto fail;
mi.codeSize=sizeof(surface_composite_frag_spv);mi.pCode=surface_composite_frag_spv;
if(spvk.CreateShaderModule(vk.device,&mi,NULL,&modules[1])!=VK_SUCCESS) {
spvk.DestroyShaderModule(vk.device,modules[0],NULL);goto fail;
}
VkPipelineShaderStageCreateInfo stages[2]={
{VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO,NULL,0,VK_SHADER_STAGE_VERTEX_BIT,modules[0],"main"},
{VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO,NULL,0,VK_SHADER_STAGE_FRAGMENT_BIT,modules[1],"main"}};
VkPipelineVertexInputStateCreateInfo vertices={VK_STRUCTURE_TYPE_PIPELINE_VERTEX_INPUT_STATE_CREATE_INFO};
VkPipelineInputAssemblyStateCreateInfo assembly={VK_STRUCTURE_TYPE_PIPELINE_INPUT_ASSEMBLY_STATE_CREATE_INFO,NULL,0,VK_PRIMITIVE_TOPOLOGY_TRIANGLE_LIST};
VkPipelineViewportStateCreateInfo viewport={VK_STRUCTURE_TYPE_PIPELINE_VIEWPORT_STATE_CREATE_INFO,NULL,0,1,NULL,1,NULL};
VkPipelineRasterizationStateCreateInfo raster={VK_STRUCTURE_TYPE_PIPELINE_RASTERIZATION_STATE_CREATE_INFO,NULL,0,VK_FALSE,VK_FALSE,
VK_POLYGON_MODE_FILL,VK_CULL_MODE_NONE,VK_FRONT_FACE_COUNTER_CLOCKWISE,VK_FALSE,0,0,0,1};
VkPipelineMultisampleStateCreateInfo samples={VK_STRUCTURE_TYPE_PIPELINE_MULTISAMPLE_STATE_CREATE_INFO,NULL,0,VK_SAMPLE_COUNT_1_BIT};
VkPipelineColorBlendAttachmentState blend={.colorWriteMask=VK_COLOR_COMPONENT_R_BIT|VK_COLOR_COMPONENT_G_BIT|VK_COLOR_COMPONENT_B_BIT|VK_COLOR_COMPONENT_A_BIT};
VkPipelineColorBlendStateCreateInfo blends={VK_STRUCTURE_TYPE_PIPELINE_COLOR_BLEND_STATE_CREATE_INFO,NULL,0,VK_FALSE,VK_LOGIC_OP_COPY,1,&blend};
VkDynamicState dynamic_states[2]={VK_DYNAMIC_STATE_VIEWPORT,VK_DYNAMIC_STATE_SCISSOR};
VkPipelineDynamicStateCreateInfo dynamic={VK_STRUCTURE_TYPE_PIPELINE_DYNAMIC_STATE_CREATE_INFO,NULL,0,2,dynamic_states};
VkGraphicsPipelineCreateInfo pci={VK_STRUCTURE_TYPE_GRAPHICS_PIPELINE_CREATE_INFO,NULL,0,2,stages,&vertices,&assembly,NULL,&viewport,
&raster,&samples,NULL,&blends,&dynamic,c->layout,c->render_pass,0,VK_NULL_HANDLE,-1};
VkResult result=spvk.CreateGraphicsPipelines(vk.device,VK_NULL_HANDLE,1,&pci,NULL,&c->pipeline);
for(int k=0;k<2;k++)spvk.DestroyShaderModule(vk.device,modules[k],NULL);
if(result!=VK_SUCCESS)goto fail;
LOG("surface_composite: one primary stereo scene, %ux%u per eye",p->size,p->size);return 1;
fail:
surf_composite_destroy(p);c->failed=1;
LOG("surface_composite: setup failed; retaining ordinary composition");return 0;
}
static const XrCompositionLayerBaseHeader *surf_composite_frame(XrSession session,const XrFrameEndInfo *info,
const XrCompositionLayerProjection *scene,const XrCompositionLayerBaseHeader *video) {
surf_projection *p=NULL;
for(int i=0;i<SURF_MAX;i++)if(video==(const void*)&surf_projections[i].layer && surfs[i].native_project) {
p=&surf_projections[i];break;
}
if(!p || !p->cache_ready || !sc_scene_supported(scene))return NULL;
if(p->presented_job.params[0][7]!=0 || p->presented_job.params[1][7]!=0)return NULL;
sc_capture_until=monotonic_ns()+500000000ll;
surf_composite *c=&sc_outputs[p-surf_projections];
surf_scene_snapshot *sources[2]={0};
for(int e=0;e<2;e++)for(int i=0;i<SC_SCENES;i++)if(sc_scenes[i].handle==scene->views[e].subImage.swapchain) {
sources[e]=&sc_scenes[i];break;
}
if(p->cached_job.space!=surfs[p-surf_projections].video_job.space)return NULL;
if(!sources[0] || !sources[1] || sources[0]->captured_release!=sources[0]->releases || sources[1]->captured_release!=sources[1]->releases)goto held;
for(int e=0;e<2;e++)if(scene->views[e].next || scene->views[e].subImage.imageArrayIndex>=sources[e]->layers)goto held;
if(!sc_setup(session,p,c))return NULL;
PFN_xrAcquireSwapchainImage acquire=(PFN_xrAcquireSwapchainImage)lookup(active_instance,"xrAcquireSwapchainImage");
PFN_xrWaitSwapchainImage wait=(PFN_xrWaitSwapchainImage)lookup(active_instance,"xrWaitSwapchainImage");
PFN_xrReleaseSwapchainImage release=(PFN_xrReleaseSwapchainImage)lookup(active_instance,"xrReleaseSwapchainImage");
XrSwapchainImageAcquireInfo ai={XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
XrSwapchainImageWaitInfo wi={XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO,NULL,0};
XrSwapchainImageReleaseInfo ri={XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};
if(!c->acquired) {
if(XR_FAILED(acquire(c->output,&ai,&c->index)))goto held;
c->acquired=1;
}
if(wait(c->output,&wi)!=XR_SUCCESS)goto held;
if(c->index>=c->count) {release(c->output,&ri);c->acquired=0;goto held;}
uint32_t index=c->index;
if(!sc_reuse(c->fences[index],&c->pending[index]))goto held;
surf_swapchain *s=&surfs[p-surf_projections];
pthread_mutex_lock(&surf_lock);surf_video_job current=s->video_job;pthread_mutex_unlock(&surf_lock);
VkCommandBuffer cmd=c->commands[index];
VkCommandBufferBeginInfo begin={VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,NULL,VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
if(vk.ResetCommandBuffer(cmd,0)!=VK_SUCCESS || vk.BeginCommandBuffer(cmd,&begin)!=VK_SUCCESS)goto held;
VkRenderPassBeginInfo rbi={VK_STRUCTURE_TYPE_RENDER_PASS_BEGIN_INFO,NULL,c->render_pass,c->framebuffers[index],
{{0,0},{2*p->size,p->size}},0,NULL};
spvk.CmdBeginRenderPass(cmd,&rbi,VK_SUBPASS_CONTENTS_INLINE);
spvk.CmdBindPipeline(cmd,VK_PIPELINE_BIND_POINT_GRAPHICS,c->pipeline);
for(int e=0;e<2;e++) {
VkDescriptorImageInfo di[2]={{c->sampler,p->video_cache_view,VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL},
{c->sampler,sources[e]->view,VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL}};
VkWriteDescriptorSet writes[2];
for(int k=0;k<2;k++)writes[k]=(VkWriteDescriptorSet){VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET,NULL,c->sets[index][e],k,0,1,
VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,&di[k]};
spvk.UpdateDescriptorSets(vk.device,2,writes,0,NULL);
spvk.CmdBindDescriptorSets(cmd,VK_PIPELINE_BIND_POINT_GRAPHICS,c->layout,0,1,&c->sets[index][e],0,NULL);
surf_composite_params params=surf_composite_parameters(&p->cached_job,&current,e,
scene->views[e].subImage.imageRect,sources[e]->width,sources[e]->height,scene->views[e].subImage.imageArrayIndex);
params.scene_options[0]=(scene->layerFlags&XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT)!=0;
params.scene_options[1]=(scene->layerFlags&XR_COMPOSITION_LAYER_UNPREMULTIPLIED_ALPHA_BIT)!=0;
for(const XrBaseInStructure *n=scene->next;n;n=n->next) {
if(n->type==XR_TYPE_COMPOSITION_LAYER_COLOR_SCALE_BIAS_KHR) {
const XrCompositionLayerColorScaleBiasKHR *color=(const void*)n;
memcpy(params.scene_scale,&color->colorScale,16);memcpy(params.scene_bias,&color->colorBias,16);
}
if(n->type==XR_TYPE_COMPOSITION_LAYER_IMAGE_LAYOUT_FB)
params.scene_options[2]=(((const XrCompositionLayerImageLayoutFB*)n)->flags&XR_COMPOSITION_LAYER_IMAGE_LAYOUT_VERTICAL_FLIP_BIT_FB)!=0;
}
VkViewport viewport={(float)(e*p->size),0,(float)p->size,(float)p->size,0,1};
VkRect2D scissor={{e*(int32_t)p->size,0},{p->size,p->size}};
spvk.CmdSetViewport(cmd,0,1,&viewport);spvk.CmdSetScissor(cmd,0,1,&scissor);
spvk.CmdPushConstants(cmd,c->layout,VK_SHADER_STAGE_FRAGMENT_BIT,0,sizeof(params),&params);
spvk.CmdDraw(cmd,3,1,0,0);
}
spvk.CmdEndRenderPass(cmd);
if(vk.EndCommandBuffer(cmd)!=VK_SUCCESS)goto held;
VkSubmitInfo submit={VK_STRUCTURE_TYPE_SUBMIT_INFO,NULL,0,NULL,NULL,1,&cmd};
if(vk.QueueSubmit(vk.queue,1,&submit,c->fences[index])!=VK_SUCCESS)goto held;
c->pending[index]=1;
if(XR_FAILED(release(c->output,&ri)))goto held;
c->acquired=0;
for(int e=0;e<2;e++) {
c->eyes[e]=scene->views[e];c->eyes[e].next=NULL;
c->eyes[e].subImage=(XrSwapchainSubImage){c->output,{{e*(int32_t)p->size,0},{(int32_t)p->size,(int32_t)p->size}},0};
}
c->layer=*scene;c->layer.layerFlags &= ~XR_COMPOSITION_LAYER_UNPREMULTIPLIED_ALPHA_BIT;c->layer.next=NULL;c->layer.views=c->eyes;c->ready=1;c->time=info->displayTime;c->frames++;
if(monotonic_ns()-c->log_time>5000000000ll) {
LOG("surface_composite: %llu new scene frames, %llu held; no scene/overlay promotion",(unsigned long long)c->frames,(unsigned long long)c->holds);
c->log_time=monotonic_ns();
}
return (const void*)&c->layer;
held:
c->holds++;
return c->ready && c->layer.space==scene->space?(const void*)&c->layer:NULL;
}
static int surf_composite_candidate(const XrFrameEndInfo *info,const XrCompositionLayerBaseHeader *video) {
if(!info || !info->layerCount || !info->layers[0] || info->layers[0]->type!=XR_TYPE_COMPOSITION_LAYER_PROJECTION ||
!emul_is_equirect(video))return 0;
const XrCompositionLayerProjection *scene=(const void*)info->layers[0];
if(!sc_scene_supported(scene))return 0;
emul_common target=emul_common_of(video);
for(uint32_t i=1;i<info->layerCount;i++) {
const XrCompositionLayerBaseHeader *layer=info->layers[i];
if(layer==video)return 1;
if(!layer)continue;
if(!emul_is_equirect(layer))return 0;
emul_common other=emul_common_of(layer);
if(other.sub.swapchain!=target.sub.swapchain || other.space!=target.space)return 0;
}
return 0;
}
static void surf_scene_release_commit(XrSwapchain handle) {
for(int i=0;i<SC_SCENES;i++)if(sc_scenes[i].handle==handle){sc_scenes[i].releases++;break;}
}
static int surf_composite_cached_only(const XrFrameEndInfo *info,surf_projection *p,
const surf_video_job *rendered,const surf_video_job *current) {
surf_composite *c=&sc_outputs[p-surf_projections];
const XrCompositionLayerProjection *scene=(const void*)info->layers[0];
return c->ready && !p->acquired && c->layer.space==scene->space && rendered->space==current->space &&
rendered->params[0][7]==0 && rendered->params[1][7]==0 && !c->failed;
}
static int surf_composite_active(surf_projection *p) {return sc_outputs[p-surf_projections].ready;}
+39
View File
@@ -0,0 +1,39 @@
#version 450
// Reproject the decoded video into the game camera, then compose in linear light.
// Every output pixel belongs to one primary projection, including fades/edges.
layout(location=0) in vec2 uv;
layout(location=0) out vec4 outputColor;
layout(binding=0) uniform sampler2D videoImage;
layout(binding=1) uniform sampler2DArray sceneImage;
layout(push_constant) uniform Params {
vec4 rotation; vec4 outputTans; vec4 videoTans;
vec4 sceneRect; vec4 options; vec4 sceneOptions; vec4 sceneScale; vec4 sceneBias;
} p;
vec3 rotate(vec4 q,vec3 v){return v+2.0*cross(q.xyz,cross(q.xyz,v)+q.w*v);}
void main(){
vec2 scenePixel=0.5/vec2(textureSize(sceneImage,0).xy);
vec2 sceneCoord=vec2(uv.x,p.sceneOptions.z>0 ? 1-uv.y : uv.y);
vec2 sceneUV=clamp(p.sceneRect.xy+sceneCoord*p.sceneRect.zw,p.sceneRect.xy+scenePixel,p.sceneRect.xy+p.sceneRect.zw-scenePixel);
vec4 scene=textureLod(sceneImage,vec3(sceneUV,p.options.y),0);
// Projection rows follow Vulkan/OpenXR's top-to-bottom convention.
vec3 ray=rotate(p.rotation,vec3(mix(p.outputTans.x,p.outputTans.y,uv.x),mix(p.outputTans.w,p.outputTans.z,uv.y),-1));
vec2 vuv=(ray.xy/-ray.z-p.videoTans.xz)/(p.videoTans.yw-p.videoTans.xz); vuv.y=1-vuv.y;
vec4 video=vec4(0);
if(ray.z<0 && all(greaterThanEqual(vuv,vec2(0))) && all(lessThanEqual(vuv,vec2(1))))
{
vec2 videoPixel=0.5/vec2(textureSize(videoImage,0));
vec4 videoRect=vec4(p.options.x*.5,0,.5,1);
vec2 videoUV=clamp(videoRect.xy+vuv*videoRect.zw,videoRect.xy+videoPixel,videoRect.xy+videoRect.zw-videoPixel);
video=textureLod(videoImage,videoUV,0);
}
float alpha=p.options.z>0 ? video.a : 1;
vec3 rgb=p.options.w>0 ? video.rgb*alpha : video.rgb;
float sceneAlpha=p.sceneOptions.x>0 ? scene.a : 1;
vec3 sceneRGB=p.sceneOptions.y>0 ? scene.rgb*sceneAlpha : scene.rgb;
// OpenXR colour transforms act on straight colour, then restore premultiplication.
vec4 sceneStraight=vec4(sceneAlpha>0 ? sceneRGB/sceneAlpha : vec3(0),sceneAlpha);
sceneStraight=sceneStraight*max(p.sceneScale,vec4(0))+p.sceneBias;
sceneAlpha=sceneStraight.a;sceneRGB=sceneStraight.rgb*sceneAlpha;
vec4 result=vec4(rgb+sceneRGB*(1-alpha),alpha+sceneAlpha*(1-alpha));
outputColor=result;
}
+29
View File
@@ -0,0 +1,29 @@
// SPDX-License-Identifier: GPL-3.0-only
// Parameters for composing a cached movie view into the game's scene camera.
#ifndef FRAMEBRIDGE_SURFACE_COMPOSITE_H
#define FRAMEBRIDGE_SURFACE_COMPOSITE_H
#include "surface_video.h"
typedef struct {
float rotation[4], output_tans[4], video_tans[4];
float scene_rect[4], options[4], scene_options[4], scene_scale[4], scene_bias[4];
} surf_composite_params;
static surf_composite_params surf_composite_parameters(const surf_video_job *rendered,
const surf_video_job *current,int eye,XrRect2Di rect,uint32_t width,uint32_t height,uint32_t array_index) {
surf_composite_params p={0};
// The current view must already be transformed into the panorama space.
XrPosef source=surf_video_pose(rendered,current,eye);
XrQuaternionf relative=lm_qmul(lm_qconj(source.orientation),current->views[eye].pose.orientation);
memcpy(p.rotation,&relative,sizeof(relative));
for(int i=0;i<4;i++) {
p.output_tans[i]=current->params[eye][8+i]/SURF_VIDEO_GUARD;
p.video_tans[i]=rendered->params[eye][8+i];
}
p.scene_rect[0]=rect.offset.x/(float)width;p.scene_rect[1]=rect.offset.y/(float)height;
p.scene_rect[2]=rect.extent.width/(float)width;p.scene_rect[3]=rect.extent.height/(float)height;
for(int i=0;i<4;i++)p.scene_scale[i]=1;
p.options[0]=(float)eye;p.options[1]=(float)array_index;
p.options[2]=(rendered->flags&XR_COMPOSITION_LAYER_BLEND_TEXTURE_SOURCE_ALPHA_BIT)!=0;
p.options[3]=(rendered->flags&XR_COMPOSITION_LAYER_UNPREMULTIPLIED_ALPHA_BIT)!=0;
return p;
}
#endif
+3
View File
@@ -0,0 +1,3 @@
#version 450
layout(location=0) out vec2 uv;
void main(){vec2 p=vec2((gl_VertexIndex<<1)&2,gl_VertexIndex&2);uv=p;gl_Position=vec4(p*2-1,0,1);}
+300
View File
@@ -0,0 +1,300 @@
// SPDX-License-Identifier: GPL-3.0-only
// Generated by glslc; Vulkan 1.0.
static const uint32_t surface_composite_vert_spv[]={
0x07230203,0x00010000,0x000d000a,0x0000002e,0x00000000,0x00020011,0x00000001,0x0006000b,
0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,0x00000000,0x00000001,
0x0008000f,0x00000000,0x00000004,0x6e69616d,0x00000000,0x0000000c,0x00000018,0x00000020,
0x00030003,0x00000002,0x000001c2,0x000a0004,0x475f4c47,0x4c474f4f,0x70635f45,0x74735f70,
0x5f656c79,0x656e696c,0x7269645f,0x69746365,0x00006576,0x00080004,0x475f4c47,0x4c474f4f,
0x6e695f45,0x64756c63,0x69645f65,0x74636572,0x00657669,0x00040005,0x00000004,0x6e69616d,
0x00000000,0x00030005,0x00000009,0x00000070,0x00060005,0x0000000c,0x565f6c67,0x65747265,
0x646e4978,0x00007865,0x00030005,0x00000018,0x00007675,0x00060005,0x0000001e,0x505f6c67,
0x65567265,0x78657472,0x00000000,0x00060006,0x0000001e,0x00000000,0x505f6c67,0x7469736f,
0x006e6f69,0x00070006,0x0000001e,0x00000001,0x505f6c67,0x746e696f,0x657a6953,0x00000000,
0x00070006,0x0000001e,0x00000002,0x435f6c67,0x4470696c,0x61747369,0x0065636e,0x00070006,
0x0000001e,0x00000003,0x435f6c67,0x446c6c75,0x61747369,0x0065636e,0x00030005,0x00000020,
0x00000000,0x00040047,0x0000000c,0x0000000b,0x0000002a,0x00040047,0x00000018,0x0000001e,
0x00000000,0x00050048,0x0000001e,0x00000000,0x0000000b,0x00000000,0x00050048,0x0000001e,
0x00000001,0x0000000b,0x00000001,0x00050048,0x0000001e,0x00000002,0x0000000b,0x00000003,
0x00050048,0x0000001e,0x00000003,0x0000000b,0x00000004,0x00030047,0x0000001e,0x00000002,
0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,0x00030016,0x00000006,0x00000020,
0x00040017,0x00000007,0x00000006,0x00000002,0x00040020,0x00000008,0x00000007,0x00000007,
0x00040015,0x0000000a,0x00000020,0x00000001,0x00040020,0x0000000b,0x00000001,0x0000000a,
0x0004003b,0x0000000b,0x0000000c,0x00000001,0x0004002b,0x0000000a,0x0000000e,0x00000001,
0x0004002b,0x0000000a,0x00000010,0x00000002,0x00040020,0x00000017,0x00000003,0x00000007,
0x0004003b,0x00000017,0x00000018,0x00000003,0x00040017,0x0000001a,0x00000006,0x00000004,
0x00040015,0x0000001b,0x00000020,0x00000000,0x0004002b,0x0000001b,0x0000001c,0x00000001,
0x0004001c,0x0000001d,0x00000006,0x0000001c,0x0006001e,0x0000001e,0x0000001a,0x00000006,
0x0000001d,0x0000001d,0x00040020,0x0000001f,0x00000003,0x0000001e,0x0004003b,0x0000001f,
0x00000020,0x00000003,0x0004002b,0x0000000a,0x00000021,0x00000000,0x0004002b,0x00000006,
0x00000023,0x40000000,0x0004002b,0x00000006,0x00000025,0x3f800000,0x0004002b,0x00000006,
0x00000028,0x00000000,0x00040020,0x0000002c,0x00000003,0x0000001a,0x00050036,0x00000002,
0x00000004,0x00000000,0x00000003,0x000200f8,0x00000005,0x0004003b,0x00000008,0x00000009,
0x00000007,0x0004003d,0x0000000a,0x0000000d,0x0000000c,0x000500c4,0x0000000a,0x0000000f,
0x0000000d,0x0000000e,0x000500c7,0x0000000a,0x00000011,0x0000000f,0x00000010,0x0004006f,
0x00000006,0x00000012,0x00000011,0x0004003d,0x0000000a,0x00000013,0x0000000c,0x000500c7,
0x0000000a,0x00000014,0x00000013,0x00000010,0x0004006f,0x00000006,0x00000015,0x00000014,
0x00050050,0x00000007,0x00000016,0x00000012,0x00000015,0x0003003e,0x00000009,0x00000016,
0x0004003d,0x00000007,0x00000019,0x00000009,0x0003003e,0x00000018,0x00000019,0x0004003d,
0x00000007,0x00000022,0x00000009,0x0005008e,0x00000007,0x00000024,0x00000022,0x00000023,
0x00050050,0x00000007,0x00000026,0x00000025,0x00000025,0x00050083,0x00000007,0x00000027,
0x00000024,0x00000026,0x00050051,0x00000006,0x00000029,0x00000027,0x00000000,0x00050051,
0x00000006,0x0000002a,0x00000027,0x00000001,0x00070050,0x0000001a,0x0000002b,0x00000029,
0x0000002a,0x00000028,0x00000025,0x00050041,0x0000002c,0x0000002d,0x00000020,0x00000021,
0x0003003e,0x0000002d,0x0000002b,0x000100fd,0x00010038,
};
static const uint32_t surface_composite_frag_spv[]={
0x07230203,0x00010000,0x000d000a,0x00000153,0x00000000,0x00020011,0x00000001,0x00020011,
0x00000032,0x0006000b,0x00000001,0x4c534c47,0x6474732e,0x3035342e,0x00000000,0x0003000e,
0x00000000,0x00000001,0x0007000f,0x00000004,0x00000004,0x6e69616d,0x00000000,0x0000003a,
0x00000151,0x00030010,0x00000004,0x00000007,0x00030003,0x00000002,0x000001c2,0x000a0004,
0x475f4c47,0x4c474f4f,0x70635f45,0x74735f70,0x5f656c79,0x656e696c,0x7269645f,0x69746365,
0x00006576,0x00080004,0x475f4c47,0x4c474f4f,0x6e695f45,0x64756c63,0x69645f65,0x74636572,
0x00657669,0x00040005,0x00000004,0x6e69616d,0x00000000,0x00060005,0x0000000e,0x61746f72,
0x76286574,0x763b3466,0x003b3366,0x00030005,0x0000000c,0x00000071,0x00030005,0x0000000d,
0x00000076,0x00050005,0x00000027,0x6e656373,0x78695065,0x00006c65,0x00050005,0x0000002c,
0x6e656373,0x616d4965,0x00006567,0x00050005,0x00000038,0x6e656373,0x6f6f4365,0x00006472,
0x00030005,0x0000003a,0x00007675,0x00040005,0x0000003f,0x61726150,0x0000736d,0x00060006,
0x0000003f,0x00000000,0x61746f72,0x6e6f6974,0x00000000,0x00060006,0x0000003f,0x00000001,
0x7074756f,0x61547475,0x0000736e,0x00060006,0x0000003f,0x00000002,0x65646976,0x6e61546f,
0x00000073,0x00060006,0x0000003f,0x00000003,0x6e656373,0x63655265,0x00000074,0x00050006,
0x0000003f,0x00000004,0x6974706f,0x00736e6f,0x00070006,0x0000003f,0x00000005,0x6e656373,
0x74704f65,0x736e6f69,0x00000000,0x00060006,0x0000003f,0x00000006,0x6e656373,0x61635365,
0x0000656c,0x00060006,0x0000003f,0x00000007,0x6e656373,0x61694265,0x00000073,0x00030005,
0x00000041,0x00000070,0x00040005,0x00000057,0x6e656373,0x00565565,0x00040005,0x00000072,
0x6e656373,0x00000065,0x00030005,0x0000007c,0x00796172,0x00040005,0x0000008e,0x61726170,
0x0000006d,0x00040005,0x00000091,0x61726170,0x0000006d,0x00030005,0x00000093,0x00767576,
0x00040005,0x000000ac,0x65646976,0x0000006f,0x00050005,0x000000c2,0x65646976,0x7869506f,
0x00006c65,0x00050005,0x000000c6,0x65646976,0x616d496f,0x00006567,0x00050005,0x000000cd,
0x65646976,0x6365526f,0x00000074,0x00040005,0x000000d2,0x65646976,0x0056556f,0x00040005,
0x000000e9,0x68706c61,0x00000061,0x00030005,0x000000f4,0x00626772,0x00050005,0x00000103,
0x6e656373,0x706c4165,0x00006168,0x00050005,0x0000010e,0x6e656373,0x42475265,0x00000000,
0x00060005,0x0000011d,0x6e656373,0x72745365,0x68676961,0x00000074,0x00040005,0x0000013f,
0x75736572,0x0000746c,0x00050005,0x00000151,0x7074756f,0x6f437475,0x00726f6c,0x00040047,
0x0000002c,0x00000022,0x00000000,0x00040047,0x0000002c,0x00000021,0x00000001,0x00040047,
0x0000003a,0x0000001e,0x00000000,0x00050048,0x0000003f,0x00000000,0x00000023,0x00000000,
0x00050048,0x0000003f,0x00000001,0x00000023,0x00000010,0x00050048,0x0000003f,0x00000002,
0x00000023,0x00000020,0x00050048,0x0000003f,0x00000003,0x00000023,0x00000030,0x00050048,
0x0000003f,0x00000004,0x00000023,0x00000040,0x00050048,0x0000003f,0x00000005,0x00000023,
0x00000050,0x00050048,0x0000003f,0x00000006,0x00000023,0x00000060,0x00050048,0x0000003f,
0x00000007,0x00000023,0x00000070,0x00030047,0x0000003f,0x00000002,0x00040047,0x000000c6,
0x00000022,0x00000000,0x00040047,0x000000c6,0x00000021,0x00000000,0x00040047,0x00000151,
0x0000001e,0x00000000,0x00020013,0x00000002,0x00030021,0x00000003,0x00000002,0x00030016,
0x00000006,0x00000020,0x00040017,0x00000007,0x00000006,0x00000004,0x00040020,0x00000008,
0x00000007,0x00000007,0x00040017,0x00000009,0x00000006,0x00000003,0x00040020,0x0000000a,
0x00000007,0x00000009,0x00050021,0x0000000b,0x00000009,0x00000008,0x0000000a,0x0004002b,
0x00000006,0x00000011,0x40000000,0x00040015,0x00000018,0x00000020,0x00000000,0x0004002b,
0x00000018,0x00000019,0x00000003,0x00040020,0x0000001a,0x00000007,0x00000006,0x00040017,
0x00000025,0x00000006,0x00000002,0x00040020,0x00000026,0x00000007,0x00000025,0x0004002b,
0x00000006,0x00000028,0x3f000000,0x00090019,0x00000029,0x00000006,0x00000001,0x00000000,
0x00000001,0x00000000,0x00000001,0x00000000,0x0003001b,0x0000002a,0x00000029,0x00040020,
0x0000002b,0x00000000,0x0000002a,0x0004003b,0x0000002b,0x0000002c,0x00000000,0x00040015,
0x0000002e,0x00000020,0x00000001,0x0004002b,0x0000002e,0x0000002f,0x00000000,0x00040017,
0x00000031,0x0000002e,0x00000003,0x00040017,0x00000033,0x0000002e,0x00000002,0x00040020,
0x00000039,0x00000001,0x00000025,0x0004003b,0x00000039,0x0000003a,0x00000001,0x0004002b,
0x00000018,0x0000003b,0x00000000,0x00040020,0x0000003c,0x00000001,0x00000006,0x000a001e,
0x0000003f,0x00000007,0x00000007,0x00000007,0x00000007,0x00000007,0x00000007,0x00000007,
0x00000007,0x00040020,0x00000040,0x00000009,0x0000003f,0x0004003b,0x00000040,0x00000041,
0x00000009,0x0004002b,0x0000002e,0x00000042,0x00000005,0x0004002b,0x00000018,0x00000043,
0x00000002,0x00040020,0x00000044,0x00000009,0x00000006,0x0004002b,0x00000006,0x00000047,
0x00000000,0x00020014,0x00000048,0x0004002b,0x00000006,0x0000004d,0x3f800000,0x0004002b,
0x00000018,0x0000004e,0x00000001,0x0004002b,0x0000002e,0x00000058,0x00000003,0x00040020,
0x00000059,0x00000009,0x00000007,0x0004002b,0x0000002e,0x00000075,0x00000004,0x0004002b,
0x0000002e,0x0000007d,0x00000001,0x0004002b,0x00000006,0x0000008c,0xbf800000,0x0004002b,
0x0000002e,0x0000009b,0x00000002,0x0007002c,0x00000007,0x000000ad,0x00000047,0x00000047,
0x00000047,0x00000047,0x0005002c,0x00000025,0x000000b4,0x00000047,0x00000047,0x00040017,
0x000000b5,0x00000048,0x00000002,0x0005002c,0x00000025,0x000000bc,0x0000004d,0x0000004d,
0x00090019,0x000000c3,0x00000006,0x00000001,0x00000000,0x00000000,0x00000000,0x00000001,
0x00000000,0x0003001b,0x000000c4,0x000000c3,0x00040020,0x000000c5,0x00000000,0x000000c4,
0x0004003b,0x000000c5,0x000000c6,0x00000000,0x0006002c,0x00000009,0x00000128,0x00000047,
0x00000047,0x00000047,0x0004002b,0x0000002e,0x00000130,0x00000006,0x0004002b,0x0000002e,
0x00000135,0x00000007,0x00040020,0x00000150,0x00000003,0x00000007,0x0004003b,0x00000150,
0x00000151,0x00000003,0x00050036,0x00000002,0x00000004,0x00000000,0x00000003,0x000200f8,
0x00000005,0x0004003b,0x00000026,0x00000027,0x00000007,0x0004003b,0x00000026,0x00000038,
0x00000007,0x0004003b,0x0000001a,0x0000004a,0x00000007,0x0004003b,0x00000026,0x00000057,
0x00000007,0x0004003b,0x00000008,0x00000072,0x00000007,0x0004003b,0x0000000a,0x0000007c,
0x00000007,0x0004003b,0x00000008,0x0000008e,0x00000007,0x0004003b,0x0000000a,0x00000091,
0x00000007,0x0004003b,0x00000026,0x00000093,0x00000007,0x0004003b,0x00000008,0x000000ac,
0x00000007,0x0004003b,0x00000026,0x000000c2,0x00000007,0x0004003b,0x00000008,0x000000cd,
0x00000007,0x0004003b,0x00000026,0x000000d2,0x00000007,0x0004003b,0x0000001a,0x000000e9,
0x00000007,0x0004003b,0x0000001a,0x000000ed,0x00000007,0x0004003b,0x0000000a,0x000000f4,
0x00000007,0x0004003b,0x0000000a,0x000000f8,0x00000007,0x0004003b,0x0000001a,0x00000103,
0x00000007,0x0004003b,0x0000001a,0x00000107,0x00000007,0x0004003b,0x0000000a,0x0000010e,
0x00000007,0x0004003b,0x0000000a,0x00000112,0x00000007,0x0004003b,0x00000008,0x0000011d,
0x00000007,0x0004003b,0x0000000a,0x00000120,0x00000007,0x0004003b,0x00000008,0x0000013f,
0x00000007,0x0004003d,0x0000002a,0x0000002d,0x0000002c,0x00040064,0x00000029,0x00000030,
0x0000002d,0x00050067,0x00000031,0x00000032,0x00000030,0x0000002f,0x0007004f,0x00000033,
0x00000034,0x00000032,0x00000032,0x00000000,0x00000001,0x0004006f,0x00000025,0x00000035,
0x00000034,0x00050050,0x00000025,0x00000036,0x00000028,0x00000028,0x00050088,0x00000025,
0x00000037,0x00000036,0x00000035,0x0003003e,0x00000027,0x00000037,0x00050041,0x0000003c,
0x0000003d,0x0000003a,0x0000003b,0x0004003d,0x00000006,0x0000003e,0x0000003d,0x00060041,
0x00000044,0x00000045,0x00000041,0x00000042,0x00000043,0x0004003d,0x00000006,0x00000046,
0x00000045,0x000500ba,0x00000048,0x00000049,0x00000046,0x00000047,0x000300f7,0x0000004c,
0x00000000,0x000400fa,0x00000049,0x0000004b,0x00000052,0x000200f8,0x0000004b,0x00050041,
0x0000003c,0x0000004f,0x0000003a,0x0000004e,0x0004003d,0x00000006,0x00000050,0x0000004f,
0x00050083,0x00000006,0x00000051,0x0000004d,0x00000050,0x0003003e,0x0000004a,0x00000051,
0x000200f9,0x0000004c,0x000200f8,0x00000052,0x00050041,0x0000003c,0x00000053,0x0000003a,
0x0000004e,0x0004003d,0x00000006,0x00000054,0x00000053,0x0003003e,0x0000004a,0x00000054,
0x000200f9,0x0000004c,0x000200f8,0x0000004c,0x0004003d,0x00000006,0x00000055,0x0000004a,
0x00050050,0x00000025,0x00000056,0x0000003e,0x00000055,0x0003003e,0x00000038,0x00000056,
0x00050041,0x00000059,0x0000005a,0x00000041,0x00000058,0x0004003d,0x00000007,0x0000005b,
0x0000005a,0x0007004f,0x00000025,0x0000005c,0x0000005b,0x0000005b,0x00000000,0x00000001,
0x0004003d,0x00000025,0x0000005d,0x00000038,0x00050041,0x00000059,0x0000005e,0x00000041,
0x00000058,0x0004003d,0x00000007,0x0000005f,0x0000005e,0x0007004f,0x00000025,0x00000060,
0x0000005f,0x0000005f,0x00000002,0x00000003,0x00050085,0x00000025,0x00000061,0x0000005d,
0x00000060,0x00050081,0x00000025,0x00000062,0x0000005c,0x00000061,0x00050041,0x00000059,
0x00000063,0x00000041,0x00000058,0x0004003d,0x00000007,0x00000064,0x00000063,0x0007004f,
0x00000025,0x00000065,0x00000064,0x00000064,0x00000000,0x00000001,0x0004003d,0x00000025,
0x00000066,0x00000027,0x00050081,0x00000025,0x00000067,0x00000065,0x00000066,0x00050041,
0x00000059,0x00000068,0x00000041,0x00000058,0x0004003d,0x00000007,0x00000069,0x00000068,
0x0007004f,0x00000025,0x0000006a,0x00000069,0x00000069,0x00000000,0x00000001,0x00050041,
0x00000059,0x0000006b,0x00000041,0x00000058,0x0004003d,0x00000007,0x0000006c,0x0000006b,
0x0007004f,0x00000025,0x0000006d,0x0000006c,0x0000006c,0x00000002,0x00000003,0x00050081,
0x00000025,0x0000006e,0x0000006a,0x0000006d,0x0004003d,0x00000025,0x0000006f,0x00000027,
0x00050083,0x00000025,0x00000070,0x0000006e,0x0000006f,0x0008000c,0x00000025,0x00000071,
0x00000001,0x0000002b,0x00000062,0x00000067,0x00000070,0x0003003e,0x00000057,0x00000071,
0x0004003d,0x0000002a,0x00000073,0x0000002c,0x0004003d,0x00000025,0x00000074,0x00000057,
0x00060041,0x00000044,0x00000076,0x00000041,0x00000075,0x0000004e,0x0004003d,0x00000006,
0x00000077,0x00000076,0x00050051,0x00000006,0x00000078,0x00000074,0x00000000,0x00050051,
0x00000006,0x00000079,0x00000074,0x00000001,0x00060050,0x00000009,0x0000007a,0x00000078,
0x00000079,0x00000077,0x00070058,0x00000007,0x0000007b,0x00000073,0x0000007a,0x00000002,
0x00000047,0x0003003e,0x00000072,0x0000007b,0x00060041,0x00000044,0x0000007e,0x00000041,
0x0000007d,0x0000003b,0x0004003d,0x00000006,0x0000007f,0x0000007e,0x00060041,0x00000044,
0x00000080,0x00000041,0x0000007d,0x0000004e,0x0004003d,0x00000006,0x00000081,0x00000080,
0x00050041,0x0000003c,0x00000082,0x0000003a,0x0000003b,0x0004003d,0x00000006,0x00000083,
0x00000082,0x0008000c,0x00000006,0x00000084,0x00000001,0x0000002e,0x0000007f,0x00000081,
0x00000083,0x00060041,0x00000044,0x00000085,0x00000041,0x0000007d,0x00000019,0x0004003d,
0x00000006,0x00000086,0x00000085,0x00060041,0x00000044,0x00000087,0x00000041,0x0000007d,
0x00000043,0x0004003d,0x00000006,0x00000088,0x00000087,0x00050041,0x0000003c,0x00000089,
0x0000003a,0x0000004e,0x0004003d,0x00000006,0x0000008a,0x00000089,0x0008000c,0x00000006,
0x0000008b,0x00000001,0x0000002e,0x00000086,0x00000088,0x0000008a,0x00060050,0x00000009,
0x0000008d,0x00000084,0x0000008b,0x0000008c,0x00050041,0x00000059,0x0000008f,0x00000041,
0x0000002f,0x0004003d,0x00000007,0x00000090,0x0000008f,0x0003003e,0x0000008e,0x00000090,
0x0003003e,0x00000091,0x0000008d,0x00060039,0x00000009,0x00000092,0x0000000e,0x0000008e,
0x00000091,0x0003003e,0x0000007c,0x00000092,0x0004003d,0x00000009,0x00000094,0x0000007c,
0x0007004f,0x00000025,0x00000095,0x00000094,0x00000094,0x00000000,0x00000001,0x00050041,
0x0000001a,0x00000096,0x0000007c,0x00000043,0x0004003d,0x00000006,0x00000097,0x00000096,
0x0004007f,0x00000006,0x00000098,0x00000097,0x00050050,0x00000025,0x00000099,0x00000098,
0x00000098,0x00050088,0x00000025,0x0000009a,0x00000095,0x00000099,0x00050041,0x00000059,
0x0000009c,0x00000041,0x0000009b,0x0004003d,0x00000007,0x0000009d,0x0000009c,0x0007004f,
0x00000025,0x0000009e,0x0000009d,0x0000009d,0x00000000,0x00000002,0x00050083,0x00000025,
0x0000009f,0x0000009a,0x0000009e,0x00050041,0x00000059,0x000000a0,0x00000041,0x0000009b,
0x0004003d,0x00000007,0x000000a1,0x000000a0,0x0007004f,0x00000025,0x000000a2,0x000000a1,
0x000000a1,0x00000001,0x00000003,0x00050041,0x00000059,0x000000a3,0x00000041,0x0000009b,
0x0004003d,0x00000007,0x000000a4,0x000000a3,0x0007004f,0x00000025,0x000000a5,0x000000a4,
0x000000a4,0x00000000,0x00000002,0x00050083,0x00000025,0x000000a6,0x000000a2,0x000000a5,
0x00050088,0x00000025,0x000000a7,0x0000009f,0x000000a6,0x0003003e,0x00000093,0x000000a7,
0x00050041,0x0000001a,0x000000a8,0x00000093,0x0000004e,0x0004003d,0x00000006,0x000000a9,
0x000000a8,0x00050083,0x00000006,0x000000aa,0x0000004d,0x000000a9,0x00050041,0x0000001a,
0x000000ab,0x00000093,0x0000004e,0x0003003e,0x000000ab,0x000000aa,0x0003003e,0x000000ac,
0x000000ad,0x00050041,0x0000001a,0x000000ae,0x0000007c,0x00000043,0x0004003d,0x00000006,
0x000000af,0x000000ae,0x000500b8,0x00000048,0x000000b0,0x000000af,0x00000047,0x000300f7,
0x000000b2,0x00000000,0x000400fa,0x000000b0,0x000000b1,0x000000b2,0x000200f8,0x000000b1,
0x0004003d,0x00000025,0x000000b3,0x00000093,0x000500be,0x000000b5,0x000000b6,0x000000b3,
0x000000b4,0x0004009b,0x00000048,0x000000b7,0x000000b6,0x000200f9,0x000000b2,0x000200f8,
0x000000b2,0x000700f5,0x00000048,0x000000b8,0x000000b0,0x0000004c,0x000000b7,0x000000b1,
0x000300f7,0x000000ba,0x00000000,0x000400fa,0x000000b8,0x000000b9,0x000000ba,0x000200f8,
0x000000b9,0x0004003d,0x00000025,0x000000bb,0x00000093,0x000500bc,0x000000b5,0x000000bd,
0x000000bb,0x000000bc,0x0004009b,0x00000048,0x000000be,0x000000bd,0x000200f9,0x000000ba,
0x000200f8,0x000000ba,0x000700f5,0x00000048,0x000000bf,0x000000b8,0x000000b2,0x000000be,
0x000000b9,0x000300f7,0x000000c1,0x00000000,0x000400fa,0x000000bf,0x000000c0,0x000000c1,
0x000200f8,0x000000c0,0x0004003d,0x000000c4,0x000000c7,0x000000c6,0x00040064,0x000000c3,
0x000000c8,0x000000c7,0x00050067,0x00000033,0x000000c9,0x000000c8,0x0000002f,0x0004006f,
0x00000025,0x000000ca,0x000000c9,0x00050050,0x00000025,0x000000cb,0x00000028,0x00000028,
0x00050088,0x00000025,0x000000cc,0x000000cb,0x000000ca,0x0003003e,0x000000c2,0x000000cc,
0x00060041,0x00000044,0x000000ce,0x00000041,0x00000075,0x0000003b,0x0004003d,0x00000006,
0x000000cf,0x000000ce,0x00050085,0x00000006,0x000000d0,0x000000cf,0x00000028,0x00070050,
0x00000007,0x000000d1,0x000000d0,0x00000047,0x00000028,0x0000004d,0x0003003e,0x000000cd,
0x000000d1,0x0004003d,0x00000007,0x000000d3,0x000000cd,0x0007004f,0x00000025,0x000000d4,
0x000000d3,0x000000d3,0x00000000,0x00000001,0x0004003d,0x00000025,0x000000d5,0x00000093,
0x0004003d,0x00000007,0x000000d6,0x000000cd,0x0007004f,0x00000025,0x000000d7,0x000000d6,
0x000000d6,0x00000002,0x00000003,0x00050085,0x00000025,0x000000d8,0x000000d5,0x000000d7,
0x00050081,0x00000025,0x000000d9,0x000000d4,0x000000d8,0x0004003d,0x00000007,0x000000da,
0x000000cd,0x0007004f,0x00000025,0x000000db,0x000000da,0x000000da,0x00000000,0x00000001,
0x0004003d,0x00000025,0x000000dc,0x000000c2,0x00050081,0x00000025,0x000000dd,0x000000db,
0x000000dc,0x0004003d,0x00000007,0x000000de,0x000000cd,0x0007004f,0x00000025,0x000000df,
0x000000de,0x000000de,0x00000000,0x00000001,0x0004003d,0x00000007,0x000000e0,0x000000cd,
0x0007004f,0x00000025,0x000000e1,0x000000e0,0x000000e0,0x00000002,0x00000003,0x00050081,
0x00000025,0x000000e2,0x000000df,0x000000e1,0x0004003d,0x00000025,0x000000e3,0x000000c2,
0x00050083,0x00000025,0x000000e4,0x000000e2,0x000000e3,0x0008000c,0x00000025,0x000000e5,
0x00000001,0x0000002b,0x000000d9,0x000000dd,0x000000e4,0x0003003e,0x000000d2,0x000000e5,
0x0004003d,0x000000c4,0x000000e6,0x000000c6,0x0004003d,0x00000025,0x000000e7,0x000000d2,
0x00070058,0x00000007,0x000000e8,0x000000e6,0x000000e7,0x00000002,0x00000047,0x0003003e,
0x000000ac,0x000000e8,0x000200f9,0x000000c1,0x000200f8,0x000000c1,0x00060041,0x00000044,
0x000000ea,0x00000041,0x00000075,0x00000043,0x0004003d,0x00000006,0x000000eb,0x000000ea,
0x000500ba,0x00000048,0x000000ec,0x000000eb,0x00000047,0x000300f7,0x000000ef,0x00000000,
0x000400fa,0x000000ec,0x000000ee,0x000000f2,0x000200f8,0x000000ee,0x00050041,0x0000001a,
0x000000f0,0x000000ac,0x00000019,0x0004003d,0x00000006,0x000000f1,0x000000f0,0x0003003e,
0x000000ed,0x000000f1,0x000200f9,0x000000ef,0x000200f8,0x000000f2,0x0003003e,0x000000ed,
0x0000004d,0x000200f9,0x000000ef,0x000200f8,0x000000ef,0x0004003d,0x00000006,0x000000f3,
0x000000ed,0x0003003e,0x000000e9,0x000000f3,0x00060041,0x00000044,0x000000f5,0x00000041,
0x00000075,0x00000019,0x0004003d,0x00000006,0x000000f6,0x000000f5,0x000500ba,0x00000048,
0x000000f7,0x000000f6,0x00000047,0x000300f7,0x000000fa,0x00000000,0x000400fa,0x000000f7,
0x000000f9,0x000000ff,0x000200f8,0x000000f9,0x0004003d,0x00000007,0x000000fb,0x000000ac,
0x0008004f,0x00000009,0x000000fc,0x000000fb,0x000000fb,0x00000000,0x00000001,0x00000002,
0x0004003d,0x00000006,0x000000fd,0x000000e9,0x0005008e,0x00000009,0x000000fe,0x000000fc,
0x000000fd,0x0003003e,0x000000f8,0x000000fe,0x000200f9,0x000000fa,0x000200f8,0x000000ff,
0x0004003d,0x00000007,0x00000100,0x000000ac,0x0008004f,0x00000009,0x00000101,0x00000100,
0x00000100,0x00000000,0x00000001,0x00000002,0x0003003e,0x000000f8,0x00000101,0x000200f9,
0x000000fa,0x000200f8,0x000000fa,0x0004003d,0x00000009,0x00000102,0x000000f8,0x0003003e,
0x000000f4,0x00000102,0x00060041,0x00000044,0x00000104,0x00000041,0x00000042,0x0000003b,
0x0004003d,0x00000006,0x00000105,0x00000104,0x000500ba,0x00000048,0x00000106,0x00000105,
0x00000047,0x000300f7,0x00000109,0x00000000,0x000400fa,0x00000106,0x00000108,0x0000010c,
0x000200f8,0x00000108,0x00050041,0x0000001a,0x0000010a,0x00000072,0x00000019,0x0004003d,
0x00000006,0x0000010b,0x0000010a,0x0003003e,0x00000107,0x0000010b,0x000200f9,0x00000109,
0x000200f8,0x0000010c,0x0003003e,0x00000107,0x0000004d,0x000200f9,0x00000109,0x000200f8,
0x00000109,0x0004003d,0x00000006,0x0000010d,0x00000107,0x0003003e,0x00000103,0x0000010d,
0x00060041,0x00000044,0x0000010f,0x00000041,0x00000042,0x0000004e,0x0004003d,0x00000006,
0x00000110,0x0000010f,0x000500ba,0x00000048,0x00000111,0x00000110,0x00000047,0x000300f7,
0x00000114,0x00000000,0x000400fa,0x00000111,0x00000113,0x00000119,0x000200f8,0x00000113,
0x0004003d,0x00000007,0x00000115,0x00000072,0x0008004f,0x00000009,0x00000116,0x00000115,
0x00000115,0x00000000,0x00000001,0x00000002,0x0004003d,0x00000006,0x00000117,0x00000103,
0x0005008e,0x00000009,0x00000118,0x00000116,0x00000117,0x0003003e,0x00000112,0x00000118,
0x000200f9,0x00000114,0x000200f8,0x00000119,0x0004003d,0x00000007,0x0000011a,0x00000072,
0x0008004f,0x00000009,0x0000011b,0x0000011a,0x0000011a,0x00000000,0x00000001,0x00000002,
0x0003003e,0x00000112,0x0000011b,0x000200f9,0x00000114,0x000200f8,0x00000114,0x0004003d,
0x00000009,0x0000011c,0x00000112,0x0003003e,0x0000010e,0x0000011c,0x0004003d,0x00000006,
0x0000011e,0x00000103,0x000500ba,0x00000048,0x0000011f,0x0000011e,0x00000047,0x000300f7,
0x00000122,0x00000000,0x000400fa,0x0000011f,0x00000121,0x00000127,0x000200f8,0x00000121,
0x0004003d,0x00000009,0x00000123,0x0000010e,0x0004003d,0x00000006,0x00000124,0x00000103,
0x00060050,0x00000009,0x00000125,0x00000124,0x00000124,0x00000124,0x00050088,0x00000009,
0x00000126,0x00000123,0x00000125,0x0003003e,0x00000120,0x00000126,0x000200f9,0x00000122,
0x000200f8,0x00000127,0x0003003e,0x00000120,0x00000128,0x000200f9,0x00000122,0x000200f8,
0x00000122,0x0004003d,0x00000009,0x00000129,0x00000120,0x0004003d,0x00000006,0x0000012a,
0x00000103,0x00050051,0x00000006,0x0000012b,0x00000129,0x00000000,0x00050051,0x00000006,
0x0000012c,0x00000129,0x00000001,0x00050051,0x00000006,0x0000012d,0x00000129,0x00000002,
0x00070050,0x00000007,0x0000012e,0x0000012b,0x0000012c,0x0000012d,0x0000012a,0x0003003e,
0x0000011d,0x0000012e,0x0004003d,0x00000007,0x0000012f,0x0000011d,0x00050041,0x00000059,
0x00000131,0x00000041,0x00000130,0x0004003d,0x00000007,0x00000132,0x00000131,0x0007000c,
0x00000007,0x00000133,0x00000001,0x00000028,0x00000132,0x000000ad,0x00050085,0x00000007,
0x00000134,0x0000012f,0x00000133,0x00050041,0x00000059,0x00000136,0x00000041,0x00000135,
0x0004003d,0x00000007,0x00000137,0x00000136,0x00050081,0x00000007,0x00000138,0x00000134,
0x00000137,0x0003003e,0x0000011d,0x00000138,0x00050041,0x0000001a,0x00000139,0x0000011d,
0x00000019,0x0004003d,0x00000006,0x0000013a,0x00000139,0x0003003e,0x00000103,0x0000013a,
0x0004003d,0x00000007,0x0000013b,0x0000011d,0x0008004f,0x00000009,0x0000013c,0x0000013b,
0x0000013b,0x00000000,0x00000001,0x00000002,0x0004003d,0x00000006,0x0000013d,0x00000103,
0x0005008e,0x00000009,0x0000013e,0x0000013c,0x0000013d,0x0003003e,0x0000010e,0x0000013e,
0x0004003d,0x00000009,0x00000140,0x000000f4,0x0004003d,0x00000009,0x00000141,0x0000010e,
0x0004003d,0x00000006,0x00000142,0x000000e9,0x00050083,0x00000006,0x00000143,0x0000004d,
0x00000142,0x0005008e,0x00000009,0x00000144,0x00000141,0x00000143,0x00050081,0x00000009,
0x00000145,0x00000140,0x00000144,0x0004003d,0x00000006,0x00000146,0x000000e9,0x0004003d,
0x00000006,0x00000147,0x00000103,0x0004003d,0x00000006,0x00000148,0x000000e9,0x00050083,
0x00000006,0x00000149,0x0000004d,0x00000148,0x00050085,0x00000006,0x0000014a,0x00000147,
0x00000149,0x00050081,0x00000006,0x0000014b,0x00000146,0x0000014a,0x00050051,0x00000006,
0x0000014c,0x00000145,0x00000000,0x00050051,0x00000006,0x0000014d,0x00000145,0x00000001,
0x00050051,0x00000006,0x0000014e,0x00000145,0x00000002,0x00070050,0x00000007,0x0000014f,
0x0000014c,0x0000014d,0x0000014e,0x0000014b,0x0003003e,0x0000013f,0x0000014f,0x0004003d,
0x00000007,0x00000152,0x0000013f,0x0003003e,0x00000151,0x00000152,0x000100fd,0x00010038,
0x00050036,0x00000009,0x0000000e,0x00000000,0x0000000b,0x00030037,0x00000008,0x0000000c,
0x00030037,0x0000000a,0x0000000d,0x000200f8,0x0000000f,0x0004003d,0x00000009,0x00000010,
0x0000000d,0x0004003d,0x00000007,0x00000012,0x0000000c,0x0008004f,0x00000009,0x00000013,
0x00000012,0x00000012,0x00000000,0x00000001,0x00000002,0x0004003d,0x00000007,0x00000014,
0x0000000c,0x0008004f,0x00000009,0x00000015,0x00000014,0x00000014,0x00000000,0x00000001,
0x00000002,0x0004003d,0x00000009,0x00000016,0x0000000d,0x0007000c,0x00000009,0x00000017,
0x00000001,0x00000044,0x00000015,0x00000016,0x00050041,0x0000001a,0x0000001b,0x0000000c,
0x00000019,0x0004003d,0x00000006,0x0000001c,0x0000001b,0x0004003d,0x00000009,0x0000001d,
0x0000000d,0x0005008e,0x00000009,0x0000001e,0x0000001d,0x0000001c,0x00050081,0x00000009,
0x0000001f,0x00000017,0x0000001e,0x0007000c,0x00000009,0x00000020,0x00000001,0x00000044,
0x00000013,0x0000001f,0x0005008e,0x00000009,0x00000021,0x00000020,0x00000011,0x00050081,
0x00000009,0x00000022,0x00000010,0x00000021,0x000200fe,0x00000022,0x00010038,
};
+331
View File
@@ -0,0 +1,331 @@
// SPDX-License-Identifier: GPL-3.0-only
// Decoder -> AImageReader -> fused GLES YUV conversion/projection -> shared RGBA
// eye views -> Vulkan copy to the runtime. The original decoded image is not
// downscaled or converted into a full-size intermediate. No picture readback or
// CPU colour conversion; no YCbCr feature forced onto the application's device.
// Published buffers are retained until the Vulkan fence completes. The full
// panorama import path remains available to the independent reference fixture.
static struct {
media_status_t (*reader_new)(int32_t,int32_t,int32_t,uint64_t,int32_t,AImageReader**);
media_status_t (*reader_window)(AImageReader*,ANativeWindow**);
media_status_t (*reader_latest)(AImageReader*,AImage**);
media_status_t (*reader_next)(AImageReader*,AImage**);
media_status_t (*image_time)(const AImage*,int64_t*);
void (*reader_delete)(AImageReader*);
media_status_t (*image_buffer)(const AImage*,AHardwareBuffer**);
void (*image_delete)(AImage*);
int (*allocate)(const AHardwareBuffer_Desc*,AHardwareBuffer**);
void (*describe)(const AHardwareBuffer*,AHardwareBuffer_Desc*);
void (*release)(AHardwareBuffer*);
jobject (*to_surface)(JNIEnv*,ANativeWindow*);
PFNEGLGETNATIVECLIENTBUFFERANDROIDPROC client;
PFNEGLCREATEIMAGEKHRPROC create;
PFNEGLDESTROYIMAGEKHRPROC destroy;
PFNGLEGLIMAGETARGETTEXTURE2DOESPROC bind;
void (*finish)(void);
PFN_vkGetAndroidHardwareBufferPropertiesANDROID properties;
PFN_vkCreateImage create_image;
PFN_vkDestroyImage destroy_image;
PFN_vkAllocateMemory allocate_memory;
PFN_vkFreeMemory free_memory;
PFN_vkBindImageMemory bind_memory;
PFN_vkCreateImageView create_view;
PFN_vkDestroyImageView destroy_view;
} sn;
static int surf_native_load(void) {
if(!vk.GetDeviceProcAddr)return 0;
sn.properties=(PFN_vkGetAndroidHardwareBufferPropertiesANDROID)vk.GetDeviceProcAddr(vk.device,
"vkGetAndroidHardwareBufferPropertiesANDROID");
if(!sn.properties){LOG("surface_native: external-memory device extension unavailable");return 0;}
void *media=dlopen("libmediandk.so",RTLD_NOW|RTLD_LOCAL);
void *window=dlopen("libnativewindow.so",RTLD_NOW|RTLD_LOCAL);
void *gles=dlopen("libGLESv3.so",RTLD_NOW|RTLD_LOCAL);
if(!media || !window || !gles){LOG("surface_native: missing platform library: %s",dlerror());return 0;}
#define SN_SYM(field,lib,name) *(void**)&sn.field=dlsym(lib,name);if(!sn.field){LOG("surface_native: missing %s",name);return 0;}
SN_SYM(reader_new,media,"AImageReader_newWithUsage")
SN_SYM(reader_window,media,"AImageReader_getWindow")
SN_SYM(reader_latest,media,"AImageReader_acquireLatestImage")
SN_SYM(reader_next,media,"AImageReader_acquireNextImage")
SN_SYM(image_time,media,"AImage_getTimestamp")
SN_SYM(reader_delete,media,"AImageReader_delete")
SN_SYM(image_buffer,media,"AImage_getHardwareBuffer") SN_SYM(image_delete,media,"AImage_delete")
SN_SYM(allocate,window,"AHardwareBuffer_allocate") SN_SYM(describe,window,"AHardwareBuffer_describe")
SN_SYM(release,window,"AHardwareBuffer_release")
#ifndef SURF_NATIVE_ALLOCATOR_TEST
// A native GPU fixture has no Java VM; only the application needs this bridge.
void *android=dlopen("libandroid.so",RTLD_NOW|RTLD_LOCAL);
if(!android)return 0;
SN_SYM(to_surface,android,"ANativeWindow_toSurface")
#endif
SN_SYM(finish,gles,"glFinish")
#undef SN_SYM
#define SN_VK(field,name) sn.field=(PFN_vk##name)vk.GetDeviceProcAddr(vk.device,"vk" #name);if(!sn.field)return 0;
SN_VK(create_image,CreateImage) SN_VK(destroy_image,DestroyImage) SN_VK(allocate_memory,AllocateMemory)
SN_VK(free_memory,FreeMemory) SN_VK(bind_memory,BindImageMemory)
SN_VK(create_view,CreateImageView) SN_VK(destroy_view,DestroyImageView)
#undef SN_VK
void *egl=dlopen("libEGL.so",RTLD_NOW|RTLD_LOCAL);
__eglMustCastToProperFunctionPointerType (*get)(const char*)=egl?
(__eglMustCastToProperFunctionPointerType (*)(const char*))dlsym(egl,"eglGetProcAddress"):NULL;
if(!get)return 0;
sn.client=(PFNEGLGETNATIVECLIENTBUFFERANDROIDPROC)get("eglGetNativeClientBufferANDROID");
sn.create=(PFNEGLCREATEIMAGEKHRPROC)get("eglCreateImageKHR");
sn.destroy=(PFNEGLDESTROYIMAGEKHRPROC)get("eglDestroyImageKHR");
sn.bind=(PFNGLEGLIMAGETARGETTEXTURE2DOESPROC)get("glEGLImageTargetTexture2DOES");
return sn.client && sn.create && sn.destroy && sn.bind;
}
static void surf_native_slot_destroy(surf_native_slot *slot) {
if(slot->view)sn.destroy_view(vk.device,slot->view,NULL);
if(slot->image)sn.destroy_image(vk.device,slot->image,NULL);
if(slot->memory)sn.free_memory(vk.device,slot->memory,NULL);
if(slot->buffer)sn.release(slot->buffer);
slot->view=VK_NULL_HANDLE;slot->image=VK_NULL_HANDLE;slot->memory=VK_NULL_HANDLE;slot->buffer=NULL;
}
static int surf_native_allocate(surf_native_slot *slot,EGLDisplay display,uint32_t width,uint32_t height) {
AHardwareBuffer_Desc desc={.width=width,.height=height,.layers=1,
.format=AHARDWAREBUFFER_FORMAT_R8G8B8A8_UNORM,
.usage=AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE|AHARDWAREBUFFER_USAGE_GPU_COLOR_OUTPUT};
if(sn.allocate(&desc,&slot->buffer))return 0;
VkAndroidHardwareBufferFormatPropertiesANDROID format={VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_FORMAT_PROPERTIES_ANDROID};
VkAndroidHardwareBufferPropertiesANDROID props={VK_STRUCTURE_TYPE_ANDROID_HARDWARE_BUFFER_PROPERTIES_ANDROID,&format};
if(sn.properties(vk.device,slot->buffer,&props)!=VK_SUCCESS || format.format!=VK_FORMAT_R8G8B8A8_UNORM)goto fail;
VkExternalMemoryImageCreateInfo external={VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_IMAGE_CREATE_INFO,NULL,
VK_EXTERNAL_MEMORY_HANDLE_TYPE_ANDROID_HARDWARE_BUFFER_BIT_ANDROID};
VkImageCreateInfo image={VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO,&external,0,VK_IMAGE_TYPE_2D,format.format,
{width,height,1},1,1,VK_SAMPLE_COUNT_1_BIT,VK_IMAGE_TILING_OPTIMAL,
VK_IMAGE_USAGE_SAMPLED_BIT|VK_IMAGE_USAGE_COLOR_ATTACHMENT_BIT|VK_IMAGE_USAGE_TRANSFER_SRC_BIT,
VK_SHARING_MODE_EXCLUSIVE,0,NULL,VK_IMAGE_LAYOUT_UNDEFINED};
if(sn.create_image(vk.device,&image,NULL,&slot->image)!=VK_SUCCESS)goto fail;
uint32_t type=0;while(type<32 && !(props.memoryTypeBits&(1u<<type)))type++;
if(type==32)goto fail;
VkImportAndroidHardwareBufferInfoANDROID imported={VK_STRUCTURE_TYPE_IMPORT_ANDROID_HARDWARE_BUFFER_INFO_ANDROID,NULL,slot->buffer};
VkMemoryDedicatedAllocateInfo dedicated={VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO,&imported,slot->image,VK_NULL_HANDLE};
VkMemoryAllocateInfo alloc={VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,&dedicated,props.allocationSize,type};
if(sn.allocate_memory(vk.device,&alloc,NULL,&slot->memory)!=VK_SUCCESS ||
sn.bind_memory(vk.device,slot->image,slot->memory,0)!=VK_SUCCESS)goto fail;
VkImageViewCreateInfo view={VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,NULL,0,slot->image,VK_IMAGE_VIEW_TYPE_2D,
format.format,{0,0,0,0},{VK_IMAGE_ASPECT_COLOR_BIT,0,1,0,1}};
if(sn.create_view(vk.device,&view,NULL,&slot->view)!=VK_SUCCESS)goto fail;
slot->egl_image=sn.create(display,EGL_NO_CONTEXT,EGL_NATIVE_BUFFER_ANDROID,sn.client(slot->buffer),NULL);
if(slot->egl_image==EGL_NO_IMAGE_KHR)goto fail;
p_glGenTextures(1,&slot->texture);p_glBindTexture(GL_TEXTURE_2D,slot->texture);
sn.bind(GL_TEXTURE_2D,slot->egl_image);
slot->width=width;slot->height=height;
return p_glGetError()==GL_NO_ERROR;
fail:
surf_native_slot_destroy(slot);return 0;
}
// Runs on the surface worker's EGL context. The pose snapshot is published
// together with the completed image, so the compositor can timewarp correctly.
static void surf_video_draw(GLuint program,const GLint uniforms[8],GLuint external,
const surf_video_job *job) {
p_glUseProgram(program);p_glActiveTexture(GL_TEXTURE0);p_glBindTexture(GL_TEXTURE_EXTERNAL_OES,external);
for(int eye=0;eye<2;eye++){
p_glViewport((GLint)(eye*job->size),0,(GLsizei)job->size,(GLsizei)job->size);
for(int i=0;i<8;i++){
const float *v=job->params[eye]+4*i;
p_glUniform4f(uniforms[i],v[0],v[1],v[2],v[3]);
}
p_glDrawArrays(GL_TRIANGLES,0,3);
}
}
#if !defined(SURF_NATIVE_ALLOCATOR_TEST) || defined(SURF_NATIVE_WORKER_TEST)
static void surf_native_worker(surf_swapchain *s,JNIEnv *env,EGLDisplay display) {
AImageReader *reader=NULL;ANativeWindow *window=NULL;
GLuint program=0,external=0,fbo=0,vao=0;
enum {PENDING_IMAGES=4,ACQUIRED_IMAGES=5};
AImage *held=NULL,*pending[PENDING_IMAGES]={0};int64_t pending_time[PENDING_IMAGES]={0};int pending_count=0;
EGLImageKHR input=EGL_NO_IMAGE_KHR;
AHardwareBuffer_Desc desc={0};uint64_t last_job=0;GLint uniforms[8]={0};
int64_t source_time=0;
surf_video_job rendered_job={0};int video_dirty=0;
#ifdef SURF_NATIVE_WORKER_TEST
// A native fixture supplies the real reader/codec without a Java VM.
(void)env;reader=surf_worker_test_reader;
#else
if(sn.reader_new((int32_t)s->width,(int32_t)s->height,AIMAGE_FORMAT_PRIVATE,
AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE,s->native_project?ACQUIRED_IMAGES:3,&reader)!=AMEDIA_OK ||
sn.reader_window(reader,&window)!=AMEDIA_OK)goto fail;
jobject local=sn.to_surface(env,window);
if(!local || (*env)->ExceptionCheck(env)){(*env)->ExceptionClear(env);goto fail;}
s->surface=(*env)->NewGlobalRef(env,local);(*env)->DeleteLocalRef(env,local);
if(!s->surface)goto fail;
#endif
const char *fragment="#version 300 es\n#extension GL_OES_EGL_image_external_essl3 : require\n"
"precision highp float;uniform samplerExternalOES tex;in vec2 uv;out vec4 color;"
"void main(){color=texture(tex,uv);}";
GLuint vertex=emul_compile(GL_VERTEX_SHADER,"",surf_vs),pixel=emul_compile(GL_FRAGMENT_SHADER,"",
s->native_project?surf_video_fragment:fragment);
if(!vertex || !pixel)goto fail;
program=p_glCreateProgram();p_glAttachShader(program,vertex);p_glAttachShader(program,pixel);p_glLinkProgram(program);
GLint ok=0;p_glGetProgramiv(program,GL_LINK_STATUS,&ok);if(!ok)goto fail;
if(s->native_project)for(int i=0;i<8;i++)uniforms[i]=p_glGetUniformLocation(program,surf_video_uniforms[i]);
p_glGenTextures(1,&external);p_glBindTexture(GL_TEXTURE_EXTERNAL_OES,external);
s_glTexParameteri(GL_TEXTURE_EXTERNAL_OES,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
s_glTexParameteri(GL_TEXTURE_EXTERNAL_OES,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
s_glTexParameteri(GL_TEXTURE_EXTERNAL_OES,GL_TEXTURE_WRAP_S,GL_CLAMP_TO_EDGE);
s_glTexParameteri(GL_TEXTURE_EXTERNAL_OES,GL_TEXTURE_WRAP_T,GL_CLAMP_TO_EDGE);
p_glGenFramebuffers(1,&fbo);p_glGenVertexArrays(1,&vao);
pthread_mutex_lock(&surf_lock);s->ready=1;pthread_cond_broadcast(&surf_cond);pthread_mutex_unlock(&surf_lock);
LOG("surface_native: shared GPU video surface ready (%s)",s->native_project?"fused visible-view conversion":"full panorama");
int64_t start=0,frames=0,views=0,draw_ns=0,reused=0,held_views=0,anchored_views=0,recentred_views=0;
while(!s->stop){
int index=-1;
surf_video_job job;
pthread_mutex_lock(&surf_lock);
if(s->video_job.seq==last_job && !s->stop){
struct timespec until;clock_gettime(CLOCK_REALTIME,&until);until.tv_nsec+=1000000;
if(until.tv_nsec>=1000000000){until.tv_sec++;until.tv_nsec-=1000000000;}
pthread_cond_timedwait(&surf_cond,&surf_lock,&until);
}
job=s->video_job;
for(int i=0;i<SURF_NATIVE_SLOTS;i++)if(s->native_slots[i].state==0){index=i;s->native_slots[i].state=1;break;}
pthread_mutex_unlock(&surf_lock);
if(index<0){struct timespec pause={0,250000};nanosleep(&pause,NULL);continue;}
surf_native_slot *slot=&s->native_slots[index];
AImage *image=NULL;
if(s->native_project){
// Acquire future pictures promptly: leaving them in a droppable
// producer queue loses pictures before their presentation time.
// Keep at most four future pictures plus the currently sampled one.
int64_t now=monotonic_ns(),target=surf_video_target(now,job.time);
while(pending_count && !surf_video_future(pending_time[0],now,target)){
if(image)sn.image_delete(image);
image=pending[0];pending_count--;
memmove(pending,pending+1,pending_count*sizeof(*pending));
memmove(pending_time,pending_time+1,pending_count*sizeof(*pending_time));
}
while(pending_count<PENDING_IMAGES && pending_count+(held!=NULL)+(image!=NULL)<ACQUIRED_IMAGES){
AImage *next=NULL;if(sn.reader_next(reader,&next)!=AMEDIA_OK)break;
int64_t timestamp=0;sn.image_time(next,&timestamp);
if(surf_video_future(timestamp,now,target)){
pending[pending_count]=next;pending_time[pending_count++]=timestamp;
}else{
if(image)sn.image_delete(image);image=next;
}
}
}else sn.reader_latest(reader,&image);
int new_image=image!=NULL;
if(new_image){
// Keep the decoder image acquired while GLES might sample it again
// for head movement. Releasing it sooner lets the decoder overwrite it.
if(input!=EGL_NO_IMAGE_KHR)sn.destroy(display,input);
if(held)sn.image_delete(held);
held=image;input=EGL_NO_IMAGE_KHR;
sn.image_time(held,&source_time);
AHardwareBuffer *buffer=NULL;
int valid=sn.image_buffer(held,&buffer)==AMEDIA_OK;
if(valid)sn.describe(buffer,&desc);
if(!valid || !desc.width || !desc.height || desc.width>8192 || desc.height>8192)goto fail;
input=sn.create(display,EGL_NO_CONTEXT,EGL_NATIVE_BUFFER_ANDROID,sn.client(buffer),NULL);
if(input==EGL_NO_IMAGE_KHR)goto fail;
p_glActiveTexture(GL_TEXTURE0);p_glBindTexture(GL_TEXTURE_EXTERNAL_OES,external);sn.bind(GL_TEXTURE_EXTERNAL_OES,input);
frames++;video_dirty=1;
}
// Importing/acquiring a decoder buffer can take time. Pick the newest
// request immediately before drawing instead of keeping the earlier pose.
pthread_mutex_lock(&surf_lock);job=s->video_job;pthread_mutex_unlock(&surf_lock);
// Coalesce video arrivals into the next application view request. Drawing
// on both clocks would do up to video-fps + headset-fps work per second.
if(!held || (s->native_project?(!job.size || job.seq==last_job):!new_image)){
pthread_mutex_lock(&surf_lock);slot->state=0;pthread_mutex_unlock(&surf_lock);continue;
}
if(s->native_project && !video_dirty && surf_video_covers(&rendered_job,&job)){
last_job=job.seq;reused++;
pthread_mutex_lock(&surf_lock);slot->state=0;pthread_mutex_unlock(&surf_lock);continue;
}
uint32_t width=s->native_project?2*job.size:desc.width,height=s->native_project?job.size:desc.height;
if(!slot->buffer || slot->width!=width || slot->height!=height){
if(slot->texture)p_glDeleteTextures(1,&slot->texture);
if(slot->egl_image)sn.destroy(display,slot->egl_image);
slot->texture=0;slot->egl_image=EGL_NO_IMAGE_KHR;surf_native_slot_destroy(slot);
if(!surf_native_allocate(slot,display,width,height))goto fail;
LOG("surface_native: %ux%u decoded input -> %ux%u shared view",desc.width,desc.height,width,height);
}
p_glBindFramebuffer(GL_FRAMEBUFFER,fbo);
p_glFramebufferTexture2D(GL_FRAMEBUFFER,GL_COLOR_ATTACHMENT0,GL_TEXTURE_2D,slot->texture,0);
if(p_glCheckFramebufferStatus(GL_FRAMEBUFFER)!=GL_FRAMEBUFFER_COMPLETE)goto fail;
p_glUseProgram(program);p_glUniform1i(p_glGetUniformLocation(program,"tex"),0);p_glBindVertexArray(vao);
int64_t draw_start=monotonic_ns();
surf_video_job draw=job;
if(s->native_project){
if(surf_video_anchor(&rendered_job,&job,&draw))anchored_views++;
else recentred_views++;
surf_video_draw(program,uniforms,external,&draw);
}
else {p_glViewport(0,0,(GLsizei)width,(GLsizei)height);p_glDrawArrays(GL_TRIANGLES,0,3);}
sn.finish(); // external writes completed before the Vulkan ownership acquire
if(!video_dirty)held_views++;
draw_ns+=monotonic_ns()-draw_start;views++;last_job=job.seq;rendered_job=draw;video_dirty=0;
GLenum error=p_glGetError();
if(error)goto fail;
pthread_mutex_lock(&surf_lock);
for(int i=0;i<SURF_NATIVE_SLOTS;i++)if(s->native_slots[i].state==2)s->native_slots[i].state=0;
slot->video=draw;slot->projected=s->native_project;slot->source_width=desc.width;slot->source_height=desc.height;
slot->source_time=source_time;
slot->seq=++s->seq;slot->state=2;s->frames++;
pthread_mutex_unlock(&surf_lock);
int64_t now=monotonic_ns();if(!start)start=now;
if(now-start>5000000000ll){LOG("surface_native: %.1f video frames/s, %.1f views/s, visible-view GPU wait %.2f ms, no picture readback",
frames*1e9/(now-start),views*1e9/(now-start),views?draw_ns/1e6/views:0);
if(s->native_project)LOG("surface_native: held-picture redraws=%lld, compositor reuse=%lld, panorama radius=%.3f",
(long long)held_views,(long long)reused,job.params[0][7]);
if(s->native_project)LOG("surface_native: fixed sampling grid=%lld views, recentered=%lld views",
(long long)anchored_views,(long long)recentred_views);
frames=views=draw_ns=reused=held_views=anchored_views=recentred_views=0;start=now;}
}
goto done;
fail:
surf_fail(s,"shared GPU video initialization/delivery failed");
done:
sn.finish();
if(input!=EGL_NO_IMAGE_KHR)sn.destroy(display,input);
if(held)sn.image_delete(held);
for(int i=0;i<pending_count;i++)sn.image_delete(pending[i]);
for(int i=0;i<SURF_NATIVE_SLOTS;i++){
surf_native_slot *slot=&s->native_slots[i];
if(slot->texture)p_glDeleteTextures(1,&slot->texture);
if(slot->egl_image)sn.destroy(display,slot->egl_image);
slot->texture=0;slot->egl_image=EGL_NO_IMAGE_KHR;
}
if(external)p_glDeleteTextures(1,&external);if(fbo)p_glDeleteFramebuffers(1,&fbo);
if(reader)sn.reader_delete(reader);
}
#endif
static void surf_native_complete(surf_swapchain *s) {
pthread_mutex_lock(&surf_lock);
if(s->native_pending>=0)s->native_slots[s->native_pending].state=0;
s->native_pending=-1;pthread_mutex_unlock(&surf_lock);
}
static void surf_native_ownership(VkCommandBuffer command,surf_native_slot *slot,int acquire) {
VkAccessFlags access=slot->projected?VK_ACCESS_TRANSFER_READ_BIT:VK_ACCESS_SHADER_READ_BIT;
VkImageLayout layout=slot->projected?VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL:VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL;
VkImageMemoryBarrier barrier={VK_STRUCTURE_TYPE_IMAGE_MEMORY_BARRIER,NULL,
acquire?0:access,acquire?access:0,
acquire?VK_IMAGE_LAYOUT_GENERAL:layout,
acquire?layout:VK_IMAGE_LAYOUT_GENERAL,
acquire?VK_QUEUE_FAMILY_FOREIGN_EXT:vk.family,acquire?vk.family:VK_QUEUE_FAMILY_FOREIGN_EXT,
slot->image,{VK_IMAGE_ASPECT_COLOR_BIT,0,1,0,1}};
vk.CmdPipelineBarrier(command,VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,VK_PIPELINE_STAGE_ALL_COMMANDS_BIT,
0,0,NULL,0,NULL,1,&barrier);
}
static void surf_native_retire(surf_swapchain *s) {
if(!s->native_mode || s->native_current<0)return;
// Teardown is the only blocking handoff. The worker is stopped before a
// swapchain is destroyed, so it cannot rewrite an image being retired.
VkCommandBufferBeginInfo begin={VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,NULL,
VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT,NULL};
vk.ResetCommandBuffer(s->cmd,0);vk.BeginCommandBuffer(s->cmd,&begin);
surf_native_ownership(s->cmd,&s->native_slots[s->native_current],0);
vk.EndCommandBuffer(s->cmd);
VkSubmitInfo submit={VK_STRUCTURE_TYPE_SUBMIT_INFO,NULL,0,NULL,NULL,1,&s->cmd,0,NULL};
if(vk.QueueSubmit(vk.queue,1,&submit,s->fence)==VK_SUCCESS){
vk.WaitForFences(vk.device,1,&s->fence,VK_TRUE,UINT64_MAX);vk.ResetFences(vk.device,1,&s->fence);
}
pthread_mutex_lock(&surf_lock);s->native_slots[s->native_current].state=0;
s->native_current=-1;pthread_mutex_unlock(&surf_lock);
}
static void surf_native_destroy(surf_swapchain *s) {
if(!s->native_mode)return;
for(int i=0;i<SURF_NATIVE_SLOTS;i++)surf_native_slot_destroy(&s->native_slots[i]);
}
+511
View File
@@ -0,0 +1,511 @@
// SPDX-License-Identifier: GPL-3.0-only
// Vulkan presentation for equirectangular Android video surfaces. The decoder's
// native GPU path projects directly from YUV on the surface worker, then copies
// the finished stereo view into a runtime swapchain. CPU-delivered/reference
// panoramas use a compute shader. No unsupported equirect layer reaches runtime.
#include "surface_projection_spv.h"
typedef struct {
XrSwapchain output;
VkImage images[MAX_IMAGES], source, rendered, video_cache;
VkImageView rendered_view, source_view, video_cache_view;
VkDeviceMemory memory, rendered_memory, video_cache_memory;
VkSampler sampler;
VkDescriptorPool descriptors;
VkDescriptorSetLayout set_layout;
VkDescriptorSet set;
VkPipelineLayout layout;
VkPipeline pipeline;
uint32_t count, size, width, height, acquired_index;
int acquired;
XrTime presented_time;
XrTime tracking_log_time;
uint64_t source_seq;
int source_ready, ready, view_source, occludes, cache_ready;
XrCompositionLayerProjection layer;
XrCompositionLayerProjectionView eyes[2];
surf_video_job presented_job, cached_job;
} surf_projection;
static surf_projection surf_projections[SURF_MAX];
static struct {
#define SP_FN(name) PFN_vk##name name;
SP_FN(CreateImage) SP_FN(DestroyImage) SP_FN(GetImageMemoryRequirements) SP_FN(BindImageMemory)
SP_FN(CreateImageView) SP_FN(DestroyImageView) SP_FN(CreateSampler) SP_FN(DestroySampler)
SP_FN(CreateDescriptorSetLayout) SP_FN(DestroyDescriptorSetLayout) SP_FN(CreateDescriptorPool)
SP_FN(DestroyDescriptorPool) SP_FN(AllocateDescriptorSets) SP_FN(UpdateDescriptorSets)
SP_FN(CreatePipelineLayout) SP_FN(DestroyPipelineLayout) SP_FN(CreateShaderModule) SP_FN(DestroyShaderModule)
SP_FN(CreateRenderPass) SP_FN(DestroyRenderPass) SP_FN(CreateFramebuffer) SP_FN(DestroyFramebuffer)
SP_FN(CreateGraphicsPipelines) SP_FN(CmdBeginRenderPass) SP_FN(CmdEndRenderPass)
SP_FN(CmdSetViewport) SP_FN(CmdSetScissor) SP_FN(CmdDraw)
SP_FN(CreateComputePipelines) SP_FN(DestroyPipeline) SP_FN(CmdBindPipeline) SP_FN(CmdBindDescriptorSets)
SP_FN(CmdPushConstants) SP_FN(CmdDispatch) SP_FN(CmdCopyImage) SP_FN(FreeMemory)
#undef SP_FN
int ready;
} spvk;
static int surf_projection_functions(void) {
if(spvk.ready)return spvk.ready>0;
spvk.ready=-1;
#define SP_LOAD(name) spvk.name=(PFN_vk##name)vk.GetDeviceProcAddr(vk.device,"vk" #name); if(!spvk.name)return 0;
SP_LOAD(CreateImage) SP_LOAD(DestroyImage) SP_LOAD(GetImageMemoryRequirements) SP_LOAD(BindImageMemory)
SP_LOAD(CreateImageView) SP_LOAD(DestroyImageView) SP_LOAD(CreateSampler) SP_LOAD(DestroySampler)
SP_LOAD(CreateDescriptorSetLayout) SP_LOAD(DestroyDescriptorSetLayout) SP_LOAD(CreateDescriptorPool)
SP_LOAD(DestroyDescriptorPool) SP_LOAD(AllocateDescriptorSets) SP_LOAD(UpdateDescriptorSets)
SP_LOAD(CreatePipelineLayout) SP_LOAD(DestroyPipelineLayout) SP_LOAD(CreateShaderModule) SP_LOAD(DestroyShaderModule)
SP_LOAD(CreateRenderPass) SP_LOAD(DestroyRenderPass) SP_LOAD(CreateFramebuffer) SP_LOAD(DestroyFramebuffer)
SP_LOAD(CreateGraphicsPipelines) SP_LOAD(CmdBeginRenderPass) SP_LOAD(CmdEndRenderPass)
SP_LOAD(CmdSetViewport) SP_LOAD(CmdSetScissor) SP_LOAD(CmdDraw)
SP_LOAD(CreateComputePipelines) SP_LOAD(DestroyPipeline) SP_LOAD(CmdBindPipeline) SP_LOAD(CmdBindDescriptorSets)
SP_LOAD(CmdPushConstants) SP_LOAD(CmdDispatch) SP_LOAD(CmdCopyImage) SP_LOAD(FreeMemory)
#undef SP_LOAD
spvk.ready=1;return 1;
}
// All images belong to the session device; failures leave cleanup-safe handles.
static int surf_private_image(uint32_t width,uint32_t height,uint32_t layers,VkFormat format,
VkImageUsageFlags usage,VkImage *image,VkDeviceMemory *memory,VkImageView *view) {
VkImageCreateInfo ci={VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO,NULL,0,VK_IMAGE_TYPE_2D,format,
{width,height,1},1,layers,VK_SAMPLE_COUNT_1_BIT,VK_IMAGE_TILING_OPTIMAL,usage,VK_SHARING_MODE_EXCLUSIVE};
if(spvk.CreateImage(vk.device,&ci,NULL,image)!=VK_SUCCESS)return 0;
VkMemoryRequirements req;spvk.GetImageMemoryRequirements(vk.device,*image,&req);
VkPhysicalDeviceMemoryProperties props;svk.GetPhysicalDeviceMemoryProperties(vk.physical,&props);
uint32_t type=UINT32_MAX;
for(uint32_t i=0;i<props.memoryTypeCount;i++)if((req.memoryTypeBits&(1u<<i)) &&
(props.memoryTypes[i].propertyFlags&VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)){type=i;break;}
if(type==UINT32_MAX)return 0;
VkMemoryAllocateInfo alloc={VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,NULL,req.size,type};
if(svk.AllocateMemory(vk.device,&alloc,NULL,memory)!=VK_SUCCESS ||
spvk.BindImageMemory(vk.device,*image,*memory,0)!=VK_SUCCESS)return 0;
VkImageViewCreateInfo vi={VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,NULL,0,*image,
layers>1?VK_IMAGE_VIEW_TYPE_2D_ARRAY:VK_IMAGE_VIEW_TYPE_2D,format,{0,0,0,0},
{VK_IMAGE_ASPECT_COLOR_BIT,0,1,0,layers}};
return spvk.CreateImageView(vk.device,&vi,NULL,view)==VK_SUCCESS;
}
static void surf_composite_destroy(surf_projection *p);
static int surf_composite_active(surf_projection *p);
static int surf_composite_candidate(const XrFrameEndInfo *info,const XrCompositionLayerBaseHeader *video);
static int surf_composite_cached_only(const XrFrameEndInfo *info,surf_projection *p,const surf_video_job *rendered,const surf_video_job *current);
static void surf_projection_destroy(surf_swapchain *s) {
surf_projection *p=&surf_projections[s-surfs];
if(!spvk.ready || spvk.ready<0)return;
if(s->in_flight) {
vk.WaitForFences(vk.device,1,&s->fence,VK_TRUE,UINT64_MAX);
vk.ResetFences(vk.device,1,&s->fence);s->in_flight=0;
}
surf_native_complete(s);
surf_native_retire(s);
surf_composite_destroy(p);
if(p->video_cache_view)spvk.DestroyImageView(vk.device,p->video_cache_view,NULL);
if(p->video_cache)spvk.DestroyImage(vk.device,p->video_cache,NULL);
if(p->video_cache_memory)spvk.FreeMemory(vk.device,p->video_cache_memory,NULL);
if(p->rendered_view)spvk.DestroyImageView(vk.device,p->rendered_view,NULL);
if(p->rendered)spvk.DestroyImage(vk.device,p->rendered,NULL);
if(p->rendered_memory)spvk.FreeMemory(vk.device,p->rendered_memory,NULL);
if(p->pipeline)spvk.DestroyPipeline(vk.device,p->pipeline,NULL);
if(p->layout)spvk.DestroyPipelineLayout(vk.device,p->layout,NULL);
if(p->descriptors)spvk.DestroyDescriptorPool(vk.device,p->descriptors,NULL);
if(p->set_layout)spvk.DestroyDescriptorSetLayout(vk.device,p->set_layout,NULL);
if(p->sampler)spvk.DestroySampler(vk.device,p->sampler,NULL);
if(p->source_view)spvk.DestroyImageView(vk.device,p->source_view,NULL);
if(p->source)spvk.DestroyImage(vk.device,p->source,NULL);
if(p->memory)spvk.FreeMemory(vk.device,p->memory,NULL);
if(p->output) {
PFN_xrDestroySwapchain destroy=(PFN_xrDestroySwapchain)lookup(active_instance,"xrDestroySwapchain");
if(destroy)destroy(p->output);
}
memset(p,0,sizeof(*p));
}
static int surf_projection_setup(XrSession session,surf_swapchain *s,surf_projection *p,uint32_t width,uint32_t height) {
if(p->pipeline || (s->native_project && p->output))return 1;
if(!surf_vk_ready() || !surf_staging(s) || !surf_projection_functions())return 0;
const char *stage="runtime swapchain";
p->size=equirect_res>=512 && equirect_res<=4096?(uint32_t)equirect_res:1536;
if(s->native_mode && !equirect_res_set)p->size=2560;
p->width=width;p->height=height;
XrSwapchainCreateInfo ci={XR_TYPE_SWAPCHAIN_CREATE_INFO,NULL,0,
XR_SWAPCHAIN_USAGE_TRANSFER_DST_BIT|XR_SWAPCHAIN_USAGE_COLOR_ATTACHMENT_BIT|XR_SWAPCHAIN_USAGE_SAMPLED_BIT,
VK_FORMAT_R8G8B8A8_SRGB,1,2*p->size,p->size,1,1,1};
PFN_xrCreateSwapchain create=(PFN_xrCreateSwapchain)lookup(active_instance,"xrCreateSwapchain");
PFN_xrEnumerateSwapchainImages enumerate=(PFN_xrEnumerateSwapchainImages)lookup(active_instance,"xrEnumerateSwapchainImages");
if(!create || !enumerate)goto fail;
XrResult xr_result=create(session,&ci,&p->output);
if(XR_FAILED(xr_result)) {
static int create_failures;
if(create_failures++<5)LOG("surface_projection: swapchain creation result=%d",xr_result);
goto fail;
}
stage="runtime image enumeration";
XrSwapchainImageVulkanKHR images[MAX_IMAGES];
for(int i=0;i<MAX_IMAGES;i++)images[i]=(XrSwapchainImageVulkanKHR){XR_TYPE_SWAPCHAIN_IMAGE_VULKAN_KHR,NULL,VK_NULL_HANDLE};
uint32_t image_count=0;
if(XR_FAILED(enumerate(p->output,MAX_IMAGES,&image_count,(XrSwapchainImageBaseHeader*)images)) ||
!image_count || image_count>MAX_IMAGES)goto fail;
p->count=image_count;
for(uint32_t i=0;i<p->count;i++)p->images[i]=images[i].image;
if(s->native_project){
stage="video composition cache";
if(!surf_private_image(2*p->size,p->size,1,VK_FORMAT_R8G8B8A8_SRGB,
VK_IMAGE_USAGE_TRANSFER_DST_BIT|VK_IMAGE_USAGE_SAMPLED_BIT,
&p->video_cache,&p->video_cache_memory,&p->video_cache_view))goto fail;
LOG("surface_projection: fused GPU video view ready (%ux%u per eye)",p->size,p->size);
return 1;
}
stage="source image";
VkImageCreateInfo image={VK_STRUCTURE_TYPE_IMAGE_CREATE_INFO,NULL,0,VK_IMAGE_TYPE_2D,VK_FORMAT_R8G8B8A8_UNORM,
{width,height,1},1,1,VK_SAMPLE_COUNT_1_BIT,VK_IMAGE_TILING_OPTIMAL,
VK_IMAGE_USAGE_TRANSFER_DST_BIT|VK_IMAGE_USAGE_SAMPLED_BIT,VK_SHARING_MODE_EXCLUSIVE,0,NULL,VK_IMAGE_LAYOUT_UNDEFINED};
VkMemoryRequirements req;
VkPhysicalDeviceMemoryProperties props;svk.GetPhysicalDeviceMemoryProperties(vk.physical,&props);
uint32_t type=UINT32_MAX;
VkMemoryAllocateInfo memory;
VkImageViewCreateInfo view={VK_STRUCTURE_TYPE_IMAGE_VIEW_CREATE_INFO,NULL,0,p->source,VK_IMAGE_VIEW_TYPE_2D,
VK_FORMAT_R8G8B8A8_UNORM,{0,0,0,0},{VK_IMAGE_ASPECT_COLOR_BIT,0,1,0,1}};
if(!s->native_mode){
if(spvk.CreateImage(vk.device,&image,NULL,&p->source)!=VK_SUCCESS)goto fail;
spvk.GetImageMemoryRequirements(vk.device,p->source,&req);
for(uint32_t i=0;i<props.memoryTypeCount;i++)if((req.memoryTypeBits&(1u<<i)) &&
(props.memoryTypes[i].propertyFlags&VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)){type=i;break;}
if(type==UINT32_MAX)goto fail;
memory=(VkMemoryAllocateInfo){VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,NULL,req.size,type};
if(svk.AllocateMemory(vk.device,&memory,NULL,&p->memory)!=VK_SUCCESS ||
spvk.BindImageMemory(vk.device,p->source,p->memory,0)!=VK_SUCCESS)goto fail;
view.image=p->source;
if(spvk.CreateImageView(vk.device,&view,NULL,&p->source_view)!=VK_SUCCESS)goto fail;
}
// The runtime supports sRGB swapchains, which cannot be storage images.
// Compute into a private UNORM image and copy its encoded pixels unchanged.
stage="private storage image";
image.extent=(VkExtent3D){2*p->size,p->size,1};
image.usage=VK_IMAGE_USAGE_STORAGE_BIT|VK_IMAGE_USAGE_TRANSFER_SRC_BIT;
if(spvk.CreateImage(vk.device,&image,NULL,&p->rendered)!=VK_SUCCESS)goto fail;
spvk.GetImageMemoryRequirements(vk.device,p->rendered,&req);
type=UINT32_MAX;
for(uint32_t i=0;i<props.memoryTypeCount;i++)if((req.memoryTypeBits&(1u<<i)) &&
(props.memoryTypes[i].propertyFlags&VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT)){type=i;break;}
if(type==UINT32_MAX)goto fail;
memory=(VkMemoryAllocateInfo){VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO,NULL,req.size,type};
if(svk.AllocateMemory(vk.device,&memory,NULL,&p->rendered_memory)!=VK_SUCCESS ||
spvk.BindImageMemory(vk.device,p->rendered,p->rendered_memory,0)!=VK_SUCCESS)goto fail;
view.image=p->rendered;
if(spvk.CreateImageView(vk.device,&view,NULL,&p->rendered_view)!=VK_SUCCESS)goto fail;
stage="sampler and descriptors";
VkSamplerCreateInfo sampler={VK_STRUCTURE_TYPE_SAMPLER_CREATE_INFO,NULL,0,VK_FILTER_LINEAR,VK_FILTER_LINEAR,
VK_SAMPLER_MIPMAP_MODE_NEAREST,VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE,VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE,
VK_SAMPLER_ADDRESS_MODE_CLAMP_TO_EDGE,0,VK_FALSE,1,VK_FALSE,VK_COMPARE_OP_ALWAYS,0,0,VK_BORDER_COLOR_FLOAT_TRANSPARENT_BLACK,VK_FALSE};
if(spvk.CreateSampler(vk.device,&sampler,NULL,&p->sampler)!=VK_SUCCESS)goto fail;
VkDescriptorSetLayoutBinding bindings[2]={{0,VK_DESCRIPTOR_TYPE_STORAGE_IMAGE,1,VK_SHADER_STAGE_COMPUTE_BIT,NULL},
{1,VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,1,VK_SHADER_STAGE_COMPUTE_BIT,NULL}};
VkDescriptorSetLayoutCreateInfo set={VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO,NULL,0,2,bindings};
if(spvk.CreateDescriptorSetLayout(vk.device,&set,NULL,&p->set_layout)!=VK_SUCCESS)goto fail;
VkDescriptorPoolSize sizes[2]={{VK_DESCRIPTOR_TYPE_STORAGE_IMAGE,1},{VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER,1}};
VkDescriptorPoolCreateInfo pool={VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO,NULL,0,1,2,sizes};
if(spvk.CreateDescriptorPool(vk.device,&pool,NULL,&p->descriptors)!=VK_SUCCESS)goto fail;
VkDescriptorSetAllocateInfo alloc={VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO,NULL,p->descriptors,1,&p->set_layout};
if(spvk.AllocateDescriptorSets(vk.device,&alloc,&p->set)!=VK_SUCCESS)goto fail;
stage="compute pipeline";
VkPushConstantRange range={VK_SHADER_STAGE_COMPUTE_BIT,0,128};
VkPipelineLayoutCreateInfo layout={VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO,NULL,0,1,&p->set_layout,1,&range};
if(spvk.CreatePipelineLayout(vk.device,&layout,NULL,&p->layout)!=VK_SUCCESS)goto fail;
VkShaderModule module;
VkShaderModuleCreateInfo shader={VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO,NULL,0,sizeof(surface_projection_spv),surface_projection_spv};
if(spvk.CreateShaderModule(vk.device,&shader,NULL,&module)!=VK_SUCCESS)goto fail;
VkComputePipelineCreateInfo pipeline={VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO,NULL,0,
{VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO,NULL,0,VK_SHADER_STAGE_COMPUTE_BIT,module,"main",NULL},p->layout,VK_NULL_HANDLE,0};
VkResult result=spvk.CreateComputePipelines(vk.device,VK_NULL_HANDLE,1,&pipeline,NULL,&p->pipeline);
spvk.DestroyShaderModule(vk.device,module,NULL);
if(result!=VK_SUCCESS)goto fail;
LOG("surface_projection: Vulkan stereo panorama renderer ready (%ux%u per eye)",p->size,p->size);return 1;
fail:;
static int failures;
if(failures++<5)LOG("surface_projection: setup failed at %s; retaining original layer behavior",stage);
surf_projection_destroy(s);return 0;
}
// Build both eyes at once. The worker carries this exact pose/space snapshot
// with its finished image rather than labelling an old image with a newer pose.
static int surf_video_request(XrSession session,const XrFrameEndInfo *info,
const XrCompositionLayerBaseHeader *layer,surf_swapchain *s,surf_projection *p) {
emul_common common=emul_common_of(layer);
pthread_mutex_lock(&surf_lock);
int same=s->video_job.seq && s->video_job.time==info->displayTime && s->video_job.space==common.space;
pthread_mutex_unlock(&surf_lock);
if(same)return 1;
surf_video_job job={.time=info->displayTime,.space=common.space,.flags=layer->layerFlags,.size=p->size};
job.views[0].type=job.views[1].type=XR_TYPE_VIEW;
int source=surf_views_select(session,info,common.space,
(PFN_xrLocateSpace)lookup(active_instance,"xrLocateSpace"),job.views);
if(!source){
PFN_xrLocateViews locate=(PFN_xrLocateViews)lookup(active_instance,"xrLocateViews");
XrViewLocateInfo li={XR_TYPE_VIEW_LOCATE_INFO,NULL,XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO,info->displayTime,common.space};
XrViewState state={XR_TYPE_VIEW_STATE};uint32_t count=0;
if(!locate || XR_FAILED(locate(session,&li,&state,2,&count,job.views)) || count!=2 ||
!(state.viewStateFlags&XR_VIEW_STATE_ORIENTATION_VALID_BIT))return 0;
source=3;
}
if(p->view_source!=source){
LOG("surface_projection: views from %s",source==1?"submitted projection":source==2?"application locate":"runtime locate");
p->view_source=source;
}
for(int eye=0;eye<2;eye++){
const XrCompositionLayerBaseHeader *selected=layer;
for(uint32_t i=0;i<info->layerCount;i++)if(emul_is_equirect(info->layers[i])){
emul_common c=emul_common_of(info->layers[i]);
if(c.sub.swapchain==s->handle && c.space==common.space && (c.eye==XR_EYE_VISIBILITY_BOTH ||
c.eye==(eye?XR_EYE_VISIBILITY_RIGHT:XR_EYE_VISIBILITY_LEFT))){selected=info->layers[i];break;}
}
emul_common c=emul_common_of(selected);XrView *view=&job.views[eye];
job.panorama[eye]=c.pose;
view->fov=surf_video_guard_fov(view->fov);
XrQuaternionf rotation=lm_qmul(lm_qconj(c.pose.orientation),view->pose.orientation);
XrVector3f origin=lm_rotate(lm_qconj(c.pose.orientation),(XrVector3f){view->pose.position.x-c.pose.position.x,
view->pose.position.y-c.pose.position.y,view->pose.position.z-c.pose.position.z});
float params[32]={rotation.x,rotation.y,rotation.z,rotation.w,origin.x,origin.y,origin.z,0,
tanf(view->fov.angleLeft),tanf(view->fov.angleRight),tanf(view->fov.angleDown),tanf(view->fov.angleUp),
c.sub.imageRect.offset.x/(float)s->width,c.sub.imageRect.offset.y/(float)s->height,
c.sub.imageRect.extent.width/(float)s->width,c.sub.imageRect.extent.height/(float)s->height,
0,0,0,(float)eye,1,1,0,0,1,1,1,1,0,0,0,0};
if(selected->type==XR_TYPE_COMPOSITION_LAYER_EQUIRECT2_KHR){
const XrCompositionLayerEquirect2KHR *e=(const void*)selected;
params[7]=e->radius;params[16]=e->centralHorizontalAngle;params[17]=e->upperVerticalAngle;params[18]=e->lowerVerticalAngle;
}else{
const XrCompositionLayerEquirectKHR *e=(const void*)selected;
params[7]=e->radius;params[20]=e->scale.x;params[21]=e->scale.y;params[22]=e->bias.x;params[23]=e->bias.y;
}
for(const XrBaseInStructure *n=selected->next;n;n=n->next)if(n->type==XR_TYPE_COMPOSITION_LAYER_COLOR_SCALE_BIAS_KHR){
const XrCompositionLayerColorScaleBiasKHR *color=(const void*)n;
memcpy(params+24,&color->colorScale,16);memcpy(params+28,&color->colorBias,16);
}
memcpy(job.params[eye],params,sizeof(params));
}
pthread_mutex_lock(&surf_lock);job.seq=s->video_job.seq+1;s->video_job=job;
pthread_cond_broadcast(&surf_cond);pthread_mutex_unlock(&surf_lock);
return 1;
}
static const XrCompositionLayerBaseHeader *surf_video_layer(surf_projection *p,const surf_video_job *current) {
if(!p->ready)return NULL;
int occludes=surf_video_occludes(&p->presented_job,current);
if(occludes!=p->occludes && !surf_composite_active(p))LOG("surface_projection: opaque primary=%d (fades and uncovered views retain lower layers)",occludes);
p->occludes=occludes;
for(int eye=0;eye<2;eye++)p->eyes[eye].pose=occludes?
surf_video_primary_pose(&p->presented_job,current,eye):surf_video_pose(&p->presented_job,current,eye);
if(current->time-p->tracking_log_time>=5000000000ll){
XrQuaternionf correction=lm_qmul(lm_qconj(p->eyes[0].pose.orientation),current->views[0].pose.orientation);
XrQuaternionf layer_delta=lm_qmul(lm_qconj(p->presented_job.panorama[0].orientation),current->panorama[0].orientation);
LOG("surface_projection: render age=%.2f ms, head correction=%.2f deg, layer rotation=%.2f deg, space=%p",
(current->time-p->presented_job.time)/1e6,lm_angle(correction)*180/LM_PI,lm_angle(layer_delta)*180/LM_PI,
(void*)(uintptr_t)current->space);
p->tracking_log_time=current->time;
}
return (const void*)&p->layer;
}
static int surf_projection_occludes(const XrCompositionLayerBaseHeader *layer) {
for(int i=0;i<SURF_MAX;i++)if(layer==(const void*)&surf_projections[i].layer)
return surfs[i].native_project && surf_projections[i].ready && surf_projections[i].occludes;
return 0;
}
static const XrCompositionLayerBaseHeader *surf_video_present(XrSession session,const XrFrameEndInfo *info,
const XrCompositionLayerBaseHeader *layer,surf_swapchain *s,surf_projection *p) {
if(!surf_projection_setup(session,s,p,s->width,s->height) || !surf_video_request(session,info,layer,s,p))return NULL;
pthread_mutex_lock(&surf_lock);surf_video_job current=s->video_job;pthread_mutex_unlock(&surf_lock);
// Request tracking updates even when our previous copy is still queued.
if(s->in_flight){
if(svk.GetFenceStatus(vk.device,s->fence)!=VK_SUCCESS)return surf_video_layer(p,&current);
vk.ResetFences(vk.device,1,&s->fence);s->in_flight=0;surf_native_complete(s);
}
// A worker can publish between the two eye calls; present the pair only once.
if(p->ready && p->presented_time==info->displayTime)return surf_video_layer(p,&current);
int slot=-1;
pthread_mutex_lock(&surf_lock);
for(int i=0;i<SURF_NATIVE_SLOTS;i++)if(s->native_slots[i].state==2){slot=i;s->native_slots[i].state=3;break;}
pthread_mutex_unlock(&surf_lock);
if(slot<0)return surf_video_layer(p,&current);
surf_native_slot *native=&s->native_slots[slot];
if(!native->projected || native->width!=2*p->size || native->height!=p->size)goto unused;
PFN_xrAcquireSwapchainImage acquire=(PFN_xrAcquireSwapchainImage)lookup(active_instance,"xrAcquireSwapchainImage");
PFN_xrWaitSwapchainImage wait=(PFN_xrWaitSwapchainImage)lookup(active_instance,"xrWaitSwapchainImage");
PFN_xrReleaseSwapchainImage release=(PFN_xrReleaseSwapchainImage)lookup(active_instance,"xrReleaseSwapchainImage");
int compose=surf_composite_candidate(info,layer);
int cached_only=compose && surf_composite_cached_only(info,p,&native->video,&current);
XrSwapchainImageAcquireInfo ai={XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO};
XrSwapchainImageWaitInfo wi={XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO,NULL,0};
XrSwapchainImageReleaseInfo ri={XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO};uint32_t index;
if(!acquire || !wait || !release)goto unused;
if(!cached_only && !p->acquired){
if(XR_FAILED(acquire(p->output,&ai,&p->acquired_index)))goto unused;
p->acquired=1;
}
index=p->acquired_index;
XrResult waited=cached_only?XR_SUCCESS:wait(p->output,&wi);
// Timeout is a positive XrResult, but the image is not yet writable. Keep
// it acquired and retry next frame rather than blocking the render thread.
if(waited!=XR_SUCCESS)goto unused;
if(!cached_only && index>=p->count){release(p->output,&ri);p->acquired=0;goto unused;}
VkCommandBufferBeginInfo begin={VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,NULL,VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT};
vk.ResetCommandBuffer(s->cmd,0);vk.BeginCommandBuffer(s->cmd,&begin);
surf_native_ownership(s->cmd,native,1);
if(!cached_only)surf_barrier(s->cmd,p->images[index],VK_IMAGE_LAYOUT_UNDEFINED,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,0,VK_ACCESS_TRANSFER_WRITE_BIT);
VkImageCopy copy={{VK_IMAGE_ASPECT_COLOR_BIT,0,0,1},{0,0,0},{VK_IMAGE_ASPECT_COLOR_BIT,0,0,1},{0,0,0},{2*p->size,p->size,1}};
if(!cached_only)spvk.CmdCopyImage(s->cmd,native->image,VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,p->images[index],VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,1,&copy);
if(compose){
surf_barrier(s->cmd,p->video_cache,p->cache_ready?VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL:VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,p->cache_ready?VK_ACCESS_SHADER_READ_BIT:0,VK_ACCESS_TRANSFER_WRITE_BIT);
spvk.CmdCopyImage(s->cmd,native->image,VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,p->video_cache,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,1,&copy);
surf_barrier(s->cmd,p->video_cache,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
VK_ACCESS_TRANSFER_WRITE_BIT,VK_ACCESS_SHADER_READ_BIT);
}
surf_native_ownership(s->cmd,native,0);
if(!cached_only)surf_barrier(s->cmd,p->images[index],VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ACCESS_TRANSFER_WRITE_BIT,VK_ACCESS_COLOR_ATTACHMENT_READ_BIT|VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT);
vk.EndCommandBuffer(s->cmd);
VkSubmitInfo submit={VK_STRUCTURE_TYPE_SUBMIT_INFO,NULL,0,NULL,NULL,1,&s->cmd};
int ok=vk.QueueSubmit(vk.queue,1,&submit,s->fence)==VK_SUCCESS;
if(!cached_only){release(p->output,&ri);p->acquired=0;}if(!ok)goto unused;
s->native_pending=slot;s->in_flight=1;s->uploads++;if(compose){p->cache_ready=1;p->cached_job=native->video;}
p->width=native->source_width;p->height=native->source_height;p->source_seq=native->seq;p->source_ready=1;
for(int eye=0;eye<2;eye++)p->eyes[eye]=(XrCompositionLayerProjectionView){XR_TYPE_COMPOSITION_LAYER_PROJECTION_VIEW,NULL,
native->video.views[eye].pose,native->video.views[eye].fov,
{p->output,{{eye*(int32_t)p->size,0},{(int32_t)p->size,(int32_t)p->size}},0}};
p->layer=(XrCompositionLayerProjection){XR_TYPE_COMPOSITION_LAYER_PROJECTION,NULL,native->video.flags,native->video.space,2,p->eyes};
p->presented_job=native->video;
p->ready=1;p->presented_time=info->displayTime;return surf_video_layer(p,&current);
unused:
pthread_mutex_lock(&surf_lock);native->state=native->seq==s->seq?2:0;pthread_mutex_unlock(&surf_lock);
return surf_video_layer(p,&current);
}
static const XrCompositionLayerBaseHeader *surf_projection_frame(XrSession session,const XrFrameEndInfo *info,
const XrCompositionLayerBaseHeader *layer) {
if(!surf_vulkan || !emul_is_equirect(layer))return NULL;
emul_common common=emul_common_of(layer);
surf_swapchain *s=surf_find(common.sub.swapchain);
if(!s)return NULL;
surf_projection *p=&surf_projections[s-surfs];
if(s->native_project)return surf_video_present(session,info,layer,s,p);
if(s->in_flight) {
if(svk.GetFenceStatus(vk.device,s->fence)!=VK_SUCCESS)return p->ready?(const XrCompositionLayerBaseHeader*)&p->layer:NULL;
vk.ResetFences(vk.device,1,&s->fence);s->in_flight=0;
surf_native_complete(s);
}
int native_index=-1;
if(s->native_mode){
pthread_mutex_lock(&surf_lock);
for(int i=0;i<SURF_NATIVE_SLOTS;i++)if(s->native_slots[i].state==2){native_index=i;s->native_slots[i].state=3;break;}
pthread_mutex_unlock(&surf_lock);
if(native_index<0 && !p->source_ready)return NULL;
}
surf_native_slot *native=native_index>=0?&s->native_slots[native_index]:NULL;
uint32_t source_width=native?native->width:(s->native_mode?p->width:s->width);
uint32_t source_height=native?native->height:(s->native_mode?p->height:s->height);
if(p->pipeline && (p->width!=source_width || p->height!=source_height)){
if(s->native_mode){p->width=source_width;p->height=source_height;}
else surf_projection_destroy(s);
}
if(!surf_projection_setup(session,s,p,source_width,source_height)){
if(native){pthread_mutex_lock(&surf_lock);native->state=native->seq==s->seq?2:0;pthread_mutex_unlock(&surf_lock);}return NULL;
}
XrView views[2]={{XR_TYPE_VIEW,NULL},{XR_TYPE_VIEW,NULL}};
XrViewState state={XR_TYPE_VIEW_STATE,NULL,0};uint32_t count=0;
XrViewLocateInfo locate={XR_TYPE_VIEW_LOCATE_INFO,NULL,XR_VIEW_CONFIGURATION_TYPE_PRIMARY_STEREO,info->displayTime,common.space};
PFN_xrLocateViews locate_views=(PFN_xrLocateViews)lookup(active_instance,"xrLocateViews");
int view_source=surf_views_select(session,info,common.space,
(PFN_xrLocateSpace)lookup(active_instance,"xrLocateSpace"),views);
if(!view_source){
if(!locate_views || XR_FAILED(locate_views(session,&locate,&state,2,&count,views)) || count!=2 ||
!(state.viewStateFlags&XR_VIEW_STATE_ORIENTATION_VALID_BIT))goto unused;
view_source=3;
}
if(p->view_source!=view_source){
LOG("surface_projection: views from %s, space=%p, frame time=%lld",view_source==1?"submitted projection":
view_source==2?"application locate":"runtime locate",(void*)(uintptr_t)common.space,(long long)info->displayTime);
p->view_source=view_source;
}
pthread_mutex_lock(&surf_lock);uint64_t seq=s->native_mode?(native?native->seq:p->source_seq):s->seq;
if(!s->native_mode && seq!=p->source_seq)memcpy(s->staging_map,s->pixels,(size_t)s->width*s->height*4);
pthread_mutex_unlock(&surf_lock);
if(!seq)goto unused;
PFN_xrAcquireSwapchainImage acquire=(PFN_xrAcquireSwapchainImage)lookup(active_instance,"xrAcquireSwapchainImage");
PFN_xrWaitSwapchainImage wait=(PFN_xrWaitSwapchainImage)lookup(active_instance,"xrWaitSwapchainImage");
PFN_xrReleaseSwapchainImage release=(PFN_xrReleaseSwapchainImage)lookup(active_instance,"xrReleaseSwapchainImage");
XrSwapchainImageAcquireInfo ai={XR_TYPE_SWAPCHAIN_IMAGE_ACQUIRE_INFO,NULL};
XrSwapchainImageWaitInfo wi={XR_TYPE_SWAPCHAIN_IMAGE_WAIT_INFO,NULL,100000000};
XrSwapchainImageReleaseInfo ri={XR_TYPE_SWAPCHAIN_IMAGE_RELEASE_INFO,NULL};uint32_t index;
if(!acquire || !wait || !release || XR_FAILED(acquire(p->output,&ai,&index)))goto unused;
if(index>=p->count || XR_FAILED(wait(p->output,&wi))) {release(p->output,&ri);goto unused;}
int previous_native=s->native_current;
surf_native_slot *sample=native?native:(s->native_mode && previous_native>=0?&s->native_slots[previous_native]:NULL);
VkDescriptorImageInfo descriptors[2]={{VK_NULL_HANDLE,p->rendered_view,VK_IMAGE_LAYOUT_GENERAL},
{p->sampler,sample?sample->view:p->source_view,VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL}};
VkWriteDescriptorSet writes[2];
for(int i=0;i<2;i++)writes[i]=(VkWriteDescriptorSet){VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET,NULL,p->set,(uint32_t)i,0,1,
i?VK_DESCRIPTOR_TYPE_COMBINED_IMAGE_SAMPLER:VK_DESCRIPTOR_TYPE_STORAGE_IMAGE,&descriptors[i],NULL,NULL};
spvk.UpdateDescriptorSets(vk.device,2,writes,0,NULL);
VkCommandBufferBeginInfo begin={VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO,NULL,VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT,NULL};
vk.ResetCommandBuffer(s->cmd,0);vk.BeginCommandBuffer(s->cmd,&begin);
if(native){
if(previous_native>=0)surf_native_ownership(s->cmd,&s->native_slots[previous_native],0);
surf_native_ownership(s->cmd,native,1);
}else if(!s->native_mode && seq!=p->source_seq) {
surf_barrier(s->cmd,p->source,p->source_ready?VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL:VK_IMAGE_LAYOUT_UNDEFINED,
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,p->source_ready?VK_ACCESS_SHADER_READ_BIT:0,VK_ACCESS_TRANSFER_WRITE_BIT);
VkBufferImageCopy copy={0,0,0,{VK_IMAGE_ASPECT_COLOR_BIT,0,0,1},{0,0,0},{s->width,s->height,1}};
svk.CmdCopyBufferToImage(s->cmd,s->staging,p->source,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,1,&copy);
surf_barrier(s->cmd,p->source,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,VK_IMAGE_LAYOUT_SHADER_READ_ONLY_OPTIMAL,
VK_ACCESS_TRANSFER_WRITE_BIT,VK_ACCESS_SHADER_READ_BIT);
}
surf_barrier(s->cmd,p->rendered,VK_IMAGE_LAYOUT_UNDEFINED,VK_IMAGE_LAYOUT_GENERAL,0,VK_ACCESS_SHADER_WRITE_BIT);
spvk.CmdBindPipeline(s->cmd,VK_PIPELINE_BIND_POINT_COMPUTE,p->pipeline);
spvk.CmdBindDescriptorSets(s->cmd,VK_PIPELINE_BIND_POINT_COMPUTE,p->layout,0,1,&p->set,0,NULL);
for(int eye=0;eye<2;eye++) {
const XrCompositionLayerBaseHeader *selected=NULL;
for(uint32_t i=0;i<info->layerCount;i++)if(emul_is_equirect(info->layers[i])) {
emul_common c=emul_common_of(info->layers[i]);
if(c.sub.swapchain==s->handle && c.space==common.space && (c.eye==XR_EYE_VISIBILITY_BOTH ||
c.eye==(eye?XR_EYE_VISIBILITY_RIGHT:XR_EYE_VISIBILITY_LEFT))){selected=info->layers[i];break;}
}
if(!selected)selected=layer;
emul_common c=emul_common_of(selected);
XrQuaternionf rotation=lm_qmul(lm_qconj(c.pose.orientation),views[eye].pose.orientation);
XrVector3f origin=lm_rotate(lm_qconj(c.pose.orientation),(XrVector3f){views[eye].pose.position.x-c.pose.position.x,
views[eye].pose.position.y-c.pose.position.y,views[eye].pose.position.z-c.pose.position.z});
float params[32]={rotation.x,rotation.y,rotation.z,rotation.w,origin.x,origin.y,origin.z,0,
tanf(views[eye].fov.angleLeft),tanf(views[eye].fov.angleRight),tanf(views[eye].fov.angleDown),tanf(views[eye].fov.angleUp),
c.sub.imageRect.offset.x/(float)s->width,c.sub.imageRect.offset.y/(float)s->height,
c.sub.imageRect.extent.width/(float)s->width,c.sub.imageRect.extent.height/(float)s->height,
0,0,0,(float)eye,1,1,0,0,1,1,1,1,0,0,0,0};
if(selected->type==XR_TYPE_COMPOSITION_LAYER_EQUIRECT2_KHR) {
const XrCompositionLayerEquirect2KHR *e=(const void*)selected;
params[7]=e->radius;params[16]=e->centralHorizontalAngle;params[17]=e->upperVerticalAngle;params[18]=e->lowerVerticalAngle;
} else {
const XrCompositionLayerEquirectKHR *e=(const void*)selected;
params[7]=e->radius;params[20]=e->scale.x;params[21]=e->scale.y;params[22]=e->bias.x;params[23]=e->bias.y;
}
for(const XrBaseInStructure *n=selected->next;n;n=n->next)if(n->type==XR_TYPE_COMPOSITION_LAYER_COLOR_SCALE_BIAS_KHR) {
const XrCompositionLayerColorScaleBiasKHR *color=(const void*)n;
memcpy(params+24,&color->colorScale,16);memcpy(params+28,&color->colorBias,16);
}
spvk.CmdPushConstants(s->cmd,p->layout,VK_SHADER_STAGE_COMPUTE_BIT,0,sizeof(params),params);
spvk.CmdDispatch(s->cmd,(p->size+15)/16,(p->size+15)/16,1);
p->eyes[eye]=(XrCompositionLayerProjectionView){XR_TYPE_COMPOSITION_LAYER_PROJECTION_VIEW,NULL,
views[eye].pose,views[eye].fov,{p->output,{{eye*(int32_t)p->size,0},{(int32_t)p->size,(int32_t)p->size}},0}};
}
surf_barrier(s->cmd,p->rendered,VK_IMAGE_LAYOUT_GENERAL,VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
VK_ACCESS_SHADER_WRITE_BIT,VK_ACCESS_TRANSFER_READ_BIT);
surf_barrier(s->cmd,p->images[index],VK_IMAGE_LAYOUT_UNDEFINED,VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,
0,VK_ACCESS_TRANSFER_WRITE_BIT);
VkImageCopy rendered_copy={{VK_IMAGE_ASPECT_COLOR_BIT,0,0,1},{0,0,0},
{VK_IMAGE_ASPECT_COLOR_BIT,0,0,1},{0,0,0},{2*p->size,p->size,1}};
spvk.CmdCopyImage(s->cmd,p->rendered,VK_IMAGE_LAYOUT_TRANSFER_SRC_OPTIMAL,
p->images[index],VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,1,&rendered_copy);
surf_barrier(s->cmd,p->images[index],VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL,VK_IMAGE_LAYOUT_COLOR_ATTACHMENT_OPTIMAL,
VK_ACCESS_TRANSFER_WRITE_BIT,VK_ACCESS_COLOR_ATTACHMENT_READ_BIT|VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT);
vk.EndCommandBuffer(s->cmd);
VkSubmitInfo submit={VK_STRUCTURE_TYPE_SUBMIT_INFO,NULL,0,NULL,NULL,1,&s->cmd,0,NULL};
int ok=vk.QueueSubmit(vk.queue,1,&submit,s->fence)==VK_SUCCESS;
release(p->output,&ri);
if(!ok)goto unused;
if(native){s->native_pending=previous_native;s->native_current=native_index;}
s->in_flight=1;p->source_ready=1;p->source_seq=seq;s->uploads++;
p->layer=(XrCompositionLayerProjection){XR_TYPE_COMPOSITION_LAYER_PROJECTION,NULL,layer->layerFlags,common.space,2,p->eyes};
p->ready=1;
return (const XrCompositionLayerBaseHeader*)&p->layer;
unused:
if(native){pthread_mutex_lock(&surf_lock);native->state=native->seq==s->seq?2:0;pthread_mutex_unlock(&surf_lock);}
return NULL;
}
#include "surface_composite.c"
+38
View File
@@ -0,0 +1,38 @@
#version 450
layout(local_size_x=16,local_size_y=16) in;
layout(binding=0,rgba8) uniform writeonly image2D outputImage;
layout(binding=1) uniform sampler2D panorama;
layout(push_constant) uniform Params {
vec4 rotation;
vec4 originRadius;
vec4 tans;
vec4 rect;
vec4 anglesEye;
vec4 scaleBias;
vec4 colorScale;
vec4 colorBias;
} p;
vec3 rotate(vec4 q,vec3 v) {return v+2.0*cross(q.xyz,cross(q.xyz,v)+q.w*v);}
void main() {
ivec2 size=imageSize(outputImage),pixel=ivec2(gl_GlobalInvocationID.xy);
int eye=int(p.anglesEye.w),width=size.x/2;
if(pixel.x>=width || pixel.y>=size.y)return;
vec2 uv=(vec2(pixel)+0.5)/vec2(width,size.y);
vec3 d=normalize(rotate(p.rotation,vec3(mix(p.tans.x,p.tans.y,uv.x),mix(p.tans.w,p.tans.z,uv.y),-1.0)));
if(p.originRadius.w>0.0) {
vec3 o=p.originRadius.xyz;
float b=dot(o,d),disc=b*b+p.originRadius.w*p.originRadius.w-dot(o,o);
if(disc<0.0){imageStore(outputImage,pixel+ivec2(eye*width,0),vec4(0));return;}
float distance=-b+sqrt(disc);
if(distance<=0.0){imageStore(outputImage,pixel+ivec2(eye*width,0),vec4(0));return;}
d=normalize(o+distance*d);
}
float lon=atan(d.x,-d.z),lat=asin(clamp(d.y,-1.0,1.0));
vec2 eq;
if(p.anglesEye.x>0.0) eq=vec2(lon/p.anglesEye.x+0.5,(lat-p.anglesEye.z)/(p.anglesEye.y-p.anglesEye.z));
else eq=vec2(lon/6.28318530718+0.5,lat/3.14159265359+0.5)*p.scaleBias.xy+p.scaleBias.zw;
vec4 color=vec4(0);
if(all(greaterThanEqual(eq,vec2(0))) && all(lessThanEqual(eq,vec2(1))))
color=clamp(textureLod(panorama,p.rect.xy+vec2(eq.x,1.0-eq.y)*p.rect.zw,0.0)*p.colorScale+p.colorBias,0.0,1.0);
imageStore(outputImage,pixel+ivec2(eye*width,0),color);
}
+224
View File
@@ -0,0 +1,224 @@
// SPDX-License-Identifier: GPL-3.0-only
// Generated from surface_projection.comp with glslangValidator -V --target-env vulkan1.0.
#pragma once
static const uint32_t surface_projection_spv[]={
0x7230203,0x10000,0x8000b,0x132,0x0,0x20011,0x1,0x20011,
0x32,0x6000b,0x1,0x4c534c47,0x6474732e,0x3035342e,0x0,0x3000e,
0x0,0x1,0x6000f,0x5,0x4,0x6e69616d,0x0,0x31,
0x60010,0x4,0x11,0x10,0x10,0x1,0x30003,0x2,
0x1c2,0x40005,0x4,0x6e69616d,0x0,0x60005,0xe,0x61746f72,
0x76286574,0x763b3466,0x3b3366,0x30005,0xc,0x71,0x30005,0xd,
0x76,0x40005,0x28,0x657a6973,0x0,0x50005,0x2b,0x7074756f,
0x6d497475,0x656761,0x40005,0x2e,0x65786970,0x6c,0x80005,0x31,
0x475f6c67,0x61626f6c,0x766e496c,0x7461636f,0x496e6f69,0x44,0x30005,0x37,
0x657965,0x40005,0x38,0x61726150,0x736d,0x60006,0x38,0x0,
0x61746f72,0x6e6f6974,0x0,0x70006,0x38,0x1,0x6769726f,0x61526e69,
0x73756964,0x0,0x50006,0x38,0x2,0x736e6174,0x0,0x50006,
0x38,0x3,0x74636572,0x0,0x60006,0x38,0x4,0x6c676e61,
0x79457365,0x65,0x60006,0x38,0x5,0x6c616373,0x61694265,0x73,
0x60006,0x38,0x6,0x6f6c6f63,0x61635372,0x656c,0x60006,0x38,
0x7,0x6f6c6f63,0x61694272,0x73,0x30005,0x3a,0x70,0x40005,
0x40,0x74646977,0x68,0x30005,0x5a,0x7675,0x30005,0x67,
0x64,0x40005,0x7a,0x61726170,0x6d,0x40005,0x7e,0x61726170,
0x6d,0x30005,0x88,0x6f,0x30005,0x8c,0x62,0x40005,
0x90,0x63736964,0x0,0x50005,0xab,0x74736964,0x65636e61,0x0,
0x30005,0xc3,0x6e6f6c,0x30005,0xca,0x74616c,0x30005,0xd5,
0x7165,0x40005,0xf9,0x6f6c6f63,0x72,0x50005,0x10b,0x6f6e6170,
0x616d6172,0x0,0x40047,0x2b,0x22,0x0,0x40047,0x2b,
0x21,0x0,0x30047,0x2b,0x19,0x40047,0x31,0xb,
0x1c,0x50048,0x38,0x0,0x23,0x0,0x50048,0x38,
0x1,0x23,0x10,0x50048,0x38,0x2,0x23,0x20,
0x50048,0x38,0x3,0x23,0x30,0x50048,0x38,0x4,
0x23,0x40,0x50048,0x38,0x5,0x23,0x50,0x50048,
0x38,0x6,0x23,0x60,0x50048,0x38,0x7,0x23,
0x70,0x30047,0x38,0x2,0x40047,0x10b,0x22,0x0,
0x40047,0x10b,0x21,0x1,0x40047,0x131,0xb,0x19,
0x20013,0x2,0x30021,0x3,0x2,0x30016,0x6,0x20,
0x40017,0x7,0x6,0x4,0x40020,0x8,0x7,0x7,
0x40017,0x9,0x6,0x3,0x40020,0xa,0x7,0x9,
0x50021,0xb,0x9,0x8,0xa,0x4002b,0x6,0x11,
0x40000000,0x40015,0x18,0x20,0x0,0x4002b,0x18,0x19,
0x3,0x40020,0x1a,0x7,0x6,0x40015,0x25,0x20,
0x1,0x40017,0x26,0x25,0x2,0x40020,0x27,0x7,
0x26,0x90019,0x29,0x6,0x1,0x0,0x0,0x0,
0x2,0x4,0x40020,0x2a,0x0,0x29,0x4003b,0x2a,
0x2b,0x0,0x40017,0x2f,0x18,0x3,0x40020,0x30,
0x1,0x2f,0x4003b,0x30,0x31,0x1,0x40017,0x32,
0x18,0x2,0x40020,0x36,0x7,0x25,0xa001e,0x38,
0x7,0x7,0x7,0x7,0x7,0x7,0x7,0x7,
0x40020,0x39,0x9,0x38,0x4003b,0x39,0x3a,0x9,
0x4002b,0x25,0x3b,0x4,0x40020,0x3c,0x9,0x6,
0x4002b,0x18,0x41,0x0,0x4002b,0x25,0x44,0x2,
0x20014,0x46,0x4002b,0x18,0x4e,0x1,0x40017,0x58,
0x6,0x2,0x40020,0x59,0x7,0x58,0x4002b,0x6,
0x5d,0x3f000000,0x4002b,0x25,0x68,0x0,0x4002b,0x18,
0x72,0x2,0x4002b,0x6,0x78,0xbf800000,0x40020,0x7b,
0x9,0x7,0x4002b,0x25,0x81,0x1,0x4002b,0x6,
0x84,0x0,0x7002c,0x7,0xa9,0x84,0x84,0x84,
0x84,0x4002b,0x6,0xcd,0x3f800000,0x4002b,0x6,0xe8,
0x40c90fdb,0x4002b,0x6,0xec,0x40490fdb,0x4002b,0x25,0xf0,
0x5,0x5002c,0x58,0xfb,0x84,0x84,0x40017,0xfc,
0x46,0x2,0x5002c,0x58,0x102,0xcd,0xcd,0x90019,
0x108,0x6,0x1,0x0,0x0,0x0,0x1,0x0,
0x3001b,0x109,0x108,0x40020,0x10a,0x0,0x109,0x4003b,
0x10a,0x10b,0x0,0x4002b,0x25,0x10d,0x3,0x4002b,
0x25,0x11d,0x6,0x4002b,0x25,0x121,0x7,0x4002b,
0x18,0x130,0x10,0x6002c,0x2f,0x131,0x130,0x130,
0x4e,0x50036,0x2,0x4,0x0,0x3,0x200f8,0x5,
0x4003b,0x27,0x28,0x7,0x4003b,0x27,0x2e,0x7,
0x4003b,0x36,0x37,0x7,0x4003b,0x36,0x40,0x7,
0x4003b,0x59,0x5a,0x7,0x4003b,0xa,0x67,0x7,
0x4003b,0x8,0x7a,0x7,0x4003b,0xa,0x7e,0x7,
0x4003b,0xa,0x88,0x7,0x4003b,0x1a,0x8c,0x7,
0x4003b,0x1a,0x90,0x7,0x4003b,0x1a,0xab,0x7,
0x4003b,0x1a,0xc3,0x7,0x4003b,0x1a,0xca,0x7,
0x4003b,0x59,0xd5,0x7,0x4003b,0x8,0xf9,0x7,
0x4003d,0x29,0x2c,0x2b,0x40068,0x26,0x2d,0x2c,
0x3003e,0x28,0x2d,0x4003d,0x2f,0x33,0x31,0x7004f,
0x32,0x34,0x33,0x33,0x0,0x1,0x4007c,0x26,
0x35,0x34,0x3003e,0x2e,0x35,0x60041,0x3c,0x3d,
0x3a,0x3b,0x19,0x4003d,0x6,0x3e,0x3d,0x4006e,
0x25,0x3f,0x3e,0x3003e,0x37,0x3f,0x50041,0x36,
0x42,0x28,0x41,0x4003d,0x25,0x43,0x42,0x50087,
0x25,0x45,0x43,0x44,0x3003e,0x40,0x45,0x50041,
0x36,0x47,0x2e,0x41,0x4003d,0x25,0x48,0x47,
0x4003d,0x25,0x49,0x40,0x500af,0x46,0x4a,0x48,
0x49,0x400a8,0x46,0x4b,0x4a,0x300f7,0x4d,0x0,
0x400fa,0x4b,0x4c,0x4d,0x200f8,0x4c,0x50041,0x36,
0x4f,0x2e,0x4e,0x4003d,0x25,0x50,0x4f,0x50041,
0x36,0x51,0x28,0x4e,0x4003d,0x25,0x52,0x51,
0x500af,0x46,0x53,0x50,0x52,0x200f9,0x4d,0x200f8,
0x4d,0x700f5,0x46,0x54,0x4a,0x5,0x53,0x4c,
0x300f7,0x56,0x0,0x400fa,0x54,0x55,0x56,0x200f8,
0x55,0x100fd,0x200f8,0x56,0x4003d,0x26,0x5b,0x2e,
0x4006f,0x58,0x5c,0x5b,0x50050,0x58,0x5e,0x5d,
0x5d,0x50081,0x58,0x5f,0x5c,0x5e,0x4003d,0x25,
0x60,0x40,0x4006f,0x6,0x61,0x60,0x50041,0x36,
0x62,0x28,0x4e,0x4003d,0x25,0x63,0x62,0x4006f,
0x6,0x64,0x63,0x50050,0x58,0x65,0x61,0x64,
0x50088,0x58,0x66,0x5f,0x65,0x3003e,0x5a,0x66,
0x60041,0x3c,0x69,0x3a,0x44,0x41,0x4003d,0x6,
0x6a,0x69,0x60041,0x3c,0x6b,0x3a,0x44,0x4e,
0x4003d,0x6,0x6c,0x6b,0x50041,0x1a,0x6d,0x5a,
0x41,0x4003d,0x6,0x6e,0x6d,0x8000c,0x6,0x6f,
0x1,0x2e,0x6a,0x6c,0x6e,0x60041,0x3c,0x70,
0x3a,0x44,0x19,0x4003d,0x6,0x71,0x70,0x60041,
0x3c,0x73,0x3a,0x44,0x72,0x4003d,0x6,0x74,
0x73,0x50041,0x1a,0x75,0x5a,0x4e,0x4003d,0x6,
0x76,0x75,0x8000c,0x6,0x77,0x1,0x2e,0x71,
0x74,0x76,0x60050,0x9,0x79,0x6f,0x77,0x78,
0x50041,0x7b,0x7c,0x3a,0x68,0x4003d,0x7,0x7d,
0x7c,0x3003e,0x7a,0x7d,0x3003e,0x7e,0x79,0x60039,
0x9,0x7f,0xe,0x7a,0x7e,0x6000c,0x9,0x80,
0x1,0x45,0x7f,0x3003e,0x67,0x80,0x60041,0x3c,
0x82,0x3a,0x81,0x19,0x4003d,0x6,0x83,0x82,
0x500ba,0x46,0x85,0x83,0x84,0x300f7,0x87,0x0,
0x400fa,0x85,0x86,0x87,0x200f8,0x86,0x50041,0x7b,
0x89,0x3a,0x81,0x4003d,0x7,0x8a,0x89,0x8004f,
0x9,0x8b,0x8a,0x8a,0x0,0x1,0x2,0x3003e,
0x88,0x8b,0x4003d,0x9,0x8d,0x88,0x4003d,0x9,
0x8e,0x67,0x50094,0x6,0x8f,0x8d,0x8e,0x3003e,
0x8c,0x8f,0x4003d,0x6,0x91,0x8c,0x4003d,0x6,
0x92,0x8c,0x50085,0x6,0x93,0x91,0x92,0x60041,
0x3c,0x94,0x3a,0x81,0x19,0x4003d,0x6,0x95,
0x94,0x60041,0x3c,0x96,0x3a,0x81,0x19,0x4003d,
0x6,0x97,0x96,0x50085,0x6,0x98,0x95,0x97,
0x50081,0x6,0x99,0x93,0x98,0x4003d,0x9,0x9a,
0x88,0x4003d,0x9,0x9b,0x88,0x50094,0x6,0x9c,
0x9a,0x9b,0x50083,0x6,0x9d,0x99,0x9c,0x3003e,
0x90,0x9d,0x4003d,0x6,0x9e,0x90,0x500b8,0x46,
0x9f,0x9e,0x84,0x300f7,0xa1,0x0,0x400fa,0x9f,
0xa0,0xa1,0x200f8,0xa0,0x4003d,0x29,0xa2,0x2b,
0x4003d,0x26,0xa3,0x2e,0x4003d,0x25,0xa4,0x37,
0x4003d,0x25,0xa5,0x40,0x50084,0x25,0xa6,0xa4,
0xa5,0x50050,0x26,0xa7,0xa6,0x68,0x50080,0x26,
0xa8,0xa3,0xa7,0x40063,0xa2,0xa8,0xa9,0x100fd,
0x200f8,0xa1,0x4003d,0x6,0xac,0x8c,0x4007f,0x6,
0xad,0xac,0x4003d,0x6,0xae,0x90,0x6000c,0x6,
0xaf,0x1,0x1f,0xae,0x50081,0x6,0xb0,0xad,
0xaf,0x3003e,0xab,0xb0,0x4003d,0x6,0xb1,0xab,
0x500bc,0x46,0xb2,0xb1,0x84,0x300f7,0xb4,0x0,
0x400fa,0xb2,0xb3,0xb4,0x200f8,0xb3,0x4003d,0x29,
0xb5,0x2b,0x4003d,0x26,0xb6,0x2e,0x4003d,0x25,
0xb7,0x37,0x4003d,0x25,0xb8,0x40,0x50084,0x25,
0xb9,0xb7,0xb8,0x50050,0x26,0xba,0xb9,0x68,
0x50080,0x26,0xbb,0xb6,0xba,0x40063,0xb5,0xbb,
0xa9,0x100fd,0x200f8,0xb4,0x4003d,0x9,0xbd,0x88,
0x4003d,0x6,0xbe,0xab,0x4003d,0x9,0xbf,0x67,
0x5008e,0x9,0xc0,0xbf,0xbe,0x50081,0x9,0xc1,
0xbd,0xc0,0x6000c,0x9,0xc2,0x1,0x45,0xc1,
0x3003e,0x67,0xc2,0x200f9,0x87,0x200f8,0x87,0x50041,
0x1a,0xc4,0x67,0x41,0x4003d,0x6,0xc5,0xc4,
0x50041,0x1a,0xc6,0x67,0x72,0x4003d,0x6,0xc7,
0xc6,0x4007f,0x6,0xc8,0xc7,0x7000c,0x6,0xc9,
0x1,0x19,0xc5,0xc8,0x3003e,0xc3,0xc9,0x50041,
0x1a,0xcb,0x67,0x4e,0x4003d,0x6,0xcc,0xcb,
0x8000c,0x6,0xce,0x1,0x2b,0xcc,0x78,0xcd,
0x6000c,0x6,0xcf,0x1,0x10,0xce,0x3003e,0xca,
0xcf,0x60041,0x3c,0xd0,0x3a,0x3b,0x41,0x4003d,
0x6,0xd1,0xd0,0x500ba,0x46,0xd2,0xd1,0x84,
0x300f7,0xd4,0x0,0x400fa,0xd2,0xd3,0xe6,0x200f8,
0xd3,0x4003d,0x6,0xd6,0xc3,0x60041,0x3c,0xd7,
0x3a,0x3b,0x41,0x4003d,0x6,0xd8,0xd7,0x50088,
0x6,0xd9,0xd6,0xd8,0x50081,0x6,0xda,0xd9,
0x5d,0x4003d,0x6,0xdb,0xca,0x60041,0x3c,0xdc,
0x3a,0x3b,0x72,0x4003d,0x6,0xdd,0xdc,0x50083,
0x6,0xde,0xdb,0xdd,0x60041,0x3c,0xdf,0x3a,
0x3b,0x4e,0x4003d,0x6,0xe0,0xdf,0x60041,0x3c,
0xe1,0x3a,0x3b,0x72,0x4003d,0x6,0xe2,0xe1,
0x50083,0x6,0xe3,0xe0,0xe2,0x50088,0x6,0xe4,
0xde,0xe3,0x50050,0x58,0xe5,0xda,0xe4,0x3003e,
0xd5,0xe5,0x200f9,0xd4,0x200f8,0xe6,0x4003d,0x6,
0xe7,0xc3,0x50088,0x6,0xe9,0xe7,0xe8,0x50081,
0x6,0xea,0xe9,0x5d,0x4003d,0x6,0xeb,0xca,
0x50088,0x6,0xed,0xeb,0xec,0x50081,0x6,0xee,
0xed,0x5d,0x50050,0x58,0xef,0xea,0xee,0x50041,
0x7b,0xf1,0x3a,0xf0,0x4003d,0x7,0xf2,0xf1,
0x7004f,0x58,0xf3,0xf2,0xf2,0x0,0x1,0x50085,
0x58,0xf4,0xef,0xf3,0x50041,0x7b,0xf5,0x3a,
0xf0,0x4003d,0x7,0xf6,0xf5,0x7004f,0x58,0xf7,
0xf6,0xf6,0x2,0x3,0x50081,0x58,0xf8,0xf4,
0xf7,0x3003e,0xd5,0xf8,0x200f9,0xd4,0x200f8,0xd4,
0x3003e,0xf9,0xa9,0x4003d,0x58,0xfa,0xd5,0x500be,
0xfc,0xfd,0xfa,0xfb,0x4009b,0x46,0xfe,0xfd,
0x300f7,0x100,0x0,0x400fa,0xfe,0xff,0x100,0x200f8,
0xff,0x4003d,0x58,0x101,0xd5,0x500bc,0xfc,0x103,
0x101,0x102,0x4009b,0x46,0x104,0x103,0x200f9,0x100,
0x200f8,0x100,0x700f5,0x46,0x105,0xfe,0xd4,0x104,
0xff,0x300f7,0x107,0x0,0x400fa,0x105,0x106,0x107,
0x200f8,0x106,0x4003d,0x109,0x10c,0x10b,0x50041,0x7b,
0x10e,0x3a,0x10d,0x4003d,0x7,0x10f,0x10e,0x7004f,
0x58,0x110,0x10f,0x10f,0x0,0x1,0x50041,0x1a,
0x111,0xd5,0x41,0x4003d,0x6,0x112,0x111,0x50041,
0x1a,0x113,0xd5,0x4e,0x4003d,0x6,0x114,0x113,
0x50083,0x6,0x115,0xcd,0x114,0x50050,0x58,0x116,
0x112,0x115,0x50041,0x7b,0x117,0x3a,0x10d,0x4003d,
0x7,0x118,0x117,0x7004f,0x58,0x119,0x118,0x118,
0x2,0x3,0x50085,0x58,0x11a,0x116,0x119,0x50081,
0x58,0x11b,0x110,0x11a,0x70058,0x7,0x11c,0x10c,
0x11b,0x2,0x84,0x50041,0x7b,0x11e,0x3a,0x11d,
0x4003d,0x7,0x11f,0x11e,0x50085,0x7,0x120,0x11c,
0x11f,0x50041,0x7b,0x122,0x3a,0x121,0x4003d,0x7,
0x123,0x122,0x50081,0x7,0x124,0x120,0x123,0x70050,
0x7,0x125,0x84,0x84,0x84,0x84,0x70050,0x7,
0x126,0xcd,0xcd,0xcd,0xcd,0x8000c,0x7,0x127,
0x1,0x2b,0x124,0x125,0x126,0x3003e,0xf9,0x127,
0x200f9,0x107,0x200f8,0x107,0x4003d,0x29,0x128,0x2b,
0x4003d,0x26,0x129,0x2e,0x4003d,0x25,0x12a,0x37,
0x4003d,0x25,0x12b,0x40,0x50084,0x25,0x12c,0x12a,
0x12b,0x50050,0x26,0x12d,0x12c,0x68,0x50080,0x26,
0x12e,0x129,0x12d,0x4003d,0x7,0x12f,0xf9,0x40063,
0x128,0x12e,0x12f,0x100fd,0x10038,0x50036,0x9,0xe,
0x0,0xb,0x30037,0x8,0xc,0x30037,0xa,0xd,
0x200f8,0xf,0x4003d,0x9,0x10,0xd,0x4003d,0x7,
0x12,0xc,0x8004f,0x9,0x13,0x12,0x12,0x0,
0x1,0x2,0x4003d,0x7,0x14,0xc,0x8004f,0x9,
0x15,0x14,0x14,0x0,0x1,0x2,0x4003d,0x9,
0x16,0xd,0x7000c,0x9,0x17,0x1,0x44,0x15,
0x16,0x50041,0x1a,0x1b,0xc,0x19,0x4003d,0x6,
0x1c,0x1b,0x4003d,0x9,0x1d,0xd,0x5008e,0x9,
0x1e,0x1d,0x1c,0x50081,0x9,0x1f,0x17,0x1e,
0x7000c,0x9,0x20,0x1,0x44,0x13,0x1f,0x5008e,
0x9,0x21,0x20,0x11,0x50081,0x9,0x22,0x10,
0x21,0x200fe,0x22,0x10038
};
+54 -2
View File
@@ -13,6 +13,29 @@
// copy on the session's queue; GLES: glTexSubImage2D in the app's context), then is submitted as usual.
// * Any failure: the call returns the runtime's own result (as before), so nothing gets worse.
#include <jni.h>
#include <android/hardware_buffer.h>
#include <android/native_window_jni.h>
#include <media/NdkImageReader.h>
#include <EGL/eglext.h>
#include <GLES2/gl2ext.h>
#include "surface_video.h"
#define SURF_NATIVE_SLOTS 4
typedef struct {
AHardwareBuffer *buffer;
VkImage image;
VkImageView view;
VkDeviceMemory memory;
EGLImageKHR egl_image;
GLuint texture;
uint32_t width,height;
int state; // 0 reusable, 1 worker writing, 2 published, 3 current/pending Vulkan sampler
uint64_t seq;
uint32_t source_width,source_height;
int64_t source_time;
surf_video_job video;
int projected;
} surf_native_slot;
#define SURF_MAX 8
@@ -49,6 +72,9 @@ typedef struct {
VkCommandBuffer cmd;
VkFence fence;
int in_flight; // a copy was submitted and its fence not yet seen signalled
int native_mode, native_pending, native_current, native_project;
surf_video_job video_job;
surf_native_slot native_slots[SURF_NATIVE_SLOTS];
} surf_swapchain;
static pthread_mutex_t surf_lock = PTHREAD_MUTEX_INITIALIZER;
@@ -68,7 +94,7 @@ static const char *surf_vs = "#version 300 es\nout vec2 uv;\nvoid main() {\n"
" uv = p; gl_Position = vec4(p * 2.0 - 1.0, 0.0, 1.0);\n}\n";
// window row 0 (glReadPixels' first row) must hold the image's top: sample t = 1 - y (SurfaceTexture t=0 is the bottom)
static const char *surf_fs = "#version 300 es\n#extension GL_OES_EGL_image_external_essl3 : require\n"
"precision mediump float;\nuniform samplerExternalOES tex;\nuniform mat4 st;\nin vec2 uv;\nout vec4 color;\n"
"precision highp float;\nuniform samplerExternalOES tex;\nuniform mat4 st;\nin vec2 uv;\nout vec4 color;\n"
"void main() { color = texture(tex, (st * vec4(uv.x, 1.0 - uv.y, 0.0, 1.0)).xy); }\n";
typedef void (*PFN_glReadPixels_)(GLint, GLint, GLsizei, GLsizei, GLenum, GLenum, void *);
@@ -129,6 +155,8 @@ static void surf_fail(surf_swapchain *s, const char *why) {
pthread_mutex_unlock(&surf_lock);
}
#include "surface_native.c"
static void *surf_worker(void *arg) {
surf_swapchain *s = arg;
JNIEnv *env = NULL;
@@ -149,6 +177,7 @@ static void *surf_worker(void *arg) {
surf_fail(s, "EGL context failed");
goto out;
}
if(s->native_mode){surf_native_worker(s,env,dpy);goto out;}
// the SurfaceTexture's external texture, and the RGBA target the frames are drawn into
p_glGenTextures(1, &ext_tex);
p_glBindTexture(0x8D65 /* GL_TEXTURE_EXTERNAL_OES */, ext_tex);
@@ -202,10 +231,15 @@ static void *surf_worker(void *arg) {
size_t bytes = (size_t)s->width * s->height * 4;
uint8_t *buf = malloc(bytes);
int64_t last_ts = -1, window_frames = 0, window_start = 0;
int64_t latch_ns = 0, read_ns = 0, copy_ns = 0, latch_calls = 0;
while (!s->stop && buf) {
struct timespec pause = {0, 8000000}; // ~120 checks a second: well above video frame rates
nanosleep(&pause, NULL);
if (s_ast_update(ast) != 0) continue;
int64_t tick = monotonic_ns();
int latch_result = s_ast_update(ast);
latch_ns += monotonic_ns() - tick;
latch_calls++;
if (latch_result != 0) continue;
int64_t ts = s_ast_timestamp(ast);
if (ts == last_ts) continue; // no new frame latched
last_ts = ts;
@@ -221,11 +255,15 @@ static void *surf_worker(void *arg) {
p_glBindVertexArray(vao);
p_glDrawArrays(GL_TRIANGLES, 0, 3);
s_glPixelStorei(GL_PACK_ALIGNMENT, 1);
tick = monotonic_ns();
s_glReadPixels(0, 0, (GLsizei)s->width, (GLsizei)s->height, GL_RGBA, GL_UNSIGNED_BYTE, buf);
read_ns += monotonic_ns() - tick;
GLenum err = p_glGetError();
tick = monotonic_ns();
pthread_mutex_lock(&surf_lock);
if (!err && s->pixels) { memcpy(s->pixels, buf, bytes); s->seq++; s->frames++; }
pthread_mutex_unlock(&surf_lock);
copy_ns += monotonic_ns() - tick;
if (err) { static int warned; if (warned++ < 5) LOG("surface_emul: GL error 0x%x drawing a frame", err); }
int64_t now = monotonic_ns();
if (!window_start) {
@@ -235,7 +273,11 @@ static void *surf_worker(void *arg) {
if (++window_frames && now - window_start > 5000000000ll) {
LOG("surface_emul: %.1f video frames/s, %d uploaded, %d upload failures", window_frames * 1e9 / (now - window_start),
s->uploads, s->upload_failures);
LOG("surface_emul: timing latch %.2f ms/call (%lld calls), readback %.2f ms/frame, publish %.2f ms/frame, timestamp %lld",
latch_calls ? latch_ns / (1e6 * latch_calls) : 0.0, (long long)latch_calls,
read_ns / (1e6 * window_frames), copy_ns / (1e6 * window_frames), (long long)ts);
window_frames = 0;
latch_ns = read_ns = copy_ns = latch_calls = 0;
window_start = now;
}
}
@@ -442,6 +484,10 @@ static XRAPI_ATTR XrResult XRAPI_CALL hook_xrCreateSwapchainAndroidSurfaceKHR(Xr
s->handle = handle;
s->width = info->width;
s->height = info->height;
s->native_pending=-1;
s->native_current=-1;
s->native_mode=surface_native && surf_vulkan && surf_native_load();
s->native_project=s->native_mode;
s->pixels = calloc((size_t)s->width * s->height, 4);
uint32_t n = 0;
if (surf_vulkan) {
@@ -470,6 +516,7 @@ static XRAPI_ATTR XrResult XRAPI_CALL hook_xrCreateSwapchainAndroidSurfaceKHR(Xr
if (!setup_ok) {
LOG("surface_emul: setup failed (images %u, pixels %d): the panel stays empty", s->image_count, s->pixels != NULL);
if (s->thread) { s->stop = 1; pthread_join(s->thread, NULL); }
surf_native_destroy(s);
if (destroy) destroy(handle);
free(s->pixels);
pthread_mutex_lock(&surf_lock); s->used = 0; pthread_mutex_unlock(&surf_lock);
@@ -485,13 +532,16 @@ static XRAPI_ATTR XrResult XRAPI_CALL hook_xrCreateSwapchainAndroidSurfaceKHR(Xr
}
// xrDestroySwapchain: an emulated swapchain's worker and surface go with it
static void surf_projection_destroy(surf_swapchain *s);
static void surf_on_destroy(XrSwapchain handle) {
pthread_mutex_lock(&surf_lock);
surf_swapchain *s = surf_find(handle);
pthread_mutex_unlock(&surf_lock);
if (!s) return;
s->stop = 1;
surf_projection_destroy(s);
if (s->thread) pthread_join(s->thread, NULL);
surf_native_destroy(s);
if (surf_vm && s->surface) {
JNIEnv *env = NULL;
int attached = 0;
@@ -508,3 +558,5 @@ static void surf_on_destroy(XrSwapchain handle) {
pthread_mutex_unlock(&surf_lock);
LOG("surface_emul: destroyed %p", (void *)(uintptr_t)handle);
}
#include "surface_projection.c"
+165
View File
@@ -0,0 +1,165 @@
// SPDX-License-Identifier: GPL-3.0-only
// Immutable geometry/pose snapshot carried with a GPU-produced video view.
#ifndef FRAMEBRIDGE_SURFACE_VIDEO_H
#define FRAMEBRIDGE_SURFACE_VIDEO_H
#include <math.h>
#include <string.h>
#include "layer_math.h"
#define SURF_VIDEO_GUARD 1.12f
// MediaCodec's timed surface releases carry CLOCK_MONOTONIC presentation times.
// Untimed releases carry media PTS instead; those must not be treated as a clock.
static int64_t surf_video_target(int64_t now,XrTime display_time) {
return display_time>now-100000000ll && display_time<now+100000000ll?display_time:now;
}
static int surf_video_future(int64_t timestamp,int64_t now,int64_t target) {
return timestamp>now-60000000000ll && timestamp<now+60000000000ll && timestamp>target;
}
// A wider frustum gives runtime timewarp real pixels beyond the visible view.
// Keep the expanded FOV on the submitted image, not just on the shader rays.
static XrFovf surf_video_guard_fov(XrFovf fov) {
return (XrFovf){atanf(tanf(fov.angleLeft)*SURF_VIDEO_GUARD),
atanf(tanf(fov.angleRight)*SURF_VIDEO_GUARD),
atanf(tanf(fov.angleUp)*SURF_VIDEO_GUARD),
atanf(tanf(fov.angleDown)*SURF_VIDEO_GUARD)};
}
typedef struct {
uint64_t seq;
XrTime time;
XrSpace space;
XrCompositionLayerFlags flags;
uint32_t size;
XrView views[2];
XrPosef panorama[2];
float params[2][32];
} surf_video_job;
// Reorient a projected infinite panorama with its layer, retaining the actual
// camera rotation used to render it. Centre it on today's eye position because
// an infinite panorama has no translational parallax. This needs no pixel copy.
static XrPosef surf_video_pose(const surf_video_job *rendered,const surf_video_job *current,int eye) {
XrPosef pose=rendered->views[eye].pose;
if(rendered->space!=current->space || rendered->params[eye][7]!=0 || current->params[eye][7]!=0)return pose;
if(memcmp(&rendered->panorama[eye].orientation,&current->panorama[eye].orientation,sizeof(XrQuaternionf))){
XrQuaternionf delta=lm_qmul(current->panorama[eye].orientation,lm_qconj(rendered->panorama[eye].orientation));
pose.orientation=lm_qmul(delta,pose.orientation);
}
pose.position=current->views[eye].pose.position;
return pose;
}
// Reuse a held picture only while the next visible frustum fits inside it.
// The compositor still timewarps at headset cadence using the actual old pose.
// Changed source frames, layer geometry or colour bypass this reuse policy.
static int surf_video_frustum_covers(const surf_video_job *old,const surf_video_job *next,float margin) {
if(!old->seq || old->size!=next->size || old->space!=next->space)return 0;
for(int eye=0;eye<2;eye++){
const float *a=old->params[eye],*b=next->params[eye];
XrQuaternionf relative=lm_qmul(lm_qconj((XrQuaternionf){a[0],a[1],a[2],a[3]}),
(XrQuaternionf){b[0],b[1],b[2],b[3]});
for(int y=0;y<2;y++)for(int x=0;x<2;x++){
XrVector3f ray=lm_rotate(relative,(XrVector3f){b[8+x]/SURF_VIDEO_GUARD,b[10+y]/SURF_VIDEO_GUARD,-1});
if(ray.z>=-0.00001f)return 0;
float u=ray.x/-ray.z,v=ray.y/-ray.z;
if(u<a[8]*margin || u>a[9]*margin || v<a[10]*margin || v>a[11]*margin)return 0;
}
}
return 1;
}
static int surf_video_covers_margin(const surf_video_job *old,const surf_video_job *next,float margin) {
if(old->flags!=next->flags)return 0;
for(int eye=0;eye<2;eye++){
const float *a=old->params[eye],*b=next->params[eye];
if(a[7]>0 && memcmp(&old->panorama[eye],&next->panorama[eye],sizeof(XrPosef)))return 0;
if(a[7]!=b[7] || memcmp(a+12,b+12,20*sizeof(float)) ||
(a[7]>0 && memcmp(a+4,b+4,3*sizeof(float))))return 0;
}
return surf_video_frustum_covers(old,next,margin);
}
static int surf_video_covers(const surf_video_job *old,const surf_video_job *next) {
// Request a new sampling camera before the visible view reaches the outer
// guard. The worker/copy handoff needs reserve while that update is in flight.
return surf_video_covers_margin(old,next,.94f);
}
// Opaque decoded YUV can occlude earlier layers only when every pixel of the
// rendered, guarded stereo frustum lies in its 180-degree hemisphere. Keep
// ordinary composition during fades, outside coverage, and for other shapes.
static int surf_video_occludes(const surf_video_job *rendered,const surf_video_job *current) {
// Occlusion uses actual image coverage, not the worker's earlier redraw
// threshold. Otherwise each asynchronous camera update switches primary
// layers briefly even though the completed image still covers both eyes.
if(rendered->flags!=current->flags || !surf_video_frustum_covers(rendered,current,1.0f))return 0;
for(int eye=0;eye<2;eye++){
const float *p=rendered->params[eye];
const float *q=current->params[eye];
// RGB/source changes require fresh pixels, not a different composition
// path. Require both the submitted picture and requested fade opaque.
if(q[7]!=0 || !isfinite(q[27]+q[31]) || q[27]+q[31]<1.0f)return 0;
if(p[7]!=0 || fabsf(p[16]-LM_PI)>.00001f ||
fabsf(p[17]-LM_PI*.5f)>.00001f || fabsf(p[18]+LM_PI*.5f)>.00001f ||
!isfinite(p[27]+p[31]) || p[27]+p[31]<1.0f ||
p[12]<0 || p[13]<0 || p[14]<=0 || p[15]<=0 ||
p[12]+p[14]>1.0f || p[13]+p[15]>1.0f)return 0;
XrQuaternionf rotation={p[0],p[1],p[2],p[3]};
for(int y=0;y<2;y++)for(int x=0;x<2;x++){
XrVector3f ray=lm_rotate(rotation,(XrVector3f){p[8+x],p[10+y],-1});
if(!isfinite(ray.z) || ray.z>=-.01f)return 0;
}
}
return 1;
}
// A primary projection must describe one rigid stereo camera. Retarget the
// infinite panorama's centre to the current head position, but rotate the eye
// offsets with the actual camera orientation carried by its rendered pixels.
static XrPosef surf_video_primary_pose(const surf_video_job *rendered,const surf_video_job *current,int eye) {
XrPosef pose=surf_video_pose(rendered,current,eye);
XrVector3f centre={
(current->views[0].pose.position.x+current->views[1].pose.position.x)*.5f,
(current->views[0].pose.position.y+current->views[1].pose.position.y)*.5f,
(current->views[0].pose.position.z+current->views[1].pose.position.z)*.5f};
XrVector3f offset={current->views[eye].pose.position.x-centre.x,
current->views[eye].pose.position.y-centre.y,current->views[eye].pose.position.z-centre.z};
offset=lm_rotate(lm_qmul(pose.orientation,lm_qconj(current->views[eye].pose.orientation)),offset);
pose.position=(XrVector3f){centre.x+offset.x,centre.y+offset.y,centre.z+offset.z};
return pose;
}
// A fresh movie picture does not require a fresh camera direction. While the
// visible view fits, keep its panorama-local sampling grid stable across movie
// frames. This avoids encoding tracking motion into the video textures at the
// decoder's cadence. Tracking remains the compositor's responsibility. Carry
// the real anchored camera/FOV with the pixels; never relabel them as a new view.
// Re-anchor at the newest requested view when either eye leaves the guard band
// or the reference space changes. Source/colour updates keep this camera but
// are still rendered with the current mapping/colour. Finite spheres unchanged.
static int surf_video_anchor(const surf_video_job *old,const surf_video_job *current,surf_video_job *draw) {
*draw=*current;
if(old->params[0][7]!=0 || old->params[1][7]!=0 || current->params[0][7]!=0 || current->params[1][7]!=0 ||
!surf_video_frustum_covers(old,current,.94f))return 0;
for(int eye=0;eye<2;eye++){
draw->views[eye]=old->views[eye];
const float *p=old->params[eye];
// Reconstruct from the fixed sampling quaternion, not a succession of
// layer deltas, so long cutscenes cannot accumulate pose-rounding drift.
draw->views[eye].pose.orientation=lm_qmul(current->panorama[eye].orientation,(XrQuaternionf){p[0],p[1],p[2],p[3]});
draw->views[eye].pose.position=current->views[eye].pose.position;
memcpy(draw->params[eye],old->params[eye],12*sizeof(float));
}
return 1;
}
// Same ray/sphere mapping as the Vulkan reference shader, sampled directly
// from the decoder's external YUV texture. No full-size RGB intermediate.
static const char *surf_video_fragment=
"#version 300 es\n#extension GL_OES_EGL_image_external_essl3 : require\n"
"precision highp float;uniform samplerExternalOES tex;in vec2 uv;out vec4 color;"
"uniform vec4 rotation,originRadius,tans,rect,anglesEye,scaleBias,colorScale,colorBias;"
"vec3 rotate(vec4 q,vec3 v){return v+2.0*cross(q.xyz,cross(q.xyz,v)+q.w*v);}"
"void main(){vec3 d=normalize(rotate(rotation,vec3(mix(tans.x,tans.y,uv.x),mix(tans.w,tans.z,uv.y),-1.0)));"
"if(originRadius.w>0.0){vec3 o=originRadius.xyz;float b=dot(o,d),disc=b*b+originRadius.w*originRadius.w-dot(o,o);"
"if(disc<0.0){color=vec4(0);return;}float distance=-b+sqrt(disc);"
"if(distance<=0.0){color=vec4(0);return;}d=normalize(o+distance*d);}"
"float lon=atan(d.x,-d.z),lat=asin(clamp(d.y,-1.0,1.0));vec2 eq;"
"if(anglesEye.x>0.0)eq=vec2(lon/anglesEye.x+0.5,(lat-anglesEye.z)/(anglesEye.y-anglesEye.z));"
"else eq=vec2(lon/6.28318530718+0.5,lat/3.14159265359+0.5)*scaleBias.xy+scaleBias.zw;"
"color=vec4(0);if(all(greaterThanEqual(eq,vec2(0)))&&all(lessThanEqual(eq,vec2(1))))"
"color=clamp(texture(tex,rect.xy+vec2(eq.x,1.0-eq.y)*rect.zw)*colorScale+colorBias,0.0,1.0);}"
;
static const char *surf_video_uniforms[8]={"rotation","originRadius","tans","rect",
"anglesEye","scaleBias","colorScale","colorBias"};
#endif
+61
View File
@@ -0,0 +1,61 @@
// SPDX-License-Identifier: GPL-3.0-only
// Use the views belonging to the submitted frame before sampling tracking again.
// Cached application views are eligible only for the exact session/display time.
#ifndef FRAMEBRIDGE_SURFACE_VIEWS_H
#define FRAMEBRIDGE_SURFACE_VIEWS_H
#include <pthread.h>
#include "layer_math.h"
static pthread_mutex_t surf_views_lock=PTHREAD_MUTEX_INITIALIZER;
static struct {
XrSession session;
XrSpace space;
XrTime time;
XrView views[2];
} surf_view_cache[4];
static void surf_views_record(XrSession session,const XrViewLocateInfo *info,const XrViewState *state,
uint32_t count,const XrView *views) {
if(!info || !state || !views || count!=2 || !info->displayTime ||
!(state->viewStateFlags&XR_VIEW_STATE_ORIENTATION_VALID_BIT))return;
pthread_mutex_lock(&surf_views_lock);
int slot=0;
for(int i=0;i<4;i++){
if(surf_view_cache[i].session==session && surf_view_cache[i].space==info->space){slot=i;break;}
if(surf_view_cache[i].time<surf_view_cache[slot].time)slot=i;
}
surf_view_cache[slot].session=session;surf_view_cache[slot].space=info->space;
surf_view_cache[slot].time=info->displayTime;
surf_view_cache[slot].views[0]=views[0];surf_view_cache[slot].views[1]=views[1];
pthread_mutex_unlock(&surf_views_lock);
}
// 1 = submitted projection views, 2 = exact application locate, 0 = not available.
static int surf_views_select(XrSession session,const XrFrameEndInfo *frame,XrSpace space,
PFN_xrLocateSpace locate,XrView views[2]) {
if(!frame)return 0;
for(uint32_t i=0;i<frame->layerCount;i++){
const XrCompositionLayerBaseHeader *layer=frame->layers[i];
if(!layer || layer->type!=XR_TYPE_COMPOSITION_LAYER_PROJECTION)continue;
const XrCompositionLayerProjection *projection=(const void*)layer;
if(projection->viewCount!=2 || !projection->views)continue;
XrPosef transform={{0,0,0,1},{0,0,0}};
if(projection->space!=space){
XrSpaceLocation location={XR_TYPE_SPACE_LOCATION,NULL,0,{{0,0,0,1},{0,0,0}}};
XrSpaceLocationFlags valid=XR_SPACE_LOCATION_ORIENTATION_VALID_BIT|XR_SPACE_LOCATION_POSITION_VALID_BIT;
if(!locate || XR_FAILED(locate(projection->space,space,frame->displayTime,&location)) ||
(location.locationFlags&valid)!=valid)continue;
transform=location.pose;
}
for(int eye=0;eye<2;eye++){
views[eye].pose=lm_pose_mul(transform,projection->views[eye].pose);
views[eye].fov=projection->views[eye].fov;
}
return 1;
}
int found=0;
pthread_mutex_lock(&surf_views_lock);
for(int i=0;i<4;i++)if(surf_view_cache[i].session==session && surf_view_cache[i].space==space &&
surf_view_cache[i].time==frame->displayTime){
views[0]=surf_view_cache[i].views[0];views[1]=surf_view_cache[i].views[1];found=2;break;
}
pthread_mutex_unlock(&surf_views_lock);return found;
}
#endif
+54 -8
View File
@@ -5,12 +5,12 @@ Everything is fetched on demand into native/.cache (git-ignored):
- Android NDK r27c (27.2.12479018) from Google's repository (checksummed via repository2-3.xml)
- OpenXR headers at the commits each component was written against (KhronosGroup/OpenXR-SDK)
- Temurin JDK 21 (javac) and Android build-tools (d8) for the Java stub classes
Then builds: FrameBridge adapter (arm64 + arm32), VrApi bridge, platform compat, language packs, GL shim, oculusos stub dex, the
Then builds: FrameBridge adapter (arm64 + arm32), VrApi bridge, platform compat, language packs, GL shim, multiview twin interposer (glmv), oculusos stub dex, the
timefix OpenXR layer for Proton games (linux-arm64, glibc; the NDK's clang builds it freestanding), the
OculusHMDConnected helper for Rift games under Proton (win-x64 PE; the NDK's clang + lld-link, no Windows SDK), and
rewrites artifacts/SHA256SUMS. Run `frameport parity` afterwards to see which games change.
OculusHMDConnected helper for Rift games under Proton (win-x64 PE; the NDK's clang + lld-link, no Windows SDK), the
live view's hardware H.264 encoder fp_venc (linux-arm64-bin, static freestanding executable), and rewrites artifacts/SHA256SUMS. Run `frameport parity` afterwards to see which games change.
python native/build.py [--only adapter,bridge,compat,langpack,glshim,eglfmt,ovrtrace,dex,xrlayer,oculushmd,xrshim,vkshim,ovrpshim,vrsettings] [--ndk PATH]
python native/build.py [--only adapter,bridge,compat,langpack,glshim,eglfmt,glmv,ovrtrace,dex,xrlayer,oculushmd,xrshim,vkshim,zinkfix,ovrpshim,vrsettings,venc] [--ndk PATH]
"""
from __future__ import annotations
@@ -158,6 +158,32 @@ def build_xrlayer(tc: Path):
shutil.copy(src / "XR_APILAYER_FRAMEPORT_timefix.json", out / "XR_APILAYER_FRAMEPORT_timefix.json")
def uapi_arm64(tc: Path) -> Path:
"""Only the kernel UAPI headers of the NDK sysroot (linux/, asm-generic/, drm/, aarch64 asm/), rebuilt every time,
so no bionic libc header can end up on a freestanding build's include path."""
inc = tc / "sysroot/usr/include"
dest = CACHE / "uapi-arm64"
shutil.rmtree(dest, ignore_errors=True)
for d in ("linux", "asm-generic", "drm"):
shutil.copytree(inc / d, dest / d)
shutil.copytree(inc / "aarch64-linux-android/asm", dest / "asm")
return dest
def build_venc(tc: Path):
"""fp_venc (Linux aarch64, runs on the Frame): SteamVR headset view -> V4L2 hardware H.264 encoder -> stdout, for the
live view. Fully static and freestanding (raw syscalls, own _start), so it needs no libc at build or run time.
Goes to linux-arm64-bin/, not linux-arm64/ (that folder is uploaded whole as the OpenXR layer)."""
src = HERE / "venc"
uapi = uapi_arm64(tc)
out = ART / "linux-arm64-bin"
out.mkdir(parents=True, exist_ok=True)
run([tc / "bin/clang", "--target=aarch64-linux-gnu", "-ffreestanding", "-nostdlib", "-nostdlibinc", "-static",
"-fno-stack-protector", "-fno-builtin", "-O2", "-Wall", "-Wextra", "-Werror", "-fuse-ld=lld",
"-Wl,--build-id=none", "-Wl,-z,max-page-size=65536", "-e", "_start", "-I", src / "include", "-I", uapi,
f"-ffile-prefix-map={HERE}=native", "fp_venc.c", "convert.c", "sys.c", "-o", out / "fp_venc"], cwd=src)
OCULUSHMD_IMPORTS = ("CreateEventW", "CreateJobObjectW", "AssignProcessToJobObject", "QueryInformationJobObject",
"CreateProcessW", "ResumeThread", "WaitForSingleObject", "GetExitCodeProcess", "CloseHandle",
"Sleep", "GetCommandLineW", "GetLastError", "GetStdHandle", "WriteFile", "ExitProcess")
@@ -249,6 +275,17 @@ def build_eglfmt(tc: Path):
"-Wl,--no-as-needed", "-lEGL", "-Wl,--as-needed", "-ldl", "-llog", "-o", ART / "arm64-v8a/libfpg.so"], cwd=src)
def build_glmv(tc: Path):
"""Single-view draws of OVR_multiview programs (see glmv/glmv.c, frame.gl_multiview_fbo). Named libfpglmv.so: as
long as "libGLESv3.so", whose dlopen string in the engine is rewritten in place. Linked to libGLESv3 so a dlsym on
its handle still finds every GL function it doesn't wrap."""
src = HERE / "glmv"
run([exe(tc, "aarch64-linux-android29-clang"), "-shared", "-fPIC", "-O2", "-Wall", "-Wextra", "-Werror",
"-fvisibility=hidden", f"-ffile-prefix-map={src}=native/glmv", "-Wl,-soname,libfpglmv.so",
"-Wl,-z,max-page-size=16384", "glmv.c", "-Wl,--no-as-needed", "-lGLESv3", "-Wl,--as-needed", "-lEGL", "-ldl",
"-llog", "-o", ART / "arm64-v8a/libfpglmv.so"], cwd=src)
def build_ovrtrace(tc: Path):
"""Meta Platform SDK tracer (see ovrtrace/ovrtrace.c): diagnostics, the game's engine library loads it first."""
src = HERE / "ovrtrace"
@@ -274,6 +311,14 @@ def build_vkshim(tc: Path):
cwd=src)
def build_zinkfix(tc: Path):
"""Shader-fix Vulkan layer for OpenGL ES games (see zinkfix/zinkfix.c): between Zink and the driver."""
src = HERE / "zinkfix"
run([exe(tc, "aarch64-linux-android29-clang"), "-shared", "-fPIC", "-O2", "-Wall", "-Wextra", "-Werror",
"-fvisibility=hidden", "-Wl,-soname,libVkLayer_fp_shaderfix.so", "-Wl,-z,max-page-size=16384", "zinkfix.c",
"-ldl", "-llog", "-o", ART / "arm64-v8a/libVkLayer_fp_shaderfix.so"], cwd=src)
def build_ovrpshim(tc: Path):
"""OVRPlugin frame-loop shim (see ovrpshim/ovrpshim.c) for Unity 2017-2018 built-in Oculus support."""
src = HERE / "ovrpshim"
@@ -326,7 +371,7 @@ def write_sums():
def main():
ap = argparse.ArgumentParser()
ap.add_argument("--only", default="adapter,bridge,compat,langpack,glshim,eglfmt,ovrtrace,dex,xrlayer,oculushmd,xrshim,vkshim,ovrpshim,vrsettings")
ap.add_argument("--only", default="adapter,bridge,compat,langpack,glshim,eglfmt,glmv,ovrtrace,dex,xrlayer,oculushmd,xrshim,vkshim,zinkfix,ovrpshim,vrsettings,venc")
ap.add_argument("--ndk")
args = ap.parse_args()
parts = set(args.only.split(","))
@@ -335,11 +380,12 @@ def main():
tc = clang_dir(ndk(args.ndk)) if parts - {"dex"} else None
steps = {"adapter": lambda: build_adapter(tc), "bridge": lambda: build_bridge(tc), "compat": lambda: build_compat(tc),
"langpack": lambda: build_langpack(tc),
"glshim": lambda: build_glshim(tc), "eglfmt": lambda: build_eglfmt(tc), "ovrtrace": lambda: build_ovrtrace(tc), "xrshim": lambda: build_xrshim(tc), "dex": build_dex, "xrlayer": lambda: build_xrlayer(tc),
"glshim": lambda: build_glshim(tc), "eglfmt": lambda: build_eglfmt(tc), "glmv": lambda: build_glmv(tc), "ovrtrace": lambda: build_ovrtrace(tc), "xrshim": lambda: build_xrshim(tc), "dex": build_dex, "xrlayer": lambda: build_xrlayer(tc),
"oculushmd": lambda: build_oculushmd(tc), "vkshim": lambda: build_vkshim(tc),
"zinkfix": lambda: build_zinkfix(tc),
"ovrpshim": lambda: build_ovrpshim(tc),
"vrsettings": lambda: build_vrsettings(tc)}
for name in ("adapter", "bridge", "compat", "langpack", "glshim", "eglfmt", "ovrtrace", "xrshim", "vkshim", "ovrpshim", "dex", "xrlayer", "oculushmd", "vrsettings"):
"vrsettings": lambda: build_vrsettings(tc), "venc": lambda: build_venc(tc)}
for name in ("adapter", "bridge", "compat", "langpack", "glshim", "eglfmt", "glmv", "ovrtrace", "xrshim", "vkshim", "zinkfix", "ovrpshim", "dex", "xrlayer", "oculushmd", "vrsettings", "venc"):
if name in parts:
log(f"build {name}")
steps[name]()
+704
View File
@@ -0,0 +1,704 @@
// SPDX-License-Identifier: GPL-3.0-only
// libfpglmv.so: single-view draws of OVR_multiview programs (frame.gl_multiview_fbo, GitHub #77).
//
// OVR_multiview says a draw is INVALID_OPERATION when the number of views of the draw framebuffer differs from the
// num_views the program's vertex shader declares. Qualcomm's Quest driver tolerates it; Mesa (the Frame's GL, Zink)
// enforces it (src/mesa/main/draw_validate.c) and drops the draw silently. Engines that compile every shader with
// `layout(num_views=2) in;` and also draw into ordinary 2D framebuffers (Doom3Quest's HUD/PDA pool: glTexImage2D
// textures + renderbuffers) then get black panels.
//
// This library sits between such an engine and libGLESv3: the patch puts it first in the engine library's DT_NEEDED
// (direct gl* imports bind here) and points the engine's dlopen("libGLESv3.so") at it (same-length name, so its
// qgl* table from dlsym binds here too); every function it doesn't define resolves to libGLESv3 through its own
// dependency (linked --no-as-needed). When a multiview program draws into a framebuffer whose color attachment has no
// views (2D texture, texture layer, renderbuffer, FBO 0), the draw uses a lazily built single-view twin of the program
// (glmv_rewrite.h: num_views layout removed, gl_ViewID_OVR -> 0u), with the original's attribute locations, uniform
// block bindings and default-block uniform values (copied before every such draw), then the original is bound again.
//
// Logcat tag GLMV. Setting gl_mv_debug=1 (libframe_settings.so / framebridge.conf / FRAMEBRIDGE_CONFIG, as the GL
// shim): glGetError after every twin draw, and per-5-s counters.
#define _GNU_SOURCE
#include <EGL/egl.h>
#include <GLES3/gl32.h>
#include <android/log.h>
#include <dlfcn.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
#include "glmv_rewrite.h"
#define TAG "GLMV"
#define LOG(...) __android_log_print(ANDROID_LOG_INFO, TAG, __VA_ARGS__)
#define EXPORT __attribute__((visibility("default")))
#ifndef GLMV_GLES_LIB // (the host test points these at a stand-in GL library)
#define GLMV_GLES_LIB "libGLESv3.so"
#define GLMV_EGL_LIB "libEGL.so"
#endif
#ifndef GL_FRAMEBUFFER_ATTACHMENT_TEXTURE_NUM_VIEWS_OVR
#define GL_FRAMEBUFFER_ATTACHMENT_TEXTURE_NUM_VIEWS_OVR 0x9630
#endif
// ---------------------------------------------------------------- real functions (only ever from libGLESv3's handle)
#define REAL_LIST(X) \
X(void, UseProgram, (GLuint)) \
X(void, LinkProgram, (GLuint)) \
X(void, DeleteProgram, (GLuint)) \
X(void, BindFramebuffer, (GLenum, GLuint)) \
X(void, DeleteFramebuffers, (GLsizei, const GLuint *)) \
X(void, FramebufferTexture2D, (GLenum, GLenum, GLenum, GLuint, GLint)) \
X(void, FramebufferRenderbuffer, (GLenum, GLenum, GLenum, GLuint)) \
X(void, FramebufferTextureLayer, (GLenum, GLenum, GLuint, GLint, GLint)) \
X(void, FramebufferTexture, (GLenum, GLenum, GLuint, GLint)) \
X(void, DrawBuffers, (GLsizei, const GLenum *)) \
X(void, DrawArrays, (GLenum, GLint, GLsizei)) \
X(void, DrawElements, (GLenum, GLsizei, GLenum, const void *)) \
X(void, DrawArraysInstanced, (GLenum, GLint, GLsizei, GLsizei)) \
X(void, DrawElementsInstanced, (GLenum, GLsizei, GLenum, const void *, GLsizei)) \
X(void, DrawRangeElements, (GLenum, GLuint, GLuint, GLsizei, GLenum, const void *)) \
X(void, DrawArraysIndirect, (GLenum, const void *)) \
X(void, DrawElementsIndirect, (GLenum, GLenum, const void *)) \
X(void, DrawElementsBaseVertex, (GLenum, GLsizei, GLenum, const void *, GLint)) \
X(void, DrawRangeElementsBaseVertex, (GLenum, GLuint, GLuint, GLsizei, GLenum, const void *, GLint)) \
X(void, DrawElementsInstancedBaseVertex, (GLenum, GLsizei, GLenum, const void *, GLsizei, GLint)) \
X(GLenum, GetError, (void)) \
X(void, GetIntegerv, (GLenum, GLint *)) \
X(void, GetFramebufferAttachmentParameteriv, (GLenum, GLenum, GLenum, GLint *)) \
X(void, GetProgramiv, (GLuint, GLenum, GLint *)) \
X(void, GetProgramInfoLog, (GLuint, GLsizei, GLsizei *, GLchar *)) \
X(void, GetShaderiv, (GLuint, GLenum, GLint *)) \
X(void, GetShaderInfoLog, (GLuint, GLsizei, GLsizei *, GLchar *)) \
X(void, GetShaderSource, (GLuint, GLsizei, GLsizei *, GLchar *)) \
X(void, GetAttachedShaders, (GLuint, GLsizei, GLsizei *, GLuint *)) \
X(GLuint, CreateShader, (GLenum)) \
X(void, ShaderSource, (GLuint, GLsizei, const GLchar *const *, const GLint *)) \
X(void, CompileShader, (GLuint)) \
X(void, DeleteShader, (GLuint)) \
X(GLuint, CreateProgram, (void)) \
X(void, AttachShader, (GLuint, GLuint)) \
X(void, BindAttribLocation, (GLuint, GLuint, const GLchar *)) \
X(void, GetActiveAttrib, (GLuint, GLuint, GLsizei, GLsizei *, GLint *, GLenum *, GLchar *)) \
X(GLint, GetAttribLocation, (GLuint, const GLchar *)) \
X(void, GetActiveUniform, (GLuint, GLuint, GLsizei, GLsizei *, GLint *, GLenum *, GLchar *)) \
X(void, GetActiveUniformsiv, (GLuint, GLsizei, const GLuint *, GLenum, GLint *)) \
X(GLint, GetUniformLocation, (GLuint, const GLchar *)) \
X(void, GetUniformfv, (GLuint, GLint, GLfloat *)) \
X(void, GetUniformiv, (GLuint, GLint, GLint *)) \
X(void, GetUniformuiv, (GLuint, GLint, GLuint *)) \
X(void, GetActiveUniformBlockName, (GLuint, GLuint, GLsizei, GLsizei *, GLchar *)) \
X(void, GetActiveUniformBlockiv, (GLuint, GLuint, GLenum, GLint *)) \
X(GLuint, GetUniformBlockIndex, (GLuint, const GLchar *)) \
X(void, UniformBlockBinding, (GLuint, GLuint, GLuint)) \
X(void, Uniform1fv, (GLint, GLsizei, const GLfloat *)) \
X(void, Uniform2fv, (GLint, GLsizei, const GLfloat *)) \
X(void, Uniform3fv, (GLint, GLsizei, const GLfloat *)) \
X(void, Uniform4fv, (GLint, GLsizei, const GLfloat *)) \
X(void, Uniform1iv, (GLint, GLsizei, const GLint *)) \
X(void, Uniform2iv, (GLint, GLsizei, const GLint *)) \
X(void, Uniform3iv, (GLint, GLsizei, const GLint *)) \
X(void, Uniform4iv, (GLint, GLsizei, const GLint *)) \
X(void, Uniform1uiv, (GLint, GLsizei, const GLuint *)) \
X(void, Uniform2uiv, (GLint, GLsizei, const GLuint *)) \
X(void, Uniform3uiv, (GLint, GLsizei, const GLuint *)) \
X(void, Uniform4uiv, (GLint, GLsizei, const GLuint *)) \
X(void, UniformMatrix2fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix3fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix4fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix2x3fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix3x2fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix2x4fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix4x2fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix3x4fv, (GLint, GLsizei, GLboolean, const GLfloat *)) \
X(void, UniformMatrix4x3fv, (GLint, GLsizei, GLboolean, const GLfloat *))
#define DECL(ret, name, args) static ret(*r_##name) args;
REAL_LIST(DECL)
#undef DECL
typedef void (*PFN_FTMV)(GLenum, GLenum, GLuint, GLint, GLint, GLsizei);
typedef void (*PFN_FTMSMV)(GLenum, GLenum, GLuint, GLint, GLsizei, GLint, GLsizei);
static PFN_FTMV r_ftmv;
static PFN_FTMSMV r_ftmsmv;
static __eglMustCastToProperFunctionPointerType (*r_eglGetProcAddress)(const char *);
static EGLBoolean (*r_eglMakeCurrent)(EGLDisplay, EGLSurface, EGLSurface, EGLContext);
static pthread_once_t once = PTHREAD_ONCE_INIT;
static int debug;
static void read_conf_file(const char *path) {
FILE *f = fopen(path, "r");
if (!f) return;
char line[256];
while (fgets(line, sizeof(line), f))
if (!strncmp(line, "gl_mv_debug=", 12)) debug = atoi(line + 12);
fclose(f);
}
// The GL shim's settings sources, later ones winning: libframe_settings.so next to this library, the game's
// framebridge.conf, then FRAMEBRIDGE_CONFIG (Lepton's per-game settings.conf).
static void read_conf(void) {
Dl_info info;
char path[600];
if (dladdr((void *)read_conf, &info) && info.dli_fname) {
const char *slash = strrchr(info.dli_fname, '/');
if (slash && slash - info.dli_fname < 500) {
snprintf(path, sizeof(path), "%.*s/libframe_settings.so", (int)(slash - info.dli_fname), info.dli_fname);
read_conf_file(path);
}
}
char pkg[256] = {0};
FILE *f = fopen("/proc/self/cmdline", "r");
if (f) {
size_t n = fread(pkg, 1, sizeof(pkg) - 1, f);
fclose(f);
pkg[n] = 0;
}
char *colon = strchr(pkg, ':');
if (colon) *colon = 0;
if (*pkg && !strchr(pkg, '/')) {
snprintf(path, sizeof(path), "/sdcard/Android/data/%s/files/framebridge.conf", pkg);
read_conf_file(path);
}
const char *env = getenv("FRAMEBRIDGE_CONFIG");
if (env && *env) read_conf_file(env);
}
static void init(void) {
read_conf();
void *gles = dlopen(GLMV_GLES_LIB, RTLD_NOW | RTLD_LOCAL);
void *egl = dlopen(GLMV_EGL_LIB, RTLD_NOW | RTLD_LOCAL);
r_eglGetProcAddress = egl ? dlsym(egl, "eglGetProcAddress") : NULL;
r_eglMakeCurrent = egl ? dlsym(egl, "eglMakeCurrent") : NULL;
int missing = 0;
#define LOAD(ret, name, args) \
r_##name = gles ? (ret(*) args)dlsym(gles, "gl" #name) : NULL; \
if (!r_##name && r_eglGetProcAddress) r_##name = (ret(*) args)r_eglGetProcAddress("gl" #name); \
missing += !r_##name;
REAL_LIST(LOAD)
#undef LOAD
if (r_eglGetProcAddress) {
r_ftmv = (PFN_FTMV)r_eglGetProcAddress("glFramebufferTextureMultiviewOVR");
r_ftmsmv = (PFN_FTMSMV)r_eglGetProcAddress("glFramebufferTextureMultisampleMultiviewOVR");
}
LOG("library active: single-view draws of multiview programs (gl_mv_debug=%d, %d GL functions missing)", debug,
missing);
}
#define INIT() pthread_once(&once, init)
// ---------------------------------------------------------------- programs
struct uni {
GLenum type;
GLint orig, var; // locations in the original and the twin
GLuint shadow[16]; // last value given to the twin
int valid;
};
struct block {
GLuint orig_index, var_index;
GLint binding; // last binding given to the twin
};
struct prog {
GLuint id;
int multiview; // the vertex shader declares num_views
int nshaders;
GLenum types[6];
char *sources[6]; // stage sources at link time (only kept for multiview programs)
GLuint twin; // single-view twin, 0 = not built yet
int twin_failed;
struct uni *unis;
int nunis;
struct block *blocks;
int nblocks;
};
static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER;
static struct prog **progs; // indexed by program name (GL names are small integers)
static GLuint nprogs;
static struct prog *find_prog(GLuint id, int create) {
struct prog *p = NULL;
pthread_mutex_lock(&lock);
if (id < nprogs) p = progs[id];
if (!p && create && id < (1u << 24)) {
if (id >= nprogs) {
GLuint n = nprogs ? nprogs : 256;
while (n <= id) n *= 2;
struct prog **grown = realloc(progs, n * sizeof(*grown));
if (grown) {
memset(grown + nprogs, 0, (n - nprogs) * sizeof(*grown));
progs = grown;
nprogs = n;
}
}
if (id < nprogs && (p = calloc(1, sizeof(*p)))) {
p->id = id;
progs[id] = p;
}
}
pthread_mutex_unlock(&lock);
return p;
}
static void drop_twin(struct prog *p) {
if (p->twin && r_DeleteProgram) r_DeleteProgram(p->twin);
p->twin = 0;
p->twin_failed = 0;
free(p->unis);
p->unis = NULL;
p->nunis = 0;
free(p->blocks);
p->blocks = NULL;
p->nblocks = 0;
}
static void drop_sources(struct prog *p) {
for (int i = 0; i < p->nshaders; ++i) free(p->sources[i]);
p->nshaders = 0;
p->multiview = 0;
}
static char *shader_source(GLuint sh) {
GLint len = 0;
r_GetShaderiv(sh, GL_SHADER_SOURCE_LENGTH, &len);
if (len <= 0) return NULL;
char *src = malloc((size_t)len + 1);
if (!src) return NULL;
GLsizei got = 0;
r_GetShaderSource(sh, len, &got, src);
src[got > 0 && got <= len ? got : 0] = 0;
return src;
}
// Before the real link: note whether the program is multiview and keep its stage sources for a later twin.
static void note_program(GLuint id) {
struct prog *p = find_prog(id, 1);
if (!p) return;
drop_twin(p);
drop_sources(p);
GLuint shaders[6];
GLsizei n = 0;
r_GetAttachedShaders(id, 6, &n, shaders);
for (GLsizei i = 0; i < n && i < 6; ++i) {
GLint type = 0;
r_GetShaderiv(shaders[i], GL_SHADER_TYPE, &type);
char *src = shader_source(shaders[i]);
if (!src) continue;
if (type == GL_VERTEX_SHADER && glmv_has_num_views(src)) p->multiview = 1;
p->types[p->nshaders] = (GLenum)type;
p->sources[p->nshaders++] = src;
}
if (!p->multiview) drop_sources(p);
}
// ---------------------------------------------------------------- the single-view twin
static int components(GLenum type, int *kind) { // kind: 0 float, 1 int/bool/sampler, 2 uint, 3 matrix
switch (type) {
case GL_FLOAT: *kind = 0; return 1;
case GL_FLOAT_VEC2: *kind = 0; return 2;
case GL_FLOAT_VEC3: *kind = 0; return 3;
case GL_FLOAT_VEC4: *kind = 0; return 4;
case GL_INT: case GL_BOOL: *kind = 1; return 1;
case GL_INT_VEC2: case GL_BOOL_VEC2: *kind = 1; return 2;
case GL_INT_VEC3: case GL_BOOL_VEC3: *kind = 1; return 3;
case GL_INT_VEC4: case GL_BOOL_VEC4: *kind = 1; return 4;
case GL_UNSIGNED_INT: *kind = 2; return 1;
case GL_UNSIGNED_INT_VEC2: *kind = 2; return 2;
case GL_UNSIGNED_INT_VEC3: *kind = 2; return 3;
case GL_UNSIGNED_INT_VEC4: *kind = 2; return 4;
case GL_FLOAT_MAT2: *kind = 3; return 4;
case GL_FLOAT_MAT3: *kind = 3; return 9;
case GL_FLOAT_MAT4: *kind = 3; return 16;
case GL_FLOAT_MAT2x3: case GL_FLOAT_MAT3x2: *kind = 3; return 6;
case GL_FLOAT_MAT2x4: case GL_FLOAT_MAT4x2: *kind = 3; return 8;
case GL_FLOAT_MAT3x4: case GL_FLOAT_MAT4x3: *kind = 3; return 12;
default: *kind = 1; return 1; // samplers and images: one int (the unit)
}
}
static GLuint compile(GLenum type, const char *src, GLuint prog_id) {
GLuint sh = r_CreateShader(type);
if (!sh) return 0;
r_ShaderSource(sh, 1, &src, NULL);
r_CompileShader(sh);
GLint ok = 0;
r_GetShaderiv(sh, GL_COMPILE_STATUS, &ok);
if (!ok) {
char log[600] = {0};
r_GetShaderInfoLog(sh, sizeof(log) - 1, NULL, log);
LOG("twin of program %u: stage 0x%x failed to compile: %s", prog_id, type, log);
r_DeleteShader(sh);
return 0;
}
return sh;
}
static void copy_attrib_locations(GLuint orig, GLuint twin) {
GLint n = 0;
r_GetProgramiv(orig, GL_ACTIVE_ATTRIBUTES, &n);
for (GLint i = 0; i < n; ++i) {
char name[256];
GLsizei len = 0;
GLint size = 0;
GLenum type = 0;
r_GetActiveAttrib(orig, (GLuint)i, sizeof(name), &len, &size, &type, name);
if (len <= 0 || !strncmp(name, "gl_", 3)) continue;
GLint loc = r_GetAttribLocation(orig, name);
if (loc >= 0) r_BindAttribLocation(twin, (GLuint)loc, name);
}
}
static void map_uniforms(struct prog *p) {
GLint n = 0;
r_GetProgramiv(p->twin, GL_ACTIVE_UNIFORMS, &n);
int cap = 0;
for (GLint i = 0; i < n; ++i) {
char name[256];
GLsizei len = 0;
GLint size = 0;
GLenum type = 0;
GLuint index = (GLuint)i;
GLint block = -1;
r_GetActiveUniform(p->twin, index, sizeof(name) - 16, &len, &size, &type, name);
r_GetActiveUniformsiv(p->twin, 1, &index, GL_UNIFORM_BLOCK_INDEX, &block);
if (len <= 0 || block != -1 || !strncmp(name, "gl_", 3)) continue; // block members live in buffers
char *bracket = len > 3 && !strcmp(name + len - 3, "[0]") ? name + len - 3 : NULL;
if (bracket) *bracket = 0;
for (GLint e = 0; e < (size > 0 ? size : 1); ++e) {
char elem[300];
if (bracket) snprintf(elem, sizeof(elem), "%s[%d]", name, e);
else snprintf(elem, sizeof(elem), "%s", name);
GLint lo = r_GetUniformLocation(p->id, elem), lt = r_GetUniformLocation(p->twin, elem);
if (lo < 0 || lt < 0) continue;
if (p->nunis == cap) {
cap = cap ? cap * 2 : 16;
struct uni *grown = realloc(p->unis, (size_t)cap * sizeof(*grown));
if (!grown) return;
p->unis = grown;
}
p->unis[p->nunis++] = (struct uni){.type = type, .orig = lo, .var = lt};
}
}
GLint nb = 0;
r_GetProgramiv(p->twin, GL_ACTIVE_UNIFORM_BLOCKS, &nb);
p->blocks = nb > 0 ? calloc((size_t)nb, sizeof(*p->blocks)) : NULL;
for (GLint i = 0; p->blocks && i < nb; ++i) {
char name[256];
GLsizei len = 0;
r_GetActiveUniformBlockName(p->twin, (GLuint)i, sizeof(name), &len, name);
GLuint oi = len > 0 ? r_GetUniformBlockIndex(p->id, name) : GL_INVALID_INDEX;
if (oi == GL_INVALID_INDEX) continue;
p->blocks[p->nblocks++] = (struct block){.orig_index = oi, .var_index = (GLuint)i, .binding = -1};
}
}
static int build_twin(struct prog *p) {
GLuint twin = r_CreateProgram(), shaders[6];
int ns = 0, layouts = 0, ids = 0;
for (int i = 0; twin && i < p->nshaders; ++i) {
int l = 0, v = 0;
char *src = glmv_rewrite(p->sources[i], &l, &v);
GLuint sh = src ? compile(p->types[i], src, p->id) : 0;
free(src);
if (!sh) { ns = -1; break; }
layouts += l;
ids += v;
r_AttachShader(twin, sh);
shaders[ns++] = sh;
}
if (twin && ns > 0) {
copy_attrib_locations(p->id, twin);
r_LinkProgram(twin);
}
for (int i = 0; i < ns; ++i) r_DeleteShader(shaders[i]); // flagged only; freed with the program
GLint ok = 0;
if (twin && ns > 0) r_GetProgramiv(twin, GL_LINK_STATUS, &ok);
if (!ok) {
char log[600] = {0};
if (twin && ns > 0) r_GetProgramInfoLog(twin, sizeof(log) - 1, NULL, log);
LOG("twin of program %u failed (%s): drawing it unchanged", p->id, ns > 0 ? log : "compile");
if (twin) r_DeleteProgram(twin);
p->twin_failed = 1;
return 0;
}
p->twin = twin;
map_uniforms(p);
LOG("twin built: program %u -> %u (%d uniforms, %d blocks, %d num_views layouts and %d gl_ViewID_OVR replaced)",
p->id, twin, p->nunis, p->nblocks, layouts, ids);
return 1;
}
static unsigned long uniform_sets; // twin uniforms changed (statistics)
static void sync_uniforms(struct prog *p) {
for (int i = 0; i < p->nunis; ++i) {
struct uni *u = &p->unis[i];
int kind, n = components(u->type, &kind);
GLuint v[16] = {0};
if (kind == 0 || kind == 3) r_GetUniformfv(p->id, u->orig, (GLfloat *)v);
else if (kind == 2) r_GetUniformuiv(p->id, u->orig, v);
else r_GetUniformiv(p->id, u->orig, (GLint *)v);
if (u->valid && !memcmp(v, u->shadow, (size_t)n * sizeof(GLuint))) continue;
memcpy(u->shadow, v, sizeof(v));
u->valid = 1;
++uniform_sets;
const GLfloat *f = (const GLfloat *)v;
const GLint *iv = (const GLint *)v;
switch (u->type) {
case GL_FLOAT: r_Uniform1fv(u->var, 1, f); break;
case GL_FLOAT_VEC2: r_Uniform2fv(u->var, 1, f); break;
case GL_FLOAT_VEC3: r_Uniform3fv(u->var, 1, f); break;
case GL_FLOAT_VEC4: r_Uniform4fv(u->var, 1, f); break;
case GL_UNSIGNED_INT: r_Uniform1uiv(u->var, 1, v); break;
case GL_UNSIGNED_INT_VEC2: r_Uniform2uiv(u->var, 1, v); break;
case GL_UNSIGNED_INT_VEC3: r_Uniform3uiv(u->var, 1, v); break;
case GL_UNSIGNED_INT_VEC4: r_Uniform4uiv(u->var, 1, v); break;
case GL_FLOAT_MAT2: r_UniformMatrix2fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT3: r_UniformMatrix3fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT4: r_UniformMatrix4fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT2x3: r_UniformMatrix2x3fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT3x2: r_UniformMatrix3x2fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT2x4: r_UniformMatrix2x4fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT4x2: r_UniformMatrix4x2fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT3x4: r_UniformMatrix3x4fv(u->var, 1, GL_FALSE, f); break;
case GL_FLOAT_MAT4x3: r_UniformMatrix4x3fv(u->var, 1, GL_FALSE, f); break;
case GL_INT_VEC2: case GL_BOOL_VEC2: r_Uniform2iv(u->var, 1, iv); break;
case GL_INT_VEC3: case GL_BOOL_VEC3: r_Uniform3iv(u->var, 1, iv); break;
case GL_INT_VEC4: case GL_BOOL_VEC4: r_Uniform4iv(u->var, 1, iv); break;
default: r_Uniform1iv(u->var, 1, iv); break; // int, bool, samplers
}
}
for (int i = 0; i < p->nblocks; ++i) {
struct block *b = &p->blocks[i];
GLint binding = 0;
r_GetActiveUniformBlockiv(p->id, b->orig_index, GL_UNIFORM_BLOCK_BINDING, &binding);
if (binding != b->binding) {
r_UniformBlockBinding(p->twin, b->var_index, (GLuint)binding);
b->binding = binding;
}
}
}
// ---------------------------------------------------------------- per-thread state (a GL context is per thread)
#define FB_UNKNOWN (-1)
#define FB_NO_COLOR (-2)
static __thread struct prog *cur_prog;
static __thread int fb_views = FB_UNKNOWN; // views of the draw framebuffer's first color attachment (cached)
static int draw_fb_views(void) {
if (fb_views != FB_UNKNOWN) return fb_views;
GLint fb = 0;
r_GetIntegerv(GL_DRAW_FRAMEBUFFER_BINDING, &fb);
if (!fb) return fb_views = 0; // the window surface
GLint type = GL_NONE;
r_GetFramebufferAttachmentParameteriv(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_FRAMEBUFFER_ATTACHMENT_OBJECT_TYPE, &type);
if (type == GL_NONE) return fb_views = FB_NO_COLOR; // depth-only: Mesa doesn't compare views
if (type != GL_TEXTURE) return fb_views = 0; // renderbuffer
GLint views = 0;
r_GetFramebufferAttachmentParameteriv(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
GL_FRAMEBUFFER_ATTACHMENT_TEXTURE_NUM_VIEWS_OVR, &views);
return fb_views = views;
}
// ---------------------------------------------------------------- statistics (per 5 s)
static unsigned long twin_draws, mv_draws;
static struct timespec last_stats;
static void stats(void) {
struct timespec now;
clock_gettime(CLOCK_MONOTONIC, &now);
if (!last_stats.tv_sec) last_stats = now;
if (now.tv_sec - last_stats.tv_sec < 5) return;
last_stats = now;
if (twin_draws || debug)
LOG("5s: %lu single-view draws of multiview programs, %lu multiview draws, %lu twin uniform updates",
twin_draws, mv_draws, uniform_sets);
twin_draws = mv_draws = uniform_sets = 0;
}
// Before a draw: bind the twin when the program is multiview and the framebuffer has no views. 1 = twin bound.
static int begin_draw(void) {
struct prog *p = cur_prog;
if (!p || !p->multiview || p->twin_failed) return 0;
int views = draw_fb_views();
if (views != 0) {
++mv_draws;
if (debug) stats();
return 0;
}
if (!p->twin && !build_twin(p)) return 0;
r_UseProgram(p->twin);
sync_uniforms(p);
if (debug) r_GetError(); // (debug only: this consumes an earlier error the game hasn't read)
return 1;
}
static void end_draw(const char *what) {
struct prog *p = cur_prog;
if (debug) {
static int logs;
GLenum err = r_GetError();
if (err && logs < 20) {
++logs;
LOG("%s with the twin of program %u: GL error 0x%x", what, p ? p->id : 0, err);
}
}
r_UseProgram(p ? p->id : 0);
++twin_draws;
stats();
}
// ---------------------------------------------------------------- exported GL functions
EXPORT void glUseProgram(GLuint program) {
INIT();
r_UseProgram(program);
cur_prog = program ? find_prog(program, 0) : NULL;
}
EXPORT void glLinkProgram(GLuint program) {
INIT();
note_program(program);
r_LinkProgram(program);
}
EXPORT void glDeleteProgram(GLuint program) {
INIT();
struct prog *p = program ? find_prog(program, 0) : NULL;
if (p) { // the record stays (pointers to it may be cached); a new program with this name relinks it
drop_twin(p);
drop_sources(p);
}
r_DeleteProgram(program);
}
EXPORT void glBindFramebuffer(GLenum target, GLuint framebuffer) {
INIT();
r_BindFramebuffer(target, framebuffer);
if (target != GL_READ_FRAMEBUFFER) fb_views = FB_UNKNOWN;
}
EXPORT void glDeleteFramebuffers(GLsizei n, const GLuint *framebuffers) {
INIT();
r_DeleteFramebuffers(n, framebuffers);
fb_views = FB_UNKNOWN;
}
EXPORT void glFramebufferTexture2D(GLenum target, GLenum attachment, GLenum textarget, GLuint texture, GLint level) {
INIT();
r_FramebufferTexture2D(target, attachment, textarget, texture, level);
fb_views = FB_UNKNOWN;
}
EXPORT void glFramebufferRenderbuffer(GLenum target, GLenum attachment, GLenum rbtarget, GLuint renderbuffer) {
INIT();
r_FramebufferRenderbuffer(target, attachment, rbtarget, renderbuffer);
fb_views = FB_UNKNOWN;
}
EXPORT void glFramebufferTextureLayer(GLenum target, GLenum attachment, GLuint texture, GLint level, GLint layer) {
INIT();
r_FramebufferTextureLayer(target, attachment, texture, level, layer);
fb_views = FB_UNKNOWN;
}
EXPORT void glFramebufferTexture(GLenum target, GLenum attachment, GLuint texture, GLint level) {
INIT();
if (r_FramebufferTexture) r_FramebufferTexture(target, attachment, texture, level);
fb_views = FB_UNKNOWN;
}
EXPORT void glDrawBuffers(GLsizei n, const GLenum *bufs) {
INIT();
r_DrawBuffers(n, bufs);
fb_views = FB_UNKNOWN;
}
static void w_ftmv(GLenum target, GLenum attachment, GLuint texture, GLint level, GLint base, GLsizei views) {
r_ftmv(target, attachment, texture, level, base, views);
fb_views = FB_UNKNOWN;
}
static void w_ftmsmv(GLenum target, GLenum attachment, GLuint texture, GLint level, GLsizei samples, GLint base,
GLsizei views) {
r_ftmsmv(target, attachment, texture, level, samples, base, views);
fb_views = FB_UNKNOWN;
}
#define DRAW(name, call) \
INIT(); \
if (!r_##name) return; \
if (begin_draw()) { \
r_##name call; \
end_draw("gl" #name); \
} else { \
r_##name call; \
}
EXPORT void glDrawArrays(GLenum mode, GLint first, GLsizei count) { DRAW(DrawArrays, (mode, first, count)) }
EXPORT void glDrawElements(GLenum mode, GLsizei count, GLenum type, const void *indices) {
DRAW(DrawElements, (mode, count, type, indices))
}
EXPORT void glDrawArraysInstanced(GLenum mode, GLint first, GLsizei count, GLsizei instances) {
DRAW(DrawArraysInstanced, (mode, first, count, instances))
}
EXPORT void glDrawElementsInstanced(GLenum mode, GLsizei count, GLenum type, const void *indices, GLsizei instances) {
DRAW(DrawElementsInstanced, (mode, count, type, indices, instances))
}
EXPORT void glDrawRangeElements(GLenum mode, GLuint start, GLuint end, GLsizei count, GLenum type,
const void *indices) {
DRAW(DrawRangeElements, (mode, start, end, count, type, indices))
}
EXPORT void glDrawArraysIndirect(GLenum mode, const void *indirect) { DRAW(DrawArraysIndirect, (mode, indirect)) }
EXPORT void glDrawElementsIndirect(GLenum mode, GLenum type, const void *indirect) {
DRAW(DrawElementsIndirect, (mode, type, indirect))
}
EXPORT void glDrawElementsBaseVertex(GLenum mode, GLsizei count, GLenum type, const void *indices, GLint base) {
DRAW(DrawElementsBaseVertex, (mode, count, type, indices, base))
}
EXPORT void glDrawRangeElementsBaseVertex(GLenum mode, GLuint start, GLuint end, GLsizei count, GLenum type,
const void *indices, GLint base) {
DRAW(DrawRangeElementsBaseVertex, (mode, start, end, count, type, indices, base))
}
EXPORT void glDrawElementsInstancedBaseVertex(GLenum mode, GLsizei count, GLenum type, const void *indices,
GLsizei instances, GLint base) {
DRAW(DrawElementsInstancedBaseVertex, (mode, count, type, indices, instances, base))
}
// A context made current on this thread (engines with a render thread move theirs between threads, e.g. Doom3Quest's
// GLimp_ActivateContext): what this thread knew about the bound program and framebuffer may be another context's.
EXPORT EGLBoolean eglMakeCurrent(EGLDisplay dpy, EGLSurface draw, EGLSurface read, EGLContext ctx) {
INIT();
if (!r_eglMakeCurrent) return EGL_FALSE;
EGLBoolean ok = r_eglMakeCurrent(dpy, draw, read, ctx);
fb_views = FB_UNKNOWN;
cur_prog = NULL;
if (ok && ctx != EGL_NO_CONTEXT && r_GetIntegerv) {
GLint program = 0;
r_GetIntegerv(GL_CURRENT_PROGRAM, &program);
cur_prog = program > 0 ? find_prog((GLuint)program, 0) : NULL;
}
return ok;
}
// Engines fetch the OVR multiview functions (and SDL-style loaders every function) through eglGetProcAddress.
EXPORT __eglMustCastToProperFunctionPointerType eglGetProcAddress(const char *name) {
INIT();
__eglMustCastToProperFunctionPointerType fn = r_eglGetProcAddress ? r_eglGetProcAddress(name) : NULL;
if (!fn || !name) return fn;
if (!strcmp(name, "glFramebufferTextureMultiviewOVR")) {
if (!r_ftmv) r_ftmv = (PFN_FTMV)fn;
return (__eglMustCastToProperFunctionPointerType)w_ftmv;
}
if (!strcmp(name, "glFramebufferTextureMultisampleMultiviewOVR")) {
if (!r_ftmsmv) r_ftmsmv = (PFN_FTMSMV)fn;
return (__eglMustCastToProperFunctionPointerType)w_ftmsmv;
}
#define OWN(sym) if (!strcmp(name, #sym)) return (__eglMustCastToProperFunctionPointerType)sym;
OWN(glUseProgram) OWN(glLinkProgram) OWN(glDeleteProgram) OWN(glBindFramebuffer) OWN(glDeleteFramebuffers)
OWN(glFramebufferTexture2D) OWN(glFramebufferRenderbuffer) OWN(glFramebufferTextureLayer)
OWN(glFramebufferTexture) OWN(glDrawBuffers) OWN(glDrawArrays) OWN(glDrawElements) OWN(glDrawArraysInstanced)
OWN(glDrawElementsInstanced) OWN(glDrawRangeElements) OWN(glDrawArraysIndirect) OWN(glDrawElementsIndirect)
OWN(glDrawElementsBaseVertex) OWN(glDrawRangeElementsBaseVertex) OWN(glDrawElementsInstancedBaseVertex)
OWN(eglMakeCurrent)
#undef OWN
return fn;
}
+87
View File
@@ -0,0 +1,87 @@
// SPDX-License-Identifier: GPL-3.0-only
// GLSL source rewrite for libfpglmv.so (see glmv.c): turns an OVR_multiview shader into its single-view twin.
// - every `layout(... num_views ...) in;` statement is blanked out (the program then declares no views, which is
// what Mesa requires for a draw into a framebuffer whose attachments aren't multiview)
// - every gl_ViewID_OVR token becomes (0u): view 0 (the left eye's matrices in Doom3Quest)
// #extension lines stay (the driver supports OVR_multiview; enabling it without a num_views layout is harmless).
// Columns and line numbers are kept (blanks pad the replacements), so compile logs point at the same places.
// Plain C, no GL: the host unit test (tests/fixtures/src/glmv_rewrite_test.c) compiles it too.
#pragma once
#include <stdlib.h>
#include <string.h>
static int glmv_ident_char(char c) {
return (c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '_';
}
// `word` at p as a whole token (not part of a longer identifier)
static int glmv_token_at(const char *start, const char *p, const char *word) {
size_t n = strlen(word);
if (strncmp(p, word, n) != 0) return 0;
if (p > start && glmv_ident_char(p[-1])) return 0;
return !glmv_ident_char(p[n]);
}
static const char *glmv_skip_ws(const char *p) {
while (*p == ' ' || *p == '\t' || *p == '\n' || *p == '\r' || *p == '\f' || *p == '\v') ++p;
return p;
}
// If a `layout(...num_views...) in;` statement starts at p, returns a pointer to its ';', else NULL.
static const char *glmv_num_views_stmt(const char *start, const char *p) {
if (!glmv_token_at(start, p, "layout")) return NULL;
const char *q = glmv_skip_ws(p + 6);
if (*q != '(') return NULL;
const char *open = q;
int depth = 0;
for (; *q; ++q) {
if (*q == '(') ++depth;
else if (*q == ')' && --depth == 0) break;
}
if (!*q) return NULL;
const char *close = q;
int has_views = 0;
for (const char *r = open + 1; r < close; ++r)
if (glmv_token_at(start, r, "num_views")) { has_views = 1; break; }
if (!has_views) return NULL;
q = glmv_skip_ws(close + 1);
if (!glmv_token_at(start, q, "in")) return NULL;
q = glmv_skip_ws(q + 2);
return *q == ';' ? q : NULL;
}
// 1 if the source declares multiview views (a vertex shader of an OVR_multiview program)
static int glmv_has_num_views(const char *src) {
for (const char *p = src; (p = strstr(p, "layout")); p += 6)
if (glmv_num_views_stmt(src, p)) return 1;
return 0;
}
// The single-view twin of `src` (malloc'd, same length), or NULL when out of memory. Counts go to *layouts / *ids.
static char *glmv_rewrite(const char *src, int *layouts, int *ids) {
static const char id_tok[] = "gl_ViewID_OVR"; // 13 characters
static const char zero[] = "(0u)";
size_t n = strlen(src);
char *out = (char *)malloc(n + 1);
if (!out) return NULL;
memcpy(out, src, n + 1);
int nl = 0, ni = 0;
for (char *p = out; (p = strstr(p, "layout"));) {
const char *semi = glmv_num_views_stmt(out, p);
if (!semi) { p += 6; continue; }
for (char *b = p; b <= semi; ++b)
if (*b != '\n' && *b != '\r') *b = ' ';
++nl;
p = (char *)semi + 1;
}
for (char *p = out; (p = strstr(p, id_tok));) {
if (!glmv_token_at(out, p, id_tok)) { p += sizeof(id_tok) - 1; continue; }
memcpy(p, zero, sizeof(zero) - 1);
memset(p + sizeof(zero) - 1, ' ', sizeof(id_tok) - sizeof(zero));
++ni;
p += sizeof(id_tok) - 1;
}
if (layouts) *layouts = nl;
if (ids) *ids = ni;
return out;
}
+34 -7
View File
@@ -30,8 +30,13 @@ static void (*real_glGetIntegerv)(GLenum, GLint *);
static __eglMustCastToProperFunctionPointerType (*real_eglGetProcAddress)(const char *);
static pthread_once_t once = PTHREAD_ONCE_INIT;
static int hide_multiview = -1; // -1: not set (Unity keeps multiview, everything else hides it)
static int hide_multiview = -1; // -1: not set (Unity and Unreal keep multiview, everything else hides it)
static int hide_msrtt = 1;
// Unreal (4.25) only turns on its mobile multiview when GL_OVR_multiview, GL_OVR_multiview2 AND
// GL_OVR_multiview_multisampled_render_to_texture are all there (FOpenGLES::ProcessExtensions). Hiding the last one
// with the MSRTT extensions switched Unreal's multiview off, so for Unreal it stays visible and its function draws
// single-sampled (glFramebufferTextureMultiviewOVR), like the EXT stand-ins below.
static int unreal;
static void read_conf_file(const char *path) {
FILE *f = fopen(path, "r");
@@ -335,12 +340,21 @@ static void init(void) {
real_glGetStringi = gles ? dlsym(gles, "glGetStringi") : NULL;
real_glGetIntegerv = gles ? dlsym(gles, "glGetIntegerv") : NULL;
real_eglGetProcAddress = egl ? dlsym(egl, "eglGetProcAddress") : NULL;
if (hide_multiview < 0) { // Unity renders multiview itself and only needs MSRTT hidden
void *unity = dlopen("libunity.so", RTLD_NOW | RTLD_NOLOAD);
hide_multiview = unity ? 0 : 1;
if (unity) dlclose(unity);
static const char *const engines[] = {"libunity.so", "libUE4.so", "libUnreal.so"};
int engine_found = 0;
for (unsigned i = 0; i < sizeof engines / sizeof *engines && !engine_found; i++) {
void *engine = dlopen(engines[i], RTLD_NOW | RTLD_NOLOAD);
if (engine) {
engine_found = 1;
unreal = i > 0;
dlclose(engine);
}
}
LOG("GL shim active: hide_multiview=%d hide_msrtt=%d", hide_multiview, hide_msrtt);
// Unity and Unreal render multiview themselves and only need MSRTT hidden
if (hide_multiview < 0) hide_multiview = !engine_found;
unreal = unreal && hide_msrtt && !hide_multiview; // only matters while MSRTT is hidden and multiview kept
LOG("GL shim active: hide_multiview=%d hide_msrtt=%d%s", hide_multiview, hide_msrtt,
unreal ? " (Unreal: multiview MSRTT kept, drawn single-sampled)" : "");
}
static int is_hidden(const char *name) {
@@ -349,7 +363,8 @@ static int is_hidden(const char *name) {
if (!strcmp(name, hidden_multiview[i])) return 1;
if (hide_msrtt)
for (size_t i = 0; i < sizeof(hidden_msrtt) / sizeof(hidden_msrtt[0]); ++i)
if (!strcmp(name, hidden_msrtt[i])) return 1;
if (!strcmp(name, hidden_msrtt[i]))
return !(unreal && !strcmp(name, "GL_OVR_multiview_multisampled_render_to_texture"));
return 0;
}
@@ -416,6 +431,14 @@ static void plain_rbsms(GLenum target, GLsizei samples, GLenum format, GLsizei w
(void)samples;
if (r_plain_rbs) r_plain_rbs(target, format, width, height);
}
static PFN_FTMV r_plain_ftmv;
static int mv_msrtt_logged;
static void plain_ftmsmv(GLenum target, GLenum attachment, GLuint texture, GLint level, GLsizei samples,
GLint base, GLsizei views) {
if (!mv_msrtt_logged++)
LOG("GL shim: multiview multisampled render-to-texture (%d samples) drawn single-sampled", (int)samples);
if (r_plain_ftmv) r_plain_ftmv(target, attachment, texture, level, base, views);
}
__attribute__((visibility("default"))) __eglMustCastToProperFunctionPointerType eglGetProcAddress(const char *name) {
pthread_once(&once, init);
@@ -432,6 +455,10 @@ __attribute__((visibility("default"))) __eglMustCastToProperFunctionPointerType
return name[2] == 'F' ? (__eglMustCastToProperFunctionPointerType)plain_ft2dms
: (__eglMustCastToProperFunctionPointerType)plain_rbsms;
}
if (unreal && !strcmp(name, "glFramebufferTextureMultisampleMultiviewOVR")) {
if (!r_plain_ftmv) r_plain_ftmv = (PFN_FTMV)real_eglGetProcAddress("glFramebufferTextureMultiviewOVR");
return (__eglMustCastToProperFunctionPointerType)plain_ftmsmv;
}
#ifdef GLSHIM_TRACE // build with -DGLSHIM_TRACE for eye-buffer / draw / error tracing
WRAP("glFramebufferTextureMultiviewOVR", real_ftmv, w_ftmv)
WRAP("glFramebufferTextureMultisampleMultiviewOVR", real_ftmsmv, w_ftmsmv)
+211
View File
@@ -0,0 +1,211 @@
# Shared hardware video decoding for Lepton
`frameport_hevc.cpp` implements Android OMX decoders for HEVC Main, H.264
(Constrained) Baseline/Main/(Constrained) High and VP9 Profile 0 using Iris
stateful V4L2 through FFmpeg. Android reports constrained H.264 streams under
separate profile values, so they are listed as stock decoders list them.
These are 8-bit 4:2:0 decoders. Format-aware clients keep Android's existing
codecs for Main10/HDR and protected content; clients selecting by MIME alone
must still respect the component's advertised profiles. Other codec types
retain Android's existing codecs. Applications that explicitly select their
own software decoder bypass this integration. Byte-buffer clients receive full-size
planar frames. Native surface clients receive YUV in Android hardware buffers,
with acquire/release fences. For large hardware-decoded NV12 surfaces (at least
4096×2048 pixels), compatible linear NV12 buffers are imported into Vulkan
and copied on the GPU without CPU pixel access or colour conversion. The decoded
AVFrame remains held until the transfer fence completes, preventing its Iris
capture buffer from being requeued. Explicit FOREIGN queue-family transfers
return both buffers to their Android/decoder owners before delivery. Imports are
cached within a bounded pool and cleared before decoder reconfiguration or flush.
Unsupported layouts, unavailable Vulkan capabilities and import failures retain
the parallel CPU copy; a submitted transfer that fails to complete reports an
OMX error instead of exposing incomplete pixels. This removes the CPU frame copy
on compatible surfaces but still performs one GPU copy; it is not direct decoding
into Android's consumer buffers. Small surfaces use disjoint parallel CPU row bands.
The same persistent worker pool handles native YUV copies, byte-buffer planar
copies and legacy RGBA conversion (helpers start at panorama sizes); all workers finish before the buffer is unlocked or the decoded
frame is released. Legacy software surfaces
retain parallel NEON RGBA conversion. Input timestamps, dynamic dimensions, EOS and
seek/flush are preserved.
This uses the tested Lepton Android 11 SoftOMX ABI. On connection, FramePort
deploys the plugin, codec XML and Podman wrapper once into the agent's shared
`~/.local/share/frameport/video-codec/versions/<manifest-sha256>` store.
Payload checksums are verified before an atomic `current` symlink exposes the
complete version. Older clients cannot downgrade a newer codec revision.
A Frame keeps an installed codec of the same or a newer manifest `revision`
(two PCs with different builds of one revision don't replace each other's), so
`build.py` bumps the revision with every change. A failed install is not
retried during the same connection.
Superseded versions are pruned, keeping the active version and the one it
replaced (a launch that resolved the previous `current` can still mount it).
Games opt in through their recipe: the patch `frame.hw_video_decode` (no APK
change; suggested when the APK decodes video through MediaCodec, ExoPlayer or
VLC) makes the agent give that game's launcher the line that puts the shared
wrapper first on Lepton's PATH (`deployment.json` `hw_video_decode`;
`upgrade_launchers` adds or removes it on existing launchers). Only that game's
matching `podman run` gains a read-only plugin/XML mount and `/dev/video-dec0`.
All other Podman operations pass through. FramePort's setting "Hardware video
decoding" turns it off for every game (`video-codec/disabled`, checked by the
launcher line and the wrapper at every start); `FRAMEPORT_NO_HW_VIDEO=1` in a
game's Steam launch options turns it off for that game. Shared Lepton, drivers and original MP4 assets are unchanged.
Unknown runtime ABIs retain the stock codecs and log why. A future native
Lepton hardware plugin takes precedence.
The wrapper resolves a fallback Podman from PATH outside its own directory
before reading deployment configuration. Lepton's Android-only `env -i PATH`
attach commands use host default paths if their PATH contains no host Podman.
Missing/malformed configuration,
recursive executable paths, mount-preparation errors and failed exec calls
retain the original arguments and launch stock Podman. The agent publishes
`deployment.json` and codec payloads before exposing the complete version.
The merged runtime XML is written to `$XDG_RUNTIME_DIR/frameport-video/`
(never into the verified version directory), named per runtime path, with
process-local temporary filenames, so concurrent app launches do not overwrite
each other's staging.
No package check or MP4 scan selects decoding: the recipe does. Older embedded
codec assets are removed when an APK is rebuilt, and agent 70's per-game
`<base>/frameport-codec` folders are removed once the launcher is converted (a
game that had one, Batman, keeps hardware decoding). The
codec has no OpenXR calls, camera poses or composition changes. Batman's
`surface_native` renderer remains package-scoped and separate. Other apps keep
their existing rendering path; hardware decoding alone does not eliminate
surface upload/conversion or GPU rendering bottlenecks.
The plugin enumerates coded formats and probes each at 1920x1080, not Batman's
8K geometry. Unavailable components are omitted so Android retains its stock
decoders. Actual-session admission can still fail (including Iris's concurrent
session-accounting issue: while any other decoder session is open, 8K is
refused with ENOMEM). SteamVR's link (`vrlinkrunthread`) holds a session for
about 12 s around every game start, when the plugin loads too (measured on the
dev Frame, 2026-10-09), so a refused session is retried every 250 ms until 20 s
after the plugin loaded (later: for 2 s); initialization then falls back to
FFmpeg software decoding inside the already-selected component. VP9 is limited
to 4096x2304 (codec list and component; larger VP9 decodes in software): at
7680x3840 Iris accepted the session but never returned a picture. For every
VP9 frame that isn't shown (alt-ref/hidden frames split from superframes) Iris
returns an empty capture buffer (bytesused 0, no LAST flag, time 0). FFmpeg's
wrapper returned those as pictures and, during the EOS drain, took the first one
for the end of the stream, losing the pictures still in the driver (two-pass
VP9 at 4K: 573 of 600 frames, measured 2026-10-10). `build.py` makes the wrapper
requeue empty non-LAST buffers instead; the drain ends at the LAST buffer, or
after a skipped empty buffer at one silent second, so a driver that ends a drain
with an empty unflagged buffer cannot block. This preserves the client's
buffer/surface contract without requiring the app to retry another codec.
During hardware playback, bounded references to two GOPs of compressed packets
permit software recovery after a driver-reported damaged picture. Replay drops
already-presented timestamps, retaining ordered output and EOS. The cache is
capped at 64 MiB/512 packets; if no usable keyframe remains, ordinary OMX error
handling applies. After recovery, later seeks flush the software decoder for
that component until reset. This is a fallback, not a repair to the Iris driver. VP9 seek tests
on this Frame exercised it. Capture buffers use FFmpeg's general-purpose budget
of 20 at ordinary sizes, which avoids the stall observed with the earlier
four-buffer H.264 configuration. At panorama sizes the budget follows the
standards' reference-picture bound plus four (10 at 8K HEVC/H.264, 13 for VP9),
instead of holding about 1 GB of 8K capture memory. Only hardware-reported
damage triggers recovery; software decoders conceal damage themselves and their
pictures are output, as Android's stock decoders do.
A seek after the stream has fully drained (replay or loop after EOS) restarts
the open hardware session in place with `V4L2_DEC_CMD_START`, keeping its
buffers; this works for all three codecs. The component checks the restart
result through a read-only FFmpeg option. A failed restart closes that session
and continues in software, rather than waiting indefinitely for hardware
output. A seek during playback reopens the
hardware session: on the tested Iris kernel, stopping and restarting a running
session's queues failed the driver's session admission ("current session not
supported", -12) and left buffers unreturned (videobuf2 kernel warnings), so it
is not used. On the same kernel, reopening VP9 after a seek produced a damaged
capture picture and a kernel fault in `lookup_swap_cgroup_id` during unmap, so
VP9 switches to software on a seek during playback rather than reopening that
hardware session; H.264 and HEVC retain hardware decoding after seeking.
This compatibility guard is independent of app/package identity.
Android CODECCONFIG parameter sets are prepended to the first H.264/HEVC
hardware access unit because FFmpeg's V4L2 wrapper does not submit extradata
to the driver. Parameter sets sent again mid-stream without a flush (streaming
apps, restarted encoders) replace the stored configuration and are submitted
in-band before the next access unit, rather than failing the session. Complete-frame parsers identify keyframes for bounded recovery,
including ACodec clients that omit the OMX input SYNCFRAME flag.
On the tested headset, disabling **hardware video decoding in Steam's interface**
and restarting Steam removed that conflicting session. This setting does not
disable the game's Iris decoder. It is a SteamOS-driver workaround, not a kernel
fix; FramePort does not silently change the global Steam setting.
Signalled bitstream colour (primaries, transfer, matrix, range) is reported to
ACodec as Android's stock decoders do, so the surface dataspace can follow it;
container values take precedence. Legacy RGBA conversion uses the resulting
matrix, BT.709 for unsignalled HD (Android's convention), and full-range BT.601
or BT.709 where signalled. A full-range BT.709 table matches the pinned Arm64
libyuv ABI. Both hardware NV12 and software planar conversions produce RGBA
channel order; native YUV output retains its original planes and dataspace.
Batman's native surface renderer and stereo composition are documented in
[Surface video playback](../../docs/SURFACE_VIDEO.md). The original decoded
panorama is retained; projected eye views are generated on the GPU.
Build on Linux/WSL with Android NDK r27c and the tested Lepton rootfs:
```
python native/hevc/build.py --ndk /path/to/android-ndk-r27c --lepton-root /path/to/Lepton/images/rootfs
```
Paths with spaces work: the builder links the repo, the NDK and the rootfs into
a space-free temp dir and compiles through those link paths (FFmpeg's configure
splits `--extra-cflags` on whitespace); with `-no-canonical-prefixes` clang keeps
the link path too, so the prefix maps give the same bytes wherever the real
folders are. Each build starts from freshly extracted FFmpeg sources.
The build requires Linux x86-64, Python 3.12+, Make, Perl, and a working host C
compiler with libc development headers (FFmpeg builds host tools). The NDK
revision is checked against `27.2.12479018`. Previous FFmpeg objects and install
output are discarded. Both FFmpeg and the plugin use that NDK's compiler and
linker. A private copy of its libc++ headers uses Android's platform `__1`
namespace; `-nostdinc++` prevents mixing these with the NDK's `__ndk1` headers.
The build uses libc++'s verbose-abort customization hook to avoid a newer NDK
abort symbol absent from Android 11's platform runtime. It does not edit the NDK
or ship a second libc++.
Compiler file-prefix maps cover source, NDK and runtime paths. FFmpeg's
generated configuration string is normalized separately because it embeds
literal configure arguments. The manifest records the NDK revision and source
checksum. Two clean builds in different directories produced the same codec
binary SHA256; `.comment` identifies NDK Clang/LLD 18.0.3 throughout. The rebuilt
plugin also passed 600 original 8K frames, a flush/seek, 120 replayed frames and
clean shutdown in an isolated headset container.
AOSP headers in `platform/` come from `android-11.0.0_r48` and retain their
original license notices. `fetch_headers.py` records the upstream paths.
The builder downloads the pinned FFmpeg 7.1.1 archive from
https://ffmpeg.org/releases/ffmpeg-7.1.1.tar.xz and verifies SHA256
`733984395e0dbbe5c046abda2dc49a5544e7e0e1e2366bba849222ae9e3a03b1`.
`build.py` applies explicit wrapper changes: VP9 V4L2 uses the same
`vp9_superframe_split` input filter as the software decoder, empty non-LAST
capture buffers (hidden VP9 frames) are requeued rather than output or taken
for the end of a drain, and the V4L2
decoder gains a flush callback that restarts a drained decoder in place
(`V4L2_DEC_CMD_START` with both queues still streaming). Its read-only
`frameport_flush_error` option reports failed or premature restarts to the
component, because `avcodec_flush_buffers` itself has no return value.
Without it a restart after EOS could only close and reopen the hardware
session. `v4l2_export.c.inc` adds a capture-buffer export interface inside FFmpeg,
so the component does not access FFmpeg's private buffer structures. The caller
must retain the AVFrame until GPU work completes. Its LGPL build
configuration and those source changes are recorded in the builder. The license
is included in the shared version directory as `COPYING.FFmpeg`.
`omx_decode_probe.cpp` and `media_codec_probe.cpp` exercise decoder selection,
original-size output, timestamps, flush/seek, EOS and teardown on the headset.
The OMX probe's `restartfail` mode uses the fault shim in
`tests/fixtures/src/video_codec_faults.c` to reject a drained-session restart;
`software_rgba` exercises software fallback and legacy surface conversion
together. `color_probe.cpp` compares both platform NEON conversion paths with
the full-range BT.709 equations, including channel order and extreme values.
The OMX probe's `native` mode hashes original-size Android YUV buffers and checks
GPU submissions and replayed pixels. `native_cpu` disables Vulkan initialization;
`native_importfail` rejects memory imports. Both verify fallback preserves the
same surface contract and pixels without GPU submissions.
The native surface fixtures cover buffer ownership, GPU projection and
composition. See the surface documentation for validation and limitations.
+262
View File
@@ -0,0 +1,262 @@
#!/usr/bin/env python3
"""Build the Android media-service plugin on Linux with NDK r27c and a Lepton rootfs.
The runtime's private SoftOMX ABI is deliberately pinned. This plugin is mounted
only when the tested library fingerprint matches; it never replaces system files.
"""
import argparse
import hashlib
import json
import os
import shutil
import subprocess
import tarfile
import tempfile
import urllib.request
from pathlib import Path
HERE = Path(__file__).resolve().parent
ARTIFACTS = HERE.parents[1] / "artifacts/hevc"
FFMPEG_SHA = "733984395e0dbbe5c046abda2dc49a5544e7e0e1e2366bba849222ae9e3a03b1"
RUNTIME_SHA = "456e912c75cd389abcf6a63bc80e2a53bdc334371d00b200c93680388ae955e2"
NDK_REVISION = "27.2.12479018"
FLUSH_SOURCE = r"""static void v4l2_flush(AVCodecContext *avctx)
{
V4L2m2mPriv *priv = avctx->priv_data;
V4L2m2mContext *s = priv->context;
struct v4l2_decoder_cmd cmd = { .cmd = V4L2_DEC_CMD_START };
av_packet_unref(&s->buf_pkt);
priv->frameport_flush_error = 0;
/* Only a drained decoder restarts in place: it has returned every picture
* and is stopped until V4L2_DEC_CMD_START (both queues keep streaming).
* Restarting a running decoder's queues failed Iris session admission and
* left its buffers unreturned (kernel warnings): callers reopen instead. */
if (!s->draining || !s->capture.done) {
priv->frameport_flush_error = AVERROR(EINVAL);
av_log(avctx, AV_LOG_WARNING, "flush before the drain completed; reopen the decoder\n");
return;
}
if (ioctl(s->fd, VIDIOC_DECODER_CMD, &cmd) < 0) {
priv->frameport_flush_error = AVERROR(errno);
av_log(avctx, AV_LOG_ERROR, "V4L2_DEC_CMD_START after drain: %s\n", av_err2str(priv->frameport_flush_error));
return;
}
s->draining = 0;
s->output.done = s->capture.done = 0;
}
"""
# Iris returns an empty capture buffer (bytesused 0, no LAST flag) for every
# VP9 frame that isn't shown (alt-ref/hidden frames split from superframes).
# FFmpeg's wrapper took any empty buffer during a drain for the end of the
# stream, so the pictures still in the driver after EOS were lost (two-pass VP9:
# 573 of 600), and outside a drain it returned them as pictures without content.
# Requeue them instead; the drain still ends at the LAST buffer (or EPIPE), and
# after a skipped empty buffer a silent second ends it too, so a driver that
# ends a drain with an empty buffer and no LAST flag cannot block the caller.
EMPTY_DECLARE = " .fd = ctx_to_m2mctx(ctx)->fd,\n };\n int i, ret;\n"
EMPTY_POLL = """ for (;;) {
ret = poll(&pfd, 1, timeout);
if (ret > 0)
break;
if (errno == EINTR)
continue;
return NULL;
}
"""
EMPTY_DRAIN = """ if (ctx_to_m2mctx(ctx)->draining && !V4L2_TYPE_IS_OUTPUT(ctx->type)) {
int bytesused"""
EMPTY_SKIP = """ if (!V4L2_TYPE_IS_OUTPUT(ctx->type) && ctx->buffers && buf.index < ctx->num_buffers &&
!(V4L2_TYPE_IS_MULTIPLANAR(buf.type) ? buf.m.planes[0].bytesused : buf.bytesused) &&
!(buf.flags & (V4L2_BUF_FLAG_LAST | V4L2_BUF_FLAG_ERROR))) {
/* an empty picture that doesn't end the stream (Iris: a hidden VP9 frame) */
if (ff_v4l2_buffer_enqueue(&ctx->buffers[buf.index]) < 0) {
ctx->done = 1;
return NULL;
}
skipped_empty = 1;
goto start;
}
"""
def skip_empty_pictures(context_text: str) -> str:
"""libavcodec/v4l2_context.c with empty non-LAST capture buffers requeued (see EMPTY_SKIP)."""
if any(context_text.count(text) != 1 for text in (EMPTY_DECLARE, EMPTY_POLL, EMPTY_DRAIN)):
raise RuntimeError("unexpected FFmpeg V4L2 context source")
return (context_text
.replace(EMPTY_DECLARE, EMPTY_DECLARE.replace("int i, ret;", "int i, ret, skipped_empty = 0;"))
.replace(EMPTY_POLL, EMPTY_POLL.replace(
" ret = poll(&pfd, 1, timeout);\n if (ret > 0)\n break;\n",
" ret = poll(&pfd, 1, skipped_empty && ctx_to_m2mctx(ctx)->draining &&\n"
" (timeout < 0 || timeout > 1000) ? 1000 : timeout);\n"
" if (ret > 0)\n break;\n"
" if (!ret && skipped_empty && ctx_to_m2mctx(ctx)->draining) {\n"
" ctx->done = 1;\n return NULL;\n }\n"))
.replace(EMPTY_DRAIN, EMPTY_SKIP + EMPTY_DRAIN))
def run(args, cwd=None, env=None):
# PWD keeps the shell's (and FFmpeg configure's) idea of the directory on the space-free link path
env = dict(env or os.environ, **({"PWD": str(cwd)} if cwd else {}))
subprocess.run([str(a) for a in args], check=True, cwd=cwd, env=env)
def space_free(links: Path, name: str, target: Path) -> Path:
"""FFmpeg's configure splits --extra-cflags on whitespace, so a repo/NDK/rootfs path with spaces breaks the
build. Build through unresolved symlinks in a space-free temp dir; the prefix maps name these link paths, so the
output is the same bytes wherever the real folders are."""
link = links / name
link.symlink_to(target, target_is_directory=True)
return link
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--ndk", type=Path, required=True)
parser.add_argument("--lepton-root", type=Path, required=True)
args = parser.parse_args()
ndk_root = args.ndk.resolve()
if not any(line.strip() == f"Pkg.Revision = {NDK_REVISION}"
for line in (ndk_root / "source.properties").read_text().splitlines()):
parser.error(f"use Android NDK r27c ({NDK_REVISION})")
ndk = ndk_root / "toolchains/llvm/prebuilt/linux-x86_64"
if not (ndk / "bin/clang++").exists():
parser.error("use the Linux NDK r27c (Windows: run this builder inside WSL)")
runtime = args.lepton_root.resolve()
with tempfile.TemporaryDirectory(prefix="fp-hevc-") as tmp:
links = Path(tmp)
if any(c.isspace() for c in str(links)):
parser.error(f"temp dir {links} has spaces: set TMPDIR to a folder without spaces")
build(parser, space_free(links, "repo", HERE.parents[1]), space_free(links, "ndk", ndk_root),
space_free(links, "rootfs", runtime))
def build(parser, repo: Path, ndk_root: Path, runtime: Path):
here = repo / HERE.relative_to(HERE.parents[1])
artifacts = repo / ARTIFACTS.relative_to(HERE.parents[1])
ndk = ndk_root / "toolchains/llvm/prebuilt/linux-x86_64"
if hashlib.sha256((runtime / "vendor/lib64/libstagefright_softomx.so").read_bytes()).hexdigest() != RUNTIME_SHA:
parser.error("unverified SoftOMX ABI; validate and update the fingerprint before rebuilding")
cache = here.parent / ".cache/hevc"
cache.mkdir(parents=True, exist_ok=True)
archive = cache / "ffmpeg-7.1.1.tar.xz"
if not archive.exists():
urllib.request.urlretrieve("https://ffmpeg.org/releases/ffmpeg-7.1.1.tar.xz", archive)
if hashlib.sha256(archive.read_bytes()).hexdigest() != FFMPEG_SHA:
raise RuntimeError("FFmpeg source checksum mismatch")
source = cache / "ffmpeg-7.1.1"
install = cache / "ffmpeg-install"
# Discard previous objects/configuration so a host compiler or changed
# configure flags cannot silently survive a rebuild (a distclean keyed on
# config.mak never ran: FFmpeg 7 keeps it in ffbuild/), and start from
# unpatched sources (the edits below are applied once).
for old in (source, install):
if old.exists():
shutil.rmtree(old)
with tarfile.open(archive) as tar:
tar.extractall(cache, filter="data")
# Unlike FFmpeg's software VP9 decoder, its V4L2 wrapper does not split
# packed VP9 superframes. Iris requires the same individual-frame input.
decoder_source = source / "libavcodec/v4l2_m2m_dec.c"
decoder_text = decoder_source.read_text()
original_vp9 = 'M2MDEC(vp9, "VP9", AV_CODEC_ID_VP9, NULL);'
if decoder_text.count(original_vp9) != 1:
raise RuntimeError("unexpected FFmpeg VP9 wrapper source")
decoder_text = decoder_text.replace(original_vp9, 'M2MDEC(vp9, "VP9", AV_CODEC_ID_VP9, "vp9_superframe_split");')
# The wrapper has no flush callback, so restarting after EOS (replay, loop)
# could only close and reopen the hardware session. Add the stateful
# decoder's restart after a completed drain (V4L2_DEC_CMD_START).
close_fn = "static av_cold int v4l2_decode_close(AVCodecContext *avctx)\n"
close_cb = " .close = v4l2_decode_close, \\\n"
if decoder_text.count(close_fn) != 1 or decoder_text.count(close_cb) != 1:
raise RuntimeError("unexpected FFmpeg V4L2 decoder source")
decoder_text = decoder_text.replace(close_fn, FLUSH_SOURCE + "\n" + close_fn).replace(
close_cb, close_cb + " .flush = v4l2_flush, \\\n")
decoder_source.write_text(decoder_text)
# avcodec_flush_buffers has no return value. Expose restart status through
# an AVOption rather than having the OMX component inspect private structs.
private_header = source / "libavcodec/v4l2_m2m.h"
private_text = private_header.read_text()
field = " int num_capture_buffers;\n} V4L2m2mPriv;"
option_end = " { NULL},\n};"
if private_text.count(field) != 1 or decoder_text.count(option_end) != 1:
raise RuntimeError("unexpected FFmpeg V4L2 private options source")
private_header.write_text(private_text.replace(field, " int num_capture_buffers;\n"
" int frameport_flush_error;\n} V4L2m2mPriv;"))
decoder_source.write_text(decoder_text.replace(option_end,
' { "frameport_flush_error", "Last drained-session restart error",\n'
' OFFSET(frameport_flush_error), AV_OPT_TYPE_INT, {.i64 = 0}, INT_MIN, 0,\n'
' FLAGS | AV_OPT_FLAG_READONLY },\n' + option_end))
context_source = source / "libavcodec/v4l2_context.c"
context_source.write_text(skip_empty_pictures(context_source.read_text()))
buffers_source = source / "libavcodec/v4l2_buffers.c"
buffers_source.write_text(buffers_source.read_text() + (here / "v4l2_export.c.inc").read_text())
env = dict(os.environ, PATH=str(ndk / "bin") + os.pathsep + os.environ["PATH"])
prefix_maps = [f"-ffile-prefix-map={repo}=.", f"-ffile-prefix-map={ndk_root}=android-ndk-r27c",
f"-ffile-prefix-map={runtime}=lepton-rootfs",
# clang otherwise resolves its own path (through the link) and embeds its real include dir
"-no-canonical-prefixes"]
run([
source / "configure", f"--prefix={install}", "--target-os=android", "--arch=aarch64",
"--enable-cross-compile", "--cc=aarch64-linux-android30-clang", "--cxx=aarch64-linux-android30-clang++",
"--ld=aarch64-linux-android30-clang", "--ar=llvm-ar", "--nm=llvm-nm", "--ranlib=llvm-ranlib",
"--extra-cflags=" + " ".join(prefix_maps),
"--extra-cxxflags=" + " ".join(prefix_maps),
"--disable-everything", "--disable-autodetect", "--enable-v4l2-m2m", "--disable-programs",
"--disable-doc", "--enable-pic", "--enable-static", "--disable-shared",
"--enable-decoder=hevc_v4l2m2m,h264_v4l2m2m,vp9_v4l2m2m,hevc,h264,vp9",
"--enable-parser=hevc,h264,vp9", "--enable-bsf=hevc_mp4toannexb,h264_mp4toannexb,vp9_superframe_split",
"--enable-demuxer=mov,matroska", "--enable-protocol=file", "--enable-avcodec", "--enable-avformat",
"--enable-avutil", "--disable-avdevice", "--disable-avfilter", "--disable-swscale",
"--disable-swresample", "--disable-postproc",
], cwd=source, env=env)
# FFmpeg embeds its configure command as a runtime diagnostic string.
# Prefix-map flags cannot rewrite string literals, so normalize that
# generated string too, retaining the options without host paths.
config = source / "config.h"
text = config.read_text()
for path, replacement in ((ndk_root, "android-ndk-r27c"), (runtime, "lepton-rootfs"), (repo, ".")):
text = text.replace(str(path), replacement)
config.write_text(text)
run(["make", f"-j{os.cpu_count() or 4}"], cwd=source, env=env)
run(["make", "install"], cwd=source, env=env)
# Use the platform libc++ namespace. Do not change the NDK's own headers or
# distribute another libc++, which would create a conflicting private ABI.
cpp = cache / "cpp"
shutil.copytree(ndk / "sysroot/usr/include/c++/v1", cpp, dirs_exist_ok=True)
site = cpp / "__config_site"
site.write_text(site.read_text().replace("_LIBCPP_ABI_NAMESPACE __ndk1", "_LIBCPP_ABI_NAMESPACE __1"))
artifacts.mkdir(parents=True, exist_ok=True)
run([
ndk / "bin/aarch64-linux-android30-clang++", "-std=gnu++17", "-O3", "-fPIC", "-shared",
*prefix_maps,
"-fno-rtti", "-fno-exceptions", "-nostdlib++", "-nostdinc++", "-Wall", "-Wextra", "-Werror",
"-D_LIBCPP_VERBOSE_ABORT(...)=__builtin_abort()",
"-Wno-unused-private-field", "-isystem", cpp, "-I", here / "platform",
"-I", here / "platform/media/openmax", "-I", install / "include", here / "frameport_hevc.cpp",
"-L", install / "lib", "-lavcodec", "-lavutil", "-L", runtime / "vendor/lib64",
"-L", runtime / "system/lib64", "-lstagefright_softomx", "-lstagefright_foundation", "-lutils",
"-llog", "-lnativewindow", "-lyuv", "-lvulkan", "-l:libc++.so", "-lm", "-ldl", "-Wl,--no-undefined",
"-Wl,-z,max-page-size=16384",
"-Wl,-soname,libstagefrighthw.so", "-o", artifacts / "libstagefrighthw.so",
], env=env)
for name in ("podman.py", "media_codecs_frameport.xml"):
shutil.copyfile(here / name, artifacts / (name + ".txt" if name == "podman.py" else name))
shutil.copyfile(source / "COPYING.LGPLv2.1", artifacts / "COPYING.FFmpeg")
files = {}
for name in ("libstagefrighthw.so", "podman.py", "media_codecs_frameport.xml", "COPYING.FFmpeg"):
path = artifacts / (name + ".txt" if name == "podman.py" else name)
files[name] = hashlib.sha256(path.read_bytes()).hexdigest()
# bump the revision with every change: a Frame keeps an installed codec of the same or a newer revision
manifest = {"revision": 8, "runtime_sha256": RUNTIME_SHA, "files": files,
"codecs": ["video/hevc", "video/avc", "video/x-vnd.on2.vp9"],
"build": {"ndk_revision": NDK_REVISION, "ffmpeg_source_sha256": FFMPEG_SHA}}
(artifacts / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n")
if __name__ == "__main__":
main()
+36
View File
@@ -0,0 +1,36 @@
// SPDX-License-Identifier: GPL-3.0-only
// Compare the actual platform NEON row functions with the full-range BT.709
// equations, including shadows/highlights and RGBA channel order in both paths.
#include "yuv_color.h"
#include <cassert>
#include <cstdio>
#include <cmath>
#include <cstring>
#include <initializer_list>
extern "C" {
void NV21ToARGBRow_Any_NEON(const uint8_t*,const uint8_t*,uint8_t*,const YuvConstants*,int);
void I422ToARGBRow_Any_NEON(const uint8_t*,const uint8_t*,const uint8_t*,uint8_t*,const YuvConstants*,int);
}
static int clipped(double v) {int n=(int)std::lround(v);return n<0?0:n>255?255:n;}
int main() {
unsigned checked=0;
for(unsigned width : {2u,8u,18u})
for(int y : {0,8,16,64,128,235,247,255})
for(int u : {16,64,128,192,240})
for(int v : {16,64,128,192,240}) {
uint8_t yy[32],uu[16],vv[16],uv[32],nv[128],planar[128];
memset(yy,y,sizeof yy);memset(uu,u,sizeof uu);memset(vv,v,sizeof vv);
for(unsigned i=0;i<width/2;i++){uv[2*i]=u;uv[2*i+1]=v;}
NV21ToARGBRow_Any_NEON(yy,uv,nv,&frameport_full709_rgba,width);
I422ToARGBRow_Any_NEON(yy,vv,uu,planar,&frameport_full709_rgba,width);
int expected[]={clipped(y+1.5748*(v-128)),clipped(y-0.187324*(u-128)-0.468124*(v-128)),
clipped(y+1.8556*(u-128)),255};
for(unsigned x=0;x<width;x++)for(unsigned c=0;c<4;c++) {
assert(nv[4*x+c]==planar[4*x+c]);
assert(std::abs((int)nv[4*x+c]-expected[c])<=2);
if(u==128 && v==128 && c<3)assert(nv[4*x+c]==y);
}
checked++;
}
printf("PASS: %u full-range BT.709 colour/width combinations, planar and NV12 RGBA\n",checked);
}
+139
View File
@@ -0,0 +1,139 @@
// SPDX-License-Identifier: GPL-3.0-only
// Import linear decoder/Android DMA buffers. Ownership returns to the foreign
// producer/consumer before the completed fence permits either buffer to move.
#pragma once
#include <vulkan/vulkan.h>
#include <unistd.h>
#include <sys/stat.h>
#include <cstdio>
#include <cstring>
#include <vector>
class FramePortDmaCopy {
VkInstance instance=VK_NULL_HANDLE;
VkPhysicalDevice physical=VK_NULL_HANDLE;
VkDevice device=VK_NULL_HANDLE;
VkQueue queue=VK_NULL_HANDLE;
VkCommandPool pool=VK_NULL_HANDLE;
VkCommandBuffer command=VK_NULL_HANDLE;
VkFence completed=VK_NULL_HANDLE;
uint32_t family=0;
PFN_vkGetMemoryFdPropertiesKHR properties=nullptr;
struct Buffer {VkBuffer buffer=VK_NULL_HANDLE;VkDeviceMemory memory=VK_NULL_HANDLE;};
struct Entry {dev_t device;ino_t inode;VkBufferUsageFlags usage;Buffer imported;};
std::vector<Entry> cache;
bool attempted=false,ready=false,unsafe=false;
template<class T>static T info(VkStructureType type){T value={};value.sType=type;return value;}
bool check(VkResult r,const char *where){if(r!=VK_SUCCESS)ALOGW("GPU video %s: %d",where,r);return r==VK_SUCCESS;}
bool import(int fd,VkBufferUsageFlags usage,Buffer &out) {
struct stat st={};if(fstat(fd,&st)||st.st_size<=0)return false;
auto query=info<VkPhysicalDeviceExternalBufferInfo>(VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_BUFFER_INFO);
query.usage=usage;query.handleType=VK_EXTERNAL_MEMORY_HANDLE_TYPE_DMA_BUF_BIT_EXT;
auto caps=info<VkExternalBufferProperties>(VK_STRUCTURE_TYPE_EXTERNAL_BUFFER_PROPERTIES);
vkGetPhysicalDeviceExternalBufferProperties(physical,&query,&caps);
if(!(caps.externalMemoryProperties.externalMemoryFeatures&VK_EXTERNAL_MEMORY_FEATURE_IMPORTABLE_BIT))return false;
auto external=info<VkExternalMemoryBufferCreateInfo>(VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO);
external.handleTypes=query.handleType;
auto create=info<VkBufferCreateInfo>(VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO);create.pNext=&external;
create.size=st.st_size;create.usage=usage;create.sharingMode=VK_SHARING_MODE_EXCLUSIVE;
if(!check(vkCreateBuffer(device,&create,nullptr,&out.buffer),"create buffer"))return false;
VkMemoryRequirements needs;vkGetBufferMemoryRequirements(device,out.buffer,&needs);
auto fdprops=info<VkMemoryFdPropertiesKHR>(VK_STRUCTURE_TYPE_MEMORY_FD_PROPERTIES_KHR);
if(!check(properties(device,query.handleType,fd,&fdprops),"fd properties"))return false;
uint32_t bits=needs.memoryTypeBits&fdprops.memoryTypeBits;
if(!bits || needs.size>(VkDeviceSize)st.st_size){ALOGW("GPU import layout mismatch: size %llu/%lld bits %#x",
(unsigned long long)needs.size,(long long)st.st_size,bits);return false;}
auto imported=info<VkImportMemoryFdInfoKHR>(VK_STRUCTURE_TYPE_IMPORT_MEMORY_FD_INFO_KHR);
imported.handleType=query.handleType;imported.fd=dup(fd);if(imported.fd<0)return false;
auto dedicated=info<VkMemoryDedicatedAllocateInfo>(VK_STRUCTURE_TYPE_MEMORY_DEDICATED_ALLOCATE_INFO);
dedicated.buffer=out.buffer;
if(caps.externalMemoryProperties.externalMemoryFeatures&VK_EXTERNAL_MEMORY_FEATURE_DEDICATED_ONLY_BIT)imported.pNext=&dedicated;
auto allocate=info<VkMemoryAllocateInfo>(VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO);allocate.pNext=&imported;
allocate.allocationSize=st.st_size;allocate.memoryTypeIndex=__builtin_ctz(bits);
if(!check(vkAllocateMemory(device,&allocate,nullptr,&out.memory),"import memory")){close(imported.fd);return false;}
return check(vkBindBufferMemory(device,out.buffer,out.memory,0),"bind buffer");
}
void release(Buffer &b){if(b.buffer)vkDestroyBuffer(device,b.buffer,nullptr);if(b.memory)vkFreeMemory(device,b.memory,nullptr);b={};}
bool cached(int fd,VkBufferUsageFlags usage,Buffer &out){
struct stat st={};if(fstat(fd,&st))return false;
for(auto &e:cache)if(e.device==st.st_dev&&e.inode==st.st_ino&&e.usage==usage){out=e.imported;return true;}
Buffer b;bool ok=import(fd,usage,b);if(!ok){release(b);return false;}
cache.push_back({st.st_dev,st.st_ino,usage,b});out=b;return true;
}
public:
void clearBuffers(){if(device){vkDeviceWaitIdle(device);for(auto &e:cache)release(e.imported);cache.clear();}}
bool available(){if(!attempted){attempted=true;ready=init();}return ready;}
bool failedAfterSubmit()const{return unsafe;}
bool init(){
auto app=info<VkApplicationInfo>(VK_STRUCTURE_TYPE_APPLICATION_INFO);app.apiVersion=VK_API_VERSION_1_1;
auto create=info<VkInstanceCreateInfo>(VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO);create.pApplicationInfo=&app;
if(!check(vkCreateInstance(&create,nullptr,&instance),"create instance"))return false;
uint32_t n=0;vkEnumeratePhysicalDevices(instance,&n,nullptr);if(!n)return false;
std::vector<VkPhysicalDevice> devices(n);vkEnumeratePhysicalDevices(instance,&n,devices.data());physical=devices[0];
VkPhysicalDeviceProperties p;vkGetPhysicalDeviceProperties(physical,&p);
if(p.apiVersion<VK_API_VERSION_1_1)return false;
vkEnumerateDeviceExtensionProperties(physical,nullptr,&n,nullptr);std::vector<VkExtensionProperties> extensions(n);
vkEnumerateDeviceExtensionProperties(physical,nullptr,&n,extensions.data());
const char *required[]={VK_KHR_EXTERNAL_MEMORY_FD_EXTENSION_NAME,VK_EXT_EXTERNAL_MEMORY_DMA_BUF_EXTENSION_NAME,VK_EXT_QUEUE_FAMILY_FOREIGN_EXTENSION_NAME};
for(auto wanted:required){bool found=false;for(auto &e:extensions)if(!strcmp(e.extensionName,wanted))found=true;
if(!found)return false;}
vkGetPhysicalDeviceQueueFamilyProperties(physical,&n,nullptr);std::vector<VkQueueFamilyProperties> families(n);
vkGetPhysicalDeviceQueueFamilyProperties(physical,&n,families.data());
family=n;for(uint32_t i=0;i<n;i++)if(families[i].queueFlags&VK_QUEUE_TRANSFER_BIT){family=i;break;}
if(family==n)return false;
float priority=1;auto q=info<VkDeviceQueueCreateInfo>(VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO);
q.queueFamilyIndex=family;q.queueCount=1;q.pQueuePriorities=&priority;
auto d=info<VkDeviceCreateInfo>(VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO);d.queueCreateInfoCount=1;d.pQueueCreateInfos=&q;
d.enabledExtensionCount=3;d.ppEnabledExtensionNames=required;
if(!check(vkCreateDevice(physical,&d,nullptr,&device),"create device"))return false;
vkGetDeviceQueue(device,family,0,&queue);
properties=(PFN_vkGetMemoryFdPropertiesKHR)vkGetDeviceProcAddr(device,"vkGetMemoryFdPropertiesKHR");if(!properties)return false;
auto cp=info<VkCommandPoolCreateInfo>(VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO);cp.queueFamilyIndex=family;
cp.flags=VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT;
if(!check(vkCreateCommandPool(device,&cp,nullptr,&pool),"create pool"))return false;
auto a=info<VkCommandBufferAllocateInfo>(VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO);
a.commandPool=pool;a.level=VK_COMMAND_BUFFER_LEVEL_PRIMARY;a.commandBufferCount=1;
auto f=info<VkFenceCreateInfo>(VK_STRUCTURE_TYPE_FENCE_CREATE_INFO);
return check(vkAllocateCommandBuffers(device,&a,&command),"allocate command buffer")&&
check(vkCreateFence(device,&f,nullptr,&completed),"create fence");
}
bool copy(int source,int dest,const VkBufferCopy *regions,unsigned count){
unsafe=false;
struct stat s={},d={};
if(!count||fstat(source,&s)||fstat(dest,&d))return false;
for(unsigned i=0;i<count;i++)if(!regions[i].size||(regions[i].size&3)||
(regions[i].srcOffset&3)||(regions[i].dstOffset&3)||regions[i].srcOffset>(VkDeviceSize)s.st_size||
regions[i].dstOffset>(VkDeviceSize)d.st_size||regions[i].size>(VkDeviceSize)s.st_size-regions[i].srcOffset||
regions[i].size>(VkDeviceSize)d.st_size-regions[i].dstOffset)return false;
// Make room for the pair before resolving either handle. Evicting
// while importing the destination could destroy the source handle.
if(cache.size()>30)clearBuffers();
Buffer buffers[2];bool ok=cached(source,VK_BUFFER_USAGE_TRANSFER_SRC_BIT,buffers[0])&&
cached(dest,VK_BUFFER_USAGE_TRANSFER_DST_BIT,buffers[1]);
VkCommandBuffer cmd=command;VkFence fence=completed;
if(ok)ok=check(vkResetCommandBuffer(cmd,0),"reset command buffer")&&check(vkResetFences(device,1,&fence),"reset fence");
if(ok){
auto b=info<VkCommandBufferBeginInfo>(VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO);b.flags=VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT;
ok=check(vkBeginCommandBuffer(cmd,&b),"begin command buffer");
VkBufferMemoryBarrier barriers[2]={};
for(unsigned i=0;i<2;i++){
barriers[i].sType=VK_STRUCTURE_TYPE_BUFFER_MEMORY_BARRIER;barriers[i].dstAccessMask=i?VK_ACCESS_TRANSFER_WRITE_BIT:VK_ACCESS_TRANSFER_READ_BIT;
barriers[i].srcQueueFamilyIndex=VK_QUEUE_FAMILY_FOREIGN_EXT;barriers[i].dstQueueFamilyIndex=family;
barriers[i].buffer=buffers[i].buffer;barriers[i].size=VK_WHOLE_SIZE;
}
vkCmdPipelineBarrier(cmd,VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT,VK_PIPELINE_STAGE_TRANSFER_BIT,0,0,nullptr,2,barriers,0,nullptr);
vkCmdCopyBuffer(cmd,buffers[0].buffer,buffers[1].buffer,count,regions);
for(auto &barrier:barriers){barrier.srcAccessMask=barrier.dstAccessMask;barrier.dstAccessMask=0;
barrier.srcQueueFamilyIndex=family;barrier.dstQueueFamilyIndex=VK_QUEUE_FAMILY_FOREIGN_EXT;}
vkCmdPipelineBarrier(cmd,VK_PIPELINE_STAGE_TRANSFER_BIT,VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT,0,0,nullptr,2,barriers,0,nullptr);
ok=ok&&check(vkEndCommandBuffer(cmd),"end command buffer");
}
if(ok){auto s=info<VkSubmitInfo>(VK_STRUCTURE_TYPE_SUBMIT_INFO);s.commandBufferCount=1;s.pCommandBuffers=&cmd;
ok=check(vkQueueSubmit(queue,1,&s,fence),"submit");
if(ok){ok=check(vkWaitForFences(device,1,&fence,VK_TRUE,10000000000ull),"wait fence");unsafe=!ok;}}
if(!ok&&device)vkDeviceWaitIdle(device);
return ok;
}
~FramePortDmaCopy(){if(device){vkDeviceWaitIdle(device);for(auto &e:cache)release(e.imported);
if(completed)vkDestroyFence(device,completed,nullptr);if(pool)vkDestroyCommandPool(device,pool,nullptr);
vkDestroyDevice(device,nullptr);}if(instance)vkDestroyInstance(instance,nullptr);}
};
+70
View File
@@ -0,0 +1,70 @@
from pathlib import Path
import base64,re,urllib.request,urllib.error,json,concurrent.futures
root=Path(__file__).parent/'platform'
root.mkdir(exist_ok=True)
tag='android-11.0.0_r48'
trees=[
('frameworks/av','media/libstagefright/omx/include'),
('frameworks/av','media/libstagefright/foundation/include'),
('frameworks/av','media/libstagefright/include'),
('frameworks/av','media/libmedia/include'),
('frameworks/native','headers'),
('frameworks/native','headers/media_plugin'),
('frameworks/native','headers/media_plugin/include'),
('frameworks/native','libs/nativebase/include'),
('frameworks/native','libs/nativewindow/include'),
('frameworks/native','libs/ui/include'),
('frameworks/native','libs/binder/include'),
('system/core','libutils/include'),
('system/core','libcutils/include'),
('system/core','libsystem/include'),
('system/core','base/include'),
('system/core','libbacktrace/include'),
('system/logging','liblog/include'),
('system/core','liblog/include'),
('hardware/libhardware','include'),
('system/core','libsystem/include'),
('system/media','audio/include'),
]
seen=set()
def fetch(name,parent=None):
if name in seen:return
seen.add(name)
dest=root/name
if dest.exists(): data=dest.read_bytes()
else:
choices=list(trees)
preferred={'utils/':'libutils','cutils/':'libcutils','android-base/':'base/',
'log/':'liblog','system/':'libsystem','hardware/':'libhardware'}
for prefix,prefer in preferred.items():
if name.startswith(prefix):choices.sort(key=lambda item:prefer not in '/'.join(item))
if parent:choices=[parent]+choices
data=None
for repo,path in choices:
url=f'https://android.googlesource.com/platform/{repo}/+/refs/tags/{tag}/{path}/{name}?format=TEXT'
for attempt in range(3):
try:
data=base64.b64decode(urllib.request.urlopen(url,timeout=15).read());break
except urllib.error.HTTPError as exc:
if exc.code==404:break
except urllib.error.URLError:pass
if data is not None:break
if data is None:
print('Unresolved platform include:',name,flush=True);return
dest.parent.mkdir(parents=True,exist_ok=True);dest.write_bytes(data)
print('Fetched',name,flush=True)
# Only follow Android-specific and same-directory quote headers; C/C++
# standard and Linux headers are supplied by the checked NDK sysroot.
for delim,inc in re.findall(rb'#\s*include\s*([<"])([^>"\n]+)',data):
inc=inc.decode()
if inc.startswith(('utils/','cutils/','media/','system/','hardware/','android-base/','log/','ui/','nativebase/','binder/','backtrace/','android/')):
fetch(inc)
elif delim==b'"' and '/' not in inc:
fetch(str(Path(name).parent/inc).replace('\\','/'))
for n in ['media/stagefright/omx/SoftVideoDecoderOMXComponent.h',
'media/stagefright/foundation/AMessage.h','media/hardware/OMXPluginBase.h',
'utils/String8.h','utils/Log.h']:
fetch(n)
print('Platform header closure:',len(list(root.rglob('*.h'))),flush=True)
+759
View File
@@ -0,0 +1,759 @@
// SPDX-License-Identifier: GPL-3.0-only
// Android OMX integration for Iris stateful V4L2 video decoding. No OpenXR
// layer/pose manipulation, asset rewriting or package-specific behaviour.
#define LOG_TAG "FramePortVideo"
#include <media/stagefright/omx/SoftVideoDecoderOMXComponent.h>
#include <media/hardware/OMXPluginBase.h>
#include <media/openmax/OMX_IndexExt.h>
#include <media/hardware/HardwareAPI.h>
#include <media/stagefright/foundation/ColorUtils.h>
#include <android/hardware_buffer.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <linux/videodev2.h>
#include <utils/String8.h>
#include <arm_neon.h>
#include <cstring>
#include <cerrno>
#include <pthread.h>
#include <time.h>
#include <poll.h>
#include "yuv_copy.h"
#include "yuv_color.h"
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavutil/opt.h>
extern const YuvConstants kYvuH709Constants, kYvuI601Constants;
extern const YuvConstants kYvuJPEGConstants;
void NV21ToARGBRow_Any_NEON(const uint8_t*,const uint8_t*,uint8_t*,const YuvConstants*,int);
void I422ToARGBRow_Any_NEON(const uint8_t*,const uint8_t*,const uint8_t*,uint8_t*,const YuvConstants*,int);
AHardwareBuffer *ANativeWindowBuffer_getHardwareBuffer(ANativeWindowBuffer*);
const native_handle_t *AHardwareBuffer_getNativeHandle(const AHardwareBuffer*);
int frameport_v4l2_export_frame(const AVFrame*);
}
#include "dma_copy.h"
namespace android {
static const CodecProfileLevel hevc_profiles[]={{OMX_VIDEO_HEVCProfileMain,OMX_VIDEO_HEVCMainTierLevel62}};
// Android reports constrained AVC streams under their own profile values;
// stock decoders list them so format-aware clients can match them.
static const CodecProfileLevel avc_profiles[]={
{(OMX_U32)OMX_VIDEO_AVCProfileConstrainedBaseline,OMX_VIDEO_AVCLevel62},
{OMX_VIDEO_AVCProfileBaseline,OMX_VIDEO_AVCLevel62},
{(OMX_U32)OMX_VIDEO_AVCProfileConstrainedHigh,OMX_VIDEO_AVCLevel62},
{OMX_VIDEO_AVCProfileMain,OMX_VIDEO_AVCLevel62},
{OMX_VIDEO_AVCProfileHigh,OMX_VIDEO_AVCLevel62}};
static const CodecProfileLevel vp9_profiles[]={{OMX_VIDEO_VP9Profile0,OMX_VIDEO_VP9Level62}};
struct DecoderKind {
const char *name,*role,*mime,*hardware,*software;
OMX_VIDEO_CODINGTYPE coding;
uint32_t fourcc;
const CodecProfileLevel *profiles;
unsigned profile_count;
// What Iris returns pictures for on the Frame: VP9 at 7680x3840 opened but
// never returned one (2026-10-09), so larger VP9 decodes in software.
unsigned max_width,max_height;
};
static const DecoderKind kinds[]={
{"OMX.frameport.hevc.decoder","video_decoder.hevc","video/hevc","hevc_v4l2m2m","hevc",
OMX_VIDEO_CodingHEVC,V4L2_PIX_FMT_HEVC,hevc_profiles,1,8192,8192},
{"OMX.frameport.avc.decoder","video_decoder.avc","video/avc","h264_v4l2m2m","h264",
OMX_VIDEO_CodingAVC,V4L2_PIX_FMT_H264,avc_profiles,5,8192,8192},
{"OMX.frameport.vp9.decoder","video_decoder.vp9","video/x-vnd.on2.vp9","vp9_v4l2m2m","vp9",
OMX_VIDEO_CodingVP9,V4L2_PIX_FMT_VP9,vp9_profiles,1,4096,2304}};
static constexpr unsigned kind_count=sizeof(kinds)/sizeof(kinds[0]);
// When the media service loaded the plugin (= the container's start).
static int64_t plugin_loaded_ns=0;
class ColorWorkers {
struct Worker {ColorWorkers *owner;unsigned band,seen;pthread_t thread;};
Worker workers[3];unsigned count=0,generation=0,complete=0;
bool started=false,stopping=false;
pthread_mutex_t mutex=PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t work=PTHREAD_COND_INITIALIZER,done=PTHREAD_COND_INITIALIZER;
AVFrame *frame=nullptr;uint8_t *output=nullptr;size_t stride=0;
const YuvConstants *matrix=nullptr;
FramePortYuvPlane planes[3]={};bool native_yuv=false;
void rows(unsigned band) {
unsigned bands=count+1;
if(frame->format==AV_PIX_FMT_YUV420P || frame->format==AV_PIX_FMT_YUVJ420P){
unsigned begin=(frame->height/2)*band/bands,end=(frame->height/2)*(band+1)/bands;
if(native_yuv){
for(unsigned plane=0;plane<3;plane++){
unsigned first=plane?begin:2*begin,last=plane?end:2*end,width=plane?frame->width/2:frame->width;
for(unsigned row=first;row<last;row++){
const uint8_t *src=frame->data[plane]+row*frame->linesize[plane];
uint8_t *dst=planes[plane].data+row*planes[plane].row_stride;
if(planes[plane].pixel_stride==1)memcpy(dst,src,width);
else for(unsigned x=0;x<width;x++)dst[x*planes[plane].pixel_stride]=src[x];
}
}
}else for(unsigned row=2*begin;row<2*end;row++)
I422ToARGBRow_Any_NEON(frame->data[0]+row*frame->linesize[0],
frame->data[2]+(row/2)*frame->linesize[2],frame->data[1]+(row/2)*frame->linesize[1],
output+row*stride*4,matrix,frame->width);
return;
}
if(native_yuv){
frameport_yuv_copy_band(frame->data[0],frame->data[1],frame->width,frame->height,
frame->linesize[0],frame->linesize[1],planes,band,bands);return;
}
int begin=frame->height*band/bands,end=frame->height*(band+1)/bands;
for(int row=begin;row<end;row++)
NV21ToARGBRow_Any_NEON(frame->data[0]+row*frame->linesize[0],
frame->data[1]+(row/2)*frame->linesize[1],output+row*stride*4,matrix,frame->width);
}
static void *thread(void *arg) {
auto &worker=*static_cast<Worker*>(arg);auto &pool=*worker.owner;unsigned seen=worker.seen;
pthread_mutex_lock(&pool.mutex);
for(;;) {
while(!pool.stopping && seen==pool.generation)pthread_cond_wait(&pool.work,&pool.mutex);
if(pool.stopping)break;
seen=pool.generation;pthread_mutex_unlock(&pool.mutex);
pool.rows(worker.band);
pthread_mutex_lock(&pool.mutex);
if(++pool.complete==pool.count)pthread_cond_signal(&pool.done);
}
pthread_mutex_unlock(&pool.mutex);return nullptr;
}
public:
void convert(AVFrame *source,uint8_t *destination,size_t pitch,const YuvConstants *constants,
const AHardwareBuffer_Planes *native=nullptr) {
// Start helpers when a large picture first appears, including after
// an adaptive stream begins at a lower resolution.
if(!started && source->width*source->height>=4096*2048) {
started=true;
for(unsigned i=0;i<3;i++) {
// Only this thread advances the generation, so a helper
// created now waits for the next picture, never a stale one.
workers[i]={this,i+1,generation,{}};
if(pthread_create(&workers[i].thread,nullptr,thread,&workers[i]))break;
count++;
}
}
pthread_mutex_lock(&mutex);
frame=source;output=destination;stride=pitch;matrix=constants;complete=0;generation++;
native_yuv=native!=nullptr;
if(native)for(unsigned i=0;i<3;i++)planes[i]={static_cast<uint8_t*>(native->planes[i].data),
native->planes[i].rowStride,native->planes[i].pixelStride};
pthread_cond_broadcast(&work);pthread_mutex_unlock(&mutex);
rows(0);
pthread_mutex_lock(&mutex);
while(complete<count)pthread_cond_wait(&done,&mutex);
pthread_mutex_unlock(&mutex);
}
~ColorWorkers() {
pthread_mutex_lock(&mutex);stopping=true;pthread_cond_broadcast(&work);pthread_mutex_unlock(&mutex);
for(unsigned i=0;i<count;i++)pthread_join(workers[i].thread,nullptr);
pthread_cond_destroy(&work);pthread_cond_destroy(&done);pthread_mutex_destroy(&mutex);
}
};
class FramePortVideo final : public SoftVideoDecoderOMXComponent {
const DecoderKind &kind;
ColorWorkers color;
FramePortDmaCopy gpu;
bool gpu_allowed=true;
AVCodecContext *codec=nullptr;
AVCodecParserContext *parser=nullptr;
AVFrame *frame=av_frame_alloc();
uint8_t *config=nullptr;
size_t config_size=0;
bool failed=false,held=false,drained=false,replace_config=false,surface=false;
bool native_enabled=false,metadata=false;
bool hardware_active=false,prefer_software=false,eos_submitted=false,replay_eos=false;
bool prepend_config=false;
Vector<AVPacket*> history;
size_t history_bytes=0;
ssize_t replay_next=-1;
int64_t last_output_pts=AV_NOPTS_VALUE,discard_until=AV_NOPTS_VALUE;
static constexpr OMX_INDEXTYPE native_index=(OMX_INDEXTYPE)0x7f010001;
static constexpr OMX_INDEXTYPE metadata_index=(OMX_INDEXTYPE)0x7f010002;
static constexpr OMX_INDEXTYPE usage_index=(OMX_INDEXTYPE)0x7f010003;
int64_t eos_pts=0;
static int64_t clockNs() {timespec ts;clock_gettime(CLOCK_MONOTONIC,&ts);return ts.tv_sec*1000000000ll+ts.tv_nsec;}
int64_t stats_start=0, stats_lock=0, stats_convert=0, stats_unlock=0;
unsigned stats_frames=0,stats_gpu=0;
bool gpuCopy(AHardwareBuffer *hardware,const AHardwareBuffer_Planes &planes) {
if(planes.planes[1].pixelStride!=2 || planes.planes[2].pixelStride!=2 ||
planes.planes[1].rowStride!=planes.planes[2].rowStride ||
(uintptr_t)planes.planes[2].data!=(uintptr_t)planes.planes[1].data+1 ||
(uintptr_t)planes.planes[1].data<(uintptr_t)planes.planes[0].data)return false;
const auto *handle=AHardwareBuffer_getNativeHandle(hardware);
// CrOS gralloc includes duplicate plane fds and a metadata fd. The
// first fd holds the linear pixels; copy() bounds-checks both planes.
if(!handle || handle->numFds<1)return false;
int source=frameport_v4l2_export_frame(frame);if(source<0)return false;
uint64_t src_uv=(uintptr_t)frame->data[1]-(uintptr_t)frame->data[0];
uint64_t dst_uv=(uintptr_t)planes.planes[1].data-(uintptr_t)planes.planes[0].data;
std::vector<VkBufferCopy> regions;
auto append=[&](VkDeviceSize src,VkDeviceSize dst,VkDeviceSize bytes){
if(!regions.empty() && regions.back().srcOffset+regions.back().size==src &&
regions.back().dstOffset+regions.back().size==dst)regions.back().size+=bytes;
else regions.push_back({src,dst,bytes});
};
for(int row=0;row<frame->height;row++)
append((uint64_t)row*frame->linesize[0],(uint64_t)row*planes.planes[0].rowStride,frame->width);
for(int row=0;row<frame->height/2;row++)
append(src_uv+(uint64_t)row*frame->linesize[1],dst_uv+(uint64_t)row*planes.planes[1].rowStride,frame->width);
bool result=gpu.copy(source,handle->data[0],regions.data(),regions.size());close(source);return result;
}
void clearHistory() {
for(size_t i=0;i<history.size();i++){AVPacket *packet=history[i];av_packet_free(&packet);}
history.clear();history_bytes=0;replay_next=-1;
}
void rememberInput(const AVPacket *packet) {
if(!hardware_active)return;
if(packet->flags&AV_PKT_FLAG_KEY){
// Keep the preceding GOP too: the new keyframe can be submitted
// before older reordered pictures have finished coming out.
ssize_t previous=-1;
for(size_t i=0;i<history.size();i++)if(history[i]->flags&AV_PKT_FLAG_KEY)previous=i;
if(previous>0){
for(ssize_t i=0;i<previous;i++){
AVPacket *old=history[i];history_bytes-=old->size;av_packet_free(&old);
}
history.removeItemsAt(0,previous);
}
}
// References to compressed input, not copies of decoded panoramas.
// Bound memory for unusually long GOPs or malformed streams.
if(history_bytes+(size_t)packet->size>64*1024*1024 || history.size()>=512)clearHistory();
if(history.empty() && !(packet->flags&AV_PKT_FLAG_KEY))return;
AVPacket *copy=av_packet_clone(packet);
if(!copy)return;
if(history.add(copy)<0){av_packet_free(&copy);return;}
history_bytes+=copy->size;
}
bool annexB() const {
// H.264/HEVC parameter sets with start codes can go in-band. VP9's
// configuration is container metadata, never bitstream data.
return kind.coding!=OMX_VIDEO_CodingVP9 && config_size>=4 && !config[0] && !config[1] &&
(config[2]==1 || (!config[2] && config[3]==1));
}
void fail(int result,const char *operation) {
char message[128];av_strerror(result,message,sizeof(message));
ALOGE("%s failed: %s",operation,message);
failed=true;notify(OMX_EventError,OMX_ErrorHardware,0,nullptr);
}
static unsigned captureBuffers(const DecoderKind &kind,uint64_t pixels) {
// FFmpeg's general-purpose capture budget (20) suits ordinary sizes.
// Reference pictures shrink as pictures grow (HEVC level 6.x DPB
// limits; H.264 level 6.2 allows fewer), so 20 panorama-sized buffers
// would hold about 1 GB at 8K. VP9 can keep 8 references at any size.
// Unknown sizes keep the full budget.
const uint64_t max_luma=35651584;
unsigned references=pixels<=max_luma/4?16:pixels<=max_luma/2?12:pixels<=max_luma*3/4?8:6;
if(kind.coding==OMX_VIDEO_CodingVP9 && references<9)references=9;
// Room for the picture being decoded, one held for output and queueing.
return references+4<20?references+4:20;
}
int openCodec(const char *name,bool hardware) {
const AVCodec *implementation=avcodec_find_decoder_by_name(name);
if(!implementation || !frame)return AVERROR(ENOMEM);
codec=avcodec_alloc_context3(implementation);
if(!codec)return AVERROR(ENOMEM);
const auto &input=editPortInfo(kInputPortIndex)->mDef.format.video;
codec->width=codec->coded_width=input.nFrameWidth;
codec->height=codec->coded_height=input.nFrameHeight;
codec->pkt_timebase=AVRational{1,1000000};
codec->pix_fmt=AV_PIX_FMT_NV12;
if(config_size) {
codec->extradata=(uint8_t*)av_mallocz(config_size+AV_INPUT_BUFFER_PADDING_SIZE);
if(!codec->extradata)return AVERROR(ENOMEM);
memcpy(codec->extradata,config,config_size);codec->extradata_size=(int)config_size;
}
AVDictionary *options=nullptr;
if(hardware){
av_dict_set(&options,"num_output_buffers","2",0);
// Leave room for reference/reordered pictures: a four-buffer
// budget stalled a valid H.264 stream on Iris.
av_dict_set_int(&options,"num_capture_buffers",
captureBuffers(kind,(uint64_t)input.nFrameWidth*input.nFrameHeight),0);
}else{codec->thread_count=4;codec->thread_type=FF_THREAD_FRAME|FF_THREAD_SLICE;}
int result=avcodec_open2(codec,implementation,&options);av_dict_free(&options);
return result;
}
bool openDecoder() {
if(codec)return true;
const auto &size=editPortInfo(kInputPortIndex)->mDef.format.video;
int result=AVERROR(ENODEV);
if(size.nFrameWidth>kind.max_width || size.nFrameHeight>kind.max_height)
ALOGW("%s %ux%u is above what Iris decodes (%ux%u); using software decoding",kind.mime,
(unsigned)size.nFrameWidth,(unsigned)size.nFrameHeight,kind.max_width,kind.max_height);
else if(!prefer_software)for(int64_t start=clockNs(),attempt=0;;attempt++) {
result=openCodec(kind.hardware,true);
if(result>=0 || (result!=AVERROR(ENOMEM) && result!=AVERROR(EBUSY)))break;
// The driver refuses a session that doesn't fit next to the open
// ones (8K while any other is open). SteamVR's link holds one for
// about 12 s around every game start, when the plugin loads too:
// wait for it then (an 8K panorama in software is a slideshow),
// otherwise 2 s, before continuing in software.
int64_t now=clockNs();
if(now-start>=2000000000ll && now-plugin_loaded_ns>=20000000000ll)break;
if(!attempt)ALOGW("Iris %s busy (%ux%u); waiting for capacity",kind.mime,(unsigned)size.nFrameWidth,
(unsigned)size.nFrameHeight);
gpu.clearBuffers();avcodec_free_context(&codec);usleep(250000);
}
hardware_active=result>=0;
if(result<0){
// Availability can change after enumeration (another app/Steam
// opens Iris). Native MediaCodec clients do not all retry a stock
// component, so retain playback within this selected component.
if(prefer_software)ALOGI("%s: continuing in software for this session",kind.mime);
else ALOGW("Iris %s initialization failed (%d); using software decoding",kind.mime,result);
gpu.clearBuffers();avcodec_free_context(&codec);
result=openCodec(kind.software,false);
if(result<0){gpu.clearBuffers();avcodec_free_context(&codec);fail(result,"software fallback initialization");return false;}
}else ALOGI("Iris hardware %s decoder active: %dx%d",kind.mime,codec->coded_width,codec->coded_height);
prepend_config=hardware_active && annexB();
parser=av_parser_init(codec->codec_id);
if(parser)parser->flags|=PARSER_FLAG_COMPLETE_FRAMES;
return true;
}
bool recoverHardware(const char *operation) {
if(!hardware_active || history.empty() || !(history[0]->flags&AV_PKT_FLAG_KEY)){
fail(AVERROR_INVALIDDATA,operation);return false;
}
ALOGW("Iris %s failed during %s; replaying %zu compressed packets in software, last pts=%lld",kind.mime,operation,history.size(),(long long)last_output_pts);
av_frame_unref(frame);held=false;av_parser_close(parser);parser=nullptr;gpu.clearBuffers();avcodec_free_context(&codec);
hardware_active=false;prefer_software=true;
int result=openCodec(kind.software,false);
if(result<0){fail(result,"software recovery initialization");return false;}
replay_next=0;replay_eos=eos_submitted;eos_submitted=false;discard_until=last_output_pts;
return true;
}
void consumeInput(BufferInfo *info) {
OMX_BUFFERHEADERTYPE *header=info->mHeader;
getPortQueue(kInputPortIndex).erase(getPortQueue(kInputPortIndex).begin());
header->nOffset=0;header->nFilledLen=0;info->mOwnedByUs=false;
notifyEmptyBufferDone(header);
}
void reportBitstreamColor() {
// Like Android's stock decoders, pass signalled colour information
// to ACodec, which uses it for the surface dataspace. Container
// values still take precedence (getColorAspectPreference).
if(frame->color_primaries==AVCOL_PRI_UNSPECIFIED && frame->color_trc==AVCOL_TRC_UNSPECIFIED &&
frame->colorspace==AVCOL_SPC_UNSPECIFIED && frame->color_range==AVCOL_RANGE_UNSPECIFIED)return;
// FFmpeg's colour enums use the ISO/IEC 23091-2 code points.
ColorAspects aspects;
ColorUtils::convertIsoColorAspectsToCodecAspects(frame->color_primaries,frame->color_trc,
frame->colorspace,frame->color_range==AVCOL_RANGE_JPEG,aspects);
if(colorAspectsDiffer(aspects,mBitstreamColorAspects)){
ALOGI("%s bitstream colour: primaries %d, transfer %d, matrix %d, range %d",kind.mime,
frame->color_primaries,frame->color_trc,frame->colorspace,frame->color_range);
mBitstreamColorAspects=aspects;handleColorAspectsChange();
}
}
const YuvConstants *rgbaMatrix() {
ColorAspects aspects;
{Mutex::Autolock lock(mColorAspectsLock);aspects=mFinalColorAspects;}
// Unsignalled HD video is conventionally BT.709, as Android assumes.
bool bt709=aspects.mMatrixCoeffs==ColorAspects::MatrixBT709_5 ||
(aspects.mMatrixCoeffs==ColorAspects::MatrixUnspecified && (frame->width>=1280 || frame->height>=720));
if(aspects.mRange==ColorAspects::RangeFull)return bt709?&frameport_full709_rgba:&kYvuJPEGConstants;
return bt709?&kYvuH709Constants:&kYvuI601Constants;
}
bool outputFrame() {
// Hardware errors are recovered in software. Software decoders
// conceal damage themselves; output their picture as stock decoders do.
if(frame->decode_error_flags && hardware_active)return recoverHardware("decoded picture");
reportBitstreamColor();
bool reset=false;
handlePortSettingsChange(&reset,frame->width,frame->height,
native_enabled?(OMX_COLOR_FORMATTYPE)AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420:
surface?OMX_COLOR_Format32BitRGBA8888:OMX_COLOR_FormatYUV420Planar);
rgbaPort();
if(reset)return false;
auto &queue=getPortQueue(kOutputPortIndex);
if(queue.empty())return false;
BufferInfo *info=*queue.begin();OMX_BUFFERHEADERTYPE *header=info->mHeader;
size_t stride=outputBufferWidth(),height=outputBufferHeight();
size_t bytes=native_enabled?sizeof(VideoNativeMetadata):surface?stride*height*4:stride*height*3/2;
bool planar=frame->format==AV_PIX_FMT_YUV420P || frame->format==AV_PIX_FMT_YUVJ420P;
if(bytes>header->nAllocLen || (frame->format!=AV_PIX_FMT_NV12 && !planar) ||
frame->width<1 || frame->height<1 || (frame->width&1) || (frame->height&1) ||
(size_t)frame->width>stride || (size_t)frame->height>height) {
fail(AVERROR(EINVAL),"output buffer format");return false;
}
// Native surfaces keep YUV planes for GPU colour conversion. Legacy
// software surfaces use parallel NEON RGBA conversion, bypassing
// Lepton's slow scalar RGB565 renderer.
if(surface || native_enabled) {
int64_t before=clockNs(),locked=before,converted=before;
const YuvConstants *matrix=native_enabled?nullptr:rgbaMatrix();
uint8_t *destination=header->pBuffer;
AHardwareBuffer *hardware=nullptr;VideoNativeMetadata *native=nullptr;
AHardwareBuffer_Planes planes={};
bool used_gpu=false;
if(native_enabled) {
native=reinterpret_cast<VideoNativeMetadata*>(header->pBuffer);
if(!metadata || native->eType!=kMetadataBufferTypeANWBuffer || !native->pBuffer) {
fail(AVERROR(EINVAL),"native output metadata");return false;
}
hardware=ANativeWindowBuffer_getHardwareBuffer(native->pBuffer);
if(!hardware){fail(AVERROR(EINVAL),"native hardware buffer");return false;}
AHardwareBuffer_Desc desc={};AHardwareBuffer_describe(hardware,&desc);
if(desc.format!=AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420 || desc.width<(unsigned)frame->width ||
desc.height<(unsigned)frame->height) {
fail(AVERROR(EINVAL),"native output geometry");return false;
}
int result=AHardwareBuffer_lockPlanes(hardware,AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN,
native->nFenceFd,nullptr,&planes);
if(native->nFenceFd>=0)close(native->nFenceFd);
native->nFenceFd=-1;
if(result){fail(AVERROR(EIO),"native output lock");return false;}
if(planes.planeCount!=3 || planes.planes[0].pixelStride!=1 ||
planes.planes[0].rowStride<(unsigned)frame->width ||
!planes.planes[0].data || !planes.planes[1].data || !planes.planes[2].data ||
!planes.planes[1].pixelStride || !planes.planes[2].pixelStride ||
planes.planes[1].rowStride<(frame->width/2-1)*planes.planes[1].pixelStride+1 ||
planes.planes[2].rowStride<(frame->width/2-1)*planes.planes[2].pixelStride+1) {
AHardwareBuffer_unlock(hardware,&native->nFenceFd);
fail(AVERROR(EINVAL),"native YUV planes");return false;
}
locked=clockNs();
if(gpu_allowed && hardware_active && !planar && !(frame->width&3) &&
(uint64_t)frame->width*frame->height>=4096*2048 && gpu.available()) {
if(AHardwareBuffer_unlock(hardware,&native->nFenceFd)){
fail(AVERROR(EIO),"native layout unlock");return false;
}
if(native->nFenceFd>=0){
pollfd fence={native->nFenceFd,POLLIN,0};int waited;
do{waited=poll(&fence,1,10000);}while(waited<0 && errno==EINTR);
close(native->nFenceFd);native->nFenceFd=-1;
if(waited<=0 || !(fence.revents&POLLIN)){fail(AVERROR(EIO),"native layout fence");return false;}
}
used_gpu=gpuCopy(hardware,planes);
if(!used_gpu){
gpu_allowed=false;
if(gpu.failedAfterSubmit()){fail(AVERROR(EIO),"GPU video transfer");return false;}
if(AHardwareBuffer_lockPlanes(hardware,AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN,-1,nullptr,&planes)){
fail(AVERROR(EIO),"native CPU fallback lock");return false;
}
ALOGW("GPU video buffer sharing unavailable; retaining CPU copy");
}
}
}
if(!native_enabled)locked=clockNs();
if(native_enabled && !used_gpu)color.convert(frame,nullptr,0,nullptr,&planes);
else if(!native_enabled)color.convert(frame,destination,stride,matrix);
converted=clockNs();
if(hardware && !used_gpu && AHardwareBuffer_unlock(hardware,&native->nFenceFd)) {
fail(AVERROR(EIO),"native output unlock");return false;
}
int64_t after=clockNs();
stats_lock+=locked-before;stats_convert+=converted-locked;stats_unlock+=after-converted;
if(!stats_start)stats_start=before;
stats_frames++;
if(used_gpu)stats_gpu++;
if(after-stats_start>=5000000000ll) {
ALOGI("output %.1f fps, native=%d, gpu=%u/%u, lock %.2f ms, copy/convert %.2f ms, unlock %.2f ms, pts=%lld",
stats_frames*1e9/(after-stats_start),native_enabled,stats_gpu,stats_frames,
stats_lock/(1e6*stats_frames),stats_convert/(1e6*stats_frames),stats_unlock/(1e6*stats_frames),
(long long)frame->pts);
stats_start=after;stats_frames=stats_gpu=0;stats_lock=stats_convert=stats_unlock=0;
}
} else {
// Byte buffers hold planar YUV; the same worker bands copy the
// planes (in parallel for panorama sizes).
uint8_t *y=header->pBuffer,*u=y+stride*height,*v=u+stride*height/4;
AHardwareBuffer_Planes planes={};planes.planeCount=3;
planes.planes[0]={y,1,(uint32_t)stride};
planes.planes[1]={u,1,(uint32_t)(stride/2)};
planes.planes[2]={v,1,(uint32_t)(stride/2)};
color.convert(frame,nullptr,0,nullptr,&planes);
}
header->nOffset=0;header->nFilledLen=(OMX_U32)bytes;
header->nTimeStamp=frame->pts==AV_NOPTS_VALUE?frame->best_effort_timestamp:frame->pts;
last_output_pts=header->nTimeStamp;
header->nFlags=OMX_BUFFERFLAG_ENDOFFRAME;
queue.erase(queue.begin());info->mOwnedByUs=false;notifyFillBufferDone(header);
av_frame_unref(frame);held=false;return true;
}
protected:
void rgbaPort() {
if(native_enabled) {
auto &def=editPortInfo(kOutputPortIndex)->mDef;
def.format.video.eColorFormat=(OMX_COLOR_FORMATTYPE)AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420;
def.nBufferSize=sizeof(VideoNativeMetadata);return;
}
if(mOutputFormat!=OMX_COLOR_Format32BitRGBA8888)return;
auto &def=editPortInfo(kOutputPortIndex)->mDef;
def.format.video.eColorFormat=OMX_COLOR_Format32BitRGBA8888;
def.nBufferSize=outputBufferWidth()*outputBufferHeight()*4;
}
OMX_ERRORTYPE internalGetParameter(OMX_INDEXTYPE index, OMX_PTR params) override {
if(index==(OMX_INDEXTYPE)OMX_IndexParamVideoAndroidRequiresSwRenderer) {
auto *value=static_cast<OMX_PARAM_U32TYPE*>(params);
if(!value || value->nSize<sizeof(*value) || value->nPortIndex!=kOutputPortIndex)
return OMX_ErrorBadParameter;
// Decoding happens on Iris; Android's Surface renderer consumes
// our ordinary planar buffers rather than vendor ANW metadata.
surface=!native_enabled;value->nU32=native_enabled?0:1;return OMX_ErrorNone;
}
if(index==usage_index) {
auto *value=static_cast<GetAndroidNativeBufferUsageParams*>(params);
if(!value || value->nSize<sizeof(*value) || value->nPortIndex!=kOutputPortIndex)return OMX_ErrorBadParameter;
value->nUsage=AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN|AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE;
return OMX_ErrorNone;
}
if(index==OMX_IndexParamPortDefinition)rgbaPort();
OMX_ERRORTYPE result=SoftVideoDecoderOMXComponent::internalGetParameter(index,params);
if(result==OMX_ErrorNone && index==OMX_IndexParamVideoPortFormat) {
auto *value=static_cast<OMX_VIDEO_PARAM_PORTFORMATTYPE*>(params);
if(value->nPortIndex==kOutputPortIndex)value->eColorFormat=native_enabled?
(OMX_COLOR_FORMATTYPE)AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420:mOutputFormat;
}
return result;
}
OMX_ERRORTYPE getExtensionIndex(const char *name,OMX_INDEXTYPE *index) override {
if(!strcmp(name,"OMX.google.android.index.enableAndroidNativeBuffers"))*index=native_index;
else if(!strcmp(name,"OMX.google.android.index.storeANWBufferInMetadata"))*index=metadata_index;
else if(!strcmp(name,"OMX.google.android.index.getAndroidNativeBufferUsage"))*index=usage_index;
else return SoftVideoDecoderOMXComponent::getExtensionIndex(name,index);
return OMX_ErrorNone;
}
OMX_ERRORTYPE internalSetParameter(OMX_INDEXTYPE index,OMX_PTR params) override {
if(index==native_index || index==metadata_index) {
auto *value=static_cast<EnableAndroidNativeBuffersParams*>(params);
if(!value || value->nSize<sizeof(*value) || value->nPortIndex!=kOutputPortIndex)return OMX_ErrorBadParameter;
if(index==native_index)native_enabled=value->enable;
else metadata=value->enable;
rgbaPort();return OMX_ErrorNone;
}
if(index==OMX_IndexParamVideoPortFormat && native_enabled) {
auto *value=static_cast<OMX_VIDEO_PARAM_PORTFORMATTYPE*>(params);
if(value && value->nSize>=sizeof(*value) && value->nPortIndex==kOutputPortIndex) {
auto copy=*value;copy.eColorFormat=OMX_COLOR_FormatYUV420Planar;
return SoftVideoDecoderOMXComponent::internalSetParameter(index,&copy);
}
}
OMX_ERRORTYPE result=SoftVideoDecoderOMXComponent::internalSetParameter(index,params);
rgbaPort();return result;
}
int getColorAspectPreference() override {return kPreferContainer;}
void onQueueFilled(OMX_U32) override {
if(failed || mOutputPortSettingsChange!=NONE || drained)return;
auto &inputs=getPortQueue(kInputPortIndex);auto &outputs=getPortQueue(kOutputPortIndex);
while(!outputs.empty()) {
if(held){if(!outputFrame())return;continue;}
if(codec) {
int result=avcodec_receive_frame(codec,frame);
if(result==0){
int64_t pts=frame->pts==AV_NOPTS_VALUE?frame->best_effort_timestamp:frame->pts;
if(discard_until!=AV_NOPTS_VALUE && pts!=AV_NOPTS_VALUE && pts<=discard_until){av_frame_unref(frame);continue;}
discard_until=AV_NOPTS_VALUE;held=true;continue;
}
if(result==AVERROR_EOF) {
BufferInfo *info=*outputs.begin();auto *header=info->mHeader;
header->nOffset=0;header->nFilledLen=0;header->nTimeStamp=eos_pts;
header->nFlags=OMX_BUFFERFLAG_EOS;outputs.erase(outputs.begin());
info->mOwnedByUs=false;notifyFillBufferDone(header);drained=true;return;
}
if(result!=AVERROR(EAGAIN)){
if(hardware_active && recoverHardware("frame retrieval"))continue;
if(!failed)fail(result,"frame retrieval");return;
}
}
if(replay_next>=0){
if((size_t)replay_next<history.size()){
int result=avcodec_send_packet(codec,history[replay_next]);
if(result<0){fail(result,"software recovery packet");return;}
replay_next++;continue;
}
clearHistory();
if(replay_eos){
int result=avcodec_send_packet(codec,nullptr);
if(result<0){fail(result,"software recovery drain");return;}
replay_eos=false;eos_submitted=true;continue;
}
}
if(inputs.empty())return;
BufferInfo *info=*inputs.begin();auto *header=info->mHeader;
if(header->nOffset>header->nAllocLen || header->nFilledLen>header->nAllocLen-header->nOffset) {
fail(AVERROR(EINVAL),"input buffer bounds");return;
}
if(header->nFlags&OMX_BUFFERFLAG_CODECCONFIG) {
if(replace_config){av_freep(&config);config_size=0;replace_config=false;}
if(config_size+header->nFilledLen>1024*1024) {
fail(AVERROR(EINVAL),"oversized codec configuration");return;
}
if(header->nFilledLen) {
void *grown=av_realloc(config,config_size+header->nFilledLen);
if(!grown){fail(AVERROR(ENOMEM),"codec configuration storage");return;}
config=(uint8_t*)grown;
memcpy(config+config_size,header->pBuffer+header->nOffset,header->nFilledLen);
config_size+=header->nFilledLen;
}
// Streaming apps and restarted encoders can send parameter
// sets again mid-stream. Stock decoders accept them; submit
// them in-band before the next access unit.
if(codec)prepend_config=annexB();
consumeInput(info);continue;
}
if(!openDecoder())return;
if(header->nFilledLen) {
AVPacket *packet=av_packet_alloc();
if(!packet){fail(AVERROR(ENOMEM),"input packet");return;}
// Android delivers parameter sets in separate CODECCONFIG
// buffers. V4L2 expects Annex B parameter sets in its input
// stream; FFmpeg's V4L2 wrapper does not submit extradata.
size_t prefix=prepend_config?config_size:0;
int result=av_new_packet(packet,(int)(prefix+header->nFilledLen));
if(result>=0) {
if(prefix)memcpy(packet->data,config,prefix);
memcpy(packet->data+prefix,header->pBuffer+header->nOffset,header->nFilledLen);
packet->pts=packet->dts=header->nTimeStamp;
if(header->nFlags&OMX_BUFFERFLAG_SYNCFRAME)packet->flags|=AV_PKT_FLAG_KEY;
// ACodec does not preserve MediaCodec's KEY_FRAME flag
// on decoder input. Parse complete access units so driver
// recovery also works for ordinary Android clients.
if(parser){
uint8_t *parsed=nullptr;int parsed_size=0;
av_parser_parse2(parser,codec,&parsed,&parsed_size,packet->data,packet->size,
packet->pts,packet->dts,AV_NOPTS_VALUE);
if(parser->key_frame==1)packet->flags|=AV_PKT_FLAG_KEY;
}
result=avcodec_send_packet(codec,packet);
if(result>=0){prepend_config=false;rememberInput(packet);}
}
av_packet_free(&packet);
if(result==AVERROR(EAGAIN))return;
if(result<0){
if(hardware_active && recoverHardware("packet submission"))continue;
if(!failed)fail(result,"packet submission");return;
}
header->nFilledLen=0;
}
if(header->nFlags&OMX_BUFFERFLAG_EOS) {
eos_pts=header->nTimeStamp;
int result=avcodec_send_packet(codec,nullptr);
if(result==AVERROR(EAGAIN))return;
if(result<0){fail(result,"stream drain");return;}
eos_submitted=true;
}
replace_config=true; // a later configuration replaces this one
consumeInput(info);
}
}
void onPortFlushCompleted(OMX_U32 port) override {
if(port==kInputPortIndex) {
gpu.clearBuffers();
// Release the held picture first (its V4L2 buffer returns to the driver).
av_frame_unref(frame);av_parser_close(parser);parser=nullptr;clearHistory();
// Software decoders flush in place. A hardware session restarts in
// place only after a completed drain (replay/loop after EOS):
// restarting a running Iris session's queues failed its admission
// check and left buffers unreturned, so other seeks reopen it.
bool usable=codec && !failed && (!hardware_active || drained);
eos_submitted=false;replay_eos=false;
last_output_pts=discard_until=AV_NOPTS_VALUE;
held=false;drained=false;failed=false;replace_config=true;
if(usable) {
// Keeps the buffers and the Iris session (no reallocation).
avcodec_flush_buffers(codec);
int64_t restart_error=0;
if(hardware_active && (av_opt_get_int(codec->priv_data,"frameport_flush_error",0,&restart_error)<0 ||
restart_error<0)) {
ALOGW("Iris %s replay restart failed (%lld); continuing in software",kind.mime,(long long)restart_error);
gpu.clearBuffers();avcodec_free_context(&codec);hardware_active=false;prefer_software=true;
} else {
parser=av_parser_init(codec->codec_id);
if(parser)parser->flags|=PARSER_FLAG_COMPLETE_FRAMES;
prepend_config=annexB();
}
} else {
if(hardware_active && kind.coding==OMX_VIDEO_CodingVP9){
// Reopening Iris VP9 sessions after a seek damaged pictures and
// faulted the kernel on this Frame: continue in software.
prefer_software=true;
ALOGW("VP9 seek: continuing in software rather than reopening Iris");
}
gpu.clearBuffers();avcodec_free_context(&codec);hardware_active=false;
}
}
}
void onReset() override {
gpu_allowed=true;
av_frame_unref(frame);av_parser_close(parser);parser=nullptr;gpu.clearBuffers();avcodec_free_context(&codec);av_freep(&config);config_size=0;clearHistory();
hardware_active=false;prefer_software=false;eos_submitted=false;replay_eos=false;
last_output_pts=discard_until=AV_NOPTS_VALUE;
held=false;drained=false;failed=false;replace_config=false;
SoftVideoDecoderOMXComponent::onReset();
surface=false;native_enabled=false;metadata=false;mOutputFormat=OMX_COLOR_FormatYUV420Planar;
updatePortDefinitions(true,false);
}
~FramePortVideo() override {av_frame_free(&frame);av_parser_close(parser);gpu.clearBuffers();avcodec_free_context(&codec);av_freep(&config);clearHistory();}
public:
FramePortVideo(const DecoderKind &type,const OMX_CALLBACKTYPE *callbacks,OMX_PTR appData,OMX_COMPONENTTYPE **component)
:SoftVideoDecoderOMXComponent(type.name,type.role,type.coding,
type.profiles,type.profile_count,320,240,callbacks,appData,component),kind(type) {
initPorts(8,2*1024*1024,4,type.mime,1);
}
};
class FramePortOMXPlugin final : public OMXPluginBase {
bool available[kind_count]={};
static bool probeCapacity(const DecoderKind &kind) {
// Check each coded format separately, at ordinary video geometry.
// A rejected 8K session must not hide hardware from a 1080p app.
int fd=open("/dev/video-dec0",O_RDWR|O_NONBLOCK);
if(fd<0)return false;
bool supported=false;
for(unsigned i=0;;i++){
v4l2_fmtdesc desc={};desc.index=i;desc.type=V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
if(ioctl(fd,VIDIOC_ENUM_FMT,&desc)<0)break;
if(desc.pixelformat==kind.fourcc)supported=true;
}
if(!supported){close(fd);return false;}
v4l2_format format={};format.type=V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
format.fmt.pix_mp.width=1920;format.fmt.pix_mp.height=1080;
format.fmt.pix_mp.pixelformat=kind.fourcc;format.fmt.pix_mp.num_planes=1;
bool ok=ioctl(fd,VIDIOC_S_FMT,&format)==0 && format.fmt.pix_mp.pixelformat==kind.fourcc;
format.type=V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;format.fmt.pix_mp.pixelformat=V4L2_PIX_FMT_NV12;
if(ok)ok=ioctl(fd,VIDIOC_S_FMT,&format)==0 && format.fmt.pix_mp.pixelformat==V4L2_PIX_FMT_NV12;
v4l2_requestbuffers buffers={};buffers.count=2;
buffers.type=V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;buffers.memory=V4L2_MEMORY_MMAP;
if(ok)ok=ioctl(fd,VIDIOC_REQBUFS,&buffers)==0;
close(fd);return ok;
}
public:
FramePortOMXPlugin() {
timespec now;clock_gettime(CLOCK_MONOTONIC,&now);plugin_loaded_ns=now.tv_sec*1000000000ll+now.tv_nsec;
for(unsigned i=0;i<kind_count;i++){
available[i]=probeCapacity(kinds[i]);
if(!available[i])ALOGW("Iris %s unavailable; keeping Android's stock decoder",kinds[i].mime);
}
}
OMX_ERRORTYPE makeComponentInstance(const char *name,const OMX_CALLBACKTYPE *callbacks,
OMX_PTR appData,OMX_COMPONENTTYPE **component) override {
for(unsigned i=0;i<kind_count;i++)if(available[i] && !strcmp(name,kinds[i].name)){
auto *decoder=new FramePortVideo(kinds[i],callbacks,appData,component);
decoder->incStrong(this);return decoder->initCheck();
}
return OMX_ErrorInvalidComponentName;
}
OMX_ERRORTYPE destroyComponentInstance(OMX_COMPONENTTYPE *component) override {
auto *decoder=static_cast<SoftOMXComponent*>(component->pComponentPrivate);
decoder->prepareForDestruction();decoder->decStrong(this);return OMX_ErrorNone;
}
OMX_ERRORTYPE enumerateComponents(OMX_STRING name,size_t size,OMX_U32 index) override {
for(unsigned i=0;i<kind_count;i++)if(available[i]){
if(index){--index;continue;}
if(size<=strlen(kinds[i].name))return OMX_ErrorBadParameter;
strcpy(name,kinds[i].name);return OMX_ErrorNone;
}
return OMX_ErrorNoMore;
}
OMX_ERRORTYPE getRolesOfComponent(const char *name,Vector<String8> *roles) override {
for(unsigned i=0;i<kind_count;i++)if(available[i] && !strcmp(name,kinds[i].name)){
roles->clear();roles->push(String8(kinds[i].role));return OMX_ErrorNone;
}
return OMX_ErrorInvalidComponentName;
}
};
}
android::OMXPluginBase *createOMXPlugin(){return new android::FramePortOMXPlugin;}
// Android loaders support both historical factory ABI spellings.
extern "C" android::OMXPluginBase *createFramePortOMXPlugin() __asm__("createOMXPlugin");
extern "C" android::OMXPluginBase *createFramePortOMXPlugin(){return createOMXPlugin();}
extern "C" void destroyOMXPlugin(android::OMXPluginBase *plugin){delete plugin;}
+78
View File
@@ -0,0 +1,78 @@
#include <media/NdkMediaCodec.h>
#include <media/NdkMediaExtractor.h>
#include <media/NdkImageReader.h>
#include <android/native_window.h>
#include <android/hardware_buffer.h>
#include <cstdio>
#include <cassert>
#include <cstring>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>
#include <time.h>
static double now(){timespec t;clock_gettime(CLOCK_MONOTONIC,&t);return t.tv_sec+t.tv_nsec*1e-9;}
int main(int argc,char **argv){
bool fallback=argc>2 && !strcmp(argv[2],"fallback");
assert(argc>=2);int fd=open(argv[1],O_RDONLY);assert(fd>=0);struct stat st;assert(!fstat(fd,&st));
auto *extractor=AMediaExtractor_new();assert(AMediaExtractor_setDataSourceFd(extractor,fd,0,st.st_size)==AMEDIA_OK);
AMediaFormat *format=nullptr;int width=0,height=0;const char *video_mime=nullptr;
for(size_t i=0;i<AMediaExtractor_getTrackCount(extractor);i++){
auto *candidate=AMediaExtractor_getTrackFormat(extractor,i);const char *mime=nullptr;
if(AMediaFormat_getString(candidate,AMEDIAFORMAT_KEY_MIME,&mime)&&
(!strcmp(mime,"video/hevc") || !strcmp(mime,"video/avc") || !strcmp(mime,"video/x-vnd.on2.vp9"))){
format=candidate;video_mime=mime;assert(AMediaExtractor_selectTrack(extractor,i)==AMEDIA_OK);break;
}AMediaFormat_delete(candidate);
}assert(format);
assert(AMediaFormat_getInt32(format,AMEDIAFORMAT_KEY_WIDTH,&width));assert(AMediaFormat_getInt32(format,AMEDIAFORMAT_KEY_HEIGHT,&height));
const char *expected=!strcmp(video_mime,"video/hevc")?"OMX.frameport.hevc.decoder":
!strcmp(video_mime,"video/avc")?"OMX.frameport.avc.decoder":"OMX.frameport.vp9.decoder";
const char *stock=!strcmp(video_mime,"video/hevc")?"OMX.google.hevc.decoder":
!strcmp(video_mime,"video/avc")?"OMX.google.h264.decoder":"OMX.google.vp9.decoder";
auto *codec=AMediaCodec_createDecoderByType(video_mime);assert(codec);char *name=nullptr;
assert(AMediaCodec_getName(codec,&name)==AMEDIA_OK);printf("Selected codec: %s\n",name);fflush(stdout);
if(argc>2 && !strcmp(argv[2],"query")){
assert(!strcmp(name,expected) || !strcmp(name,stock));
AMediaCodec_releaseName(codec,name);AMediaCodec_delete(codec);
AMediaFormat_delete(format);AMediaExtractor_delete(extractor);close(fd);
puts("PASS default HEVC selection");return 0;
}
assert(!strcmp(name,fallback?stock:expected));AMediaCodec_releaseName(codec,name);
AImageReader *reader=nullptr;ANativeWindow *window=nullptr;
if(argc>2){assert(AImageReader_newWithUsage(width,height,AIMAGE_FORMAT_PRIVATE,AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE,3,&reader)==AMEDIA_OK);assert(AImageReader_getWindow(reader,&window)==AMEDIA_OK);AMediaFormat_setInt32(format,"allow-frame-drop",0);}
printf("Input format: %s\n",AMediaFormat_toString(format));
const char *keys[]={"csd-0","csd-1"};for(const char *key:keys){void *p=nullptr;size_t n=0;if(AMediaFormat_getBuffer(format,key,&p,&n)){auto *b=(unsigned char*)p;printf("%s: %zu bytes %02x %02x %02x %02x\n",key,n,b[0],b[1],b[2],b[3]);}}
assert(AMediaCodec_configure(codec,format,window,nullptr,0)==AMEDIA_OK);assert(AMediaCodec_start(codec)==AMEDIA_OK);
for(unsigned phase=0;phase<2;phase++){
unsigned goal=fallback?(phase?2:6):(phase?120:600),sent=0,received=0,images=0;bool input_eos=false,output_eos=false;double start=now();
while(!output_eos && now()-start<45){
if(!input_eos){ssize_t id=AMediaCodec_dequeueInputBuffer(codec,1000);if(id>=0){
size_t capacity=0;auto *data=AMediaCodec_getInputBuffer(codec,id,&capacity);assert(data);
if(sent==goal){assert(AMediaCodec_queueInputBuffer(codec,id,0,0,sent*1000000/60,AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM)==AMEDIA_OK);input_eos=true;}
else {ssize_t size=AMediaExtractor_readSampleData(extractor,data,capacity);assert(size>0);int64_t pts=AMediaExtractor_getSampleTime(extractor);
if(sent<2)printf("Input packet: %zd bytes %02x %02x %02x %02x\n",size,data[0],data[1],data[2],data[3]);
assert(AMediaCodec_queueInputBuffer(codec,id,0,size,pts,AMediaExtractor_getSampleFlags(extractor)&1?AMEDIACODEC_BUFFER_FLAG_KEY_FRAME:0)==AMEDIA_OK);
assert(AMediaExtractor_advance(extractor));sent++;
}
}}
AMediaCodecBufferInfo info={};ssize_t id=AMediaCodec_dequeueOutputBuffer(codec,&info,1000);
if(id>=0){if(received<2 || (info.flags&AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM))printf("Buffer id=%zd size=%d flags=%u pts=%lld\n",id,info.size,info.flags,(long long)info.presentationTimeUs);if(info.size)received++;output_eos=info.flags&AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM;
assert(AMediaCodec_releaseOutputBuffer(codec,id,window!=nullptr)==AMEDIA_OK);
// Surface delivery is asynchronous. Drain each rendered picture before
// submitting another, so this unpaced fixture does not test frame dropping.
if(reader && info.size){double deadline=now()+1;unsigned before=images;
do{AImage *image=nullptr;if(AImageReader_acquireNextImage(reader,&image)==AMEDIA_OK){
int32_t w=0,h=0;assert(AImage_getWidth(image,&w)==AMEDIA_OK && AImage_getHeight(image,&h)==AMEDIA_OK);
assert(w==width && h==height);images++;AImage_delete(image);
}else usleep(100);}while(images==before && now()<deadline);
assert(images>before);
}
}else if(id==AMEDIACODEC_INFO_OUTPUT_FORMAT_CHANGED){auto *out=AMediaCodec_getOutputFormat(codec);printf("Output: %s\n",AMediaFormat_toString(out));AMediaFormat_delete(out);fflush(stdout);}
if(reader){AImage *image=nullptr;while(AImageReader_acquireNextImage(reader,&image)==AMEDIA_OK){images++;AImage_delete(image);}}
}
printf("Phase %u: %u decoded frames, %u surface images, %.3f seconds\n",phase,received,images,now()-start);fflush(stdout);
assert(output_eos && received==goal);if(reader)assert(images>=goal-2);
if(!phase){assert(AMediaCodec_flush(codec)==AMEDIA_OK);assert(AMediaExtractor_seekTo(extractor,0,AMEDIAEXTRACTOR_SEEK_PREVIOUS_SYNC)==AMEDIA_OK);}
}
assert(AMediaCodec_stop(codec)==AMEDIA_OK);AMediaCodec_delete(codec);if(reader)AImageReader_delete(reader);
AMediaFormat_delete(format);AMediaExtractor_delete(extractor);close(fd);puts(fallback?"PASS stock fallback playback and seeking":"PASS MediaCodec default selection, full-resolution playback and seeking");
}
+32
View File
@@ -0,0 +1,32 @@
<?xml version="1.0" encoding="utf-8"?>
<MediaCodecs>
<Decoders>
<MediaCodec name="OMX.frameport.hevc.decoder" type="video/hevc" rank="64">
<Limit name="size" min="128x128" max="8192x8192" />
<Limit name="alignment" value="2x2" />
<Limit name="block-size" value="16x16" />
<Limit name="block-count" range="1-138240" />
<Limit name="blocks-per-second" range="1-7864320" />
<Limit name="bitrate" range="1-245000000" />
<Limit name="concurrent-instances" max="1" />
</MediaCodec>
<MediaCodec name="OMX.frameport.avc.decoder" type="video/avc" rank="64">
<Limit name="size" min="128x128" max="8192x8192" />
<Limit name="alignment" value="2x2" />
<Limit name="block-size" value="16x16" />
<Limit name="block-count" range="1-138240" />
<Limit name="blocks-per-second" range="1-7864320" />
<Limit name="bitrate" range="1-245000000" />
<Limit name="concurrent-instances" max="1" />
</MediaCodec>
<MediaCodec name="OMX.frameport.vp9.decoder" type="video/x-vnd.on2.vp9" rank="64">
<Limit name="size" min="128x128" max="4096x2304" />
<Limit name="alignment" value="2x2" />
<Limit name="block-size" value="16x16" />
<Limit name="block-count" range="1-36864" />
<Limit name="blocks-per-second" range="1-2211840" />
<Limit name="bitrate" range="1-245000000" />
<Limit name="concurrent-instances" max="1" />
</MediaCodec>
</Decoders>
</MediaCodecs>
+297
View File
@@ -0,0 +1,297 @@
// Exercises the actual Android OMX component, including port renegotiation,
// output conversion, timestamps and normal state/buffer lifetimes.
#include <media/hardware/OMXPluginBase.h>
#include <media/hardware/HardwareAPI.h>
#include <media/stagefright/foundation/ColorUtils.h>
#include <media/openmax/OMX_Component.h>
#include <media/openmax/OMX_IndexExt.h>
#include <utils/String8.h>
#include <pthread.h>
#include <dlfcn.h>
#include <cstdio>
#include <cstring>
#include <cassert>
#include <cstdlib>
#include <unistd.h>
#include <android/hardware_buffer.h>
#include <cutils/native_handle.h>
#include <sys/stat.h>
extern "C" const native_handle_t *AHardwareBuffer_getNativeHandle(const AHardwareBuffer*);
namespace android { ANativeWindowBuffer *AHardwareBuffer_to_ANativeWindowBuffer(AHardwareBuffer*); }
extern "C" {
#include <libavformat/avformat.h>
#include <libavcodec/bsf.h>
#include <libavutil/time.h>
}
struct Test {
pthread_mutex_t lock=PTHREAD_MUTEX_INITIALIZER;
pthread_cond_t changed=PTHREAD_COND_INITIALIZER;
OMX_COMPONENTTYPE *component=nullptr;
OMX_BUFFERHEADERTYPE *input_free[64],*output_free[64];
unsigned in_count=0,out_count=0,frames=0;
bool resize=false,eos=false,error=false;
unsigned state=OMX_StateLoaded,port_event=0;
OMX_COMMANDTYPE port_command=OMX_CommandStateSet;
int64_t last_pts=-1;
uint64_t first_hash=0;
uint64_t pictures[120]={};
bool replay=false;
bool native=false;
unsigned width=0,height=0;
AHardwareBuffer *hardware[16]={};
};
static OMX_ERRORTYPE event(OMX_HANDLETYPE,OMX_PTR data,OMX_EVENTTYPE type,OMX_U32 a,OMX_U32 b,OMX_PTR) {
auto &t=*(Test*)data;pthread_mutex_lock(&t.lock);
if(type==OMX_EventError){fprintf(stderr,"OMX error: %x %x\n",a,b);t.error=true;}
// As ACodec: only definition changes reallocate; crop/colour are updates.
if(type==OMX_EventPortSettingsChanged && a==1 && (b==0 || b==OMX_IndexParamPortDefinition))t.resize=true;
if(type==OMX_EventCmdComplete) {
if(a==OMX_CommandStateSet)t.state=b;
else {t.port_event=b==OMX_ALL?2:b+1;t.port_command=(OMX_COMMANDTYPE)a;}
}
pthread_cond_broadcast(&t.changed);pthread_mutex_unlock(&t.lock);return OMX_ErrorNone;
}
static OMX_ERRORTYPE empty(OMX_HANDLETYPE,OMX_PTR data,OMX_BUFFERHEADERTYPE *header) {
auto &t=*(Test*)data;pthread_mutex_lock(&t.lock);assert(t.in_count<64);
t.input_free[t.in_count++]=header;pthread_cond_broadcast(&t.changed);pthread_mutex_unlock(&t.lock);return OMX_ErrorNone;
}
static OMX_ERRORTYPE fill(OMX_HANDLETYPE,OMX_PTR data,OMX_BUFFERHEADERTYPE *header) {
auto &t=*(Test*)data;pthread_mutex_lock(&t.lock);assert(t.out_count<64);
if(header->nFilledLen) {
AHardwareBuffer_Planes planes={};AHardwareBuffer *hardware=nullptr;
if(t.native){
auto *meta=reinterpret_cast<android::VideoNativeMetadata*>(header->pBuffer);
for(auto *b:t.hardware)if(b && android::AHardwareBuffer_to_ANativeWindowBuffer(b)==meta->pBuffer)hardware=b;
assert(hardware);
assert(AHardwareBuffer_lockPlanes(hardware,AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN,meta->nFenceFd,nullptr,&planes)==0);
if(meta->nFenceFd>=0)close(meta->nFenceFd);meta->nFenceFd=-1;
}
auto byte=[&](unsigned i)->uint8_t{
if(!t.native)return header->pBuffer[i];
unsigned y=t.width*t.height,plane=i<y?0:i<y+y/4?1:2;
unsigned index=plane?i-y-(plane==2?y/4:0):i,width=plane?t.width/2:t.width;
return ((uint8_t*)planes.planes[plane].data)[(index/width)*planes.planes[plane].rowStride+
(index%width)*planes.planes[plane].pixelStride];
};
if(t.frames<120){
uint64_t hash=1469598103934665603ull;
unsigned pixel_bytes=t.native?t.width*t.height*3/2:header->nFilledLen;
unsigned step=pixel_bytes/1024;if(!step)step=1;
for(unsigned i=0;i<pixel_bytes;i+=step){hash^=byte(i);hash*=1099511628211ull;}
if(t.replay && t.pictures[t.frames]!=hash){
fprintf(stderr,"Picture %u changed after seek/recovery\n",t.frames);t.error=true;
}
t.pictures[t.frames]=hash;
}
if(!t.frames){
uint64_t hash=1469598103934665603ull;
unsigned pixel_bytes=t.native?t.width*t.height*3/2:header->nFilledLen;
for(unsigned i=0;i<pixel_bytes;i++){hash^=byte(i);hash*=1099511628211ull;}
printf("First-picture hash: %016llx\n",(unsigned long long)hash);fflush(stdout);
if(t.first_hash && t.first_hash!=hash){fprintf(stderr,"Seek did not reproduce the first picture\n");t.error=true;}
t.first_hash=hash;
}
if(header->nTimeStamp<t.last_pts){fprintf(stderr,"PTS went backwards\n");t.error=true;}
t.last_pts=header->nTimeStamp;t.frames++;
if(hardware)assert(AHardwareBuffer_unlock(hardware,nullptr)==0);
}
if(header->nFlags&OMX_BUFFERFLAG_EOS)t.eos=true;
t.output_free[t.out_count++]=header;
pthread_cond_broadcast(&t.changed);pthread_mutex_unlock(&t.lock);return OMX_ErrorNone;
}
template<class T>static T parameter(unsigned port) {T p={};p.nSize=sizeof(p);p.nVersion.s.nVersionMajor=1;p.nPortIndex=port;return p;}
static void wait_state(Test &t,unsigned state) {
pthread_mutex_lock(&t.lock);while(!t.error && t.state!=state)pthread_cond_wait(&t.changed,&t.lock);
assert(!t.error);pthread_mutex_unlock(&t.lock);
}
static void wait_port(Test &t,OMX_COMMANDTYPE command) {
pthread_mutex_lock(&t.lock);while(!t.error && !(t.port_event==2 && t.port_command==command))pthread_cond_wait(&t.changed,&t.lock);
assert(!t.error);t.port_event=0;pthread_mutex_unlock(&t.lock);
}
static void allocate(Test &t,unsigned port,OMX_BUFFERHEADERTYPE **headers,unsigned &count) {
auto def=parameter<OMX_PARAM_PORTDEFINITIONTYPE>(port);
assert(t.component->GetParameter(t.component,OMX_IndexParamPortDefinition,&def)==OMX_ErrorNone);
count=def.nBufferCountActual;assert(count<=16);
printf("Allocate port %u: %ux%u, %u buffers x %u bytes\n",port,def.format.video.nFrameWidth,def.format.video.nFrameHeight,count,def.nBufferSize);fflush(stdout);
for(unsigned i=0;i<count;i++)assert(t.component->AllocateBuffer(t.component,&headers[i],port,nullptr,def.nBufferSize)==OMX_ErrorNone);
if(port==1 && t.native){
t.width=def.format.video.nFrameWidth;t.height=def.format.video.nFrameHeight;
for(unsigned i=0;i<count;i++){
AHardwareBuffer_Desc d={};d.width=t.width;d.height=t.height;d.layers=1;d.format=AHARDWAREBUFFER_FORMAT_Y8Cb8Cr8_420;
d.usage=AHARDWAREBUFFER_USAGE_CPU_READ_OFTEN|AHARDWAREBUFFER_USAGE_CPU_WRITE_OFTEN|AHARDWAREBUFFER_USAGE_GPU_SAMPLED_IMAGE;
assert(AHardwareBuffer_allocate(&d,&t.hardware[i])==0);
if(!i){auto *handle=AHardwareBuffer_getNativeHandle(t.hardware[i]);
printf("Native buffer handles: %d\n",handle->numFds);
for(int fd=0;fd<handle->numFds;fd++){struct stat st={};assert(fstat(handle->data[fd],&st)==0);
printf(" fd %d: device=%llu inode=%llu bytes=%lld\n",fd,(unsigned long long)st.st_dev,
(unsigned long long)st.st_ino,(long long)st.st_size);}fflush(stdout);}
auto *meta=reinterpret_cast<android::VideoNativeMetadata*>(headers[i]->pBuffer);
meta->eType=android::kMetadataBufferTypeANWBuffer;meta->pBuffer=android::AHardwareBuffer_to_ANativeWindowBuffer(t.hardware[i]);
meta->nFenceFd=-1;
}
}
}
static void queue_output(Test &t) {
for(;;) {
pthread_mutex_lock(&t.lock);
if(!t.out_count || t.resize || t.eos){pthread_mutex_unlock(&t.lock);return;}
auto *header=t.output_free[--t.out_count];pthread_mutex_unlock(&t.lock);
header->nFilledLen=0;header->nFlags=0;
assert(t.component->FillThisBuffer(t.component,header)==OMX_ErrorNone);
}
}
int main(int argc,char **argv) {
alarm(60);
assert(argc==2 || argc==3);const char *path=getenv("FRAMEPORT_PROBE_CODEC");
void *lib=dlopen(path?path:"/probe/libstagefrighthw.so",RTLD_NOW);
if(!lib){fprintf(stderr,"%s\n",dlerror());return 1;}
auto factory=(android::OMXPluginBase*(*)())dlsym(lib,"createOMXPlugin");assert(factory);
auto *plugin=factory();Test t;OMX_CALLBACKTYPE callbacks={event,empty,fill};
t.native=argc==3 && (!strcmp(argv[2],"native") || !strcmp(argv[2],"native_cpu") || !strcmp(argv[2],"native_importfail"));
if(t.native && !strcmp(argv[2],"native_cpu")){
auto deny=(void(*)())dlsym(RTLD_DEFAULT,"frameport_probe_deny_gpu");assert(deny);deny();
}
if(t.native && !strcmp(argv[2],"native_importfail")){
auto deny=(void(*)())dlsym(RTLD_DEFAULT,"frameport_probe_deny_gpu_import");assert(deny);deny();
}
if(argc==3 && (!strcmp(argv[2],"software") || !strcmp(argv[2],"software_rgba"))){
auto deny=(void(*)())dlsym(RTLD_DEFAULT,"frameport_probe_deny_device");assert(deny);deny();
}
if(argc==3 && !strcmp(argv[2],"recover")){
auto corrupt=(void(*)())dlsym(RTLD_DEFAULT,"frameport_probe_corrupt_picture");assert(corrupt);corrupt();
}
if(argc==3 && !strcmp(argv[2],"restartfail")){
auto deny=(void(*)())dlsym(RTLD_DEFAULT,"frameport_probe_deny_restart");assert(deny);deny();
}
AVFormatContext *format=nullptr;assert(avformat_open_input(&format,argv[1],nullptr,nullptr)>=0);
unsigned video=0;while(video<format->nb_streams && format->streams[video]->codecpar->codec_type!=AVMEDIA_TYPE_VIDEO)video++;
assert(video<format->nb_streams);auto *stream=format->streams[video];
const char *name=nullptr,*filter=nullptr;
switch(stream->codecpar->codec_id){
case AV_CODEC_ID_HEVC:name="OMX.frameport.hevc.decoder";filter="hevc_mp4toannexb";break;
case AV_CODEC_ID_H264:name="OMX.frameport.avc.decoder";filter="h264_mp4toannexb";break;
case AV_CODEC_ID_VP9:name="OMX.frameport.vp9.decoder";break;
default:assert(false);
}
printf("Testing %s\n",name);fflush(stdout);
assert(plugin->makeComponentInstance(name,&callbacks,&t,&t.component)==OMX_ErrorNone);
if(t.native)for(const char *name:{"OMX.google.android.index.enableAndroidNativeBuffers","OMX.google.android.index.storeANWBufferInMetadata"}){
OMX_INDEXTYPE index;assert(t.component->GetExtensionIndex(t.component,(char*)name,&index)==OMX_ErrorNone);
auto mode=parameter<android::EnableAndroidNativeBuffersParams>(1);mode.enable=OMX_TRUE;
assert(t.component->SetParameter(t.component,index,&mode)==OMX_ErrorNone);
}
// ACodec supplies the container's colour metadata before decoding. Use the
// same contract so full-range fixtures exercise the intended conversion.
auto *par=stream->codecpar;
if(par->color_primaries!=AVCOL_PRI_UNSPECIFIED || par->color_trc!=AVCOL_TRC_UNSPECIFIED ||
par->color_space!=AVCOL_SPC_UNSPECIFIED || par->color_range!=AVCOL_RANGE_UNSPECIFIED) {
OMX_INDEXTYPE index;
char extension[]="OMX.google.android.index.describeColorAspects";
assert(t.component->GetExtensionIndex(t.component,extension,&index)==OMX_ErrorNone);
auto aspects=parameter<android::DescribeColorAspectsParams>(1);
android::ColorUtils::convertIsoColorAspectsToCodecAspects(par->color_primaries,par->color_trc,
par->color_space,par->color_range==AVCOL_RANGE_JPEG,aspects.sAspects);
assert(t.component->SetConfig(t.component,index,&aspects)==OMX_ErrorNone);
}
if(argc==3 && (!strcmp(argv[2],"rgba") || !strcmp(argv[2],"software_rgba"))) {
auto mode=parameter<OMX_PARAM_U32TYPE>(1);
assert(t.component->GetParameter(t.component,(OMX_INDEXTYPE)OMX_IndexParamVideoAndroidRequiresSwRenderer,&mode)==OMX_ErrorNone);
assert(mode.nU32==1);
}
AVBSFContext *bsf=nullptr;
if(filter){assert(av_bsf_alloc(av_bsf_get_by_name(filter),&bsf)>=0);
assert(avcodec_parameters_copy(bsf->par_in,stream->codecpar)>=0);bsf->time_base_in=stream->time_base;assert(av_bsf_init(bsf)>=0);}
auto input_def=parameter<OMX_PARAM_PORTDEFINITIONTYPE>(0);
assert(t.component->GetParameter(t.component,OMX_IndexParamPortDefinition,&input_def)==OMX_ErrorNone);
input_def.format.video.nFrameWidth=stream->codecpar->width;input_def.format.video.nFrameHeight=stream->codecpar->height;
assert(t.component->SetParameter(t.component,OMX_IndexParamPortDefinition,&input_def)==OMX_ErrorNone);
OMX_BUFFERHEADERTYPE *inputs[16],*outputs[16];unsigned input_count,output_count;
assert(t.component->SendCommand(t.component,OMX_CommandStateSet,OMX_StateIdle,nullptr)==OMX_ErrorNone);
allocate(t,0,inputs,input_count);allocate(t,1,outputs,output_count);wait_state(t,OMX_StateIdle);
for(unsigned i=0;i<input_count;i++)t.input_free[t.in_count++]=inputs[i];
for(unsigned i=0;i<output_count;i++)t.output_free[t.out_count++]=outputs[i];
assert(t.component->SendCommand(t.component,OMX_CommandStateSet,OMX_StateExecuting,nullptr)==OMX_ErrorNone);wait_state(t,OMX_StateExecuting);
AVPacket *packet=av_packet_alloc();
// midseek: seek mid-playback (no EOS first), as players usually do.
bool midseek=argc==3 && !strcmp(argv[2],"midseek");
for(unsigned phase=0;phase<2;phase++) {
bool sent_config=false,sent_eos=false,resent_config=false;unsigned submitted=0;
unsigned goal=phase?120:midseek?300:600;
int64_t start=av_gettime_relative();
for(;;) {
queue_output(t);
pthread_mutex_lock(&t.lock);
while(!t.error && !t.resize && !t.eos && (sent_eos || !t.in_count) && !t.out_count)pthread_cond_wait(&t.changed,&t.lock);
assert(!t.error);
bool resized=t.resize;t.resize=false;bool finished=t.eos;
pthread_mutex_unlock(&t.lock);
if(finished)break;
if(midseek && !phase && submitted>=goal) {
pthread_mutex_lock(&t.lock);bool enough=t.frames>=200;pthread_mutex_unlock(&t.lock);
if(enough)break;
usleep(1000);
}
if(resized) {
assert(t.component->SendCommand(t.component,OMX_CommandPortDisable,1,nullptr)==OMX_ErrorNone);
pthread_mutex_lock(&t.lock);while(t.out_count<output_count && !t.error)pthread_cond_wait(&t.changed,&t.lock);assert(!t.error);t.out_count=0;pthread_mutex_unlock(&t.lock);
for(unsigned i=0;i<output_count;i++)assert(t.component->FreeBuffer(t.component,1,outputs[i])==OMX_ErrorNone);
if(t.native)for(auto &b:t.hardware)if(b){AHardwareBuffer_release(b);b=nullptr;}
wait_port(t,OMX_CommandPortDisable);
assert(t.component->SendCommand(t.component,OMX_CommandPortEnable,1,nullptr)==OMX_ErrorNone);
allocate(t,1,outputs,output_count);wait_port(t,OMX_CommandPortEnable);
pthread_mutex_lock(&t.lock);for(unsigned i=0;i<output_count;i++)t.output_free[t.out_count++]=outputs[i];pthread_mutex_unlock(&t.lock);
continue;
}
if(sent_eos || (midseek && !phase && submitted>=goal))continue;
pthread_mutex_lock(&t.lock);
auto *header=t.in_count?t.input_free[--t.in_count]:nullptr;pthread_mutex_unlock(&t.lock);
if(!header)continue;
header->nOffset=0;header->nFilledLen=0;header->nTimeStamp=0;header->nFlags=0;
// midconfig: parameter sets sent again mid-stream without a flush.
bool midconfig=argc==3 && !strcmp(argv[2],"midconfig") && !phase && submitted==300 && !resent_config;
if(!sent_config || midconfig) {
const auto *params=bsf?bsf->par_out:stream->codecpar;
if(midconfig){resent_config=true;puts("Resending codec configuration mid-stream.");fflush(stdout);}
assert((unsigned)params->extradata_size<=header->nAllocLen);
if(params->extradata_size)memcpy(header->pBuffer,params->extradata,params->extradata_size);
header->nFilledLen=params->extradata_size;header->nFlags=OMX_BUFFERFLAG_CODECCONFIG;sent_config=true;
} else if(submitted>=goal) {
header->nFlags=OMX_BUFFERFLAG_EOS;header->nTimeStamp=goal*1000000/60;sent_eos=true;
} else {
int result;
do{result=av_read_frame(format,packet);assert(result>=0);if(packet->stream_index!=(int)video)av_packet_unref(packet);}while(packet->stream_index!=(int)video);
if(bsf){assert(av_bsf_send_packet(bsf,packet)>=0);assert(av_bsf_receive_packet(bsf,packet)>=0);}
av_packet_rescale_ts(packet,stream->time_base,AVRational{1,1000000});
assert((unsigned)packet->size<=header->nAllocLen);memcpy(header->pBuffer,packet->data,packet->size);header->nFilledLen=packet->size;
header->nTimeStamp=packet->pts;header->nFlags=OMX_BUFFERFLAG_ENDOFFRAME;
// Recovery runs model ACodec decoder input, which omits SYNCFRAME.
if((packet->flags&AV_PKT_FLAG_KEY) && !(argc==3 && !strcmp(argv[2],"recover")))
header->nFlags|=OMX_BUFFERFLAG_SYNCFRAME;
if(submitted<2){printf("Input phase %u: pts=%lld key=%d bytes=%d\n",phase,(long long)packet->pts,
!!(packet->flags&AV_PKT_FLAG_KEY),packet->size);fflush(stdout);}
av_packet_unref(packet);submitted++;
}
assert(t.component->EmptyThisBuffer(t.component,header)==OMX_ErrorNone);
}
double seconds=(av_gettime_relative()-start)/1000000.0;
printf("OMX decode + full-size delivery: %u frames in %.3fs = %.1ffps\n",t.frames,seconds,t.frames/seconds);fflush(stdout);
assert(midseek && !phase?t.frames>=200:t.frames==goal);
if(!phase) {
assert(t.component->SendCommand(t.component,OMX_CommandFlush,OMX_ALL,nullptr)==OMX_ErrorNone);
wait_port(t,OMX_CommandFlush);
assert(av_seek_frame(format,video,0,AVSEEK_FLAG_BACKWARD)>=0);if(bsf)av_bsf_flush(bsf);
pthread_mutex_lock(&t.lock);t.eos=false;t.frames=0;t.last_pts=-1;t.replay=true;pthread_mutex_unlock(&t.lock);
puts("Replaying after a complete OMX flush and seek to the beginning.");
}
}
assert(t.component->SendCommand(t.component,OMX_CommandStateSet,OMX_StateIdle,nullptr)==OMX_ErrorNone);wait_state(t,OMX_StateIdle);
assert(t.component->SendCommand(t.component,OMX_CommandStateSet,OMX_StateLoaded,nullptr)==OMX_ErrorNone);
for(unsigned i=0;i<input_count;i++)assert(t.component->FreeBuffer(t.component,0,inputs[i])==OMX_ErrorNone);
for(unsigned i=0;i<output_count;i++)assert(t.component->FreeBuffer(t.component,1,outputs[i])==OMX_ErrorNone);
if(t.native)for(auto &b:t.hardware)if(b){AHardwareBuffer_release(b);b=nullptr;}
wait_state(t,OMX_StateLoaded);assert(plugin->destroyComponentInstance(t.component)==OMX_ErrorNone);
delete plugin;dlclose(lib);av_packet_free(&packet);av_bsf_free(&bsf);avformat_close_input(&format);
if(t.native){auto count=(unsigned(*)())dlsym(RTLD_DEFAULT,"frameport_probe_gpu_submissions");assert(count);
printf("GPU submissions: %u\n",count());assert(!strcmp(argv[2],"native")?count()>0:count()==0);}
puts("PASS: OMX decode, ordered timestamps, EOS, seek and shutdown.");
}
+350
View File
@@ -0,0 +1,350 @@
/*
* Copyright (C) 2009 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#pragma once
/**
* @addtogroup Logging
* @{
*/
/**
* \file
*
* Support routines to send messages to the Android log buffer,
* which can later be accessed through the `logcat` utility.
*
* Each log message must have
* - a priority
* - a log tag
* - some text
*
* The tag normally corresponds to the component that emits the log message,
* and should be reasonably small.
*
* Log message text may be truncated to less than an implementation-specific
* limit (1023 bytes).
*
* Note that a newline character ("\n") will be appended automatically to your
* log message, if not already there. It is not possible to send several
* messages and have them appear on a single line in logcat.
*
* Please use logging in moderation:
*
* - Sending log messages eats CPU and slow down your application and the
* system.
*
* - The circular log buffer is pretty small, so sending many messages
* will hide other important log messages.
*
* - In release builds, only send log messages to account for exceptional
* conditions.
*/
#include <stdarg.h>
#include <stddef.h>
#include <stdint.h>
#include <sys/cdefs.h>
#if !defined(__BIONIC__) && !defined(__INTRODUCED_IN)
#define __INTRODUCED_IN(x)
#endif
#ifdef __cplusplus
extern "C" {
#endif
/**
* Android log priority values, in increasing order of priority.
*/
typedef enum android_LogPriority {
/** For internal use only. */
ANDROID_LOG_UNKNOWN = 0,
/** The default priority, for internal use only. */
ANDROID_LOG_DEFAULT, /* only for SetMinPriority() */
/** Verbose logging. Should typically be disabled for a release apk. */
ANDROID_LOG_VERBOSE,
/** Debug logging. Should typically be disabled for a release apk. */
ANDROID_LOG_DEBUG,
/** Informational logging. Should typically be disabled for a release apk. */
ANDROID_LOG_INFO,
/** Warning logging. For use with recoverable failures. */
ANDROID_LOG_WARN,
/** Error logging. For use with unrecoverable failures. */
ANDROID_LOG_ERROR,
/** Fatal logging. For use when aborting. */
ANDROID_LOG_FATAL,
/** For internal use only. */
ANDROID_LOG_SILENT, /* only for SetMinPriority(); must be last */
} android_LogPriority;
/**
* Writes the constant string `text` to the log, with priority `prio` and tag
* `tag`.
*/
int __android_log_write(int prio, const char* tag, const char* text);
/**
* Writes a formatted string to the log, with priority `prio` and tag `tag`.
* The details of formatting are the same as for
* [printf(3)](http://man7.org/linux/man-pages/man3/printf.3.html).
*/
int __android_log_print(int prio, const char* tag, const char* fmt, ...)
__attribute__((__format__(printf, 3, 4)));
/**
* Equivalent to `__android_log_print`, but taking a `va_list`.
* (If `__android_log_print` is like `printf`, this is like `vprintf`.)
*/
int __android_log_vprint(int prio, const char* tag, const char* fmt, va_list ap)
__attribute__((__format__(printf, 3, 0)));
/**
* Writes an assertion failure to the log (as `ANDROID_LOG_FATAL`) and to
* stderr, before calling
* [abort(3)](http://man7.org/linux/man-pages/man3/abort.3.html).
*
* If `fmt` is non-null, `cond` is unused. If `fmt` is null, the string
* `Assertion failed: %s` is used with `cond` as the string argument.
* If both `fmt` and `cond` are null, a default string is provided.
*
* Most callers should use
* [assert(3)](http://man7.org/linux/man-pages/man3/assert.3.html) from
* `&lt;assert.h&gt;` instead, or the `__assert` and `__assert2` functions
* provided by bionic if more control is needed. They support automatically
* including the source filename and line number more conveniently than this
* function.
*/
void __android_log_assert(const char* cond, const char* tag, const char* fmt, ...)
__attribute__((__noreturn__)) __attribute__((__format__(printf, 3, 4)));
/**
* Identifies a specific log buffer for __android_log_buf_write()
* and __android_log_buf_print().
*/
typedef enum log_id {
LOG_ID_MIN = 0,
/** The main log buffer. This is the only log buffer available to apps. */
LOG_ID_MAIN = 0,
/** The radio log buffer. */
LOG_ID_RADIO = 1,
/** The event log buffer. */
LOG_ID_EVENTS = 2,
/** The system log buffer. */
LOG_ID_SYSTEM = 3,
/** The crash log buffer. */
LOG_ID_CRASH = 4,
/** The statistics log buffer. */
LOG_ID_STATS = 5,
/** The security log buffer. */
LOG_ID_SECURITY = 6,
/** The kernel log buffer. */
LOG_ID_KERNEL = 7,
LOG_ID_MAX,
/** Let the logging function choose the best log target. */
LOG_ID_DEFAULT = 0x7FFFFFFF
} log_id_t;
/**
* Writes the constant string `text` to the log buffer `id`,
* with priority `prio` and tag `tag`.
*
* Apps should use __android_log_write() instead.
*/
int __android_log_buf_write(int bufID, int prio, const char* tag, const char* text);
/**
* Writes a formatted string to log buffer `id`,
* with priority `prio` and tag `tag`.
* The details of formatting are the same as for
* [printf(3)](http://man7.org/linux/man-pages/man3/printf.3.html).
*
* Apps should use __android_log_print() instead.
*/
int __android_log_buf_print(int bufID, int prio, const char* tag, const char* fmt, ...)
__attribute__((__format__(printf, 4, 5)));
/**
* Logger data struct used for writing log messages to liblog via __android_log_write_logger_data()
* and sending log messages to user defined loggers specified in __android_log_set_logger().
*/
struct __android_log_message {
size_t
struct_size; /** Must be set to sizeof(__android_log_message) and is used for versioning. */
int32_t buffer_id; /** {@link log_id_t} values. */
int32_t priority; /** {@link android_LogPriority} values. */
const char* tag; /** The tag for the log message. */
const char* file; /** Optional file name, may be set to nullptr. */
uint32_t line; /** Optional line number, ignore if file is nullptr. */
const char* message; /** The log message itself. */
};
/**
* Prototype for the 'logger' function that is called for every log message.
*/
typedef void (*__android_logger_function)(const struct __android_log_message* log_message);
/**
* Prototype for the 'abort' function that is called when liblog will abort due to
* __android_log_assert() failures.
*/
typedef void (*__android_aborter_function)(const char* abort_message);
#if !defined(__ANDROID__) || __ANDROID_API__ >= 30
/**
* Writes the log message specified by log_message. log_message includes additional file name and
* line number information that a logger may use. log_message is versioned for backwards
* compatibility.
* This assumes that loggability has already been checked through __android_log_is_loggable().
* Higher level logging libraries, such as libbase, first check loggability, then format their
* buffers, then pass the message to liblog via this function, and therefore we do not want to
* duplicate the loggability check here.
*
* @param log_message the log message itself, see {@link __android_log_message}.
*
* Available since API level 30.
*/
void __android_log_write_log_message(struct __android_log_message* log_message) __INTRODUCED_IN(30);
/**
* Sets a user defined logger function. All log messages sent to liblog will be set to the
* function pointer specified by logger for processing. It is not expected that log messages are
* already terminated with a new line. This function should add new lines if required for line
* separation.
*
* @param logger the new function that will handle log messages.
*
* Available since API level 30.
*/
void __android_log_set_logger(__android_logger_function logger) __INTRODUCED_IN(30);
/**
* Writes the log message to logd. This is an __android_logger_function and can be provided to
* __android_log_set_logger(). It is the default logger when running liblog on a device.
*
* @param log_message the log message to write, see {@link __android_log_message}.
*
* Available since API level 30.
*/
void __android_log_logd_logger(const struct __android_log_message* log_message) __INTRODUCED_IN(30);
/**
* Writes the log message to stderr. This is an __android_logger_function and can be provided to
* __android_log_set_logger(). It is the default logger when running liblog on host.
*
* @param log_message the log message to write, see {@link __android_log_message}.
*
* Available since API level 30.
*/
void __android_log_stderr_logger(const struct __android_log_message* log_message)
__INTRODUCED_IN(30);
/**
* Sets a user defined aborter function that is called for __android_log_assert() failures. This
* user defined aborter function is highly recommended to abort and be noreturn, but is not strictly
* required to.
*
* @param aborter the new aborter function, see {@link __android_aborter_function}.
*
* Available since API level 30.
*/
void __android_log_set_aborter(__android_aborter_function aborter) __INTRODUCED_IN(30);
/**
* Calls the stored aborter function. This allows for other logging libraries to use the same
* aborter function by calling this function in liblog.
*
* @param abort_message an additional message supplied when aborting, for example this is used to
* call android_set_abort_message() in __android_log_default_aborter().
*
* Available since API level 30.
*/
void __android_log_call_aborter(const char* abort_message) __INTRODUCED_IN(30);
/**
* Sets android_set_abort_message() on device then aborts(). This is the default aborter.
*
* @param abort_message an additional message supplied when aborting. This functions calls
* android_set_abort_message() with its contents.
*
* Available since API level 30.
*/
void __android_log_default_aborter(const char* abort_message) __attribute__((noreturn))
__INTRODUCED_IN(30);
/**
* Use the per-tag properties "log.tag.<tagname>" along with the minimum priority from
* __android_log_set_minimum_priority() to determine if a log message with a given prio and tag will
* be printed. A non-zero result indicates yes, zero indicates false.
*
* If both a priority for a tag and a minimum priority are set by
* __android_log_set_minimum_priority(), then the lowest of the two values are to determine the
* minimum priority needed to log. If only one is set, then that value is used to determine the
* minimum priority needed. If none are set, then default_priority is used.
*
* @param prio the priority to test, takes {@link android_LogPriority} values.
* @param tag the tag to test.
* @param len the length of the tag.
* @param default_prio the default priority to use if no properties or minimum priority are set.
* @return an integer where 1 indicates that the message is loggable and 0 indicates that it is not.
*
* Available since API level 30.
*/
int __android_log_is_loggable(int prio, const char* tag, int default_prio) __INTRODUCED_IN(30);
int __android_log_is_loggable_len(int prio, const char* tag, size_t len, int default_prio)
__INTRODUCED_IN(30);
/**
* Sets the minimum priority that will be logged for this process.
*
* @param priority the new minimum priority to set, takes @{link android_LogPriority} values.
* @return the previous set minimum priority as @{link android_LogPriority} values, or
* ANDROID_LOG_DEFAULT if none was set.
*
* Available since API level 30.
*/
int32_t __android_log_set_minimum_priority(int32_t priority) __INTRODUCED_IN(30);
/**
* Gets the minimum priority that will be logged for this process. If none has been set by a
* previous __android_log_set_minimum_priority() call, this returns ANDROID_LOG_DEFAULT.
*
* @return the current minimum priority as @{link android_LogPriority} values, or
* ANDROID_LOG_DEFAULT if none is set.
*
* Available since API level 30.
*/
int32_t __android_log_get_minimum_priority(void) __INTRODUCED_IN(30);
/**
* Sets the default tag if no tag is provided when writing a log message. Defaults to
* getprogname(). This truncates tag to the maximum log message size, though appropriate tags
* should be much smaller.
*
* @param tag the new log tag.
*
* Available since API level 30.
*/
void __android_log_set_default_tag(const char* tag) __INTRODUCED_IN(30);
#endif
#ifdef __cplusplus
}
#endif
/** @} */
Loaded 100 of 345 files, more files were not shown because too many files have changed in this diff. Show more