mirror of
https://github.com/saphid/frame-control.git
synced 2026-10-06 04:04:21 +02:00
Compare commits
73
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
48158d1fe9 | ||
|
|
66b6d46e0e | ||
|
|
d970107716 | ||
|
|
6fbc450eea | ||
|
|
f7573e3565 | ||
|
|
4f6d40625a | ||
|
|
c6ed6c9ea1 | ||
|
|
6d03317970 | ||
|
|
976008065f | ||
|
|
8b5ada1272 | ||
|
|
48a9914124 | ||
|
|
002c859572 | ||
|
|
b5cf8253e6 | ||
|
|
beccfec307 | ||
|
|
a6137351d9 | ||
|
|
6a8e3fadbf | ||
|
|
643cb65c79 | ||
|
|
1b90c64b73 | ||
|
|
39afb23d97 | ||
|
|
bda9d77d2d | ||
|
|
4fae62ba14 | ||
|
|
1d05f57fbf | ||
|
|
484a50a189 | ||
|
|
7f769ca2f0 | ||
|
|
71200c5179 | ||
|
|
00b45bafc5 | ||
|
|
9e4dcdd147 | ||
|
|
33a92a2e1d | ||
|
|
f076527722 | ||
|
|
e67802f15d | ||
|
|
73eef14ecd | ||
|
|
c3ceea9bcd | ||
|
|
dcf9689f64 | ||
|
|
97d70d0c80 | ||
|
|
9eeca79b5d | ||
|
|
224340edc9 | ||
|
|
b393e90854 | ||
|
|
45f720883a | ||
|
|
c814cb95d0 | ||
|
|
ff2c4ebfe0 | ||
|
|
03322d3166 | ||
|
|
2d08486278 | ||
|
|
f780ab2c6a | ||
|
|
396f2a830a | ||
|
|
59761ae015 | ||
|
|
48eedbc2eb | ||
|
|
2b89eeba1d | ||
|
|
75e92db2fa | ||
|
|
10f96656e3 | ||
|
|
a1fa4ce140 | ||
|
|
a7663b3a5e | ||
|
|
a6fff434a4 | ||
|
|
cd40a194ed | ||
|
|
0037b1ef4b | ||
|
|
50405ccf88 | ||
|
|
32196b4260 | ||
|
|
7d328e20a5 | ||
|
|
fbe7ba9575 | ||
|
|
a3c6e5c984 | ||
|
|
5e12245932 | ||
|
|
df1810bae3 | ||
|
|
3f0e7b198a | ||
|
|
9110557e23 | ||
|
|
0016d9200c | ||
|
|
fe87a9d826 | ||
|
|
1b26c4f92c | ||
|
|
3db311da13 | ||
|
|
c711e136d4 | ||
|
|
13f629af35 | ||
|
|
7efce19f4c | ||
|
|
623ce683d4 | ||
|
|
e5f3c96275 | ||
|
|
0cb289d571 |
No files matched your search
@@ -48,3 +48,9 @@ running `--help`: `paste-to-frame.sh`, `serve-bootstrap.sh` and
|
||||
- `sudo` on the Frame asks for the user's Developer Mode password. Hand those
|
||||
steps to the user (open Terminal) and keep automation to non-sudo commands.
|
||||
- The Mac uses BSD userland and zsh (no `timeout`, use `head -n`).
|
||||
- **First-party first.** For any new capability, investigate the first-party
|
||||
way before anything else: Valve (SteamOS, Steam, Steam Link), Apple (the Mac
|
||||
and iPhone), and KDE (the Frame's desktop is Plasma). It's usually the best
|
||||
answer. If it isn't, write down why not. If it is, find what Frame Control
|
||||
can do to make it easier to set up (install over SSH, pre-seed settings,
|
||||
pair automatically, tell the user the one setting to turn on).
|
||||
@@ -0,0 +1,9 @@
|
||||
# GitHub handles approved to bypass contribution auto-close
|
||||
# Format: <username> <capability>
|
||||
# capability:
|
||||
# issue future issues stay open
|
||||
# pr future issues and PRs stay open
|
||||
# Maintainers add people by replying `lgtmi` or `lgtm` on an issue
|
||||
# (.github/workflows/approve-contributor.yml); editing this file by hand works too.
|
||||
|
||||
fbl100 pr
|
||||
@@ -0,0 +1 @@
|
||||
ko_fi: alexsouthwell
|
||||
@@ -0,0 +1,51 @@
|
||||
name: Bug report
|
||||
description: Report something that's broken
|
||||
labels: ["bug"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**Before you start:** read [CONTRIBUTING.md](https://github.com/saphid/frame-control/blob/main/CONTRIBUTING.md).
|
||||
|
||||
Issues from new contributors are auto-closed by default. A maintainer reviews them and reopens worthwhile ones. The [website feedback form](https://frame-control.pages.dev/feedback/) skips that queue.
|
||||
|
||||
Keep this short. If it doesn't fit on one screen, it's too long. Write in your own voice.
|
||||
|
||||
- type: textarea
|
||||
id: description
|
||||
attributes:
|
||||
label: What happened?
|
||||
description: Be specific. Include error messages and the last lines of the server log (Frame → Show Server Log).
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: repro
|
||||
attributes:
|
||||
label: Steps to reproduce
|
||||
description: Minimal steps to trigger the bug.
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: textarea
|
||||
id: expected
|
||||
attributes:
|
||||
label: Expected behavior
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: input
|
||||
id: version
|
||||
attributes:
|
||||
label: Frame Control version
|
||||
description: e.g. v0.3.1
|
||||
validations:
|
||||
required: false
|
||||
|
||||
- type: input
|
||||
id: os
|
||||
attributes:
|
||||
label: Computer and SteamOS build
|
||||
description: e.g. Windows 11, SteamOS 20260922.6101926 (Steam Settings → System)
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,5 @@
|
||||
blank_issues_enabled: false
|
||||
contact_links:
|
||||
- name: Feedback form (no GitHub account needed, skips the queue)
|
||||
url: https://frame-control.pages.dev/feedback/
|
||||
about: Bugs, ideas and questions from the website become issues here without being auto-closed.
|
||||
@@ -0,0 +1,36 @@
|
||||
name: Idea or contribution proposal
|
||||
description: Propose a change or feature (required for new contributors before opening a PR)
|
||||
labels: ["enhancement"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
**Before you start:** read [CONTRIBUTING.md](https://github.com/saphid/frame-control/blob/main/CONTRIBUTING.md).
|
||||
|
||||
Issues from new contributors are auto-closed by default. A maintainer reviews them and reopens worthwhile ones.
|
||||
|
||||
Keep this short. If it doesn't fit on one screen, it's too long. Write in your own voice.
|
||||
|
||||
- type: textarea
|
||||
id: what
|
||||
attributes:
|
||||
label: What do you want to change?
|
||||
description: Be specific and concise.
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: why
|
||||
attributes:
|
||||
label: Why?
|
||||
description: What problem does this solve?
|
||||
validations:
|
||||
required: true
|
||||
|
||||
- type: textarea
|
||||
id: how
|
||||
attributes:
|
||||
label: How? (optional)
|
||||
description: Brief technical approach, and whether you'd like to implement it yourself.
|
||||
validations:
|
||||
required: false
|
||||
@@ -0,0 +1,238 @@
|
||||
# Contributor gate adapted from badlogic/pi-mono (MIT) at 6f7551516b84.
|
||||
# See CONTRIBUTING.md for how it works.
|
||||
|
||||
name: Approve Contributor
|
||||
|
||||
on:
|
||||
issue_comment:
|
||||
types: [created]
|
||||
|
||||
jobs:
|
||||
approve:
|
||||
# Only maintainers' comments that might approve someone join the queue, and
|
||||
# they run one at a time so two lgtm replies can't race on APPROVED_CONTRIBUTORS.
|
||||
# (The script below still checks for write access.)
|
||||
if: >-
|
||||
contains(github.event.comment.body, 'lgtm') &&
|
||||
contains(fromJSON('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)
|
||||
concurrency:
|
||||
group: approve-contributor
|
||||
cancel-in-progress: false
|
||||
queue: max
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: write
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
||||
with:
|
||||
ref: ${{ github.event.repository.default_branch }}
|
||||
|
||||
- name: Update contributor approval
|
||||
id: update
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
script: |
|
||||
const fs = require('fs');
|
||||
|
||||
const APPROVED_FILE = '.github/APPROVED_CONTRIBUTORS';
|
||||
const VALID_CAPABILITIES = new Set(['issue', 'pr']);
|
||||
const issueAuthor = context.payload.issue.user.login;
|
||||
const commenter = context.payload.comment.user.login;
|
||||
const commentBody = (context.payload.comment.body || '').trim();
|
||||
|
||||
const approvalAtStartPattern = /^[\s.]*(?:@[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?(?:\s*,\s*|[.:]\s*|\s+))*(lgtmi|lgtm)(?=$|[\s]|[^\p{L}\p{N}_\s])/iu;
|
||||
const approvalAtEndPattern = /(?:^|[\s.])(lgtmi|lgtm)\s*(?:[^\p{L}\p{N}_\s])?\s*$/iu;
|
||||
const approvalMatch = commentBody.match(approvalAtStartPattern) ?? commentBody.match(approvalAtEndPattern);
|
||||
|
||||
if (!approvalMatch) {
|
||||
console.log('Comment does not start or end with lgtm or lgtmi');
|
||||
core.setOutput('status', 'skipped');
|
||||
return;
|
||||
}
|
||||
|
||||
const targetCapability = approvalMatch[1].toLowerCase() === 'lgtmi' ? 'issue' : 'pr';
|
||||
|
||||
try {
|
||||
const { data: permissionLevel } = await github.rest.repos.getCollaboratorPermissionLevel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
username: commenter,
|
||||
});
|
||||
|
||||
if (!['admin', 'maintain', 'write'].includes(permissionLevel.permission)) {
|
||||
console.log(`${commenter} does not have write access`);
|
||||
core.setOutput('status', 'skipped');
|
||||
return;
|
||||
}
|
||||
} catch {
|
||||
console.log(`${commenter} does not have collaborator access`);
|
||||
core.setOutput('status', 'skipped');
|
||||
return;
|
||||
}
|
||||
|
||||
function parseMentionedUsers(body) {
|
||||
const users = [];
|
||||
const seenUsers = new Set();
|
||||
const mentionPattern = /(^|[^A-Za-z0-9_])@([A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?)(?![A-Za-z0-9-]|\/)/g;
|
||||
|
||||
for (const match of body.matchAll(mentionPattern)) {
|
||||
const username = match[2];
|
||||
const normalizedUser = username.toLowerCase();
|
||||
if (seenUsers.has(normalizedUser)) {
|
||||
continue;
|
||||
}
|
||||
seenUsers.add(normalizedUser);
|
||||
users.push(username);
|
||||
}
|
||||
|
||||
return users;
|
||||
}
|
||||
|
||||
function parseApprovedUsers(content) {
|
||||
const lines = content.split('\n');
|
||||
const entries = [];
|
||||
const users = new Map();
|
||||
|
||||
for (const line of lines) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed || trimmed.startsWith('#')) {
|
||||
entries.push({ type: 'other', line });
|
||||
continue;
|
||||
}
|
||||
|
||||
const parts = trimmed.split(/\s+/);
|
||||
if (parts.length !== 2) {
|
||||
console.log(`Skipping malformed line: ${line}`);
|
||||
entries.push({ type: 'other', line });
|
||||
continue;
|
||||
}
|
||||
|
||||
const [username, capability] = parts;
|
||||
const normalizedCapability = capability.toLowerCase();
|
||||
if (!VALID_CAPABILITIES.has(normalizedCapability)) {
|
||||
console.log(`Skipping line with invalid capability: ${line}`);
|
||||
entries.push({ type: 'other', line });
|
||||
continue;
|
||||
}
|
||||
|
||||
const normalizedUser = username.toLowerCase();
|
||||
const entry = { type: 'user', username, normalizedUser, capability: normalizedCapability };
|
||||
entries.push(entry);
|
||||
users.set(normalizedUser, entry);
|
||||
}
|
||||
|
||||
return { entries, users };
|
||||
}
|
||||
|
||||
function stringifyApprovedUsers(entries) {
|
||||
const normalizedEntries = [...entries];
|
||||
|
||||
while (normalizedEntries.length > 0) {
|
||||
const lastEntry = normalizedEntries[normalizedEntries.length - 1];
|
||||
if (lastEntry.type !== 'other' || lastEntry.line.trim() !== '') {
|
||||
break;
|
||||
}
|
||||
normalizedEntries.pop();
|
||||
}
|
||||
|
||||
return `${normalizedEntries
|
||||
.map((entry) => (entry.type === 'user' ? `${entry.username} ${entry.capability}` : entry.line))
|
||||
.join('\n')}\n`;
|
||||
}
|
||||
|
||||
const content = fs.readFileSync(APPROVED_FILE, 'utf8');
|
||||
const { entries, users } = parseApprovedUsers(content);
|
||||
const mentionedUsers = parseMentionedUsers(commentBody);
|
||||
const approvalTargets = mentionedUsers.length > 0 ? mentionedUsers : [issueAuthor];
|
||||
const changedTargets = [];
|
||||
const alreadyTargets = [];
|
||||
|
||||
for (const username of approvalTargets) {
|
||||
const normalizedUser = username.toLowerCase();
|
||||
const existingEntry = users.get(normalizedUser);
|
||||
const existingCapability = existingEntry?.capability ?? null;
|
||||
|
||||
if (existingCapability === 'pr' || existingCapability === targetCapability) {
|
||||
alreadyTargets.push(existingEntry?.username ?? username);
|
||||
console.log(`${username} is already approved for ${existingCapability}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (existingEntry) {
|
||||
existingEntry.capability = targetCapability;
|
||||
changedTargets.push(existingEntry.username);
|
||||
} else {
|
||||
const entry = { type: 'user', username, normalizedUser, capability: targetCapability };
|
||||
entries.push(entry);
|
||||
users.set(normalizedUser, entry);
|
||||
changedTargets.push(username);
|
||||
}
|
||||
|
||||
console.log(`Set ${username} capability to ${targetCapability}`);
|
||||
}
|
||||
|
||||
core.setOutput('capability', targetCapability);
|
||||
core.setOutput('changed_targets', JSON.stringify(changedTargets));
|
||||
core.setOutput('already_targets', JSON.stringify(alreadyTargets));
|
||||
|
||||
if (changedTargets.length === 0) {
|
||||
core.setOutput('status', 'already');
|
||||
return;
|
||||
}
|
||||
|
||||
fs.writeFileSync(APPROVED_FILE, stringifyApprovedUsers(entries));
|
||||
core.setOutput('status', 'changed');
|
||||
|
||||
- name: Commit and push
|
||||
if: steps.update.outputs.status == 'changed'
|
||||
run: |
|
||||
git config user.name "github-actions[bot]"
|
||||
git config user.email "github-actions[bot]@users.noreply.github.com"
|
||||
git add .github/APPROVED_CONTRIBUTORS
|
||||
git diff --staged --quiet || git commit -m "chore: approve contributors from issue #${{ github.event.issue.number }}"
|
||||
# main may have moved since checkout; replay the approval on top of it.
|
||||
git pull --rebase origin "${{ github.event.repository.default_branch }}"
|
||||
git push
|
||||
|
||||
- name: Comment on issue
|
||||
if: steps.update.outputs.status == 'changed' || steps.update.outputs.status == 'already'
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
env:
|
||||
CAPABILITY: ${{ steps.update.outputs.capability }}
|
||||
CHANGED_TARGETS: ${{ steps.update.outputs.changed_targets }}
|
||||
ALREADY_TARGETS: ${{ steps.update.outputs.already_targets }}
|
||||
with:
|
||||
script: |
|
||||
const capability = process.env.CAPABILITY;
|
||||
const changedTargets = JSON.parse(process.env.CHANGED_TARGETS || '[]');
|
||||
const alreadyTargets = JSON.parse(process.env.ALREADY_TARGETS || '[]');
|
||||
const defaultBranch = context.payload.repository.default_branch;
|
||||
const formatTargets = (targets) => targets.map((target) => `@${target}`).join(', ');
|
||||
const bodyLines = [];
|
||||
|
||||
if (changedTargets.length > 0) {
|
||||
if (capability === 'issue') {
|
||||
bodyLines.push(`${formatTargets(changedTargets)} approved for issues. Future issues will not be auto-closed. PRs still require \`lgtm\` at the start of a maintainer reply (optionally after one or more \`@username\` mentions) or at the end.`);
|
||||
} else {
|
||||
bodyLines.push(`${formatTargets(changedTargets)} approved for issues and PRs. Future issues and PRs will not be auto-closed.`);
|
||||
}
|
||||
}
|
||||
|
||||
if (alreadyTargets.length > 0) {
|
||||
const verb = alreadyTargets.length === 1 ? 'is' : 'are';
|
||||
bodyLines.push(`${formatTargets(alreadyTargets)} ${verb} already approved.`);
|
||||
}
|
||||
|
||||
bodyLines.push('', `See [CONTRIBUTING.md](https://github.com/${context.repo.owner}/${context.repo.repo}/blob/${defaultBranch}/CONTRIBUTING.md).`);
|
||||
const body = bodyLines.join('\n');
|
||||
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
body,
|
||||
});
|
||||
|
||||
@@ -35,7 +35,11 @@ jobs:
|
||||
- name: Server tests
|
||||
run: python -m unittest discover -s tests -v
|
||||
- name: App syntax
|
||||
run: node --check app/main.js && node --check app/build/make-icon.js && node --check app/build/fetch-deps.js && node --check app/preload.js && node --check app/install-link.js
|
||||
run: node --check app/main.js && node --check app/build/make-icon.js && node --check app/build/fetch-deps.js && node --check app/preload.js && node --check app/install-link.js && node --check app/updater.js
|
||||
- name: Updater tests
|
||||
run: node --test app/test/updater.test.js
|
||||
- name: Website
|
||||
run: node --test site/test/*.test.mjs && node --check site/public/js/site.js && node --check site/public/js/feedback.js
|
||||
|
||||
# The server runs on each desktop OS the app ships for, on the Python version
|
||||
# the app bundles (app/build/fetch-deps.js) and, on Ubuntu, a newer one.
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
# Contributor gate adapted from badlogic/pi-mono (MIT) at 6f7551516b84.
|
||||
# See CONTRIBUTING.md for how it works.
|
||||
|
||||
name: Issue Gate
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [opened]
|
||||
|
||||
jobs:
|
||||
check-contributor:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
steps:
|
||||
- name: Check issue author
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
script: |
|
||||
const APPROVED_FILE = '.github/APPROVED_CONTRIBUTORS';
|
||||
const VALID_CAPABILITIES = new Set(['issue', 'pr']);
|
||||
const TRUSTED_BOT_AUTHORS = new Set(['dependabot[bot]', 'sentry[bot]', 'claude[bot]']);
|
||||
const issueAuthor = context.payload.issue.user.login;
|
||||
const defaultBranch = context.payload.repository.default_branch;
|
||||
const isBotAuthor = issueAuthor.endsWith('[bot]');
|
||||
|
||||
if (TRUSTED_BOT_AUTHORS.has(issueAuthor)) {
|
||||
console.log(`Skipping trusted bot: ${issueAuthor}`);
|
||||
return;
|
||||
}
|
||||
|
||||
async function getPermission(username) {
|
||||
try {
|
||||
const { data: permissionLevel } = await github.rest.repos.getCollaboratorPermissionLevel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
username,
|
||||
});
|
||||
return permissionLevel.permission;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function getTextFile(path) {
|
||||
const { data: fileContent } = await github.rest.repos.getContent({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
path,
|
||||
ref: defaultBranch,
|
||||
});
|
||||
|
||||
if (!('content' in fileContent) || typeof fileContent.content !== 'string') {
|
||||
throw new Error(`Expected file content for ${path}`);
|
||||
}
|
||||
|
||||
return Buffer.from(fileContent.content, 'base64').toString('utf8');
|
||||
}
|
||||
|
||||
function parseApprovedUsers(content) {
|
||||
const users = new Map();
|
||||
|
||||
for (const rawLine of content.split('\n')) {
|
||||
const line = rawLine.trim();
|
||||
if (!line || line.startsWith('#')) continue;
|
||||
|
||||
const parts = line.split(/\s+/);
|
||||
if (parts.length !== 2) {
|
||||
console.log(`Skipping malformed line: ${rawLine}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const [username, capability] = parts;
|
||||
const normalizedCapability = capability.toLowerCase();
|
||||
if (!VALID_CAPABILITIES.has(normalizedCapability)) {
|
||||
console.log(`Skipping line with invalid capability: ${rawLine}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
users.set(username.toLowerCase(), normalizedCapability);
|
||||
}
|
||||
|
||||
return users;
|
||||
}
|
||||
|
||||
const permission = await getPermission(issueAuthor);
|
||||
if (!isBotAuthor && ['admin', 'maintain', 'write'].includes(permission)) {
|
||||
console.log(`${issueAuthor} is a collaborator with ${permission} access`);
|
||||
return;
|
||||
}
|
||||
|
||||
const approvedContent = await getTextFile(APPROVED_FILE);
|
||||
const approvedUsers = parseApprovedUsers(approvedContent);
|
||||
const capability = approvedUsers.get(issueAuthor.toLowerCase());
|
||||
|
||||
if (!isBotAuthor && (capability === 'issue' || capability === 'pr')) {
|
||||
console.log(`${issueAuthor} is approved for ${capability}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const message = [
|
||||
'This issue was auto-closed. All issues from new contributors are auto-closed by default.',
|
||||
'',
|
||||
`Maintainers review auto-closed issues regularly and reopen worthwhile ones. Issues that do not meet the quality bar in [CONTRIBUTING.md](https://github.com/${context.repo.owner}/${context.repo.repo}/blob/${defaultBranch}/CONTRIBUTING.md) will not be reopened or receive a reply.`,
|
||||
'',
|
||||
'Just want to report a bug or share an idea? The [website feedback form](https://frame-control.pages.dev/feedback/) skips this queue.',
|
||||
'',
|
||||
'If a maintainer replies `lgtmi` on one of your issues, your future issues will stay open. If a maintainer replies `lgtm`, your future issues and PRs will stay open. The command must be at the start of the reply (optionally after one or more `@username` mentions) or at the end.',
|
||||
'',
|
||||
`See [CONTRIBUTING.md](https://github.com/${context.repo.owner}/${context.repo.repo}/blob/${defaultBranch}/CONTRIBUTING.md).`,
|
||||
].join('\n');
|
||||
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
body: message,
|
||||
});
|
||||
|
||||
await github.rest.issues.addLabels({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
labels: ['untriaged'],
|
||||
});
|
||||
|
||||
await github.rest.issues.update({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
state: 'closed',
|
||||
state_reason: 'not_planned',
|
||||
});
|
||||
@@ -0,0 +1,145 @@
|
||||
# Contributor gate adapted from badlogic/pi-mono (MIT) at 6f7551516b84.
|
||||
# See CONTRIBUTING.md for how it works.
|
||||
|
||||
name: Issue Triage Labels
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [reopened, labeled]
|
||||
|
||||
jobs:
|
||||
update-labels:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
steps:
|
||||
- name: Update triage labels
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
script: |
|
||||
const UNTRIAGED_LABEL = 'untriaged';
|
||||
const NO_ACTION_LABEL = 'no-action';
|
||||
const LAST_READ_LABEL = 'last-read';
|
||||
const TO_DISCUSS_LABEL = 'to-discuss';
|
||||
const INPROGRESS_LABEL = 'inprogress';
|
||||
|
||||
function issueHasLabel(issue, labelName) {
|
||||
return (issue.labels ?? []).some((label) => label.name === labelName);
|
||||
}
|
||||
|
||||
async function removeLabelIfPresent(issueNumber, issue, labelName) {
|
||||
if (!issueHasLabel(issue, labelName)) {
|
||||
console.log(`Issue #${issueNumber} does not have ${labelName}`);
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
await github.rest.issues.removeLabel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issueNumber,
|
||||
name: labelName,
|
||||
});
|
||||
console.log(`Removed ${labelName} from #${issueNumber}`);
|
||||
} catch (error) {
|
||||
if (error.status === 404) {
|
||||
console.log(`Label ${labelName} was already absent from #${issueNumber}`);
|
||||
return;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
if (context.payload.action === 'reopened') {
|
||||
await removeLabelIfPresent(context.issue.number, context.payload.issue, UNTRIAGED_LABEL);
|
||||
await removeLabelIfPresent(context.issue.number, context.payload.issue, NO_ACTION_LABEL);
|
||||
return;
|
||||
}
|
||||
|
||||
if (context.payload.action === 'labeled' && context.payload.label?.name === NO_ACTION_LABEL) {
|
||||
await removeLabelIfPresent(context.issue.number, context.payload.issue, UNTRIAGED_LABEL);
|
||||
return;
|
||||
}
|
||||
|
||||
if (context.payload.action !== 'labeled' || context.payload.label?.name !== LAST_READ_LABEL) {
|
||||
console.log('Not a last-read label event');
|
||||
return;
|
||||
}
|
||||
|
||||
const currentIssueNumber = context.issue.number;
|
||||
const lastReadIssues = await github.paginate(github.rest.issues.listForRepo, {
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
state: 'all',
|
||||
labels: LAST_READ_LABEL,
|
||||
per_page: 100,
|
||||
});
|
||||
|
||||
const previousIssueNumbers = lastReadIssues
|
||||
.filter((issue) => !issue.pull_request)
|
||||
.map((issue) => issue.number)
|
||||
.filter((issueNumber) => issueNumber !== currentIssueNumber);
|
||||
|
||||
if (previousIssueNumbers.length === 0) {
|
||||
console.log('No previous last-read issue found');
|
||||
return;
|
||||
}
|
||||
|
||||
const previousIssueNumber = Math.max(...previousIssueNumbers);
|
||||
if (currentIssueNumber <= previousIssueNumber) {
|
||||
console.log(
|
||||
`Last-read was added to old issue #${currentIssueNumber}; latest last-read is #${previousIssueNumber}`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
const untriagedIssues = await github.paginate(github.rest.issues.listForRepo, {
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
state: 'all',
|
||||
labels: UNTRIAGED_LABEL,
|
||||
per_page: 100,
|
||||
});
|
||||
|
||||
const issuesToMark = untriagedIssues
|
||||
.filter((issue) => !issue.pull_request)
|
||||
.filter((issue) => issue.number >= previousIssueNumber && issue.number <= currentIssueNumber)
|
||||
.sort((a, b) => a.number - b.number);
|
||||
|
||||
if (issuesToMark.length === 0) {
|
||||
console.log(`No untriaged issues found from #${previousIssueNumber} to #${currentIssueNumber}`);
|
||||
return;
|
||||
}
|
||||
|
||||
for (const issue of issuesToMark) {
|
||||
if (issueHasLabel(issue, TO_DISCUSS_LABEL)) {
|
||||
console.log(`Skipped ${NO_ACTION_LABEL} for #${issue.number} because it has ${TO_DISCUSS_LABEL}`);
|
||||
} else {
|
||||
await github.rest.issues.addLabels({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issue.number,
|
||||
labels: [NO_ACTION_LABEL],
|
||||
});
|
||||
console.log(`Added ${NO_ACTION_LABEL} to #${issue.number}`);
|
||||
}
|
||||
|
||||
await github.rest.issues.update({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issue.number,
|
||||
state: 'closed',
|
||||
state_reason: 'not_planned',
|
||||
});
|
||||
console.log(`Closed #${issue.number} as not planned`);
|
||||
|
||||
await removeLabelIfPresent(issue.number, issue, INPROGRESS_LABEL);
|
||||
|
||||
await github.rest.issues.removeLabel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: issue.number,
|
||||
name: UNTRIAGED_LABEL,
|
||||
});
|
||||
console.log(`Removed ${UNTRIAGED_LABEL} from #${issue.number}`);
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
# Contributor gate adapted from badlogic/pi-mono (MIT) at 6f7551516b84.
|
||||
# See CONTRIBUTING.md for how it works.
|
||||
|
||||
name: PR Gate
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
types: [opened]
|
||||
|
||||
jobs:
|
||||
check-contributor:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
issues: write
|
||||
pull-requests: write
|
||||
steps:
|
||||
- name: Check if contributor is approved
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
script: |
|
||||
const APPROVED_FILE = '.github/APPROVED_CONTRIBUTORS';
|
||||
const VALID_CAPABILITIES = new Set(['issue', 'pr']);
|
||||
const TRUSTED_BOT_AUTHORS = new Set(['dependabot[bot]', 'sentry[bot]', 'claude[bot]']);
|
||||
const prAuthor = context.payload.pull_request.user.login;
|
||||
const defaultBranch = context.payload.repository.default_branch;
|
||||
const isBotAuthor = prAuthor.endsWith('[bot]');
|
||||
|
||||
if (TRUSTED_BOT_AUTHORS.has(prAuthor)) {
|
||||
console.log(`Skipping trusted bot: ${prAuthor}`);
|
||||
return;
|
||||
}
|
||||
|
||||
async function getPermission(username) {
|
||||
try {
|
||||
const { data: permissionLevel } = await github.rest.repos.getCollaboratorPermissionLevel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
username,
|
||||
});
|
||||
return permissionLevel.permission;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async function getTextFile(path) {
|
||||
const { data: fileContent } = await github.rest.repos.getContent({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
path,
|
||||
ref: defaultBranch,
|
||||
});
|
||||
|
||||
if (!('content' in fileContent) || typeof fileContent.content !== 'string') {
|
||||
throw new Error(`Expected file content for ${path}`);
|
||||
}
|
||||
|
||||
return Buffer.from(fileContent.content, 'base64').toString('utf8');
|
||||
}
|
||||
|
||||
function parseApprovedUsers(content) {
|
||||
const users = new Map();
|
||||
|
||||
for (const rawLine of content.split('\n')) {
|
||||
const line = rawLine.trim();
|
||||
if (!line || line.startsWith('#')) continue;
|
||||
|
||||
const parts = line.split(/\s+/);
|
||||
if (parts.length !== 2) {
|
||||
console.log(`Skipping malformed line: ${rawLine}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
const [username, capability] = parts;
|
||||
const normalizedCapability = capability.toLowerCase();
|
||||
if (!VALID_CAPABILITIES.has(normalizedCapability)) {
|
||||
console.log(`Skipping line with invalid capability: ${rawLine}`);
|
||||
continue;
|
||||
}
|
||||
|
||||
users.set(username.toLowerCase(), normalizedCapability);
|
||||
}
|
||||
|
||||
return users;
|
||||
}
|
||||
|
||||
async function closePullRequest(message) {
|
||||
await github.rest.issues.createComment({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.payload.pull_request.number,
|
||||
body: message,
|
||||
});
|
||||
|
||||
await github.rest.pulls.update({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
pull_number: context.payload.pull_request.number,
|
||||
state: 'closed',
|
||||
});
|
||||
}
|
||||
|
||||
const permission = await getPermission(prAuthor);
|
||||
if (!isBotAuthor && ['admin', 'maintain', 'write'].includes(permission)) {
|
||||
console.log(`${prAuthor} is a collaborator with ${permission} access`);
|
||||
return;
|
||||
}
|
||||
|
||||
const approvedContent = await getTextFile(APPROVED_FILE);
|
||||
const approvedUsers = parseApprovedUsers(approvedContent);
|
||||
const capability = approvedUsers.get(prAuthor.toLowerCase());
|
||||
|
||||
if (!isBotAuthor && capability === 'pr') {
|
||||
console.log(`${prAuthor} is approved for PRs`);
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`${prAuthor} is not approved, closing PR`);
|
||||
|
||||
const message = [
|
||||
'This PR was auto-closed. Only contributors approved with `lgtm` can open PRs. Open an issue first and ask a maintainer for approval.',
|
||||
'',
|
||||
`Maintainers review auto-closed issues regularly. Issues that do not meet the quality bar in [CONTRIBUTING.md](https://github.com/${context.repo.owner}/${context.repo.repo}/blob/${defaultBranch}/CONTRIBUTING.md) will not be reopened or receive a reply.`,
|
||||
'',
|
||||
'If a maintainer replies `lgtmi`, your future issues will stay open. If a maintainer replies `lgtm`, your future issues and PRs will stay open. The command must be at the start of the reply (optionally after one or more `@username` mentions) or at the end.',
|
||||
'',
|
||||
`See [CONTRIBUTING.md](https://github.com/${context.repo.owner}/${context.repo.repo}/blob/${defaultBranch}/CONTRIBUTING.md).`,
|
||||
].join('\n');
|
||||
|
||||
await closePullRequest(message);
|
||||
@@ -0,0 +1,34 @@
|
||||
# Contributor gate adapted from badlogic/pi-mono (MIT) at 6f7551516b84.
|
||||
# See CONTRIBUTING.md for how it works.
|
||||
|
||||
name: Remove In Progress Label On Close
|
||||
|
||||
on:
|
||||
issues:
|
||||
types: [closed]
|
||||
|
||||
jobs:
|
||||
remove-label:
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
issues: write
|
||||
steps:
|
||||
- name: Remove inprogress label
|
||||
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
|
||||
with:
|
||||
script: |
|
||||
const labelName = 'inprogress';
|
||||
const labels = context.payload.issue.labels ?? [];
|
||||
const hasLabel = labels.some((label) => label.name === labelName);
|
||||
|
||||
if (!hasLabel) {
|
||||
console.log(`Issue does not have ${labelName} label`);
|
||||
return;
|
||||
}
|
||||
|
||||
await github.rest.issues.removeLabel({
|
||||
owner: context.repo.owner,
|
||||
repo: context.repo.repo,
|
||||
issue_number: context.issue.number,
|
||||
name: labelName,
|
||||
});
|
||||
+2
-1
@@ -1,7 +1,8 @@
|
||||
.DS_Store
|
||||
__pycache__/
|
||||
apk-catalog/data/cache/
|
||||
apk-catalog/data/index-v2.json*
|
||||
apk-catalog/data/index-v2*.json*
|
||||
compat-db/.env.lakebed.server
|
||||
compat-db/.lakebed/
|
||||
tests/smoke/results/
|
||||
mac/bin/
|
||||
@@ -0,0 +1,71 @@
|
||||
# Contributing to Frame Control
|
||||
|
||||
This guide exists to save both sides time. The process is borrowed from
|
||||
[pi](https://github.com/badlogic/pi-mono/blob/main/CONTRIBUTING.md).
|
||||
|
||||
## Just want to report something?
|
||||
|
||||
Use the [feedback form](https://frame-control.pages.dev/feedback/). It needs no
|
||||
GitHub account, and what you send becomes an issue here that stays open.
|
||||
|
||||
## The One Rule
|
||||
|
||||
**You must understand your code.** If you can't explain what your change does
|
||||
and how it interacts with the rest of the app, your PR will be closed.
|
||||
|
||||
Using AI to write code is fine. Submitting AI-generated slop you don't
|
||||
understand is not.
|
||||
|
||||
## Contribution gate
|
||||
|
||||
Issues and PRs opened on GitHub by new contributors are auto-closed by default.
|
||||
A maintainer reviews auto-closed issues regularly and reopens worthwhile ones.
|
||||
Issues that don't meet the quality bar below won't be reopened or get a reply.
|
||||
|
||||
Approval happens through maintainer replies on issues:
|
||||
|
||||
- `lgtmi`: your future issues won't be auto-closed
|
||||
- `lgtm`: your future issues and PRs won't be auto-closed
|
||||
|
||||
The word must be at the start of the reply (optionally after one or more
|
||||
`@username` mentions) or at the end. Only `lgtm` lets you open PRs. Approved
|
||||
people are listed in [`.github/APPROVED_CONTRIBUTORS`](.github/APPROVED_CONTRIBUTORS).
|
||||
|
||||
## Quality bar for issues
|
||||
|
||||
Use one of the issue templates, and keep it short, concrete and worth reading.
|
||||
|
||||
- If it doesn't fit on one screen, it's too long.
|
||||
- Write in your own voice. If you must use an LLM, say so in a clearly labelled
|
||||
follow-up comment.
|
||||
- State the bug or request clearly, and why it matters.
|
||||
- For bugs, include your OS, your SteamOS build (Steam Settings → System), and
|
||||
the server log (**Frame → Show Server Log** in the app).
|
||||
- If you want to implement the change yourself, say so.
|
||||
|
||||
## Before opening a PR
|
||||
|
||||
Don't open a PR until a maintainer has approved you with `lgtm`. Open an
|
||||
[idea or contribution proposal](https://github.com/saphid/frame-control/issues/new?template=idea.yml)
|
||||
first.
|
||||
|
||||
Then check your change:
|
||||
|
||||
```sh
|
||||
python3 -m unittest discover -s tests # server tests; no headset needed
|
||||
node --test site/test/*.test.mjs # website feedback function
|
||||
```
|
||||
|
||||
Say what you tested, and whether you tried it on a real Steam Frame.
|
||||
|
||||
## Blocking
|
||||
|
||||
If you ignore this document twice, or spam the tracker with agent-generated
|
||||
issues, your GitHub account will be blocked from the repo.
|
||||
|
||||
## Why auto-close?
|
||||
|
||||
This is a hobby project with one maintainer. Auto-closing is a buffer against
|
||||
burnout and tracker spam: issues get reviewed on the maintainer's schedule, and
|
||||
the good ones are reopened. Short, concrete, reproducible reports and thoughtful
|
||||
contributions are welcome.
|
||||
@@ -12,12 +12,17 @@ See what the headset sees, install games and Android apps, move files and text a
|
||||
[](https://github.com/saphid/steam-frame/actions/workflows/checks.yml)
|
||||
[](LICENSE)
|
||||
|
||||
[**Download**](#install) · [Features](#features) · [Set up the headset](#set-up-the-headset) · [Feedback](#feedback) · [Docs](#going-further)
|
||||
[**Website**](https://frame-control.pages.dev) · [**Download**](#install) · [Trailer](#trailer) · [Features](#features) · [Set up the headset](#set-up-the-headset) · [Feedback](#feedback) · [Docs](#going-further)
|
||||
|
||||
<br>
|
||||
|
||||
<img src="docs/img/frame-control.png" alt="Frame Control's Games tab: installed games, sideloaded titles, and your Steam library with Frame ratings" width="900">
|
||||
|
||||
<a id="trailer"></a>
|
||||
<a href="https://github.com/saphid/steam-frame/releases/download/trailer/frame-control-trailer.mp4"><img src="docs/img/trailer.jpg" alt="Watch the Frame Control trailer" width="900"></a>
|
||||
|
||||
<sub>The trailer: 66 seconds, with sound. Downloads the MP4 from the trailer release.</sub>
|
||||
|
||||
<sub>Unofficial hobby project, not affiliated with Valve. Free and open source.</sub>
|
||||
|
||||
</div>
|
||||
@@ -102,6 +107,10 @@ already ships (sideloading a game copies Valve's own devkit scripts to
|
||||
a computer. Build it from [`ios/`](ios) in Xcode; see [docs/iphone.md](docs/iphone.md).
|
||||
|
||||
The app brings its own Python and `adb`; SSH is built into macOS and Windows.
|
||||
From 0.4 it updates itself: when a new version is published, a banner offers
|
||||
**Update and restart**. It sends anonymous usage statistics, which you can turn
|
||||
off. Sharing compatibility results and error details is opt-in. See
|
||||
[docs/privacy.md](docs/privacy.md).
|
||||
Google doesn't publish `adb` for arm64 Linux, so that build uses your
|
||||
distribution's. If you already have `adb`, the app uses yours.
|
||||
|
||||
@@ -170,13 +179,19 @@ entry to `~/.ssh/config` and keys at `~/.ssh/id_ed25519_frame` and
|
||||
## Feedback
|
||||
|
||||
This is a first public test, so reports are really useful, especially from
|
||||
Windows and Linux. Please [open an issue](https://github.com/saphid/steam-frame/issues/new)
|
||||
with:
|
||||
Windows and Linux. The quickest way is **Report a problem** in the app (the
|
||||
warning-sign button at the top, or **Help → Report a Problem…**). It adds
|
||||
diagnostics with personal details removed, shows you exactly what's included,
|
||||
and sends it privately to the maintainer; nothing is published. Without the app,
|
||||
use the [feedback form](https://frame-control.pages.dev/feedback/). Please include:
|
||||
|
||||
- what you tried and what happened
|
||||
- your computer's OS and your SteamOS build (Steam Settings → System)
|
||||
- the server log: **Frame → Show Server Log** in the app
|
||||
|
||||
Issues and PRs opened directly on GitHub by new contributors are auto-closed
|
||||
until a maintainer approves them; see [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
||||
## Going further
|
||||
|
||||
This repo also holds the scripts behind the app and field notes on how the
|
||||
@@ -192,9 +207,12 @@ Frame's software fits together, all checked against a real headset and labelled
|
||||
| [Sideloading Linux and Windows games](docs/sideloading.md) | A .zip, folder or .exe as a Steam Devkit Game, runtime detection |
|
||||
| [Install links for websites](docs/web-install.md) | `frame-control://install` links and manifests, the rules, a button to paste |
|
||||
| [Steam games](docs/steam-games.md) · [VR video](docs/vr-video.md) · [WebXR in Chromium](docs/webxr-chromium.md) | Installing and buying, watching VR180/360, the Chromium build |
|
||||
| [Mac in the headset](docs/mac-in-headset.md) | Mac windows and screens as panels in the Frame, with laser and keyboard input |
|
||||
| [VR mods and custom songs](docs/mods.md) | Per-game feasibility, real-Frame results and blockers; no installer yet |
|
||||
| [SSH](docs/ssh.md) · [Streaming](docs/streaming.md) · [Files](docs/file-transfer.md) · [Panels](docs/panels.md) · [Tailscale](docs/tailscale.md) | Topic notes |
|
||||
| [Frame Control for iPhone](docs/iphone.md) | The iPhone and iPad app, how it runs the server on the Frame, pairing |
|
||||
| [Recovery and OS images](docs/recovery-and-images.md) | Where to download the Frame's OS, what's inside, testing without the headset |
|
||||
| [AI agents and assistant](docs/agents.md) | Key-free MCP tools, human approvals, and an opt-in assistant panel |
|
||||
| [Testing](docs/testing.md) | Unit tests, end-to-end tests against a fake Frame in Docker, and the headset smoke test |
|
||||
| [Open questions](docs/open-questions.md) | What's still unchecked |
|
||||
|
||||
@@ -228,7 +246,8 @@ cd app && npm install && npm start # run the app from the checkout
|
||||
|
||||
The server is Python stdlib only; the app is Electron. GitHub Actions runs the
|
||||
tests on macOS, Windows and Linux, and a `v*` tag builds all three installers
|
||||
into the release. See [building](docs/frame-control.md#building).
|
||||
into a draft release, which reaches users once published. See
|
||||
[building](docs/frame-control.md#building) and [releasing](docs/releasing.md).
|
||||
|
||||
## License
|
||||
|
||||
|
||||
+98
-2
@@ -10,6 +10,7 @@ const net = require("net");
|
||||
const os = require("os");
|
||||
const path = require("path");
|
||||
const { SCHEME, parseInstallLink, linkFromArgv } = require("./install-link");
|
||||
const updater = require("./updater");
|
||||
|
||||
const run = promisify(execFile);
|
||||
|
||||
@@ -114,7 +115,11 @@ function ping(target) {
|
||||
}
|
||||
|
||||
async function startServer() {
|
||||
// The version and whether this is a built app go to ui/frame_telemetry.py, which
|
||||
// sends nothing from a source checkout.
|
||||
const env = { ...process.env, PATH: await loginPath(), FRAME_CONTROL_APP: "1",
|
||||
FRAME_CONTROL_VERSION: app.getVersion(), FRAME_CONTROL_LOG: LOG,
|
||||
...(app.isPackaged ? { FRAME_CONTROL_PACKAGED: "1" } : {}),
|
||||
...(fs.existsSync(TOOLS) ? { FRAME_CONTROL_TOOLS: TOOLS } : {}) };
|
||||
python = await findPython(env);
|
||||
if (!python) throw new Error(`Frame Control needs Python 3.8 or later. ${PYTHON_HELP}`);
|
||||
@@ -244,6 +249,9 @@ function fromUi(e) {
|
||||
|
||||
ipcMain.handle("clipboard:read", (e) => fromUi(e) ? clipboard.readText() : "");
|
||||
ipcMain.handle("connection:setup", (e) => { if (fromUi(e)) setUpConnection(); });
|
||||
ipcMain.handle("update:get", (e) => fromUi(e) ? publicUpdate() : null);
|
||||
ipcMain.handle("update:check", (e) => fromUi(e) ? checkForUpdate({ manual: true }).then(publicUpdate) : null);
|
||||
ipcMain.handle("update:install", (e) => { if (fromUi(e)) installUpdate(); });
|
||||
|
||||
// frame-control://install links from websites (docs/web-install.md). They can
|
||||
// arrive before the window or server exists (macOS open-url on a cold launch),
|
||||
@@ -276,6 +284,81 @@ ipcMain.on("install-link:ready", (e) => {
|
||||
deliverLinks();
|
||||
});
|
||||
|
||||
// ---- updates (app/updater.js, docs/releasing.md) ----
|
||||
// Checked shortly after launch and every few hours; the page shows a banner and
|
||||
// the Update button calls installUpdate.
|
||||
const UPDATE_EVERY = 6 * 3600 * 1000;
|
||||
const update = { status: "idle", current: app.getVersion(), latest: null, error: null, progress: 0, how: null };
|
||||
|
||||
function publicUpdate() {
|
||||
const r = update.latest;
|
||||
return { status: update.status, current: update.current, error: update.error, progress: update.progress,
|
||||
latest: r && { version: r.version, notes: r.notes, page: r.page },
|
||||
canInstall: !!update.how && update.how.method !== "manual", why: update.how && update.how.why };
|
||||
}
|
||||
|
||||
function setUpdate(fields) {
|
||||
Object.assign(update, fields);
|
||||
if (win && linkPage === win.webContents) win.webContents.send("update:state", publicUpdate());
|
||||
}
|
||||
|
||||
async function checkForUpdate({ manual = false } = {}) {
|
||||
if (["checking", "downloading", "ready"].includes(update.status)) return;
|
||||
setUpdate({ status: "checking", error: null });
|
||||
try {
|
||||
const latest = await updater.latestRelease();
|
||||
const how = updater.updateMethod({ platform: process.platform, isPackaged: app.isPackaged,
|
||||
execPath: process.execPath, env: process.env,
|
||||
exists: fs.existsSync, writable: updater.writable });
|
||||
if (updater.isNewer(latest.version, update.current)) {
|
||||
setUpdate({ status: "available", latest, how });
|
||||
if (manual) offerUpdateDialog();
|
||||
} else {
|
||||
setUpdate({ status: "none", latest, how });
|
||||
if (manual) dialog.showMessageBox(win, { type: "info", message: "Frame Control is up to date",
|
||||
detail: `You have ${update.current}, the newest version.` });
|
||||
}
|
||||
} catch (e) {
|
||||
// A failed check: nothing to install, and never an older release kept from before.
|
||||
setUpdate({ status: "check-failed", error: e.message, latest: null });
|
||||
if (manual) dialog.showMessageBox(win, { type: "warning", message: "Couldn't check for updates", detail: e.message });
|
||||
}
|
||||
}
|
||||
|
||||
async function offerUpdateDialog() {
|
||||
const r = update.latest;
|
||||
const { response } = await dialog.showMessageBox(win, {
|
||||
type: "info", message: `Frame Control ${r.version} is available`,
|
||||
detail: `You have ${update.current}.` + (update.how.method === "manual" ? ` Download it from the release page (${update.how.why}).` : ""),
|
||||
buttons: [update.how.method === "manual" ? "Open Release Page" : "Update and Restart", "Later"], defaultId: 0, cancelId: 1,
|
||||
});
|
||||
if (response === 0) installUpdate();
|
||||
}
|
||||
|
||||
async function installUpdate() {
|
||||
// "error" here only ever means an install failed, so trying again is safe.
|
||||
if (update.status !== "available" && update.status !== "error") return;
|
||||
if (!update.latest || !updater.isNewer(update.latest.version, update.current)) return;
|
||||
if (!update.how || update.how.method === "manual") { shell.openExternal(update.latest.page); return; }
|
||||
setUpdate({ status: "downloading", progress: 0, error: null });
|
||||
try {
|
||||
const start = await updater.prepare(update.latest, update.how,
|
||||
(done, total) => { if (total) setUpdate({ progress: done / total }); }, update.current);
|
||||
setUpdate({ status: "ready", progress: 1 });
|
||||
start();
|
||||
quitting = true;
|
||||
app.quit();
|
||||
} catch (e) {
|
||||
setUpdate({ status: "error", error: e.message });
|
||||
}
|
||||
}
|
||||
|
||||
function scheduleUpdateChecks() {
|
||||
if (process.env.FRAME_CONTROL_NO_UPDATE_CHECK === "1") return;
|
||||
setTimeout(checkForUpdate, 8000);
|
||||
setInterval(checkForUpdate, UPDATE_EVERY).unref();
|
||||
}
|
||||
|
||||
function registerScheme() {
|
||||
// A checkout runs as `electron .`, so the OS must be told the script too.
|
||||
// (macOS takes the scheme from Info.plist, which only the built app has.)
|
||||
@@ -337,7 +420,11 @@ async function setUpConnection() {
|
||||
|
||||
function buildMenu() {
|
||||
const template = [
|
||||
...(IS_MAC ? [{ role: "appMenu" }] : []),
|
||||
...(IS_MAC ? [{ label: app.name, submenu: [
|
||||
{ role: "about" }, { label: "Check for Updates…", click: () => checkForUpdate({ manual: true }) },
|
||||
{ type: "separator" }, { role: "services" }, { type: "separator" },
|
||||
{ role: "hide" }, { role: "hideOthers" }, { role: "unhide" }, { type: "separator" }, { role: "quit" },
|
||||
] }] : []),
|
||||
{ role: "fileMenu" },
|
||||
{ role: "editMenu" },
|
||||
{
|
||||
@@ -364,7 +451,15 @@ function buildMenu() {
|
||||
...(IS_MAC ? [{ role: "windowMenu" }] : []),
|
||||
{
|
||||
role: "help",
|
||||
submenu: [{ label: "Project on GitHub", click: () => shell.openExternal("https://github.com/saphid/steam-frame") }],
|
||||
submenu: [
|
||||
...(IS_MAC ? [] : [{ label: "Check for Updates…", click: () => checkForUpdate({ manual: true }) }]),
|
||||
{ label: "Report a Problem…", click: () => {
|
||||
if (win && url && linkPage === win.webContents) win.webContents.send("report:open");
|
||||
else shell.openExternal("https://frame-control.pages.dev/feedback/"); // the page isn't up
|
||||
} },
|
||||
{ label: "Release Notes", click: () => shell.openExternal(updater.RELEASES) },
|
||||
{ label: "Project on GitHub", click: () => shell.openExternal("https://github.com/saphid/steam-frame") },
|
||||
],
|
||||
},
|
||||
];
|
||||
Menu.setApplicationMenu(Menu.buildFromTemplate(template));
|
||||
@@ -387,6 +482,7 @@ if (!app.requestSingleInstanceLock()) {
|
||||
registerScheme();
|
||||
buildMenu();
|
||||
createWindow();
|
||||
scheduleUpdateChecks();
|
||||
});
|
||||
app.on("activate", () => { if (!win) createWindow(); });
|
||||
app.on("window-all-closed", () => app.quit());
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "frame-control",
|
||||
"version": "0.3.1",
|
||||
"version": "0.4.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "frame-control",
|
||||
"version": "0.3.1",
|
||||
"version": "0.4.0",
|
||||
"license": "MIT",
|
||||
"devDependencies": {
|
||||
"electron": "^44.4.5",
|
||||
|
||||
+17
-5
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "frame-control",
|
||||
"productName": "Frame Control",
|
||||
"version": "0.3.1",
|
||||
"version": "0.4.0",
|
||||
"description": "Desktop app for managing a Valve Steam Frame over SSH",
|
||||
"private": true,
|
||||
"main": "main.js",
|
||||
@@ -9,8 +9,8 @@
|
||||
"scripts": {
|
||||
"start": "env -u ELECTRON_RUN_AS_NODE electron .",
|
||||
"icon": "env -u ELECTRON_RUN_AS_NODE electron build/make-icon.js",
|
||||
"dist": "node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --publish never",
|
||||
"dist:dir": "node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --dir",
|
||||
"dist": "sh ../mac/frame-mac-view/build.sh && node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --publish never",
|
||||
"dist:dir": "sh ../mac/frame-mac-view/build.sh && node build/fetch-deps.js mac arm64 && electron-builder --mac --arm64 --dir",
|
||||
"dist:linux": "node build/fetch-deps.js linux x64 arm64 && electron-builder --linux --x64 --arm64 --publish never",
|
||||
"dist:win": "node build/fetch-deps.js win x64 && electron-builder --win --x64 --publish never"
|
||||
},
|
||||
@@ -37,6 +37,7 @@
|
||||
"main.js",
|
||||
"preload.js",
|
||||
"install-link.js",
|
||||
"updater.js",
|
||||
"package.json",
|
||||
"build/icon.png"
|
||||
],
|
||||
@@ -46,7 +47,8 @@
|
||||
"to": "ui",
|
||||
"filter": [
|
||||
"*.py",
|
||||
"*.html"
|
||||
"*.html",
|
||||
"telemetry.json"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -105,9 +107,19 @@
|
||||
"dmg",
|
||||
"zip"
|
||||
],
|
||||
"extraResources": [
|
||||
{
|
||||
"from": "../mac/bin",
|
||||
"to": "mac/bin",
|
||||
"filter": [
|
||||
"frame-mac-view"
|
||||
]
|
||||
}
|
||||
],
|
||||
"extendInfo": {
|
||||
"NSAppleEventsUsageDescription": "Frame Control opens Terminal for SSH sessions and for power actions that need the Developer Mode password.",
|
||||
"NSLocalNetworkUsageDescription": "Frame Control connects to your Steam Frame over SSH on the local network."
|
||||
"NSLocalNetworkUsageDescription": "Frame Control connects to your Steam Frame over SSH on the local network.",
|
||||
"NSScreenCaptureUsageDescription": "Frame Control shows your Mac's windows and screens inside the Steam Frame when you ask it to."
|
||||
},
|
||||
"artifactName": "Frame-Control-mac-${arch}.${ext}"
|
||||
},
|
||||
|
||||
@@ -5,12 +5,28 @@
|
||||
// It can open Set Up Connection when the headset can't be reached.
|
||||
// It also receives frame-control://install links (docs/web-install.md): only
|
||||
// what the link asked for, never an install; the page asks the user first.
|
||||
// And it passes update state both ways: see app/updater.js.
|
||||
const { contextBridge, ipcRenderer, webUtils } = require("electron");
|
||||
|
||||
contextBridge.exposeInMainWorld("frameApp", {
|
||||
readClipboard: () => ipcRenderer.invoke("clipboard:read"),
|
||||
setUpConnection: () => ipcRenderer.invoke("connection:setup"),
|
||||
pathForFile: (file) => { try { return webUtils.getPathForFile(file) || ""; } catch { return ""; } },
|
||||
// Updates (app/updater.js): the page shows a banner and an Update button.
|
||||
update: {
|
||||
get: () => ipcRenderer.invoke("update:get"),
|
||||
check: () => ipcRenderer.invoke("update:check"),
|
||||
install: () => ipcRenderer.invoke("update:install"),
|
||||
onState: (cb) => {
|
||||
ipcRenderer.removeAllListeners("update:state");
|
||||
ipcRenderer.on("update:state", (_e, s) => cb(s));
|
||||
},
|
||||
},
|
||||
// Help → Report a Problem… opens the page's report dialog (ui/frame_report.py).
|
||||
onReportProblem: (cb) => {
|
||||
ipcRenderer.removeAllListeners("report:open");
|
||||
ipcRenderer.on("report:open", () => cb());
|
||||
},
|
||||
onInstallLink: (cb) => {
|
||||
ipcRenderer.removeAllListeners("install-link");
|
||||
ipcRenderer.on("install-link", (_e, req) => cb({ kind: req.kind, target: req.target }));
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
// Run: node --test app/test/
|
||||
const test = require("node:test");
|
||||
const assert = require("node:assert");
|
||||
const { isNewer, assetName, updateMethod, macBundle } = require("../updater");
|
||||
|
||||
test("versions compare numerically, and a release beats its pre-releases", () => {
|
||||
assert.ok(isNewer("0.3.10", "0.3.9"));
|
||||
assert.ok(isNewer("v1.0.0", "0.9.9"));
|
||||
assert.ok(!isNewer("0.3.1", "0.3.1"));
|
||||
assert.ok(!isNewer("0.3.0", "0.3.1"));
|
||||
assert.ok(isNewer("1.0.0", "1.0.0-beta.1"));
|
||||
assert.ok(!isNewer("1.0.0-beta.1", "1.0.0"));
|
||||
assert.ok(!isNewer("garbage", "0.1.0"));
|
||||
});
|
||||
|
||||
test("asset names match what electron-builder publishes", () => {
|
||||
assert.strictEqual(assetName("darwin", "arm64", "mac-zip"), "Frame-Control-mac-arm64.zip");
|
||||
assert.strictEqual(assetName("win32", "x64", "nsis"), "Frame-Control-Setup-x64.exe");
|
||||
assert.strictEqual(assetName("linux", "x64", "appimage"), "Frame-Control-linux-x86_64.AppImage");
|
||||
assert.strictEqual(assetName("linux", "arm64", "appimage"), "Frame-Control-linux-arm64.AppImage");
|
||||
});
|
||||
|
||||
const base = { isPackaged: true, env: {}, exists: () => false, writable: () => true };
|
||||
|
||||
test("macOS updates in place only from a writable, non-translocated location", () => {
|
||||
const exe = "/Applications/Frame Control.app/Contents/MacOS/Frame Control";
|
||||
assert.strictEqual(macBundle(exe), "/Applications/Frame Control.app");
|
||||
assert.deepStrictEqual(updateMethod({ ...base, platform: "darwin", execPath: exe }),
|
||||
{ method: "mac-zip", bundle: "/Applications/Frame Control.app" });
|
||||
const dmg = "/Volumes/Frame Control 0.3.1/Frame Control.app/Contents/MacOS/Frame Control";
|
||||
assert.strictEqual(updateMethod({ ...base, platform: "darwin", execPath: dmg }).method, "manual");
|
||||
const trans = "/private/var/folders/x/AppTranslocation/ABC/d/Frame Control.app/Contents/MacOS/Frame Control";
|
||||
assert.strictEqual(updateMethod({ ...base, platform: "darwin", execPath: trans }).method, "manual");
|
||||
assert.strictEqual(updateMethod({ ...base, platform: "darwin", execPath: exe, writable: () => false }).method, "manual");
|
||||
});
|
||||
|
||||
test("Windows needs the installer's copy; Linux needs an AppImage", () => {
|
||||
const exe = "C:\\Users\\a\\AppData\\Local\\Programs\\Frame Control\\Frame Control.exe";
|
||||
assert.strictEqual(updateMethod({ ...base, platform: "win32", execPath: exe, exists: () => true }).method, "nsis");
|
||||
assert.strictEqual(updateMethod({ ...base, platform: "win32", execPath: exe }).method, "manual");
|
||||
assert.strictEqual(updateMethod({ ...base, platform: "linux", execPath: "/opt/x", env: { APPIMAGE: "/home/a/F.AppImage" } }).method,
|
||||
"appimage");
|
||||
assert.strictEqual(updateMethod({ ...base, platform: "linux", execPath: "/opt/Frame Control/frame-control" }).method, "manual");
|
||||
assert.strictEqual(updateMethod({ ...base, isPackaged: false, platform: "darwin", execPath: "x" }).method, "manual");
|
||||
});
|
||||
|
||||
test("update.json assets always download from this repository's release", () => {
|
||||
const r = require("../updater").fromManifest({ version: "0.4.0", notes: "n", assets: [
|
||||
{ name: "Frame-Control-mac-arm64.zip", url: "https://evil.example/x.zip", digest: "sha256:" + "a".repeat(64) }] });
|
||||
assert.strictEqual(r.assets[0].url, "https://github.com/saphid/frame-control/releases/download/v0.4.0/Frame-Control-mac-arm64.zip");
|
||||
assert.throws(() => require("../updater").fromManifest({ version: "nope", assets: [] }));
|
||||
});
|
||||
|
||||
test("prepare refuses a release that isn't newer (no downgrades)", async () => {
|
||||
const { prepare } = require("../updater");
|
||||
const release = { version: "0.3.1", assets: [] };
|
||||
await assert.rejects(prepare(release, { method: "appimage", appImage: "/nonexistent/x" }, null, "0.4.0"), /isn't newer/);
|
||||
await assert.rejects(prepare(release, { method: "appimage", appImage: "/nonexistent/x" }, null, "0.3.1"), /isn't newer/);
|
||||
});
|
||||
+250
@@ -0,0 +1,250 @@
|
||||
// Update checks and self-update for the desktop app (docs/releasing.md).
|
||||
//
|
||||
// The newest version is GitHub's "latest" release of saphid/frame-control. Drafts
|
||||
// and pre-releases never count, so a build reaches people only when the
|
||||
// maintainer publishes it after testing (scripts/publish-release.sh). That
|
||||
// script attaches update.json (version, notes, each asset's SHA-256), read
|
||||
// through github.com's latest/download link: the REST API allows only 60
|
||||
// unauthenticated requests an hour per IP address, shared by everyone behind
|
||||
// the same router, so it's only the fallback.
|
||||
//
|
||||
// Every download is checked against the SHA-256 digest GitHub records for the
|
||||
// asset before anything is replaced. How the update is applied:
|
||||
// macOS the .zip: unpacked next to the running app, swapped in by a small
|
||||
// script once the app has quit, then reopened.
|
||||
// Windows the NSIS installer, run silently over the current install; it
|
||||
// reopens the app. A copy unpacked from the .zip is updated by hand.
|
||||
// Linux the AppImage replaces itself; .deb installs are updated by hand.
|
||||
// When the app can't update itself it opens the release page instead.
|
||||
const { execFile, spawn } = require("child_process");
|
||||
const crypto = require("crypto");
|
||||
const fs = require("fs");
|
||||
const https = require("https");
|
||||
const os = require("os");
|
||||
const path = require("path");
|
||||
|
||||
const REPO = "saphid/frame-control"; // renamed from saphid/steam-frame; GitHub redirects the old name
|
||||
const LATEST = `https://api.github.com/repos/${REPO}/releases/latest`;
|
||||
const MANIFEST = `https://github.com/${REPO}/releases/latest/download/update.json`;
|
||||
const RELEASES = `https://github.com/${REPO}/releases`;
|
||||
|
||||
// "0.3.1" or "v0.3.1" -> [0, 3, 1]; pre-release suffixes sort before the release.
|
||||
function parseVersion(v) {
|
||||
const m = String(v || "").trim().replace(/^v/i, "").match(/^(\d+)\.(\d+)\.(\d+)(?:-([0-9A-Za-z.-]+))?$/);
|
||||
return m ? { nums: [+m[1], +m[2], +m[3]], pre: m[4] || null } : null;
|
||||
}
|
||||
|
||||
function isNewer(candidate, current) {
|
||||
const a = parseVersion(candidate), b = parseVersion(current);
|
||||
if (!a || !b) return false;
|
||||
for (let i = 0; i < 3; i++) if (a.nums[i] !== b.nums[i]) return a.nums[i] > b.nums[i];
|
||||
if (a.pre === b.pre) return false;
|
||||
if (!a.pre) return true; // 1.0.0 is newer than 1.0.0-beta
|
||||
if (!b.pre) return false;
|
||||
return a.pre > b.pre;
|
||||
}
|
||||
|
||||
// The asset this copy of the app updates from, by the names electron-builder gives them.
|
||||
function assetName(platform, arch, method) {
|
||||
if (method === "mac-zip") return `Frame-Control-mac-${arch}.zip`;
|
||||
if (method === "nsis") return `Frame-Control-Setup-${arch}.exe`;
|
||||
if (method === "appimage") return `Frame-Control-linux-${arch === "x64" ? "x86_64" : arch}.AppImage`;
|
||||
return null;
|
||||
}
|
||||
|
||||
// How this copy can update itself: mac-zip, nsis, appimage, or manual (with why).
|
||||
function updateMethod({ platform, isPackaged, execPath, env, exists, writable }) {
|
||||
if (!isPackaged) return { method: "manual", why: "running from a source checkout" };
|
||||
if (platform === "darwin") {
|
||||
const bundle = macBundle(execPath);
|
||||
if (!bundle) return { method: "manual", why: "can't find the app bundle" };
|
||||
if (bundle.includes("/AppTranslocation/") || bundle.startsWith("/Volumes/")) {
|
||||
return { method: "manual", why: "move Frame Control to Applications first" };
|
||||
}
|
||||
if (!writable(path.dirname(bundle))) return { method: "manual", why: `${path.dirname(bundle)} isn't writable` };
|
||||
return { method: "mac-zip", bundle };
|
||||
}
|
||||
if (platform === "win32") {
|
||||
// electron-builder's NSIS install puts its uninstaller next to the app.
|
||||
const dir = path.dirname(execPath);
|
||||
if (exists(path.join(dir, "Uninstall Frame Control.exe"))) return { method: "nsis" };
|
||||
return { method: "manual", why: "not installed with the installer" };
|
||||
}
|
||||
if (platform === "linux" && env.APPIMAGE) {
|
||||
if (!writable(path.dirname(env.APPIMAGE))) return { method: "manual", why: "the AppImage's folder isn't writable" };
|
||||
return { method: "appimage", appImage: env.APPIMAGE };
|
||||
}
|
||||
return { method: "manual", why: "installed from a package" };
|
||||
}
|
||||
|
||||
function macBundle(execPath) {
|
||||
const i = execPath.indexOf(".app/Contents/MacOS/");
|
||||
return i < 0 ? null : execPath.slice(0, i + 4);
|
||||
}
|
||||
|
||||
function get(url, { headers = {}, timeout = 20000, redirects = 5 } = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const req = https.get(url, { headers: { "user-agent": "FrameControl-updater", ...headers }, timeout }, (res) => {
|
||||
if ([301, 302, 303, 307, 308].includes(res.statusCode) && res.headers.location && redirects > 0) {
|
||||
res.resume();
|
||||
const next = new URL(res.headers.location, url);
|
||||
if (next.protocol !== "https:") return reject(new Error("refusing a non-HTTPS redirect"));
|
||||
return resolve(get(next.href, { headers, timeout, redirects: redirects - 1 }));
|
||||
}
|
||||
if (res.statusCode !== 200) { res.resume(); return reject(new Error(`HTTP ${res.statusCode} from ${new URL(url).host}`)); }
|
||||
resolve(res);
|
||||
});
|
||||
req.on("timeout", () => req.destroy(new Error("timed out")));
|
||||
req.on("error", reject);
|
||||
});
|
||||
}
|
||||
|
||||
async function getJson(url, headers) {
|
||||
const res = await get(url, { headers });
|
||||
let body = "";
|
||||
for await (const chunk of res) body += chunk;
|
||||
return JSON.parse(body);
|
||||
}
|
||||
|
||||
// update.json and the API's release both become { version, notes, page, assets }.
|
||||
function fromManifest(m) {
|
||||
if (!parseVersion(m.version) || !Array.isArray(m.assets)) throw new Error("update.json is malformed");
|
||||
const base = `https://github.com/${REPO}/releases/download/v${String(m.version).replace(/^v/i, "")}/`;
|
||||
return { version: String(m.version).replace(/^v/i, ""), notes: String(m.notes || "").slice(0, 4000),
|
||||
page: m.page || RELEASES,
|
||||
// Assets always come from this repository's release, whatever the manifest says.
|
||||
assets: m.assets.map((a) => ({ name: String(a.name), url: base + encodeURIComponent(String(a.name)),
|
||||
size: a.size, digest: a.digest || null })) };
|
||||
}
|
||||
|
||||
function fromApi(r) {
|
||||
if (r.draft || r.prerelease) throw new Error("GitHub returned an unpublished release");
|
||||
return { version: String(r.tag_name || "").replace(/^v/i, ""), notes: String(r.body || "").slice(0, 4000),
|
||||
page: r.html_url || RELEASES,
|
||||
assets: (r.assets || []).map((a) => ({ name: a.name, url: a.browser_download_url, size: a.size,
|
||||
digest: a.digest || null })) };
|
||||
}
|
||||
|
||||
async function latestRelease() {
|
||||
try {
|
||||
return fromManifest(await getJson(MANIFEST));
|
||||
} catch (e) {
|
||||
if (!/HTTP 404/.test(e.message)) throw e; // releases before update.json existed
|
||||
}
|
||||
return fromApi(await getJson(LATEST, { accept: "application/vnd.github+json" }));
|
||||
}
|
||||
|
||||
async function download(asset, dest, onProgress) {
|
||||
const m = /^sha256:([0-9a-f]{64})$/.exec(asset.digest || "");
|
||||
if (!m) throw new Error(`GitHub has no SHA-256 for ${asset.name}, so it can't be checked`);
|
||||
const res = await get(asset.url, { timeout: 60000 });
|
||||
const total = +res.headers["content-length"] || asset.size || 0;
|
||||
const hash = crypto.createHash("sha256");
|
||||
// "wx": a new file only, never through an existing file or symlink at that path.
|
||||
const out = fs.createWriteStream(dest, { mode: 0o755, flags: "wx" });
|
||||
let done = 0;
|
||||
await new Promise((resolve, reject) => {
|
||||
res.on("data", (chunk) => { hash.update(chunk); done += chunk.length; onProgress && onProgress(done, total); });
|
||||
res.on("error", reject);
|
||||
out.on("error", reject);
|
||||
out.on("finish", resolve);
|
||||
res.pipe(out);
|
||||
});
|
||||
if (hash.digest("hex") !== m[1]) {
|
||||
fs.rmSync(dest, { force: true });
|
||||
throw new Error(`${asset.name} didn't match its SHA-256; nothing was changed`);
|
||||
}
|
||||
}
|
||||
|
||||
const run = (cmd, args) => new Promise((resolve, reject) =>
|
||||
execFile(cmd, args, { timeout: 120000 }, (err, stdout, stderr) => err ? reject(new Error((stderr || err.message).trim())) : resolve(stdout)));
|
||||
|
||||
// Waits for this process to exit, swaps the new bundle in (putting the old one
|
||||
// back if that fails), and reopens the app.
|
||||
const MAC_SWAP = `set -u
|
||||
pid="$1"; app="$2"; new="$3"; stage="$4"
|
||||
while kill -0 "$pid" 2>/dev/null; do sleep 0.2; done
|
||||
old="$stage/old.app"
|
||||
if mv "$app" "$old"; then
|
||||
if mv "$new" "$app"; then rm -rf "$old"; else mv "$old" "$app"; fi
|
||||
fi
|
||||
xattr -dr com.apple.quarantine "$app" 2>/dev/null
|
||||
rm -rf "$stage"
|
||||
open "$app"
|
||||
`;
|
||||
|
||||
async function applyMac(release, bundle, onProgress) {
|
||||
const asset = release.assets.find((a) => a.name === assetName("darwin", process.arch, "mac-zip"));
|
||||
if (!asset) throw new Error(`the release has no ${assetName("darwin", process.arch, "mac-zip")}`);
|
||||
// Staged beside the app, so the final move stays on one volume.
|
||||
const stage = fs.mkdtempSync(path.join(path.dirname(bundle), ".frame-control-update-"));
|
||||
try {
|
||||
const zip = path.join(stage, asset.name);
|
||||
await download(asset, zip, onProgress);
|
||||
await run("/usr/bin/ditto", ["-x", "-k", zip, stage]);
|
||||
fs.rmSync(zip, { force: true });
|
||||
const name = fs.readdirSync(stage).find((n) => n.endsWith(".app"));
|
||||
if (!name) throw new Error("the download has no app in it");
|
||||
const fresh = path.join(stage, name);
|
||||
const version = (await run("/usr/bin/plutil", ["-extract", "CFBundleShortVersionString", "raw",
|
||||
path.join(fresh, "Contents", "Info.plist")])).trim();
|
||||
if (version !== release.version) throw new Error(`the download is version ${version}, not ${release.version}`);
|
||||
const script = path.join(stage, "swap.sh");
|
||||
fs.writeFileSync(script, MAC_SWAP);
|
||||
return () => spawn("/bin/sh", [script, String(process.pid), bundle, fresh, stage],
|
||||
{ detached: true, stdio: "ignore" }).unref();
|
||||
} catch (e) {
|
||||
fs.rmSync(stage, { recursive: true, force: true });
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
async function applyNsis(release, onProgress) {
|
||||
const asset = release.assets.find((a) => a.name === assetName("win32", process.arch, "nsis"));
|
||||
if (!asset) throw new Error(`the release has no ${assetName("win32", process.arch, "nsis")}`);
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), "frame-control-update-"));
|
||||
const exe = path.join(dir, asset.name);
|
||||
await download(asset, exe, onProgress);
|
||||
// /S: silent, into the existing install. --force-run: open the app afterwards.
|
||||
return () => spawn(exe, ["--updated", "/S", "--force-run"], { detached: true, stdio: "ignore" }).unref();
|
||||
}
|
||||
|
||||
async function applyAppImage(release, appImage, onProgress) {
|
||||
const asset = release.assets.find((a) => a.name === assetName("linux", process.arch, "appimage"));
|
||||
if (!asset) throw new Error(`the release has no ${assetName("linux", process.arch, "appimage")}`);
|
||||
// A private folder beside the AppImage, so the final rename stays on one filesystem.
|
||||
const stage = fs.mkdtempSync(path.join(path.dirname(appImage), ".frame-control-update-"));
|
||||
try {
|
||||
const next = path.join(stage, asset.name);
|
||||
await download(asset, next, onProgress);
|
||||
fs.chmodSync(next, 0o755);
|
||||
fs.renameSync(next, appImage); // the running copy keeps its open file
|
||||
} finally {
|
||||
fs.rmSync(stage, { recursive: true, force: true });
|
||||
}
|
||||
// Without FUSE the AppImage runs extracted (--appimage-extract-and-run, which isn't passed
|
||||
// on to the app); a FUSE mount lives under /tmp/.mount_*. Keep the same mode on restart.
|
||||
const extracted = process.env.APPIMAGE_EXTRACT_AND_RUN === "1" || !process.execPath.includes("/.mount_");
|
||||
const env = { ...process.env, APPIMAGE: appImage, ...(extracted ? { APPIMAGE_EXTRACT_AND_RUN: "1" } : {}) };
|
||||
// Started only once this process has exited, or the new copy would lose the single-instance lock.
|
||||
return () => spawn("/bin/sh", ["-c", 'while kill -0 "$1" 2>/dev/null; do sleep 0.2; done; exec "$2"',
|
||||
"sh", String(process.pid), appImage], { detached: true, stdio: "ignore", env }).unref();
|
||||
}
|
||||
|
||||
// Downloads and prepares the update; returns a function that starts the swap,
|
||||
// to be called just before the app quits.
|
||||
async function prepare(release, how, onProgress, current) {
|
||||
if (!isNewer(release.version, current)) throw new Error(`${release.version} isn't newer than ${current}`);
|
||||
if (how.method === "mac-zip") return applyMac(release, how.bundle, onProgress);
|
||||
if (how.method === "nsis") return applyNsis(release, onProgress);
|
||||
if (how.method === "appimage") return applyAppImage(release, how.appImage, onProgress);
|
||||
throw new Error(how.why || "this copy can't update itself");
|
||||
}
|
||||
|
||||
function writable(dir) {
|
||||
try { fs.accessSync(dir, fs.constants.W_OK); return true; } catch { return false; }
|
||||
}
|
||||
|
||||
module.exports = { REPO, RELEASES, parseVersion, isNewer, assetName, updateMethod, macBundle, latestRelease,
|
||||
fromManifest, fromApi,
|
||||
download, prepare, writable };
|
||||
@@ -0,0 +1,24 @@
|
||||
<!doctype html>
|
||||
<!-- Control experiment, run on the Frame itself: how often does Chromium on
|
||||
gamescope put a canvas animation on screen, with no network or decoding
|
||||
involved? Draws every animation frame for ?s= seconds and writes the
|
||||
frame-gap histogram per second into the title, where xprop can read it. -->
|
||||
<html><head><meta charset="utf-8"><title>fmv-present</title></head>
|
||||
<body style="margin:0;background:#000"><canvas id="c" width="1280" height="720" style="width:100%;height:100%"></canvas>
|
||||
<script>
|
||||
const q = new URLSearchParams(location.search), secs = +q.get("s") || 15, tag = q.get("tag") || "";
|
||||
const ctx = document.getElementById("c").getContext("2d", { alpha: false, desynchronized: true });
|
||||
const perSec = []; let t0 = 0, last = 0, n = 0, gaps = [];
|
||||
function frame(t) {
|
||||
if (!t0) t0 = last = t;
|
||||
ctx.fillStyle = "#123"; ctx.fillRect(0, 0, 1280, 720);
|
||||
ctx.fillStyle = "#fff"; ctx.fillRect((n * 21) % 1280, 0, 20, 720); n++;
|
||||
const s = Math.floor((t - t0) / 1000);
|
||||
(perSec[s] = perSec[s] || []).push(t - last); last = t;
|
||||
if (t - t0 < secs * 1000) return requestAnimationFrame(frame);
|
||||
const out = perSec.map(g => g.length).join(",");
|
||||
document.title = `[${tag}] done fps/s=${out}`;
|
||||
}
|
||||
document.title = `[${tag}] running`;
|
||||
requestAnimationFrame(frame);
|
||||
</script></body></html>
|
||||
@@ -0,0 +1,32 @@
|
||||
<!doctype html>
|
||||
<!-- Benchmark content: a long page of text that scrolls itself at a steady
|
||||
speed, so every frame changes the way reading a real page does. -->
|
||||
<html lang="en"><head><meta charset="utf-8"><title>fmv-bench scroll</title>
|
||||
<style>
|
||||
body { font: 17px/1.55 -apple-system, "Helvetica Neue", sans-serif; margin: 0 auto; max-width: 900px; padding: 24px; color: #1d1d1f; background: #fff; }
|
||||
h2 { font-size: 22px; margin: 28px 0 8px; } code { background: #f2f2f5; padding: 1px 4px; border-radius: 3px; }
|
||||
</style></head><body><div id="doc"></div>
|
||||
<script>
|
||||
const words = "latency frame panel stream encoder decoder network display pixel window capture budget quality motion text sharp adaptive controller queue socket clock".split(" ");
|
||||
let seed = 7; const rnd = () => (seed = (seed * 16807) % 2147483647) / 2147483647;
|
||||
let html = "";
|
||||
for (let s = 0; s < 120; s++) {
|
||||
html += `<h2>Section ${s + 1}: ${words[s % words.length]} and ${words[(s * 7) % words.length]}</h2>`;
|
||||
for (let p = 0; p < 4; p++) {
|
||||
let para = [];
|
||||
for (let w = 0; w < 70; w++) para.push(rnd() < 0.05 ? `<code>${words[Math.floor(rnd() * words.length)]}()</code>` : words[Math.floor(rnd() * words.length)]);
|
||||
html += `<p>${para.join(" ")}.</p>`;
|
||||
}
|
||||
}
|
||||
document.getElementById("doc").innerHTML = html;
|
||||
// 240 points a second, whatever the display's refresh rate.
|
||||
// The title carries the page's own frame rate, so the source's rate is known.
|
||||
let last = performance.now(), frames = 0, since = last;
|
||||
function step(now) {
|
||||
const dy = (now - last) * 0.24; last = now;
|
||||
if (++frames && now - since >= 1000) { document.title = `fmv-bench scroll ${frames} fps`; frames = 0; since = now; }
|
||||
if (window.scrollY + innerHeight >= document.body.scrollHeight - 2) window.scrollTo(0, 0); else window.scrollBy(0, dy);
|
||||
requestAnimationFrame(step);
|
||||
}
|
||||
requestAnimationFrame(step);
|
||||
</script></body></html>
|
||||
@@ -0,0 +1,9 @@
|
||||
<!doctype html>
|
||||
<!-- Benchmark content: a plain text editor, focused, for typing tests. -->
|
||||
<html lang="en"><head><meta charset="utf-8"><title>fmv-bench type</title>
|
||||
<style>
|
||||
html, body { margin: 0; height: 100%; background: #fff; }
|
||||
textarea { box-sizing: border-box; width: 100%; height: 100%; border: 0; padding: 20px; outline: none; resize: none;
|
||||
font: 20px/1.5 ui-monospace, Menlo, monospace; color: #111; caret-color: transparent; } /* a blinking caret would pass for a reply to a key */
|
||||
</style></head><body><textarea id="t" autofocus spellcheck="false"></textarea>
|
||||
<script>document.getElementById("t").focus();</script></body></html>
|
||||
@@ -0,0 +1,677 @@
|
||||
{
|
||||
"label": "usb1",
|
||||
"date": "2026-09-28T21:16:03",
|
||||
"commit": "1d05f57",
|
||||
"config": {
|
||||
"quality": "balanced",
|
||||
"mode": "separate",
|
||||
"net": "none",
|
||||
"delay_ms": 0,
|
||||
"buffer_ms": 250,
|
||||
"host": "10.86.200.233",
|
||||
"ssh_opts": [],
|
||||
"encoder": "",
|
||||
"duration_s": 15.0,
|
||||
"browser_flags": []
|
||||
},
|
||||
"frame_build": "20260925.6191901",
|
||||
"mac": "26.5.2",
|
||||
"headset": "",
|
||||
"scenarios": [
|
||||
{
|
||||
"scenario": "test",
|
||||
"frames_sent": 911,
|
||||
"frames_drawn": 911,
|
||||
"frames_shown": 610,
|
||||
"duration_s": 15.2,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0,
|
||||
"n": 911
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 0.1,
|
||||
"n": 911
|
||||
},
|
||||
"encode": {
|
||||
"p50": 4.4,
|
||||
"p95": 5.0,
|
||||
"p99": 5.2,
|
||||
"n": 911
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.2,
|
||||
"n": 911
|
||||
},
|
||||
"network": {
|
||||
"p50": 1.1,
|
||||
"p95": 1.6,
|
||||
"p99": 2.1,
|
||||
"n": 911
|
||||
},
|
||||
"decode": {
|
||||
"p50": 1.2,
|
||||
"p95": 2.6,
|
||||
"p99": 3.9,
|
||||
"n": 911
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.4,
|
||||
"p95": 0.9,
|
||||
"p99": 1.5,
|
||||
"n": 911
|
||||
},
|
||||
"present": {
|
||||
"p50": 5.8,
|
||||
"p95": 13.6,
|
||||
"p99": 16.7,
|
||||
"n": 610
|
||||
},
|
||||
"content": {
|
||||
"p50": 7.3,
|
||||
"p95": 8.8,
|
||||
"p99": 10.0,
|
||||
"n": 911
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 13.6,
|
||||
"p95": 20.4,
|
||||
"p99": 24.9,
|
||||
"n": 610
|
||||
}
|
||||
},
|
||||
"fps": 59.9,
|
||||
"fps_shown": 40.1,
|
||||
"late_pct": 0.22,
|
||||
"stall_max": 32.3,
|
||||
"stalls_over_100ms": 0,
|
||||
"mbps": 0.48,
|
||||
"keyframes": 1,
|
||||
"size": "1280x720",
|
||||
"captured": 912,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"p50": 16.6,
|
||||
"p95": 24.3,
|
||||
"p99": 24.5,
|
||||
"n": 30
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"p50": 20.8,
|
||||
"p95": 31.8,
|
||||
"p99": 33.8,
|
||||
"n": 22
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"p50": 0.7,
|
||||
"p95": 1.0,
|
||||
"p99": 1.0,
|
||||
"n": 30
|
||||
},
|
||||
"mac": {
|
||||
"p50": 9.1,
|
||||
"p95": 15.0,
|
||||
"p99": 15.2,
|
||||
"n": 30
|
||||
},
|
||||
"back": {
|
||||
"p50": 7.2,
|
||||
"p95": 8.5,
|
||||
"p99": 12.7,
|
||||
"n": 30
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 30,
|
||||
"sent": 30,
|
||||
"seen": 30
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 60,
|
||||
"mbps": 0.5,
|
||||
"content_p50": 7.3,
|
||||
"content_p95": 8.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 60,
|
||||
"mbps": 0.5,
|
||||
"content_p50": 7.1,
|
||||
"content_p95": 8.9,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 59,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 7.2,
|
||||
"content_p95": 8.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 60,
|
||||
"mbps": 0.49,
|
||||
"content_p50": 7.1,
|
||||
"content_p95": 9.0,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 60,
|
||||
"mbps": 0.49,
|
||||
"content_p50": 6.9,
|
||||
"content_p95": 8.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 60,
|
||||
"mbps": 0.58,
|
||||
"content_p50": 7.3,
|
||||
"content_p95": 8.1,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 7.4,
|
||||
"content_p95": 8.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 7.4,
|
||||
"content_p95": 8.8,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 7.3,
|
||||
"content_p95": 8.9,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 7.3,
|
||||
"content_p95": 8.8,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 60,
|
||||
"mbps": 0.45,
|
||||
"content_p50": 7.3,
|
||||
"content_p95": 8.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 60,
|
||||
"mbps": 0.45,
|
||||
"content_p50": 7.4,
|
||||
"content_p95": 9.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 7.4,
|
||||
"content_p95": 8.4,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 7.6,
|
||||
"content_p95": 9.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 7.6,
|
||||
"content_p95": 8.5,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 12,
|
||||
"mbps": 0.09,
|
||||
"content_p50": 7.5,
|
||||
"content_p95": 9.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 7.3,
|
||||
"content_p95": 8.8,
|
||||
"input_p50": 16.6,
|
||||
"input_p95": 24.3,
|
||||
"grades": {
|
||||
"input_replies": "local",
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"input_p50": "local",
|
||||
"input_p95": "local",
|
||||
"fps": "local",
|
||||
"late_pct": "local",
|
||||
"stall_max": "local"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 0.917,
|
||||
"ceiling": 5529600,
|
||||
"fps": 60,
|
||||
"inFlight": 1,
|
||||
"scale": 1,
|
||||
"slack": 41.666,
|
||||
"target": 5529600,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [],
|
||||
"source_fps": 60.0,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 16.7,
|
||||
"frame_viewer_pct": 48.5,
|
||||
"frame_tailscaled_pct": 0.5,
|
||||
"frame_sshd_pct": 1.1,
|
||||
"frame_gamescope_pct": 12.0,
|
||||
"frame_total_pct": 39.3
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 137 Hz",
|
||||
"show_s": 1.15,
|
||||
"panel": "valve.steam.desktopgame.2001639889",
|
||||
"src": "test"
|
||||
},
|
||||
{
|
||||
"scenario": "scroll",
|
||||
"frames_sent": 848,
|
||||
"frames_drawn": 848,
|
||||
"frames_shown": 617,
|
||||
"duration_s": 15.1,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": -3.9,
|
||||
"p95": 4.2,
|
||||
"p99": 7.3,
|
||||
"n": 848
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 0.1,
|
||||
"n": 848
|
||||
},
|
||||
"encode": {
|
||||
"p50": 6.7,
|
||||
"p95": 9.7,
|
||||
"p99": 17.7,
|
||||
"n": 848
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.4,
|
||||
"n": 848
|
||||
},
|
||||
"network": {
|
||||
"p50": 1.7,
|
||||
"p95": 2.7,
|
||||
"p99": 5.7,
|
||||
"n": 848
|
||||
},
|
||||
"decode": {
|
||||
"p50": 5.9,
|
||||
"p95": 10.3,
|
||||
"p99": 13.1,
|
||||
"n": 848
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.7,
|
||||
"p95": 1.6,
|
||||
"p99": 2.2,
|
||||
"n": 848
|
||||
},
|
||||
"present": {
|
||||
"p50": 4.4,
|
||||
"p95": 15.8,
|
||||
"p99": 20.6,
|
||||
"n": 617
|
||||
},
|
||||
"content": {
|
||||
"p50": 15.7,
|
||||
"p95": 23.0,
|
||||
"p99": 39.3,
|
||||
"n": 848
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 21.3,
|
||||
"p95": 34.1,
|
||||
"p99": 45.6,
|
||||
"n": 617
|
||||
}
|
||||
},
|
||||
"fps": 56.1,
|
||||
"fps_shown": 40.7,
|
||||
"late_pct": 3.3,
|
||||
"stall_max": 238.5,
|
||||
"stalls_over_100ms": 1,
|
||||
"mbps": 9.01,
|
||||
"keyframes": 1,
|
||||
"size": "1920x1290",
|
||||
"captured": 857,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"n": 0
|
||||
},
|
||||
"mac": {
|
||||
"n": 0
|
||||
},
|
||||
"back": {
|
||||
"n": 0
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 0,
|
||||
"sent": 0,
|
||||
"seen": 0
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 44,
|
||||
"mbps": 8.13,
|
||||
"content_p50": 15.7,
|
||||
"content_p95": 35.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 56,
|
||||
"mbps": 7.04,
|
||||
"content_p50": 16.3,
|
||||
"content_p95": 47.3,
|
||||
"bitrate": 7430400,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 57,
|
||||
"mbps": 8.58,
|
||||
"content_p50": 16.0,
|
||||
"content_p95": 22.6,
|
||||
"bitrate": 10055362,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 56,
|
||||
"mbps": 9.29,
|
||||
"content_p50": 16.4,
|
||||
"content_p95": 21.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 58,
|
||||
"mbps": 9.01,
|
||||
"content_p50": 15.6,
|
||||
"content_p95": 21.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 57,
|
||||
"mbps": 9.13,
|
||||
"content_p50": 15.9,
|
||||
"content_p95": 21.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 56,
|
||||
"mbps": 11.63,
|
||||
"content_p50": 16.1,
|
||||
"content_p95": 22.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 57,
|
||||
"mbps": 8.7,
|
||||
"content_p50": 16.3,
|
||||
"content_p95": 22.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 57,
|
||||
"mbps": 9.24,
|
||||
"content_p50": 15.7,
|
||||
"content_p95": 21.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 57,
|
||||
"mbps": 9.21,
|
||||
"content_p50": 15.0,
|
||||
"content_p95": 20.5,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 56,
|
||||
"mbps": 8.55,
|
||||
"content_p50": 15.7,
|
||||
"content_p95": 20.6,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 58,
|
||||
"mbps": 9.51,
|
||||
"content_p50": 14.7,
|
||||
"content_p95": 19.9,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 57,
|
||||
"mbps": 8.75,
|
||||
"content_p50": 16.0,
|
||||
"content_p95": 20.4,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 58,
|
||||
"mbps": 9.25,
|
||||
"content_p50": 15.0,
|
||||
"content_p95": 20.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 57,
|
||||
"mbps": 9.13,
|
||||
"content_p50": 15.3,
|
||||
"content_p95": 21.0,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 7,
|
||||
"mbps": 1.17,
|
||||
"content_p50": 14.2,
|
||||
"content_p95": 15.3,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 15.7,
|
||||
"content_p95": 23.0,
|
||||
"input_p50": null,
|
||||
"input_p95": null,
|
||||
"grades": {
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"fps": "acceptable",
|
||||
"late_pct": "acceptable",
|
||||
"stall_max": "acceptable"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 1.25,
|
||||
"ceiling": 14860800,
|
||||
"fps": 60,
|
||||
"inFlight": 0,
|
||||
"scale": 1,
|
||||
"slack": 41.666,
|
||||
"target": 14860800,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"t": 1.06,
|
||||
"e": "down to 7430 kbit/s: queue 35 ms, held 4, oldest 32 ms, base 1 ms, sent 6282 got 4914 wants 11892"
|
||||
}
|
||||
],
|
||||
"source_fps": 56.8,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 8.8,
|
||||
"frame_viewer_pct": 74.8,
|
||||
"frame_tailscaled_pct": 0.3,
|
||||
"frame_sshd_pct": 1.2,
|
||||
"frame_gamescope_pct": 10.5,
|
||||
"frame_total_pct": 39.0
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 137 Hz",
|
||||
"show_s": 1.16,
|
||||
"panel": "valve.steam.desktopgame.2001968301",
|
||||
"src": "separate:7914"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,690 @@
|
||||
{
|
||||
"label": "usb2",
|
||||
"date": "2026-09-28T21:17:48",
|
||||
"commit": "1d05f57",
|
||||
"config": {
|
||||
"quality": "balanced",
|
||||
"mode": "separate",
|
||||
"net": "none",
|
||||
"delay_ms": 0,
|
||||
"buffer_ms": 250,
|
||||
"host": "10.86.200.233",
|
||||
"ssh_opts": [],
|
||||
"encoder": "",
|
||||
"duration_s": 15.0,
|
||||
"browser_flags": []
|
||||
},
|
||||
"frame_build": "20260925.6191901",
|
||||
"mac": "26.5.2",
|
||||
"headset": "",
|
||||
"scenarios": [
|
||||
{
|
||||
"scenario": "test",
|
||||
"frames_sent": 913,
|
||||
"frames_drawn": 913,
|
||||
"frames_shown": 856,
|
||||
"duration_s": 15.2,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0,
|
||||
"n": 913
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 0.1,
|
||||
"n": 913
|
||||
},
|
||||
"encode": {
|
||||
"p50": 4.3,
|
||||
"p95": 4.9,
|
||||
"p99": 5.2,
|
||||
"n": 913
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.2,
|
||||
"n": 913
|
||||
},
|
||||
"network": {
|
||||
"p50": 0.9,
|
||||
"p95": 1.2,
|
||||
"p99": 1.7,
|
||||
"n": 913
|
||||
},
|
||||
"decode": {
|
||||
"p50": 1.1,
|
||||
"p95": 2.5,
|
||||
"p99": 3.8,
|
||||
"n": 913
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.3,
|
||||
"p95": 0.8,
|
||||
"p99": 1.2,
|
||||
"n": 913
|
||||
},
|
||||
"present": {
|
||||
"p50": 2.6,
|
||||
"p95": 8.6,
|
||||
"p99": 15.9,
|
||||
"n": 856
|
||||
},
|
||||
"content": {
|
||||
"p50": 6.9,
|
||||
"p95": 8.4,
|
||||
"p99": 9.6,
|
||||
"n": 913
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 9.7,
|
||||
"p95": 15.8,
|
||||
"p99": 23.1,
|
||||
"n": 856
|
||||
}
|
||||
},
|
||||
"fps": 60.0,
|
||||
"fps_shown": 56.3,
|
||||
"late_pct": 0.0,
|
||||
"stall_max": 22.0,
|
||||
"stalls_over_100ms": 0,
|
||||
"mbps": 0.57,
|
||||
"keyframes": 1,
|
||||
"size": "1280x720",
|
||||
"captured": 913,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"p50": 16.9,
|
||||
"p95": 24.1,
|
||||
"p99": 26.1,
|
||||
"n": 33
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"p50": 19.7,
|
||||
"p95": 26.6,
|
||||
"p99": 29.2,
|
||||
"n": 31
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"p50": 0.7,
|
||||
"p95": 2.5,
|
||||
"p99": 5.7,
|
||||
"n": 33
|
||||
},
|
||||
"mac": {
|
||||
"p50": 8.1,
|
||||
"p95": 15.4,
|
||||
"p99": 17.2,
|
||||
"n": 33
|
||||
},
|
||||
"back": {
|
||||
"p50": 7.3,
|
||||
"p95": 8.3,
|
||||
"p99": 8.9,
|
||||
"n": 33
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 30,
|
||||
"sent": 33,
|
||||
"seen": 33
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 60,
|
||||
"mbps": 0.49,
|
||||
"content_p50": 7.1,
|
||||
"content_p95": 8.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 7.0,
|
||||
"content_p95": 8.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 6.9,
|
||||
"content_p95": 8.4,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 7.0,
|
||||
"content_p95": 9.1,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 60,
|
||||
"mbps": 0.5,
|
||||
"content_p50": 6.8,
|
||||
"content_p95": 8.5,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 60,
|
||||
"mbps": 0.67,
|
||||
"content_p50": 7.0,
|
||||
"content_p95": 8.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 60,
|
||||
"mbps": 0.59,
|
||||
"content_p50": 7.0,
|
||||
"content_p95": 8.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 60,
|
||||
"mbps": 0.63,
|
||||
"content_p50": 6.7,
|
||||
"content_p95": 7.5,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 60,
|
||||
"mbps": 0.62,
|
||||
"content_p50": 6.9,
|
||||
"content_p95": 8.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 60,
|
||||
"mbps": 0.64,
|
||||
"content_p50": 6.8,
|
||||
"content_p95": 8.0,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 60,
|
||||
"mbps": 0.67,
|
||||
"content_p50": 6.8,
|
||||
"content_p95": 8.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 60,
|
||||
"mbps": 0.62,
|
||||
"content_p50": 6.7,
|
||||
"content_p95": 7.9,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 60,
|
||||
"mbps": 0.59,
|
||||
"content_p50": 6.8,
|
||||
"content_p95": 7.9,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 60,
|
||||
"mbps": 0.57,
|
||||
"content_p50": 6.7,
|
||||
"content_p95": 8.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 60,
|
||||
"mbps": 0.58,
|
||||
"content_p50": 7.0,
|
||||
"content_p95": 8.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 13,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 6.4,
|
||||
"content_p95": 7.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 6.9,
|
||||
"content_p95": 8.4,
|
||||
"input_p50": 16.9,
|
||||
"input_p95": 24.1,
|
||||
"grades": {
|
||||
"input_replies": "local",
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"input_p50": "local",
|
||||
"input_p95": "local",
|
||||
"fps": "local",
|
||||
"late_pct": "local",
|
||||
"stall_max": "local"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 0.709,
|
||||
"ceiling": 5529600,
|
||||
"fps": 60,
|
||||
"inFlight": 0,
|
||||
"scale": 1,
|
||||
"slack": 41.666,
|
||||
"target": 5529600,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [],
|
||||
"source_fps": 60.1,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 17.9,
|
||||
"frame_viewer_pct": 51.9,
|
||||
"frame_tailscaled_pct": 0.2,
|
||||
"frame_sshd_pct": 1.1,
|
||||
"frame_gamescope_pct": 9.6,
|
||||
"frame_total_pct": 27.9
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 134 Hz",
|
||||
"show_s": 1.15,
|
||||
"panel": "valve.steam.desktopgame.2001639889",
|
||||
"src": "test"
|
||||
},
|
||||
{
|
||||
"scenario": "scroll",
|
||||
"frames_sent": 868,
|
||||
"frames_drawn": 868,
|
||||
"frames_shown": 868,
|
||||
"duration_s": 15.2,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": -4.2,
|
||||
"p95": 0.5,
|
||||
"p99": 6.7,
|
||||
"n": 868
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 0.2,
|
||||
"n": 868
|
||||
},
|
||||
"encode": {
|
||||
"p50": 6.8,
|
||||
"p95": 10.2,
|
||||
"p99": 17.9,
|
||||
"n": 868
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.4,
|
||||
"n": 868
|
||||
},
|
||||
"network": {
|
||||
"p50": 1.5,
|
||||
"p95": 2.3,
|
||||
"p99": 7.6,
|
||||
"n": 868
|
||||
},
|
||||
"decode": {
|
||||
"p50": 5.9,
|
||||
"p95": 10.2,
|
||||
"p99": 20.3,
|
||||
"n": 868
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.7,
|
||||
"p95": 1.4,
|
||||
"p99": 2.1,
|
||||
"n": 868
|
||||
},
|
||||
"present": {
|
||||
"p50": 0.9,
|
||||
"p95": 6.1,
|
||||
"p99": 6.8,
|
||||
"n": 868
|
||||
},
|
||||
"content": {
|
||||
"p50": 15.3,
|
||||
"p95": 23.5,
|
||||
"p99": 53.1,
|
||||
"n": 868
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 17.7,
|
||||
"p95": 25.6,
|
||||
"p99": 57.5,
|
||||
"n": 868
|
||||
}
|
||||
},
|
||||
"fps": 57.2,
|
||||
"fps_shown": 57.2,
|
||||
"late_pct": 3.23,
|
||||
"stall_max": 108.0,
|
||||
"stalls_over_100ms": 1,
|
||||
"mbps": 9.84,
|
||||
"keyframes": 1,
|
||||
"size": "1920x1290",
|
||||
"captured": 870,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"p50": 24.2,
|
||||
"p95": 36.3,
|
||||
"p99": 36.3,
|
||||
"n": 6
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"p50": 24.8,
|
||||
"p95": 38.5,
|
||||
"p99": 38.5,
|
||||
"n": 6
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"p50": 2.0,
|
||||
"p95": 5.4,
|
||||
"p99": 5.4,
|
||||
"n": 6
|
||||
},
|
||||
"mac": {
|
||||
"p50": 7.5,
|
||||
"p95": 15.8,
|
||||
"p99": 15.8,
|
||||
"n": 6
|
||||
},
|
||||
"back": {
|
||||
"p50": 15.3,
|
||||
"p95": 16.1,
|
||||
"p99": 16.1,
|
||||
"n": 6
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 0,
|
||||
"sent": 6,
|
||||
"seen": 6
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 53,
|
||||
"mbps": 9.21,
|
||||
"content_p50": 16.1,
|
||||
"content_p95": 71.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 57,
|
||||
"mbps": 9.64,
|
||||
"content_p50": 15.9,
|
||||
"content_p95": 46.8,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 58,
|
||||
"mbps": 8.99,
|
||||
"content_p50": 14.8,
|
||||
"content_p95": 19.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 57,
|
||||
"mbps": 9.72,
|
||||
"content_p50": 15.2,
|
||||
"content_p95": 22.3,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 58,
|
||||
"mbps": 9.64,
|
||||
"content_p50": 16.1,
|
||||
"content_p95": 23.5,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 57,
|
||||
"mbps": 11.27,
|
||||
"content_p50": 15.6,
|
||||
"content_p95": 20.9,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 58,
|
||||
"mbps": 10.44,
|
||||
"content_p50": 15.1,
|
||||
"content_p95": 19.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 57,
|
||||
"mbps": 9.31,
|
||||
"content_p50": 15.0,
|
||||
"content_p95": 19.9,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 58,
|
||||
"mbps": 11.69,
|
||||
"content_p50": 15.9,
|
||||
"content_p95": 22.0,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 58,
|
||||
"mbps": 10.31,
|
||||
"content_p50": 14.5,
|
||||
"content_p95": 20.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 57,
|
||||
"mbps": 10.47,
|
||||
"content_p50": 15.2,
|
||||
"content_p95": 20.6,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 58,
|
||||
"mbps": 9.44,
|
||||
"content_p50": 15.3,
|
||||
"content_p95": 20.5,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 57,
|
||||
"mbps": 9.17,
|
||||
"content_p50": 15.9,
|
||||
"content_p95": 18.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 58,
|
||||
"mbps": 8.77,
|
||||
"content_p50": 15.1,
|
||||
"content_p95": 18.6,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 58,
|
||||
"mbps": 9.72,
|
||||
"content_p50": 15.2,
|
||||
"content_p95": 19.8,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 9,
|
||||
"mbps": 1.33,
|
||||
"content_p50": 14.8,
|
||||
"content_p95": 17.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 15.3,
|
||||
"content_p95": 23.5,
|
||||
"input_p50": 24.2,
|
||||
"input_p95": 36.3,
|
||||
"grades": {
|
||||
"input_replies": "local",
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"input_p50": "local",
|
||||
"input_p95": "local",
|
||||
"fps": "acceptable",
|
||||
"late_pct": "acceptable",
|
||||
"stall_max": "acceptable"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 1.375,
|
||||
"ceiling": 14860800,
|
||||
"fps": 60,
|
||||
"inFlight": 1,
|
||||
"scale": 1,
|
||||
"slack": 41.666,
|
||||
"target": 14860800,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [],
|
||||
"source_fps": 57.2,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 11.4,
|
||||
"frame_viewer_pct": 83.5,
|
||||
"frame_tailscaled_pct": 0.3,
|
||||
"frame_sshd_pct": 1.5,
|
||||
"frame_gamescope_pct": 9.7,
|
||||
"frame_total_pct": 32.5
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz",
|
||||
"show_s": 1.15,
|
||||
"panel": "valve.steam.desktopgame.2001786096",
|
||||
"src": "separate:8187"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,682 @@
|
||||
{
|
||||
"label": "wifi1",
|
||||
"date": "2026-09-28T21:16:56",
|
||||
"commit": "1d05f57",
|
||||
"config": {
|
||||
"quality": "balanced",
|
||||
"mode": "separate",
|
||||
"net": "none",
|
||||
"delay_ms": 0,
|
||||
"buffer_ms": 250,
|
||||
"host": "frame",
|
||||
"ssh_opts": [],
|
||||
"encoder": "",
|
||||
"duration_s": 15.0,
|
||||
"browser_flags": []
|
||||
},
|
||||
"frame_build": "20260925.6191901",
|
||||
"mac": "26.5.2",
|
||||
"headset": "",
|
||||
"scenarios": [
|
||||
{
|
||||
"scenario": "test",
|
||||
"frames_sent": 912,
|
||||
"frames_drawn": 912,
|
||||
"frames_shown": 611,
|
||||
"duration_s": 15.2,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0,
|
||||
"n": 912
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 0.1,
|
||||
"n": 912
|
||||
},
|
||||
"encode": {
|
||||
"p50": 4.3,
|
||||
"p95": 4.9,
|
||||
"p99": 5.1,
|
||||
"n": 912
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.2,
|
||||
"n": 912
|
||||
},
|
||||
"network": {
|
||||
"p50": 4.0,
|
||||
"p95": 5.3,
|
||||
"p99": 7.8,
|
||||
"n": 912
|
||||
},
|
||||
"decode": {
|
||||
"p50": 1.1,
|
||||
"p95": 2.8,
|
||||
"p99": 4.0,
|
||||
"n": 912
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.4,
|
||||
"p95": 0.8,
|
||||
"p99": 1.2,
|
||||
"n": 912
|
||||
},
|
||||
"present": {
|
||||
"p50": 5.9,
|
||||
"p95": 14.0,
|
||||
"p99": 18.5,
|
||||
"n": 611
|
||||
},
|
||||
"content": {
|
||||
"p50": 10.1,
|
||||
"p95": 12.3,
|
||||
"p99": 13.9,
|
||||
"n": 912
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 16.7,
|
||||
"p95": 23.6,
|
||||
"p99": 28.3,
|
||||
"n": 611
|
||||
}
|
||||
},
|
||||
"fps": 60.0,
|
||||
"fps_shown": 40.1,
|
||||
"late_pct": 0.11,
|
||||
"stall_max": 25.8,
|
||||
"stalls_over_100ms": 0,
|
||||
"mbps": 0.48,
|
||||
"keyframes": 1,
|
||||
"size": "1280x720",
|
||||
"captured": 912,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"p50": 26.4,
|
||||
"p95": 34.2,
|
||||
"p99": 53.5,
|
||||
"n": 30
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"p50": 28.6,
|
||||
"p95": 41.5,
|
||||
"p99": 55.9,
|
||||
"n": 18
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"p50": 8.9,
|
||||
"p95": 20.4,
|
||||
"p99": 28.0,
|
||||
"n": 30
|
||||
},
|
||||
"mac": {
|
||||
"p50": 5.1,
|
||||
"p95": 14.5,
|
||||
"p99": 16.2,
|
||||
"n": 30
|
||||
},
|
||||
"back": {
|
||||
"p50": 10.1,
|
||||
"p95": 13.7,
|
||||
"p99": 13.8,
|
||||
"n": 30
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 30,
|
||||
"sent": 30,
|
||||
"seen": 30
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 60,
|
||||
"mbps": 0.51,
|
||||
"content_p50": 10.3,
|
||||
"content_p95": 11.9,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.5,
|
||||
"content_p95": 12.1,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 60,
|
||||
"mbps": 0.49,
|
||||
"content_p50": 10.1,
|
||||
"content_p95": 12.8,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 9.9,
|
||||
"content_p95": 11.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 9.7,
|
||||
"content_p95": 11.8,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 60,
|
||||
"mbps": 0.59,
|
||||
"content_p50": 10.2,
|
||||
"content_p95": 11.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.2,
|
||||
"content_p95": 12.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 9.7,
|
||||
"content_p95": 12.1,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.4,
|
||||
"content_p95": 13.1,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 9.9,
|
||||
"content_p95": 12.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 9.8,
|
||||
"content_p95": 12.4,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 10.3,
|
||||
"content_p95": 12.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 60,
|
||||
"mbps": 0.49,
|
||||
"content_p50": 10.2,
|
||||
"content_p95": 12.0,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 10.0,
|
||||
"content_p95": 11.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 10.2,
|
||||
"content_p95": 12.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 12,
|
||||
"mbps": 0.1,
|
||||
"content_p50": 10.7,
|
||||
"content_p95": 11.4,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 10.1,
|
||||
"content_p95": 12.3,
|
||||
"input_p50": 26.4,
|
||||
"input_p95": 34.2,
|
||||
"grades": {
|
||||
"input_replies": "local",
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"input_p50": "local",
|
||||
"input_p95": "local",
|
||||
"fps": "local",
|
||||
"late_pct": "local",
|
||||
"stall_max": "local"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 4.958,
|
||||
"ceiling": 5529600,
|
||||
"fps": 60,
|
||||
"inFlight": 1,
|
||||
"scale": 1,
|
||||
"slack": 47.417,
|
||||
"target": 2764800,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"t": 16.58,
|
||||
"e": "down to 2764 kbit/s: queue 19 ms, held 6, oldest 210 ms, base 4 ms, sent 322 got 265 wants 459"
|
||||
}
|
||||
],
|
||||
"source_fps": 60.0,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 15.7,
|
||||
"frame_viewer_pct": 42.5,
|
||||
"frame_tailscaled_pct": 5.5,
|
||||
"frame_sshd_pct": 0.7,
|
||||
"frame_gamescope_pct": 8.7,
|
||||
"frame_total_pct": 26.1
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 136 Hz",
|
||||
"show_s": 1.26,
|
||||
"panel": "valve.steam.desktopgame.2001639889",
|
||||
"src": "test"
|
||||
},
|
||||
{
|
||||
"scenario": "scroll",
|
||||
"frames_sent": 852,
|
||||
"frames_drawn": 852,
|
||||
"frames_shown": 614,
|
||||
"duration_s": 15.2,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": -4.4,
|
||||
"p95": 2.6,
|
||||
"p99": 4.2,
|
||||
"n": 852
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 0.1,
|
||||
"n": 852
|
||||
},
|
||||
"encode": {
|
||||
"p50": 6.7,
|
||||
"p95": 9.8,
|
||||
"p99": 16.7,
|
||||
"n": 852
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.2,
|
||||
"n": 852
|
||||
},
|
||||
"network": {
|
||||
"p50": 5.3,
|
||||
"p95": 13.1,
|
||||
"p99": 28.2,
|
||||
"n": 852
|
||||
},
|
||||
"decode": {
|
||||
"p50": 6.0,
|
||||
"p95": 10.9,
|
||||
"p99": 19.0,
|
||||
"n": 852
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.7,
|
||||
"p95": 1.4,
|
||||
"p99": 1.8,
|
||||
"n": 852
|
||||
},
|
||||
"present": {
|
||||
"p50": 4.1,
|
||||
"p95": 16.2,
|
||||
"p99": 20.6,
|
||||
"n": 614
|
||||
},
|
||||
"content": {
|
||||
"p50": 19.5,
|
||||
"p95": 31.4,
|
||||
"p99": 64.5,
|
||||
"n": 852
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 25.4,
|
||||
"p95": 38.8,
|
||||
"p99": 76.7,
|
||||
"n": 614
|
||||
}
|
||||
},
|
||||
"fps": 56.2,
|
||||
"fps_shown": 40.5,
|
||||
"late_pct": 4.69,
|
||||
"stall_max": 253.7,
|
||||
"stalls_over_100ms": 2,
|
||||
"mbps": 9.02,
|
||||
"keyframes": 1,
|
||||
"size": "1920x1290",
|
||||
"captured": 866,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"n": 0
|
||||
},
|
||||
"mac": {
|
||||
"n": 0
|
||||
},
|
||||
"back": {
|
||||
"n": 0
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 0,
|
||||
"sent": 0,
|
||||
"seen": 0
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 45,
|
||||
"mbps": 7.77,
|
||||
"content_p50": 19.0,
|
||||
"content_p95": 163.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 51,
|
||||
"mbps": 6.54,
|
||||
"content_p50": 19.4,
|
||||
"content_p95": 107.0,
|
||||
"bitrate": 7430400,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 57,
|
||||
"mbps": 8.64,
|
||||
"content_p50": 19.9,
|
||||
"content_p95": 31.8,
|
||||
"bitrate": 10055362,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 58,
|
||||
"mbps": 9.46,
|
||||
"content_p50": 19.1,
|
||||
"content_p95": 24.9,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 58,
|
||||
"mbps": 9.11,
|
||||
"content_p50": 19.4,
|
||||
"content_p95": 22.9,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 57,
|
||||
"mbps": 9.18,
|
||||
"content_p50": 19.9,
|
||||
"content_p95": 25.3,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 58,
|
||||
"mbps": 11.64,
|
||||
"content_p50": 19.8,
|
||||
"content_p95": 49.6,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 57,
|
||||
"mbps": 9.12,
|
||||
"content_p50": 20.5,
|
||||
"content_p95": 31.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 58,
|
||||
"mbps": 9.29,
|
||||
"content_p50": 19.5,
|
||||
"content_p95": 25.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 57,
|
||||
"mbps": 9.43,
|
||||
"content_p50": 19.8,
|
||||
"content_p95": 49.6,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 58,
|
||||
"mbps": 8.59,
|
||||
"content_p50": 19.2,
|
||||
"content_p95": 24.8,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 57,
|
||||
"mbps": 9.42,
|
||||
"content_p50": 19.9,
|
||||
"content_p95": 27.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 58,
|
||||
"mbps": 9.0,
|
||||
"content_p50": 20.6,
|
||||
"content_p95": 31.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 57,
|
||||
"mbps": 8.84,
|
||||
"content_p50": 19.6,
|
||||
"content_p95": 25.3,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 57,
|
||||
"mbps": 9.17,
|
||||
"content_p50": 19.3,
|
||||
"content_p95": 27.8,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 9,
|
||||
"mbps": 1.52,
|
||||
"content_p50": 19.3,
|
||||
"content_p95": 30.0,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 19.5,
|
||||
"content_p95": 31.4,
|
||||
"input_p50": null,
|
||||
"input_p95": null,
|
||||
"grades": {
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"fps": "acceptable",
|
||||
"late_pct": "acceptable",
|
||||
"stall_max": "bad"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 6.959,
|
||||
"ceiling": 14860800,
|
||||
"fps": 60,
|
||||
"inFlight": 2,
|
||||
"scale": 1,
|
||||
"slack": 43.54,
|
||||
"target": 14860800,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"t": 1.09,
|
||||
"e": "down to 7430 kbit/s: queue 131 ms, held 3, oldest 231 ms, base 6 ms, sent 4272 got 3724 wants 9321"
|
||||
}
|
||||
],
|
||||
"source_fps": 57.0,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 8.5,
|
||||
"frame_viewer_pct": 71.7,
|
||||
"frame_tailscaled_pct": 8.4,
|
||||
"frame_sshd_pct": 0.9,
|
||||
"frame_gamescope_pct": 8.4,
|
||||
"frame_total_pct": 30.1
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz",
|
||||
"show_s": 1.25,
|
||||
"panel": "valve.steam.desktopgame.2001195625",
|
||||
"src": "separate:8050"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,682 @@
|
||||
{
|
||||
"label": "wifi2",
|
||||
"date": "2026-09-28T21:18:40",
|
||||
"commit": "1d05f57",
|
||||
"config": {
|
||||
"quality": "balanced",
|
||||
"mode": "separate",
|
||||
"net": "none",
|
||||
"delay_ms": 0,
|
||||
"buffer_ms": 250,
|
||||
"host": "frame",
|
||||
"ssh_opts": [],
|
||||
"encoder": "",
|
||||
"duration_s": 15.0,
|
||||
"browser_flags": []
|
||||
},
|
||||
"frame_build": "20260925.6191901",
|
||||
"mac": "26.5.2",
|
||||
"headset": "",
|
||||
"scenarios": [
|
||||
{
|
||||
"scenario": "test",
|
||||
"frames_sent": 902,
|
||||
"frames_drawn": 902,
|
||||
"frames_shown": 604,
|
||||
"duration_s": 15.2,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0,
|
||||
"n": 902
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 0.1,
|
||||
"n": 902
|
||||
},
|
||||
"encode": {
|
||||
"p50": 4.3,
|
||||
"p95": 5.0,
|
||||
"p99": 5.3,
|
||||
"n": 902
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.3,
|
||||
"n": 902
|
||||
},
|
||||
"network": {
|
||||
"p50": 3.8,
|
||||
"p95": 5.3,
|
||||
"p99": 8.0,
|
||||
"n": 902
|
||||
},
|
||||
"decode": {
|
||||
"p50": 1.1,
|
||||
"p95": 2.8,
|
||||
"p99": 3.8,
|
||||
"n": 902
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.3,
|
||||
"p95": 0.8,
|
||||
"p99": 1.2,
|
||||
"n": 902
|
||||
},
|
||||
"present": {
|
||||
"p50": 5.9,
|
||||
"p95": 17.0,
|
||||
"p99": 18.6,
|
||||
"n": 604
|
||||
},
|
||||
"content": {
|
||||
"p50": 9.8,
|
||||
"p95": 12.1,
|
||||
"p99": 14.4,
|
||||
"n": 902
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 15.2,
|
||||
"p95": 27.0,
|
||||
"p99": 28.4,
|
||||
"n": 604
|
||||
}
|
||||
},
|
||||
"fps": 59.3,
|
||||
"fps_shown": 39.7,
|
||||
"late_pct": 0.55,
|
||||
"stall_max": 187.5,
|
||||
"stalls_over_100ms": 1,
|
||||
"mbps": 0.47,
|
||||
"keyframes": 1,
|
||||
"size": "1280x720",
|
||||
"captured": 913,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"p50": 27.8,
|
||||
"p95": 34.9,
|
||||
"p99": 206.9,
|
||||
"n": 30
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"p50": 35.1,
|
||||
"p95": 52.6,
|
||||
"p99": 207.2,
|
||||
"n": 24
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"p50": 9.8,
|
||||
"p95": 22.5,
|
||||
"p99": 184.8,
|
||||
"n": 30
|
||||
},
|
||||
"mac": {
|
||||
"p50": 5.3,
|
||||
"p95": 14.4,
|
||||
"p99": 19.6,
|
||||
"n": 30
|
||||
},
|
||||
"back": {
|
||||
"p50": 9.8,
|
||||
"p95": 13.4,
|
||||
"p99": 13.9,
|
||||
"n": 30
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 30,
|
||||
"sent": 30,
|
||||
"seen": 30
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 9.3,
|
||||
"content_p95": 11.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 50,
|
||||
"mbps": 0.39,
|
||||
"content_p50": 9.1,
|
||||
"content_p95": 14.4,
|
||||
"bitrate": 2764800,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 9.2,
|
||||
"content_p95": 11.8,
|
||||
"bitrate": 2764800,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 9.4,
|
||||
"content_p95": 10.5,
|
||||
"bitrate": 2764800,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 9.5,
|
||||
"content_p95": 12.5,
|
||||
"bitrate": 3091280,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.0,
|
||||
"content_p95": 12.8,
|
||||
"bitrate": 4757991,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 60,
|
||||
"mbps": 0.57,
|
||||
"content_p50": 9.5,
|
||||
"content_p95": 11.7,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 9.8,
|
||||
"content_p95": 11.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.2,
|
||||
"content_p95": 12.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.0,
|
||||
"content_p95": 13.2,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.2,
|
||||
"content_p95": 12.5,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 60,
|
||||
"mbps": 0.45,
|
||||
"content_p50": 9.9,
|
||||
"content_p95": 11.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 60,
|
||||
"mbps": 0.46,
|
||||
"content_p50": 10.0,
|
||||
"content_p95": 11.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 60,
|
||||
"mbps": 0.47,
|
||||
"content_p50": 10.3,
|
||||
"content_p95": 12.3,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 60,
|
||||
"mbps": 0.48,
|
||||
"content_p50": 9.8,
|
||||
"content_p95": 11.6,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 12,
|
||||
"mbps": 0.1,
|
||||
"content_p50": 9.3,
|
||||
"content_p95": 11.5,
|
||||
"bitrate": 5529600,
|
||||
"tier": 0,
|
||||
"w": 1280,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 9.8,
|
||||
"content_p95": 12.1,
|
||||
"input_p50": 27.8,
|
||||
"input_p95": 34.9,
|
||||
"grades": {
|
||||
"input_replies": "local",
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"input_p50": "local",
|
||||
"input_p95": "local",
|
||||
"fps": "local",
|
||||
"late_pct": "local",
|
||||
"stall_max": "acceptable"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 5.084,
|
||||
"ceiling": 5529600,
|
||||
"fps": 60,
|
||||
"inFlight": 1,
|
||||
"scale": 1,
|
||||
"slack": 49.603,
|
||||
"target": 5529600,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"t": 1.77,
|
||||
"e": "down to 2764 kbit/s: queue 23 ms, held 5, oldest 0 ms, base 5 ms, sent 315 got 315 wants 472"
|
||||
}
|
||||
],
|
||||
"source_fps": 60.1,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 16.1,
|
||||
"frame_viewer_pct": 40.9,
|
||||
"frame_tailscaled_pct": 5.2,
|
||||
"frame_sshd_pct": 0.7,
|
||||
"frame_gamescope_pct": 8.4,
|
||||
"frame_total_pct": 25.2
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 136 Hz",
|
||||
"show_s": 1.27,
|
||||
"panel": "valve.steam.desktopgame.2001639889",
|
||||
"src": "test"
|
||||
},
|
||||
{
|
||||
"scenario": "scroll",
|
||||
"frames_sent": 832,
|
||||
"frames_drawn": 832,
|
||||
"frames_shown": 602,
|
||||
"duration_s": 15.1,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": -3.9,
|
||||
"p95": 0.8,
|
||||
"p99": 6.8,
|
||||
"n": 832
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.1,
|
||||
"p99": 1.9,
|
||||
"n": 832
|
||||
},
|
||||
"encode": {
|
||||
"p50": 6.8,
|
||||
"p95": 10.5,
|
||||
"p99": 18.8,
|
||||
"n": 832
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.5,
|
||||
"n": 832
|
||||
},
|
||||
"network": {
|
||||
"p50": 5.8,
|
||||
"p95": 18.7,
|
||||
"p99": 56.7,
|
||||
"n": 832
|
||||
},
|
||||
"decode": {
|
||||
"p50": 5.9,
|
||||
"p95": 11.7,
|
||||
"p99": 26.0,
|
||||
"n": 832
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.7,
|
||||
"p95": 1.4,
|
||||
"p99": 1.7,
|
||||
"n": 832
|
||||
},
|
||||
"present": {
|
||||
"p50": 4.4,
|
||||
"p95": 18.5,
|
||||
"p99": 25.0,
|
||||
"n": 602
|
||||
},
|
||||
"content": {
|
||||
"p50": 20.2,
|
||||
"p95": 36.9,
|
||||
"p99": 90.7,
|
||||
"n": 832
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 26.4,
|
||||
"p95": 45.4,
|
||||
"p99": 101.0,
|
||||
"n": 602
|
||||
}
|
||||
},
|
||||
"fps": 54.9,
|
||||
"fps_shown": 39.7,
|
||||
"late_pct": 6.96,
|
||||
"stall_max": 204.9,
|
||||
"stalls_over_100ms": 2,
|
||||
"mbps": 9.04,
|
||||
"keyframes": 1,
|
||||
"size": "1920x1290",
|
||||
"captured": 852,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"n": 0
|
||||
},
|
||||
"mac": {
|
||||
"n": 0
|
||||
},
|
||||
"back": {
|
||||
"n": 0
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"asked": 0,
|
||||
"sent": 0,
|
||||
"seen": 0
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 49,
|
||||
"mbps": 8.68,
|
||||
"content_p50": 19.8,
|
||||
"content_p95": 119.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 47,
|
||||
"mbps": 6.06,
|
||||
"content_p50": 19.8,
|
||||
"content_p95": 88.6,
|
||||
"bitrate": 7430400,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 52,
|
||||
"mbps": 8.07,
|
||||
"content_p50": 19.4,
|
||||
"content_p95": 25.5,
|
||||
"bitrate": 10055362,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 56,
|
||||
"mbps": 9.79,
|
||||
"content_p50": 21.4,
|
||||
"content_p95": 29.1,
|
||||
"bitrate": 13549185,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 57,
|
||||
"mbps": 8.93,
|
||||
"content_p50": 21.5,
|
||||
"content_p95": 35.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 57,
|
||||
"mbps": 10.59,
|
||||
"content_p50": 21.1,
|
||||
"content_p95": 35.4,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 56,
|
||||
"mbps": 11.55,
|
||||
"content_p50": 19.7,
|
||||
"content_p95": 66.9,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 57,
|
||||
"mbps": 8.8,
|
||||
"content_p50": 18.8,
|
||||
"content_p95": 24.8,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 57,
|
||||
"mbps": 9.17,
|
||||
"content_p50": 19.5,
|
||||
"content_p95": 24.4,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 57,
|
||||
"mbps": 9.39,
|
||||
"content_p50": 19.4,
|
||||
"content_p95": 23.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 57,
|
||||
"mbps": 8.53,
|
||||
"content_p50": 19.5,
|
||||
"content_p95": 38.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 56,
|
||||
"mbps": 9.23,
|
||||
"content_p50": 20.7,
|
||||
"content_p95": 30.9,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 53,
|
||||
"mbps": 8.87,
|
||||
"content_p50": 20.6,
|
||||
"content_p95": 35.4,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 57,
|
||||
"mbps": 8.94,
|
||||
"content_p50": 20.4,
|
||||
"content_p95": 28.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 57,
|
||||
"mbps": 9.14,
|
||||
"content_p50": 21.0,
|
||||
"content_p95": 35.5,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 7,
|
||||
"mbps": 1.2,
|
||||
"content_p50": 21.8,
|
||||
"content_p95": 23.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 20.2,
|
||||
"content_p95": 36.9,
|
||||
"input_p50": null,
|
||||
"input_p95": null,
|
||||
"grades": {
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"fps": "acceptable",
|
||||
"late_pct": "bad",
|
||||
"stall_max": "acceptable"
|
||||
},
|
||||
"controller": {
|
||||
"adapt": true,
|
||||
"baseRtt": 6.833,
|
||||
"ceiling": 14860800,
|
||||
"fps": 60,
|
||||
"inFlight": 2,
|
||||
"scale": 1,
|
||||
"slack": 44.791,
|
||||
"target": 14860800,
|
||||
"tier": 0
|
||||
},
|
||||
"events": [
|
||||
{
|
||||
"t": 1.22,
|
||||
"e": "down to 7430 kbit/s: queue 74 ms, held 7, oldest 235 ms, base 6 ms, sent 5436 got 4487 wants 11082"
|
||||
}
|
||||
],
|
||||
"source_fps": 56.4,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 8.8,
|
||||
"frame_viewer_pct": 69.5,
|
||||
"frame_tailscaled_pct": 9.5,
|
||||
"frame_sshd_pct": 0.8,
|
||||
"frame_gamescope_pct": 8.4,
|
||||
"frame_total_pct": 31.8
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 139 Hz",
|
||||
"show_s": 1.25,
|
||||
"panel": "valve.steam.desktopgame.2001116820",
|
||||
"src": "separate:8327"
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -0,0 +1,984 @@
|
||||
{
|
||||
"label": "adapt-clean",
|
||||
"date": "2026-09-28T17:13:59",
|
||||
"commit": "a3c6e5c+dirty",
|
||||
"config": {
|
||||
"quality": "balanced",
|
||||
"mode": "separate",
|
||||
"net": "none",
|
||||
"delay_ms": 0,
|
||||
"buffer_ms": 250,
|
||||
"host": "frame",
|
||||
"ssh_opts": [],
|
||||
"encoder": "",
|
||||
"duration_s": 20.0,
|
||||
"browser_flags": []
|
||||
},
|
||||
"frame_build": "20260925.6191901",
|
||||
"mac": "26.5.2",
|
||||
"headset": ":39.639829 [Info] - 0 - entering standby",
|
||||
"scenarios": [
|
||||
{
|
||||
"scenario": "test",
|
||||
"frames_sent": 653,
|
||||
"frames_drawn": 653,
|
||||
"frames_shown": 649,
|
||||
"duration_s": 20.7,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0,
|
||||
"n": 653
|
||||
},
|
||||
"queue": {
|
||||
"p50": 11.2,
|
||||
"p95": 16.1,
|
||||
"p99": 17.1,
|
||||
"n": 653
|
||||
},
|
||||
"encode": {
|
||||
"p50": 3.5,
|
||||
"p95": 4.2,
|
||||
"p99": 4.4,
|
||||
"n": 653
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.2,
|
||||
"n": 653
|
||||
},
|
||||
"network": {
|
||||
"p50": 2.9,
|
||||
"p95": 7.0,
|
||||
"p99": 12.0,
|
||||
"n": 653
|
||||
},
|
||||
"decode": {
|
||||
"p50": 1.0,
|
||||
"p95": 2.4,
|
||||
"p99": 2.8,
|
||||
"n": 653
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.3,
|
||||
"p95": 0.6,
|
||||
"p99": 0.9,
|
||||
"n": 653
|
||||
},
|
||||
"present": {
|
||||
"p50": 0.5,
|
||||
"p95": 0.9,
|
||||
"p99": 5.9,
|
||||
"n": 649
|
||||
},
|
||||
"content": {
|
||||
"p50": 19.8,
|
||||
"p95": 26.2,
|
||||
"p99": 30.1,
|
||||
"n": 653
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 20.3,
|
||||
"p95": 26.9,
|
||||
"p99": 31.1,
|
||||
"n": 649
|
||||
}
|
||||
},
|
||||
"fps": 31.5,
|
||||
"fps_shown": 31.3,
|
||||
"late_pct": 98.01,
|
||||
"stall_max": 144.0,
|
||||
"stalls_over_100ms": 2,
|
||||
"mbps": 0.13,
|
||||
"keyframes": 2,
|
||||
"size": "960x540",
|
||||
"captured": 1336,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"p50": 43.5,
|
||||
"p95": 61.8,
|
||||
"p99": 63.4,
|
||||
"n": 40
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"p50": 44.3,
|
||||
"p95": 62.3,
|
||||
"p99": 63.9,
|
||||
"n": 40
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"p50": 11.1,
|
||||
"p95": 29.2,
|
||||
"p99": 36.1,
|
||||
"n": 40
|
||||
},
|
||||
"mac": {
|
||||
"p50": 10.4,
|
||||
"p95": 28.2,
|
||||
"p99": 32.4,
|
||||
"n": 40
|
||||
},
|
||||
"back": {
|
||||
"p50": 17.5,
|
||||
"p95": 27.9,
|
||||
"p99": 28.8,
|
||||
"n": 40
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"sent": 40,
|
||||
"seen": 40
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 31,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 21.5,
|
||||
"content_p95": 27.1,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 31,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 22.4,
|
||||
"content_p95": 26.3,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 32,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 18.9,
|
||||
"content_p95": 24.8,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 31,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 18.2,
|
||||
"content_p95": 24.6,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 32,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 21.0,
|
||||
"content_p95": 25.6,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 32,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 19.5,
|
||||
"content_p95": 24.3,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 31,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 22.2,
|
||||
"content_p95": 25.6,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 32,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 19.9,
|
||||
"content_p95": 24.0,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 31,
|
||||
"mbps": 0.21,
|
||||
"content_p50": 22.7,
|
||||
"content_p95": 26.9,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 31,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 20.2,
|
||||
"content_p95": 26.1,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 32,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 18.2,
|
||||
"content_p95": 23.2,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 31,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 19.8,
|
||||
"content_p95": 24.2,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 32,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 19.2,
|
||||
"content_p95": 24.8,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 32,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 21.0,
|
||||
"content_p95": 24.3,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 30,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 20.4,
|
||||
"content_p95": 96.7,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 32,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 18.2,
|
||||
"content_p95": 24.4,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 16,
|
||||
"fps": 31,
|
||||
"mbps": 0.12,
|
||||
"content_p50": 18.4,
|
||||
"content_p95": 25.1,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 17,
|
||||
"fps": 32,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 17.2,
|
||||
"content_p95": 23.5,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 18,
|
||||
"fps": 32,
|
||||
"mbps": 0.19,
|
||||
"content_p50": 19.4,
|
||||
"content_p95": 25.2,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 19,
|
||||
"fps": 31,
|
||||
"mbps": 0.14,
|
||||
"content_p50": 21.1,
|
||||
"content_p95": 84.3,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 20,
|
||||
"fps": 24,
|
||||
"mbps": 0.08,
|
||||
"content_p50": 20.8,
|
||||
"content_p95": 25.1,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 19.8,
|
||||
"content_p95": 26.2,
|
||||
"input_p50": 43.5,
|
||||
"input_p95": 61.8,
|
||||
"grades": {
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"input_p50": "local",
|
||||
"input_p95": "local",
|
||||
"fps": "bad",
|
||||
"late_pct": "bad",
|
||||
"stall_max": "acceptable"
|
||||
},
|
||||
"source_fps": 64.5,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 11.3,
|
||||
"frame_viewer_pct": 35.9,
|
||||
"frame_tailscaled_pct": 3.5,
|
||||
"frame_sshd_pct": 0.6,
|
||||
"frame_gamescope_pct": 7.5,
|
||||
"frame_total_pct": 16.1
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz",
|
||||
"show_s": 4.68,
|
||||
"panel": "valve.steam.desktopgame.2001639889",
|
||||
"src": "test"
|
||||
},
|
||||
{
|
||||
"scenario": "scroll",
|
||||
"frames_sent": 937,
|
||||
"frames_drawn": 937,
|
||||
"frames_shown": 720,
|
||||
"duration_s": 20.7,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": -3.9,
|
||||
"p95": 0.9,
|
||||
"p99": 8.2,
|
||||
"n": 937
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 16.3,
|
||||
"p99": 20.1,
|
||||
"n": 937
|
||||
},
|
||||
"encode": {
|
||||
"p50": 6.4,
|
||||
"p95": 7.2,
|
||||
"p99": 9.3,
|
||||
"n": 937
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.2,
|
||||
"p99": 0.2,
|
||||
"n": 937
|
||||
},
|
||||
"network": {
|
||||
"p50": 7.3,
|
||||
"p95": 13.0,
|
||||
"p99": 22.8,
|
||||
"n": 937
|
||||
},
|
||||
"decode": {
|
||||
"p50": 6.2,
|
||||
"p95": 12.7,
|
||||
"p99": 18.6,
|
||||
"n": 937
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.6,
|
||||
"p95": 1.1,
|
||||
"p99": 1.9,
|
||||
"n": 937
|
||||
},
|
||||
"present": {
|
||||
"p50": 2.9,
|
||||
"p95": 18.5,
|
||||
"p99": 24.5,
|
||||
"n": 720
|
||||
},
|
||||
"content": {
|
||||
"p50": 19.9,
|
||||
"p95": 37.2,
|
||||
"p99": 50.2,
|
||||
"n": 937
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 27.0,
|
||||
"p95": 45.7,
|
||||
"p99": 55.0,
|
||||
"n": 720
|
||||
}
|
||||
},
|
||||
"fps": 45.3,
|
||||
"fps_shown": 34.8,
|
||||
"late_pct": 28.63,
|
||||
"stall_max": 99.3,
|
||||
"stalls_over_100ms": 0,
|
||||
"mbps": 7.59,
|
||||
"keyframes": 2,
|
||||
"size": "1920x1290",
|
||||
"captured": 1215,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"n": 0
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"n": 0
|
||||
},
|
||||
"mac": {
|
||||
"n": 0
|
||||
},
|
||||
"back": {
|
||||
"n": 0
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"sent": 0,
|
||||
"seen": 0
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 0,
|
||||
"fps": 30,
|
||||
"mbps": 4.01,
|
||||
"content_p50": 31.9,
|
||||
"content_p95": 52.3,
|
||||
"bitrate": 4867140,
|
||||
"tier": 2,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 1,
|
||||
"fps": 30,
|
||||
"mbps": 4.12,
|
||||
"content_p50": 29.9,
|
||||
"content_p95": 45.3,
|
||||
"bitrate": 3807054,
|
||||
"tier": 2,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 2,
|
||||
"fps": 30,
|
||||
"mbps": 3.26,
|
||||
"content_p50": 31.8,
|
||||
"content_p95": 52.5,
|
||||
"bitrate": 3235995,
|
||||
"tier": 2,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 30,
|
||||
"mbps": 3.72,
|
||||
"content_p50": 24.5,
|
||||
"content_p95": 37.7,
|
||||
"bitrate": 4238740,
|
||||
"tier": 2,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 30,
|
||||
"mbps": 5.09,
|
||||
"content_p50": 25.2,
|
||||
"content_p95": 39.6,
|
||||
"bitrate": 5992063,
|
||||
"tier": 2,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 31,
|
||||
"mbps": 7.29,
|
||||
"content_p50": 29.1,
|
||||
"content_p95": 53.5,
|
||||
"bitrate": 5526067,
|
||||
"tier": 2,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 35,
|
||||
"mbps": 4.58,
|
||||
"content_p50": 25.3,
|
||||
"content_p95": 32.4,
|
||||
"bitrate": 6018152,
|
||||
"tier": 1,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 46,
|
||||
"mbps": 7.12,
|
||||
"content_p50": 25.1,
|
||||
"content_p95": 34.3,
|
||||
"bitrate": 8412933,
|
||||
"tier": 1,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 8,
|
||||
"fps": 46,
|
||||
"mbps": 8.25,
|
||||
"content_p50": 27.1,
|
||||
"content_p95": 40.3,
|
||||
"bitrate": 10760191,
|
||||
"tier": 1,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 9,
|
||||
"fps": 47,
|
||||
"mbps": 9.3,
|
||||
"content_p50": 27.8,
|
||||
"content_p95": 41.2,
|
||||
"bitrate": 13717060,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 55,
|
||||
"mbps": 8.98,
|
||||
"content_p50": 16.5,
|
||||
"content_p95": 21.7,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 11,
|
||||
"fps": 51,
|
||||
"mbps": 9.32,
|
||||
"content_p50": 18.6,
|
||||
"content_p95": 35.4,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 56,
|
||||
"mbps": 9.53,
|
||||
"content_p50": 15.4,
|
||||
"content_p95": 26.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 13,
|
||||
"fps": 55,
|
||||
"mbps": 8.46,
|
||||
"content_p50": 18.6,
|
||||
"content_p95": 23.2,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 54,
|
||||
"mbps": 9.82,
|
||||
"content_p50": 15.9,
|
||||
"content_p95": 28.4,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 15,
|
||||
"fps": 54,
|
||||
"mbps": 11.11,
|
||||
"content_p50": 15.5,
|
||||
"content_p95": 26.1,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 16,
|
||||
"fps": 54,
|
||||
"mbps": 9.79,
|
||||
"content_p50": 17.4,
|
||||
"content_p95": 25.3,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 17,
|
||||
"fps": 57,
|
||||
"mbps": 9.3,
|
||||
"content_p50": 16.3,
|
||||
"content_p95": 23.5,
|
||||
"bitrate": 14860800,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 18,
|
||||
"fps": 55,
|
||||
"mbps": 8.3,
|
||||
"content_p50": 19.5,
|
||||
"content_p95": 31.0,
|
||||
"bitrate": 7984791,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 19,
|
||||
"fps": 51,
|
||||
"mbps": 8.56,
|
||||
"content_p50": 16.4,
|
||||
"content_p95": 29.1,
|
||||
"bitrate": 10220855,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 20,
|
||||
"fps": 40,
|
||||
"mbps": 7.13,
|
||||
"content_p50": 18.3,
|
||||
"content_p95": 23.9,
|
||||
"bitrate": 12025604,
|
||||
"tier": 0,
|
||||
"w": 1920,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 19.9,
|
||||
"content_p95": 37.2,
|
||||
"input_p50": null,
|
||||
"input_p95": null,
|
||||
"grades": {
|
||||
"content_p50": "local",
|
||||
"content_p95": "local",
|
||||
"fps": "acceptable",
|
||||
"late_pct": "bad",
|
||||
"stall_max": "local"
|
||||
},
|
||||
"source_fps": 58.7,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 6.6,
|
||||
"frame_viewer_pct": 72.4,
|
||||
"frame_tailscaled_pct": 8.1,
|
||||
"frame_sshd_pct": 0.9,
|
||||
"frame_gamescope_pct": 10.0,
|
||||
"frame_total_pct": 24.2
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 141 Hz",
|
||||
"show_s": 6.43,
|
||||
"panel": "valve.steam.desktopgame.2001414593",
|
||||
"src": "separate:9384"
|
||||
},
|
||||
{
|
||||
"scenario": "type",
|
||||
"frames_sent": 25,
|
||||
"frames_drawn": 25,
|
||||
"frames_shown": 25,
|
||||
"duration_s": 14.7,
|
||||
"stages_ms": {
|
||||
"capture": {
|
||||
"p50": -0.3,
|
||||
"p95": 0.9,
|
||||
"p99": 1.1,
|
||||
"n": 25
|
||||
},
|
||||
"queue": {
|
||||
"p50": 0.0,
|
||||
"p95": 0.0,
|
||||
"p99": 0.0,
|
||||
"n": 25
|
||||
},
|
||||
"encode": {
|
||||
"p50": 5.0,
|
||||
"p95": 33.4,
|
||||
"p99": 33.6,
|
||||
"n": 25
|
||||
},
|
||||
"socket": {
|
||||
"p50": 0.1,
|
||||
"p95": 0.1,
|
||||
"p99": 0.2,
|
||||
"n": 25
|
||||
},
|
||||
"network": {
|
||||
"p50": 6.1,
|
||||
"p95": 14.9,
|
||||
"p99": 24.4,
|
||||
"n": 25
|
||||
},
|
||||
"decode": {
|
||||
"p50": 3.0,
|
||||
"p95": 10.5,
|
||||
"p99": 11.4,
|
||||
"n": 25
|
||||
},
|
||||
"draw": {
|
||||
"p50": 0.6,
|
||||
"p95": 1.0,
|
||||
"p99": 1.1,
|
||||
"n": 25
|
||||
},
|
||||
"present": {
|
||||
"p50": 0.8,
|
||||
"p95": 1.4,
|
||||
"p99": 1.7,
|
||||
"n": 25
|
||||
},
|
||||
"content": {
|
||||
"p50": 12.6,
|
||||
"p95": 41.0,
|
||||
"p99": 46.3,
|
||||
"n": 25
|
||||
},
|
||||
"content_shown": {
|
||||
"p50": 13.9,
|
||||
"p95": 41.8,
|
||||
"p99": 46.6,
|
||||
"n": 25
|
||||
}
|
||||
},
|
||||
"fps": 1.7,
|
||||
"fps_shown": 1.7,
|
||||
"late_pct": 100.0,
|
||||
"stall_max": 3099.2,
|
||||
"stalls_over_100ms": 21,
|
||||
"mbps": 0.04,
|
||||
"keyframes": 4,
|
||||
"size": "960x646",
|
||||
"captured": 79,
|
||||
"viewer_never_drawn": 0,
|
||||
"input_ms": {
|
||||
"p50": 41.3,
|
||||
"p95": 60.3,
|
||||
"p99": 68.6,
|
||||
"n": 18
|
||||
},
|
||||
"input_shown_ms": {
|
||||
"p50": 43.0,
|
||||
"p95": 61.1,
|
||||
"p99": 69.6,
|
||||
"n": 18
|
||||
},
|
||||
"input_parts_ms": {
|
||||
"uplink": {
|
||||
"p50": 5.3,
|
||||
"p95": 11.6,
|
||||
"p99": 14.2,
|
||||
"n": 18
|
||||
},
|
||||
"mac": {
|
||||
"p50": 23.1,
|
||||
"p95": 35.0,
|
||||
"p99": 42.6,
|
||||
"n": 18
|
||||
},
|
||||
"back": {
|
||||
"p50": 11.8,
|
||||
"p95": 19.2,
|
||||
"p99": 22.7,
|
||||
"n": 18
|
||||
}
|
||||
},
|
||||
"inputs": {
|
||||
"sent": 66,
|
||||
"seen": 18
|
||||
},
|
||||
"timeline": [
|
||||
{
|
||||
"t": 3,
|
||||
"fps": 1,
|
||||
"mbps": 0.01,
|
||||
"content_p50": 11.8,
|
||||
"content_p95": 11.8,
|
||||
"bitrate": 1224075,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 4,
|
||||
"fps": 1,
|
||||
"mbps": 0.02,
|
||||
"content_p50": 14.7,
|
||||
"content_p95": 14.7,
|
||||
"bitrate": 2091896,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 5,
|
||||
"fps": 4,
|
||||
"mbps": 0.1,
|
||||
"content_p50": 18.6,
|
||||
"content_p95": 37.4,
|
||||
"bitrate": 3071304,
|
||||
"tier": 3,
|
||||
"w": 1440,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 6,
|
||||
"fps": 5,
|
||||
"mbps": 0.13,
|
||||
"content_p50": 4.2,
|
||||
"content_p95": 19.2,
|
||||
"bitrate": 4031277,
|
||||
"tier": 3,
|
||||
"w": 1440,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 7,
|
||||
"fps": 6,
|
||||
"mbps": 0.04,
|
||||
"content_p50": 12.0,
|
||||
"content_p95": 46.3,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 10,
|
||||
"fps": 1,
|
||||
"mbps": 0.01,
|
||||
"content_p50": 12.2,
|
||||
"content_p95": 12.2,
|
||||
"bitrate": 300000,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 12,
|
||||
"fps": 1,
|
||||
"mbps": 0.02,
|
||||
"content_p50": -1.2,
|
||||
"content_p95": -1.2,
|
||||
"bitrate": 842857,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
},
|
||||
{
|
||||
"t": 14,
|
||||
"fps": 1,
|
||||
"mbps": 0.02,
|
||||
"content_p50": 14.4,
|
||||
"content_p95": 14.4,
|
||||
"bitrate": 2091896,
|
||||
"tier": 4,
|
||||
"w": 960,
|
||||
"net": null
|
||||
}
|
||||
],
|
||||
"adapt": [],
|
||||
"content_p50": 12.6,
|
||||
"content_p95": 41.0,
|
||||
"input_p50": 41.3,
|
||||
"input_p95": 60.3,
|
||||
"grades": {
|
||||
"content_p50": "local",
|
||||
"content_p95": "acceptable",
|
||||
"input_p50": "local",
|
||||
"input_p95": "local"
|
||||
},
|
||||
"source_fps": 5.4,
|
||||
"cpu": {
|
||||
"mac_agent_pct": 1.2,
|
||||
"frame_viewer_pct": 5.0,
|
||||
"frame_tailscaled_pct": 1.5,
|
||||
"frame_sshd_pct": 0.2,
|
||||
"frame_gamescope_pct": 4.7,
|
||||
"frame_total_pct": 8.5
|
||||
},
|
||||
"viewer": "h264 software; ANGLE (Mesa, zink Vulkan 1.4(Turnip Adreno (TM) 750 (MESA_TURNIP)), OpenGL 4.6); raf 140 Hz",
|
||||
"show_s": 5.12,
|
||||
"panel": "valve.steam.desktopgame.2001910179",
|
||||
"src": "separate:9510"
|
||||
}
|
||||
]
|
||||
}
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
+26
-3
@@ -1,9 +1,32 @@
|
||||
# compat-db: Frame Control's compatibility database
|
||||
|
||||
A private [Lakebed](https://docs.lakebed.dev/) capsule holding compatibility
|
||||
reports for Android apps on the Steam Frame. For now only the maintainer's
|
||||
copy of Frame Control has the key to read or write it. Everyone else's reports
|
||||
stay on their own Mac (see `shared()` in `ui/frame_compat_db.py`).
|
||||
reports for Android apps on the Steam Frame. Only the maintainer's copy of
|
||||
Frame Control has the key to read or write it (see `shared()` in
|
||||
`ui/frame_compat_db.py`). Everyone else's reports stay on their computer
|
||||
unless they turn on **Share compatibility results**. Then the reports also go
|
||||
to PostHog as `compat_report` events, and the maintainer syncs them in (below).
|
||||
|
||||
## Community reports
|
||||
|
||||
```sh
|
||||
python3 ui/frame_compat_db.py sync --dry-run # what would be added
|
||||
python3 ui/frame_compat_db.py sync # add them
|
||||
```
|
||||
|
||||
`sync` reads `compat_report` events through PostHog's query API and adds
|
||||
them with `via` set to `community`, `community-probe` or `community-install`.
|
||||
It skips invalid reports and anything over 30 per reporter per day. Each run
|
||||
re-reads the last 30 days, because an offline copy sends its reports late,
|
||||
with the time they were made. `posthog-sync.json`, next to the outbox,
|
||||
remembers which reports it has handled and each reporter's daily count, so
|
||||
nothing is added twice and the cap holds across runs.
|
||||
|
||||
It needs:
|
||||
|
||||
- the PostHog project id: `"project"` in `ui/telemetry.json`
|
||||
- a personal API key with `query:read`: `POSTHOG_PERSONAL_API_KEY`, or in the
|
||||
Keychain (service `frame-control-posthog`, account `personal-api-key`)
|
||||
|
||||
- Live: `https://frame-compat.lakebed.app` (deploy `dep_dDmcsosVSiFirpW6`,
|
||||
claimed, so it doesn't expire). The browser page only says it's private.
|
||||
|
||||
+185
@@ -0,0 +1,185 @@
|
||||
# Frame Control for AI agents
|
||||
|
||||
**Documented interface:** Frame Control's own stdlib Python MCP adapter wraps
|
||||
its loopback HTTP API. No API key, hosted service, model SDK or third-party
|
||||
helper app is needed. The assistant is our HTML/Python implementation hosted
|
||||
in the platform Chromium browser. Its optional LLM endpoint is user configuration.
|
||||
Installing other apps is an optional management action, never a prerequisite.
|
||||
|
||||
## Connect an MCP client
|
||||
|
||||
The default MCP command starts a private HTTP backend on a free loopback port,
|
||||
with a fresh local access key. It stops that backend when the MCP client closes
|
||||
stdin or sends SIGTERM. It uses its own SSH control socket, so closing it does
|
||||
not close the desktop app's connection. No manually started server is needed.
|
||||
|
||||
Add this stdio server to your MCP client (use absolute paths):
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"frame-control": {
|
||||
"command": "python3",
|
||||
"args": ["/absolute/path/frame-control/ui/frame_mcp.py"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
For Codex, the equivalent registration is:
|
||||
|
||||
```sh
|
||||
codex mcp add frame-control -- python3 /absolute/path/frame-control/ui/frame_mcp.py
|
||||
```
|
||||
|
||||
New agent sessions load the entry. An already running session may need its MCP
|
||||
connections reloaded; registration does not retroactively add tools to its
|
||||
initial tool inventory. Keep the checkout at that path while it is registered.
|
||||
Use `codex mcp remove frame-control` to remove only this registration.
|
||||
|
||||
To reuse a running server instead, pass `--url http://127.0.0.1:47810`.
|
||||
The desktop app uses a random port; use that port with `--url`, or run the
|
||||
checkout server above. If the HTTP server uses `FRAME_UI_KEY`, pass the same
|
||||
value in the MCP process environment. This is local access control, not an LLM
|
||||
API key. The adapter only accepts loopback HTTP servers, refuses redirects and
|
||||
ignores environment proxies. Stdout contains newline-delimited JSON-RPC only.
|
||||
It supports MCP initialization, ping, tool listing and tool calls; no sampling,
|
||||
resources, prompts or streaming transport.
|
||||
|
||||
| Tool | Arguments | Effect |
|
||||
|---|---|---|
|
||||
| `computer_state` | none | Read-only gamescope window IDs/focus and bounded AT-SPI tree; reports incomplete observations |
|
||||
| `status` | none | Battery, services, installed games and Flatpaks |
|
||||
| `screenshot` | `view`: `headset` (default) or `desktop` | Returns PNG image content to the MCP client |
|
||||
| `job` | `id` | Background install status; poll until `done`, inspect `error` |
|
||||
| `launch` | `appid` | Launch an installed Steam app |
|
||||
| `install` / `uninstall` | `id` | Install from Flathub / remove a user Flatpak |
|
||||
| `send_text` | `text` | Frame desktop clipboard; desktop must be open |
|
||||
| `send_file` | `path` | File on the HTTP server computer, up to 16 MiB, copied to Frame `~/Downloads` |
|
||||
| `panel` | `id` | Launch an installed Flatpak as a panel using the existing launcher |
|
||||
| `power` | `action`: `suspend`, `reboot`, `poweroff` | Open a terminal for the user to enter the sudo password |
|
||||
| `keep_awake` | `action`: `on`, `off`, `status` | Optional keep-awake script interface |
|
||||
|
||||
Only install free software with its developer's consent. There is no purchase,
|
||||
entitlement bypass or arbitrary shell tool. `install` returns a background job
|
||||
ID; it does not claim the installation has finished. APK and sideloaded title
|
||||
installs remain in the main UI for now.
|
||||
|
||||
### Approval is a separate human action
|
||||
|
||||
Every mutation first returns an `approvalUrl`, exact action and `confirmation`
|
||||
token. Ask the user to open that URL and choose **Approve this action** or
|
||||
**Reject**. Then repeat the same tool and arguments with the token in
|
||||
`confirmation`. The server refuses execution before approval, changed arguments,
|
||||
expired tokens and reuse. A file approval binds the content hash as well as the
|
||||
path. Approvals last five minutes and disappear when the HTTP server restarts.
|
||||
A failed execution also consumes the approval; review a fresh request to retry.
|
||||
The panel does not execute an action merely because it was approved.
|
||||
|
||||
MCP has no approval tool. This is protection against accidental model tool
|
||||
calls, not a sandbox against a client with independent shell/HTTP access to your
|
||||
computer. Grant the MCP client only the access you intend. Status, captures and computer-state observations
|
||||
are returned directly to that client, which may forward them to its configured
|
||||
model. The assistant's separate opt-in does not govern an external MCP client.
|
||||
|
||||
Power still requires the existing password prompt in a local terminal. MCP
|
||||
never receives passwords. Power via `FRAME_LOCAL=1` is unsupported: use the main
|
||||
UI. The panel launcher and keep-awake adapter require zsh on the computer.
|
||||
|
||||
[PR #16](https://github.com/saphid/frame-control/pull/16) owns
|
||||
`scripts/keep-awake.sh on|off|status`. This branch does not copy or change it.
|
||||
Until that script is present, the tool reports it unavailable. Keep-awake is
|
||||
never automatic: `on` changes the shared idle timers; explicitly approve `off`
|
||||
to restore them after work. It is not a per-agent lease; coordinate with other
|
||||
users. No changes are made to the analytics/update interfaces in
|
||||
[PR #17](https://github.com/saphid/frame-control/pull/17). Prompts, keys, model
|
||||
replies, screenshots and approval payloads are not sent to analytics.
|
||||
|
||||
## Assistant panel
|
||||
|
||||
Open **Tools → Open assistant**, or `http://127.0.0.1:47810/assistant`.
|
||||
To put the same page in the headset, with the HTTP server still running:
|
||||
|
||||
```sh
|
||||
python3 scripts/assistant-on-frame.py --port 47810
|
||||
```
|
||||
|
||||
This starts an SSH reverse forward bound to Frame loopback (port 47812 by
|
||||
default), then a dedicated Chromium profile tagged as a SteamVR panel. Keep the
|
||||
command running. Ctrl-C closes this browser profile and the tunnel; it leaves
|
||||
other Chromium windows and the existing HTTP server alone. A failed cleanup
|
||||
prints the temporary profile path so it can be removed when the Frame returns.
|
||||
Use `--frame-port` if the default is busy. Chromium must already be available as
|
||||
`org.chromium.Chromium`; the launcher never installs anything automatically.
|
||||
Place the panel with SteamVR's normal docking controls.
|
||||
|
||||
Enter your full **chat-completions endpoint**, model name and optional key.
|
||||
An OpenAI-compatible local server works without a key; no OpenAI account is
|
||||
required. HTTP is allowed only on loopback; other endpoints require HTTPS.
|
||||
Loopback refers to the computer running the HTTP server, even in the headset.
|
||||
Endpoints with embedded credentials, query strings or redirects are refused.
|
||||
|
||||
Check the message consent box and press **Send message**. Screenshot context is
|
||||
a separate unchecked box and sends one fresh capture with that request. Both
|
||||
boxes reset after sending, and changing endpoint/model revokes consent. Nothing
|
||||
is sent when opening the page or entering configuration. There is no model
|
||||
list fetch, saved history, automatic screenshot capture or assistant telemetry.
|
||||
Each send is independent: previous messages and replies are not included.
|
||||
|
||||
Configuration, credentials and chat remain in page memory; close/reload the page
|
||||
or choose **Clear everything** to clear them. A request already sent cannot be
|
||||
recalled. Only the chosen endpoint gets the request; proxy environment variables
|
||||
and redirects are disabled. Its privacy and retention policy still applies.
|
||||
Replies are plain text and cannot call tools or operate the Frame. A model must
|
||||
support image inputs to accept screenshot context.
|
||||
|
||||
## Evidence and limits
|
||||
|
||||
**Verified 2026-09-28, SteamOS 0.4.1, BUILD_ID 20260925.6191901:** loopback HTTP
|
||||
status through an SSH reverse tunnel; platform Chromium created a separate
|
||||
SteamVR panel (confirmed in `GAMESCOPE_FOCUSABLE_APPS`); headset capture returned
|
||||
a PNG. These checks preceded the UI implementation. No power or global settings
|
||||
were changed.
|
||||
|
||||
**Inferred:** visual comfort and controller keyboard usability while wearing
|
||||
the headset; panel creation in gamescope alone does not establish these.
|
||||
Windows/Linux launcher support, live third-party model endpoints, installs,
|
||||
uninstalls, power and keep-awake changes are not covered by that feasibility
|
||||
check. See the PR for the final unit and end-to-end results.
|
||||
|
||||
**Verified end to end on the same Frame/build (2026-09-28):** a stdio MCP client
|
||||
initialized, read status, retrieved a headset PNG, and transferred a test file
|
||||
only after approval through the Chromium page. Remote file bytes matched;
|
||||
reusing the confirmation was rejected. The actual headset Chromium page sent
|
||||
text and then separately opted-in image context to a local test endpoint and
|
||||
displayed its replies. Without consent there were zero endpoint requests.
|
||||
The test endpoint returned canned replies: model inference and a live external
|
||||
provider remain **unverified**. The launcher’s Ctrl-C cleanup was checked;
|
||||
profiles, SSH tunnels and the test file were removed. No installs, removals,
|
||||
launches of user games, power operations or keep-awake changes were performed.
|
||||
|
||||
**Verified locally:** unit coverage includes the stdio subprocess, approval
|
||||
binding/expiry/replay/concurrency, file-change rejection, and a real local HTTP
|
||||
endpoint for opt-in, text/image payloads and redirect refusal. Fake-Frame
|
||||
regressions are in `tests/e2e/test_agents.py`; local Docker execution was blocked
|
||||
because the Docker daemon was unavailable. The ARM64 fake-Frame CI job passed
|
||||
on this branch (run 36421345682).
|
||||
|
||||
**Verified on the same Frame/build:** both Ctrl-C and SIGTERM close the dedicated
|
||||
browser profile and SSH tunnel and remove the profile and panel log.
|
||||
|
||||

|
||||
|
||||
## Computer-use coverage
|
||||
|
||||
MCP is the tool transport, not a limit on what an agent can do. A screenshot,
|
||||
accessibility snapshot, click or keystroke can all be MCP tools when we have a
|
||||
reliable underlying implementation. See [the investigation](computer-use.md)
|
||||
for the verified boundaries. `computer_state` adds observation, not an input
|
||||
channel: it cannot click an approval button or send keyboard/mouse events.
|
||||
|
||||
**Verified 2026-09-29, SteamOS 0.4.1, BUILD_ID 20260925.6191901:** the command saved
|
||||
by `codex mcp add` launched without a prestarted server, negotiated MCP, listed
|
||||
12 tools, read live Frame status and returned X11 window state plus AT-SPI
|
||||
observations. It exited 0 at EOF. Steam's accessibility tree had inaccessible
|
||||
children, reported as `incomplete: true`; this is not a complete actionable UI.
|
||||
@@ -0,0 +1,144 @@
|
||||
# Announcing changes
|
||||
|
||||
How we tell people about Frame Control features and fixes as they merge. The
|
||||
same few sentences feed the X post, the release notes and the website, so they
|
||||
are written once, in the pull request, while the change is fresh.
|
||||
|
||||
## What gets announced
|
||||
|
||||
| Kind | Announce? | Example |
|
||||
|---|---|---|
|
||||
| **New** — something you can now do | Yes, its own post | Stream Mac windows into the Frame as panels |
|
||||
| **Better** — something existing got noticeably easier, faster or wider | Yes, its own post or a roundup | APKs install without the Android SDK |
|
||||
| **Fixed** — something broken that users hit | Yes if people reported it or it blocked a flow; otherwise the next roundup | Mac mirror showed a zoomed-in corner |
|
||||
| **Release** — a tagged build | Always, one post linking the release | Frame Control 0.3.1 |
|
||||
| Tests, refactors, CI, docs-only, website polish | No | Fake Frame tests, screenshot crop |
|
||||
|
||||
If a change isn't worth a sentence to someone who owns a Frame, it isn't
|
||||
announced.
|
||||
|
||||
## The voice
|
||||
|
||||
Write it the way the README and release notes already read.
|
||||
|
||||
- **Lead with what the person can now do**, in their words: "Install older
|
||||
versions of an app when the newest won't run on the Frame", not "Add APK
|
||||
version fallback resolver".
|
||||
- **Plain and specific.** Name the thing, give the number: "about 30 fps",
|
||||
"4,500 apps", "up to 8 older versions". No "blazing", "game-changing",
|
||||
"excited to announce", "huge", or exclamation marks.
|
||||
- **Say where it works.** Platforms and what it was tested on, briefly:
|
||||
"Tested on a real Frame from macOS 27." Don't claim what wasn't tested.
|
||||
- **Say the catch.** If it needs a setup step, an unsigned build, or only works
|
||||
on one OS, say so in the same post.
|
||||
- **Sentence case**, full sentences, British spelling to match the docs.
|
||||
Contractions are fine.
|
||||
- **No emoji in the text.** One image, GIF or short clip carries the tone
|
||||
instead. The only symbol is the kind label below.
|
||||
- **Unofficial, always.** Never imply Valve made or endorses it. Say "Steam
|
||||
Frame" for the headset and "Frame Control" for the app.
|
||||
- **Credit people.** If a user reported the bug or suggested the feature and is
|
||||
happy to be named, thank them by handle.
|
||||
|
||||
## The formats
|
||||
|
||||
Every announceable PR ends with an `## Announcement` section holding these.
|
||||
The reviewer checks it like code.
|
||||
|
||||
### 1. The post (X, and any other social account)
|
||||
|
||||
```
|
||||
<Kind>: <what you can do now, one sentence>
|
||||
|
||||
<one or two sentences: how it works, the catch, or what it was tested on>
|
||||
|
||||
<link>
|
||||
```
|
||||
|
||||
- `<Kind>` is `New`, `Better` or `Fixed`.
|
||||
- 280 characters maximum including the link (X counts any link as 23).
|
||||
- One link: the release if it has shipped, otherwise the PR.
|
||||
- One visual when the change is visible: a screenshot from the app, a GIF, or a
|
||||
short clip from the headset. Alt text describes what it shows.
|
||||
- No hashtags, except `#SteamFrame` on releases and on posts about something
|
||||
new, because people search for it.
|
||||
|
||||
### 2. The release-note line
|
||||
|
||||
One bullet under **New in x.y.z**, same as the current release notes: the
|
||||
first half of the post's first sentence, no kind label, no link.
|
||||
|
||||
### 3. Release post
|
||||
|
||||
```
|
||||
Frame Control <version>: <the headline change>
|
||||
|
||||
<one sentence on the headline change>. Also: <two or three short items>.
|
||||
|
||||
Windows, macOS and Linux: <release link>
|
||||
#SteamFrame
|
||||
```
|
||||
|
||||
The release title on GitHub uses the same `Frame Control <version>: <headline>`
|
||||
line, as 0.3.0 and 0.3.1 already do.
|
||||
|
||||
### Roundups
|
||||
|
||||
Small fixes that don't earn their own post wait for a roundup, posted with the
|
||||
next release or when three or more have piled up:
|
||||
|
||||
```
|
||||
Fixed in Frame Control this week:
|
||||
- <fix>
|
||||
- <fix>
|
||||
- <fix>
|
||||
|
||||
<link>
|
||||
```
|
||||
|
||||
## Examples from what has already merged
|
||||
|
||||
**#11, older APK versions**
|
||||
|
||||
```
|
||||
New: when an Android app is too new for the Frame, Frame Control now offers
|
||||
older versions that will install.
|
||||
|
||||
It checks F-Droid, its archive and IzzyOnDroid, and verifies each download
|
||||
before it goes on the headset.
|
||||
|
||||
https://github.com/saphid/steam-frame/pull/11
|
||||
```
|
||||
|
||||
**#8, Mac mirror fixes**
|
||||
|
||||
```
|
||||
Fixed: mirroring your Mac into the Steam Frame now fits the whole desktop in
|
||||
the panel, asks for the right password, and shows the cursor.
|
||||
|
||||
Tested end to end on a real Frame from macOS 27.
|
||||
|
||||
https://github.com/saphid/steam-frame/pull/8
|
||||
```
|
||||
|
||||
**v0.3.1**
|
||||
|
||||
```
|
||||
Frame Control 0.3.1: install APKs without the Android SDK
|
||||
|
||||
Frame Control now reads APK files itself, so there's nothing extra to install.
|
||||
Also: Linux and Windows game sideloading, and one-click install links.
|
||||
|
||||
Windows, macOS and Linux: https://github.com/saphid/steam-frame/releases/tag/v0.3.1
|
||||
#SteamFrame
|
||||
```
|
||||
|
||||
## Posting
|
||||
|
||||
Nothing is posted without a person approving it. The flow is:
|
||||
|
||||
1. The PR carries its `## Announcement` section.
|
||||
2. On merge, the post is drafted from that section (manually for now).
|
||||
3. Alex approves or edits it, then it's posted from the project account.
|
||||
4. Replies and questions that turn out to be bugs become GitHub issues labelled
|
||||
`feedback`, same as the website form.
|
||||
@@ -5,6 +5,12 @@ in **Lepton**, Valve's Waydroid-based container. Lepton is built for games,
|
||||
not general Android use
|
||||
([GamingOnLinux](https://www.gamingonlinux.com/2026/09/lepton-from-valve-to-run-android-games-on-linux-is-now-open-source/)).
|
||||
|
||||
**VR streaming clients:** WiVRn 26.9 and ALVR 20.14.1 install, but both fail
|
||||
OpenXR instance creation on the checked Frame because its Android runtime lacks
|
||||
`XR_KHR_convert_timespec_time` (**verified** 2026-09-28, SteamOS 0.4.1,
|
||||
BUILD_ID 20260925.6191901). They are not Frame Control dependencies. See
|
||||
[the feasibility results and options](linux-vr-streaming.md).
|
||||
|
||||
## Install from the Mac: one app, one Lepton instance (verified 2026-09-25)
|
||||
|
||||
Use Frame Control's **Android apps** section (search, Install, Test, Report), drop
|
||||
@@ -46,6 +52,33 @@ Lepton Development must be installed once. Over SSH,
|
||||
`ssh frame 'steam steam://install/3056000'` queues it, but the install still
|
||||
needs to be confirmed or started in the headset.
|
||||
|
||||
## When an app needs a newer Android
|
||||
|
||||
Lepton is Android 11 (API 30), with arm64-v8a only. If Frame Control refuses
|
||||
an APK, it shows compatible versions from F-Droid's main and archive repos and
|
||||
IzzyOnDroid. It shows at most eight version names, newest first, preferring an
|
||||
arm64-only build, and says how many compatible builds it found in total.
|
||||
Each index is reduced to its compatible builds once a day and cached (about
|
||||
16 MB). The first lookup takes about 30 s and 100 MB of memory; later ones are
|
||||
instant.
|
||||
Choose **Install** to download a listed version, verify its SHA-256 against
|
||||
the index, and install it as its own app.
|
||||
|
||||
You can also inspect a file or look up a package from the command line:
|
||||
|
||||
```sh
|
||||
python3 ui/frame_android.py info some-app.apk
|
||||
python3 ui/frame_android.py versions some-app.apk
|
||||
python3 ui/frame_android.py versions org.example.app
|
||||
```
|
||||
|
||||
The search links open APKMirror, APKPure, Uptodown, F-Droid and GitHub. Pick a
|
||||
version whose minimum is Android 11 or lower and that has an arm64-v8a build
|
||||
(or no native code). Frame Control does not fetch APKs from those search sites.
|
||||
Older versions may lack fixes, and being installable does not guarantee an
|
||||
app will run: see the missing services below. Android may refuse a downgrade
|
||||
or an update signed by a different publisher; removing the app deletes its data.
|
||||
|
||||
## Installed apps disappear when Lepton Development closes (verified 2026-09-25)
|
||||
|
||||
Lepton Development runs in a throwaway "dev" context. When it exits for any
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# Computer use through Frame Control MCP
|
||||
|
||||
The MCP transport can carry semantic actions or visual computer-use actions.
|
||||
The limits are the Frame's underlying interfaces, permissions and whether an
|
||||
action can be targeted and verified. A stereoscopic headset screenshot alone
|
||||
is not a reliable coordinate system for clicking a particular app window.
|
||||
|
||||
## What exists, and the right route
|
||||
|
||||
| Surface | Evidence and route | Remaining work or boundary |
|
||||
|---|---|---|
|
||||
| Frame management | **Verified:** existing SSH/HTTP operations for status, capture and file transfer work through MCP. Typed install/launch/power tools wrap the existing API. | Extend typed operations before adding generic mouse automation. Preserve explicit approval for consequential changes. |
|
||||
| App/window observation | **Verified 2026-09-29:** `computer_state` reads gamescope X11 window/app/process triples, focused app and the installed AT-SPI library. | Bounded to 96 accessible nodes and six levels. Trees may be truncated, stale, hidden or incomplete. Snapshot paths and XIDs are observations, never durable action permissions. |
|
||||
| Chromium page content | **Verified previously:** the assistant rendered and could be exercised through CDP in an isolated Frame Chromium profile. | A shipped click/type surface needs exact owned browser/target binding, fresh element references, lifecycle cleanup, consent and post-action readback. Do not expose unrestricted JavaScript or attach to arbitrary existing profiles automatically. |
|
||||
| Steam UI | **Verified 2026-09-29:** the AT-SPI service listed the Steam client's Chromium process and frame nodes, but child traversal was incomplete. Existing `frame_steam.py` uses Steam's loopback CDP endpoint for specific operations. | Prefer those narrow Steam interfaces. Presence of AT-SPI does not prove controls are actionable, and generic pointer injection is not proved for VR menus. |
|
||||
| Other Linux apps | **Verified 2026-09-29:** Frame ships libX11, libXtst and libatspi; `/dev/uinput` is writable by the current user. | Library presence and access permissions do not prove that a game accepts input. Global virtual input can affect whichever app has focus. Do not ship a blind keyboard/mouse tool on this evidence alone. |
|
||||
| Panel focus and layouts | **Documented in [#41](https://github.com/saphid/frame-control/pull/41):** `POST /api/panels` accepts `list`, `focus` and `open`. Focus was verified there. | Reuse that owned interface after integration. Its tested gamescope-owned overlay transform setters return `PermissionDenied`; no reliable saved spatial-layout interface was established. Do not duplicate its implementation here. |
|
||||
| Shared keyboard/trackpad | **Documented in [#19](https://github.com/saphid/frame-control/pull/19):** `/api/input` supplies state/start and event submission, implemented with a bundled KDE Connect daemon. | This branch does not import, launch or depend on that daemon. The user's own-implementation rule remains authoritative. A first-party input implementation or permitted bundled-library route needs its own delivery evidence before MCP integration. |
|
||||
| Physical/device boundaries | **Documented:** an asleep Frame may be off the network; power authorization can require the user's password; physical pairing and headset fit/comfort require the user. | MCP cannot bypass offline hardware, consent, compositor permissions or physical verification. Keep explicit human handoffs. |
|
||||
|
||||
## Reusing the existing computer-use work
|
||||
|
||||
**Documented:** the installed `cua-driver` skill has the right control pattern:
|
||||
observe an exact window, use a semantic target if available, fall back to pixels
|
||||
from that same snapshot, then read back the result. Its browser route requires
|
||||
an exact process/window/target binding and session-scoped element references.
|
||||
Those are useful design rules for Frame tools.
|
||||
|
||||
**Verified locally 2026-09-29:** `cua-driver describe get_window_state` describes
|
||||
host-local process/window IDs and macOS AX inspection. It does not establish an
|
||||
SSH Frame target. The installed skill's advertised Linux companion file is
|
||||
missing. A native ARM64 Frame backend, its dependencies and remote transport
|
||||
have not been verified. We therefore do not claim that the existing Mac driver
|
||||
can control the Frame by passing it a Frame PID or screenshot, and we do not
|
||||
make the feature depend on installing that application.
|
||||
|
||||
Frame Control's `computer_state` is our own Python implementation over installed
|
||||
platform libraries. It sends the probe over SSH stdin, writes no helper to disk,
|
||||
and exits after one observation. Missing displays/libraries return explicit
|
||||
errors; a 15-second process deadline prevents a stalled accessibility call from
|
||||
leaving a probe behind. Window names and accessibility text are untrusted app
|
||||
content, never instructions to an agent.
|
||||
|
||||
**Recommended next implementation:** an isolated Chromium session with typed
|
||||
snapshot/click/type/scroll tools and exact fresh target binding, then individually
|
||||
verified native app actions. Use the headset capture to judge appearance, not to
|
||||
invent a screen-to-window coordinate transform. Direct tool calls must retain
|
||||
approval rules; a generic computer-use tool must not become a route around the
|
||||
MCP approval panel, install confirmation or power confirmation.
|
||||
|
||||
## Isolated browser input proof
|
||||
|
||||
**Verified 2026-09-29, SteamOS 0.4.1, BUILD_ID 20260925.6191901:** a temporary
|
||||
Frame Chromium profile loaded a local test page through an SSH reverse tunnel.
|
||||
CDP `Input.insertText` entered the test string in its own input. A CDP
|
||||
`Input.dispatchMouseEvent` press/release on its own button copied that string
|
||||
to the page's result; DOM readback matched exactly. The browser profile,
|
||||
loopback forwards and panel log were removed afterward. No user app was typed
|
||||
into, no global settings were changed and no third-party helper app was used.
|
||||
|
||||
AT-SPI did **not** expose the test page's controls in that same probe, even with
|
||||
Chromium's renderer-accessibility flag. It returned the partial Steam-client
|
||||
tree instead. The reason remains **unverified**; this is an evidence gap, not
|
||||
proof that Frame accessibility cannot work. For a first implementation,
|
||||
Chromium's proven page-specific CDP route is stronger than assuming complete
|
||||
AT-SPI coverage. This proof does not ship unrestricted click/type tools or
|
||||
establish input delivery to SteamVR's menus.
|
||||
@@ -0,0 +1,47 @@
|
||||
Frame client feasibility, 2026-09-28
|
||||
SteamOS VERSION_ID=0.4.1 BUILD_ID=20260925.6191901; uname -m=aarch64
|
||||
SteamVR: vrserver log reports 2.18.1; process 2256 remained alive across checks.
|
||||
Base: dcf9689f6459e576d35fc507eb702ca6b2bf4dad (main).
|
||||
Unmodified upstream release APKs; installed with ui/frame_android.py install APK --vr.
|
||||
No Linux host attached. No pairing, streamed video, input, audio or worn-headset checks.
|
||||
Selected logcat lines only; timestamps in Android logs are UTC.
|
||||
|
||||
WiVRn-release.apk
|
||||
https://github.com/WiVRn/WiVRn/releases/tag/v26.9
|
||||
sha256=1df6649ec77224fcc821af0ab4897222bdf3d7eb6ce6ad636461336724111331
|
||||
E/OpenXR-Loader( 1140): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
|
||||
I/WiVRn ( 1140): [2026-09-28 12:07:58.987] [WiVRn] [info] Failed to create OpenXR instance version 1.1.58: XR_ERROR_EXTENSION_NOT_PRESENT
|
||||
E/OpenXR-Loader( 1140): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
|
||||
I/WiVRn ( 1140): [2026-09-28 12:07:59.034] [WiVRn] [info] Failed to create OpenXR instance version 1.0.58: XR_ERROR_EXTENSION_NOT_PRESENT
|
||||
E/WiVRn ( 1140): [2026-09-28 12:07:59.035] [WiVRn] [error] Error during initialization: Failed to create OpenXR instance: XR_ERROR_EXTENSION_NOT_PRESENT
|
||||
|
||||
alvr_client_android.apk
|
||||
https://github.com/alvr-org/ALVR/releases/tag/v20.14.1
|
||||
sha256=be68feeb02665e3d69f1cdbcabf38ea4d15c42868a7ec6b5e698dbefee4e4e36
|
||||
E/OpenXR-Loader( 1139): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
|
||||
I/RustStdoutStderr( 1139): Error [GENERAL | xrCreateInstance | OpenXR-Loader] : LoaderInstance::CreateInstance, no support found for requested extension: XR_KHR_convert_timespec_time
|
||||
E/[ALVR NATIVE-RUST]( 1139): ALVR panicked: What happened:
|
||||
E/[ALVR NATIVE-RUST]( 1139): panicked at alvr/client_openxr/src/lib.rs:220:10:
|
||||
E/[ALVR NATIVE-RUST]( 1139): called `Result::unwrap()` on an `Err` value: ERROR_EXTENSION_NOT_PRESENT
|
||||
|
||||
Cleanup verified: both app directories, compatdata directories and Steam shortcuts absent.
|
||||
Both test containers stopped and removed. No Steam/SteamVR restart or global setting changes.
|
||||
|
||||
Valve release notes fetched from ISteamNews/GetNewsForApp/v2 (appid=250820).
|
||||
|
||||
SteamVR Beta Updated - 2.18.1
|
||||
https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1844751498219787
|
||||
Added tethered Quest support over USB (must be used with Steam Link Beta)
|
||||
|
||||
Introducing SteamVR 2.17
|
||||
https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1843481262693486
|
||||
Adds initial support for USB streaming. Note: Requires new Steam Client Beta.
|
||||
Fix crash using Steam Link on Linux when games submit invalid textures.
|
||||
Improve streaming recovery when using Steam Link on Linux.
|
||||
|
||||
SteamVR Beta Updated - 2.17.8
|
||||
https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1842212951314598
|
||||
Fix crash using Steam Link on Linux when games submit invalid textures.
|
||||
Improve streaming recovery when using Steam Link on Linux.
|
||||
Adds initial support for USB streaming.
|
||||
USB streaming can be used without WiFi by opting into the Steam Client Beta.
|
||||
@@ -0,0 +1,128 @@
|
||||
# Mod feasibility test, 2026-09-28
|
||||
|
||||
**Verified observations**, with limits below. Device: `aarch64`, SteamOS
|
||||
`VERSION_ID=0.4.1`, `BUILD_ID=20260925.6191901`; Proton version file:
|
||||
`1788505046 proton-11.0-2c-arm64`; `vrcmd --stats`: SteamVR `2.18.1`.
|
||||
Times below are the Frame's AEST clock.
|
||||
|
||||
## Inventory and allowed content
|
||||
|
||||
`ssh frame 'python3 - owned' < ui/frame_steam.py` returned 868 games before
|
||||
the test. Beat Saber (620980) and Skyrim VR (611670) were absent. Half-Life 2:
|
||||
VR Mod – Episode One (2177750) was installed. Hogwarts Legacy (990080),
|
||||
Horizon Zero Dawn (1151640) and Horizon Forbidden West (2420110) were listed
|
||||
but not installed. The full personal library is not committed.
|
||||
|
||||
**Documented, owner report after these tests:** Alex has Beat Saber on a
|
||||
Quest 2, which is charging. That copy was not accessed or inspected during
|
||||
this test. Its version, transfer path and Frame compatibility remain unknown;
|
||||
Steam ownership is still not established.
|
||||
|
||||
Gravitas's public Steam metadata reported `is_free: true`, developer Galaxy
|
||||
Shark Studios, Windows only. Frame Control's existing Steam helper requested
|
||||
its install. Steam first returned free-license state 3, then config state 7,
|
||||
then queued the download. The helper did not accept state 3; a later read
|
||||
found state 7. The source of that transition was not observed. A later
|
||||
`install 1067310` returned `state: installed`. No purchase occurred.
|
||||
|
||||
UEVR was downloaded from the author's [1.05 release](https://github.com/praydog/UEVR/releases/tag/1.05),
|
||||
not a mirror. `UEVR.zip` was 7,399,455 bytes. Its SHA-256 matched the author's
|
||||
`UEVR.zip.sha256` (a UTF-16 text file):
|
||||
|
||||
```text
|
||||
af4f2f91306802d7ee4e8497d483a547ac8e9a3067dbafb81324100524215d3c
|
||||
```
|
||||
|
||||
Microsoft's Windows x64 .NET runtime and Windows Desktop runtime 6.0.36 ZIPs
|
||||
were downloaded using URLs in the official release metadata. Both SHA-512
|
||||
hashes matched that metadata. They were extracted into a test-only user
|
||||
directory, not installed globally. No other mod manager was used.
|
||||
|
||||
## Half-Life 2 VR: visible setup, not gameplay
|
||||
|
||||
Launched the already-installed Episode One using
|
||||
`steam steam://rungameid/2177750`. The existing manifest recorded build
|
||||
25413453 and a shared base depot from app 658920, build 25413418.
|
||||
|
||||
Selected fresh Steam / SteamVR log lines:
|
||||
|
||||
```text
|
||||
22:07:50 proton waitforexitandrun .../Half-Life 2 VR/ep1vr.exe
|
||||
22:08:03 SetApplicationPid: Setting app steam.app.2177750 PID to 27990
|
||||
22:08:03 Successfully loaded binding file '.../hlvr/cfg/steamvr/bindings_frame.json' for app 'steam.app.2177750'.
|
||||
```
|
||||
|
||||
The game's `episodicvr/console.log` reached `Creating VR hand HUD...`,
|
||||
`Creating VR weapon HUD...` and `Calibrating VR base position`. It also
|
||||
contained missing material/weapon warnings. SteamVR logged a missing
|
||||
`frame_hmd` binding as well as the successful controller binding load;
|
||||
controller input was not exercised.
|
||||
|
||||
`ui/frame_vrshot.py` produced a stereo capture showing the mod's
|
||||
“First time setup” and “Dominant hand” dialog. This establishes visible
|
||||
startup, not a played level or comfortable performance. The capture includes
|
||||
room passthrough and is deliberately not published. The test's `hl2.exe`
|
||||
process was terminated; the pre-existing game installation was preserved.
|
||||
|
||||
## Gravitas and UEVR: injection unverified
|
||||
|
||||
The game was launched through Steam. The injector was then started in the
|
||||
same `steamapps/compatdata/1067310` prefix using the shipped
|
||||
`SteamLinuxRuntime_4-arm64/_v2-entry-point` and Proton 11 ARM64. The first
|
||||
attempt reported:
|
||||
|
||||
```text
|
||||
Application: UEVRInjector.exe
|
||||
Message: You must install .NET to run this application.
|
||||
Architecture: x64
|
||||
App host version: 6.0.35
|
||||
.NET location: Not found
|
||||
```
|
||||
|
||||
With `DOTNET_ROOT` and `DOTNET_ROOT_X64` pointing at the test-only Windows
|
||||
runtime directory, Proton loaded `Microsoft.NETCore.App/6.0.36` and
|
||||
`Microsoft.WindowsDesktop.App/6.0.36`. A 25-second launcher timeout was too
|
||||
short to establish whether the UI worked. A longer attempt produced a
|
||||
625×372 `UEVR` X11 window on display `:1`. This is window creation evidence,
|
||||
not a successful injection. That launcher returned exit 0; this was not
|
||||
treated as proof that a mod worked.
|
||||
|
||||
A combined launch set `PROTON_REMOTE_DEBUG_CMD` to `UEVRInjector.exe` and
|
||||
ran Gravitas's `Drop.exe` in the same runtime/prefix. The process
|
||||
`SkyArk/Binaries/Win64/Drop-Win64-Shipping.exe` and a 1600×900 window titled
|
||||
`SkyArk (64-bit, PCD3D_SM5)` appeared. The launcher exited **1** before an
|
||||
injection or stereo game scene could be verified. Output included:
|
||||
|
||||
```text
|
||||
Proton: Error while copying to ".../windows/system32/amdxcffx64.dll": No such file or directory
|
||||
Error [GENERAL | xrCreateInstance | OpenXR-Loader] : xrCreateInstance failed
|
||||
X Error of failed request: BadWindow (invalid Window parameter)
|
||||
Major opcode of failed request: 10 (X_UnmapWindow)
|
||||
X Error of failed request: XI_BadDevice (invalid Device parameter)
|
||||
Minor opcode of failed request: 28 (X_GetDeviceButtonMapping)
|
||||
```
|
||||
|
||||
These errors do **not** establish an ARM64/FEX incompatibility. Other work
|
||||
was launching apps on the shared Frame. SteamVR's PIDs changed during the
|
||||
experiment; `ps` recorded the replacement `vrserver` and `vrcompositor`
|
||||
starting at **22:13:03**, corroborated by `steamvr.service` journal startup
|
||||
lines. This test did not request a Steam/SteamVR restart. The combined
|
||||
attempt ran at 22:14:17–22:14:27, after that restart. A stable, coordinated
|
||||
session is needed to distinguish launcher/environment problems from game or
|
||||
mod incompatibility.
|
||||
|
||||
## Cleanup and remaining checks
|
||||
|
||||
- No game executable or OpenVR DLL was replaced. No global runtime or power
|
||||
setting was changed by this test. No R.E.A.L., OpenComposite or Beat Saber
|
||||
payload was installed.
|
||||
- Removed the test-only UEVR/.NET directory and downloaded ZIPs from the
|
||||
Frame. Final process checks found no `UEVRInjector.exe`,
|
||||
`Drop-Win64-Shipping.exe`, `ep1vr.exe` or `hl2.exe`. SteamVR was running.
|
||||
- Gravitas and its Steam-created prefix remain installed for a repeat test;
|
||||
the pre-existing HL2 VR install remains. Steam may retain normal shader
|
||||
caches, logs and prefix temporary files.
|
||||
- Still unverified: UEVR injection and removal, R.E.A.L. releases and
|
||||
permissions, OpenComposite, Beat Saber playback on either build, and our
|
||||
own manager's end-to-end install/uninstall. No installer UI is justified
|
||||
by these results. See the [support table and next checks](../mods.md).
|
||||
+22
-5
@@ -56,9 +56,11 @@ counts them while they run.
|
||||
Steam library), then launch, stop, test or remove it. **Report an APK** records
|
||||
whether any APK worked (F-Droid or not: pick a file, type a package, or use an
|
||||
installed app). Your reports are saved on your computer and change the verdicts
|
||||
you see. They aren't uploaded anywhere: the shared database is maintainer-only
|
||||
for now (see [compat-db/README.md](../compat-db/README.md)). Uses the app's bundled
|
||||
`adb`, or yours if you have one.
|
||||
you see. With **Share compatibility results** on (Privacy & updates), they also
|
||||
go to the shared database ([privacy.md](privacy.md),
|
||||
[compat-db/README.md](../compat-db/README.md)). A failed install records
|
||||
itself when the APK was the problem, and after an install the app offers a
|
||||
20-second test. Uses the app's bundled `adb`, or yours if you have one.
|
||||
- **Android display**: pick a running Lepton instance (by the app in it) and set
|
||||
its resolution (Native 1920×1080, or Sharp 2560×1440 with density scaled to
|
||||
match), UI scale (Smaller / Default / Larger, or an exact dpi) and text size
|
||||
@@ -70,6 +72,10 @@ counts them while they run.
|
||||
Runtime picked from the program's header), listed under **Sideloaded titles**
|
||||
with Launch and Remove; see [sideloading.md](sideloading.md). Send typed text, or your computer's clipboard, to the
|
||||
Frame clipboard.
|
||||
- **Mac in the headset** (macOS): show any Mac window, or a whole screen, as
|
||||
its own panel in the headset. Place it with the SteamVR dashboard, click and
|
||||
scroll with the laser, and type on the Mac. Streams hardware H.264 through
|
||||
an SSH tunnel; see [mac-in-headset.md](mac-in-headset.md).
|
||||
- **Flatpaks**: install and remove them (quick picks: Moonlight, Firefox, VLC,
|
||||
Remmina).
|
||||
- **One-click tools**: SSH or SFTP in a terminal window, Steam Link, and remote
|
||||
@@ -148,5 +154,16 @@ npm run dist:win # Windows: installer and .zip
|
||||
npm run dist:linux # Linux: AppImage and .deb, x64 and arm64
|
||||
```
|
||||
|
||||
Pushing a `v*` tag builds all three in GitHub Actions and attaches them to the
|
||||
release (`.github/workflows/release.yml`).
|
||||
Pushing a `v*` tag builds all three in GitHub Actions and attaches them to a
|
||||
draft release (`.github/workflows/release.yml`). Running copies are offered it
|
||||
once you publish it: see [releasing.md](releasing.md).
|
||||
|
||||
## AI agents and assistant
|
||||
|
||||
**Documented:** [the MCP adapter and assistant panel](agents.md) are Frame
|
||||
Control implementations. MCP wraps this HTTP API without API keys. Changes
|
||||
require a separate user approval; power also retains its password prompt. The
|
||||
assistant uses a user-chosen endpoint and sends nothing until the user opts in
|
||||
for a message. Screenshot context is separately opt-in. Model replies cannot
|
||||
operate the headset. Tools → Open assistant opens the page; the linked guide
|
||||
covers putting it in a Chromium panel on the Frame.
|
||||
@@ -33,6 +33,7 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
|
||||
| SteamVR's `steamvr-v4l2cam.service` (`/opt/steamvr/bin/linuxarm64/v4l2cam --output=99`) copies the headset view (the `system.HeadsetView` mirror, one undistorted image) into the v4l2loopback device `/dev/video99` ("SteamVR"), 1920×1080 RGB24. `ffmpeg -f v4l2 -i /dev/video99` reads it at about 70 new frames/s; the first frame read can be black. The Frame's hardware encoder (`iris_encoder`, `/dev/video-enc0`) crashes ffmpeg's `h264_v4l2m2m`, so encode with `libx264 -preset ultrafast -tune zerolatency`: 720p30 takes about 0.7 of a core and 1080p60 about 1.7 (of 8). gamescope also publishes a PipeWire `gamescope` video source, but the Frame's GStreamer has no `pipewiresrc`. **Verified 2026-09-26.** | Frame Control's live video (`/api/stream`) |
|
||||
| Battery: `/sys/class/power_supply/max1720x_bat_7-36` gives µV/µA (current is positive while charging), `time_to_full_now`/`time_to_empty_now` in seconds, and `temp` in tenths of °C. The charger shows up as `tcpm-source-psy-…` (`type=USB`, `usb_type=C PD [PD_PPS]`), for example 12 V × 1.67 A. | Frame Control's battery card |
|
||||
| `vrcmd --stats` reports `activity_level` (3 = standby). | Telling whether the headset is being worn |
|
||||
| **Testing VR apps without wearing the headset.** In standby SteamVR keeps OpenXR sessions hidden, so they render one frame and stop. `vrcmd` (in `/opt/steamvr/bin/linuxarm64`) settings use `section.key`: `vrcmd --set-settings-bool power.pauseCompositorOnStandby 0` and `vrcmd --set-settings-float power.turnOffScreensTimeout 3600`, then `vrcmd --handlewakeup`, keep the compositor running, and the scene app becomes visible. If it stays `visible-blurred`, the Steam dashboard is open: `SteamClient.OpenVR.VROverlay.HideDashboard()` in Steam's `SharedJSContext` (CDP on 8080) closes it. The headset view then captures with `ui/frame_vrshot.py`. Restore afterwards with `--set-settings-bool power.pauseCompositorOnStandby 1` and `--set-settings-float power.turnOffScreensTimeout 5`. The bool setter reads `true` as false, so use 1/0. A Steam launch that stalls in standby at `ShowInterstitials` or `CreatingProcess` (see `console_log.txt`) continues with `SteamClient.Apps.ContinueGameAction(<action id>, "<appid>", "<task>")`. **Verified 2026-09-27.** | Proving VR output remotely, [webxr-chromium.md](webxr-chromium.md) |
|
||||
| The SteamVR dashboard has docking: Float in World, Move, Size, Curvature, controller docking, Theater, Multitasking View. **Inferred** from `/opt/steamvr/resources/webinterface/dashboard/` and not yet driven by hand. | [panels.md](panels.md) |
|
||||
| SteamVR settings live in `~/.config/openvr/config/steamvr.vrsettings`, not under `~/.local/share/Steam/config/`. `dashboard.lastAccessedExternalOverlayKey` names the last panel you used. | Settings tweaks |
|
||||
| The Steam client's journal (`journalctl --user`) carries SteamVR system UI lines such as `[Overlays] Created: …` and `vroverlay_uid<appid>`. It's the quickest way to see panels come and go. | Debugging |
|
||||
@@ -51,13 +52,17 @@ Lepton (Android 11, podman container "lepton-dev") ← its own panel, app 305600
|
||||
| **T3 Code desktop runs natively.** The stock release `T3-Code-0.0.42-arm64.AppImage` in `~/Applications/T3CodeDesktop/` starts with no extra setup: glibc 2.39, `libfuse.so.2`, GTK 3, NSS and libsecret are on the image. `panel-on-frame.sh --name t3code-desktop -- '~/Applications/T3CodeDesktop/T3-Code.AppImage'` gives it its own panel (`valve.steam.desktopgame.2000281357`, `--ozone-platform=x11`). Its bundled server listens on `127.0.0.1:3773` and shows up in onboarding as the `frame` computer, with `passwordStore: gnome-libsecret`. The image has no agent CLI and no `node`. Agents run through the LAN CLIProxyAPI (`llm-proxy.lan:8317`, which resolves on the Frame). Claude Code 2.1.283 comes from `claude.ai/install.sh`, and Codex 0.157.1 from the `codex-aarch64-unknown-linux-musl` release tarball, both into `~/.local/bin`. `with-cliproxy` and a mode-600 `~/.config/cliproxyapi/secrets.env` are copied from the Mac. The wrappers `claude-cliproxy` and `codex-cliproxy` (a `-c model_provider=cliproxy`, `wire_api="responses"`, `env_key="CLIPROXY_API_KEY"`) are set as `providers.claudeAgent.binaryPath` and `providers.codex.binaryPath` in `~/.t3/userdata/settings.json`, and T3 picked that up without a restart. Through the wrappers, `claude auth status` reports `loggedIn: true` (`oauth_token`), and both CLIs answered a prompt with `kimi-k3`. `gamescopectl screenshot` captured another layer (the Lepton T3 app) rather than this panel. `DISPLAY=:0 xwd -id <win>` piped to `ffmpeg` captures the window itself (1920×1080). **Verified 2026-09-26**, BUILD_ID 20260922.6101926. | Running T3 Code as a host on the Frame |
|
||||
| Power actions need `sudo`, which asks for the Developer Mode password over SSH. | Frame Control's power buttons |
|
||||
| **SSH server:** OpenSSH 9.7p1. It offers `publickey,password` (keyboard-interactive is off, PAM on) and also asks `userdbctl ssh-authorized-keys` for keys. OpenSSH ≥ 8.8 rejects SHA-1 `ssh-rsa` signatures by default, so a client whose RSA support is SHA-1 only (the Swift library Citadel, for one) can't log in with the RSA key that devkit pairing installs; use ed25519 (**inferred** from OpenSSH defaults). **Verified 2026-09-27**, BUILD_ID 20260922.6101926. | [iphone.md](iphone.md), `ui/frame_connect.py` |
|
||||
| **A Chromium app window on gamescope's `:0` with its own `STEAM_GAME` becomes a panel, even when started over SSH.** `chromium-xr/chrome --ozone-platform=x11 --app=URL --window-size=1280,720` got a 1920×1080 window, and tagging it produced `[Overlays] Created: valve.steam.desktopgame.<id>` in `~/.local/share/Steam/logs/vrwebhelper_systemui.txt`. Chromium XR decoded a 1080p H.264 WebCodecs stream from the Mac at about 60 fps. XTest events sent to `:0` (libXtst through Python ctypes) did not reach the page. The Frame has `libXtst`, `xprop`, `xwininfo`, `curl` and the `C.utf8` locale. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | [mac-in-headset.md](mac-in-headset.md) |
|
||||
| **Streaming video into a panel: what the Frame adds.** Chromium XR decodes H.264 in **software** (1920×1290 at about 9 Mbit/s took 4–6 ms per frame); its GL is ANGLE → zink → Turnip on the Adreno 750, and it logs `GetVSyncParametersIfAvailable() failed`. An idle page's `requestAnimationFrame` runs at about 140 Hz; when the headset is worn or woken, SteamVR picks the 4320×2160 @ 144 Hz mode. **An unworn headset throttles panel apps** whatever they draw: a local canvas page ran at 58 fps for about 6 s, then 28, then about 15 once in standby (vrserver logs `entering standby`, with `power.pauseCompositorOnStandby 1`). `vrcmd --handlewakeup` gave 137–144 fps for about 2 s, then 36. So a panel's frame rate and compositor delay can only be measured while it's worn. `tailscaled` runs with `--tun=userspace-networking` and cost about 8% of a core at 9 Mbit/s (0.5% over the LAN address). Wi-Fi power saving is on (`iw … get power_save`). **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | [mac-in-headset.md](mac-in-headset.md#measuring) |
|
||||
| **USB-C networking.** Plugged into a Mac, the Frame is a USB network device: macOS names the port "Steam Frame" (here `en9`, 10.86.200.234/29), and the Frame's `usb0` is 10.86.200.233. Ping is about 0.9 ms, and SSH works with the usual host key (`-o HostName=10.86.200.233 -o HostKeyAlias=<tailscale name>`). Steam's Remote Play discovery also broadcasts over it. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. | Frame Control's Mac stream uses it when present ([mac-in-headset.md](mac-in-headset.md)) |
|
||||
| **Tools on the image:** Python 3.12.3, `ffmpeg`, `openssl`, `curl`, `rsync`, `zip`/`unzip`, `flatpak`, `wpctl`, `podman`. **No `adb`.** `steamos` is uid 1000, in `wheel`, and sudoers has `%wheel ALL=(ALL) ALL`, so `sudo -S` takes the Developer Mode password on stdin. **Verified 2026-09-27.** | Running Frame Control's server on the Frame (`FRAME_LOCAL=1`, [iphone.md](iphone.md)) |
|
||||
| **Each Lepton instance is a podman container** named `lepton-steamlaunch-<instance id>`, labelled with its ADB port (`podman ps --format '{{.Names}} {{.Labels.adb_port}}'`). `podman exec <container> /system/bin/sh -c '…'` runs Android's shell inside it with no adb at all (used for `pidof` and `logcat` by the app tester). Running `wm size`/`wm density` that way is untested. **Verified 2026-09-27.** | `ui/frame_android.py`, the iPhone app's display settings |
|
||||
| **Asleep means off the network.** In standby the Frame stops answering on its LAN address, `frame.local` and Tailscale alike (`Host is down`, `No route to host`, timeouts), and ping fails. It was unreachable for about 2.5 hours until woken. Nothing over SSH can wake it. **Verified 2026-09-27.** | Frame Control's offline banner and retries |
|
||||
| **What puts it to sleep is Steam's idle timer**, not logind. The journal shows `steamui_system: Switching to power state: [ k_ESystemPowerState_Sleep ] reason: 'ComputeNextPowerState: active: 3600 < 3600 (k_EACState_Connected)'`, then Steam suspends. SSH work doesn't count as activity. The timers are the client settings `system_idle_suspend_ac_sec` (3600) and `system_idle_suspend_battery_sec` (900); 0 means Never (Settings → Power → Sleep after inactivity). They can be written over DevTools the way the settings page does. logind refuses a `systemd-inhibit --mode=block` sleep lock from an SSH session (`Interactive authentication required`) but accepts one started with `systemd-run --user`. `scripts/keep-awake.sh on|off|status` does both and restores the old timers on `off`. **Verified 2026-09-28**, BUILD_ID 20260925.6191901. Whether Steam's suspend honours the inhibitor on its own is **inferred** (polkit gives `steamos` no `suspend-ignore-inhibit`), not tested. | Keeping the Frame awake for agent work |
|
||||
| **Battery at full on a charger** can read `Discharging` at about 0 W (for example 99 %, 0.0 W, USB-C PD 18 W). Treat under 0.5 W on a charger as "not charging", not "draining". **Verified 2026-09-27.** | Frame Control's battery card |
|
||||
| **The OS image is downloadable.** Valve's recovery images for the Frame are at `https://steamdeck-images.steamos.cloud/recovery/`. The root filesystem inside is btrfs, and it runs as an SSH test target on ARM64 Linux without the headset (`tests/frame-container/frame-image.sh`). **Verified 2026-09-27.** | [recovery-and-images.md](recovery-and-images.md) |
|
||||
| **Boot / recovery menu.** Hold Power ~10 s until the LED goes off, then power on while holding the **AUX button on top of the Power button** (not the volume keys) until a text menu appears. Entries: `Current` (SteamOS-A/B + build), `Previous` (the other A/B slot), `Boot from USB`, `Repair Steam Installation`, `Erase User Data` (factory reset), `ADB mode`, `Battery Ship Mode`. It auto-boots `Current` after a ~15 s countdown. **Volume Up/Down (left side) move, AUX (right side) selects.** For a boot loop, Valve says pick `Previous` (keeps user data); then `Repair Steam Installation`; `Erase User Data` wipes `~` (SSH keys, Tailscale, Flatpaks, T3 setup). Last resort is a full re-image, two ways: (1) USB: write `steamframe-oobe-repair-<build>.img.bz2` to an 8 GB+ USB-C stick (Balena Etcher on the Mac), pick `Boot from USB`, then use "Wipe Device & Install SteamOS" / "Repair SteamOS" (keeps games and personal content) from the recovery desktop; (2) cable/EDL: `steamframe-oobe-repair-qdl-<build>.tar.gz`, run `flash.sh` (Linux) or `flash.cmd` (Windows), then with the Frame off for 10 s hold Power + Vol Up + Vol Down for 10 s and plug it in; it reflashes and reboots. Both images: `https://steamdeck-images.steamos.cloud/recovery/` (build 20260922.5153644, 0.3.0, 3.8 GiB each, no published checksums); local copies in `~/Downloads/steam-frame-recovery/`. File names, checksums and what's inside: [recovery-and-images.md](recovery-and-images.md). Source: Valve's [SteamOS Recovery FAQ](https://help.steampowered.com/en/faqs/view/1B71-EDF2-EB6D-2BB3) and [Installation and Repair FAQ](https://help.steampowered.com/en/faqs/view/65B4-2AA3-5F37-4227), plus a menu photo in [EloiStree/HelloSteamFrame#9](https://github.com/EloiStree/HelloSteamFrame/issues/9). **Inferred** (Valve docs, 2026-09-26); not yet tried on our Frame. | Recovering from a boot loop |
|
||||
| **Boot loop cause: the SteamVR health check.** `steamvr.service` runs `/usr/share/deckard/steamvr-health-check`, which appends `frog:glasses:` to `$XDG_RUNTIME_DIR/steamvr-short-session-tracker` on every failed or <10 s SteamVR run. At 3 it runs `steam-health-check --repair-now`, which **deletes all of `~/.local/share/Steam` (games, login, Developer Mode) and `~/.steam`**, keeping only `registry.vdf`. At 4 it also tries `steamos-bootconf set-mode reboot-other` (fails as the user: `bootenv: Permission denied`). SteamVR normally fails 1–2 times per boot while it waits for the Steam client (`SteamAPI_InitEx failed … Steam is probably not running`, then `fatal stalled cross-thread pipe`). Once Steam has been wiped, it has to re-download a ~210 MB client on every boot, so SteamVR keeps failing, Steam keeps getting wiped and the Frame reboots, in a loop. Also, the Steam updater can deadlock at `Installing update...` (main process blocked writing to the `-child-update-ui` process, which is stuck in `drm_syncobj_array_wait_timeout`). Killing only the `-child-update-ui` process lets the install finish (`package/*.installed` appears). **Fix without sudo:** over USB-C ADB (`adb -s frame shell` works as `steamos` while the Frame is looping; SSH is refused once Developer Mode is lost), truncate both `/run/user/1000/steam{,vr}-short-session-tracker` files and `chmod 444` them (the health check then logs `Permission denied` and does nothing; this is tmpfs, so it resets on reboot). Unstick the updater if needed, let Steam finish installing, then hold Power 10 s and start the Frame normally. `systemctl reboot` over ADB needs interactive auth. After the fix, sign in to Steam and turn Developer Mode back on. **Verified 2026-09-26**, BUILD_ID 20260922.6101926, slot B (clean boot: 0 SteamVR failures, SSH and Tailscale back). | Diagnosing a boot loop |
|
||||
| **Boot loop cause: the SteamVR health check.** `steamvr.service` runs `/usr/share/deckard/steamvr-health-check`, which appends `frog:glasses:` to `$XDG_RUNTIME_DIR/steamvr-short-session-tracker` on every failed or <10 s SteamVR run. At 3 it runs `steam-health-check --repair-now`, which **deletes all of `~/.local/share/Steam` (games, login, Developer Mode) and `~/.steam`**, keeping only `registry.vdf`. At 4 it also tries `steamos-bootconf set-mode reboot-other` (fails as the user: `bootenv: Permission denied`). SteamVR normally fails 1–2 times per boot while it waits for the Steam client (`SteamAPI_InitEx failed … Steam is probably not running`, then `fatal stalled cross-thread pipe`). Once Steam has been wiped, it has to re-download a ~210 MB client on every boot, so SteamVR keeps failing, Steam keeps getting wiped and the Frame reboots, in a loop. Also, the Steam updater can deadlock at `Installing update...` (main process blocked writing to the `-child-update-ui` process, which is stuck in `drm_syncobj_array_wait_timeout`). Killing only the `-child-update-ui` process lets the install finish (`package/*.installed` appears). **Fix without sudo:** over USB-C ADB (`adb -s frame shell` works as `steamos` while the Frame is looping; SSH is refused once Developer Mode is lost), truncate both `/run/user/1000/steam{,vr}-short-session-tracker` files and `chmod 444` them (the health check then logs `Permission denied` and does nothing; this is tmpfs, so it resets on reboot). Unstick the updater if needed, let Steam finish installing, then hold Power 10 s and start the Frame normally. `systemctl reboot` over ADB needs interactive auth. After the fix, sign in to Steam and turn Developer Mode back on. **Verified 2026-09-26**, BUILD_ID 20260922.6101926, slot B (clean boot: 0 SteamVR failures, SSH and Tailscale back). **Seen again 2026-09-28** on BUILD_ID 20260925.6191901, beta client 1790377368. The Frame rebooted by itself while off the network. At 21:13 the check tried `reboot-other` (`bootenv: Permission denied`) and reset the unpacked Steam install, and the tracker had 15 entries. The tracker fix above, applied over SSH, stopped the resets (the checks then log `Permission denied`), but Steam itself kept exiting about 17 s after each start (34 restarts). Cause not found yet. | Diagnosing a boot loop |
|
||||
|
||||
## Debug recipes
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 142 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 125 KiB |
@@ -87,6 +87,20 @@ check (a wrong or missing password refused; the right one reaches `systemctl`),
|
||||
a changed host key refused with "Pair with the Frame again", and a wrong
|
||||
pairing password reported the same way.
|
||||
|
||||
Also verified in the Simulator against the Frame (2026-09-27): the setup screen
|
||||
found the Frame by itself over Bonjour (`frame · 192.168.1.237`); a paired app
|
||||
waiting for a sleeping Frame connected 4 s after it answered; an upload from the
|
||||
app's web view landed in `~/Downloads`; the share sheet offers Save Image
|
||||
(needs `NSPhotoLibraryAddUsageDescription`, now declared); an install link opens
|
||||
the confirm dialog and downloads nothing until Install; Steam Link without the
|
||||
app installed opens its App Store page.
|
||||
|
||||
Things iOS asks the first time: **Local Network** (tap Allow, or the app can't
|
||||
see the Frame), and **Paste** when you send the iPhone's clipboard (tap Allow
|
||||
Paste, or set Settings → Apps → Frame Control → Paste from Other Apps → Allow).
|
||||
Sending text to the Frame's clipboard needs the desktop panel open in the
|
||||
headset, as on the desktop app.
|
||||
|
||||
Not yet exercised: Android display changes through podman (no Android app was
|
||||
running), a real sleep/restart/shut down on the Frame, and a physical iPhone.
|
||||
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
# PC VR streaming from Linux
|
||||
|
||||
**Recommendation, 2026-09-28:** test Valve's current SteamVR/Steam Link path
|
||||
on a Linux gaming PC before building another streamer. Valve now documents
|
||||
Linux streaming fixes and USB support. We have no Linux host attached, so
|
||||
Linux-to-Frame VR streaming remains **unverified here**.
|
||||
|
||||
This is the feasibility and options report for
|
||||
[#24](https://github.com/saphid/frame-control/issues/24), not a shipped streaming
|
||||
feature. Frame Control's features must use our own implementation or standard
|
||||
platform components. WiVRn and ALVR are research comparisons, not dependencies.
|
||||
An optional install shortcut is the most we would offer for a third-party app.
|
||||
Our own streamer requires Alex's choice before implementation.
|
||||
|
||||
## What was checked on the Frame
|
||||
|
||||
**Verified** on 2026-09-28: aarch64, SteamOS **0.4.1**, BUILD_ID
|
||||
`20260925.6191901`, SteamVR **2.18.1**. Version and build are recorded separately;
|
||||
earlier docs associate this build with other SteamOS version labels.
|
||||
|
||||
| Client | Installation | Runtime result |
|
||||
|---|---|---|
|
||||
| WiVRn **26.9**, upstream `WiVRn-release.apk` | API 29, arm64-v8a; installed in its own immersive Lepton instance | OpenXR instance creation fails: missing `XR_KHR_convert_timespec_time`. Both 1.1.58 and 1.0.58 attempts return `XR_ERROR_EXTENSION_NOT_PRESENT` |
|
||||
| ALVR **20.14.1**, upstream `alvr_client_android.apk` | API 26, arm64-v8a; installed in its own immersive Lepton instance | Same missing extension. Client panics at `client_openxr/src/lib.rs:220` with `ERROR_EXTENSION_NOT_PRESENT` |
|
||||
|
||||
The [evidence excerpt](evidence/linux-vr/2026-09-28.txt) includes APK SHA-256s,
|
||||
upstream release links, loader errors and cleanup results. These are failures
|
||||
before an OpenXR session, not successful VR clients. WiVRn was launched twice;
|
||||
ALVR's container remained up despite its client panic. Container liveness alone
|
||||
does not establish VR compatibility.
|
||||
|
||||
Both APKs already declare `MAIN` and `LAUNCHER`. They were installed unmodified
|
||||
using this branch's existing `python3 ui/frame_android.py install APK --vr`,
|
||||
then launched through their Steam shortcuts. Logs came from the instance's
|
||||
`podman exec … /system/bin/logcat`; the user journal also retained WiVRn's errors
|
||||
after its container exited. No headset was worn and no host was connected.
|
||||
|
||||
All test app files, compatdata, shortcuts and containers were removed afterwards.
|
||||
SteamVR's original process remained running. No global settings changed.
|
||||
|
||||
### Relation to the VR APK branch
|
||||
|
||||
**Documented from source:** [PR #20](https://github.com/saphid/frame-control/pull/20)
|
||||
was read, not edited (branch inspected at
|
||||
[`038dcd4`](https://github.com/saphid/frame-control/commit/038dcd48cd75336f6a86c63c7878bfc9c52deec9)).
|
||||
Its compatibility layer handles OpenXR version negotiation, some controller
|
||||
profiles and refresh-rate requests. It does **not** implement
|
||||
`XR_KHR_convert_timespec_time`. Its launcher fix is unnecessary for these APKs.
|
||||
This report has **no unmerged code dependency** on that PR, and neither APK was
|
||||
tested with its layer injected.
|
||||
|
||||
**Documented from upstream source:** WiVRn requests the extension in
|
||||
[`application.cpp`](https://github.com/WiVRn/WiVRn/blob/bbc6e4cc36c355fa6180980abd231673dc15115d/client/application.cpp#L1286)
|
||||
and uses it to convert `CLOCK_MONOTONIC` into `XrTime` in
|
||||
[`instance::now()`](https://github.com/WiVRn/WiVRn/blob/bbc6e4cc36c355fa6180980abd231673dc15115d/client/xr/instance.cpp#L335).
|
||||
ALVR also [requests it unconditionally](https://github.com/alvr-org/ALVR/blob/a9f6542fa507a841f40ab4f3fcb531427cd02550/alvr/client_openxr/src/lib.rs#L188).
|
||||
Simply deleting the extension request or returning made-up timestamps would
|
||||
not prove correct tracking or timing. A real fix needs a valid clock mapping
|
||||
and further runtime tests. No such patch was made.
|
||||
|
||||
### Native SteamOS aarch64 clients
|
||||
|
||||
**Verified:** the Frame has a native OpenXR runtime manifest at
|
||||
`~/.config/openxr/1/active_runtime.json`, pointing to SteamVR's
|
||||
`bin/linuxarm64/vrclient.so`.
|
||||
|
||||
**Documented:** WiVRn's [26.9 README](https://github.com/WiVRn/WiVRn/blob/bbc6e4cc36c355fa6180980abd231673dc15115d/README.md)
|
||||
describes its Linux client as debugging-only, without audio or hardware decode.
|
||||
ALVR 20.14.1's [non-Android decoder](https://github.com/alvr-org/ALVR/blob/a9f6542fa507a841f40ab4f3fcb531427cd02550/alvr/client_core/src/video_decoder/mod.rs)
|
||||
returns no decoded frames. The inspected releases ship Android clients, not a
|
||||
ready-to-run native Frame client.
|
||||
|
||||
**Inferred:** a native port is possible research, but neither release offers a
|
||||
demonstrated native alternative to the blocked APKs. Native builds, native
|
||||
extension enumeration, hardware decoding and audio were **not tested**. The
|
||||
Android extension failure does not establish that the native runtime lacks it.
|
||||
|
||||
## (a) Valve's own path — recommended first
|
||||
|
||||
**Documented**, from Valve's release notes rather than launch-window reports:
|
||||
|
||||
- [SteamVR 2.17.8 beta](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1842212951314598)
|
||||
says “Fix crash using Steam Link on Linux when games submit invalid textures”
|
||||
and “Improve streaming recovery when using Steam Link on Linux.” It also
|
||||
adds initial USB streaming, with Steam Client Beta required to use USB
|
||||
without Wi-Fi.
|
||||
- [SteamVR 2.17 release](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1843481262693486)
|
||||
repeats Linux streaming fixes and initial USB support. USB is no longer
|
||||
solely a claim about an old beta, but version/channel requirements still
|
||||
need checking on the actual host.
|
||||
- [SteamVR 2.18.1 beta](https://steamstore-a.akamaihd.net/news/externalpost/steam_community_announcements/1844751498219787)
|
||||
adds USB-tethered **Quest** support with Steam Link Beta. That entry is not
|
||||
proof of a Frame/Linux combination.
|
||||
- The [Steam Link page](https://store.steampowered.com/app/353380/Steam_Link/)
|
||||
lists Linux desktop clients, while its Quest VR requirements still say
|
||||
Windows 10 or newer. Desktop Steam Link support is not equivalent to VR host
|
||||
support, and the Quest requirements are not a Frame support matrix.
|
||||
|
||||
**Inferred:** Valve has a Linux VR streaming path worth testing. The old blanket
|
||||
claim “Linux cannot stream VR” is no longer justified by the evidence. These
|
||||
release notes do not establish which Linux GPU/driver/Frame combinations work.
|
||||
USB changes the transport; it does not by itself prove host encoder support.
|
||||
|
||||
**Not verified:** Linux host discovery, pairing, wireless or USB streaming,
|
||||
stereo rendering, controllers, haptics, audio, latency, or a game. Frame-only
|
||||
inspection cannot establish any of these. Flat Remote Play and a desktop shown
|
||||
on a panel are not substitutes for this test.
|
||||
|
||||
Next test, once a Linux gaming PC is available: record distro, GPU/driver,
|
||||
Steam client channel/version and SteamVR version; use the Frame's built-in
|
||||
Steam connection flow, first wirelessly and then over a data-capable USB cable.
|
||||
Launch a free OpenXR sample or developer-consented VR game. Verify stereo,
|
||||
head/controller tracking, haptics and audio while worn; retain both ends' logs
|
||||
and measure latency and recovery after a link interruption. Restore any test
|
||||
channel changes. Do not change the shared headset's channel just for this report.
|
||||
|
||||
If that works, Frame Control can provide our own host checks, setup guidance
|
||||
and session controls around Valve's existing platform. First establish which
|
||||
controls have a usable interface; no stable automated pairing API has been
|
||||
verified. **Estimate (inferred):** 2–5 engineer-days for the hardware feasibility
|
||||
pass; another 1–2 weeks for a small integration if those interfaces exist.
|
||||
|
||||
## (b) Our own streaming — proposal only
|
||||
|
||||
This is a new VR transport and device integration, not a desktop capture feature.
|
||||
A plausible first target is **one Linux GPU family, one host, one Frame**, using
|
||||
SteamVR on both ends. Our host driver would expose a remote HMD/controllers,
|
||||
receive poses and inputs, and obtain stereo textures for hardware encoding.
|
||||
Our Frame OpenXR app would decode, submit the correct eye views and render poses
|
||||
at predicted display times, and return tracking/input. SteamVR/OpenXR, bundled
|
||||
codec/transport libraries and platform GPU APIs fit the ownership rule; a
|
||||
WiVRn/ALVR/Monado server dependency would not.
|
||||
|
||||
**Inferred design risks:** Linux SteamVR texture-sharing/driver interfaces and
|
||||
Frame decode-to-GPU interoperability need a spike before committing to this
|
||||
architecture. Sending an already-composited desktop mirror loses the stereo,
|
||||
pose and timing information we need. Late reprojection, clock conversion,
|
||||
backpressure, controller bindings, audio sync and reconnects are substantial
|
||||
work. A runtime shim must not assume `XrTime` equals monotonic nanoseconds.
|
||||
|
||||
### Reuse from `mac-in-headset`
|
||||
|
||||
**Documented from our code**, read-only at
|
||||
[`1b90c64`](https://github.com/saphid/frame-control/commit/1b90c64b73bace54c63a3aae154c5d29a9448d72):
|
||||
|
||||
- Reuse the ideas for low-latency encoding without B-frames, dropping work
|
||||
before encoding, bounded queues, keyframe recovery, adaptive bitrate,
|
||||
per-frame timing and authenticated session setup.
|
||||
- Its VideoToolbox encoder and ScreenCaptureKit capture are macOS-specific.
|
||||
Linux needs a new GPU encoder path (for example VA-API or NVENC via bundled
|
||||
libraries) and VR texture capture, not a port of window capture.
|
||||
- Its WebSocket over SSH is useful for a first controlled transport experiment
|
||||
and control messages. Reliable TCP can stall behind lost packets; a VR media
|
||||
path needs measured deadline behaviour, likely datagrams with loss recovery
|
||||
using an ordinary bundled transport library. Do not invent cryptography.
|
||||
- Its Chromium/WebCodecs panel viewer is not a VR client. That branch reports
|
||||
software H.264 decoding and occasional long Wi-Fi stalls on the Frame.
|
||||
Its desktop latency measurements are not motion-to-photon measurements or
|
||||
evidence that a 90/120 Hz stereo stream will work.
|
||||
|
||||
**Size/effort estimate (inferred, one experienced full-time engineer, hardware
|
||||
available):**
|
||||
|
||||
| Phase | Deliverable / stop condition | Effort |
|
||||
|---|---|---|
|
||||
| Feasibility | Linux driver texture access, Frame hardware decode into OpenXR, pose/clock loop; stop if any cannot meet frame deadlines | 2–4 weeks |
|
||||
| First end-to-end prototype | One GPU/codec, stereo sample over a controlled LAN, head/controllers, logs and teardown | 4–8 additional weeks |
|
||||
| Usable limited beta | Audio/haptics, pairing, recovery, bitrate/loss handling, installer, worn testing and latency work | 6–12 additional weeks |
|
||||
| Wider support | Multiple GPU vendors/distros, USB and Wi-Fi variation, long-session stability | 2–4 additional months |
|
||||
|
||||
Planning range: **12–24 engineer-weeks for a limited beta**, roughly
|
||||
**10–25k lines of our code plus tests/tooling**, excluding bundled libraries.
|
||||
This is a low-confidence scope estimate, not a delivery promise; an unsupported
|
||||
driver or decode interface could block it entirely. Foveated streaming,
|
||||
eye tracking and parity with Valve are excluded. A Linux gaming PC and repeatable
|
||||
worn-headset testing are prerequisites. **Do not build this until Alex chooses.**
|
||||
|
||||
## (c) Optional “install WiVRn” shortcut only
|
||||
|
||||
Allowed as a clearly optional convenience, never a prerequisite for a Frame
|
||||
Control feature. **Documented:** WiVRn's server Flatpak ID is
|
||||
`io.github.wivrn.wivrn`; its client/server versions must match, and its Flatpak
|
||||
includes xrizer/OpenComposite. Those are properties of an independently
|
||||
installed third-party stack, not components of our implementation.
|
||||
|
||||
**Recommendation:** defer the shortcut while the current client fails before
|
||||
session creation. If offered later, label that compatibility result and let
|
||||
the user choose the install; do not present “install” as “streaming works.”
|
||||
**Estimate (inferred):** 1–2 engineer-days for an optional host-side shortcut
|
||||
with package/version detection and honest status, excluding third-party fixes.
|
||||
No shortcut, host install, pairing automation or streaming UI was built here.
|
||||
|
||||
Choose **(a)** for the next hardware test. Keep **(b)** as a separately approved
|
||||
project if Valve's path fails or lacks a required capability. **(c)** does not
|
||||
solve the verified client blocker and should not be the product's foundation.
|
||||
@@ -0,0 +1,470 @@
|
||||
# Mac in the headset
|
||||
|
||||
Frame Control can show any Mac window, or a whole Mac screen, as its own panel
|
||||
in the Steam Frame. You place each panel anywhere in the room with the SteamVR
|
||||
dashboard. The laser clicks and drags, the thumbstick scrolls, and you type on
|
||||
the Mac's own keyboard. Find it under **Tools → Mac in the headset** (macOS
|
||||
only).
|
||||
|
||||
The confidence labels are the same as in [ssh.md](ssh.md).
|
||||
|
||||
## Why this design
|
||||
|
||||
First-party options come first, as the repo's rule asks, with the reason
|
||||
each one was or wasn't chosen. The full list for every device is in
|
||||
[streaming.md](streaming.md#first-party-options-and-why-they-do-or-dont-fit).
|
||||
Checked 2026-09-28.
|
||||
|
||||
| Goal | First-party option | Chosen? | Why |
|
||||
|---|---|---|---|
|
||||
| One Mac screen in the headset | **Apple Screen Sharing** (VNC) → Remmina (Remmina 1.4.43 is already installed on this Frame) | Kept as the fallback (`panel-on-frame.sh mac-screen`) | It's the closest to first-party and needs nothing new. But VNC sends compressed tiles rather than video, so moving content is slow: noticeable lag even on a good 5 GHz link (**verified** 2026-09-27, see [streaming.md](streaming.md)), and the Mac's pointer isn't in the picture without a helper. It shows only whole screens |
|
||||
| One Mac screen | **Steam Remote Play**, Mac as host (Valve) | For Mac games only; see [Steam's own streaming](#steams-own-streaming) | The Mac's and the Frame's Steam clients already find each other (**verified**). But Remote Play streams a game (the whole desktop only while the game is out of focus, untested from a Mac), never single windows, and a Mac can't host the Frame's VR streaming |
|
||||
| One Mac screen | **AirPlay** (Apple) | No | Apple licenses AirPlay receivers only to TV and speaker makers, and nothing official runs on Linux. UxPlay is an unofficial receiver, and it mirrors a whole screen, not single windows |
|
||||
| One Mac screen | **Sidecar / Mac Virtual Display** (Apple) | No | These work only with an iPad or Apple Vision Pro |
|
||||
| **Each Mac window as its own panel** | None | – | No first-party way does this: Apple's per-app streaming is only for Vision Pro, and Valve's desktop streaming needs a Windows SteamVR host. So Frame Control does it itself |
|
||||
| Mac keyboard and trackpad driving the headset | **Bluetooth HID** | No | macOS can't act as a Bluetooth keyboard or mouse. A real Bluetooth keyboard paired with the Frame still works |
|
||||
| Mac keyboard and trackpad | **KDE Connect** (KDE) | No | The Frame has no `kdeconnectd` and it isn't on Flathub. Its Mac app has no keyboard or mouse sharing (**inferred**), and on Wayland it can only reach the desktop panel |
|
||||
| Mac keyboard and trackpad | **xrdp** (Valve, Developer Mode) | No | It runs a separate Linux session that you view on the Mac. It isn't the headset's view, and it doesn't carry input the other way |
|
||||
|
||||
What that leaves is our own stream: nothing to install on the Mac or the
|
||||
Frame, and whole screens or single windows. Here the Mac's own keyboard and
|
||||
trackpad need no forwarding, because the windows are still on the Mac. The
|
||||
laser is the only input that has to be sent back.
|
||||
|
||||
Other routes that were compared:
|
||||
|
||||
| Option | One screen | Each window | Speed | Verdict |
|
||||
|---|---|---|---|---|
|
||||
| Sunshine → Moonlight | ✓ | – | Good | Sunshine's macOS support is still experimental ([discussion #777](https://github.com/orgs/LizardByte/discussions/777)), and it captures whole screens only |
|
||||
| Virtual Desktop, Immersed | – | – | – | No Frame client as of September 2026 |
|
||||
| **Frame Control's own stream** | ✓ | ✓ | Hardware H.264, sending only changed frames | **Built** |
|
||||
|
||||
To type into VR surfaces other than these panels (SteamVR's dashboard,
|
||||
games), the Frame supports a uinput keyboard and mouse without sudo
|
||||
(verified 2026-09-27: `steamos` is in `input`, and `/dev/uinput` is
|
||||
`root:input 660`). That's a separate feature, not part of this one.
|
||||
|
||||
## Steam's own streaming
|
||||
|
||||
The Frame is built around Steam streaming, so this was checked first
|
||||
(2026-09-28). It fits Mac games, not Mac windows.
|
||||
|
||||
- **VR streaming from the Mac: no.** The Frame streams VR from a PC running
|
||||
SteamVR ("Steam Link" with foveated streaming). SteamVR dropped macOS in
|
||||
2020, and Valve lists PCs, laptops, Steam Deck and Steam Machine as
|
||||
hosts, never a Mac (**documented**:
|
||||
[UploadVR](https://www.uploadvr.com/steamvr-drops-mac-support/),
|
||||
[Road to VR](https://roadtovr.com/steam-frame-game-certification-specs/)).
|
||||
- **Flat Remote Play from the Mac: probably, for Steam games.**
|
||||
- Steam on this Mac has streaming on, and the two Steam clients already
|
||||
see each other. The Frame's `remote_connections.txt` shows it
|
||||
connecting directly to "Alexs-MacBook-Pro-7" at 192.168.1.211:27036,
|
||||
and the Mac's shows the Frame connecting over Wi-Fi and over the USB-C
|
||||
link (**verified** in both clients' logs).
|
||||
- Whether a stream then starts, and how a flat game looks in the headset
|
||||
(reviews describe a theater screen), is **not tested yet**. The Frame's
|
||||
Steam was crash-looping during this session (below).
|
||||
- Mac-hosted Remote Play has a long-standing report of the stream
|
||||
closing as the game loads
|
||||
([Steam forum](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/),
|
||||
**reported**).
|
||||
- **The Mac desktop through Steam: untested; single windows: no.** Valve
|
||||
says Remote Play shows the host's desktop when the game loses focus
|
||||
([Steam Remote Play FAQ](https://help.steampowered.com/en/faqs/view/0689-74B8-92AC-10F2),
|
||||
**documented**), so a whole Mac screen may be reachable by starting a
|
||||
game, then switching away from it. Nobody has tried that from a Mac
|
||||
host. It would still be one screen in one panel: Remote Play has
|
||||
nothing like one panel per Mac window, so Frame Control's own stream
|
||||
stays the way to see separate windows.
|
||||
- **Steam has a "stream desktop" call, and it pairs with a Mac
|
||||
(verified 2026-09-28).** In the Remote Play device list, the Frame's
|
||||
Steam UI calls `SteamClient.RemotePlay.StartDesktopStream(<client id>)`
|
||||
for a connected device. Called over CDP with the Mac's client ID, it made
|
||||
the Mac's Steam show "Authorize Device" and ask for a 4-digit code shown
|
||||
on the Frame. Once the code was entered, the Mac logged
|
||||
`k_ERemoteDeviceAuthorizationSuccess`. No stream started in that attempt,
|
||||
and a second attempt, now that the device is authorized, is the next
|
||||
test. If it streams the Mac's desktop, that's a whole-screen option built
|
||||
into Steam: one panel, Valve's encoder and transport. It still wouldn't
|
||||
give each window its own panel.
|
||||
- **What Steam's work did give us: the USB-C link.** Plugged into the Mac,
|
||||
the Frame appears as a network port called "Steam Frame". Steam's Remote
|
||||
Play discovery uses it, and so does Frame Control's stream now (see
|
||||
"USB-C, when it's plugged in" below).
|
||||
|
||||
Next steps, once Steam on the Frame is healthy:
|
||||
|
||||
1. Call `StartDesktopStream` again now that the Frame is authorized, and
|
||||
compare its latency and sharpness with Frame Control's stream of the same
|
||||
screen.
|
||||
2. Stream the one Mac game installed here (Fortune Mill) from the Frame's
|
||||
library, over Wi-Fi and over USB-C.
|
||||
3. Record whether it starts, how it's shown, and its latency. Steam's
|
||||
streaming overlay shows this; our benchmark can't measure it.
|
||||
4. While streaming, switch away from the game on the Mac, and see whether
|
||||
the Mac's desktop appears in the headset, and whether its keyboard and
|
||||
pointer work.
|
||||
5. If it works, Frame Control's Games page could offer "Stream from the
|
||||
Mac" for Mac-installed games.
|
||||
|
||||
## How it works
|
||||
|
||||
```
|
||||
Mac Frame
|
||||
ScreenCaptureKit (one window or display)
|
||||
→ VideoToolbox H.264 (hardware, low-latency,
|
||||
no B-frames)
|
||||
→ frame-mac-view, 127.0.0.1 ──ssh -R──→ 127.0.0.1:479xx
|
||||
→ Chromium app window per stream
|
||||
(WebCodecs decode), on gamescope's
|
||||
X display, tagged STEAM_GAME
|
||||
→ its own SteamVR panel
|
||||
← CGEvent (clicks, drags, wheel, keys) ←──── pointer, wheel and key events
|
||||
```
|
||||
|
||||
- **The agent** is `mac/bin/frame-mac-view`, built from `mac/frame-mac-view`
|
||||
(Swift, no dependencies; `build.sh`). Frame Control's server starts it on
|
||||
first use and stops it on quit.
|
||||
- It captures with ScreenCaptureKit, which sends frames only when something
|
||||
changes, so idle windows cost nothing.
|
||||
- It encodes in hardware with VideoToolbox's low-latency rate control (plain
|
||||
real-time mode where that's unavailable).
|
||||
- While anyone is watching, it keeps the Mac's display awake. A sleeping
|
||||
display isn't drawn, so there would be nothing to capture.
|
||||
- **The link** is an `ssh -R` tunnel on its own connection. It's encrypted and
|
||||
works anywhere `ssh frame` works, Tailscale included, with no firewall
|
||||
changes on the Mac. If the headset sleeps or the network drops, Frame
|
||||
Control reopens the tunnel on the same port, and open viewers reconnect by
|
||||
themselves.
|
||||
- **USB-C, when it's plugged in.** Connected to the Mac by cable, the Frame
|
||||
is also a USB network device: macOS lists a network port called "Steam
|
||||
Frame", and the Frame's `usb0` answers in under 1 ms. Frame Control
|
||||
checks for it each time it opens the tunnel and uses it when it's there,
|
||||
with the Frame's usual SSH host key. Otherwise it uses the normal path.
|
||||
`FRAME_MACVIEW_USB=0` turns this off. **Verified** 2026-09-28, in two
|
||||
interleaved pairs of runs:
|
||||
|
||||
| | USB-C | Wi-Fi (Tailscale) |
|
||||
|---|---|---|
|
||||
| test: content p50 / p95 | 6.9–7.3 / 8.4–8.8 ms | 9.8–10.1 / 12.1–12.3 ms |
|
||||
| test: click to drawn p50 | 16.6–16.9 ms | 26.4–27.8 ms |
|
||||
| scroll: content p95 | 23.0–23.5 ms | 31.4–36.9 ms |
|
||||
| scroll: late frames | 3.2–3.3% | 4.7–7.0% |
|
||||
|
||||
(`bench/results/2026-09-28-*-usb1.json`, `-usb2`, `-wifi1`, `-wifi2`.)
|
||||
- **Access.**
|
||||
- Frame Control's own key never leaves the Mac.
|
||||
- Each viewer is opened with a **single-use ticket**. It's tied to one
|
||||
window or display and expires after a minute. It's spent as soon as the
|
||||
viewer confirms it has received its reconnect key. Until then, a retry
|
||||
gets the same key, so a connection lost at that moment doesn't strand
|
||||
the viewer. Stop revokes tickets that haven't been used yet.
|
||||
- After that, the viewer holds a reconnect key for that one source, in
|
||||
memory only. **Stop** revokes it.
|
||||
- Remaining risk: a program running as `steamos` on the Frame could read a
|
||||
ticket from Chromium's command line in the first second or so and use it
|
||||
first. That gets it the one source being opened, not the Mac, and the
|
||||
real viewer would then fail to connect. Android apps in Lepton run in
|
||||
their own podman container, so they shouldn't see the Frame's process
|
||||
list (inferred, not checked).
|
||||
- **The viewer** is `ui/mac-view.html`, served by the agent. It opens on the
|
||||
Frame as a Chromium app window, preferring Chromium XR (`~/chromium-xr`,
|
||||
built with H.264) over Flathub Chromium.
|
||||
- The page puts `[fcNNNNN]` in its title. The launcher finds the window by
|
||||
that tag and sets `STEAM_GAME` to a stable id per source, which gives it
|
||||
its own panel (see [panels.md](panels.md)). The same Mac window gets the
|
||||
same panel id each time.
|
||||
- It decodes with WebCodecs. If it falls behind, it skips to the next
|
||||
keyframe instead of showing old frames late.
|
||||
- It falls back to JPEG stills (**Compatible** quality) where H.264 isn't
|
||||
available.
|
||||
- **Flow control.** The agent never lets frames queue up anywhere on the
|
||||
way. It skips capture frames *before* encoding, so no reference frame goes
|
||||
missing, and it lowers the bitrate, then the frame rate, then the size, to
|
||||
fit the link (see [Adapting to the network](#adapting-to-the-network)).
|
||||
- **Input.**
|
||||
- A click on a window's panel brings that Mac window to the front
|
||||
(Accessibility API), then clicks at the same point. Double clicks, right
|
||||
clicks, drags and the wheel work too.
|
||||
- Keys typed into the panel are sent as Mac key codes. Any keys or buttons
|
||||
still held down are released if the viewer loses focus or disconnects, or
|
||||
when the stream stops. Characters the key
|
||||
table doesn't know, such as those from other keyboard layouts, are typed
|
||||
as text.
|
||||
- The Mac's own keyboard and trackpad keep working as normal. Click a panel
|
||||
with the laser, then type on the Mac.
|
||||
|
||||
## Permissions (Mac)
|
||||
|
||||
- **Screen Recording**, to see windows. Without it, the card asks for it.
|
||||
- **Accessibility**, so input from the headset reaches the Mac. Without it the
|
||||
stream still works, and the viewer says clicks won't go through.
|
||||
|
||||
Both are granted to Frame Control. After granting, press **Refresh**, which
|
||||
restarts the helper so it picks them up. The app is ad-hoc signed, so macOS
|
||||
may ask again after an update.
|
||||
|
||||
## Quality settings
|
||||
|
||||
| Setting | Long side | fps | Codec | Use |
|
||||
|---|---|---|---|---|
|
||||
| Sharp | 2560 | 60 | H.264, ~0.14 bits/pixel | Text-heavy windows on a strong link |
|
||||
| Balanced (default) | 1920 | 60 | H.264, ~0.1 bits/pixel | Most things |
|
||||
| Light | 1280 | 30 | H.264 | Weak Wi-Fi or Tailscale off the LAN |
|
||||
| Compatible | 1280 | 20 | JPEG | A Frame browser without H.264 |
|
||||
|
||||
## Measuring
|
||||
|
||||
Every frame carries a sequence number, and the agent records its journey on
|
||||
the Mac's clock (`Sources/Stats.swift`):
|
||||
|
||||
| Stage | From → to |
|
||||
|---|---|
|
||||
| capture | the Mac composited it (ScreenCaptureKit's display time) → the agent got it |
|
||||
| queue, encode | → encoding started → the encoder finished |
|
||||
| network | → the viewer received it |
|
||||
| decode, draw | → WebCodecs decoded it → it was drawn on the page's canvas |
|
||||
| present | → the page's next animation frame |
|
||||
|
||||
- **Clock sync.** The viewer syncs its clock to the Mac's the way NTP does:
|
||||
it pings over the stream's own WebSocket and keeps the sample with the
|
||||
shortest round trip. It then reports, in Mac time, when each frame arrived
|
||||
(right away, so the agent can pace itself) and when it was decoded and
|
||||
drawn (in batches every 250 ms).
|
||||
- **Input.** The first frame captured after a click or key carries that
|
||||
event's id. So input latency is the viewer's event → injected on the Mac →
|
||||
the first frame after it → drawn in the headset.
|
||||
- **Where to see it.**
|
||||
- `GET /stats` (key required) returns every frame and input record.
|
||||
- `/status` includes a two-second summary, which Frame Control's card
|
||||
shows next to each live stream.
|
||||
- In the headset, add `?stats=1` to the viewer or press
|
||||
Ctrl+Alt+Shift+S for an overlay.
|
||||
- **The benchmark.** `scripts/macview-bench.py` runs fixed scenarios on the
|
||||
real Frame, from the Mac, with nobody wearing the headset:
|
||||
- **test** is the moving test pattern.
|
||||
- **scroll** is a Chrome page on its own display scrolling at 240 pt/s,
|
||||
which gives about 9 Mbit/s of real 1920×1290 video.
|
||||
- **type** types into a Chrome text box, first fast and then with pauses.
|
||||
|
||||
It writes `bench/results/<date>-<commit>-<label>.json`, compares two
|
||||
results, and runs interleaved A/B tests between agent settings (`ab`).
|
||||
Wi-Fi changes from minute to minute, so single runs at different times
|
||||
aren't comparable. Throttled links come from a shaping relay on the Mac,
|
||||
which needs no sudo (`--net 50@0,3@8,50@16` means 50 Mbit/s, then 3 from
|
||||
8 s, then 50 from 16 s).
|
||||
- **What's graded.** "Content" runs from when the Mac composited a frame
|
||||
(or when ScreenCaptureKit delivered it, if that was earlier) to when it was
|
||||
drawn in the viewer. The Frame's compositor adds its own delay after that.
|
||||
That part is reported, but not graded: an unworn Frame throttles panels to
|
||||
about 36 fps after a few seconds, and to 15 fps in standby, whatever they
|
||||
draw. This was **verified** with a local canvas page that ran on the Frame
|
||||
with no network involved (`bench/pages/present.html`). The compositor's
|
||||
share needs a run with the headset worn.
|
||||
|
||||
Targets: content p50 ≤ 25 ms (p95 ≤ 40), click to photon p50 ≤ 50 ms
|
||||
(p95 ≤ 70), 60 fps with ≤ 1% late frames, no stall over 100 ms, and adapting
|
||||
to a new link rate within 1 s.
|
||||
|
||||
Baseline on 2026-09-28 (**verified**, home Wi-Fi, Tailscale, Balanced,
|
||||
`bench/results/2026-09-28-a3c6e5c-dirty-baseline-fixed.json`; ms p50/p95):
|
||||
|
||||
| Scenario | Content | Input to drawn | fps drawn | Notes |
|
||||
|---|---|---|---|---|
|
||||
| test (1280×720) | 9.9 | 28.8 / 39.0 | 59 | encode 4.0, network 4.0, decode 1.3 |
|
||||
| scroll (1920×1290) | 14.7 | – | 50.4 | encode 6.7, decode 6.4 (software), 9.4 Mbit/s, 16% late |
|
||||
| type (1920×1290) | 16.7 | 51.2 / 73.2 | – | most of the input time is the Mac app reacting |
|
||||
|
||||
After this work, with the controller on (**verified**, same setup,
|
||||
`bench/results/2026-09-28-9e4dcdd-final.json`; ms p50/p95). Content is
|
||||
now measured from the earlier of display time and delivery, which adds
|
||||
about 5 ms to scroll compared with the baseline's way of measuring:
|
||||
|
||||
| Scenario | Content | Input to drawn | fps drawn | Grades |
|
||||
|---|---|---|---|---|
|
||||
| test | 10.5 / 16.7 | 29.6 / 36.3 | 60 | all within target |
|
||||
| scroll | 19.8 / 27.4 | – | 55.9 | fps, late frames (4.8%) and worst gap (222 ms) only "acceptable": Wi-Fi stalls (the Mac captured 57 fps in this run; earlier runs got 46–53 from virtual displays) |
|
||||
| type | 15.0 / 21.4 | 43.0 / 60.5 | – | all within target |
|
||||
|
||||
Of the targets, click to photon is met without the Frame's compositor (the
|
||||
headset has to be worn to measure its share), and so is content latency.
|
||||
The frame-rate and no-stall targets aren't yet met while scrolling.
|
||||
|
||||
**Worn** (**verified** 2026-09-28 22:00, `vrcmd --stats` activity level 1,
|
||||
home Wi-Fi over Tailscale, `bench/results/2026-09-28-39afb23-worn.json`;
|
||||
ms p50/p95):
|
||||
|
||||
| Scenario | Content | Input to drawn | fps drawn | What happened |
|
||||
|---|---|---|---|---|
|
||||
| test | 10.8 / 15.3 | 25.7 / 34.4 | 58 | as unworn |
|
||||
| scroll | 22.2 / 141 | – | 37.7 | Wi-Fi queued 56–364 ms and held frames for 220–270 ms; the controller went to 2.6 Mbit/s and 45 fps |
|
||||
| type | 13.5 / 24.3 | 56.8 / 106 | – | slower replies than unworn (43 / 60) |
|
||||
|
||||
The Frame's CPU wasn't the limit: about 27% in total, and the viewer took
|
||||
45% of one core. The link to a headset on someone's head is much rougher
|
||||
than to one lying still. The adaptation keeps latency bounded there, but
|
||||
the frame rate drops. The USB-C cable avoids Wi-Fi entirely. Frames
|
||||
actually shown were still about 39 fps for the test pattern while worn, so
|
||||
the unworn throttling isn't the whole story. The cause is **unknown**.
|
||||
|
||||
What was learned (all **verified**, unless marked):
|
||||
|
||||
- The biggest costs are encoding (4–7 ms), network (4–6 ms), and decoding
|
||||
on the Frame. Chromium XR on the Frame decodes H.264 in **software**.
|
||||
- Wi-Fi alone stalls for 240–580 ms now and then, over Tailscale and over
|
||||
the LAN alike. Over the LAN (`--host 192.168.1.237`) latency was no
|
||||
better, but the Frame used about 8% less CPU, because tailscaled runs in
|
||||
userspace there.
|
||||
- With no controller, a link that slows down queues without limit. In one
|
||||
run, frames arrived 1.9 s late, and at worst 9.4 s late.
|
||||
- Tried, and no help, so not kept: VideoToolbox options (require hardware,
|
||||
no frame delay, prioritise speed, a hard data-rate cap), Chromium flags
|
||||
(`--disable-gpu-vsync`, `--disable-frame-rate-limit`,
|
||||
`--use-angle=vulkan`), and a 120 Hz virtual display.
|
||||
- Inconclusive, so off by default: keeping the Frame's Wi-Fi awake during
|
||||
typing (`FRAME_MAC_VIEW_WARM=40`, a tiny message every 40 ms for 5 s
|
||||
after input). Over three interleaved runs each, input p50 went 44 → 47 ms
|
||||
and p95 92 → 71 ms, and the ranges overlapped widely
|
||||
(`…-ab-keepwarm.json`). In that run, and in the baseline's typing, the
|
||||
harness typed spaces as "+" (a URL-encoding bug, since fixed), so they
|
||||
went through the text path rather than as space keys.
|
||||
- Pointer moves now go out on an 8 ms timer, not on the page's next
|
||||
animation frame, which an unworn Frame slows to 15–36 Hz. This is
|
||||
**inferred** to help dragging; the benchmark has no drag scenario yet.
|
||||
- Kept: the encoder's timestamps never jump more than two frame intervals.
|
||||
Before this, the first frame after a pause got a quarter of a second's
|
||||
bit budget, and one P-frame reached 204 KB.
|
||||
|
||||
## Adapting to the network
|
||||
|
||||
`Sources/Controller.swift`, per stream, latency first:
|
||||
|
||||
- **The gate.** The viewer acknowledges every frame as it arrives. A new
|
||||
frame is sent only while the oldest unacknowledged one is younger than
|
||||
the path's usual round trip, plus one frame interval, plus room for this
|
||||
link's normal jitter (1.5 times its recent spread, 25–80 ms). So frames
|
||||
never queue in SSH, TCP or the Wi-Fi driver. While the link is stuck, the
|
||||
newest picture waits and goes out as soon as it moves.
|
||||
- **The bitrate.** The link counts as congested when, for two checks in a
|
||||
row (100 ms apart), round trips grow by more than 40 ms while the stream
|
||||
uses much of its budget, or the gate holds frames back, or a frame is
|
||||
stuck for 100 ms. Then the bitrate drops to a bit under what actually got
|
||||
through: at least a fifth off, and at most half. Once the link has been
|
||||
clear for a second, it rises by 10% steps, never above the quality
|
||||
setting's bitrate.
|
||||
- **The tier.** When the bitrate stays low, and the stream is really
|
||||
limited by the link rather than having little to send, it steps down:
|
||||
60 → 45 → 30 fps, then 75%, then 50% of the pixels. It goes straight to
|
||||
the tier the bitrate supports after half a second, and steps back up one
|
||||
tier at a time after two seconds with room to spare.
|
||||
- `FRAME_MAC_VIEW_ADAPT=0` turns it off, for comparison.
|
||||
|
||||
Measured on the real Frame, 2026-09-28 (**verified**; interleaved A/B, off
|
||||
versus on, medians of the runs, ms):
|
||||
|
||||
| Link | Scenario | Content p95, off → on | fps drawn, off → on | Result file |
|
||||
|---|---|---|---|---|
|
||||
| Clean Wi-Fi (3 runs each) | scroll | 32.3 → 33.6 | 57 → 56.2 | `…-ab-adapt-clean2.json` |
|
||||
| Clean Wi-Fi | test | 15 → 14.2 | 60 → 59.7 | same |
|
||||
| 50 → 3 → 50 Mbit/s at 8 s and 16 s (2 runs each) | scroll | **4670 → 72** | 45 → 42 | `…-ab-adapt-step3.json` |
|
||||
| 50 → 3 → 50 Mbit/s | test | 66 → 16 | 60 → 60 | same |
|
||||
|
||||
- **Clean link.** On a clean link it costs nothing measurable. An earlier
|
||||
version with a fixed gate slack lost 9 fps to Wi-Fi jitter while
|
||||
scrolling (47 → 38 fps), and that's why the slack now follows the link's
|
||||
jitter.
|
||||
- **Throttled link.** Without the controller, frames queued for up to 5.6 s
|
||||
and never caught up while the link was slow (p95 1.8–5.6 s, second by
|
||||
second). With it, in the two runs:
|
||||
|
||||
| | Run 1 | Run 2 |
|
||||
|---|---|---|
|
||||
| Worst second's p95 just after the drop | 219 ms | 428 ms |
|
||||
| p95 back under 100 ms for 3 s in a row | after 1 s | after 4 s |
|
||||
| Stepped down to 1440 px at 30 fps | 1.8 s after the drop | 3.5 s after |
|
||||
| p95 per second after that, on the 3 Mbit/s link | 55–94 ms | 60–148 ms |
|
||||
|
||||
Sending one frame takes about 27 ms on that link by itself. After the
|
||||
link recovered, the stream was back at full size and 60 fps in about
|
||||
7.5 s. It steps up one tier every two seconds, on purpose, so it doesn't
|
||||
bounce. The 1 s adaptation target was met in one run of two.
|
||||
- **A hiccup on a small stream.** In one clean run, a Wi-Fi hiccup made an
|
||||
earlier version halve the test pattern's bitrate five times and drop it
|
||||
to half size. That cut couldn't help: the stream only sends
|
||||
0.47 Mbit/s. Now the controller estimates what a stream wants (captures
|
||||
per second × average frame size). While a stream wants about half its
|
||||
budget or less, no cut takes it below twice what it wants, so it doesn't change
|
||||
tier.
|
||||
|
||||
## Checked so far (2026-09-28)
|
||||
|
||||
- **Mac (checked by hand, macOS 26.5.2).**
|
||||
- The agent builds, and lists windows and displays.
|
||||
- In a browser, the test pattern decoded at about 60 fps (H.264) and 30 fps
|
||||
(JPEG, about 22 Mbps).
|
||||
- A click in the viewer arrived at the same point in the source.
|
||||
- `pmset -g assertions` showed the display-awake assertion only while a
|
||||
stream was being watched.
|
||||
- **Automated** (`tests/test_macview.py`, on CI's macOS runner): the agent
|
||||
builds (a build failure fails the job).
|
||||
- `/ping` and the page are open; everything else needs the key.
|
||||
- Tickets work once and only for their own source, and Stop revokes
|
||||
reconnect keys.
|
||||
- A WebSocket frame claiming 2^63 bytes closes that socket, and the agent
|
||||
keeps running.
|
||||
- A stream sends an SPS-led H.264 keyframe, and sends another when asked.
|
||||
- Stop ends the stream on the Mac even if the viewer ignores it.
|
||||
- The test doesn't decode video, time it, or check where clicks land.
|
||||
- **Linux aarch64 (verified in a stand-in, not on the Frame).** In an Arch
|
||||
Linux ARM container with sshd, Xvfb as `:0` and Chromium 153:
|
||||
- The real `show` path worked in 1–2.3 s: tunnel, ticket, launcher,
|
||||
window found and tagged `STEAM_GAME`.
|
||||
- Frame Control's key didn't appear anywhere in the stand-in's process
|
||||
list.
|
||||
- After the tunnel was killed, it came back on the same port within 4 s,
|
||||
and the viewer reconnected by itself.
|
||||
- Chromium decoded the stream in software with the GPU off.
|
||||
- Stop closed the window, and Chromium exited.
|
||||
- This test found and fixed a bug: in a C locale, `xwininfo` can't print a
|
||||
title with non-ASCII characters, so the launcher reads `_NET_WM_NAME`
|
||||
with `xprop`.
|
||||
- **On the Frame (verified 2026-09-28, SteamOS build 20260925.6191901,
|
||||
from the Mac, nobody wearing the headset).** Frame Control's **Show** with
|
||||
the test pattern:
|
||||
- The panel was ready in 1.45 s. SteamVR logged `[Overlays] Created:
|
||||
valve.steam.desktopgame.2001639889` (in `vrwebhelper_systemui.txt`).
|
||||
- The window was tagged `STEAM_GAME`, and gamescope sized it to 1920×1080
|
||||
although 1280×720 was asked for.
|
||||
- Chromium XR decoded it live at about 60 fps: the frame counter advanced
|
||||
62 in 1.04 s.
|
||||
- Over home Wi-Fi, the frames in the viewer window had been drawn on the
|
||||
Mac about 11–17 ms earlier, plus `xwd`'s own time. That's measured
|
||||
against the two clocks, which were 37–39 ms apart (±4 ms, measured over
|
||||
one SSH session). SteamVR's compositor and the display come on top.
|
||||
- Clicks and keys injected with XTest into gamescope's Xwayland didn't
|
||||
reach the page. That's inconclusive, not a failure: XTest on gamescope
|
||||
isn't how real input arrives. The laser should arrive as a left mouse
|
||||
button and the thumbstick as a wheel, because the window has an app id
|
||||
(from gamescope's source; see [panels.md](panels.md)).
|
||||
- **Not yet checked:**
|
||||
- Clicking, dragging and scrolling with the laser while wearing the
|
||||
headset.
|
||||
- Real window capture and input on a Mac with both permissions granted.
|
||||
- Keys from SteamVR's on-screen keyboard.
|
||||
- Whether Flathub Chromium has H.264. Chromium XR is used when it's
|
||||
installed, as it is on this Frame.
|
||||
- Latency with real, busy windows at Sharp.
|
||||
- A benchmark run while wearing the headset. Only then does the Frame show
|
||||
panels at full rate, so only then can the compositor's share of the
|
||||
latency, and the frame rate you actually see, be measured.
|
||||
|
||||
## Limits
|
||||
|
||||
- Only windows on the Mac's current desktop (Space) are listed, and
|
||||
minimised windows can't be captured.
|
||||
- A window's panel shows only that window. Its menus and sheets are separate
|
||||
windows on the Mac, so open them from the Mac or use **Whole screen**.
|
||||
- Keys go to whichever Mac window is in front. Clicking a panel brings its
|
||||
window to the front first.
|
||||
- Ctrl stays Ctrl. On the Mac, copy is ⌘C, so use Meta+C on a keyboard paired
|
||||
with the Frame.
|
||||
@@ -0,0 +1,200 @@
|
||||
# Real separation of Mac windows in the headset
|
||||
|
||||
Research and experiments on 2026-09-28 (macOS 26.5.2, Apple M5 Pro), for
|
||||
making each streamed Mac window truly independent. Builds on
|
||||
[mac-in-headset.md](mac-in-headset.md). Labels: **verified** (tried here),
|
||||
**documented**, **source** (read in someone's code) or **reported**.
|
||||
|
||||
## What "separate" lacks today
|
||||
|
||||
Today each window is captured on its own
|
||||
(`SCContentFilter(desktopIndependentWindow:)`), but it still lives on the
|
||||
Mac's one desktop:
|
||||
|
||||
- **Clicks.** A click has to raise the window first, so it reorders the
|
||||
Mac's windows, steals focus and moves the real cursor.
|
||||
- **Child windows.** A window's menus, popovers, sheets and tooltips are
|
||||
separate windows, so they aren't in its panel. Apple documents that the
|
||||
single-window filter leaves them out.
|
||||
- **Size.** A panel's size is tied to the window's size on the Mac screen.
|
||||
- **Hidden windows.** Minimised windows, and windows on other Spaces, can't
|
||||
be shown live.
|
||||
|
||||
## How the Mac draws, and where pixels can be read
|
||||
|
||||
Apps draw with Core Animation into IOSurfaces. WindowServer's compositor
|
||||
stacks every window onto each display, including virtual ones, and sends
|
||||
the result to the screen. Pixels can be read at three points:
|
||||
|
||||
| Where | How | Gets | Doesn't get |
|
||||
|---|---|---|---|
|
||||
| One window's content | ScreenCaptureKit `desktopIndependentWindow` (public, what we use) | Live, zero-copy, even when covered by other windows (**documented**) | Menus, popovers, sheets. It pauses while the window is minimised (**reported**) |
|
||||
| One window plus its children | `SCStreamConfiguration.includeChildWindows` (macOS 14.2+, **reported**) | Menus, popovers and sheets attached to the window | Anything the app puts outside the window's bounds |
|
||||
| A whole display | ScreenCaptureKit display filter, with apps or windows included or excluded | Everything on that display, cursor included | Only what's on that display |
|
||||
| A snapshot of any window | Private `CGSHWCaptureWindowList`, which AltTab uses for minimised windows and other Spaces (**source**: `alt-tab-macos` `PrivateApis.swift`) | Minimised windows and other Spaces | It's one still image, not a live stream |
|
||||
| Another app's layer tree | Private `CALayerHost` with a context id | A live, zero-copy picture | It only works when the other app cooperates, so it's no good for arbitrary windows (**reported**) |
|
||||
|
||||
`CGWindowListCreateImage` and `CGDisplayStream` are obsolete in the macOS 15
|
||||
SDK (**reported** by MacPorts and JUCE). Nothing reads another app's pixels
|
||||
without the Screen Recording permission.
|
||||
|
||||
## The idea: each window gets its own virtual display
|
||||
|
||||
A virtual display (the private `CGVirtualDisplay`, as used by BetterDisplay,
|
||||
DeskPad and quest-display) is a compositor target with no physical screen.
|
||||
Put one streamed window alone on its own virtual display, sized to its
|
||||
panel, and capture the whole display:
|
||||
|
||||
- **Nothing can overlap it.** A plain click at that point always lands on
|
||||
that window, so input doesn't need raising or background-event tricks.
|
||||
- **Menus, sheets, popovers, tooltips and context menus appear on the same
|
||||
display, so they're in the panel.** With "Displays have separate Spaces"
|
||||
on (it is on this Mac), each display also has its own menu bar, so the
|
||||
app's menu bar can be part of the panel.
|
||||
- **The panel's size is the display's size,** in HiDPI. Resizing the panel
|
||||
means changing the display mode and resizing the window to fill it
|
||||
(Accessibility API).
|
||||
- **Minimised windows and other Spaces stop being a problem,** because
|
||||
streamed windows live on their own displays.
|
||||
- **The cursor goes where the laser points.** That display's panel is the
|
||||
one being used, much as Vision Pro's Mac Virtual Display works. Keys from
|
||||
the Mac's keyboard go to the window last clicked.
|
||||
|
||||
### Verified here (no permission needed)
|
||||
|
||||
- **Creating one.** It needs no permission or entitlement. 1920×1080 HiDPI
|
||||
gives a 3840×2160-pixel, 60 Hz display, placed next to the built-in
|
||||
screen, and `NSScreen.screensHaveSeparateSpaces` was true.
|
||||
- **Many at once.** 16 were created at once with no error.
|
||||
- **Placement.** `CGConfigureDisplayOrigin` far away failed with error
|
||||
1014: macOS keeps displays edge to edge. Disabling one with the private
|
||||
`CGSConfigureDisplayEnabled` from a command-line tool also failed with 1014.
|
||||
- **Removal is deferred while the physical screen sleeps.** With the
|
||||
built-in display asleep, virtual displays were not removed when released,
|
||||
or when their process exited, even from their own `.app`. A second display
|
||||
with the same vendor, product and serial couldn't be created. All of them
|
||||
disappeared as soon as the screen woke (`caffeinate -u`). The helper must
|
||||
therefore:
|
||||
- keep a fixed pool of displays and reuse them rather than create new ones;
|
||||
- keep the screen awake while they exist, which it already does while
|
||||
streaming.
|
||||
|
||||
### Built: "Give each window its own display"
|
||||
|
||||
Frame Control's helper now does this (`mac/frame-mac-view/Sources/Separate.swift`),
|
||||
and it's the default in the Tools card. Streams of this kind are named
|
||||
`separate:<window id>`.
|
||||
|
||||
- For each window shown, it creates a HiDPI virtual display. The display is
|
||||
sized to the window plus a menu bar, with a fresh serial each time, so a
|
||||
display left over while the screen slept can't block a new one.
|
||||
- The window is moved onto the display and resized to fill it (Accessibility
|
||||
API), and the whole display is captured.
|
||||
- On **Stop** the window goes back where it was, and the display is released.
|
||||
- Plain per-window capture is still there: untick "Give each window its own
|
||||
display". Separate mode needs Accessibility (to move the window), and
|
||||
without it, Show says so rather than quietly falling back.
|
||||
|
||||
Verified 2026-09-28 (macOS 26.5.2, M5 Pro), using a TextEdit test document
|
||||
and the benchmark's Chrome windows:
|
||||
|
||||
- **Separation works.** The window moved onto its own display and streamed,
|
||||
with its first frames in 3.8 s. The display showed TextEdit's own menu bar.
|
||||
- **Child windows come along.** A context menu and the Page Setup sheet
|
||||
opened on the same display and were in the capture.
|
||||
- **Input.** Typing through the stream reached the window. The benchmark's
|
||||
typing scenario does this on every run.
|
||||
- **Stop.** Stop put the window back at exactly its old size (656×422), and
|
||||
the display was removed.
|
||||
- **The helper must run a real Cocoa event loop.** Earlier, it ran only a
|
||||
`RunLoop`. With that, AppKit never learned about new displays:
|
||||
- `NSScreen.screens` never listed them, so placing the window always waited
|
||||
3 s and then gave up;
|
||||
- the display's HiDPI mode never applied, so windows were captured at 1×
|
||||
(1280×860 instead of 1920×1290 pixels) and text was soft.
|
||||
|
||||
Running `NSApplication` (with no Dock icon) fixed both. The screen now
|
||||
appears within 100 ms, and captures are at 2× (**verified** in
|
||||
`bench/results/*-baseline.json` against `*-baseline-fixed.json`).
|
||||
- **Frame rate.** Chrome drew at 60–61 fps on the virtual display, but
|
||||
ScreenCaptureKit delivered only about 46–53 fps from it, whereas the
|
||||
built-in ProMotion screen gave 121 fps (**verified**). Neither a 120 Hz
|
||||
virtual display (`FRAME_MAC_VIEW_VD_HZ=120`) nor a looser
|
||||
`minimumFrameInterval` changed that. Cause unknown.
|
||||
- **Windows that keep their own size** (Calculator, 230×408). The window
|
||||
moved and streamed, but stayed small in the display's corner, so the panel
|
||||
was mostly wallpaper. Now only that corner is captured
|
||||
(`SCStreamConfiguration.sourceRect`): the menu bar and the window, at least
|
||||
480×360 points so menus fit. Clicks map to the cropped area. The first
|
||||
frame arrived in 0.6–1.1 s, and on Stop the window went back and the
|
||||
display was removed. A display's menu bar names the active app, so it
|
||||
shows the window's own menus only once the panel has been clicked.
|
||||
- **Several windows at once** (on the Mac, with a local stand-in viewer that
|
||||
acknowledges frames). Three and four Chrome windows, each scrolling on its
|
||||
own display at 1920×1290 pixels, streamed at 53–56 fps and about
|
||||
9 Mbit/s each. The helper used 20% (three) or 24% (four) of one CPU core.
|
||||
The hardware encoder is shared: its time per frame went from 6.7 ms for
|
||||
one stream to 7–15 ms for three and 11–25 ms for four, so each extra
|
||||
moving window adds latency to the others. Once in four runs, before a fix,
|
||||
one window left its display when several displays appeared at the same
|
||||
moment, and its stream stopped: nothing on that display was changing any
|
||||
more. The helper now checks every second and puts the window back. Three
|
||||
further runs with four windows were clean, and every display was gone
|
||||
afterwards (checked with `CGGetOnlineDisplayList`).
|
||||
- **Quitting.** On SIGTERM, or when Frame Control goes away, the helper ends
|
||||
every stream first, so windows go back before their displays disappear.
|
||||
Verified: a 900×600 window was back at 900×600 after SIGTERM. Closing a
|
||||
viewer 0.05–1.5 s into startup left the window at its old size and no
|
||||
extra display.
|
||||
- **A consent prompt.** macOS 26 asks whether to let ScreenCaptureKit apps
|
||||
"bypass the system private window picker". The prompt appeared *on the
|
||||
virtual display*, so it would show up inside the headset panel. Allow it
|
||||
once on the Mac.
|
||||
|
||||
### Still to check
|
||||
|
||||
1. That a context menu opens over the text being clicked, not just somewhere
|
||||
on the display. This needs a retest after the consent prompt above is
|
||||
allowed.
|
||||
2. Stage Manager, which is reported to undo Accessibility resizes.
|
||||
3. Where the Dock and the cursor go on a virtual display.
|
||||
4. Closing the lid with virtual displays attached (clamshell).
|
||||
5. Many moving windows at once in the headset: whether to lower the bitrate
|
||||
or frame rate of panels you aren't using, so the one you are using keeps
|
||||
the encoder to itself.
|
||||
6. All of it in the headset, with the laser.
|
||||
|
||||
## Input without disturbing the Mac (a finer option)
|
||||
|
||||
For windows that stay on the Mac's own screen, input can go to a window
|
||||
behind others without raising it:
|
||||
|
||||
- **Background click.** `CGEventPostToPid` with the CGEvent fields 91 and
|
||||
92, which say which window a click is for (`kCGMouseEventWindowUnderMousePointer`
|
||||
and `...ThatCanHandleThisEvent`), plus a window-relative location
|
||||
(**reported**, reverse-engineered, needs testing).
|
||||
- **Focus without raise.** yabai makes a window key without raising it by
|
||||
posting event records with the private `SLPSPostEventRecordTo` (**source**:
|
||||
`yabai/src/window_manager.c`).
|
||||
- **Known failures.**
|
||||
- Chromium and Electron ignore background clicks unless they're built
|
||||
with the 2026 `acceptsFirstMouse` fix (electron/electron#54493).
|
||||
- Games and canvas apps often need real activation.
|
||||
- Password fields (Secure Event Input) drop synthetic keys.
|
||||
- IME composition, as for Chinese or Japanese input, isn't reliable.
|
||||
|
||||
With a virtual display per window, most of this isn't needed. It's worth
|
||||
having for hovering over one panel while typing in another.
|
||||
|
||||
## Encoding and transport
|
||||
|
||||
- **Codec.** Keep H.264 4:2:0. Apple's High Performance Screen Sharing uses
|
||||
4:4:4 over two virtual displays (**documented**), but the Frame's Chromium
|
||||
decodes H.264 in software, and no 4:4:4 decode path is confirmed there.
|
||||
- **Sharper static text.** When a panel has been still for a moment, send a
|
||||
high-quality refresh, either a keyframe at low quantisation or a lossless
|
||||
WebP overlay, so static text is sharp. Moving content stays as video.
|
||||
- **Transport.** Keep WebSocket over SSH for now, as it works everywhere.
|
||||
WebRTC (UDP, congestion control) is the proven step up for Wi-Fi.
|
||||
WebTransport over QUIC is promising, but its server side on macOS is
|
||||
unverified.
|
||||
+101
@@ -0,0 +1,101 @@
|
||||
# Flat-to-VR mods and Beat Saber songs
|
||||
|
||||
**Status: feasibility work, not an installer.** Frame Control does not yet
|
||||
manage mods or Beat Saber songs. [Issue #26](https://github.com/saphid/frame-control/issues/26)
|
||||
stays open: neither UEVR injection nor Beat Saber custom-song playback has
|
||||
been verified on this Frame. Alex has an owned copy on a Quest 2; that copy
|
||||
has not been inspected. There is no Mods button until the underlying
|
||||
install, playback and removal have been checked.
|
||||
|
||||
The mod manager must be Frame Control's own implementation. Mods and songs
|
||||
are permitted third-party content; BSManager, ModsBeforeFriday, MO2 and other
|
||||
managers must not be dependencies. Steam, Proton and SteamVR remain platform
|
||||
dependencies. No game purchases, entitlement bypasses, withdrawn builds or
|
||||
unofficial mod mirrors are part of this work.
|
||||
|
||||
## Per-game support
|
||||
|
||||
Checked 2026-09-28 on SteamOS **0.4.1**, BUILD_ID **20260925.6191901**, aarch64,
|
||||
with **Proton 11.0-2c ARM64** and SteamVR **2.18.1**. “Verified” describes only
|
||||
the observation stated, not a promise that the game is playable. “Documented”
|
||||
means an upstream source describes it; “inferred” means it still needs a test.
|
||||
|
||||
| Game / build | Mod or content | Evidence and support status | Next check |
|
||||
|---|---|---|---|
|
||||
| Half-Life 2: VR Mod – Episode One, Steam 2177750, build 25413453 | Official Steam community mod, shared base depot 658920 build 25413418 | **Verified: startup only.** Already installed; launched through Proton ARM64. The stereo headset capture showed its first-time setup, and SteamVR loaded `bindings_frame.json`. Gameplay, controller interaction, fresh installation and removal are unverified. | Complete first-time setup and play a level before offering a tested install shortcut. |
|
||||
| Gravitas, Steam 1067310, Windows | UEVR 1.05 | **Verified: prerequisites and windows only.** Free Steam install completed. The game produced a `SkyArk (64-bit, PCD3D_SM5)` window. UEVR needed .NET; with official .NET 6.0.36 libraries it produced a `UEVR` window. A combined run exited 1 with X11 errors before injection was verified. **Inferred: compatibility remains unknown**, not proven broken. | Retry during a stable headset session; verify injection, stereo scene output, controls and removal. |
|
||||
| Beat Saber, Steam 620980, Windows / Proton | Basic custom songs; later SongCore and version-matched mods | **Verified: absent from the 868-game library returned by this Frame.** Store metadata lists Windows, not Linux. **Documented:** the PC game reads basic maps from `Beat Saber_Data/CustomLevels` without a mod manager. Playback on Frame is unverified. | An already-owned, legitimately installed copy is required. Do not buy it as part of this task. |
|
||||
| Beat Saber, claimed native ARM64 build | Custom songs / native mods | **Inferred: unverified.** The research mentions this build but supplies no verified official distributable or tested layout. CPU architecture alone does not identify Android versus Linux, the game version or the mod ABI. | Establish official provenance, ownership, binary type and version before touching files. Do not apply Quest patches to an unidentified build. |
|
||||
| Beat Saber, Alex's Quest 2 copy | Custom songs / Android mods | **Documented: owner-reported copy on Quest 2**, currently charging. No APK, version or installed mods inspected; no Frame playback verified. This does not establish ownership of the Steam build. | When the Quest is available, inspect the owned copy's version and supported transfer path, then test Lepton/OpenXR compatibility without bypassing entitlement checks. |
|
||||
| Hogwarts Legacy, Steam 990080 | R.E.A.L. | **Verified: listed in this Frame's owned library, not installed.** Official release access and redistribution permission were not established; the referenced author Patreon page returned HTTP 403. No archive downloaded or game tested. | Obtain a current free release from the author and confirm its terms before any test. A news report saying “free” is not a redistribution grant. |
|
||||
| Horizon Zero Dawn, Steam 1151640; Horizon Forbidden West, Steam 2420110 | R.E.A.L. | **Verified: both listed as owned, neither installed.** Same source/permission blocker as above; runtime support is unverified. | Check each game's supported version against an accessible official release. |
|
||||
| Half-Life 2 VR / other OpenVR games | OpenComposite, per-game replacement | **Documented:** forwards OpenVR calls to OpenXR. **Inferred: Frame compatibility unknown.** Not installed or tested. HL2 VR reached setup with the shipped OpenVR path already. | Test a specific game and replacement DLL only if needed; preserve its original DLL. Never switch the shared headset's runtime globally. |
|
||||
| Doom / Quake / Half-Life Team Beef ports | Author's VR ports plus separately owned or free game data | **Inferred: untested.** Android ARM64 support does not establish OpenXR extension or controller compatibility on Lepton. | Choose an official release and legally usable data set, then test that exact port. |
|
||||
| Skyrim VR, Steam 611670 | SKSEVR / HIGGS / PLANCK stack | **Verified: Skyrim VR is absent from this library.** Owning flat Skyrim or Special Edition is not the VR game's entitlement. Runtime and mod support are unverified. | An already-owned VR copy and version-matched official mod releases are required. |
|
||||
|
||||
The [test record](evidence/mods-2026-09-28.md) distinguishes process startup,
|
||||
visible output and failures. It also records a SteamVR restart during the
|
||||
shared session, which prevents attributing the failed UEVR attempt to FEX.
|
||||
|
||||
## Beat Saber: songs first
|
||||
|
||||
**Documented:** the [BSMG PC guide](https://bsmg.wiki/pc-modding.html)
|
||||
describes extracting each map into its own directory below
|
||||
`Beat Saber/Beat Saber_Data/CustomLevels`. Basic custom songs do not require
|
||||
SongCore; maps that require mod features need their matching dependencies.
|
||||
This is a candidate for our own file manager, not a verified Frame feature.
|
||||
|
||||
Alex's Quest 2 copy is a separate Android candidate. Its ownership does not
|
||||
make the PC `CustomLevels` layout applicable. Until the actual build is
|
||||
inspected, neither a direct song-copy recipe nor APK patching is justified.
|
||||
|
||||
Only maps whose music and chart are permitted for distribution may be used
|
||||
as test fixtures or bundled content. A public download alone does not establish
|
||||
those rights. Start with an original or explicitly licensed basic map.
|
||||
|
||||
**Documented:** [ModsBeforeFriday](https://github.com/Lauriethefish/ModsBeforeFriday)
|
||||
targets Quest Beat Saber over WebUSB/ADB. It is not a generic native ARM64
|
||||
modding protocol. [BSManager](https://github.com/Zagrios/bs-manager/releases/tag/v1.6.0)
|
||||
publishes an aarch64 Flatpak, but its architecture says nothing about Beat
|
||||
Saber or its plugins running on Frame. Neither app is an installation step
|
||||
or dependency for Frame Control.
|
||||
|
||||
## Requirements for our manager
|
||||
|
||||
These are **planned**, not implemented or verified:
|
||||
|
||||
1. Resolve the selected Steam game's real library, installed build, executable
|
||||
architecture and Proton prefix. Confirm ownership through Steam; a directory
|
||||
or app manifest alone is not proof. Keep downloading, installed and playable
|
||||
as separate states.
|
||||
2. Download a pinned mod version from the author's official release. Record
|
||||
the URL, version, license and digest. Verify the published digest when
|
||||
available; an upstream SHA-256 detects corruption but is not a signature.
|
||||
Do not treat “free to download” as permission to redistribute.
|
||||
3. Stage and validate archives before writing into the game or prefix. Reject
|
||||
path traversal, links escaping the destination, archive bombs and unexpected
|
||||
executable content in song packs. Check song metadata and its referenced
|
||||
files, not just the `.zip` suffix.
|
||||
4. Refuse changes while the game is running. Back up originals and journal
|
||||
every managed file and digest. Apply changes atomically where possible and
|
||||
roll back partial failures. Keep runtime prerequisites scoped to this game.
|
||||
5. Uninstall only files still matching our receipt; restore originals without
|
||||
overwriting later user edits. Preserve saves, unrelated mods and songs.
|
||||
Song removal must target one managed map, never the whole CustomLevels tree.
|
||||
6. Expose one-click actions beside the game only after real-Frame install,
|
||||
playback and uninstall pass. Test filesystem and download failure handling
|
||||
with fake-Frame fixtures; those cannot prove FEX injection or VR rendering.
|
||||
|
||||
## Sources
|
||||
|
||||
- [UEVR 1.05 official release](https://github.com/praydog/UEVR/releases/tag/1.05)
|
||||
and [author's usage instructions](https://github.com/praydog/UEVR#getting-started).
|
||||
- [Microsoft .NET 6 release metadata](https://builds.dotnet.microsoft.com/dotnet/release-metadata/6.0/releases.json),
|
||||
including the SHA-512 hashes used for the test runtimes.
|
||||
- [OpenComposite's OpenXR branch](https://gitlab.com/znixian/OpenOVR/-/tree/openxr),
|
||||
including per-game installation and the need to preserve original DLLs.
|
||||
- [R.E.A.L. author post referenced by the research](https://www.patreon.com/realvr/posts/but-wheres-link-165840151)
|
||||
(HTTP 403 from this environment; contents not verified).
|
||||
- [Half-Life 2 VR official site](https://halflife2vr.com/) and
|
||||
[Episode One on Steam](https://store.steampowered.com/app/2177750/).
|
||||
- [Beat Saber store metadata](https://store.steampowered.com/api/appdetails?appids=620980).
|
||||
+28
-13
@@ -33,14 +33,19 @@ build 20260922.6101926, kernel 6.18, aarch64):
|
||||
- **10.** `install-apps.sh remmina --vnc-host <mac>.local` installed Remmina as
|
||||
a `--user` Flatpak over SSH and wrote the profile. The desktop's
|
||||
`XDG_DATA_DIRS` includes the user Flatpak exports, so it shows up in the menu.
|
||||
The Frame can reach the Mac's Screen Sharing port (5900). The Remmina
|
||||
connection itself hasn't been tried in the headset yet (part of 11).
|
||||
The Frame can reach the Mac's Screen Sharing port (5900).
|
||||
- **11.** Answered 2026-09-27 (BUILD_ID 20260925.6191901, macOS 27.0): the
|
||||
pre-seeded profile connects and shows the Mac in its own panel. It asks for
|
||||
the Mac account login rather than the VNC password, needs scale-to-fit at
|
||||
Retina resolutions, and doesn't show the Mac cursor without
|
||||
`scripts/mac-cursor-ring.lua`. It's usable but noticeably laggy. See
|
||||
[streaming.md](streaming.md).
|
||||
|
||||
- **Panels.** An X11 window on gamescope's `:0` with its own `STEAM_GAME` id
|
||||
gets its own SteamVR overlay (`valve.steam.desktopgame.<id>`). Three were
|
||||
created side by side with `panel-on-frame.sh`. See [panels.md](panels.md).
|
||||
|
||||
Still open: 4, 6, 7, 11 (in-headset connect), 12–15, 16 (off-LAN and after a reboot), 17–21.
|
||||
Still open: 4, 6, 7, 12–15, 16 (off-LAN and after a reboot), 17–21.
|
||||
|
||||
## Check on the headset (in order)
|
||||
|
||||
@@ -70,12 +75,10 @@ Still open: 4, 6, 7, 11 (in-headset connect), 12–15, 16 (off-LAN and after a r
|
||||
`ssh frame 'command -v wl-copy xclip rsync flatpak'`.
|
||||
10. **Can Flatpaks be installed `--user` over SSH, and do they appear in the
|
||||
headset's desktop?** Test with `./scripts/install-apps.sh remmina`.
|
||||
11. **Remmina → macOS Screen Sharing:** does it connect, and is it usable at
|
||||
Retina resolutions? Is the pre-seeded profile path
|
||||
(`~/.var/app/org.remmina.Remmina/data/remmina/`) the one Remmina
|
||||
actually reads?
|
||||
12. **Moonlight Flatpak (aarch64) + Sunshine on macOS:** worth trying only if
|
||||
VNC is too slow.
|
||||
11. ~~**Remmina → macOS Screen Sharing**~~: answered 2026-09-27; see above
|
||||
and [streaming.md](streaming.md).
|
||||
12. **Moonlight Flatpak (aarch64) + Sunshine on macOS:** VNC works but is
|
||||
noticeably laggy, so this is worth trying.
|
||||
13. **KDE Connect**: is it preinstalled or installable on the Frame, and does
|
||||
it pair with KDE Connect for macOS?
|
||||
14. **Bluetooth keyboard pairing** on the Frame, for the rare times you do need
|
||||
@@ -86,8 +89,9 @@ Still open: 4, 6, 7, 11 (in-headset connect), 12–15, 16 (off-LAN and after a r
|
||||
runs as a lingering user service with no sudo; see [tailscale.md](tailscale.md).
|
||||
Still open: reaching the Frame from outside the home network, and the service
|
||||
starting after a reboot.
|
||||
17. **Floating panels in the headset** (see [panels.md](panels.md)): do the
|
||||
panels from `panel-on-frame.sh` show up, take input, and offer **Float in
|
||||
17. **Floating panels in the headset** (see [panels.md](panels.md)): panels
|
||||
from `panel-on-frame.sh` show up and take controller input (verified
|
||||
2026-09-27 with `mac-screen`). Still open: do they offer **Float in
|
||||
World** / **Move** / **Size**? Do floating positions survive closing and
|
||||
reopening the app, or a reboot?
|
||||
18. **`LEPTON_NO_CLEANUP=1 %command%`** as Lepton Development's launch
|
||||
@@ -125,12 +129,23 @@ Still open: 4, 6, 7, 11 (in-headset connect), 12–15, 16 (off-LAN and after a r
|
||||
- The Mac EDL flashing script in `~/Downloads/steam-frame-recovery/` hasn't
|
||||
been run against a Frame.
|
||||
|
||||
## Mac in the headset (2026-09-28)
|
||||
|
||||
The test pattern streams to the Frame as its own panel at about 60 fps
|
||||
(verified, build 20260925.6191901; see
|
||||
[mac-in-headset.md](mac-in-headset.md#checked-so-far-2026-09-28)). Still to
|
||||
check in the headset:
|
||||
|
||||
- Laser clicks, drags and thumbstick scrolling in a viewer panel.
|
||||
- Real window capture and input once Screen Recording and Accessibility are
|
||||
granted to Frame Control.
|
||||
- Keys from SteamVR's on-screen keyboard.
|
||||
- Whether Flathub Chromium decodes H.264 (otherwise use Compatible).
|
||||
|
||||
## Unconfirmed claims made in these docs
|
||||
|
||||
- `/home` and `/etc` persist across Frame OS updates. This is inferred from
|
||||
Steam Deck behaviour.
|
||||
- The whole Mac → Frame desktop path (VNC → Remmina). Each part is documented
|
||||
separately, but the combination is untested.
|
||||
- Steam Remote Play with a Mac as host is broken. That's based on community
|
||||
reports, not tested with the Frame.
|
||||
- `connect.sh --harden`, `serve-bootstrap.sh` and
|
||||
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
# Privacy and analytics
|
||||
|
||||
Frame Control sends anonymous analytics to [PostHog](https://posthog.com)
|
||||
(US cloud) so the maintainer can see how many people use it, which features
|
||||
matter and where installs fail. You choose how much in **Privacy & updates**,
|
||||
the last panel on the page. `ui/frame_telemetry.py` is the whole
|
||||
implementation.
|
||||
|
||||
## The three levels
|
||||
|
||||
| Level | Default | What it sends |
|
||||
|---|---|---|
|
||||
| Anonymous usage statistics | On, after a notice on first run | The events in the table below |
|
||||
| Share compatibility results | Off | Your Android compatibility reports and tests |
|
||||
| Send error details | Off | Scrubbed error messages and tracebacks |
|
||||
|
||||
Nothing is sent until the first-run notice has been shown. The notice's
|
||||
**Share more to help fix problems** button turns on the second and third
|
||||
levels together. Either can be turned off later. Turning a level off
|
||||
deletes that level's events that haven't been sent yet.
|
||||
|
||||
**Show what's been sent** in the panel lists the last 50 events that left your
|
||||
computer, exactly as they were sent.
|
||||
|
||||
## Anonymous
|
||||
|
||||
- Events carry a random id, made when Frame Control first runs and kept in
|
||||
its data folder (`telemetry/settings.json`). It isn't derived from your
|
||||
computer, account or network. To get a new one, delete that file.
|
||||
- Events are sent without person profiles (`$process_person_profile: false`)
|
||||
and without location lookup (`$geoip_disable: true`). Each carries a
|
||||
placeholder address (`$ip: 0.0.0.0`), so PostHog stores that instead of
|
||||
yours.
|
||||
- Every event includes the app version, OS name (macOS, Windows or Linux),
|
||||
CPU architecture and Python version.
|
||||
|
||||
## Usage events
|
||||
|
||||
| Event | When | Properties besides the common ones |
|
||||
|---|---|---|
|
||||
| `app_installed` | First run | |
|
||||
| `app_updated` | First run of a new version | `from_version` |
|
||||
| `app_opened` | At most once a day | |
|
||||
| `frame_connected` | The first time a SteamOS build is seen | `steamos_build`, `steamos_version` |
|
||||
| `tab_viewed` | The first click on each tab in a session | `tab` |
|
||||
| `install_finished` | Any install finishes, working or not | `kind` (apk, flatpak, steam, title, web), `ok`, `seconds`, `error_category`, `installer_code`, and see below |
|
||||
| `update_offered`, `update_started`, `update_failed` | The update banner | `to_version`, `error_category` |
|
||||
|
||||
`install_finished` never includes a file name, path or error message. An
|
||||
error becomes one category from a fixed list (for example `apk_wrong_abi` or
|
||||
`frame_unreachable`), plus Android's own `INSTALL_FAILED_…` code when there
|
||||
is one. It names what was installed only when that's already public:
|
||||
|
||||
- F-Droid catalogue apps: `package`. Never the version, since a local build can reuse a
|
||||
catalogue app's package name
|
||||
- Flathub apps: `flatpak_id`
|
||||
- Steam games: `steam_appid`
|
||||
- A sideloaded title: only its runtime (Proton or Linux)
|
||||
|
||||
Any other APK is sent as `catalog: false`, with no name.
|
||||
|
||||
## Compatibility results (opt-in)
|
||||
|
||||
Each report becomes a `compat_report` event with the fields the Report dialog
|
||||
shows: package, version, result or rating, your notes, how it was run, and the
|
||||
SteamOS and Lepton builds. Before sending:
|
||||
|
||||
- the notes, app name and version are scrubbed like error messages (see
|
||||
below)
|
||||
- the APK's source is kept only if it's `F-Droid` or the public host name of
|
||||
a download link (`https://example.com/…`). File names, user names,
|
||||
passwords, ports, paths, IP addresses and local host names are dropped
|
||||
|
||||
When you turn this on, reports you made earlier on this computer are shared
|
||||
too.
|
||||
|
||||
The maintainer's `python3 ui/frame_compat_db.py sync` copies these events
|
||||
into the compatibility database, marked `via=community…`. It takes at most
|
||||
30 per reporter per day.
|
||||
|
||||
## Error details (opt-in)
|
||||
|
||||
`$exception` events carry an error message, the Frame Control file, line and
|
||||
function it came from, and the request that failed (for example
|
||||
`POST /api/android install`). Before anything is sent, the message is
|
||||
scrubbed:
|
||||
|
||||
- your home folder becomes `~`, and any user name becomes `<user>`
|
||||
- IP and MAC addresses, email addresses, `.local`, `.lan` and Tailscale host
|
||||
names, Steam ids, SSH and PEM keys, API tokens and long hex strings are
|
||||
replaced
|
||||
- URLs are cut down to their scheme and a public host name, or `<url>`. User
|
||||
names, passwords, ports, paths and queries are dropped
|
||||
- `token=`, `key=`, `password=` and similar values are replaced
|
||||
|
||||
The same error is sent at most once every 10 minutes.
|
||||
|
||||
## Report a problem
|
||||
|
||||
**Report a problem** is the warning-sign button in the header, also in the
|
||||
Privacy panel and under **Help → Report a Problem…**. It sends the report
|
||||
privately to Frame Control's PostHog project as a `problem_report` event, the
|
||||
same way as the analytics above, so only the maintainer can read it and
|
||||
nothing is published. It works whatever the analytics settings are, because
|
||||
the person sends it deliberately. The report has the kind, title and text you
|
||||
wrote, how to reach you if you gave it, a short reference shown after sending,
|
||||
and the diagnostics below. It has its own random id, so it isn't linked to
|
||||
your analytics events.
|
||||
|
||||
With **Include diagnostics** ticked (the default), the report adds:
|
||||
|
||||
- the app version and whether it's a built app
|
||||
- the OS, its release and CPU, and the Python version
|
||||
- the Frame's SteamOS build, if it has connected since the app started
|
||||
- which analytics levels are on
|
||||
|
||||
**Also include recent activity and the server log** is off by default,
|
||||
because those lines can name files and apps. When ticked, it adds the newest
|
||||
Activity lines and server log lines, without the request lines.
|
||||
|
||||
Everything is scrubbed like error details and limited to what fits in the
|
||||
report. Environment details are kept first, then the newest lines. **Show
|
||||
exactly what's included** shows the snapshot that will be sent, and later
|
||||
activity isn't added to it. If PostHog can't be reached, **Copy report** puts
|
||||
the whole report on the clipboard.
|
||||
|
||||
The maintainer reads reports on the Frame Control dashboard in PostHog, or
|
||||
with `python3 ui/frame_report.py inbox [days]`, which uses the same personal
|
||||
API key as `frame_compat_db.py sync`.
|
||||
|
||||
## Turning it all off
|
||||
|
||||
Untick the boxes, or set `DO_NOT_TRACK=1` or `FRAME_CONTROL_TELEMETRY=0` in
|
||||
the environment that starts Frame Control. A copy run from a source checkout
|
||||
never sends anything unless `FRAME_CONTROL_TELEMETRY=1` is set.
|
||||
|
||||
## Update checks
|
||||
|
||||
The desktop app asks GitHub for the latest release shortly after starting,
|
||||
then every 6 hours: the latest release's `update.json` on GitHub, or
|
||||
`api.github.com/repos/saphid/frame-control/releases/latest` if that fails.
|
||||
Those requests carry no id. To stop it, set
|
||||
`FRAME_CONTROL_NO_UPDATE_CHECK=1`. See [releasing.md](releasing.md).
|
||||
@@ -0,0 +1,59 @@
|
||||
# Releasing and updates
|
||||
|
||||
Frame Control checks for updates itself. The desktop app offers a new version
|
||||
only once it's GitHub's **latest release**, and drafts and pre-releases never
|
||||
count. So a build reaches people only when you publish it, after testing it.
|
||||
|
||||
## Steps
|
||||
|
||||
1. Bump `version` in `app/package.json`, commit, and push a tag:
|
||||
|
||||
```sh
|
||||
git tag v0.4.0 && git push origin v0.4.0
|
||||
```
|
||||
|
||||
`.github/workflows/release.yml` builds macOS, Windows and Linux, and
|
||||
attaches everything to a **draft** release for that tag. Nobody is
|
||||
offered a draft.
|
||||
|
||||
2. Download the draft's installers and test them. An installed copy of the
|
||||
previous version won't offer the draft, so install it directly.
|
||||
|
||||
3. Write the release notes on the draft. The update banner links to them.
|
||||
|
||||
4. Publish:
|
||||
|
||||
```sh
|
||||
scripts/publish-release.sh v0.4.0
|
||||
```
|
||||
|
||||
The script checks that all eight installers are attached, each with the
|
||||
SHA-256 digest GitHub records. It attaches `update.json` (the version, the
|
||||
notes and each installer's digest), then publishes the release and marks it
|
||||
latest. From then on, running copies see the update. They check about 8
|
||||
seconds after starting, then every 6 hours, and anyone can use **Check for
|
||||
Updates…** (the app menu on macOS, the Help menu elsewhere).
|
||||
|
||||
To pull a bad release, mark the previous one as latest
|
||||
(`gh release edit v0.3.9 --latest`) or turn the bad one back into a draft.
|
||||
Copies that already updated stay on it. Nothing downgrades them.
|
||||
|
||||
## How a copy updates itself
|
||||
|
||||
`app/updater.js` reads `update.json` from
|
||||
`github.com/saphid/frame-control/releases/latest/download/`. It falls back to the
|
||||
REST API only when a release has no manifest, because the API allows just 60
|
||||
unauthenticated requests an hour per IP address, shared by a whole household.
|
||||
Then it downloads the installer for its platform and checks it
|
||||
against the SHA-256 digest GitHub publishes for the asset. It refuses if the
|
||||
digest is missing or doesn't match. Then:
|
||||
|
||||
| Installed from | Update |
|
||||
|---|---|
|
||||
| macOS `.dmg`, app in a writable folder such as Applications | The `.zip` is unpacked next to the app and its version checked. After the app quits, a small script swaps the new app in, putting the old one back if that fails, and reopens it. Updates don't get the download quarantine, so there's no `xattr` step. |
|
||||
| Windows installer | The new `Setup` runs silently over the install (`/S --force-run`) and reopens the app. |
|
||||
| Linux AppImage | The new AppImage replaces the old file and is started. |
|
||||
| macOS app still on the disk image or translocated, Windows `.zip`, Linux `.deb` | The banner opens the release page instead. |
|
||||
|
||||
Version 0.3.1 and earlier have no updater, so people on them have to download
|
||||
the new version once by hand.
|
||||
+2
-1
@@ -93,7 +93,8 @@ controls to place each panel. See [docs/panels.md](panels.md).
|
||||
| `scripts/install-apps.sh` | Mac → Frame | Install Flatpaks (Remmina, Moonlight, …) on the Frame over SSH as `--user` (**verified** with Remmina) |
|
||||
| `scripts/paste-to-frame.sh` | Mac → Frame | Send the Mac clipboard (or stdin) to the Frame clipboard (**verified**) |
|
||||
| `scripts/install-apk.sh` | Mac → Frame | Install APKs, each as its own persistent Lepton instance with a Steam library shortcut (`--dev`: old ADB path into Lepton Development) (**verified**; see [docs/apks.md](apks.md)) |
|
||||
| `scripts/panel-on-frame.sh` | Mac → Frame | Start an app as its own floating VR panel, outside the desktop (**verified**: overlays created; in-headset placement not yet checked) |
|
||||
| `scripts/panel-on-frame.sh` | Mac → Frame | Start an app as its own floating VR panel, outside the desktop (**verified**, including `mac-screen` in the headset) |
|
||||
| `scripts/mac-cursor-ring.lua` | Mac | Hammerspoon script: a ring around the Mac pointer so it shows in the VNC mirror (**verified**) |
|
||||
| `scripts/run-on-frame.sh` | Mac → Frame | Start an app on the headset desktop, e.g. `mac-screen` opens Remmina straight into the Mac (**verified**) |
|
||||
| `scripts/frame-ui.sh` | Mac | Start the Frame Control web UI (`ui/server.py`) and open it (**verified**) |
|
||||
| `scripts/apk-catalog.sh` | Mac | Refresh the rated F-Droid catalogue that Frame Control's Android section shows (**verified**) |
|
||||
|
||||
@@ -4,6 +4,10 @@ Frame Control's **Get games** section lists the games you own with each one's
|
||||
Steam Frame rating, installs them on the Frame, and searches the Steam store.
|
||||
This page covers how it works underneath, so you can do the same from a shell.
|
||||
|
||||
For flat-to-VR mods and Beat Saber custom songs, see the
|
||||
[per-game feasibility table](mods.md). Mod support is separate from Steam's
|
||||
Frame rating; there is no mod installer yet.
|
||||
|
||||
## How it works
|
||||
|
||||
The Frame's Steam client runs with `-cef-enable-debugging`. So its UI, a
|
||||
|
||||
+90
-18
@@ -1,9 +1,13 @@
|
||||
# Screen and desktop streaming
|
||||
|
||||
This covers two directions:
|
||||
This covers three directions, plus input:
|
||||
|
||||
- **A. Frame → Mac**: see and control the headset from the Mac.
|
||||
- **B. Mac → Frame**: use the Mac's desktop inside the headset.
|
||||
- **C. iPhone → Frame**: mirror the phone inside the headset.
|
||||
- **PC VR from Linux**: [feasibility and options](linux-vr-streaming.md),
|
||||
including Valve's streaming and USB support. No Linux host tested yet.
|
||||
- **Input**: type and point in the Frame from the Mac or iPhone.
|
||||
|
||||
The confidence labels are the same as in [ssh.md](ssh.md).
|
||||
|
||||
@@ -22,17 +26,20 @@ Mac with keyboard, mouse, and clipboard.
|
||||
|
||||
## B. Show the Mac's desktop inside the Frame
|
||||
|
||||
The Frame's streaming features are built around a **Windows PC running
|
||||
SteamVR** plus the USB Wi-Fi 6E dongle. Even Linux hosts had VR-streaming
|
||||
problems at launch
|
||||
The Frame's VR streaming uses **SteamVR** on the host. Linux hosts had
|
||||
VR-streaming problems at launch
|
||||
([Steam discussion](https://steamcommunity.com/app/4165890/discussions/0/528765047224280796/),
|
||||
[gbl08ma](https://gbl08ma.com/posts/steam-frame-a-linux-machine-doesnt-support-linux/)).
|
||||
Valve's later 2.17.8 notes explicitly describe Steam Link fixes on Linux and
|
||||
initial USB streaming support (**documented**, not tested from a Linux host
|
||||
here). See [the current comparison](linux-vr-streaming.md#a-valves-own-path--recommended-first).
|
||||
**macOS isn't a supported SteamVR host**, so for the Mac we're only looking at
|
||||
flat 2D desktop streaming into a window on the Frame's Linux desktop.
|
||||
|
||||
| Option | Setup | Confidence | Verdict |
|
||||
|---|---|---|---|
|
||||
| **macOS Screen Sharing (VNC) → Remmina on the Frame** | **Mac:** System Settings → General → Sharing → Screen Sharing on → (i) → enable "VNC viewers may control screen with password". **Frame:** `./scripts/install-apps.sh remmina` from the Mac, then open Remmina in the headset and connect to `vnc://<mac>.local` | **Inferred.** Remmina is on Flathub for **aarch64** with VNC and RDP ([Flathub](https://flathub.org/apps/org.remmina.Remmina)). The Frame desktop runs Flatpaks ([UploadVR](https://www.uploadvr.com/flatpaks-open-source-steam-frame/)). macOS VNC is built in. | **Recommended.** Nothing to install on the Mac, and it's easy to set up. Latency is fine for productivity but not for games. You'll type the Mac's hostname once in Remmina on the headset, then save the profile. To avoid even that, the script can pre-seed a Remmina profile over SSH (see below). |
|
||||
| **Frame Control → Tools → Mac in the headset** | Nothing to install. Allow Screen Recording and Accessibility for Frame Control, then press **Show** next to any window or screen | **Verified 2026-09-28** on the Frame (panel in 1.5 s, measured with `scripts/macview-bench.py`: about 10–20 ms from the Mac drawing a frame to the viewer drawing it); laser input not yet tried while wearing it | **Recommended.** Hardware H.264 over an SSH tunnel, adapting to the link. Each window becomes its own panel you can place anywhere. Laser clicks and scrolls, and the Mac's keyboard types. See [mac-in-headset.md](mac-in-headset.md) |
|
||||
| **macOS Screen Sharing (VNC) → Remmina on the Frame** | **Mac:** System Settings → General → Sharing → Screen Sharing on → (i) → enable "VNC viewers may control screen with password". **Frame:** `./scripts/install-apps.sh remmina` from the Mac, then open Remmina in the headset and connect to `vnc://<mac>.local` | **Verified 2026-09-27** (Frame BUILD_ID 20260925.6191901, macOS 27.0), in its own panel via `panel-on-frame.sh mac-screen`. Remmina is on Flathub for **aarch64** with VNC and RDP ([Flathub](https://flathub.org/apps/org.remmina.Remmina)). The Frame desktop runs Flatpaks ([UploadVR](https://www.uploadvr.com/flatpaks-open-source-steam-frame/)). macOS VNC is built in. | **Fallback** (whole screens only). Nothing to install on the Mac, and it's easy to set up. Noticeable lag, even at lower Remmina quality settings on a good 5 GHz link, where neither Wi-Fi nor the Frame's CPU was the bottleneck. Usable for reading and coding, but not for games. You'll type the Mac's hostname once in Remmina on the headset, then save the profile. To avoid even that, the script can pre-seed a Remmina profile over SSH (see below). |
|
||||
| Sunshine (Mac) → Moonlight (Frame Flatpak) | `brew install` Sunshine on the Mac, then `./scripts/install-apps.sh moonlight` | Moonlight Flatpak supports **aarch64** ([Flathub](https://flathub.org/apps/com.moonlight_stream.Moonlight)). **Sunshine on macOS is poorly supported**: install problems on Apple Silicon/Sequoia, and no virtual gamepads ([LizardByte discussion #777](https://github.com/orgs/LizardByte/discussions/777)). | Try it if VNC is too laggy. Expect some friction. |
|
||||
| Steam Remote Play with the Mac as host | Steam on the Mac, Steam Link/Remote Play on the Frame | macOS-hosted Remote Play is reported broken or flaky in 2024–2026 ([Steam discussion](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/)) | Not recommended. It's only for games, if it works at all. |
|
||||
| Immersed / Virtual Desktop | Vendor apps | Immersed has a Mac agent but no known Frame client. Virtual Desktop's developer said he'd "try" to port it ([NewsBreak](https://www.newsbreak.com/news/4892834783961-virtual-desktop-dev-says-he-ll-try-to-bring-the-app-to-steam-frame)). | Not available as of 2026-09-25. Check again later. |
|
||||
@@ -45,20 +52,85 @@ them on the Frame in DeoVR instead: see [vr-video.md](vr-video.md).
|
||||
|
||||
`scripts/install-apps.sh remmina --vnc-host <your-mac>.local` writes
|
||||
`~/.var/app/org.remmina.Remmina/data/remmina/mac-screen-sharing.remmina` on the Frame over
|
||||
SSH. The profile then appears in Remmina's list, and you just click it. You'll
|
||||
still be asked for the VNC password in the headset the first time, unless you
|
||||
choose to save it. Remmina stores passwords encrypted with a per-install key,
|
||||
so the script doesn't try to write the password. (The Remmina file format is
|
||||
standard; the Flatpak data path is inferred.)
|
||||
SSH. The profile then appears in Remmina's list, and you just click it. It
|
||||
scales the Mac's desktop to fit the window (`scale=1`, `viewmode=1`). Without
|
||||
that, Remmina shows a Retina Mac's native pixels 1:1, so you see a zoomed-in
|
||||
corner. (Verified 2026-09-27.)
|
||||
|
||||
## Input and text entry without the virtual keyboard
|
||||
**Expect a Mac login prompt, not the VNC password.** macOS offers Apple's own
|
||||
authentication (RFB security type 30) ahead of plain VNC auth (type 2), and
|
||||
Remmina picks it. So Remmina asks for your **Mac account name and login
|
||||
password**; the "VNC viewers may control screen" password isn't used. To store
|
||||
the password without typing it in the headset, run on the Frame:
|
||||
|
||||
- **A Bluetooth keyboard and mouse** paired to the Frame is the obvious way to
|
||||
avoid the virtual keyboard. Road to VR says there are "only a few things
|
||||
you'd actually want to do" on the Linux desktop unless you connect a
|
||||
keyboard and mouse.
|
||||
(Pairing a BT keyboard on the Frame is inferred from SteamOS; not verified.)
|
||||
- **Clipboard from the Mac**: `scripts/paste-to-frame.sh` (see
|
||||
[file-transfer.md](file-transfer.md#clipboard)).
|
||||
```sh
|
||||
printf '%s' "$PASSWORD" | flatpak run org.remmina.Remmina \
|
||||
--update-profile ~/.var/app/org.remmina.Remmina/data/remmina/mac-screen-sharing.remmina \
|
||||
--set-option password
|
||||
```
|
||||
|
||||
Remmina encrypts it into the profile with its own key, because there's no
|
||||
secret service in the SSH session. (Verified 2026-09-27.)
|
||||
|
||||
### The Mac's cursor
|
||||
|
||||
The mirror doesn't show the Mac's pointer, with either `showcursor` value.
|
||||
macOS keeps the pointer out of the picture it sends, and Remmina's cursor mode
|
||||
draws the cursor shape only at the Frame's own pointer, which doesn't follow
|
||||
the Mac trackpad. `scripts/mac-cursor-ring.lua` works around this: a
|
||||
[Hammerspoon](https://www.hammerspoon.org/) script that draws a ring around the
|
||||
Mac pointer as a real window, so it's part of the mirrored picture. Setup is in
|
||||
its header. (Verified 2026-09-27.)
|
||||
|
||||
Going the other way, pointing a controller at the panel moves the Mac's mouse,
|
||||
because Remmina forwards input (`viewonly=0`).
|
||||
|
||||
## First-party options, and why they do or don't fit
|
||||
|
||||
Checked 2026-09-28. The first-party way is usually the best one, so these are
|
||||
listed first; the sections above and below explain the alternatives.
|
||||
|
||||
| Goal | First-party option | Fits? | Why, and what would make it easier |
|
||||
|---|---|---|---|
|
||||
| Type and point from the **iPhone** | **KDE Connect** (KDE; official [iOS app](https://apps.apple.com/app/kde-connect/id1580245991)) remote touchpad and keyboard, plus clipboard and files | **Best candidate, untested on the Frame** | The Frame doesn't have it (verified: no `kdeconnectd`), it isn't on Flathub, and the root is read-only, so it would have to run from `~` or a container. On Wayland it types through KWin, so it can only reach the desktop panel, not SteamVR or games (**inferred**). Steam Deck users report its remote input breaking after SteamOS updates ([SteamOS #1939](https://github.com/ValveSoftware/SteamOS/issues/1939)). If it works, Frame Control could install it and pair it for you. |
|
||||
| Type and point from the **Mac** | KDE Connect for macOS (KDE builds) | Same as above | Its Mac app sends clipboard and files but has no keyboard/mouse sharing (**inferred**). |
|
||||
| Either | **Bluetooth keyboard and mouse** paired in SteamOS (Valve) | Yes, with real hardware | Neither device can pretend to be one: iOS refuses the HID service ([Apple forums](https://developer.apple.com/forums/thread/733916)), and macOS has no built-in way. |
|
||||
| Either | **xrdp** in Developer Mode (Valve) | No | Input goes into a *separate* desktop shown on the Mac, not into what you see in the headset. |
|
||||
| **Mac screen** in the Frame | **Screen Sharing** (Apple's VNC server) + Remmina (already installed on this Frame, profile pre-seeded by `install-apps.sh`) | **Yes, closest to first-party** | Only the Mac side is first-party; Remmina is the client. Turn on System Settings → General → Sharing → Screen Sharing → (i) → "VNC viewers may control screen with password". Still to test in the headset (open question 11). |
|
||||
| Mac screen | **Steam Remote Play** with the Mac as host (Valve) | Probably not | macOS isn't a SteamVR host, and Mac-hosted Remote Play is reported broken ([Steam forum](https://steamcommunity.com/groups/homestream/discussions/1/574921459914429988/)). One quick test is worth doing: Steam on the Mac, then Remote Play from the Frame's Steam. |
|
||||
| Mac or **iPhone screen** | **AirPlay** (Apple) | Not officially | It's Apple's own mirroring for both, but Apple only licenses receivers to TV and speaker makers; nothing official runs on Linux. UxPlay (below) is the unofficial receiver. |
|
||||
|
||||
## C. Show the iPhone's screen inside the Frame
|
||||
|
||||
iOS only shares its screen two ways: **AirPlay** (Screen Mirroring in Control
|
||||
Centre) or a **ReplayKit broadcast extension** in an app. Nothing else can
|
||||
capture it.
|
||||
|
||||
| Option | What it takes | Confidence | Verdict |
|
||||
|---|---|---|---|
|
||||
| **UxPlay** (an open-source AirPlay receiver) on the Frame | Build it for aarch64 (no Flathub package; there's a Snap and distro packages), run it in `~` or a podman container, and advertise it over mDNS. The iPhone *and* the Mac then see "Frame" in Screen Mirroring, with nothing to install on either | **Inferred.** It runs on ARM64 Linux such as the Raspberry Pi ([UxPlay](https://github.com/FDH2/UxPlay)). Not tried on the Frame: needs mDNS registration and its ports (7000, 7001, 7100 and a UDP range) reachable | **Recommended to try first.** It's the only receiver-side option, and it covers the Mac too. The window shows in the Frame's Linux desktop panel |
|
||||
| A broadcast extension in Frame Control | ReplayKit sends the screen to a small extension (50 MB memory limit), which encodes H.264 and sends it through the app's SSH tunnel to the page, shown the same way as the Frame's live view in reverse | **Inferred** from Apple's ReplayKit docs | Full control and no network setup, but several days' work, and the picture only shows where Frame Control's page is open in the headset |
|
||||
|
||||
## Input: type and point in the Frame from the Mac or iPhone
|
||||
|
||||
**Verified 2026-09-27** on the headset: `steamos` is in the `input` group and
|
||||
`/dev/uinput` is `crw-rw-r-- root input`, so **our own code can create a
|
||||
virtual keyboard and mouse without sudo**. The Frame has no `python-evdev`,
|
||||
`ydotool`, `wtype` or KDE Connect; `kwin_wayland` and `plasmashell` run only
|
||||
while the desktop panel is open in the headset.
|
||||
|
||||
| Option | Mac | iPhone | Notes |
|
||||
|---|---|---|---|
|
||||
| **A uinput keyboard and mouse in Frame Control's server** | ✓ | ✓ | **Recommended.** The server opens `/dev/uinput` with `ctypes` (standard library only) and the page sends key and pointer events through the tunnel it already has. On the phone: a trackpad area (drag to move, tap to click, two fingers to scroll) and the iOS keyboard for typing. On the Mac: a "control the Frame" mode that captures the keyboard and pointer (Esc to release). Uinput devices look like real hardware to the kernel, so libinput, KWin and gamescope should take them; [frame-voice](https://github.com/DeeJanuz/frame-voice) already types into a Frame through a uinput keyboard. **Untested**: which surfaces in VR (desktop panel, SteamVR dashboard, games, Android apps in Lepton) accept the pointer. About a day or two of work |
|
||||
| **Bluetooth keyboard and mouse** | – | – | Real hardware paired in SteamOS settings. The iPhone can't pretend to be a Bluetooth keyboard: iOS won't advertise the HID service ([Apple forums](https://developer.apple.com/forums/thread/733916)) |
|
||||
| **Deskflow** (formerly Input Leap / Barrier) | ✓ | – | Moves the Mac's own mouse and keyboard onto the Frame's screen edge. Flathub has an aarch64 build ([Flathub](https://flathub.org/apps/org.deskflow.deskflow)); on Wayland it needs the InputCapture/libei portal, and only works while Plasma is running. No iPhone client |
|
||||
| **KDE Connect** | ~ | ✓ | Its iOS app has a remote touchpad and keyboard, but the Frame would need KDE Connect installed (not on Flathub; `pacman` on a read-only root). More moving parts than the uinput route |
|
||||
| **Remmina / Steam Link / RDP** | ✓ | – | Input only reaches the streamed session, not the headset's own apps |
|
||||
|
||||
Other ways to get text in:
|
||||
|
||||
- **Clipboard from the Mac**: `scripts/paste-to-frame.sh`, or Frame Control's
|
||||
clipboard box (see [file-transfer.md](file-transfer.md#clipboard)). Needs
|
||||
the desktop panel open.
|
||||
- **RDP session**: Windows App syncs the clipboard with xrdp, but only inside
|
||||
that RDP session.
|
||||
@@ -148,3 +148,11 @@ For example, on 2026-09-27 the smoke test found that Steam's `create-shortcut`
|
||||
refuses ids with a hyphen (`missing/invalid arguments`), which the fake had
|
||||
accepted. The fake now refuses them the same way, and Frame Control makes ids
|
||||
Steam accepts.
|
||||
|
||||
## Agent interfaces
|
||||
|
||||
`tests/test_agent.py` exercises MCP stdio, exact-action human approvals and the
|
||||
assistant against an in-process HTTP endpoint with canned responses (no keys or
|
||||
external calls). `tests/e2e/test_agents.py` runs the MCP/HTTP/SSH path against the
|
||||
fake Frame for approved installs, clipboard and file transfer. Headset Chromium
|
||||
rendering and real screenshots still need a device; see [agent evidence](agents.md#evidence-and-limits).
|
||||
+64
-69
@@ -3,9 +3,11 @@
|
||||
Goal: open a web VR180 or 360 player (DeoVR and DL8 embeds, WebXR samples),
|
||||
press its VR button, and watch in 3D in the headset.
|
||||
|
||||
A standalone, public version of this build (build script, the SO_PEERCRED
|
||||
patch, and a Frame-side installer that adds "Chromium XR" to the Steam library)
|
||||
is at [saphid/chromium-webxr-steam-frame](https://github.com/saphid/chromium-webxr-steam-frame).
|
||||
The build and installer now live in their own public repo,
|
||||
[saphid/chromium-webxr-steam-frame](https://github.com/saphid/chromium-webxr-steam-frame):
|
||||
a build script for an x86-64 Linux host, the SO_PEERCRED patch, and a
|
||||
Frame-side installer that adds "Chromium XR" to the Steam library. This page
|
||||
keeps the findings and what was verified on this Frame.
|
||||
|
||||
## Why Flathub Chromium can't
|
||||
|
||||
@@ -45,76 +47,52 @@ gets both.
|
||||
SteamVR (`bin/linuxarm64/vrclient.so`, `VALVE_runtime_is_steamvr`). The
|
||||
Linux backend uses Vulkan (`XR_USE_GRAPHICS_API_VULKAN`).
|
||||
|
||||
## Building it
|
||||
## Building and installing it
|
||||
|
||||
[`scripts/build-chromium-xr.sh`](../scripts/build-chromium-xr.sh)
|
||||
cross-compiles arm64 Linux Chromium on an x64 Linux host. It doesn't need
|
||||
sudo: the arm64 sysroot comes from Chromium's own script. It needs about
|
||||
90 GB of disk. It shallow-fetches the CL ref (patchset 44), runs
|
||||
`gclient sync --no-history`, installs the sysroot, applies one extra seccomp
|
||||
fix (below), builds `chrome` with `symbol_level=0` and proprietary codecs, and
|
||||
packs `chromium-xr-arm64.tar.xz` (about 145 MB, GPU libraries included).
|
||||
Progress is logged to `~/chromium-xr/stage`. The build aborts if the disk
|
||||
holding `~/chromium-xr` drops below 12 GB free.
|
||||
Follow the [public repo's README](https://github.com/saphid/chromium-webxr-steam-frame#build).
|
||||
In short: `build/build.sh` on an x64 Linux host (no sudo, about 90 GB of
|
||||
disk) produces `chromium-xr-arm64.tar.xz` (about 145 MB), and
|
||||
`frame/install.sh` on the Frame unpacks it to `~/chromium-xr`, installs the
|
||||
`chromium-xr` launcher in `~/.local/bin`, and adds the Steam library shortcut
|
||||
through the Steam client's DevTools port, the same way as T3 Code
|
||||
([apks.md](apks.md)). Launching the shortcut gives Chromium its own panel,
|
||||
`valve.steam.desktopgame.<appid>`, like any other app.
|
||||
|
||||
First run, 2026-09-25, on a 12-core, 31 GB x64 Linux box: 9 h 33 min for
|
||||
First build, 2026-09-25, on a 12-thread, 31 GB x64 Linux box: 9 h 33 min for
|
||||
94,835 steps, giving Chromium 156.0.8071.0. A rebuild after a one-file change
|
||||
takes under a minute, plus about 4 minutes to repack.
|
||||
|
||||
**The extra fix.** The CL's XR seccomp policy refuses `getsockopt`. SteamVR's
|
||||
To debug from the Mac, launch it as a panel with DevTools on the Frame
|
||||
(verified 2026-09-27):
|
||||
`scripts/panel-on-frame.sh -- '~/.local/bin/chromium-xr' --remote-debugging-port=9223 URL`
|
||||
([panels.md](panels.md)). DevTools has no authentication. It listens on
|
||||
loopback, but with the userspace Tailscale from [tailscale.md](tailscale.md)
|
||||
running, loopback ports are reachable from your tailnet. Close the browser
|
||||
when you're done. Chromium runs one browser per profile, so close the
|
||||
Steam-launched one first or the flag is ignored.
|
||||
|
||||
**The SO_PEERCRED fix.** The XR seccomp policy refuses `getsockopt`. SteamVR's
|
||||
client calls `getsockopt(SOL_SOCKET, SO_PEERCRED)` inside `xrCreateInstance`,
|
||||
so the XR process died with a seccomp crash (arm64 syscall 209). The script
|
||||
allows that one option. That's needed but not enough: `launch` still turns
|
||||
so the XR process died with a seccomp crash (arm64 syscall 209). The patch
|
||||
allows that one option. It's needed but not enough: the launcher still turns
|
||||
seccomp off (below), so the patch only matters once that's fixed too.
|
||||
|
||||
## Running it on the Frame
|
||||
**Seccomp is off.** The launcher passes `--disable-seccomp-filter-sandbox`.
|
||||
With the XR seccomp policy on, SteamVR's client reads `/proc/self/status`
|
||||
through Chrome's file broker and gets the broker's pid. SteamVR then binds
|
||||
the app to the wrong process ("Unable to init path manager:
|
||||
VRInitError_Init_Internal") and `xrCreateInstance` fails. The broker can't
|
||||
answer `/proc/self` for another process, so fixing this needs a change in
|
||||
Chromium's broker client or in the CL. The namespace sandbox stays on, but
|
||||
seccomp is off for every process, so use this profile for VR sites rather
|
||||
than everyday browsing.
|
||||
|
||||
[`scripts/chromium-xr.sh`](../scripts/chromium-xr.sh):
|
||||
|
||||
```sh
|
||||
BUILD_HOST=my-linux-box scripts/chromium-xr.sh install # your build host; scp, unpack to ~/chromium-xr
|
||||
scripts/chromium-xr.sh launch [URL] # its own VR panel, --enable-features=OpenXR
|
||||
scripts/chromium-xr.sh steam # adds "Chromium XR" to the Steam library
|
||||
scripts/chromium-xr.sh check # prints isSessionSupported('immersive-vr')
|
||||
```
|
||||
|
||||
It runs natively, not as a Flatpak. `launch` opens it as its own panel on
|
||||
gamescope's X display, the same way as [`panel-on-frame.sh`](../scripts/panel-on-frame.sh) ([panels.md](panels.md)), so
|
||||
the Plasma desktop doesn't need to be open. It uses its own profile
|
||||
(`~/.config/chromium-xr`) and DevTools on loopback port 9223, so it doesn't
|
||||
collide with the Flatpak's 9222. When a page enters VR, Chrome asks
|
||||
**Allow VR?** in the browser panel; choose *Allow this time* or *Allow while
|
||||
visiting the site*.
|
||||
|
||||
Both ways of starting it run
|
||||
[`frame/chromium-xr/launch.sh`](../frame/chromium-xr/launch.sh), copied to
|
||||
`~/Applications/ChromiumXR/launch.sh` on the Frame, which holds Chrome's flags.
|
||||
It sits outside `~/chromium-xr` so `install` doesn't delete it.
|
||||
|
||||
**From the Steam library (verified 2026-09-26).** `steam` adds a non-Steam
|
||||
shortcut called "Chromium XR" (with the Flathub Chromium icon, if that's
|
||||
installed) through the Steam client's DevTools port, the same way as T3 Code
|
||||
([apks.md](apks.md)), without restarting Steam. It saves the app id in
|
||||
`~/Applications/ChromiumXR/shortcut-appid`, so rerunning it, even after you
|
||||
rename the shortcut in the library, doesn't add a second one. On this Frame
|
||||
the shortcut app id is 2240749789. Launching it from the library gives
|
||||
Chromium its own panel, `valve.steam.desktopgame.2240749789`, like any other
|
||||
app. A Steam launch doesn't open a DevTools port, so `check` needs `launch`.
|
||||
Chromium runs one browser per profile: while the Steam-launched one is open,
|
||||
`launch` opens its URL in that window, without DevTools, then prints
|
||||
`failed: ... exited` because no new window appeared. Close it first.
|
||||
|
||||
**Seccomp is off.** The wrapper passes `--disable-seccomp-filter-sandbox`. With
|
||||
the XR seccomp policy on, SteamVR's client reads `/proc/self/status` through
|
||||
Chrome's file broker and gets the broker's pid. SteamVR then binds the app to
|
||||
the wrong process ("Unable to init path manager: VRInitError_Init_Internal")
|
||||
and `xrCreateInstance` fails. The broker can't answer `/proc/self` for another
|
||||
process, so fixing this needs a change in Chromium's broker client or in the
|
||||
CL. The namespace sandbox stays on, but seccomp is off for every process, so
|
||||
use this profile for VR sites rather than everyday browsing. DevTools on
|
||||
port 9223 has no authentication. It listens on loopback, but with the
|
||||
userspace Tailscale from [tailscale.md](tailscale.md) running, loopback ports
|
||||
are reachable from your tailnet. Close the browser when you're done.
|
||||
**Upstream (2026-09-27).** CL 8441736 (the XR sandbox) has merged into
|
||||
Chromium, still refusing `getsockopt`; CL 8132979 is still in review. Valve
|
||||
and the CLs' author are working on Steam Frame support
|
||||
([utzcoz/chromium-webxr-linux#5](https://github.com/utzcoz/chromium-webxr-linux/issues/5)).
|
||||
Both sandbox problems above, with the patch, are reported in
|
||||
[utzcoz/chromium-webxr-linux#7](https://github.com/utzcoz/chromium-webxr-linux/issues/7).
|
||||
|
||||
**Verified 2026-09-26** (Frame BUILD_ID 20260922.6101926, SteamVR 2.17.10,
|
||||
this build):
|
||||
@@ -138,9 +116,26 @@ this build):
|
||||
[`webxr_vr_video`](https://threejs.org/examples/webxr_vr_video.html) demo,
|
||||
a stereo 360 video, played in 3D after pressing Enter VR.
|
||||
|
||||
**Not verified yet:**
|
||||
- **Launched from the Steam library, verified remotely 2026-09-27** with
|
||||
nobody wearing the headset (standby workaround in
|
||||
[how-the-frame-works.md](how-the-frame-works.md)). The installer's Steam
|
||||
shortcut starts Chromium, and SteamVR takes it as scene app
|
||||
`steam.app.<shortcut id>`. A minimal WebXR session that clears every frame
|
||||
to red ran at about 75 frames per second, and the stereo headset capture
|
||||
showed both eyes solid red. Steam preloads its overlay
|
||||
(`gameoverlayrenderer.so`), which crashed Chromium's zygote about 30 s after
|
||||
a Steam launch. The public repo's launcher now removes it from
|
||||
`LD_PRELOAD`. With the headset outside its playspace, SteamVR shows
|
||||
passthrough wherever the page leaves transparent pixels.
|
||||
|
||||
- Frame rate and dropped frames during playback (nothing was measured; it
|
||||
looked fine).
|
||||
- Third-party VR180 players (DeoVR and DL8 web embeds).
|
||||
- Controller and hand input inside a WebXR page.
|
||||
- **Frame rate and input, measured 2026-09-27** (standby workaround, red
|
||||
test session): 72 fps with every frame at 13.9–14 ms over 16 s, and SteamVR
|
||||
dropped frames only at startup. The right controller showed up as an
|
||||
`oculus-touch` `tracked-pointer` with an `xr-standard` gamepad and a
|
||||
25-joint hand, with poses on every frame. A real squeeze reached the page
|
||||
as `squeezestart`/`squeeze`. Haptics aren't exposed (no actuators).
|
||||
Details are in the public repo's technical notes.
|
||||
|
||||
**Not verified yet:** trigger, thumbstick and face buttons, the left
|
||||
controller, bare-hand tracking, and third-party VR180 players (DeoVR and
|
||||
DL8 web embeds).
|
||||
@@ -1,27 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Frame-side: start the WebXR Chromium build (~/chromium-xr). The Steam
|
||||
# library shortcut "Chromium XR" runs this, and so does
|
||||
# `scripts/chromium-xr.sh launch` (which adds a DevTools port). Extra
|
||||
# arguments go to Chrome, so a URL opens that page.
|
||||
#
|
||||
# Lives in ~/Applications/ChromiumXR, outside ~/chromium-xr, so reinstalling
|
||||
# the build doesn't delete it.
|
||||
set -euo pipefail
|
||||
|
||||
CHROME="$HOME/chromium-xr/chrome"
|
||||
[[ -x "$CHROME" ]] || { echo "launch.sh: no build at $CHROME (run chromium-xr.sh install)" >&2; exit 1; }
|
||||
|
||||
# Without --no-first-run and --password-store=basic, startup can stop at a
|
||||
# first-run or keyring prompt.
|
||||
# --disable-seccomp-filter-sandbox: under the XR seccomp policy, SteamVR's
|
||||
# client reads /proc/self/status through the file broker, gets the broker's
|
||||
# pid, and SteamVR binds the app to the wrong process, so xrCreateInstance
|
||||
# fails. The namespace sandbox stays on, but seccomp is off for every
|
||||
# process, so keep this profile for VR sites.
|
||||
exec "$CHROME" \
|
||||
--user-data-dir="$HOME/.config/chromium-xr" \
|
||||
--enable-features=OpenXR \
|
||||
--ozone-platform=x11 \
|
||||
--no-first-run --no-default-browser-check --password-store=basic \
|
||||
--disable-seccomp-filter-sandbox \
|
||||
"$@"
|
||||
@@ -19,6 +19,7 @@
|
||||
9657F7BC23E3352E5AB30777 /* SetupView.swift in Sources */ = {isa = PBXBuildFile; fileRef = DB544223FC60A59CC3E8EF5F /* SetupView.swift */; };
|
||||
A8C7AED25A6280682FCE45DC /* Citadel in Frameworks */ = {isa = PBXBuildFile; productRef = 6BA549B6CC0A0CB847126456 /* Citadel */; };
|
||||
DC043FB74BE2D23F3A5826BF /* FrameControlTests.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1740B691F9C25E5FB6F9EFC3 /* FrameControlTests.swift */; };
|
||||
E6898C714A92D3979F73B6E1 /* FrameFinder.swift in Sources */ = {isa = PBXBuildFile; fileRef = 2F288DF6636A417F0CA3A6CD /* FrameFinder.swift */; };
|
||||
F94D0252F8CC5854314B84B2 /* AppModel.swift in Sources */ = {isa = PBXBuildFile; fileRef = 8A11F3431826A26B247C0695 /* AppModel.swift */; };
|
||||
/* End PBXBuildFile section */
|
||||
|
||||
@@ -36,6 +37,7 @@
|
||||
16644E7FDA7ADD5B232EB700 /* FrameLink.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FrameLink.swift; sourceTree = "<group>"; };
|
||||
1740B691F9C25E5FB6F9EFC3 /* FrameControlTests.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FrameControlTests.swift; sourceTree = "<group>"; };
|
||||
237D9AF04EEA257AB382F60E /* Keys.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = Keys.swift; sourceTree = "<group>"; };
|
||||
2F288DF6636A417F0CA3A6CD /* FrameFinder.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = FrameFinder.swift; sourceTree = "<group>"; };
|
||||
6B5B6718C5EA77FA67F6B14C /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist; path = Info.plist; sourceTree = "<group>"; };
|
||||
6FBC8D0B5ED7BF1C06F99892 /* FrameControlTests.xctest */ = {isa = PBXFileReference; includeInIndex = 0; lastKnownFileType = wrapper.cfbundle; path = FrameControlTests.xctest; sourceTree = BUILT_PRODUCTS_DIR; };
|
||||
8A11F3431826A26B247C0695 /* AppModel.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = AppModel.swift; sourceTree = "<group>"; };
|
||||
@@ -74,6 +76,7 @@
|
||||
5064A5B6FE5B17B18E5FA4B8 /* SSH */ = {
|
||||
isa = PBXGroup;
|
||||
children = (
|
||||
2F288DF6636A417F0CA3A6CD /* FrameFinder.swift */,
|
||||
16644E7FDA7ADD5B232EB700 /* FrameLink.swift */,
|
||||
EDC7BA8014DBC302D08FD397 /* HeadsetServer.swift */,
|
||||
237D9AF04EEA257AB382F60E /* Keys.swift */,
|
||||
@@ -255,6 +258,7 @@
|
||||
files = (
|
||||
F94D0252F8CC5854314B84B2 /* AppModel.swift in Sources */,
|
||||
78427FC66780623F31E7501E /* FrameControlApp.swift in Sources */,
|
||||
E6898C714A92D3979F73B6E1 /* FrameFinder.swift in Sources */,
|
||||
12B21D3319BAF5AE79948560 /* FrameLink.swift in Sources */,
|
||||
0DEE50BD563B1D8C328C4C0A /* HeadsetServer.swift in Sources */,
|
||||
4622FE0F0D6499CD642C29A2 /* InstallLink.swift in Sources */,
|
||||
|
||||
@@ -92,7 +92,7 @@ final class AppModel: ObservableObject {
|
||||
}
|
||||
|
||||
/// "host", "host:port" or "[v6]:port", plus a user name.
|
||||
static func parse(host: String, user: String) -> FrameSettings? {
|
||||
nonisolated static func parse(host: String, user: String) -> FrameSettings? {
|
||||
var target = FrameSettings(host: host.trimmingCharacters(in: .whitespaces), user: user.trimmingCharacters(in: .whitespaces))
|
||||
if target.host.hasPrefix("["), let close = target.host.firstIndex(of: "]") {
|
||||
let rest = target.host[target.host.index(after: close)...]
|
||||
|
||||
@@ -14,6 +14,11 @@ struct FrameControlApp: App {
|
||||
// and connect to FRAME_TEST_HOST with it (`simctl launch` passes
|
||||
// SIMCTL_CHILD_FRAME_TEST_HOST through as FRAME_TEST_HOST).
|
||||
print("FRAME_CONTROL_KEY: \(model.authorizedKeysLine)")
|
||||
// FRAME_TEST_LANDSCAPE=1 turns the app on its side, to check the safe areas there.
|
||||
if ProcessInfo.processInfo.environment["FRAME_TEST_LANDSCAPE"] != nil,
|
||||
let scene = UIApplication.shared.connectedScenes.first as? UIWindowScene {
|
||||
scene.requestGeometryUpdate(.iOS(interfaceOrientations: .landscapeRight))
|
||||
}
|
||||
// FRAME_TEST_PAIR="host|user|password" runs the real password pairing.
|
||||
if model.settings == nil, let pair = ProcessInfo.processInfo.environment["FRAME_TEST_PAIR"] {
|
||||
let f = pair.components(separatedBy: "|")
|
||||
|
||||
@@ -43,8 +43,14 @@
|
||||
<key>NSAllowsLocalNetworking</key>
|
||||
<true/>
|
||||
</dict>
|
||||
<key>NSBonjourServices</key>
|
||||
<array>
|
||||
<string>_steamos-devkit._tcp</string>
|
||||
</array>
|
||||
<key>NSLocalNetworkUsageDescription</key>
|
||||
<string>Frame Control connects to your Steam Frame over your local network with SSH.</string>
|
||||
<string>Frame Control finds your Steam Frame on your network and connects to it.</string>
|
||||
<key>NSPhotoLibraryAddUsageDescription</key>
|
||||
<string>Frame Control saves headset captures and screenshots to your photo library when you ask it to.</string>
|
||||
<key>UILaunchScreen</key>
|
||||
<dict>
|
||||
<key>UIColorName</key>
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
import Foundation
|
||||
import Network
|
||||
|
||||
/// Finds the Frame on the local network so nobody has to type its address.
|
||||
/// A Frame in Developer Mode advertises Valve's devkit service over Bonjour
|
||||
/// (`_steamos-devkit._tcp`); failing that, `fallback` (the saved address, or
|
||||
/// frame.local) is checked by opening its SSH port. Both repeat until stopped.
|
||||
@MainActor
|
||||
final class FrameFinder: ObservableObject {
|
||||
struct Found: Equatable {
|
||||
let host: String // what to connect to
|
||||
let name: String // what to call it
|
||||
}
|
||||
|
||||
@Published private(set) var found: Found?
|
||||
/// When the search began, to tell "still looking" from "can't find it".
|
||||
@Published private(set) var since = Date()
|
||||
|
||||
private var browser: NWBrowser?
|
||||
private var probeTask: Task<Void, Never>?
|
||||
private var fallback = "frame.local"
|
||||
|
||||
func start(fallback: String?) {
|
||||
stop()
|
||||
self.fallback = (fallback?.isEmpty == false ? fallback : nil) ?? "frame.local"
|
||||
found = nil
|
||||
since = Date()
|
||||
browse()
|
||||
probeTask = Task { [weak self] in
|
||||
while !Task.isCancelled {
|
||||
guard let self else { return }
|
||||
let host = self.fallback
|
||||
if self.found == nil, await Self.sshAnswers(host: host) {
|
||||
self.found = Found(host: host, name: host)
|
||||
}
|
||||
try? await Task.sleep(nanoseconds: 3_000_000_000)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func stop() {
|
||||
browser?.cancel()
|
||||
browser = nil
|
||||
probeTask?.cancel()
|
||||
probeTask = nil
|
||||
}
|
||||
|
||||
private func browse() {
|
||||
let browser = NWBrowser(for: .bonjour(type: "_steamos-devkit._tcp", domain: nil), using: .tcp)
|
||||
browser.browseResultsChangedHandler = { [weak self] results, _ in
|
||||
for result in results {
|
||||
guard case let .service(name, _, _, _) = result.endpoint else { continue }
|
||||
Self.resolve(result.endpoint) { host in
|
||||
Task { @MainActor in
|
||||
guard let self, let host else { return }
|
||||
// A found device is used over the fallback probe.
|
||||
self.found = Found(host: host, name: name)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
browser.start(queue: .main)
|
||||
self.browser = browser
|
||||
}
|
||||
|
||||
/// The device's IP address: connect to the service and read where it went.
|
||||
nonisolated private static func resolve(_ endpoint: NWEndpoint, done: @escaping @Sendable (String?) -> Void) {
|
||||
let connection = NWConnection(to: endpoint, using: .tcp)
|
||||
let once = Once()
|
||||
connection.stateUpdateHandler = { state in
|
||||
switch state {
|
||||
case .ready:
|
||||
var host: String?
|
||||
if case let .hostPort(h, _)? = connection.currentPath?.remoteEndpoint {
|
||||
host = "\(h)".components(separatedBy: "%").first // drop an IPv6 interface suffix
|
||||
}
|
||||
connection.cancel()
|
||||
if once.claim() { done(host) }
|
||||
case .failed, .cancelled:
|
||||
if once.claim() { done(nil) }
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
connection.start(queue: .global())
|
||||
DispatchQueue.global().asyncAfter(deadline: .now() + 5) {
|
||||
connection.cancel()
|
||||
if once.claim() { done(nil) }
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether something answers on the SSH port of "host" or "host:port" within a few seconds.
|
||||
nonisolated static func sshAnswers(host address: String) async -> Bool {
|
||||
var host = address, port: UInt16 = 22
|
||||
if let target = AppModel.parse(host: address, user: "steamos") {
|
||||
host = target.host
|
||||
port = UInt16(target.port)
|
||||
}
|
||||
return await sshAnswers(host: host, port: port)
|
||||
}
|
||||
|
||||
nonisolated static func sshAnswers(host: String, port: UInt16) async -> Bool {
|
||||
await withCheckedContinuation { (c: CheckedContinuation<Bool, Never>) in
|
||||
let connection = NWConnection(host: NWEndpoint.Host(host), port: NWEndpoint.Port(rawValue: port) ?? 22, using: .tcp)
|
||||
let once = Once()
|
||||
connection.stateUpdateHandler = { state in
|
||||
switch state {
|
||||
case .ready:
|
||||
connection.cancel()
|
||||
if once.claim() { c.resume(returning: true) }
|
||||
case .failed, .waiting:
|
||||
connection.cancel()
|
||||
if once.claim() { c.resume(returning: false) }
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
connection.start(queue: .global())
|
||||
DispatchQueue.global().asyncAfter(deadline: .now() + 3) {
|
||||
connection.cancel()
|
||||
if once.claim() { c.resume(returning: false) }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -11,6 +11,9 @@ struct RootView: View {
|
||||
SetupView(model: model)
|
||||
case .connecting(let step):
|
||||
ConnectingView(step: step, host: model.settings?.host) { model.showSetup() }
|
||||
case .failed(let message) where model.retrying && !model.needsPairing:
|
||||
WaitingView(host: model.settings.map { $0.port == 22 ? $0.host : "\($0.host):\($0.port)" } ?? "", detail: message, deviceName: model.deviceName,
|
||||
reachable: { Task { await model.connect(quiet: true) } }, change: { model.showSetup() })
|
||||
case .failed(let message):
|
||||
FailedView(message: message, canRetry: model.settings != nil, retrying: model.retrying, needsPairing: model.needsPairing,
|
||||
retry: { Task { await model.connect() } }, change: { model.showSetup() })
|
||||
@@ -73,3 +76,46 @@ struct FailedView: View {
|
||||
.frame(maxWidth: 480)
|
||||
}
|
||||
}
|
||||
|
||||
/// A paired Frame that isn't answering is almost always asleep: say how to wake
|
||||
/// it, and connect the moment it does (its SSH port is checked every 3 s).
|
||||
struct WaitingView: View {
|
||||
let host: String
|
||||
let detail: String
|
||||
let deviceName: String
|
||||
let reachable: () -> Void
|
||||
let change: () -> Void
|
||||
@State private var pulse = false
|
||||
|
||||
var body: some View {
|
||||
VStack(spacing: 18) {
|
||||
Image("AppIconImage").resizable().frame(width: 76, height: 76)
|
||||
.clipShape(RoundedRectangle(cornerRadius: 17))
|
||||
.opacity(pulse ? 1 : 0.55)
|
||||
.animation(.easeInOut(duration: 1.2).repeatForever(autoreverses: true), value: pulse)
|
||||
Text("Waiting for your Frame").font(.title3.bold())
|
||||
Text("Put the headset on, or press its power button, to wake it. Frame Control connects by itself as soon as it's awake.")
|
||||
.multilineTextAlignment(.center)
|
||||
VStack(alignment: .leading, spacing: 10) {
|
||||
Tip(icon: "wifi", text: "Same Wi-Fi as this \(deviceName), or both on Tailscale.")
|
||||
Tip(icon: "bolt.horizontal", text: "Asleep, the Frame drops off the network entirely; nothing can wake it remotely.")
|
||||
}
|
||||
.padding(14)
|
||||
.background(Color.framePanel, in: RoundedRectangle(cornerRadius: 12))
|
||||
Text(detail).font(.footnote).foregroundStyle(Color.frameMuted).multilineTextAlignment(.center)
|
||||
Button("Connect to a different Frame", action: change).font(.footnote)
|
||||
}
|
||||
.padding(28)
|
||||
.frame(maxWidth: 480)
|
||||
.onAppear { pulse = true }
|
||||
.task(id: host) {
|
||||
while !Task.isCancelled {
|
||||
try? await Task.sleep(nanoseconds: 3_000_000_000)
|
||||
if !host.isEmpty, await FrameFinder.sshAnswers(host: host) {
|
||||
reachable()
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,82 +1,197 @@
|
||||
import SwiftUI
|
||||
import UIKit
|
||||
|
||||
/// Pairing: the Developer Mode password is used once, to add this phone's own
|
||||
/// key to the Frame. It isn't stored.
|
||||
/// First run: two steps. Wake the Frame (the app finds it by itself), then type the
|
||||
/// Developer Mode password once. Everything else waits under "Other ways to connect".
|
||||
struct SetupView: View {
|
||||
@ObservedObject var model: AppModel
|
||||
@State private var host = ""
|
||||
@State private var user = "steamos"
|
||||
@StateObject private var finder = FrameFinder()
|
||||
@State private var password = ""
|
||||
@FocusState private var focus: Field?
|
||||
private enum Field { case host, user, password }
|
||||
@State private var manualHost = ""
|
||||
@State private var user = "steamos"
|
||||
@State private var showOther = false
|
||||
@State private var showHelp = false
|
||||
@FocusState private var passwordFocused: Bool
|
||||
|
||||
/// Where Connect goes: what the finder saw, else what was typed, else frame.local.
|
||||
private var host: String {
|
||||
let typed = manualHost.trimmingCharacters(in: .whitespaces)
|
||||
return finder.found?.host ?? (typed.isEmpty ? "frame.local" : typed)
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
NavigationStack {
|
||||
Form {
|
||||
Section {
|
||||
VStack(alignment: .leading, spacing: 10) {
|
||||
Image("AppIconImage").resizable().frame(width: 64, height: 64).clipShape(RoundedRectangle(cornerRadius: 14))
|
||||
Text("Connect to your Steam Frame").font(.title2.bold())
|
||||
Text("See what the headset sees, install games and Android apps, and send files and text, from this \(model.deviceName).")
|
||||
.foregroundStyle(Color.frameMuted)
|
||||
}
|
||||
.padding(.vertical, 6)
|
||||
.listRowBackground(Color.clear)
|
||||
}
|
||||
Section {
|
||||
Label("On the Frame, open Steam Settings → System and turn on Developer Mode.", systemImage: "1.circle")
|
||||
Label("Then Developer → Set User Password.", systemImage: "2.circle")
|
||||
Label("Enter the headset's address and that password here, once.", systemImage: "3.circle")
|
||||
} header: { Text("Before you start") }
|
||||
Section {
|
||||
TextField("frame.local or 192.168.1.20", text: $host)
|
||||
.textContentType(.URL).keyboardType(.URL).autocorrectionDisabled().textInputAutocapitalization(.never)
|
||||
.focused($focus, equals: .host).submitLabel(.next).onSubmit { focus = .password }
|
||||
TextField("User", text: $user)
|
||||
.autocorrectionDisabled().textInputAutocapitalization(.never).focused($focus, equals: .user)
|
||||
SecureField("Developer Mode password", text: $password)
|
||||
.textContentType(.password).focused($focus, equals: .password).submitLabel(.go).onSubmit(pair)
|
||||
} header: { Text("Headset") } footer: {
|
||||
Text("The password is only used to add this \(model.deviceName)'s own SSH key to the Frame; it isn't saved. The Frame and this \(model.deviceName) need to be on the same network, or both on Tailscale.")
|
||||
}
|
||||
Section {
|
||||
Button(action: pair) {
|
||||
Text("Pair").frame(maxWidth: .infinity).fontWeight(.semibold)
|
||||
}
|
||||
.disabled(host.trimmingCharacters(in: .whitespaces).isEmpty || password.isEmpty)
|
||||
if model.settings != nil {
|
||||
Button("Use \(model.settings!.host) again") { Task { await model.connect() } }
|
||||
Button("Forget this headset", role: .destructive) { Task { await model.forget() } }
|
||||
}
|
||||
}
|
||||
Section {
|
||||
Text(model.authorizedKeysLine)
|
||||
.font(.system(.caption2, design: .monospaced)).lineLimit(3).textSelection(.enabled)
|
||||
Button("Copy this \(model.deviceName)'s key") { UIPasteboard.general.string = model.authorizedKeysLine }
|
||||
Button("Connect with the key") {
|
||||
let (h, u) = (host, user)
|
||||
Task { await model.useKey(host: h, user: u) }
|
||||
}
|
||||
.disabled(host.trimmingCharacters(in: .whitespaces).isEmpty)
|
||||
} header: { Text("Or add the key yourself") } footer: {
|
||||
Text("If you already reach the Frame over SSH, add this line to ~/.ssh/authorized_keys there, then connect without a password.")
|
||||
}
|
||||
ScrollView {
|
||||
VStack(alignment: .leading, spacing: 22) {
|
||||
header
|
||||
StepCard(number: 1, title: "Wake your Frame", done: finder.found != nil) { wakeStep }
|
||||
StepCard(number: 2, title: "Enter its Developer Mode password", done: false) { passwordStep }
|
||||
otherWays
|
||||
}
|
||||
.scrollContentBackground(.hidden)
|
||||
.background(Color.frameBackground)
|
||||
.navigationTitle("Frame Control")
|
||||
.navigationBarTitleDisplayMode(.inline)
|
||||
.padding(20)
|
||||
.frame(maxWidth: 560)
|
||||
.frame(maxWidth: .infinity)
|
||||
}
|
||||
.scrollDismissesKeyboard(.interactively)
|
||||
.background(Color.frameBackground)
|
||||
.onAppear {
|
||||
host = model.settings?.host ?? "frame.local"
|
||||
manualHost = model.settings?.host ?? ""
|
||||
user = model.settings?.user ?? "steamos"
|
||||
var fallback = model.settings?.host
|
||||
#if DEBUG
|
||||
fallback = ProcessInfo.processInfo.environment["FRAME_TEST_FALLBACK"] ?? fallback // Simulator test hook
|
||||
#endif
|
||||
finder.start(fallback: fallback)
|
||||
}
|
||||
.onDisappear { finder.stop() }
|
||||
// After a few seconds of not finding it, say exactly what to check.
|
||||
.task(id: finder.since) {
|
||||
try? await Task.sleep(nanoseconds: 8_000_000_000)
|
||||
showHelp = true
|
||||
}
|
||||
}
|
||||
|
||||
private func pair() {
|
||||
private var header: some View {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
Image("AppIconImage").resizable().frame(width: 56, height: 56).clipShape(RoundedRectangle(cornerRadius: 13))
|
||||
Text("Connect to your Steam Frame").font(.title2.bold())
|
||||
Text("One time only. After this, the app connects by itself whenever your Frame is awake.")
|
||||
.foregroundStyle(Color.frameMuted)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: step 1
|
||||
|
||||
@ViewBuilder private var wakeStep: some View {
|
||||
if let found = finder.found {
|
||||
Label {
|
||||
VStack(alignment: .leading, spacing: 2) {
|
||||
Text("Found your Frame").fontWeight(.semibold)
|
||||
Text(found.name == found.host ? found.host : "\(found.name) · \(found.host)")
|
||||
.font(.footnote).foregroundStyle(Color.frameMuted)
|
||||
}
|
||||
} icon: {
|
||||
Image(systemName: "checkmark.circle.fill").foregroundStyle(.green)
|
||||
}
|
||||
} else {
|
||||
HStack(spacing: 10) {
|
||||
ProgressView()
|
||||
Text("Looking for it on this network…").foregroundStyle(Color.frameMuted)
|
||||
}
|
||||
Text("Put the headset on, or press its power button, so it's awake.")
|
||||
if showHelp {
|
||||
VStack(alignment: .leading, spacing: 10) {
|
||||
Text("Still can't see it? Check:").font(.subheadline.weight(.semibold))
|
||||
Tip(icon: "wifi", text: "The Frame and this \(model.deviceName) are on the same Wi-Fi.")
|
||||
Tip(icon: "hammer", text: "Developer Mode is on: on the Frame, Steam Settings → System → Enable Developer Mode.")
|
||||
Tip(icon: "network", text: "Local Network is allowed for Frame Control: \(model.deviceName) Settings → Apps → Frame Control.")
|
||||
}
|
||||
.padding(.top, 4)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: step 2
|
||||
|
||||
@ViewBuilder private var passwordStep: some View {
|
||||
SecureField("Developer Mode password", text: $password)
|
||||
.textContentType(.password)
|
||||
.submitLabel(.go)
|
||||
.focused($passwordFocused)
|
||||
.onSubmit(connect)
|
||||
.padding(12)
|
||||
.background(Color.black.opacity(0.28), in: RoundedRectangle(cornerRadius: 10))
|
||||
Text("Haven't set one? On the Frame: Steam Settings → Developer → Set User Password. It's only used now, to let this \(model.deviceName) in; it isn't saved.")
|
||||
.font(.footnote).foregroundStyle(Color.frameMuted)
|
||||
Button(action: connect) {
|
||||
Text(finder.found == nil ? "Connect to \(host)" : "Connect")
|
||||
.fontWeight(.semibold).frame(maxWidth: .infinity).padding(.vertical, 4)
|
||||
}
|
||||
.buttonStyle(.borderedProminent)
|
||||
.controlSize(.large)
|
||||
.disabled(password.isEmpty)
|
||||
}
|
||||
|
||||
// MARK: everything else, out of the way
|
||||
|
||||
private var otherWays: some View {
|
||||
DisclosureGroup(isExpanded: $showOther) {
|
||||
VStack(alignment: .leading, spacing: 14) {
|
||||
VStack(alignment: .leading, spacing: 6) {
|
||||
Text("Address").font(.footnote).foregroundStyle(Color.frameMuted)
|
||||
TextField("frame.local, an IP, or a Tailscale name", text: $manualHost)
|
||||
.keyboardType(.URL).textInputAutocapitalization(.never).autocorrectionDisabled()
|
||||
.onSubmit { finder.start(fallback: manualHost) }
|
||||
.padding(10).background(Color.black.opacity(0.28), in: RoundedRectangle(cornerRadius: 8))
|
||||
TextField("User", text: $user)
|
||||
.textInputAutocapitalization(.never).autocorrectionDisabled()
|
||||
.padding(10).background(Color.black.opacity(0.28), in: RoundedRectangle(cornerRadius: 8))
|
||||
Text("Typing an address here uses it instead of searching.").font(.caption).foregroundStyle(Color.frameMuted)
|
||||
}
|
||||
VStack(alignment: .leading, spacing: 6) {
|
||||
Text("Already reach the Frame over SSH? Add this \(model.deviceName)'s key to ~/.ssh/authorized_keys there, then connect without a password.")
|
||||
.font(.footnote).foregroundStyle(Color.frameMuted)
|
||||
HStack {
|
||||
Button("Copy key") { UIPasteboard.general.string = model.authorizedKeysLine }
|
||||
Spacer()
|
||||
Button("Connect with the key") {
|
||||
let (h, u) = (host, user)
|
||||
Task { await model.useKey(host: h, user: u) }
|
||||
}
|
||||
}
|
||||
}
|
||||
if let saved = model.settings {
|
||||
Button("Forget \(saved.host)", role: .destructive) { Task { await model.forget() } }
|
||||
}
|
||||
}
|
||||
.padding(.top, 10)
|
||||
} label: {
|
||||
Text("Other ways to connect").foregroundStyle(Color.frameMuted)
|
||||
}
|
||||
.onChange(of: manualHost) { _, value in
|
||||
// A typed address replaces the search.
|
||||
if !value.trimmingCharacters(in: .whitespaces).isEmpty, finder.found?.host != value { finder.start(fallback: value) }
|
||||
}
|
||||
}
|
||||
|
||||
private func connect() {
|
||||
guard !password.isEmpty else { passwordFocused = true; return }
|
||||
let (h, u, p) = (host, user, password)
|
||||
password = ""
|
||||
finder.stop()
|
||||
Task { await model.pair(host: h, user: u, password: p) }
|
||||
}
|
||||
}
|
||||
|
||||
/// A numbered step with a tick once it's done.
|
||||
struct StepCard<Content: View>: View {
|
||||
let number: Int
|
||||
let title: String
|
||||
let done: Bool
|
||||
@ViewBuilder let content: Content
|
||||
|
||||
var body: some View {
|
||||
VStack(alignment: .leading, spacing: 12) {
|
||||
HStack(spacing: 10) {
|
||||
ZStack {
|
||||
Circle().fill(done ? Color.green : Color.frameBlue).frame(width: 26, height: 26)
|
||||
if done { Image(systemName: "checkmark").font(.caption.bold()) } else { Text("\(number)").font(.subheadline.bold()) }
|
||||
}
|
||||
.foregroundStyle(.white)
|
||||
Text(title).font(.headline)
|
||||
}
|
||||
content
|
||||
}
|
||||
.padding(16)
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
.background(Color.framePanel, in: RoundedRectangle(cornerRadius: 14))
|
||||
}
|
||||
}
|
||||
|
||||
struct Tip: View {
|
||||
let icon: String
|
||||
let text: String
|
||||
|
||||
var body: some View {
|
||||
Label { Text(text).font(.subheadline).fixedSize(horizontal: false, vertical: true) } icon: { Image(systemName: icon).foregroundStyle(Color.frameBlue) }
|
||||
}
|
||||
}
|
||||
+3
-1
@@ -39,7 +39,9 @@ targets:
|
||||
UISupportedInterfaceOrientations: [UIInterfaceOrientationPortrait, UIInterfaceOrientationLandscapeLeft, UIInterfaceOrientationLandscapeRight]
|
||||
UISupportedInterfaceOrientations~ipad: [UIInterfaceOrientationPortrait, UIInterfaceOrientationPortraitUpsideDown, UIInterfaceOrientationLandscapeLeft, UIInterfaceOrientationLandscapeRight]
|
||||
UIUserInterfaceStyle: Dark
|
||||
NSLocalNetworkUsageDescription: Frame Control connects to your Steam Frame over your local network with SSH.
|
||||
NSLocalNetworkUsageDescription: Frame Control finds your Steam Frame on your network and connects to it.
|
||||
NSBonjourServices: [_steamos-devkit._tcp]
|
||||
NSPhotoLibraryAddUsageDescription: Frame Control saves headset captures and screenshots to your photo library when you ask it to.
|
||||
NSAppTransportSecurity:
|
||||
NSAllowsLocalNetworking: true
|
||||
LSApplicationQueriesSchemes: [ssh, sftp, steamlink, rdp]
|
||||
|
||||
@@ -0,0 +1,28 @@
|
||||
// Private CoreGraphics classes for virtual displays (as used by DeskPad,
|
||||
// BetterDisplay and quest-display). Not in any SDK; see
|
||||
// docs/mac-window-separation.md.
|
||||
#import <Foundation/Foundation.h>
|
||||
#import <CoreGraphics/CoreGraphics.h>
|
||||
@interface CGVirtualDisplayDescriptor : NSObject
|
||||
@property (retain, nonatomic) dispatch_queue_t queue;
|
||||
@property (retain, nonatomic) NSString *name;
|
||||
@property (nonatomic) unsigned int maxPixelsHigh;
|
||||
@property (nonatomic) unsigned int maxPixelsWide;
|
||||
@property (nonatomic) CGSize sizeInMillimeters;
|
||||
@property (nonatomic) unsigned int serialNum;
|
||||
@property (nonatomic) unsigned int productID;
|
||||
@property (nonatomic) unsigned int vendorID;
|
||||
@property (copy, nonatomic) void (^terminationHandler)(id, id);
|
||||
@end
|
||||
@interface CGVirtualDisplayMode : NSObject
|
||||
- (instancetype)initWithWidth:(unsigned int)width height:(unsigned int)height refreshRate:(double)refreshRate;
|
||||
@end
|
||||
@interface CGVirtualDisplaySettings : NSObject
|
||||
@property (retain, nonatomic) NSArray *modes;
|
||||
@property (nonatomic) unsigned int hiDPI;
|
||||
@end
|
||||
@interface CGVirtualDisplay : NSObject
|
||||
@property (readonly, nonatomic) unsigned int displayID;
|
||||
- (instancetype)initWithDescriptor:(CGVirtualDisplayDescriptor *)descriptor;
|
||||
- (BOOL)applySettings:(CGVirtualDisplaySettings *)settings;
|
||||
@end
|
||||
@@ -0,0 +1,414 @@
|
||||
// Where pictures come from: one Mac window or one display (ScreenCaptureKit),
|
||||
// or a generated test pattern that needs no permissions and shows the input
|
||||
// the viewer sends, for checking the whole path without touching the Mac.
|
||||
import AppKit
|
||||
import CoreMedia
|
||||
import CoreVideo
|
||||
import Foundation
|
||||
import ScreenCaptureKit
|
||||
|
||||
enum Source: Equatable {
|
||||
case window(CGWindowID)
|
||||
case display(CGDirectDisplayID)
|
||||
case separate(CGWindowID) // the window on a display of its own
|
||||
case test
|
||||
|
||||
init?(_ s: String) {
|
||||
let parts = s.split(separator: ":", maxSplits: 1).map(String.init)
|
||||
switch (parts.first, parts.count > 1 ? UInt32(parts[1]) : nil) {
|
||||
case ("window", let id?): self = .window(id)
|
||||
case ("display", let id?): self = .display(id)
|
||||
case ("separate", let id?): self = .separate(id)
|
||||
case ("test", _): self = .test
|
||||
default: return nil
|
||||
}
|
||||
}
|
||||
|
||||
var key: String {
|
||||
switch self {
|
||||
case .window(let id): return "window:\(id)"
|
||||
case .display(let id): return "display:\(id)"
|
||||
case .separate(let id): return "separate:\(id)"
|
||||
case .test: return "test"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Size to capture at: the source's pixel size, shrunk to fit a box whose
|
||||
/// long side is `maxLong` (default 1920), even numbers for the encoder.
|
||||
func fitSize(width: Double, height: Double, maxLong: Int) -> (Int, Int) {
|
||||
let scale = min(1, Double(maxLong) / max(width, height, 1))
|
||||
let w = max(16, Int((width * scale / 2).rounded()) * 2), h = max(16, Int((height * scale / 2).rounded()) * 2)
|
||||
return (w, h)
|
||||
}
|
||||
|
||||
/// What CGWindowList says about a window right now (no permission needed for
|
||||
/// bounds; titles need Screen Recording).
|
||||
struct WindowInfo {
|
||||
let id: CGWindowID
|
||||
let pid: pid_t
|
||||
let app: String
|
||||
let title: String
|
||||
let bounds: CGRect
|
||||
let layer: Int
|
||||
let onScreen: Bool
|
||||
|
||||
static func all(onScreenOnly: Bool = true) -> [WindowInfo] {
|
||||
let opts: CGWindowListOption = onScreenOnly ? [.optionOnScreenOnly, .excludeDesktopElements] : [.excludeDesktopElements]
|
||||
let list = CGWindowListCopyWindowInfo(opts, kCGNullWindowID) as? [[CFString: Any]] ?? []
|
||||
return list.compactMap(WindowInfo.init)
|
||||
}
|
||||
|
||||
static func find(_ id: CGWindowID) -> WindowInfo? {
|
||||
let list = CGWindowListCopyWindowInfo(.optionIncludingWindow, id) as? [[CFString: Any]] ?? []
|
||||
return list.compactMap(WindowInfo.init).first { $0.id == id }
|
||||
}
|
||||
|
||||
init?(_ d: [CFString: Any]) {
|
||||
guard let id = d[kCGWindowNumber] as? CGWindowID, let pid = d[kCGWindowOwnerPID] as? pid_t,
|
||||
let b = d[kCGWindowBounds] as? [String: Any], let rect = CGRect(dictionaryRepresentation: b as CFDictionary)
|
||||
else { return nil }
|
||||
self.id = id
|
||||
self.pid = pid
|
||||
app = d[kCGWindowOwnerName] as? String ?? ""
|
||||
title = d[kCGWindowName] as? String ?? ""
|
||||
bounds = rect
|
||||
layer = d[kCGWindowLayer] as? Int ?? 0
|
||||
onScreen = d[kCGWindowIsOnscreen] as? Bool ?? false
|
||||
}
|
||||
}
|
||||
|
||||
protocol CaptureSource: AnyObject {
|
||||
/// The picture, its presentation time, and when the Mac composited it (µs, host clock).
|
||||
var onFrame: ((CVPixelBuffer, CMTime, Int64) -> Void)? { get set }
|
||||
var onEnded: ((String) -> Void)? { get set }
|
||||
var onChange: (() -> Void)? { get set } // title or size changed
|
||||
/// True once the window or display is gone for good.
|
||||
var gone: Bool { get }
|
||||
/// Points on the Mac's global display space that the picture covers.
|
||||
var frameRect: CGRect { get }
|
||||
var title: String { get }
|
||||
var app: String { get }
|
||||
var pid: pid_t? { get }
|
||||
func start(maxLong: Int, fps: Int, completion: @escaping (String?) -> Void)
|
||||
/// A smaller or larger picture from now on (the long side, in pixels).
|
||||
func setMaxLong(_ maxLong: Int)
|
||||
func stop()
|
||||
func pointer(x: Double, y: Double, text: String?) // for the test pattern
|
||||
}
|
||||
|
||||
extension CaptureSource {
|
||||
func pointer(x: Double, y: Double, text: String?) {}
|
||||
}
|
||||
|
||||
/// ScreenCaptureKit, for a window or a display.
|
||||
final class SCKSource: NSObject, CaptureSource, SCStreamOutput, SCStreamDelegate {
|
||||
let source: Source
|
||||
var onFrame: ((CVPixelBuffer, CMTime, Int64) -> Void)?
|
||||
var onEnded: ((String) -> Void)?
|
||||
private(set) var frameRect = CGRect.zero
|
||||
private(set) var title = ""
|
||||
private(set) var app = ""
|
||||
private(set) var pid: pid_t?
|
||||
private var stream: SCStream?
|
||||
private var config = SCStreamConfiguration()
|
||||
private var maxLong = 1920
|
||||
private var scale = 2.0
|
||||
private var poll: DispatchSourceTimer?
|
||||
private let queue = DispatchQueue(label: "frame-mac-view.capture", qos: .userInteractive)
|
||||
/// Set by stop(), on `queue`; a start still enumerating windows checks it
|
||||
/// before it starts capturing, so a viewer that left early leaves nothing on.
|
||||
private var cancelled = false
|
||||
/// The window or display no longer exists, so retrying can't help.
|
||||
private(set) var gone = false
|
||||
var onChange: (() -> Void)? // title or size changed
|
||||
/// For a display: capture only this part of it (display points, top-left
|
||||
/// origin). Set before start().
|
||||
var crop: CGRect?
|
||||
|
||||
init(_ source: Source) { self.source = source }
|
||||
|
||||
/// What's captured, in global points: the display, or the cropped part of it.
|
||||
private func displayRect(_ id: CGDirectDisplayID) -> CGRect {
|
||||
let b = CGDisplayBounds(id)
|
||||
guard let c = crop else { return b }
|
||||
return c.offsetBy(dx: b.minX, dy: b.minY).intersection(b)
|
||||
}
|
||||
|
||||
func start(maxLong: Int, fps: Int, completion: @escaping (String?) -> Void) {
|
||||
self.maxLong = maxLong
|
||||
SCShareableContent.getExcludingDesktopWindows(true, onScreenWindowsOnly: false) { [self] content, error in
|
||||
queue.async { self.begin(content, error, fps: fps, completion: completion) }
|
||||
}
|
||||
}
|
||||
|
||||
private func begin(_ content: SCShareableContent?, _ error: Error?, fps: Int, completion: @escaping (String?) -> Void) {
|
||||
if cancelled { return }
|
||||
guard let content else {
|
||||
return completion("Screen Recording isn't allowed for Frame Control (\(error?.localizedDescription ?? "no content"))")
|
||||
}
|
||||
let filter: SCContentFilter
|
||||
switch source {
|
||||
case .window(let id):
|
||||
guard let w = content.windows.first(where: { $0.windowID == id }) else {
|
||||
gone = true
|
||||
return completion("that window has closed")
|
||||
}
|
||||
filter = SCContentFilter(desktopIndependentWindow: w)
|
||||
title = w.title ?? ""
|
||||
app = w.owningApplication?.applicationName ?? ""
|
||||
pid = w.owningApplication?.processID
|
||||
frameRect = w.frame
|
||||
case .display(let id):
|
||||
guard let d = content.displays.first(where: { $0.displayID == id }) else {
|
||||
gone = true
|
||||
return completion("that display isn't connected")
|
||||
}
|
||||
filter = SCContentFilter(display: d, excludingWindows: [])
|
||||
title = displayName(id)
|
||||
app = "Mac"
|
||||
frameRect = displayRect(id)
|
||||
if crop != nil { config.sourceRect = frameRect.offsetBy(dx: -CGDisplayBounds(id).minX, dy: -CGDisplayBounds(id).minY) }
|
||||
case .test, .separate:
|
||||
return completion("not a ScreenCaptureKit source")
|
||||
}
|
||||
scale = Double(filter.pointPixelScale)
|
||||
// A display's own mode knows best: ScreenCaptureKit can still say 1
|
||||
// for a virtual display that has only just switched to HiDPI.
|
||||
if case .display(let id) = source, let mode = CGDisplayCopyDisplayMode(id), mode.width > 0 {
|
||||
scale = max(scale, Double(mode.pixelWidth) / Double(mode.width))
|
||||
}
|
||||
let (w, h) = fitSize(width: frameRect.width * scale, height: frameRect.height * scale, maxLong: maxLong)
|
||||
config.width = w
|
||||
config.height = h
|
||||
config.minimumFrameInterval = CMTime(value: 1, timescale: CMTimeScale(fps))
|
||||
config.pixelFormat = kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange
|
||||
config.colorMatrix = CGDisplayStream.yCbCrMatrix_ITU_R_709_2
|
||||
config.colorSpaceName = CGColorSpace.sRGB
|
||||
config.showsCursor = true
|
||||
config.queueDepth = 5
|
||||
config.scalesToFit = true
|
||||
config.preservesAspectRatio = true
|
||||
config.capturesAudio = false
|
||||
let stream = SCStream(filter: filter, configuration: config, delegate: self)
|
||||
do {
|
||||
try stream.addStreamOutput(self, type: .screen, sampleHandlerQueue: queue)
|
||||
} catch {
|
||||
return completion("couldn't capture: \(error.localizedDescription)")
|
||||
}
|
||||
self.stream = stream
|
||||
stream.startCapture { error in
|
||||
self.queue.async {
|
||||
if self.cancelled {
|
||||
stream.stopCapture { _ in }
|
||||
return
|
||||
}
|
||||
if let error { return completion("couldn't capture: \(error.localizedDescription)") }
|
||||
self.startPolling()
|
||||
completion(nil)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func stop() {
|
||||
queue.async {
|
||||
self.cancelled = true
|
||||
self.poll?.cancel()
|
||||
self.poll = nil
|
||||
self.stream?.stopCapture { _ in }
|
||||
self.stream = nil
|
||||
}
|
||||
}
|
||||
|
||||
/// Windows move, resize, retitle and close, and displays change mode;
|
||||
/// ScreenCaptureKit doesn't say.
|
||||
private func startPolling() {
|
||||
if case .display(let id) = source { return pollDisplay(id) }
|
||||
guard case .window(let id) = source else { return }
|
||||
let t = DispatchSource.makeTimerSource(queue: queue)
|
||||
t.schedule(deadline: .now() + 1, repeating: 1)
|
||||
t.setEventHandler { [weak self] in
|
||||
guard let self else { return }
|
||||
guard let info = WindowInfo.find(id) else {
|
||||
self.stop()
|
||||
self.onEnded?("the window closed")
|
||||
return
|
||||
}
|
||||
var changed = false
|
||||
if !info.title.isEmpty, info.title != self.title { self.title = info.title; changed = true }
|
||||
let old = self.frameRect
|
||||
self.frameRect = info.bounds
|
||||
if abs(old.width - info.bounds.width) > 1 || abs(old.height - info.bounds.height) > 1 {
|
||||
let (w, h) = fitSize(width: info.bounds.width * self.scale, height: info.bounds.height * self.scale, maxLong: self.maxLong)
|
||||
self.config.width = w
|
||||
self.config.height = h
|
||||
self.stream?.updateConfiguration(self.config) { _ in }
|
||||
changed = true
|
||||
}
|
||||
if changed { self.onChange?() }
|
||||
}
|
||||
t.resume()
|
||||
poll = t
|
||||
}
|
||||
|
||||
func setMaxLong(_ n: Int) {
|
||||
queue.async {
|
||||
self.maxLong = n
|
||||
guard self.stream != nil else { return }
|
||||
let (w, h) = fitSize(width: self.frameRect.width * self.scale, height: self.frameRect.height * self.scale, maxLong: n)
|
||||
guard w != self.config.width || h != self.config.height else { return }
|
||||
self.config.width = w
|
||||
self.config.height = h
|
||||
self.stream?.updateConfiguration(self.config) { _ in }
|
||||
}
|
||||
}
|
||||
|
||||
private func pollDisplay(_ id: CGDirectDisplayID) {
|
||||
let t = DispatchSource.makeTimerSource(queue: queue)
|
||||
t.schedule(deadline: .now() + 0.5, repeating: 1)
|
||||
t.setEventHandler { [weak self] in
|
||||
guard let self, let mode = CGDisplayCopyDisplayMode(id), mode.width > 0 else { return }
|
||||
let bounds = self.displayRect(id), scale = Double(mode.pixelWidth) / Double(mode.width)
|
||||
let (w, h) = fitSize(width: bounds.width * scale, height: bounds.height * scale, maxLong: self.maxLong)
|
||||
self.frameRect = bounds
|
||||
guard w != self.config.width || h != self.config.height else { return }
|
||||
self.scale = scale
|
||||
self.config.width = w
|
||||
self.config.height = h
|
||||
self.stream?.updateConfiguration(self.config) { _ in }
|
||||
self.onChange?()
|
||||
}
|
||||
t.resume()
|
||||
poll = t
|
||||
}
|
||||
|
||||
func stream(_ stream: SCStream, didOutputSampleBuffer sample: CMSampleBuffer, of type: SCStreamOutputType) {
|
||||
guard type == .screen, sample.isValid, let pb = CMSampleBufferGetImageBuffer(sample) else { return }
|
||||
// Only complete frames carry new pixels; idle and blank ones don't.
|
||||
let info = (CMSampleBufferGetSampleAttachmentsArray(sample, createIfNecessary: false) as? [[SCStreamFrameInfo: Any]])?.first
|
||||
if let raw = info?[.status] as? Int, let status = SCFrameStatus(rawValue: raw), status != .complete {
|
||||
return
|
||||
}
|
||||
let pts = CMSampleBufferGetPresentationTimeStamp(sample)
|
||||
// When WindowServer composited it: the moment it showed on the Mac.
|
||||
let shown = (info?[.displayTime] as? UInt64).map(hostUs) ?? hostUs(pts)
|
||||
onFrame?(pb, pts, shown)
|
||||
}
|
||||
|
||||
func stream(_ stream: SCStream, didStopWithError error: Error) {
|
||||
onEnded?("capture stopped: \(error.localizedDescription)")
|
||||
}
|
||||
}
|
||||
|
||||
func displayName(_ id: CGDirectDisplayID) -> String {
|
||||
for screen in NSScreen.screens {
|
||||
if (screen.deviceDescription[NSDeviceDescriptionKey("NSScreenNumber")] as? NSNumber)?.uint32Value == id {
|
||||
return screen.localizedName
|
||||
}
|
||||
}
|
||||
return "Display \(id)"
|
||||
}
|
||||
|
||||
/// A moving test card: bars, a clock and a frame counter, plus a dot where the
|
||||
/// viewer's pointer is and the last key it sent, so input can be checked too.
|
||||
final class TestSource: CaptureSource {
|
||||
var onFrame: ((CVPixelBuffer, CMTime, Int64) -> Void)?
|
||||
var onEnded: ((String) -> Void)?
|
||||
var onChange: (() -> Void)?
|
||||
let gone = false
|
||||
let frameRect = CGRect(x: 0, y: 0, width: 1280, height: 720)
|
||||
let title = "Test pattern"
|
||||
let app = "Frame Control"
|
||||
let pid: pid_t? = nil
|
||||
private var timer: DispatchSourceTimer?
|
||||
private var pool: CVPixelBufferPool?
|
||||
private var n = 0
|
||||
private var dot: (Double, Double)?
|
||||
private var lastText = "Point or type in the headset"
|
||||
private let queue = DispatchQueue(label: "frame-mac-view.test")
|
||||
private var size = (1280, 720)
|
||||
|
||||
func start(maxLong: Int, fps: Int, completion: @escaping (String?) -> Void) {
|
||||
size = fitSize(width: 1280, height: 720, maxLong: maxLong)
|
||||
makePool()
|
||||
let t = DispatchSource.makeTimerSource(queue: queue)
|
||||
t.schedule(deadline: .now(), repeating: 1.0 / Double(fps))
|
||||
t.setEventHandler { [weak self] in self?.draw() }
|
||||
t.resume()
|
||||
timer = t
|
||||
completion(nil)
|
||||
}
|
||||
|
||||
func stop() {
|
||||
timer?.cancel()
|
||||
timer = nil
|
||||
}
|
||||
|
||||
func setMaxLong(_ n: Int) {
|
||||
queue.async {
|
||||
self.size = fitSize(width: 1280, height: 720, maxLong: n)
|
||||
self.makePool()
|
||||
}
|
||||
}
|
||||
|
||||
private func makePool() {
|
||||
let attrs: [CFString: Any] = [kCVPixelBufferPixelFormatTypeKey: kCVPixelFormatType_32BGRA,
|
||||
kCVPixelBufferWidthKey: size.0, kCVPixelBufferHeightKey: size.1,
|
||||
kCVPixelBufferIOSurfacePropertiesKey: [:] as CFDictionary]
|
||||
pool = nil
|
||||
CVPixelBufferPoolCreate(nil, nil, attrs as CFDictionary, &pool)
|
||||
}
|
||||
|
||||
func pointer(x: Double, y: Double, text: String?) {
|
||||
queue.async {
|
||||
if x >= 0 { self.dot = (x, y) }
|
||||
if let text { self.lastText = text }
|
||||
}
|
||||
}
|
||||
|
||||
private func draw() {
|
||||
guard let pool else { return }
|
||||
var out: CVPixelBuffer?
|
||||
CVPixelBufferPoolCreatePixelBuffer(nil, pool, &out)
|
||||
guard let pb = out else { return }
|
||||
CVPixelBufferLockBaseAddress(pb, [])
|
||||
defer { CVPixelBufferUnlockBaseAddress(pb, []) }
|
||||
let (w, h) = size
|
||||
guard let ctx = CGContext(data: CVPixelBufferGetBaseAddress(pb), width: w, height: h, bitsPerComponent: 8,
|
||||
bytesPerRow: CVPixelBufferGetBytesPerRow(pb), space: CGColorSpace(name: CGColorSpace.sRGB)!,
|
||||
bitmapInfo: CGImageAlphaInfo.premultipliedFirst.rawValue | CGBitmapInfo.byteOrder32Little.rawValue)
|
||||
else { return }
|
||||
let colors: [(CGFloat, CGFloat, CGFloat)] = [(0.75, 0.75, 0.75), (0.75, 0.75, 0), (0, 0.75, 0.75), (0, 0.75, 0),
|
||||
(0.75, 0, 0.75), (0.75, 0, 0), (0, 0, 0.75)]
|
||||
let bw = CGFloat(w) / CGFloat(colors.count)
|
||||
for (i, c) in colors.enumerated() {
|
||||
ctx.setFillColor(red: c.0, green: c.1, blue: c.2, alpha: 1)
|
||||
ctx.fill(CGRect(x: CGFloat(i) * bw, y: CGFloat(h) * 0.35, width: bw + 1, height: CGFloat(h) * 0.65))
|
||||
}
|
||||
ctx.setFillColor(red: 0.08, green: 0.09, blue: 0.11, alpha: 1)
|
||||
ctx.fill(CGRect(x: 0, y: 0, width: w, height: Int(Double(h) * 0.35)))
|
||||
// A bar sweeping once a second shows motion and dropped frames.
|
||||
let x = CGFloat(n % 60) / 60 * CGFloat(w)
|
||||
ctx.setFillColor(red: 1, green: 1, blue: 1, alpha: 1)
|
||||
ctx.fill(CGRect(x: x, y: CGFloat(h) * 0.35, width: max(4, CGFloat(w) / 120), height: CGFloat(h) * 0.65))
|
||||
let f = DateFormatter()
|
||||
f.dateFormat = "HH:mm:ss.SSS"
|
||||
let label = "Frame Control test pattern \(f.string(from: Date())) frame \(n)\n\(lastText)"
|
||||
let text = NSAttributedString(string: label, attributes: [
|
||||
.font: NSFont.monospacedSystemFont(ofSize: CGFloat(h) / 24, weight: .medium),
|
||||
.foregroundColor: NSColor.white,
|
||||
])
|
||||
let ns = NSGraphicsContext(cgContext: ctx, flipped: false)
|
||||
NSGraphicsContext.saveGraphicsState()
|
||||
NSGraphicsContext.current = ns
|
||||
text.draw(at: CGPoint(x: CGFloat(w) * 0.03, y: CGFloat(h) * 0.08))
|
||||
NSGraphicsContext.restoreGraphicsState()
|
||||
if let (px, py) = dot {
|
||||
let r = CGFloat(h) / 40
|
||||
ctx.setFillColor(red: 1, green: 0.2, blue: 0.2, alpha: 1)
|
||||
ctx.fillEllipse(in: CGRect(x: CGFloat(px) * CGFloat(w) - r, y: (1 - CGFloat(py)) * CGFloat(h) - r, width: 2 * r, height: 2 * r))
|
||||
}
|
||||
n += 1
|
||||
onFrame?(pb, CMClockGetTime(CMClockGetHostTimeClock()), nowUs())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
// Adapts each stream to its network, latency first. The viewer acknowledges
|
||||
// every frame as it arrives ("rx"); from those acks the controller knows how
|
||||
// long frames take to get through and how fast the link delivers them.
|
||||
//
|
||||
// - The gate: a new frame is sent only while the oldest unacknowledged one is
|
||||
// younger than the path's usual round trip plus a little slack. So frames
|
||||
// never queue up in SSH, TCP or the Wi-Fi driver; while the link is stuck
|
||||
// the newest picture waits and goes out as soon as it moves again.
|
||||
// - The bitrate: when frames start queueing (the round trip grows) or the
|
||||
// gate has to hold frames back, it drops to a bit under what the link
|
||||
// actually delivered; once things are clear it probes up again slowly, never
|
||||
// above the quality setting's bitrate (the ceiling).
|
||||
// - The tier: as the bitrate falls, fewer frames per second (60, 45, 30), then
|
||||
// a smaller picture (75%, then 50% of the panel's pixels).
|
||||
// See docs/mac-in-headset.md ("Adapting to the network"). Thread-safe.
|
||||
import Foundation
|
||||
|
||||
final class RateController {
|
||||
struct Tier: Equatable {
|
||||
let fps: Int
|
||||
let scale: Double // of the picture's long side
|
||||
}
|
||||
|
||||
static let tiers = [Tier(fps: 60, scale: 1), Tier(fps: 45, scale: 1), Tier(fps: 30, scale: 1),
|
||||
Tier(fps: 30, scale: 0.75), Tier(fps: 30, scale: 0.5)]
|
||||
/// A tier is used while the target bitrate is at least this share of the ceiling.
|
||||
static let floors = [0.45, 0.28, 0.16, 0.08, 0]
|
||||
static let enabled = ProcessInfo.processInfo.environment["FRAME_MAC_VIEW_ADAPT"] != "0"
|
||||
|
||||
let maxFps: Int
|
||||
private let lock = NSLock()
|
||||
private var ceiling = 0 // bits/s at full size and frame rate
|
||||
private(set) var target = 0
|
||||
private(set) var tier = 0
|
||||
private var unacked: [(seq: UInt32, sent: Int64, bytes: Int)] = []
|
||||
/// Round trips (send -> ack arrives here), for the baseline: the lowest
|
||||
/// in the last 10 s is the path without any queue.
|
||||
private var rtts: [(t: Int64, v: Int64)] = []
|
||||
private var acked: [(t: Int64, bytes: Int)] = [] // the last second
|
||||
private var sentLog: [(t: Int64, bytes: Int)] = []
|
||||
private var captures: [Int64] = [] // the last second
|
||||
private var frameBytes = 0 // average recent frame, kept while the gate holds everything back
|
||||
private var held = 0 // frames the gate held back since the last update
|
||||
private var lastSignal = false
|
||||
private var sawAck = false
|
||||
private var lastDecrease: Int64 = 0
|
||||
private var lastIncrease: Int64 = 0
|
||||
private var belowSince: Int64 = 0, aboveSince: Int64 = 0
|
||||
/// What changed, for the timeline: (time, event).
|
||||
private(set) var events: [(Int64, String)] = []
|
||||
|
||||
init(maxFps: Int) { self.maxFps = maxFps }
|
||||
|
||||
private func locked<T>(_ f: () -> T) -> T { lock.lock(); defer { lock.unlock() }; return f() }
|
||||
|
||||
/// The quality setting's bitrate at full size; the first call also starts there.
|
||||
func setCeiling(_ bps: Int) {
|
||||
locked {
|
||||
if target == 0 || target > bps { target = bps }
|
||||
ceiling = bps
|
||||
}
|
||||
}
|
||||
|
||||
var fps: Int { locked { fpsLocked } }
|
||||
private var fpsLocked: Int { min(maxFps, RateController.tiers[tier].fps) }
|
||||
var scale: Double { locked { RateController.tiers[tier].scale } }
|
||||
var baseRtt: Int64 { locked { baseline() } }
|
||||
|
||||
private func baseline() -> Int64 { rtts.map(\.v).min() ?? 0 }
|
||||
|
||||
/// How late a frame may be before the gate holds the next one: one frame
|
||||
/// interval, plus room for the jitter this link normally has (1.5 times
|
||||
/// its recent spread), so ordinary Wi-Fi jitter doesn't cost frames but a
|
||||
/// real queue does. Updated in update().
|
||||
private var slack: Int64 = 40_000
|
||||
|
||||
/// Whether a frame may be sent now without queueing behind earlier ones.
|
||||
/// `counts`: a held frame is a sign of congestion (not when merely
|
||||
/// re-checking whether a held frame can go yet).
|
||||
func maySend(now: Int64, counts: Bool = true) -> Bool {
|
||||
locked {
|
||||
guard RateController.enabled, sawAck else { return true } // not heard from the viewer yet
|
||||
// Unacknowledged for 2 s: gone with a reconnection, not in a queue.
|
||||
unacked.removeAll { now - $0.sent > 2_000_000 }
|
||||
guard let oldest = unacked.first else { return true }
|
||||
// Age is what bounds latency. The count only stops a burst, and it
|
||||
// allows a full round trip of frames, so a long but clear path
|
||||
// (100 ms away) still gets every frame.
|
||||
let interval = Int64(1_000_000 / max(1, fpsLocked))
|
||||
let window = max(3, Int((baseline() + slack) / interval) + 1)
|
||||
if unacked.count < window, now - oldest.sent <= baseline() + slack { return true }
|
||||
if counts { held += 1 }
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/// A picture was captured (sent or not): with the frame sizes, what this
|
||||
/// stream would send if the link allowed.
|
||||
func captured(at t: Int64) {
|
||||
locked {
|
||||
captures.append(t)
|
||||
if captures.count > 256 { captures.removeFirst(captures.count - 256) }
|
||||
}
|
||||
}
|
||||
|
||||
func sent(seq: UInt32, bytes: Int, at t: Int64) {
|
||||
locked {
|
||||
// Bounded even if the viewer never acknowledges (an old viewer, or
|
||||
// the controller is off).
|
||||
unacked.append((seq, t, bytes))
|
||||
if unacked.count > 512 { unacked.removeFirst(unacked.count - 512) }
|
||||
sentLog.append((t, bytes))
|
||||
if sentLog.count > 1024 { sentLog.removeFirst(sentLog.count - 1024) }
|
||||
}
|
||||
}
|
||||
|
||||
/// The viewer has frame `seq`. Returns true if that may let a held frame go.
|
||||
func acked(seq: UInt32, at now: Int64) -> Bool {
|
||||
locked {
|
||||
sawAck = true
|
||||
guard let i = unacked.firstIndex(where: { $0.seq == seq }) else { return false }
|
||||
let f = unacked[i]
|
||||
unacked.removeSubrange(0...i) // TCP delivers in order: earlier ones arrived too
|
||||
rtts.append((now, now - f.sent))
|
||||
acked.append((now, f.bytes))
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
/// Called every 100 ms. Returns the new bitrate target, or nil if the
|
||||
/// controller is off.
|
||||
func update(now: Int64) -> Int? {
|
||||
locked {
|
||||
rtts.removeAll { now - $0.t > 10_000_000 }
|
||||
acked.removeAll { now - $0.t > 500_000 }
|
||||
sentLog.removeAll { now - $0.t > 500_000 }
|
||||
unacked.removeAll { now - $0.sent > 2_000_000 }
|
||||
captures.removeAll { now - $0 > 1_000_000 }
|
||||
guard RateController.enabled, ceiling > 0 else { return nil }
|
||||
let base = baseline()
|
||||
let spread = rtts.filter { now - $0.t < 2_000_000 }.map(\.v).sorted()
|
||||
let jitter = spread.isEmpty ? 0 : spread[spread.count * 9 / 10] - base
|
||||
let interval = Int64(1_000_000 / max(1, fpsLocked))
|
||||
slack = interval + min(max(jitter * 3 / 2, 25_000), 80_000)
|
||||
let recent = rtts.filter { now - $0.t < 300_000 }.map(\.v).sorted()
|
||||
let queueing = recent.isEmpty ? 0 : recent[recent.count / 2] - base
|
||||
let oldestAge = unacked.first.map { now - $0.sent } ?? 0
|
||||
let stuck = oldestAge > base + 100_000
|
||||
let delivered = acked.reduce(0) { $0 + $1.bytes } * 16 // bits/s over the last half second
|
||||
let sending = sentLog.reduce(0) { $0 + $1.bytes } * 16
|
||||
// Demand: captures per second (up to the tier's rate) times the
|
||||
// average frame. A test card or a mostly still window wants far
|
||||
// less than its budget; when the link hiccups, cutting its bitrate
|
||||
// can't help, and it would only look link-limited afterwards.
|
||||
if !sentLog.isEmpty { frameBytes = sentLog.reduce(0) { $0 + $1.bytes } / sentLog.count }
|
||||
let demand = min(captures.count, fpsLocked) * frameBytes * 8
|
||||
// Delay alone isn't our queue: Wi-Fi jitters by itself. It only
|
||||
// counts while this stream uses a good part of its budget (so its
|
||||
// own data could be what's queueing). Frames the gate had to hold,
|
||||
// or one stuck in flight, show demand the link isn't carrying
|
||||
// whatever was sent (the gate itself keeps what's sent low).
|
||||
let busy = sending >= target / 2
|
||||
// Twice in a row (200 ms), so one late ack doesn't count.
|
||||
let signal = (busy && queueing > 40_000) || held >= 3 || stuck
|
||||
let congested = signal && lastSignal
|
||||
lastSignal = signal
|
||||
let heldNow = held
|
||||
held = 0
|
||||
let floorBps = 300_000
|
||||
if congested, now - lastDecrease > 300_000 {
|
||||
// Down to a bit under what got through: at least a fifth off, at
|
||||
// most half (a stall delivers nothing, but the link is still there).
|
||||
let measured = Int(Double(delivered) * 0.9)
|
||||
var next = max(floorBps, min(target * 4 / 5, max(measured, target / 2)))
|
||||
// App-limited (it wants about half its budget or less): never
|
||||
// below twice what it wants, however many cuts in a row. The
|
||||
// extra quarter is hysteresis, so frame sizes wobbling at the
|
||||
// floor don't switch the protection off.
|
||||
if demand > 0, demand * 2 <= target * 5 / 4 { next = max(next, min(target, demand * 2)) }
|
||||
target = next
|
||||
lastDecrease = now
|
||||
events.append((now, "down to \(target / 1000) kbit/s: queue \(queueing / 1000) ms, held \(heldNow), "
|
||||
+ "oldest \(oldestAge / 1000) ms, base \(base / 1000) ms, sent \(sending / 1000) got \(delivered / 1000) "
|
||||
+ "wants \(demand / 1000)"))
|
||||
} else if !congested, now - lastDecrease > 1_000_000, now - lastIncrease > 250_000, target < ceiling,
|
||||
sending > target * 6 / 10 || now - lastDecrease > 3_000_000 {
|
||||
// Clear for a second and using what it has: probe up.
|
||||
target = min(ceiling, Int(Double(target) * 1.1) + 50_000)
|
||||
lastIncrease = now
|
||||
}
|
||||
// Fewer frames or pixels only help a stream that fills its budget;
|
||||
// a small one (a still window, a test card) keeps its tier.
|
||||
retier(now: now, linkLimited: sending >= target * 7 / 10)
|
||||
if events.count > 200 { events.removeFirst(events.count - 200) }
|
||||
return target
|
||||
}
|
||||
}
|
||||
|
||||
/// Steps down quickly, straight to the tier the bitrate supports, and back
|
||||
/// up one tier at a time only when there's clearly room (hysteresis).
|
||||
private func retier(now: Int64, linkLimited: Bool) {
|
||||
let share = Double(target) / Double(max(ceiling, 1))
|
||||
if tier < RateController.tiers.count - 1, share < RateController.floors[tier], linkLimited {
|
||||
if belowSince == 0 { belowSince = now }
|
||||
if now - belowSince > 500_000 {
|
||||
tier = RateController.floors.firstIndex { share >= $0 } ?? RateController.tiers.count - 1
|
||||
belowSince = 0
|
||||
events.append((now, "tier \(tier)"))
|
||||
}
|
||||
} else {
|
||||
belowSince = 0
|
||||
}
|
||||
if tier > 0, share > RateController.floors[tier - 1] * 1.25 {
|
||||
if aboveSince == 0 { aboveSince = now }
|
||||
if now - aboveSince > 2_000_000 {
|
||||
tier -= 1
|
||||
aboveSince = 0
|
||||
events.append((now, "tier \(tier)"))
|
||||
}
|
||||
} else {
|
||||
aboveSince = 0
|
||||
}
|
||||
}
|
||||
|
||||
func state() -> [String: Any] {
|
||||
locked {
|
||||
["target": target, "ceiling": ceiling, "tier": tier, "fps": min(maxFps, RateController.tiers[tier].fps),
|
||||
"scale": RateController.tiers[tier].scale, "baseRtt": Double(baseline()) / 1000,
|
||||
"inFlight": unacked.count, "slack": Double(slack) / 1000, "adapt": RateController.enabled]
|
||||
}
|
||||
}
|
||||
|
||||
func eventList() -> [[String: Any]] { locked { events.map { ["t": $0.0, "e": $0.1] } } }
|
||||
}
|
||||
@@ -0,0 +1,217 @@
|
||||
// VideoToolbox encoding: H.264 for the normal path (hardware, low-latency rate
|
||||
// control, no B-frames, Annex B output that WebCodecs takes without a
|
||||
// description), or JPEG stills for viewers that can't decode H.264.
|
||||
import CoreMedia
|
||||
import Foundation
|
||||
import VideoToolbox
|
||||
|
||||
enum Codec: String {
|
||||
case h264, jpeg
|
||||
}
|
||||
|
||||
final class Encoder {
|
||||
let codec: Codec
|
||||
let fps: Int
|
||||
let bitsPerPixel: Double
|
||||
private(set) var width = 0
|
||||
private(set) var height = 0
|
||||
private var session: VTCompressionSession?
|
||||
private var forceKey = true
|
||||
private var lastPts = CMTime.invalid
|
||||
private var lastGiven = CMTime.invalid
|
||||
/// Bits per second to aim for; nil means from the size, frame rate and
|
||||
/// bits per pixel. Changing it takes effect on the next frame.
|
||||
private var target: Int?
|
||||
private(set) var bitrate = 0
|
||||
/// Called on VideoToolbox's thread with one access unit (or JPEG) per
|
||||
/// frame, and the sequence number it was submitted with.
|
||||
var onFrame: ((Data, Bool, CMTime, UInt32) -> Void)?
|
||||
var onError: ((String) -> Void)?
|
||||
/// Called once per frame handed to VideoToolbox, however it went.
|
||||
var onDone: ((UInt32) -> Void)?
|
||||
/// Experiment switches (FRAME_MAC_VIEW_ENCODER, comma-separated), so a
|
||||
/// benchmark can compare them without a rebuild.
|
||||
static let options = Set((ProcessInfo.processInfo.environment["FRAME_MAC_VIEW_ENCODER"] ?? "")
|
||||
.split(separator: ",").map(String.init))
|
||||
|
||||
init(codec: Codec, fps: Int, bitsPerPixel: Double) {
|
||||
self.codec = codec
|
||||
self.fps = fps
|
||||
self.bitsPerPixel = bitsPerPixel
|
||||
}
|
||||
|
||||
/// The bitrate the size and quality setting would give.
|
||||
func defaultBitrate(width w: Int, height h: Int) -> Int {
|
||||
Int(min(max(Double(w * h * fps) * bitsPerPixel, 2_000_000), 60_000_000))
|
||||
}
|
||||
|
||||
/// Live, without a new keyframe. Call on the encoding queue.
|
||||
func setBitrate(_ bps: Int?) {
|
||||
target = bps
|
||||
guard codec == .h264, let s = session else { return }
|
||||
let b = bps ?? defaultBitrate(width: width, height: height)
|
||||
guard b != bitrate else { return }
|
||||
bitrate = b
|
||||
VTSessionSetProperty(s, key: kVTCompressionPropertyKey_AverageBitRate, value: b as CFTypeRef)
|
||||
// A hard ceiling too: no more than 200 ms' worth of bits in any 200 ms,
|
||||
// so a keyframe can't hold the link for long.
|
||||
if Encoder.options.contains("cap") {
|
||||
VTSessionSetProperty(s, key: kVTCompressionPropertyKey_DataRateLimits,
|
||||
value: [b / 8 / 5, 0.2] as CFArray)
|
||||
}
|
||||
}
|
||||
|
||||
deinit { invalidate() }
|
||||
|
||||
func requestKeyFrame() { forceKey = true }
|
||||
|
||||
func invalidate() {
|
||||
if let s = session {
|
||||
VTCompressionSessionCompleteFrames(s, untilPresentationTimeStamp: .invalid)
|
||||
VTCompressionSessionInvalidate(s)
|
||||
}
|
||||
session = nil
|
||||
}
|
||||
|
||||
private func makeSession(width w: Int, height h: Int) -> Bool {
|
||||
invalidate()
|
||||
var spec: [CFString: Any] = [:]
|
||||
if codec == .h264 { spec[kVTVideoEncoderSpecification_EnableLowLatencyRateControl] = true }
|
||||
if Encoder.options.contains("hw") { spec[kVTVideoEncoderSpecification_RequireHardwareAcceleratedVideoEncoder] = true }
|
||||
var s: VTCompressionSession?
|
||||
let type = codec == .h264 ? kCMVideoCodecType_H264 : kCMVideoCodecType_JPEG
|
||||
func create(_ spec: [CFString: Any]) -> OSStatus {
|
||||
VTCompressionSessionCreate(allocator: nil, width: Int32(w), height: Int32(h), codecType: type,
|
||||
encoderSpecification: spec as CFDictionary, imageBufferAttributes: nil,
|
||||
compressedDataAllocator: nil, outputCallback: nil, refcon: nil,
|
||||
compressionSessionOut: &s)
|
||||
}
|
||||
var err = create(spec)
|
||||
// Low-latency rate control needs Apple's hardware encoder; without it
|
||||
// (some VMs), plain real-time encoding still works.
|
||||
if err != noErr, !spec.isEmpty { err = create([:]) }
|
||||
guard err == noErr, let s else {
|
||||
onError?("couldn't start the \(codec.rawValue) encoder (VideoToolbox \(err))")
|
||||
return false
|
||||
}
|
||||
func set(_ key: CFString, _ value: Any) { VTSessionSetProperty(s, key: key, value: value as CFTypeRef) }
|
||||
set(kVTCompressionPropertyKey_RealTime, true)
|
||||
set(kVTCompressionPropertyKey_ColorPrimaries, kCVImageBufferColorPrimaries_ITU_R_709_2)
|
||||
set(kVTCompressionPropertyKey_TransferFunction, kCVImageBufferTransferFunction_ITU_R_709_2)
|
||||
set(kVTCompressionPropertyKey_YCbCrMatrix, kCVImageBufferYCbCrMatrix_ITU_R_709_2)
|
||||
if codec == .h264 {
|
||||
set(kVTCompressionPropertyKey_ProfileLevel, kVTProfileLevel_H264_ConstrainedHigh_AutoLevel)
|
||||
set(kVTCompressionPropertyKey_AllowFrameReordering, false)
|
||||
set(kVTCompressionPropertyKey_ExpectedFrameRate, fps)
|
||||
if Encoder.options.contains("nodelay") { set(kVTCompressionPropertyKey_MaxFrameDelayCount, 0) }
|
||||
if Encoder.options.contains("speed") { set(kVTCompressionPropertyKey_PrioritizeEncodingSpeedOverQuality, true) }
|
||||
// A keyframe every 10 s at most, so a viewer that lost one recovers
|
||||
// even if it never asks. Viewers ask for one when they start.
|
||||
set(kVTCompressionPropertyKey_MaxKeyFrameIntervalDuration, 10)
|
||||
} else {
|
||||
set(kVTCompressionPropertyKey_Quality, 0.8)
|
||||
}
|
||||
VTCompressionSessionPrepareToEncodeFrames(s)
|
||||
session = s
|
||||
lastPts = .invalid
|
||||
lastGiven = .invalid
|
||||
width = w
|
||||
height = h
|
||||
forceKey = true
|
||||
bitrate = 0
|
||||
setBitrate(target)
|
||||
return true
|
||||
}
|
||||
|
||||
/// False if the frame never reached VideoToolbox (then onDone won't come).
|
||||
@discardableResult
|
||||
func encode(_ pb: CVPixelBuffer, pts given: CMTime, seq: UInt32) -> Bool {
|
||||
// The encoder's own timeline: strictly increasing (a resent picture
|
||||
// stamped "now" can be followed by a capture stamped a moment earlier),
|
||||
// and never more than two frames on from the last one. Rate control
|
||||
// budgets bits by elapsed time, so after a pause (the link held frames
|
||||
// back, or nothing changed) one frame would otherwise get a quarter
|
||||
// second's worth of bits: 200 KB that then hold a slow link for half a second.
|
||||
let w = CVPixelBufferGetWidth(pb), h = CVPixelBufferGetHeight(pb)
|
||||
if session == nil || w != width || h != height {
|
||||
guard makeSession(width: w, height: h) else { return false } // a new timeline too
|
||||
}
|
||||
guard let s = session else { return false }
|
||||
var pts = given
|
||||
if lastPts.isValid, lastGiven.isValid {
|
||||
let gap = CMTimeSubtract(given, lastGiven)
|
||||
let most = CMTime(value: 2, timescale: CMTimeScale(fps))
|
||||
let step = CMTimeCompare(gap, most) > 0 ? most : gap
|
||||
pts = CMTimeAdd(lastPts, CMTimeMaximum(step, CMTime(value: 1, timescale: 1_000_000)))
|
||||
}
|
||||
lastGiven = given
|
||||
var props: CFDictionary?
|
||||
if forceKey {
|
||||
props = [kVTEncodeFrameOptionKey_ForceKeyFrame: true] as CFDictionary
|
||||
forceKey = false
|
||||
}
|
||||
let codec = self.codec
|
||||
lastPts = pts
|
||||
let status = VTCompressionSessionEncodeFrame(s, imageBuffer: pb, presentationTimeStamp: pts, duration: .invalid,
|
||||
frameProperties: props, infoFlagsOut: nil) { [weak self] status, _, sample in
|
||||
guard let self else { return }
|
||||
defer { self.onDone?(seq) }
|
||||
guard status == noErr, let sample else { return }
|
||||
if codec == .jpeg {
|
||||
if let data = Self.bytes(sample) { self.onFrame?(data, true, pts, seq) }
|
||||
} else if let (data, key) = Self.annexB(sample) {
|
||||
self.onFrame?(data, key, pts, seq)
|
||||
}
|
||||
}
|
||||
if status != noErr { forceKey = true }
|
||||
return status == noErr
|
||||
}
|
||||
|
||||
private static func bytes(_ sample: CMSampleBuffer) -> Data? {
|
||||
guard let block = CMSampleBufferGetDataBuffer(sample) else { return nil }
|
||||
var length = 0
|
||||
var ptr: UnsafeMutablePointer<CChar>?
|
||||
guard CMBlockBufferGetDataPointer(block, atOffset: 0, lengthAtOffsetOut: nil, totalLengthOut: &length,
|
||||
dataPointerOut: &ptr) == noErr, let ptr else { return nil }
|
||||
return Data(bytes: ptr, count: length)
|
||||
}
|
||||
|
||||
/// AVCC sample -> Annex B access unit, with SPS and PPS before keyframes.
|
||||
static func annexB(_ sample: CMSampleBuffer) -> (Data, Bool)? {
|
||||
guard let avcc = bytes(sample) else { return nil }
|
||||
var key = true
|
||||
if let atts = CMSampleBufferGetSampleAttachmentsArray(sample, createIfNecessary: false) as? [[CFString: Any]],
|
||||
let first = atts.first, first[kCMSampleAttachmentKey_NotSync] as? Bool == true {
|
||||
key = false
|
||||
}
|
||||
let start: [UInt8] = [0, 0, 0, 1]
|
||||
var out = Data()
|
||||
if key, let fmt = CMSampleBufferGetFormatDescription(sample) {
|
||||
var count = 0
|
||||
CMVideoFormatDescriptionGetH264ParameterSetAtIndex(fmt, parameterSetIndex: 0, parameterSetPointerOut: nil,
|
||||
parameterSetSizeOut: nil, parameterSetCountOut: &count,
|
||||
nalUnitHeaderLengthOut: nil)
|
||||
for i in 0..<count {
|
||||
var p: UnsafePointer<UInt8>?
|
||||
var n = 0
|
||||
if CMVideoFormatDescriptionGetH264ParameterSetAtIndex(fmt, parameterSetIndex: i, parameterSetPointerOut: &p,
|
||||
parameterSetSizeOut: &n, parameterSetCountOut: nil,
|
||||
nalUnitHeaderLengthOut: nil) == noErr, let p {
|
||||
out.append(contentsOf: start)
|
||||
out.append(p, count: n)
|
||||
}
|
||||
}
|
||||
}
|
||||
let bytes = [UInt8](avcc)
|
||||
var i = 0
|
||||
while i + 4 <= bytes.count {
|
||||
let n = Int(bytes[i]) << 24 | Int(bytes[i + 1]) << 16 | Int(bytes[i + 2]) << 8 | Int(bytes[i + 3])
|
||||
i += 4
|
||||
guard n > 0, i + n <= bytes.count else { break }
|
||||
out.append(contentsOf: start)
|
||||
out.append(contentsOf: bytes[i..<(i + n)])
|
||||
i += n
|
||||
}
|
||||
return (out, key)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
// Turns the viewer's pointer and key events into real Mac input. Needs the
|
||||
// Accessibility permission ("control your computer"); without it macOS drops
|
||||
// the events silently, so the agent reports the permission to the viewer.
|
||||
import AppKit
|
||||
import ApplicationServices
|
||||
import Foundation
|
||||
|
||||
@_silgen_name("_AXUIElementGetWindow")
|
||||
private func _AXUIElementGetWindow(_ element: AXUIElement, _ id: UnsafeMutablePointer<CGWindowID>) -> AXError
|
||||
|
||||
enum Input {
|
||||
static let source = CGEventSource(stateID: .hidSystemState)
|
||||
/// What each viewer (session id) is holding down, so one panel closing
|
||||
/// lets go of its own keys and buttons and nobody else's. Main thread.
|
||||
private static var buttonsHeld: [Int: Set<Int>] = [:]
|
||||
private static var keysHeld: [Int: Set<CGKeyCode>] = [:]
|
||||
|
||||
private static var lastDown: (time: TimeInterval, point: CGPoint, button: Int, count: Int)?
|
||||
|
||||
static var allowed: Bool { AXIsProcessTrusted() }
|
||||
|
||||
/// Brings a window to the front so a click lands on it, not on whatever
|
||||
/// covers it, and typing goes to it.
|
||||
static func focus(window id: CGWindowID, pid: pid_t) {
|
||||
if frontWindow() == id { return }
|
||||
let app = AXUIElementCreateApplication(pid)
|
||||
var value: CFTypeRef?
|
||||
if AXUIElementCopyAttributeValue(app, kAXWindowsAttribute as CFString, &value) == .success,
|
||||
let windows = value as? [AXUIElement] {
|
||||
for w in windows {
|
||||
var wid: CGWindowID = 0
|
||||
if _AXUIElementGetWindow(w, &wid) == .success, wid == id {
|
||||
AXUIElementPerformAction(w, kAXRaiseAction as CFString)
|
||||
AXUIElementSetAttributeValue(w, kAXMainAttribute as CFString, kCFBooleanTrue)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
AXUIElementSetAttributeValue(app, kAXFrontmostAttribute as CFString, kCFBooleanTrue)
|
||||
NSRunningApplication(processIdentifier: pid)?.activate()
|
||||
}
|
||||
|
||||
static func frontWindow() -> CGWindowID? {
|
||||
WindowInfo.all().first { $0.layer == 0 }?.id
|
||||
}
|
||||
|
||||
/// `button` uses the browser's numbering: 0 left, 1 middle, 2 right.
|
||||
static func mouse(_ kind: String, button: Int, at p: CGPoint, owner: Int) {
|
||||
let b: CGMouseButton = button == 2 ? .right : button == 1 ? .center : .left
|
||||
let type: CGEventType
|
||||
switch kind {
|
||||
case "down":
|
||||
type = b == .left ? .leftMouseDown : b == .right ? .rightMouseDown : .otherMouseDown
|
||||
buttonsHeld[owner, default: []].insert(button)
|
||||
case "up":
|
||||
type = b == .left ? .leftMouseUp : b == .right ? .rightMouseUp : .otherMouseUp
|
||||
buttonsHeld[owner]?.remove(button)
|
||||
// Another viewer still holding it keeps it down.
|
||||
if buttonsHeld.values.contains(where: { $0.contains(button) }) { return }
|
||||
default: // a drag is whatever this viewer is holding
|
||||
let mine = buttonsHeld[owner] ?? []
|
||||
if mine.contains(0) { type = .leftMouseDragged }
|
||||
else if mine.contains(2) { type = .rightMouseDragged }
|
||||
else if mine.contains(1) { type = .otherMouseDragged }
|
||||
else { type = .mouseMoved }
|
||||
}
|
||||
let mine = buttonsHeld[owner] ?? []
|
||||
let held: CGMouseButton = mine.contains(0) ? .left : mine.contains(2) ? .right : mine.contains(1) ? .center : .left
|
||||
guard let e = CGEvent(mouseEventSource: source, mouseType: type, mouseCursorPosition: p,
|
||||
mouseButton: kind == "move" ? held : b) else { return }
|
||||
if kind == "down" || kind == "up" {
|
||||
let now = ProcessInfo.processInfo.systemUptime
|
||||
var count = 1
|
||||
if kind == "down" {
|
||||
if let l = lastDown, l.button == button, now - l.time < NSEvent.doubleClickInterval,
|
||||
abs(l.point.x - p.x) < 5, abs(l.point.y - p.y) < 5 { count = l.count + 1 }
|
||||
lastDown = (now, p, button, count)
|
||||
} else if let l = lastDown, l.button == button {
|
||||
count = l.count
|
||||
}
|
||||
e.setIntegerValueField(.mouseEventClickState, value: Int64(count))
|
||||
}
|
||||
e.post(tap: .cghidEventTap)
|
||||
}
|
||||
|
||||
static func scroll(dx: Double, dy: Double, at p: CGPoint, owner: Int) {
|
||||
mouse("move", button: 0, at: p, owner: owner)
|
||||
// Browsers report pixels with +y meaning "scroll down"; macOS's +y is up.
|
||||
guard let e = CGEvent(scrollWheelEvent2Source: source, units: .pixel, wheelCount: 2,
|
||||
wheel1: Int32(-dy.rounded()), wheel2: Int32(-dx.rounded()), wheel3: 0) else { return }
|
||||
e.location = p
|
||||
e.post(tap: .cghidEventTap)
|
||||
}
|
||||
|
||||
/// Lets go of every button and key this viewer is holding down, so a
|
||||
/// dropped connection can't leave, say, Shift or ⌘ stuck on.
|
||||
static func releaseAll(owner: Int) {
|
||||
let p = CGEvent(source: nil)?.location ?? .zero
|
||||
let buttons = buttonsHeld.removeValue(forKey: owner) ?? []
|
||||
let keys = keysHeld.removeValue(forKey: owner) ?? []
|
||||
// Only what no other viewer is still holding.
|
||||
for b in buttons where !buttonsHeld.values.contains(where: { $0.contains(b) }) {
|
||||
let type: CGEventType = b == 2 ? .rightMouseUp : b == 1 ? .otherMouseUp : .leftMouseUp
|
||||
CGEvent(mouseEventSource: source, mouseType: type, mouseCursorPosition: p,
|
||||
mouseButton: b == 2 ? .right : b == 1 ? .center : .left)?.post(tap: .cghidEventTap)
|
||||
}
|
||||
for k in keys where !keysHeld.values.contains(where: { $0.contains(k) }) {
|
||||
CGEvent(keyboardEventSource: source, virtualKey: k, keyDown: false)?.post(tap: .cghidEventTap)
|
||||
}
|
||||
}
|
||||
|
||||
static func key(code: String, key: String, down: Bool, mods: [String], owner: Int) {
|
||||
var flags = CGEventFlags()
|
||||
if mods.contains("shift") { flags.insert(.maskShift) }
|
||||
if mods.contains("ctrl") { flags.insert(.maskControl) }
|
||||
if mods.contains("alt") { flags.insert(.maskAlternate) }
|
||||
if mods.contains("meta") { flags.insert(.maskCommand) }
|
||||
if let vk = keyCodes[code] {
|
||||
guard let e = CGEvent(keyboardEventSource: source, virtualKey: vk, keyDown: down) else { return }
|
||||
if down {
|
||||
keysHeld[owner, default: []].insert(vk)
|
||||
} else {
|
||||
keysHeld[owner]?.remove(vk)
|
||||
if keysHeld.values.contains(where: { $0.contains(vk) }) { return } // still held elsewhere
|
||||
}
|
||||
e.flags = flags
|
||||
e.post(tap: .cghidEventTap)
|
||||
} else if down, !key.isEmpty, key.count <= 4, key.unicodeScalars.allSatisfy({ $0.value >= 0x20 }) {
|
||||
// A key the table doesn't know (a non-US layout, the headset's
|
||||
// on-screen keyboard): type its character instead.
|
||||
text(key)
|
||||
}
|
||||
}
|
||||
|
||||
static func text(_ s: String) {
|
||||
let units = Array(s.utf16)
|
||||
for chunk in stride(from: 0, to: units.count, by: 16) {
|
||||
let part = Array(units[chunk..<min(chunk + 16, units.count)])
|
||||
for down in [true, false] {
|
||||
guard let e = CGEvent(keyboardEventSource: source, virtualKey: 0, keyDown: down) else { continue }
|
||||
e.keyboardSetUnicodeString(stringLength: part.count, unicodeString: part)
|
||||
e.post(tap: .cghidEventTap)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// DOM KeyboardEvent.code -> macOS virtual key code (ANSI positions).
|
||||
static let keyCodes: [String: CGKeyCode] = [
|
||||
"KeyA": 0x00, "KeyS": 0x01, "KeyD": 0x02, "KeyF": 0x03, "KeyH": 0x04, "KeyG": 0x05, "KeyZ": 0x06, "KeyX": 0x07,
|
||||
"KeyC": 0x08, "KeyV": 0x09, "IntlBackslash": 0x0A, "KeyB": 0x0B, "KeyQ": 0x0C, "KeyW": 0x0D, "KeyE": 0x0E,
|
||||
"KeyR": 0x0F, "KeyY": 0x10, "KeyT": 0x11, "Digit1": 0x12, "Digit2": 0x13, "Digit3": 0x14, "Digit4": 0x15,
|
||||
"Digit6": 0x16, "Digit5": 0x17, "Equal": 0x18, "Digit9": 0x19, "Digit7": 0x1A, "Minus": 0x1B, "Digit8": 0x1C,
|
||||
"Digit0": 0x1D, "BracketRight": 0x1E, "KeyO": 0x1F, "KeyU": 0x20, "BracketLeft": 0x21, "KeyI": 0x22,
|
||||
"KeyP": 0x23, "Enter": 0x24, "KeyL": 0x25, "KeyJ": 0x26, "Quote": 0x27, "KeyK": 0x28, "Semicolon": 0x29,
|
||||
"Backslash": 0x2A, "Comma": 0x2B, "Slash": 0x2C, "KeyN": 0x2D, "KeyM": 0x2E, "Period": 0x2F, "Tab": 0x30,
|
||||
"Space": 0x31, "Backquote": 0x32, "Backspace": 0x33, "Escape": 0x35, "MetaRight": 0x36, "MetaLeft": 0x37,
|
||||
"ShiftLeft": 0x38, "CapsLock": 0x39, "AltLeft": 0x3A, "ControlLeft": 0x3B, "ShiftRight": 0x3C,
|
||||
"AltRight": 0x3D, "ControlRight": 0x3E, "F17": 0x40, "NumpadDecimal": 0x41, "NumpadMultiply": 0x43,
|
||||
"NumpadAdd": 0x45, "NumLock": 0x47, "NumpadDivide": 0x4B, "NumpadEnter": 0x4C, "NumpadSubtract": 0x4E,
|
||||
"F18": 0x4F, "F19": 0x50, "NumpadEqual": 0x51, "Numpad0": 0x52, "Numpad1": 0x53, "Numpad2": 0x54,
|
||||
"Numpad3": 0x55, "Numpad4": 0x56, "Numpad5": 0x57, "Numpad6": 0x58, "Numpad7": 0x59, "F20": 0x5A,
|
||||
"Numpad8": 0x5B, "Numpad9": 0x5C, "F5": 0x60, "F6": 0x61, "F7": 0x62, "F3": 0x63, "F8": 0x64, "F9": 0x65,
|
||||
"F11": 0x67, "F13": 0x69, "F16": 0x6A, "F14": 0x6B, "F10": 0x6D, "ContextMenu": 0x6E, "F12": 0x6F,
|
||||
"F15": 0x71, "Insert": 0x72, "Help": 0x72, "Home": 0x73, "PageUp": 0x74, "Delete": 0x75, "F4": 0x76,
|
||||
"End": 0x77, "F2": 0x78, "PageDown": 0x79, "F1": 0x7A, "ArrowLeft": 0x7B, "ArrowRight": 0x7C,
|
||||
"ArrowDown": 0x7D, "ArrowUp": 0x7E, "OSLeft": 0x37, "OSRight": 0x36,
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,295 @@
|
||||
// "Separate" mode: a streamed window gets a virtual display of its own. The
|
||||
// window is moved onto it and sized to fill it, and the whole display is
|
||||
// captured, so its menus, sheets, popovers and tooltips come along, nothing
|
||||
// can cover it, and clicks land on it without raising anything. On stop the
|
||||
// window goes back where it was. See docs/mac-window-separation.md.
|
||||
import AppKit
|
||||
import ApplicationServices
|
||||
import CoreMedia
|
||||
import Foundation
|
||||
|
||||
@_silgen_name("_AXUIElementGetWindow")
|
||||
private func axWindowID(_ element: AXUIElement, _ id: UnsafeMutablePointer<CGWindowID>) -> AXError
|
||||
|
||||
/// One of our virtual displays, HiDPI (2 pixels per point). Main thread only.
|
||||
final class VirtualDisplay {
|
||||
static let vendor: UInt32 = 0xF0C0
|
||||
/// How often macOS composites our displays (FRAME_MAC_VIEW_VD_HZ to experiment).
|
||||
static let refreshRate = Double(ProcessInfo.processInfo.environment["FRAME_MAC_VIEW_VD_HZ"] ?? "") ?? 60
|
||||
private var display: CGVirtualDisplay?
|
||||
let id: CGDirectDisplayID
|
||||
/// Removal is deferred while the Mac's screen sleeps, and an identity
|
||||
/// can't be reused until it's gone, so each display gets a fresh serial.
|
||||
private static var serial = (UInt32(truncatingIfNeeded: ProcessInfo.processInfo.processIdentifier) & 0xFFFF) << 12
|
||||
|
||||
init?(name: String, width: Int, height: Int) {
|
||||
var made: CGVirtualDisplay?
|
||||
for _ in 0..<8 where made == nil {
|
||||
VirtualDisplay.serial &+= 1
|
||||
let d = CGVirtualDisplayDescriptor()
|
||||
d.queue = DispatchQueue.main
|
||||
d.name = name
|
||||
d.maxPixelsWide = UInt32(2 * width)
|
||||
d.maxPixelsHigh = UInt32(2 * height)
|
||||
d.sizeInMillimeters = CGSize(width: Double(width) * 0.2646, height: Double(height) * 0.2646)
|
||||
d.vendorID = VirtualDisplay.vendor
|
||||
d.productID = 0x5E9A
|
||||
d.serialNum = VirtualDisplay.serial
|
||||
d.terminationHandler = { _, _ in }
|
||||
guard let vd = CGVirtualDisplay(descriptor: d) else { continue }
|
||||
let s = CGVirtualDisplaySettings()
|
||||
s.hiDPI = 1
|
||||
let hz = VirtualDisplay.refreshRate
|
||||
s.modes = [CGVirtualDisplayMode(width: UInt32(2 * width), height: UInt32(2 * height), refreshRate: hz)!,
|
||||
CGVirtualDisplayMode(width: UInt32(width), height: UInt32(height), refreshRate: hz)!]
|
||||
if vd.apply(s), vd.displayID != 0 { made = vd }
|
||||
}
|
||||
guard let made else { return nil }
|
||||
display = made
|
||||
id = made.displayID
|
||||
// Pick the HiDPI mode: `width` points drawn with twice the pixels.
|
||||
let modes = (CGDisplayCopyAllDisplayModes(id, [kCGDisplayShowDuplicateLowResolutionModes: true] as CFDictionary)
|
||||
as? [CGDisplayMode]) ?? []
|
||||
if let hidpi = modes.first(where: {
|
||||
$0.width == width && $0.height == height && $0.pixelWidth == 2 * width && $0.refreshRate == VirtualDisplay.refreshRate
|
||||
}) ?? modes.first(where: { $0.width == width && $0.height == height && $0.pixelWidth == 2 * width }) {
|
||||
var cfg: CGDisplayConfigRef?
|
||||
CGBeginDisplayConfiguration(&cfg)
|
||||
CGConfigureDisplayWithDisplayMode(cfg, id, hidpi, nil)
|
||||
CGCompleteDisplayConfiguration(cfg, .forSession)
|
||||
}
|
||||
}
|
||||
|
||||
var bounds: CGRect { CGDisplayBounds(id) }
|
||||
|
||||
var isHiDPI: Bool { CGDisplayCopyDisplayMode(id).map { $0.pixelWidth > $0.width } ?? false }
|
||||
|
||||
var screen: NSScreen? {
|
||||
NSScreen.screens.first {
|
||||
($0.deviceDescription[NSDeviceDescriptionKey("NSScreenNumber")] as? NSNumber)?.uint32Value == id
|
||||
}
|
||||
}
|
||||
|
||||
/// Where a window may go, in global (top-left origin) points: the display
|
||||
/// minus its menu bar and anything else macOS reserves there.
|
||||
var usableRect: CGRect {
|
||||
let b = bounds
|
||||
guard let s = screen else { return b }
|
||||
let top = s.frame.maxY - s.visibleFrame.maxY, bottom = s.visibleFrame.minY - s.frame.minY
|
||||
let left = s.visibleFrame.minX - s.frame.minX, right = s.frame.maxX - s.visibleFrame.maxX
|
||||
return CGRect(x: b.minX + left, y: b.minY + top, width: b.width - left - right, height: b.height - top - bottom)
|
||||
}
|
||||
|
||||
func release() { display = nil }
|
||||
|
||||
static func isOurs(_ id: CGDirectDisplayID) -> Bool { CGDisplayVendorNumber(id) == vendor }
|
||||
}
|
||||
|
||||
/// A window through the Accessibility API.
|
||||
struct AXWindow {
|
||||
let element: AXUIElement
|
||||
|
||||
init?(windowID: CGWindowID, pid: pid_t) {
|
||||
let app = AXUIElementCreateApplication(pid)
|
||||
var value: CFTypeRef?
|
||||
guard AXUIElementCopyAttributeValue(app, kAXWindowsAttribute as CFString, &value) == .success,
|
||||
let windows = value as? [AXUIElement] else { return nil }
|
||||
guard let match = windows.first(where: { w in
|
||||
var id: CGWindowID = 0
|
||||
return axWindowID(w, &id) == .success && id == windowID
|
||||
}) else { return nil }
|
||||
element = match
|
||||
}
|
||||
|
||||
var frame: CGRect? {
|
||||
var p: CFTypeRef?, s: CFTypeRef?
|
||||
guard AXUIElementCopyAttributeValue(element, kAXPositionAttribute as CFString, &p) == .success,
|
||||
AXUIElementCopyAttributeValue(element, kAXSizeAttribute as CFString, &s) == .success else { return nil }
|
||||
var point = CGPoint.zero, size = CGSize.zero
|
||||
AXValueGetValue(p as! AXValue, .cgPoint, &point)
|
||||
AXValueGetValue(s as! AXValue, .cgSize, &size)
|
||||
return CGRect(origin: point, size: size)
|
||||
}
|
||||
|
||||
var isFullScreen: Bool {
|
||||
var v: CFTypeRef?
|
||||
return AXUIElementCopyAttributeValue(element, "AXFullScreen" as CFString, &v) == .success && (v as? Bool) == true
|
||||
}
|
||||
|
||||
/// Moving to another display: position first (so the size fits there),
|
||||
/// then size, then position again in case the size change moved it.
|
||||
func setFrame(_ r: CGRect) {
|
||||
var origin = r.origin, size = r.size
|
||||
let pos = AXValueCreate(.cgPoint, &origin)!, sz = AXValueCreate(.cgSize, &size)!
|
||||
AXUIElementSetAttributeValue(element, kAXPositionAttribute as CFString, pos)
|
||||
AXUIElementSetAttributeValue(element, kAXSizeAttribute as CFString, sz)
|
||||
AXUIElementSetAttributeValue(element, kAXPositionAttribute as CFString, pos)
|
||||
}
|
||||
}
|
||||
|
||||
/// A window on a display of its own, captured as that display.
|
||||
final class SeparateSource: CaptureSource {
|
||||
let windowID: CGWindowID
|
||||
var onFrame: ((CVPixelBuffer, CMTime, Int64) -> Void)?
|
||||
var onEnded: ((String) -> Void)?
|
||||
var onChange: (() -> Void)?
|
||||
private(set) var title = ""
|
||||
private(set) var app = ""
|
||||
private(set) var pid: pid_t?
|
||||
private(set) var gone = false
|
||||
private var display: VirtualDisplay?
|
||||
private var inner: SCKSource?
|
||||
private var window: AXWindow?
|
||||
private var original: CGRect?
|
||||
private var crop: CGRect? // display points, when only part of the display is captured
|
||||
private var poll: DispatchSourceTimer?
|
||||
private var stopped = false
|
||||
private let queue = DispatchQueue(label: "frame-mac-view.separate")
|
||||
|
||||
init(windowID: CGWindowID) { self.windowID = windowID }
|
||||
|
||||
var frameRect: CGRect { inner?.frameRect ?? .zero }
|
||||
var displayID: CGDirectDisplayID? { display?.id }
|
||||
|
||||
func start(maxLong: Int, fps: Int, completion: @escaping (String?) -> Void) {
|
||||
guard AXIsProcessTrusted() else {
|
||||
return completion("Separate windows need the Accessibility permission (to move the window)")
|
||||
}
|
||||
guard let info = WindowInfo.find(windowID) else {
|
||||
gone = true
|
||||
return completion("that window has closed")
|
||||
}
|
||||
title = info.title
|
||||
app = info.app
|
||||
pid = info.pid
|
||||
DispatchQueue.main.async { [self] in
|
||||
guard !stopped else { return }
|
||||
guard let w = AXWindow(windowID: windowID, pid: info.pid), let frame = w.frame else {
|
||||
return completion("couldn't reach that window through Accessibility")
|
||||
}
|
||||
if w.isFullScreen { return completion("take the window out of full screen first") }
|
||||
// A display the window's size, plus room for the menu bar, in
|
||||
// points; within what a panel can show sharply.
|
||||
let width = min(max(Int(frame.width.rounded()), 640), 2560) / 2 * 2
|
||||
let height = min(max(Int(frame.height.rounded()) + 40, 480), 1600) / 2 * 2
|
||||
guard let vd = VirtualDisplay(name: app.isEmpty ? "Frame Control" : "\(app) (Frame)", width: width, height: height) else {
|
||||
return completion("macOS wouldn't create a display for this window")
|
||||
}
|
||||
display = vd
|
||||
window = w
|
||||
original = frame
|
||||
// The new screen shows up in NSScreen a moment later; its menu bar
|
||||
// decides where the window can go.
|
||||
placeWindow(attempt: 0, maxLong: maxLong, fps: fps, completion: completion)
|
||||
}
|
||||
}
|
||||
|
||||
private func placeWindow(attempt: Int, maxLong: Int, fps: Int, completion: @escaping (String?) -> Void) {
|
||||
guard !stopped, let vd = display, let w = window else { return }
|
||||
// Wait for the screen to appear, and for its HiDPI mode, so the
|
||||
// first frames are captured sharp.
|
||||
if vd.screen == nil || !vd.isHiDPI, attempt < 30 {
|
||||
DispatchQueue.main.asyncAfter(deadline: .now() + 0.1) {
|
||||
self.placeWindow(attempt: attempt + 1, maxLong: maxLong, fps: fps, completion: completion)
|
||||
}
|
||||
return
|
||||
}
|
||||
let target = vd.usableRect
|
||||
w.setFrame(target)
|
||||
guard let now = w.frame, vd.bounds.intersects(now) else {
|
||||
restore()
|
||||
return completion("macOS didn't let the window move to its own display (Stage Manager can prevent this)")
|
||||
}
|
||||
let sck = SCKSource(.display(vd.id))
|
||||
// A window that keeps its own size (Calculator, some dialogs) sits in
|
||||
// the display's top-left corner: send only that corner, menu bar
|
||||
// included, with room around it for menus, not a panel of wallpaper.
|
||||
let b = vd.bounds
|
||||
if now.width < target.width - 8 || now.height < target.height - 8 {
|
||||
let w = min(b.width, max(now.maxX - b.minX, 480)), h = min(b.height, max(now.maxY - b.minY, 360))
|
||||
crop = CGRect(x: 0, y: 0, width: (w / 2).rounded(.up) * 2, height: (h / 2).rounded(.up) * 2)
|
||||
sck.crop = crop
|
||||
}
|
||||
sck.onFrame = { [weak self] pb, pts, shown in self?.onFrame?(pb, pts, shown) }
|
||||
sck.onEnded = { [weak self] reason in self?.onEnded?(reason) }
|
||||
inner = sck
|
||||
sck.start(maxLong: maxLong, fps: fps) { [weak self] error in
|
||||
// Back on main, where stop() runs too, so a viewer that left during
|
||||
// startup can't leave a capture running.
|
||||
DispatchQueue.main.async {
|
||||
guard let self, !self.stopped else { return sck.stop() }
|
||||
if let error {
|
||||
self.restore()
|
||||
return completion(error)
|
||||
}
|
||||
self.startPolling()
|
||||
completion(nil)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The window can close or be retitled; ScreenCaptureKit won't say.
|
||||
private func startPolling() {
|
||||
let t = DispatchSource.makeTimerSource(queue: queue)
|
||||
t.schedule(deadline: .now() + 1, repeating: 1)
|
||||
t.setEventHandler { [weak self] in
|
||||
guard let self else { return }
|
||||
guard let info = WindowInfo.find(self.windowID) else {
|
||||
self.gone = true
|
||||
self.onEnded?("the window closed")
|
||||
return
|
||||
}
|
||||
if !info.title.isEmpty, info.title != self.title {
|
||||
self.title = info.title
|
||||
self.onChange?()
|
||||
}
|
||||
// Displays added at the same moment can rearrange each other, and
|
||||
// someone may drag the window away: put it back on its display.
|
||||
if !info.bounds.isEmpty {
|
||||
DispatchQueue.main.async { self.keepOnDisplay(info.bounds) }
|
||||
}
|
||||
}
|
||||
t.resume()
|
||||
poll = t
|
||||
}
|
||||
|
||||
private func keepOnDisplay(_ now: CGRect) {
|
||||
guard !stopped, let vd = display, let w = window else { return }
|
||||
let b = vd.bounds
|
||||
if let c = crop {
|
||||
// Only part of the display is sent: the window must stay inside it.
|
||||
guard !c.offsetBy(dx: b.minX, dy: b.minY).insetBy(dx: -2, dy: -2).contains(now) else { return }
|
||||
} else {
|
||||
guard !b.contains(CGPoint(x: now.midX, y: now.minY + 10)) else { return }
|
||||
}
|
||||
let target = vd.usableRect
|
||||
w.setFrame(CGRect(origin: target.origin, size: now.size.width < target.width - 8 ? now.size : target.size))
|
||||
}
|
||||
|
||||
/// Lifecycle state (stopped, inner, poll, window, display) is only touched on main.
|
||||
func stop() {
|
||||
DispatchQueue.main.async { [self] in
|
||||
stopped = true
|
||||
poll?.cancel()
|
||||
poll = nil
|
||||
inner?.stop()
|
||||
restore()
|
||||
}
|
||||
}
|
||||
|
||||
/// Puts the window back where it was, then removes the display. In that
|
||||
/// order: removing a display first would let macOS pick where it goes.
|
||||
private func restore() {
|
||||
if let w = window, let r = original, WindowInfo.find(windowID) != nil { w.setFrame(r) }
|
||||
window = nil
|
||||
original = nil
|
||||
let vd = display
|
||||
display = nil
|
||||
// Give WindowServer a moment to move the window before the display goes.
|
||||
DispatchQueue.main.asyncAfter(deadline: .now() + 0.3) { vd?.release() }
|
||||
}
|
||||
|
||||
func setMaxLong(_ n: Int) { DispatchQueue.main.async { self.inner?.setMaxLong(n) } }
|
||||
|
||||
func pointer(x: Double, y: Double, text: String?) {}
|
||||
}
|
||||
@@ -0,0 +1,269 @@
|
||||
// A small HTTP/1.1 + WebSocket server on Network.framework, loopback only.
|
||||
// Enough for the agent's JSON endpoints, the viewer page and one WebSocket per
|
||||
// stream; not a general web server.
|
||||
import CryptoKit
|
||||
import Foundation
|
||||
import Network
|
||||
|
||||
struct Request {
|
||||
let method: String
|
||||
let path: String
|
||||
let query: [String: String]
|
||||
let headers: [String: String] // lower-cased names
|
||||
}
|
||||
|
||||
final class Server {
|
||||
let queue = DispatchQueue(label: "frame-mac-view.server")
|
||||
private let listener: NWListener
|
||||
private let handle: (Request, HTTPConnection) -> Void
|
||||
|
||||
init(port: UInt16, handle: @escaping (Request, HTTPConnection) -> Void) throws {
|
||||
let tcp = NWProtocolTCP.Options()
|
||||
tcp.noDelay = true
|
||||
let params = NWParameters(tls: nil, tcp: tcp)
|
||||
// Port 0 lets the system pick; `port` then says which.
|
||||
params.requiredLocalEndpoint = .hostPort(host: "127.0.0.1", port: NWEndpoint.Port(rawValue: port) ?? .any)
|
||||
params.allowLocalEndpointReuse = true
|
||||
listener = try NWListener(using: params)
|
||||
self.handle = handle
|
||||
}
|
||||
|
||||
var port: UInt16? { listener.port?.rawValue }
|
||||
|
||||
func start(ready: @escaping (Error?) -> Void) {
|
||||
listener.stateUpdateHandler = { state in
|
||||
switch state {
|
||||
case .ready: ready(nil)
|
||||
case .failed(let e): ready(e)
|
||||
default: break
|
||||
}
|
||||
}
|
||||
listener.newConnectionHandler = { [weak self] conn in
|
||||
guard let self else { return }
|
||||
HTTPConnection(conn, queue: self.queue, handle: self.handle).start()
|
||||
}
|
||||
listener.start(queue: queue)
|
||||
}
|
||||
}
|
||||
|
||||
final class HTTPConnection {
|
||||
let conn: NWConnection
|
||||
let queue: DispatchQueue
|
||||
private let handle: (Request, HTTPConnection) -> Void
|
||||
private var buffer = Data()
|
||||
private var retained: HTTPConnection? // alive until the response is sent
|
||||
|
||||
init(_ conn: NWConnection, queue: DispatchQueue, handle: @escaping (Request, HTTPConnection) -> Void) {
|
||||
self.conn = conn
|
||||
self.queue = queue
|
||||
self.handle = handle
|
||||
}
|
||||
|
||||
func start() {
|
||||
retained = self
|
||||
conn.start(queue: queue)
|
||||
readHead()
|
||||
}
|
||||
|
||||
private func readHead() {
|
||||
conn.receive(minimumIncompleteLength: 1, maximumLength: 16384) { [self] data, _, done, error in
|
||||
if let data { buffer.append(data) }
|
||||
if let end = buffer.range(of: Data("\r\n\r\n".utf8)) {
|
||||
guard let req = Self.parse(buffer[..<end.lowerBound]) else { return respond(400, text: "bad request") }
|
||||
handle(req, self)
|
||||
} else if error != nil || done || buffer.count > 16384 {
|
||||
close()
|
||||
} else {
|
||||
readHead()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static func parse(_ head: Data) -> Request? {
|
||||
guard let text = String(data: head, encoding: .utf8) else { return nil }
|
||||
let lines = text.components(separatedBy: "\r\n")
|
||||
let parts = lines[0].split(separator: " ")
|
||||
guard parts.count >= 2, let url = URLComponents(string: String(parts[1])) else { return nil }
|
||||
var headers: [String: String] = [:]
|
||||
for line in lines.dropFirst() {
|
||||
guard let colon = line.firstIndex(of: ":") else { continue }
|
||||
headers[line[..<colon].lowercased()] = line[line.index(after: colon)...].trimmingCharacters(in: .whitespaces)
|
||||
}
|
||||
var query: [String: String] = [:]
|
||||
for item in url.queryItems ?? [] { query[item.name] = item.value ?? "" }
|
||||
return Request(method: String(parts[0]), path: url.path, query: query, headers: headers)
|
||||
}
|
||||
|
||||
func respond(_ status: Int, body: Data, type: String) {
|
||||
let reason = [200: "OK", 400: "Bad Request", 403: "Forbidden", 404: "Not Found", 409: "Conflict", 500: "Internal Server Error"][status] ?? "Error"
|
||||
var head = "HTTP/1.1 \(status) \(reason)\r\nContent-Type: \(type)\r\nContent-Length: \(body.count)\r\n"
|
||||
head += "Cache-Control: no-store\r\nConnection: close\r\n\r\n"
|
||||
conn.send(content: Data(head.utf8) + body, isComplete: true, completion: .contentProcessed { [self] _ in close() })
|
||||
}
|
||||
|
||||
func respond(_ status: Int, text: String) {
|
||||
respond(status, body: Data(text.utf8), type: "text/plain; charset=utf-8")
|
||||
}
|
||||
|
||||
func respond(_ status: Int = 200, json: Any) {
|
||||
let body = (try? JSONSerialization.data(withJSONObject: json, options: [.sortedKeys])) ?? Data("{}".utf8)
|
||||
respond(status, body: body, type: "application/json")
|
||||
}
|
||||
|
||||
func close() {
|
||||
conn.cancel()
|
||||
retained = nil
|
||||
}
|
||||
|
||||
/// Completes the WebSocket handshake and hands the connection over.
|
||||
func upgrade(_ req: Request) -> WebSocket? {
|
||||
guard req.headers["upgrade"]?.lowercased() == "websocket", let key = req.headers["sec-websocket-key"] else {
|
||||
respond(400, text: "expected a WebSocket upgrade")
|
||||
return nil
|
||||
}
|
||||
let accept = Data(Insecure.SHA1.hash(data: Data((key + "258EAFA5-E914-47DA-95CA-C5AB0DC85B11").utf8))).base64EncodedString()
|
||||
let head = "HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: \(accept)\r\n\r\n"
|
||||
conn.send(content: Data(head.utf8), completion: .contentProcessed { _ in })
|
||||
let ws = WebSocket(conn, queue: queue)
|
||||
retained = nil
|
||||
return ws
|
||||
}
|
||||
}
|
||||
|
||||
/// Server side of RFC 6455. Sends are counted until the network stack has
|
||||
/// taken them, so the stream can skip frames instead of queueing seconds of
|
||||
/// video on a slow link.
|
||||
final class WebSocket {
|
||||
static let maxMessage: UInt64 = 1 << 20
|
||||
let conn: NWConnection
|
||||
let queue: DispatchQueue
|
||||
var onText: ((String) -> Void)?
|
||||
var onClose: (() -> Void)?
|
||||
private var buffer = Data()
|
||||
private var fragments = Data()
|
||||
private var fragmentOpcode: UInt8 = 0
|
||||
private var closed = false
|
||||
private var retained: WebSocket?
|
||||
private let lock = NSLock()
|
||||
private var _pending = 0
|
||||
var onDrain: (() -> Void)?
|
||||
|
||||
/// Bytes handed to the connection that it hasn't sent yet. Any thread.
|
||||
var pendingBytes: Int { lock.lock(); defer { lock.unlock() }; return _pending }
|
||||
|
||||
init(_ conn: NWConnection, queue: DispatchQueue) {
|
||||
self.conn = conn
|
||||
self.queue = queue
|
||||
}
|
||||
|
||||
func start() {
|
||||
retained = self
|
||||
read()
|
||||
}
|
||||
|
||||
func sendText(_ s: String) { send(opcode: 1, Data(s.utf8)) }
|
||||
func sendJSON(_ obj: Any) {
|
||||
if let d = try? JSONSerialization.data(withJSONObject: obj), let s = String(data: d, encoding: .utf8) { sendText(s) }
|
||||
}
|
||||
/// `taken` runs once the network stack has taken the whole frame.
|
||||
func sendBinary(_ d: Data, taken: (() -> Void)? = nil) { send(opcode: 2, d, taken: taken) }
|
||||
|
||||
func send(opcode: UInt8, _ payload: Data, taken: (() -> Void)? = nil) {
|
||||
var frame = Data([0x80 | opcode])
|
||||
let n = payload.count
|
||||
if n < 126 {
|
||||
frame.append(UInt8(n))
|
||||
} else if n < 65536 {
|
||||
frame.append(126)
|
||||
frame.append(contentsOf: [UInt8(n >> 8), UInt8(n & 0xff)])
|
||||
} else {
|
||||
frame.append(127)
|
||||
for shift in stride(from: 56, through: 0, by: -8) { frame.append(UInt8((UInt64(n) >> UInt64(shift)) & 0xff)) }
|
||||
}
|
||||
frame.append(payload)
|
||||
let size = frame.count
|
||||
lock.lock(); _pending += size; lock.unlock()
|
||||
conn.send(content: frame, completion: .contentProcessed { [weak self] _ in
|
||||
guard let self else { return }
|
||||
self.lock.lock(); self._pending -= size; self.lock.unlock()
|
||||
taken?()
|
||||
self.onDrain?()
|
||||
})
|
||||
}
|
||||
|
||||
func close() {
|
||||
guard !closed else { return }
|
||||
closed = true
|
||||
send(opcode: 8, Data([0x03, 0xe8])) // 1000, normal closure
|
||||
conn.send(content: nil, isComplete: true, completion: .contentProcessed { [weak self] _ in self?.conn.cancel() })
|
||||
finish()
|
||||
}
|
||||
|
||||
private func finish() {
|
||||
let cb = onClose
|
||||
onClose = nil
|
||||
onText = nil
|
||||
onDrain = nil
|
||||
cb?()
|
||||
retained = nil
|
||||
}
|
||||
|
||||
private func read() {
|
||||
conn.receive(minimumIncompleteLength: 1, maximumLength: 65536) { [weak self] data, _, done, error in
|
||||
guard let self, !self.closed else { return }
|
||||
if let data { self.buffer.append(data) }
|
||||
self.parse()
|
||||
if error != nil || done {
|
||||
self.closed = true
|
||||
self.conn.cancel()
|
||||
self.finish()
|
||||
} else if !self.closed {
|
||||
self.read()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func parse() {
|
||||
while buffer.count >= 2 {
|
||||
let b = [UInt8](buffer.prefix(14))
|
||||
let fin = b[0] & 0x80 != 0, opcode = b[0] & 0x0f, masked = b[1] & 0x80 != 0
|
||||
// Lengths are unsigned and clients only send small control
|
||||
// messages, so anything big (or a 64-bit length with the top bit
|
||||
// set) is refused before it's turned into an Int.
|
||||
var len64 = UInt64(b[1] & 0x7f), off = 2
|
||||
if len64 == 126 {
|
||||
guard b.count >= 4 else { return }
|
||||
len64 = UInt64(b[2]) << 8 | UInt64(b[3]); off = 4
|
||||
} else if len64 == 127 {
|
||||
guard b.count >= 10 else { return }
|
||||
len64 = 0
|
||||
for i in 2..<10 { len64 = len64 << 8 | UInt64(b[i]) }
|
||||
off = 10
|
||||
}
|
||||
guard len64 <= WebSocket.maxMessage else { return close() }
|
||||
let len = Int(len64)
|
||||
let maskOff = off
|
||||
if masked { off += 4 }
|
||||
guard buffer.count >= off + len else { return }
|
||||
let start = buffer.startIndex
|
||||
var payload = Data(buffer[(start + off)..<(start + off + len)])
|
||||
if masked {
|
||||
let mask = [UInt8](buffer[(start + maskOff)..<(start + maskOff + 4)])
|
||||
payload.withUnsafeMutableBytes { p in
|
||||
for i in 0..<len { p[i] ^= mask[i & 3] }
|
||||
}
|
||||
}
|
||||
buffer.removeFirst(off + len)
|
||||
switch opcode {
|
||||
case 0, 1, 2:
|
||||
if opcode != 0 { fragmentOpcode = opcode; fragments = Data() }
|
||||
guard fragments.count + payload.count <= Int(WebSocket.maxMessage) else { return close() }
|
||||
fragments.append(payload)
|
||||
if fin, fragmentOpcode == 1, let s = String(data: fragments, encoding: .utf8) { onText?(s) }
|
||||
case 8: close(); return
|
||||
case 9: send(opcode: 10, payload)
|
||||
default: break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
// Per-frame timing, so every change to the stream can be judged by numbers.
|
||||
// All times are the agent's host clock in microseconds (the clock
|
||||
// ScreenCaptureKit stamps frames with). The viewer syncs to it over the
|
||||
// WebSocket and reports when each frame arrived, decoded and was drawn.
|
||||
import CoreMedia
|
||||
import Foundation
|
||||
|
||||
private let timebase: mach_timebase_info_data_t = {
|
||||
var t = mach_timebase_info_data_t()
|
||||
mach_timebase_info(&t)
|
||||
return t
|
||||
}()
|
||||
|
||||
/// Now on the host clock, in microseconds.
|
||||
func nowUs() -> Int64 { hostUs(mach_absolute_time()) }
|
||||
|
||||
/// A mach_absolute_time value (as in SCStreamFrameInfo.displayTime) in microseconds.
|
||||
func hostUs(_ ticks: UInt64) -> Int64 { Int64(ticks / 1000 * UInt64(timebase.numer) / UInt64(timebase.denom)) }
|
||||
|
||||
/// A host-clock CMTime (ScreenCaptureKit's presentation times) in microseconds.
|
||||
func hostUs(_ t: CMTime) -> Int64 { t.isValid ? Int64(CMTimeGetSeconds(t) * 1_000_000) : nowUs() }
|
||||
|
||||
/// One frame's journey. Zero means "didn't happen" or "not reported yet".
|
||||
struct FrameRecord {
|
||||
var seq: UInt32 = 0
|
||||
var key = false
|
||||
var bytes = 0
|
||||
var width = 0, height = 0
|
||||
var capture: Int64 = 0 // the Mac composited it (display time)
|
||||
var arrived: Int64 = 0 // ScreenCaptureKit handed it to us
|
||||
var encodeStart: Int64 = 0
|
||||
var encodeEnd: Int64 = 0
|
||||
var sent: Int64 = 0 // handed to the WebSocket
|
||||
var wire: Int64 = 0 // the socket took it
|
||||
var received: Int64 = 0 // viewer, agent clock
|
||||
var decoded: Int64 = 0
|
||||
var drawn: Int64 = 0
|
||||
var vsync: Int64 = 0 // the viewer's next animation frame after drawing it
|
||||
var echo: UInt32 = 0 // the first frame after input `echo`
|
||||
var tier = 0
|
||||
var bitrate = 0
|
||||
|
||||
var json: [String: Any] {
|
||||
["s": seq, "k": key ? 1 : 0, "b": bytes, "w": width, "h": height, "cap": capture, "arr": arrived,
|
||||
"e0": encodeStart, "e1": encodeEnd, "snd": sent, "wire": wire, "rx": received, "dec": decoded,
|
||||
"drw": drawn, "vs": vsync, "echo": echo, "tier": tier, "br": bitrate]
|
||||
}
|
||||
}
|
||||
|
||||
/// One input event from the viewer and what came of it.
|
||||
struct InputRecord {
|
||||
var id: UInt32
|
||||
var kind: String
|
||||
var viewer: Int64 // the viewer's event time, agent clock
|
||||
var injected: Int64 = 0 // posted as a CGEvent
|
||||
var frame: UInt32 = 0 // first frame captured after it (0: none yet)
|
||||
var capture: Int64 = 0
|
||||
|
||||
var json: [String: Any] {
|
||||
["id": id, "kind": kind, "tv": viewer, "inj": injected, "frame": frame, "cap": capture]
|
||||
}
|
||||
}
|
||||
|
||||
func percentile(_ sorted: [Double], _ p: Double) -> Double? {
|
||||
guard !sorted.isEmpty else { return nil }
|
||||
let i = min(sorted.count - 1, max(0, Int((p * Double(sorted.count - 1)).rounded())))
|
||||
return sorted[i]
|
||||
}
|
||||
|
||||
/// Frames and inputs of one stream, kept for the last few thousand frames.
|
||||
/// Any thread.
|
||||
final class StreamStats {
|
||||
static let capacity = 4096
|
||||
private let lock = NSLock()
|
||||
private var frames = [FrameRecord?](repeating: nil, count: StreamStats.capacity)
|
||||
private var inputs: [UInt32: InputRecord] = [:]
|
||||
private var inputOrder: [UInt32] = []
|
||||
private(set) var nextSeq: UInt32 = 1
|
||||
var captured = 0 // frames ScreenCaptureKit delivered
|
||||
var skipped = 0 // held back from encoding (link, encoder, pacing or gate busy); only the newest may go later
|
||||
var viewerDropped = 0 // not shown by the viewer (behind, or waiting for a keyframe)
|
||||
var rtt: Double = 0 // ms, the viewer's best recent ping
|
||||
var clockSynced = false
|
||||
var decoder = "" // what the viewer reports about its decoder
|
||||
|
||||
func withLock<T>(_ f: () -> T) -> T { lock.lock(); defer { lock.unlock() }; return f() }
|
||||
|
||||
func newFrame(_ r: FrameRecord) -> UInt32 {
|
||||
withLock {
|
||||
var r = r
|
||||
r.seq = nextSeq
|
||||
nextSeq &+= 1
|
||||
frames[Int(r.seq) % StreamStats.capacity] = r
|
||||
return r.seq
|
||||
}
|
||||
}
|
||||
|
||||
func update(_ seq: UInt32, _ f: (inout FrameRecord) -> Void) {
|
||||
withLock {
|
||||
let i = Int(seq) % StreamStats.capacity
|
||||
guard var r = frames[i], r.seq == seq else { return }
|
||||
f(&r)
|
||||
frames[i] = r
|
||||
}
|
||||
}
|
||||
|
||||
func frame(_ seq: UInt32) -> FrameRecord? {
|
||||
withLock {
|
||||
let r = frames[Int(seq) % StreamStats.capacity]
|
||||
return r?.seq == seq ? r : nil
|
||||
}
|
||||
}
|
||||
|
||||
func addInput(_ r: InputRecord) {
|
||||
withLock {
|
||||
inputs[r.id] = r
|
||||
inputOrder.append(r.id)
|
||||
if inputOrder.count > 512 { inputs[inputOrder.removeFirst()] = nil }
|
||||
}
|
||||
}
|
||||
|
||||
func updateInput(_ id: UInt32, _ f: (inout InputRecord) -> Void) {
|
||||
withLock { if var r = inputs[id] { f(&r); inputs[id] = r } }
|
||||
}
|
||||
|
||||
/// Frames with seq > `since` that were sent at least `settle` µs ago, so
|
||||
/// the viewer has had time to report on them.
|
||||
func settled(since: UInt32, settle: Int64 = 1_500_000) -> [FrameRecord] {
|
||||
let cutoff = nowUs() - settle
|
||||
return withLock {
|
||||
frames.compactMap { $0 }.filter { $0.seq > since && $0.sent > 0 && $0.sent < cutoff }.sorted { $0.seq < $1.seq }
|
||||
}
|
||||
}
|
||||
|
||||
func inputList() -> [InputRecord] { withLock { inputOrder.compactMap { inputs[$0] } } }
|
||||
|
||||
/// Percentiles over the last `window` µs, for /status and the overlay.
|
||||
func summary(window: Int64 = 2_000_000) -> [String: Any] {
|
||||
let now = nowUs(), from = now - window
|
||||
let recent = withLock { frames.compactMap { $0 }.filter { $0.sent > from } }
|
||||
func ms(_ pick: (FrameRecord) -> (Int64, Int64)) -> [String: Double] {
|
||||
let v = recent.compactMap { r -> Double? in
|
||||
let (a, b) = pick(r)
|
||||
return a > 0 && b > 0 ? Double(b - a) / 1000 : nil
|
||||
}.sorted()
|
||||
guard !v.isEmpty else { return [:] }
|
||||
return ["p50": (percentile(v, 0.5)! * 10).rounded() / 10, "p95": (percentile(v, 0.95)! * 10).rounded() / 10]
|
||||
}
|
||||
let secs = Double(window) / 1_000_000
|
||||
let shown = recent.filter { $0.vsync > 0 }.count
|
||||
let bytes = recent.reduce(0) { $0 + $1.bytes }
|
||||
var s: [String: Any] = [
|
||||
"capture": ms { ($0.capture, $0.arrived) },
|
||||
"queue": ms { ($0.arrived, $0.encodeStart) },
|
||||
"encode": ms { ($0.encodeStart, $0.encodeEnd) },
|
||||
"network": ms { ($0.encodeEnd, $0.received) },
|
||||
"decode": ms { ($0.received, $0.decoded) },
|
||||
"draw": ms { ($0.decoded, $0.vsync) },
|
||||
"total": ms { ($0.capture, $0.vsync) },
|
||||
"fps": (Double(shown) / secs * 10).rounded() / 10,
|
||||
"sentFps": (Double(recent.count) / secs * 10).rounded() / 10,
|
||||
"mbps": (Double(bytes * 8) / secs / 1e5).rounded() / 10,
|
||||
"rtt": (rtt * 10).rounded() / 10,
|
||||
"synced": clockSynced,
|
||||
]
|
||||
withLock {
|
||||
s["captured"] = captured
|
||||
s["skipped"] = skipped
|
||||
s["dropped"] = viewerDropped
|
||||
s["decoder"] = decoder
|
||||
let done = inputOrder.suffix(20).compactMap { inputs[$0] }.compactMap { i -> Double? in
|
||||
guard i.frame != 0, let f = frames[Int(i.frame) % StreamStats.capacity], f.seq == i.frame, f.vsync > 0
|
||||
else { return nil }
|
||||
return Double(f.vsync - i.viewer) / 1000
|
||||
}.sorted()
|
||||
if !done.isEmpty { s["input"] = ["p50": percentile(done, 0.5)!, "n": done.count] }
|
||||
}
|
||||
return s
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,755 @@
|
||||
// frame-mac-view: streams Mac windows or displays to viewers on the Steam
|
||||
// Frame and plays their pointer and key input back on the Mac.
|
||||
//
|
||||
// frame-mac-view serve --port 47811 --page ui/mac-view.html (token in FRAME_MAC_VIEW_TOKEN)
|
||||
// frame-mac-view windows | displays | permissions (JSON on stdout)
|
||||
// frame-mac-view request-permissions (shows macOS's prompts)
|
||||
//
|
||||
// It listens on 127.0.0.1 only. Frame Control reaches it from the Frame
|
||||
// through an SSH reverse tunnel, and every request must carry the token.
|
||||
//
|
||||
// HTTP. Frame Control's key (?k=, from FRAME_MAC_VIEW_TOKEN) never leaves the
|
||||
// Mac; the Frame gets a single-use ticket per viewer instead.
|
||||
// GET /ping, /view open: tunnel check, viewer page
|
||||
// GET /stream?src=...&t=TICKET|r=KEY WebSocket (&codec=h264|jpeg&max=&fps=&bpp=)
|
||||
// GET /status, /windows, /displays ?k=: permissions, streams, sources
|
||||
// POST /ticket?src=... ?k=: a ticket for one viewer of src
|
||||
// POST /close[?src=...] ?k=: end those streams, close their windows
|
||||
// POST /permissions ?k=: show macOS's permission prompts
|
||||
// GET /stats[?id=&since=] ?k=: per-frame timing records (for benchmarks)
|
||||
// POST /bench?src=&action=... ?k=: ask viewers of src to type, click or show their overlay
|
||||
// src is window:<CGWindowID>, display:<CGDirectDisplayID> or test.
|
||||
import AppKit
|
||||
import ApplicationServices
|
||||
import Foundation
|
||||
import IOKit.pwr_mgt
|
||||
import ScreenCaptureKit
|
||||
import Security
|
||||
|
||||
let version = "1"
|
||||
|
||||
func windowsJSON() -> [[String: Any]] {
|
||||
let me = ProcessInfo.processInfo.processIdentifier
|
||||
let skip: Set<String> = ["Window Server", "Dock", "Control Centre", "Control Center", "Notification Centre",
|
||||
"Notification Center", "Spotlight", "SystemUIServer", "Wallpaper", "WindowManager"]
|
||||
return WindowInfo.all().filter {
|
||||
$0.layer == 0 && $0.pid != me && !skip.contains($0.app) && $0.bounds.width >= 120 && $0.bounds.height >= 80
|
||||
}.map {
|
||||
["id": $0.id, "src": "window:\($0.id)", "pid": $0.pid, "app": $0.app, "title": $0.title,
|
||||
"w": Int($0.bounds.width), "h": Int($0.bounds.height)]
|
||||
}
|
||||
}
|
||||
|
||||
func displaysJSON() -> [[String: Any]] {
|
||||
var ids = [CGDirectDisplayID](repeating: 0, count: 16)
|
||||
var n: UInt32 = 0
|
||||
// Online, not active: a display that's asleep is still one you can stream.
|
||||
CGGetOnlineDisplayList(16, &ids, &n)
|
||||
return ids.prefix(Int(n)).filter {
|
||||
CGDisplayMirrorsDisplay($0) == kCGNullDirectDisplay && !VirtualDisplay.isOurs($0) // not a separated window's
|
||||
}.map { id in
|
||||
let b = CGDisplayBounds(id)
|
||||
return ["id": id, "src": "display:\(id)", "name": displayName(id), "w": Int(b.width), "h": Int(b.height),
|
||||
"main": CGDisplayIsMain(id) != 0]
|
||||
}
|
||||
}
|
||||
|
||||
func permissionsJSON() -> [String: Any] {
|
||||
["screen": CGPreflightScreenCaptureAccess(), "accessibility": AXIsProcessTrusted()]
|
||||
}
|
||||
|
||||
func printJSON(_ obj: Any) {
|
||||
let d = (try? JSONSerialization.data(withJSONObject: obj, options: [.sortedKeys, .prettyPrinted])) ?? Data()
|
||||
FileHandle.standardOutput.write(d + Data("\n".utf8))
|
||||
}
|
||||
|
||||
/// A number from a JSON message, whatever its JSON type.
|
||||
func num(_ v: Any?) -> Double? { (v as? NSNumber)?.doubleValue }
|
||||
|
||||
/// One viewer watching one source.
|
||||
final class Session {
|
||||
let id: Int
|
||||
let source: Source
|
||||
let capture: CaptureSource
|
||||
let encoder: Encoder
|
||||
let ws: WebSocket
|
||||
let codec: Codec
|
||||
let reconnectKey: String
|
||||
let stats = StreamStats()
|
||||
let controller: RateController
|
||||
private var appliedScale = 1.0
|
||||
private var lastSubmit: Int64 = 0
|
||||
private var paceScheduled = false
|
||||
private let lock = NSLock()
|
||||
/// A captured picture waiting to be encoded (or the last one encoded).
|
||||
private struct Picture {
|
||||
let pb: CVPixelBuffer
|
||||
let pts: CMTime
|
||||
let capture: Int64
|
||||
let arrived: Int64
|
||||
var echo: UInt32 // the input this picture is the first reply to
|
||||
}
|
||||
private var last: Picture?
|
||||
private var skipped = false
|
||||
private var stopped = false
|
||||
private var inFlight = 0
|
||||
/// Input posted to the Mac whose effect hasn't been captured yet: the
|
||||
/// next picture composited after `after` is tagged with it.
|
||||
private var pendingEcho: (id: UInt32, after: Int64)?
|
||||
private var statsTimer: DispatchSourceTimer?
|
||||
/// Frames already queued for the network; beyond this, or with two frames
|
||||
/// already in the encoder, new frames are skipped (before encoding, so no
|
||||
/// reference frame goes missing) and the newest picture is sent once
|
||||
/// things catch up. Only that newest picture is kept meanwhile.
|
||||
let maxPending: Int
|
||||
static let maxInFlight = 2
|
||||
var onEnd: ((Session) -> Void)?
|
||||
var onAck: (() -> Void)?
|
||||
var onFinished: ((String) -> Void)?
|
||||
private let encodeQueue = DispatchQueue(label: "frame-mac-view.encode", qos: .userInteractive)
|
||||
|
||||
init(id: Int, source: Source, capture: CaptureSource, ws: WebSocket, codec: Codec, fps: Int, bitsPerPixel: Double,
|
||||
reconnectKey: String) {
|
||||
self.id = id
|
||||
self.source = source
|
||||
self.capture = capture
|
||||
self.ws = ws
|
||||
self.codec = codec
|
||||
self.reconnectKey = reconnectKey
|
||||
encoder = Encoder(codec: codec, fps: fps, bitsPerPixel: bitsPerPixel)
|
||||
controller = RateController(maxFps: fps)
|
||||
maxPending = codec == .jpeg ? 3 << 20 : 1 << 20
|
||||
}
|
||||
|
||||
/// Frames come faster than the current tier's frame rate: this one waits,
|
||||
/// and goes when its turn comes (unless a newer one replaces it).
|
||||
/// Call with `lock` held.
|
||||
private func paced(now: Int64) -> Bool {
|
||||
let fps = controller.fps
|
||||
guard fps < controller.maxFps, lastSubmit > 0 else { return false }
|
||||
let interval = Int64(1_000_000 / fps), due = lastSubmit + interval * 9 / 10
|
||||
guard now < due else { return false }
|
||||
if !paceScheduled {
|
||||
paceScheduled = true
|
||||
encodeQueue.asyncAfter(deadline: .now() + .microseconds(Int(due - now))) { [weak self] in
|
||||
guard let self else { return }
|
||||
self.lock.lock(); self.paceScheduled = false; self.lock.unlock()
|
||||
self.resendIfRoom()
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
/// Encodes `pb` unless the link or the encoder is busy, in which case it
|
||||
/// becomes the picture to send next.
|
||||
private func offer(_ pb: CVPixelBuffer, pts: CMTime, capture shown: Int64) {
|
||||
let arrived = nowUs()
|
||||
stats.withLock { stats.captured += 1 }
|
||||
controller.captured(at: arrived)
|
||||
lock.lock()
|
||||
// A reply to input stays with whichever picture ends up being sent.
|
||||
var echo = skipped ? last?.echo ?? 0 : 0
|
||||
if let p = pendingEcho, shown >= p.after {
|
||||
echo = p.id
|
||||
pendingEcho = nil
|
||||
}
|
||||
let picture = Picture(pb: pb, pts: pts, capture: shown, arrived: arrived, echo: echo)
|
||||
last = picture
|
||||
let busy = stopped || ws.pendingBytes > maxPending || inFlight >= Session.maxInFlight
|
||||
|| paced(now: arrived) || !controller.maySend(now: arrived)
|
||||
if busy { skipped = true } else { inFlight += 1; last?.echo = 0; lastSubmit = arrived }
|
||||
lock.unlock()
|
||||
if busy { stats.withLock { stats.skipped += 1 } } else { submit(picture) }
|
||||
}
|
||||
|
||||
private func submit(_ p: Picture) {
|
||||
encodeQueue.async {
|
||||
var r = FrameRecord()
|
||||
r.capture = p.capture
|
||||
r.arrived = p.arrived
|
||||
r.echo = p.echo
|
||||
r.bitrate = self.encoder.bitrate
|
||||
r.tier = self.controller.tier
|
||||
r.encodeStart = nowUs()
|
||||
let seq = self.stats.newFrame(r)
|
||||
if p.echo != 0 { self.stats.updateInput(p.echo) { $0.frame = seq; $0.capture = p.capture } }
|
||||
if !self.encoder.encode(p.pb, pts: p.pts, seq: seq) { self.finished() }
|
||||
}
|
||||
}
|
||||
|
||||
/// An encode finished (or failed): send the newest skipped picture if
|
||||
/// there's room now.
|
||||
private func finished() {
|
||||
lock.lock()
|
||||
inFlight = max(0, inFlight - 1)
|
||||
lock.unlock()
|
||||
resendIfRoom()
|
||||
}
|
||||
|
||||
private func resendIfRoom() {
|
||||
lock.lock()
|
||||
var next: Picture?
|
||||
let now = nowUs()
|
||||
if !stopped, skipped, ws.pendingBytes <= maxPending / 2, inFlight < Session.maxInFlight, let l = last,
|
||||
!paced(now: now), controller.maySend(now: now, counts: false) {
|
||||
next = l
|
||||
skipped = false
|
||||
inFlight += 1
|
||||
last?.echo = 0
|
||||
lastSubmit = now
|
||||
}
|
||||
lock.unlock()
|
||||
// Stamped now: VideoToolbox needs increasing times, and the capture
|
||||
// time stays in the record, so the wait counts as latency.
|
||||
if let p = next {
|
||||
submit(Picture(pb: p.pb, pts: CMClockGetTime(CMClockGetHostTimeClock()), capture: p.capture,
|
||||
arrived: p.arrived, echo: p.echo))
|
||||
}
|
||||
}
|
||||
|
||||
func start(maxLong: Int, fps: Int) {
|
||||
encoder.onFrame = { [weak self] data, key, pts, seq in
|
||||
guard let self else { return }
|
||||
let t = nowUs()
|
||||
// The quality setting's bitrate, at full size, is the most it gets.
|
||||
if self.appliedScale == 1 {
|
||||
self.controller.setCeiling(self.encoder.defaultBitrate(width: self.encoder.width, height: self.encoder.height))
|
||||
}
|
||||
self.controller.sent(seq: seq, bytes: data.count + 17, at: t)
|
||||
var echo: UInt32 = 0
|
||||
let (w, h) = (self.encoder.width, self.encoder.height)
|
||||
self.stats.update(seq) {
|
||||
$0.encodeEnd = t
|
||||
$0.sent = t
|
||||
$0.bytes = data.count
|
||||
$0.key = key
|
||||
$0.width = w
|
||||
$0.height = h
|
||||
echo = $0.echo
|
||||
}
|
||||
// Header: flags (1 = keyframe), pts µs, sequence number, and the
|
||||
// input this frame is the first reply to; all big-endian.
|
||||
var msg = Data(capacity: data.count + 17)
|
||||
msg.append(key ? 1 : 0)
|
||||
var us = UInt64(max(0, CMTimeGetSeconds(pts)) * 1_000_000).bigEndian
|
||||
msg.append(Data(bytes: &us, count: 8))
|
||||
var s = seq.bigEndian, e = echo.bigEndian
|
||||
msg.append(Data(bytes: &s, count: 4))
|
||||
msg.append(Data(bytes: &e, count: 4))
|
||||
msg.append(data)
|
||||
let stats = self.stats
|
||||
self.ws.sendBinary(msg) { stats.update(seq) { $0.wire = nowUs() } }
|
||||
}
|
||||
encoder.onDone = { [weak self] _ in self?.finished() }
|
||||
encoder.onError = { [weak self] message in self?.ws.sendJSON(["t": "error", "message": message]) }
|
||||
capture.onFrame = { [weak self] pb, pts, shown in self?.offer(pb, pts: pts, capture: shown) }
|
||||
capture.onEnded = { [weak self] reason in
|
||||
guard let self else { return }
|
||||
self.ws.sendJSON(["t": "closed", "reason": reason])
|
||||
self.onFinished?(self.source.key)
|
||||
self.end()
|
||||
}
|
||||
ws.onDrain = { [weak self] in self?.resendIfRoom() }
|
||||
ws.onText = { [weak self] text in self?.handle(text) }
|
||||
ws.onClose = { [weak self] in self?.end() }
|
||||
ws.start()
|
||||
// Reconnect with this (kept in the page's memory, never on a command line).
|
||||
ws.sendJSON(["t": "hello", "r": reconnectKey])
|
||||
capture.onChange = { [weak self] in self?.sendInfo() }
|
||||
// The numbers, for the viewer's overlay.
|
||||
// Adapt ten times a second; the numbers go to the viewer once a second.
|
||||
let t = DispatchSource.makeTimerSource(queue: encodeQueue)
|
||||
t.schedule(deadline: .now() + 0.1, repeating: 0.1)
|
||||
var ticks = 0
|
||||
t.setEventHandler { [weak self] in
|
||||
guard let self, !self.isStopped else { return }
|
||||
self.adapt(maxLong: maxLong)
|
||||
ticks += 1
|
||||
guard ticks % 10 == 0 else { return }
|
||||
var s = self.stats.summary()
|
||||
s.merge(self.controller.state()) { a, _ in a }
|
||||
s["t"] = "stats"
|
||||
s["bitrate"] = self.encoder.bitrate
|
||||
s["size"] = "\(self.encoder.width)×\(self.encoder.height)"
|
||||
self.ws.sendJSON(s)
|
||||
}
|
||||
t.resume()
|
||||
statsTimer = t
|
||||
capture.start(maxLong: maxLong, fps: fps) { [weak self] error in
|
||||
guard let self else { return }
|
||||
if let error {
|
||||
if self.capture.gone {
|
||||
// Final, like a window closing mid-stream: the viewer closes
|
||||
// and its key is revoked, rather than retrying for ever.
|
||||
self.ws.sendJSON(["t": "closed", "reason": error])
|
||||
self.onFinished?(self.source.key)
|
||||
} else {
|
||||
self.ws.sendJSON(["t": "error", "message": error])
|
||||
}
|
||||
self.end()
|
||||
return
|
||||
}
|
||||
self.sendInfo()
|
||||
}
|
||||
}
|
||||
|
||||
/// Applies the controller's bitrate and tier. On the encoding queue.
|
||||
private func adapt(maxLong: Int) {
|
||||
guard let bps = controller.update(now: nowUs()) else { return }
|
||||
encoder.setBitrate(bps)
|
||||
let scale = controller.scale
|
||||
if scale != appliedScale {
|
||||
appliedScale = scale
|
||||
capture.setMaxLong(max(320, Int(Double(maxLong) * scale)))
|
||||
}
|
||||
resendIfRoom() // a lower tier may let a held frame go now
|
||||
}
|
||||
|
||||
/// While someone is typing or clicking, the viewer sends a tiny message this
|
||||
/// often (ms, 0 = off), so the Frame's Wi-Fi doesn't doze between the input
|
||||
/// and the frame that answers it (power saving is on there).
|
||||
static let warmMs = Int(ProcessInfo.processInfo.environment["FRAME_MAC_VIEW_WARM"] ?? "") ?? 0
|
||||
|
||||
func sendInfo() {
|
||||
ws.sendJSON(["t": "info", "src": source.key, "title": capture.title, "app": capture.app, "codec": codec.rawValue,
|
||||
"input": source == .test || Input.allowed, "warm": Session.warmMs,
|
||||
"aspect": Double(capture.frameRect.width / max(capture.frameRect.height, 1))])
|
||||
}
|
||||
|
||||
var isStopped: Bool { lock.lock(); defer { lock.unlock() }; return stopped }
|
||||
|
||||
func end() {
|
||||
lock.lock()
|
||||
let was = stopped
|
||||
stopped = true
|
||||
last = nil
|
||||
lock.unlock()
|
||||
guard !was else { return }
|
||||
statsTimer?.cancel()
|
||||
capture.stop()
|
||||
encodeQueue.async { self.encoder.invalidate() }
|
||||
let owner = id
|
||||
DispatchQueue.main.async { Input.releaseAll(owner: owner) }
|
||||
ws.close()
|
||||
onEnd?(self)
|
||||
}
|
||||
|
||||
/// A point in the picture (0...1 each way) -> Mac global coordinates.
|
||||
private func point(_ x: Double, _ y: Double) -> CGPoint {
|
||||
let r = capture.frameRect
|
||||
return CGPoint(x: r.minX + min(max(x, 0), 1) * r.width, y: r.minY + min(max(y, 0), 1) * r.height)
|
||||
}
|
||||
|
||||
/// Input with an id (from the viewer) has been posted: the next picture
|
||||
/// the Mac composites is its first chance to show.
|
||||
private func injected(_ m: [String: Any], kind: String) {
|
||||
guard let iid = (m["i"] as? NSNumber)?.uint32Value, iid != 0 else { return }
|
||||
let now = nowUs()
|
||||
stats.addInput(InputRecord(id: iid, kind: kind, viewer: Int64(num(m["tv"]) ?? 0), injected: now))
|
||||
lock.lock(); pendingEcho = (iid, now); lock.unlock()
|
||||
}
|
||||
|
||||
/// Timing reports from the viewer, in the agent's clock.
|
||||
private func report(_ t: String, _ m: [String: Any]) -> Bool {
|
||||
switch t {
|
||||
case "ping":
|
||||
ws.sendJSON(["t": "pong", "c": m["c"] ?? 0, "a": nowUs()])
|
||||
case "rx":
|
||||
guard let s = (m["s"] as? NSNumber)?.uint32Value else { break }
|
||||
if let r = num(m["r"]) { stats.update(s) { $0.received = Int64(r) } }
|
||||
if controller.acked(seq: s, at: nowUs()) { resendIfRoom() }
|
||||
case "fd":
|
||||
for f in m["f"] as? [[NSNumber]] ?? [] where f.count >= 4 {
|
||||
stats.update(f[0].uint32Value) {
|
||||
$0.decoded = f[1].int64Value
|
||||
$0.drawn = f[2].int64Value
|
||||
$0.vsync = f[3].int64Value
|
||||
}
|
||||
}
|
||||
let dropped = Int(num(m["drop"]) ?? 0)
|
||||
stats.withLock { stats.viewerDropped += dropped }
|
||||
case "w":
|
||||
break // keep-warm filler, see sendInfo
|
||||
case "clock":
|
||||
stats.withLock {
|
||||
stats.rtt = num(m["rtt"]) ?? 0
|
||||
stats.clockSynced = true
|
||||
if let d = m["dec"] as? String { stats.decoder = d }
|
||||
}
|
||||
default:
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
/// Asks the viewer to do something for a benchmark (type, click, show its overlay).
|
||||
func bench(_ m: [String: Any]) { ws.sendJSON(m.merging(["t": "bench"]) { a, _ in a }) }
|
||||
|
||||
private func handle(_ text: String) {
|
||||
guard !isStopped, let d = text.data(using: .utf8),
|
||||
let m = try? JSONSerialization.jsonObject(with: d) as? [String: Any], let t = m["t"] as? String else { return }
|
||||
if report(t, m) { return }
|
||||
let x = m["x"] as? Double ?? -1, y = m["y"] as? Double ?? -1
|
||||
if t == "ack" {
|
||||
onAck?()
|
||||
onAck = nil
|
||||
return
|
||||
}
|
||||
if t == "key-frame" {
|
||||
encodeQueue.async { self.encoder.requestKeyFrame() } // before the resend, same queue
|
||||
lock.lock(); skipped = last != nil; lock.unlock()
|
||||
resendIfRoom()
|
||||
return
|
||||
}
|
||||
if source == .test {
|
||||
let desc: String?
|
||||
switch t {
|
||||
case "m": desc = (m["e"] as? String) == "move" ? nil : "mouse \(m["e"] ?? "") button \(m["b"] ?? 0)"
|
||||
case "wheel": desc = "wheel \(m["dx"] ?? 0), \(m["dy"] ?? 0)"
|
||||
case "k": desc = (m["e"] as? String) == "down" ? "key \(m["code"] ?? "") \"\(m["key"] ?? "")\"" : nil
|
||||
case "text": desc = "text \"\(m["s"] ?? "")\""
|
||||
default: desc = nil
|
||||
}
|
||||
capture.pointer(x: x, y: y, text: desc)
|
||||
if desc != nil { injected(m, kind: t) }
|
||||
return
|
||||
}
|
||||
DispatchQueue.main.async { [self] in
|
||||
guard !isStopped else { return } // Stop revokes input at once
|
||||
switch t {
|
||||
case "m":
|
||||
let kind = m["e"] as? String ?? "move", b = m["b"] as? Int ?? 0
|
||||
let p = point(x, y)
|
||||
if kind == "down", case .window(let wid) = source, let pid = capture.pid {
|
||||
Input.focus(window: wid, pid: pid)
|
||||
}
|
||||
Input.mouse(kind, button: b, at: p, owner: id)
|
||||
if kind == "down" { injected(m, kind: "click") }
|
||||
case "wheel":
|
||||
Input.scroll(dx: m["dx"] as? Double ?? 0, dy: m["dy"] as? Double ?? 0, at: point(x, y), owner: id)
|
||||
injected(m, kind: "wheel")
|
||||
case "k":
|
||||
let down = (m["e"] as? String) == "down"
|
||||
Input.key(code: m["code"] as? String ?? "", key: m["key"] as? String ?? "",
|
||||
down: down, mods: m["mods"] as? [String] ?? [], owner: id)
|
||||
if down { injected(m, kind: "key") }
|
||||
case "text":
|
||||
if let s = m["s"] as? String, s.count <= 4096 { Input.text(s); injected(m, kind: "text") }
|
||||
case "release":
|
||||
Input.releaseAll(owner: id)
|
||||
case "focus":
|
||||
if case .window(let wid) = source, let pid = capture.pid { Input.focus(window: wid, pid: pid) }
|
||||
default: break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func randomKey() -> String {
|
||||
var bytes = [UInt8](repeating: 0, count: 24)
|
||||
_ = SecRandomCopyBytes(kSecRandomDefault, bytes.count, &bytes)
|
||||
return Data(bytes).base64EncodedString().replacingOccurrences(of: "+", with: "-")
|
||||
.replacingOccurrences(of: "/", with: "_").replacingOccurrences(of: "=", with: "")
|
||||
}
|
||||
|
||||
func sameSecret(_ a: String, _ b: String) -> Bool {
|
||||
guard !a.isEmpty, a.utf8.count == b.utf8.count else { return false }
|
||||
return zip(a.utf8, b.utf8).reduce(0, { $0 | ($1.0 ^ $1.1) }) == 0
|
||||
}
|
||||
|
||||
final class Agent {
|
||||
/// Frame Control's own key. It stays on the Mac: the Frame only ever sees
|
||||
/// single-use tickets and per-stream reconnect keys, both tied to one source.
|
||||
let token: String
|
||||
let page: URL?
|
||||
var sessions: [Int: Session] = [:] { didSet { keepDisplayAwake(!sessions.isEmpty) } }
|
||||
var nextId = 1
|
||||
let lock = NSLock()
|
||||
private var assertion: IOPMAssertionID = 0
|
||||
/// ticket -> (source, expiry, reconnect key once redeemed). A ticket opens
|
||||
/// one viewer within a minute; it may be redeemed again, for the same key,
|
||||
/// only until that viewer confirms it has the key ("ack").
|
||||
private var tickets: [String: (src: String, expiry: Date, key: String?)] = [:]
|
||||
/// reconnect key -> source, until Stop for that source.
|
||||
private var reconnectKeys: [String: String] = [:]
|
||||
/// Sources that ended for good (the window closed), for Frame Control.
|
||||
private var finished = Set<String>()
|
||||
|
||||
/// Ends every stream, then exits once separated windows are back on the
|
||||
/// Mac's own screens (they're moved back, then their displays go 0.3 s later).
|
||||
/// Idempotent: SIGTERM and the parent going away can both call it, and a
|
||||
/// session already ended by /close may still be putting its window back.
|
||||
private var quitting = false
|
||||
func quit() {
|
||||
lock.lock()
|
||||
let all = Array(sessions.values), again = quitting
|
||||
quitting = true
|
||||
lock.unlock()
|
||||
guard !again else { return }
|
||||
for s in all { s.ws.sendJSON(["t": "close"]); s.end() }
|
||||
DispatchQueue.main.asyncAfter(deadline: .now() + 0.8) { exit(0) }
|
||||
}
|
||||
|
||||
/// While anyone watches, keep the Mac's display on: a sleeping display
|
||||
/// stops being drawn, so there'd be nothing to capture (and it would lock).
|
||||
private func keepDisplayAwake(_ on: Bool) {
|
||||
if on, assertion == 0 {
|
||||
IOPMAssertionCreateWithName(kIOPMAssertionTypePreventUserIdleDisplaySleep as CFString,
|
||||
IOPMAssertionLevel(kIOPMAssertionLevelOn),
|
||||
"Frame Control is showing this Mac in a Steam Frame" as CFString, &assertion)
|
||||
} else if !on, assertion != 0 {
|
||||
IOPMAssertionRelease(assertion)
|
||||
assertion = 0
|
||||
}
|
||||
}
|
||||
|
||||
init(token: String, page: URL?) {
|
||||
self.token = token
|
||||
self.page = page
|
||||
}
|
||||
|
||||
/// Whether this request may open a stream of `src`: Frame Control's key,
|
||||
/// an unused ticket for that source, or a live reconnect key for it.
|
||||
private func mayStream(_ req: Request, src: String) -> String? {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
let now = Date()
|
||||
tickets = tickets.filter { $0.value.expiry > now }
|
||||
if sameSecret(req.query["k"] ?? "", token) { return randomKey() }
|
||||
if let t = req.query["t"], let entry = tickets[t], entry.src == src {
|
||||
// A retry before the viewer got its key gets the same key back.
|
||||
let key = entry.key ?? randomKey()
|
||||
tickets[t] = (src, entry.expiry, key)
|
||||
reconnectKeys[key] = src
|
||||
return key
|
||||
}
|
||||
if let r = req.query["r"], reconnectKeys[r] == src { return r }
|
||||
return nil
|
||||
}
|
||||
|
||||
/// The viewer has its reconnect key, so the ticket that led to it can't
|
||||
/// be used again (also when the ack comes over a reconnection).
|
||||
func acknowledged(key: String) {
|
||||
lock.lock(); tickets = tickets.filter { $0.value.key != key }; lock.unlock()
|
||||
}
|
||||
|
||||
func handle(_ req: Request, _ c: HTTPConnection) {
|
||||
// Open to anything that reaches the port: a liveness check for the
|
||||
// tunnel, and the viewer page, which holds no secrets.
|
||||
switch (req.method, req.path) {
|
||||
case ("GET", "/ping"): return c.respond(200, text: "frame-mac-view")
|
||||
case ("GET", "/view"):
|
||||
guard let page, let body = try? Data(contentsOf: page) else { return c.respond(404, text: "no viewer page") }
|
||||
return c.respond(200, body: body, type: "text/html; charset=utf-8")
|
||||
case ("GET", "/stream"):
|
||||
guard let src = Source(req.query["src"] ?? "") else { return c.respond(400, text: "bad src") }
|
||||
guard let key = mayStream(req, src: src.key) else { return c.respond(403, text: "forbidden") }
|
||||
return stream(req, c, src: src, key: key)
|
||||
default: break
|
||||
}
|
||||
guard sameSecret(req.query["k"] ?? req.headers["x-token"] ?? "", token) else {
|
||||
return c.respond(403, text: "forbidden")
|
||||
}
|
||||
switch (req.method, req.path) {
|
||||
case ("GET", "/status"):
|
||||
lock.lock()
|
||||
let list = sessions.values.map { s -> [String: Any] in
|
||||
var e: [String: Any] = ["id": s.id, "src": s.source.key, "title": s.capture.title, "app": s.capture.app,
|
||||
"stats": s.stats.summary(), "bitrate": s.encoder.bitrate,
|
||||
"controller": s.controller.state()]
|
||||
if let d = (s.capture as? SeparateSource)?.displayID { e["display"] = d }
|
||||
return e
|
||||
}
|
||||
lock.unlock()
|
||||
var s = permissionsJSON()
|
||||
s["version"] = version
|
||||
s["streams"] = list
|
||||
lock.lock(); s["finished"] = Array(finished); lock.unlock()
|
||||
c.respond(json: s)
|
||||
case ("GET", "/windows"):
|
||||
c.respond(json: ["windows": windowsJSON(), "screen": CGPreflightScreenCaptureAccess()])
|
||||
case ("GET", "/displays"):
|
||||
c.respond(json: ["displays": displaysJSON()])
|
||||
case ("POST", "/ticket"):
|
||||
guard let src = Source(req.query["src"] ?? "") else { return c.respond(400, text: "bad src") }
|
||||
let t = randomKey()
|
||||
lock.lock()
|
||||
tickets[t] = (src.key, Date().addingTimeInterval(60), nil)
|
||||
finished.remove(src.key)
|
||||
lock.unlock()
|
||||
c.respond(json: ["ticket": t])
|
||||
case ("POST", "/close"):
|
||||
// Tell viewers to close their windows, but don't rely on them:
|
||||
// the sessions end here and can't reconnect.
|
||||
let src = req.query["src"]
|
||||
lock.lock()
|
||||
let matching = sessions.values.filter { src == nil || $0.source.key == src }
|
||||
reconnectKeys = reconnectKeys.filter { src != nil && $0.value != src }
|
||||
tickets = tickets.filter { src != nil && $0.value.src != src } // not yet used ones too
|
||||
lock.unlock()
|
||||
for s in matching { s.ws.sendJSON(["t": "close"]) }
|
||||
DispatchQueue.global().asyncAfter(deadline: .now() + 0.3) { for s in matching { s.end() } }
|
||||
c.respond(json: ["closed": matching.count])
|
||||
case ("GET", "/stats"):
|
||||
// Frames the viewer has had time to report on, oldest first.
|
||||
let since = UInt32(req.query["since"] ?? "") ?? 0
|
||||
let settle = Int64(req.query["settle"] ?? "") ?? 1_500_000
|
||||
lock.lock()
|
||||
let list = sessions.values.filter { req.query["id"] == nil || "\($0.id)" == req.query["id"] }
|
||||
lock.unlock()
|
||||
c.respond(json: ["now": nowUs(), "streams": list.map { s -> [String: Any] in
|
||||
["id": s.id, "src": s.source.key, "frames": s.stats.settled(since: since, settle: settle).map(\.json),
|
||||
"inputs": s.stats.inputList().map(\.json), "summary": s.stats.summary(),
|
||||
"captured": s.stats.withLock { s.stats.captured }, "controller": s.controller.state(),
|
||||
"events": s.controller.eventList()]
|
||||
}])
|
||||
case ("POST", "/bench"):
|
||||
lock.lock()
|
||||
let list = sessions.values.filter { $0.source.key == req.query["src"] }
|
||||
lock.unlock()
|
||||
var m: [String: Any] = [:]
|
||||
for (k, v) in req.query where k != "k" && k != "src" { m[k] = Double(v) ?? v as Any }
|
||||
for s in list { s.bench(m) }
|
||||
c.respond(json: ["sent": list.count])
|
||||
case ("GET", "/snapshot"):
|
||||
// One JPEG of a display (for checks and thumbnails): ?display=ID
|
||||
guard let id = UInt32(req.query["display"] ?? "") else { return c.respond(400, text: "display=ID") }
|
||||
snapshot(display: id) { data, error in
|
||||
if let data { c.respond(200, body: data, type: "image/jpeg") } else { c.respond(500, text: error ?? "failed") }
|
||||
}
|
||||
case ("POST", "/permissions"):
|
||||
DispatchQueue.main.async { requestPermissions() }
|
||||
c.respond(json: permissionsJSON())
|
||||
default:
|
||||
c.respond(404, text: "not found")
|
||||
}
|
||||
}
|
||||
|
||||
private func stream(_ req: Request, _ c: HTTPConnection, src: Source, key: String) {
|
||||
let codec = Codec(rawValue: req.query["codec"] ?? "h264") ?? .h264
|
||||
let maxLong = min(max(Int(req.query["max"] ?? "") ?? 1920, 320), 3840)
|
||||
let fps = min(max(Int(req.query["fps"] ?? "") ?? 60, 5), 120)
|
||||
let bpp = min(max(Double(req.query["bpp"] ?? "") ?? 0.1, 0.02), 0.5)
|
||||
guard let ws = c.upgrade(req) else { return }
|
||||
let capture: CaptureSource
|
||||
switch src {
|
||||
case .test: capture = TestSource()
|
||||
case .separate(let id): capture = SeparateSource(windowID: id)
|
||||
default: capture = SCKSource(src)
|
||||
}
|
||||
lock.lock()
|
||||
// One live viewer per key: a reconnection (or a second use of an
|
||||
// unacknowledged ticket) replaces the one before.
|
||||
let replaced = sessions.values.filter { $0.reconnectKey == key }
|
||||
let session = Session(id: nextId, source: src, capture: capture, ws: ws, codec: codec, fps: fps,
|
||||
bitsPerPixel: bpp, reconnectKey: key)
|
||||
nextId += 1
|
||||
sessions[session.id] = session
|
||||
lock.unlock()
|
||||
for old in replaced { old.end() }
|
||||
session.onEnd = { [weak self] s in
|
||||
guard let self else { return }
|
||||
self.lock.lock(); self.sessions[s.id] = nil; self.lock.unlock()
|
||||
}
|
||||
// The source is gone (its window closed): its viewers can't come back.
|
||||
session.onFinished = { [weak self] src in
|
||||
guard let self else { return }
|
||||
self.lock.lock()
|
||||
self.finished.insert(src)
|
||||
self.reconnectKeys = self.reconnectKeys.filter { $0.value != src }
|
||||
self.lock.unlock()
|
||||
}
|
||||
session.onAck = { [weak self] in self?.acknowledged(key: key) }
|
||||
session.start(maxLong: maxLong, fps: fps)
|
||||
}
|
||||
}
|
||||
|
||||
func snapshot(display id: CGDirectDisplayID, completion: @escaping (Data?, String?) -> Void) {
|
||||
SCShareableContent.getExcludingDesktopWindows(false, onScreenWindowsOnly: true) { content, error in
|
||||
guard let d = content?.displays.first(where: { $0.displayID == id }) else {
|
||||
return completion(nil, error?.localizedDescription ?? "no such display")
|
||||
}
|
||||
let filter = SCContentFilter(display: d, excludingWindows: [])
|
||||
let cfg = SCStreamConfiguration()
|
||||
cfg.width = Int(Double(d.width) * Double(filter.pointPixelScale))
|
||||
cfg.height = Int(Double(d.height) * Double(filter.pointPixelScale))
|
||||
cfg.showsCursor = true
|
||||
SCScreenshotManager.captureImage(contentFilter: filter, configuration: cfg) { image, error in
|
||||
guard let image else { return completion(nil, error?.localizedDescription ?? "no image") }
|
||||
let rep = NSBitmapImageRep(cgImage: image)
|
||||
completion(rep.representation(using: .jpeg, properties: [.compressionFactor: 0.85]), nil)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func requestPermissions() {
|
||||
if !CGPreflightScreenCaptureAccess() { CGRequestScreenCaptureAccess() }
|
||||
if !AXIsProcessTrusted() {
|
||||
AXIsProcessTrustedWithOptions([kAXTrustedCheckOptionPrompt.takeUnretainedValue() as String: true] as CFDictionary)
|
||||
}
|
||||
}
|
||||
|
||||
func argument(_ name: String, in args: [String]) -> String? {
|
||||
guard let i = args.firstIndex(of: name), i + 1 < args.count else { return nil }
|
||||
return args[i + 1]
|
||||
}
|
||||
|
||||
let args = Array(CommandLine.arguments.dropFirst())
|
||||
switch args.first {
|
||||
case "windows":
|
||||
printJSON(["windows": windowsJSON(), "screen": CGPreflightScreenCaptureAccess()])
|
||||
case "displays":
|
||||
printJSON(["displays": displaysJSON()])
|
||||
case "permissions":
|
||||
printJSON(permissionsJSON())
|
||||
case "request-permissions":
|
||||
requestPermissions()
|
||||
printJSON(permissionsJSON())
|
||||
case "serve":
|
||||
let port = UInt16(argument("--port", in: args) ?? "") ?? 0
|
||||
let token = ProcessInfo.processInfo.environment["FRAME_MAC_VIEW_TOKEN"] ?? ""
|
||||
guard !token.isEmpty else {
|
||||
FileHandle.standardError.write(Data("frame-mac-view: set FRAME_MAC_VIEW_TOKEN\n".utf8))
|
||||
exit(2)
|
||||
}
|
||||
let page = argument("--page", in: args).map { URL(fileURLWithPath: $0) }
|
||||
let agent = Agent(token: token, page: page)
|
||||
// Quit when the parent goes away (it holds our stdin open), or on SIGTERM,
|
||||
// but first end every stream, so separated windows go back where they
|
||||
// were before their displays disappear.
|
||||
if args.contains("--exit-on-eof") {
|
||||
DispatchQueue.global().async {
|
||||
while FileHandle.standardInput.availableData.count > 0 {}
|
||||
agent.quit()
|
||||
}
|
||||
}
|
||||
signal(SIGTERM, SIG_IGN)
|
||||
let sigterm = DispatchSource.makeSignalSource(signal: SIGTERM, queue: .main)
|
||||
sigterm.setEventHandler { agent.quit() }
|
||||
sigterm.resume()
|
||||
let server: Server
|
||||
do {
|
||||
server = try Server(port: port) { req, c in agent.handle(req, c) }
|
||||
} catch {
|
||||
FileHandle.standardError.write(Data("frame-mac-view: \(error)\n".utf8))
|
||||
exit(1)
|
||||
}
|
||||
server.start { [server] error in
|
||||
if let error {
|
||||
FileHandle.standardError.write(Data("frame-mac-view: can't listen on 127.0.0.1:\(port): \(error)\n".utf8))
|
||||
exit(1)
|
||||
}
|
||||
print("frame-mac-view listening on 127.0.0.1:\(server.port ?? port)")
|
||||
fflush(stdout)
|
||||
}
|
||||
// A real (background, no Dock icon) AppKit event loop, not just a run
|
||||
// loop: without it this process never hears that displays were added or
|
||||
// changed mode, so NSScreen and CGDisplayCopyDisplayMode stay stale for
|
||||
// the displays Separate mode creates.
|
||||
let app = NSApplication.shared
|
||||
app.setActivationPolicy(.prohibited)
|
||||
withExtendedLifetime((server, sigterm)) { app.run() }
|
||||
default:
|
||||
FileHandle.standardError.write(Data("usage: frame-mac-view serve|windows|displays|permissions|request-permissions\n".utf8))
|
||||
exit(2)
|
||||
}
|
||||
Executable
+11
@@ -0,0 +1,11 @@
|
||||
#!/bin/sh
|
||||
# Builds the Mac streaming agent into mac/bin/frame-mac-view (arm64, macOS 14+).
|
||||
# Frame Control runs it for "Mac in the headset"; the Electron app bundles it.
|
||||
set -eu
|
||||
here=$(cd "$(dirname "$0")" && pwd)
|
||||
out=${1:-$here/../bin/frame-mac-view}
|
||||
mkdir -p "$(dirname "$out")"
|
||||
xcrun swiftc -O -swift-version 5 -target arm64-apple-macos14.0 \
|
||||
-import-objc-header "$here/Sources/CGVirtualDisplay.h" \
|
||||
-o "$out" "$here"/Sources/*.swift
|
||||
echo "built $out"
|
||||
Executable
+50
@@ -0,0 +1,50 @@
|
||||
#!/bin/sh
|
||||
# Wraps the agent in "Frame Mac View Lab.app" for testing from a terminal or
|
||||
# an agent session: macOS then asks for Screen Recording and Accessibility
|
||||
# for this app rather than for the terminal. Signed with an Apple Development
|
||||
# identity if there is one, so the permissions survive rebuilds.
|
||||
# mac/frame-mac-view/lab.sh build the app into mac/bin/
|
||||
# mac/frame-mac-view/lab.sh serve also start it (token, port and log in ~/Library/Caches/frame-mac-view-lab,
|
||||
# readable only by you: the token opens every window and injects input)
|
||||
# mac/frame-mac-view/lab.sh serve-only restart it without rebuilding
|
||||
set -eu
|
||||
here=$(cd "$(dirname "$0")" && pwd)
|
||||
app="$here/../bin/Frame Mac View Lab.app"
|
||||
if [ "${1:-}" != serve-only ]; then
|
||||
mkdir -p "$app/Contents/MacOS"
|
||||
sh "$here/build.sh" "$app/Contents/MacOS/frame-mac-view" >/dev/null
|
||||
cat > "$app/Contents/Info.plist" <<PLIST
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0"><dict>
|
||||
<key>CFBundleIdentifier</key><string>com.saphid.frame-mac-view-lab</string>
|
||||
<key>CFBundleExecutable</key><string>frame-mac-view</string>
|
||||
<key>CFBundleName</key><string>Frame Mac View Lab</string>
|
||||
<key>CFBundlePackageType</key><string>APPL</string>
|
||||
<key>CFBundleShortVersionString</key><string>1</string>
|
||||
<key>LSUIElement</key><true/>
|
||||
<key>NSScreenCaptureUsageDescription</key><string>Tests streaming Mac windows into the Steam Frame.</string>
|
||||
</dict></plist>
|
||||
PLIST
|
||||
id=$(security find-identity -v -p codesigning 2>/dev/null | awk '/Apple Development/ {print $2; exit}')
|
||||
codesign -f -s "${id:--}" "$app" >/dev/null 2>&1
|
||||
echo "built $app (signed ${id:-ad hoc})"
|
||||
fi
|
||||
if [ "${1:-}" = serve ] || [ "${1:-}" = serve-only ]; then
|
||||
pkill -f "Frame Mac View Lab.app/Contents/MacOS/frame-mac-view" 2>/dev/null || true
|
||||
dir="$HOME/Library/Caches/frame-mac-view-lab"
|
||||
mkdir -p "$dir" && chmod 700 "$dir"
|
||||
umask 077
|
||||
token=$(openssl rand -hex 16)
|
||||
rm -f "$dir/token" "$dir/port" "$dir/log"
|
||||
echo "$token" > "$dir/token"
|
||||
: > "$dir/log"
|
||||
# Experiment switches pass through: FRAME_MAC_VIEW_ENCODER (Encoder.swift),
|
||||
# _VD_HZ (Separate.swift), _ADAPT (Controller.swift), _WARM (main.swift).
|
||||
open -n "$app" --env "FRAME_MAC_VIEW_TOKEN=$token" --env "FRAME_MAC_VIEW_ENCODER=${FRAME_MAC_VIEW_ENCODER:-}" --env "FRAME_MAC_VIEW_VD_HZ=${FRAME_MAC_VIEW_VD_HZ:-}" \
|
||||
--env "FRAME_MAC_VIEW_ADAPT=${FRAME_MAC_VIEW_ADAPT:-}" --env "FRAME_MAC_VIEW_WARM=${FRAME_MAC_VIEW_WARM:-}" --stdout "$dir/log" --stderr "$dir/log" \
|
||||
--args serve --port 0 --page "$here/../../ui/mac-view.html"
|
||||
for _ in 1 2 3 4 5 6 7 8 9 10; do grep -q listening "$dir/log" && break; sleep 0.3; done
|
||||
sed -n 's/.*127\.0\.0\.1:\([0-9]*\).*/\1/p' "$dir/log" | head -n 1 > "$dir/port"
|
||||
echo "lab on 127.0.0.1:$(cat "$dir/port")"
|
||||
fi
|
||||
@@ -0,0 +1,100 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Open Frame Control's assistant as a Chromium panel. Ctrl-C closes it and its SSH tunnel.
|
||||
|
||||
Start ui/server.py first. Requires the platform Chromium Flatpak and zsh on the
|
||||
computer (the existing panel launcher). No model endpoint or key is configured.
|
||||
"""
|
||||
import argparse
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shlex
|
||||
import signal
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import uuid
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument('--port', type=int, default=47810, help='local Frame Control port')
|
||||
parser.add_argument('--frame-port', type=int, default=47812, help='Frame loopback tunnel port')
|
||||
args = parser.parse_args()
|
||||
alias = os.environ.get('FRAME_ALIAS', 'frame')
|
||||
if not re.fullmatch(r'[A-Za-z0-9][A-Za-z0-9._-]*', alias) or any(not 1 <= p <= 65535 for p in (args.port, args.frame_port)):
|
||||
parser.error('Invalid alias or port')
|
||||
if not shutil.which('zsh'):
|
||||
parser.error('The panel launcher requires zsh on this computer')
|
||||
sys.path.insert(0, str(ROOT / 'ui'))
|
||||
from frame_mcp import Client
|
||||
Client('http://127.0.0.1:' + str(args.port), os.environ.get('FRAME_UI_KEY', '1')).request('/api/host')
|
||||
profile = '/tmp/frame-control-assistant-' + uuid.uuid4().hex
|
||||
log_path = ''
|
||||
signal.signal(signal.SIGTERM, lambda *_: (_ for _ in ()).throw(KeyboardInterrupt))
|
||||
tunnel = subprocess.Popen(['ssh', '-N', '-o', 'BatchMode=yes', '-o', 'ConnectTimeout=8',
|
||||
'-o', 'ExitOnForwardFailure=yes', '-o', 'ServerAliveInterval=15',
|
||||
'-o', 'ServerAliveCountMax=2', '-R',
|
||||
f'127.0.0.1:{args.frame_port}:127.0.0.1:{args.port}', alias])
|
||||
try:
|
||||
# Check the forwarded page before starting a browser; no arbitrary sleeps.
|
||||
probe = subprocess.run(['ssh', '-o', 'BatchMode=yes', '-o', 'ConnectTimeout=8', alias,
|
||||
'curl --retry 5 --retry-connrefused --retry-delay 1 --max-time 10 -fsS ' +
|
||||
shlex.quote(f'http://127.0.0.1:{args.frame_port}/assistant')],
|
||||
stdout=subprocess.DEVNULL, timeout=30)
|
||||
if probe.returncode or tunnel.poll() is not None:
|
||||
raise RuntimeError('Could not forward Frame Control to the Frame')
|
||||
launched = subprocess.run(['zsh', str(ROOT / 'scripts/panel-on-frame.sh'), '--name', 'Frame Control Assistant',
|
||||
'org.chromium.Chromium', '--user-data-dir=' + profile, '--no-first-run',
|
||||
'--disable-background-networking', '--disable-sync',
|
||||
f'--app=http://127.0.0.1:{args.frame_port}/assistant'], check=True, timeout=45, stdout=subprocess.PIPE, text=True)
|
||||
print(launched.stdout, end='', flush=True)
|
||||
match = re.search(r'log (/tmp/panel-on-frame\.[A-Za-z0-9]+)', launched.stdout)
|
||||
if match:
|
||||
log_path = match.group(1)
|
||||
print('Assistant panel open. Ctrl-C closes this panel and its tunnel.', flush=True)
|
||||
tunnel.wait()
|
||||
raise RuntimeError('SSH tunnel ended')
|
||||
except KeyboardInterrupt:
|
||||
return 0
|
||||
finally:
|
||||
tunnel.terminate()
|
||||
try:
|
||||
tunnel.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
tunnel.kill()
|
||||
tunnel.wait()
|
||||
# Only this unique browser profile, never a shared Chromium instance.
|
||||
cleanup = '''import os, pathlib, signal, shutil, sys, time
|
||||
profile = sys.argv[1]
|
||||
needle = ('--user-data-dir=' + profile).encode()
|
||||
owned = []
|
||||
for p in pathlib.Path('/proc').iterdir():
|
||||
try:
|
||||
if p.name.isdigit() and p.stat().st_uid == os.getuid() and needle in (p / 'cmdline').read_bytes().split(b'\\0'):
|
||||
owned.append(int(p.name))
|
||||
except OSError:
|
||||
pass
|
||||
for sig in (signal.SIGTERM, signal.SIGKILL):
|
||||
for pid in owned:
|
||||
try: os.kill(pid, sig)
|
||||
except ProcessLookupError: pass
|
||||
time.sleep(.3)
|
||||
shutil.rmtree(profile, ignore_errors=True)
|
||||
if sys.argv[2]:
|
||||
pathlib.Path(sys.argv[2]).unlink(missing_ok=True)
|
||||
'''
|
||||
result = subprocess.run(['ssh', '-o', 'BatchMode=yes', '-o', 'ConnectTimeout=8', alias,
|
||||
'python3 - ' + shlex.quote(profile) + ' ' + shlex.quote(log_path)], input=cleanup, text=True, timeout=20)
|
||||
if result.returncode:
|
||||
print('Cleanup failed; close the assistant panel and remove ' + profile + ' on the Frame.', file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
try:
|
||||
sys.exit(main())
|
||||
except (OSError, RuntimeError, subprocess.SubprocessError) as exc:
|
||||
print(str(exc), file=sys.stderr)
|
||||
sys.exit(1)
|
||||
@@ -1,135 +0,0 @@
|
||||
#!/bin/bash
|
||||
# Linux-side (x64 host): cross-compile arm64 Chromium with the Linux OpenXR CLs
|
||||
# (8441736 + 8132979, bug 506004811), plus a one-option seccomp fix, so WebXR
|
||||
# immersive-vr works on the Frame.
|
||||
# Needs ~90 GB free, no sudo. Takes hours; run it detached on the build host:
|
||||
# scp scripts/build-chromium-xr.sh buildhost:chromium-xr/build.sh
|
||||
# ssh buildhost 'cd ~/chromium-xr && tmux new -d -s chromium-xr "./build.sh > build.log 2>&1"'
|
||||
# Progress: ~/chromium-xr/stage. Output: ~/chromium-xr/chromium-xr-arm64.tar.xz,
|
||||
# which scripts/chromium-xr.sh install copies to the Frame.
|
||||
# Re-running resumes: existing checkout and out/XR are reused.
|
||||
set -euo pipefail
|
||||
W=~/chromium-xr
|
||||
cd "$W"
|
||||
stage(){ echo "$(date -Is) $*" | tee -a "$W/stage"; }
|
||||
# Returns non-zero below 12 GB free; set -e turns that into an exit at top level.
|
||||
guard(){ avail=$(df --output=avail -BG "$W" | tail -n 1 | tr -dc 0-9); if [ "$avail" -lt 12 ]; then stage "ABORT: only ${avail}G free for $W"; return 3; fi; }
|
||||
[ -d depot_tools ] || git clone -q https://chromium.googlesource.com/chromium/tools/depot_tools.git
|
||||
export PATH="$W/depot_tools:$PATH" DEPOT_TOOLS_UPDATE=1 DEPOT_TOOLS_METRICS=0
|
||||
CL_REF=refs/changes/79/8132979/44
|
||||
if [ ! -f .gclient ]; then
|
||||
cat > .gclient <<'G'
|
||||
solutions = [{ "name": "src", "url": "https://chromium.googlesource.com/chromium/src.git",
|
||||
"managed": False, "custom_deps": {}, "custom_vars": { "checkout_nacl": False } }]
|
||||
target_os = ["linux"]
|
||||
target_cpu = ["arm64"]
|
||||
G
|
||||
fi
|
||||
# Keyed on a real commit, so an interrupted first fetch is retried on re-run.
|
||||
if ! git -C src rev-parse -q --verify HEAD >/dev/null 2>&1; then
|
||||
stage "clone src at $CL_REF"
|
||||
mkdir -p src
|
||||
[ -d src/.git ] || git -C src init -q
|
||||
git -C src remote get-url origin >/dev/null 2>&1 || git -C src remote add origin https://chromium.googlesource.com/chromium/src.git
|
||||
git -C src fetch -q --depth=1 origin "$CL_REF"
|
||||
git -C src checkout -q FETCH_HEAD
|
||||
fi
|
||||
guard
|
||||
stage "src at $(git -C src log -1 --format='%h %s')"
|
||||
rev=$(git -C src rev-parse HEAD)
|
||||
# Sync once per revision: once the patch below is applied, gclient sync
|
||||
# refuses to run on the modified checkout, so re-runs must skip it.
|
||||
if [ "$(cat "$W/synced" 2>/dev/null)" != "$rev" ]; then
|
||||
stage "gclient sync"
|
||||
gclient sync --nohooks --no-history -D --shallow --revision "src@$rev" -j 8
|
||||
guard
|
||||
stage "runhooks"
|
||||
gclient runhooks
|
||||
src/build/linux/sysroot_scripts/install-sysroot.py --arch=arm64
|
||||
echo "$rev" > "$W/synced"
|
||||
fi
|
||||
guard
|
||||
cd src
|
||||
# CL 8441736's XR seccomp policy refuses getsockopt, and SteamVR's IPC client
|
||||
# calls getsockopt(SO_PEERCRED) inside xrCreateInstance, which crashes the XR
|
||||
# process (verified on the Frame 2026-09-26). Allow only that option.
|
||||
IFS= read -r -d '' PEERCRED_PATCH <<'P' || true
|
||||
diff --git a/sandbox/policy/linux/bpf_xr_policy_linux.cc b/sandbox/policy/linux/bpf_xr_policy_linux.cc
|
||||
index 435e13d396..297453f582 100644
|
||||
--- a/sandbox/policy/linux/bpf_xr_policy_linux.cc
|
||||
+++ b/sandbox/policy/linux/bpf_xr_policy_linux.cc
|
||||
@@ -11,6 +11,7 @@
|
||||
#include "sandbox/linux/system_headers/linux_syscalls.h"
|
||||
#include "sandbox/policy/linux/sandbox_linux.h"
|
||||
|
||||
+using sandbox::bpf_dsl::AllOf;
|
||||
using sandbox::bpf_dsl::Allow;
|
||||
using sandbox::bpf_dsl::Arg;
|
||||
using sandbox::bpf_dsl::Error;
|
||||
@@ -27,8 +28,8 @@ XrProcessPolicy::~XrProcessPolicy() = default;
|
||||
ResultExpr XrProcessPolicy::EvaluateSyscall(int system_call_number) const {
|
||||
switch (system_call_number) {
|
||||
// The runtime reaches its compositor over an AF_UNIX socket and passes fds
|
||||
- // with SCM_RIGHTS, neither of which the GPU policy allows. get/setsockopt
|
||||
- // stay disallowed; add a narrow level/optname restriction if ever needed.
|
||||
+ // with SCM_RIGHTS, neither of which the GPU policy allows. setsockopt
|
||||
+ // stays disallowed; getsockopt is limited to SO_PEERCRED below.
|
||||
#if defined(__NR_getpeername)
|
||||
case __NR_getpeername:
|
||||
#endif
|
||||
@@ -49,6 +50,16 @@ ResultExpr XrProcessPolicy::EvaluateSyscall(int system_call_number) const {
|
||||
case __NR_get_robust_list:
|
||||
#endif
|
||||
return Allow();
|
||||
+#if defined(__NR_getsockopt)
|
||||
+ case __NR_getsockopt: {
|
||||
+ // SteamVR's IPC client checks who is on the other end of its socket
|
||||
+ // with SO_PEERCRED. Nothing else is readable.
|
||||
+ const Arg<int> level(1);
|
||||
+ const Arg<int> optname(2);
|
||||
+ return If(AllOf(level == SOL_SOCKET, optname == SO_PEERCRED), Allow())
|
||||
+ .Else(Error(EPERM));
|
||||
+ }
|
||||
+#endif
|
||||
#if defined(__NR_kill)
|
||||
case __NR_kill: {
|
||||
// SteamVR probes its sibling processes for liveness with kill(pid, 0).
|
||||
P
|
||||
if ! printf '%s\n' "$PEERCRED_PATCH" | git apply --reverse --check 2>/dev/null; then
|
||||
printf '%s\n' "$PEERCRED_PATCH" | git apply
|
||||
stage "applied SO_PEERCRED patch"
|
||||
fi
|
||||
mkdir -p out/XR
|
||||
cat > out/XR/args.gn <<'A'
|
||||
target_os = "linux"
|
||||
target_cpu = "arm64"
|
||||
is_debug = false
|
||||
is_official_build = false
|
||||
is_component_build = false
|
||||
dcheck_always_on = false
|
||||
symbol_level = 0
|
||||
blink_symbol_level = 0
|
||||
v8_symbol_level = 0
|
||||
proprietary_codecs = true
|
||||
ffmpeg_branding = "Chrome"
|
||||
use_remoteexec = false
|
||||
use_siso = true
|
||||
treat_warnings_as_errors = false
|
||||
A
|
||||
stage "gn gen"
|
||||
gn gen out/XR
|
||||
gn args out/XR --list=enable_openxr --short | tee -a "$W/stage"
|
||||
stage "build"
|
||||
( while sleep 600; do guard || { pkill -u "$(id -u)" -f "siso|ninja"; exit 3; }; done ) &
|
||||
GUARD=$!
|
||||
trap 'kill $GUARD 2>/dev/null || true' EXIT
|
||||
autoninja -C out/XR chrome chrome_sandbox chrome_crashpad_handler
|
||||
stage "package"
|
||||
cd out/XR
|
||||
files=(chrome chrome_sandbox chrome_crashpad_handler *.pak *.bin icudtl.dat locales)
|
||||
# GPU libraries aren't produced by every config; pack the ones that exist.
|
||||
for f in libEGL.so libGLESv2.so libvk_swiftshader.so libvulkan.so.1 vk_swiftshader_icd.json; do
|
||||
[ -e "$f" ] && files+=("$f")
|
||||
done
|
||||
tar -cJf "$W/chromium-xr-arm64.tar.xz" "${files[@]}"
|
||||
stage "DONE $(ls -la $W/chromium-xr-arm64.tar.xz)"
|
||||
@@ -1,140 +0,0 @@
|
||||
#!/usr/bin/env zsh
|
||||
# Mac-side: install and launch the WebXR-enabled Chromium build on the Frame.
|
||||
#
|
||||
# Flathub Chromium can't enter immersive WebXR on Linux: upstream only wires
|
||||
# the OpenXR device on Windows (see docs/webxr-chromium.md). This deploys an
|
||||
# arm64 build with the Linux OpenXR CLs, made on a Linux host by
|
||||
# scripts/build-chromium-xr.sh, into ~/chromium-xr on the Frame (not a
|
||||
# Flatpak, so SteamVR's sockets and the XR sandbox work unmodified).
|
||||
#
|
||||
# Usage:
|
||||
# scripts/chromium-xr.sh install [TARBALL] # default: scp from $BUILD_HOST
|
||||
# scripts/chromium-xr.sh launch [URL] # opens as its own panel in the headset
|
||||
# scripts/chromium-xr.sh steam # adds "Chromium XR" to the Steam library
|
||||
# scripts/chromium-xr.sh check # isSessionSupported via DevTools
|
||||
set -euo pipefail
|
||||
|
||||
FRAME_ALIAS=${FRAME_ALIAS:-frame}
|
||||
BUILD_HOST=${BUILD_HOST:-}
|
||||
BUILD_TARBALL=${BUILD_TARBALL:-chromium-xr/chromium-xr-arm64.tar.xz}
|
||||
DEVTOOLS_PORT=${DEVTOOLS_PORT:-9223}
|
||||
STEAM_NAME=${STEAM_NAME:-Chromium XR}
|
||||
here=${0:A:h}
|
||||
wrapper='~/Applications/ChromiumXR/launch.sh'
|
||||
|
||||
# frame/chromium-xr/launch.sh holds Chrome's flags; both launch paths run it.
|
||||
push_wrapper() {
|
||||
# Write then rename, so a dropped connection can't leave a torn script.
|
||||
ssh "$FRAME_ALIAS" 'mkdir -p ~/Applications/ChromiumXR && cd ~/Applications/ChromiumXR && cat > launch.sh.new && chmod +x launch.sh.new && mv launch.sh.new launch.sh' \
|
||||
< "$here/../frame/chromium-xr/launch.sh"
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
install)
|
||||
tarball=${2:-}
|
||||
if [[ -z "$tarball" ]]; then
|
||||
[[ -n "$BUILD_HOST" ]] || { print -u2 "Pass a tarball, or set BUILD_HOST to the build machine"; exit 2; }
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
tarball=$tmp/chromium-xr-arm64.tar.xz
|
||||
scp -q "$BUILD_HOST:$BUILD_TARBALL" "$tarball"
|
||||
fi
|
||||
ssh "$FRAME_ALIAS" 'rm -rf ~/chromium-xr.new && mkdir -p ~/chromium-xr.new'
|
||||
ssh "$FRAME_ALIAS" 'tar -xJf - -C ~/chromium-xr.new' < "$tarball"
|
||||
# Check the new build runs before replacing the old one.
|
||||
ssh "$FRAME_ALIAS" '~/chromium-xr.new/chrome --version && rm -rf ~/chromium-xr && mv ~/chromium-xr.new ~/chromium-xr'
|
||||
;;
|
||||
launch)
|
||||
# Its own VR panel on gamescope's X display, so the Plasma desktop doesn't
|
||||
# need to be open. DevTools is only on for this path (for `check`).
|
||||
push_wrapper
|
||||
exec "$here/panel-on-frame.sh" --name chromium-xr -- "$wrapper" \
|
||||
--remote-debugging-port="$DEVTOOLS_PORT" \
|
||||
"${2:-https://immersive-web.github.io/webxr-samples/}"
|
||||
;;
|
||||
steam)
|
||||
# A non-Steam shortcut, added through the Steam client's DevTools port
|
||||
# without restarting Steam (see docs/apks.md). Launching it from the
|
||||
# library gives Chromium its own panel like any game. Safe to rerun: it
|
||||
# refreshes the wrapper and only adds the shortcut if it's missing.
|
||||
ssh "$FRAME_ALIAS" 'test -x ~/chromium-xr/chrome' ||
|
||||
{ print -u2 "No build in ~/chromium-xr on the Frame: run 'chromium-xr.sh install' first"; exit 1; }
|
||||
push_wrapper
|
||||
# The app id is kept next to the wrapper, so renaming the shortcut in the
|
||||
# library doesn't make a rerun add a second one.
|
||||
shortcuts=$here/../frame/android/steam_shortcuts.py
|
||||
home=$(ssh "$FRAME_ALIAS" 'printf %s "$HOME"')
|
||||
saved=$(ssh "$FRAME_ALIAS" 'cat ~/Applications/ChromiumXR/shortcut-appid 2>/dev/null || true')
|
||||
existing=$(ssh "$FRAME_ALIAS" python3 - list < "$shortcuts" |
|
||||
python3 -c 'import json,sys
|
||||
apps = json.load(sys.stdin)
|
||||
ids = [a["appid"] for a in apps if str(a["appid"]) == sys.argv[2]] or [a["appid"] for a in apps if a["name"] == sys.argv[1]]
|
||||
print(ids[0] if ids else "")' "$STEAM_NAME" "$saved")
|
||||
if [[ -n "$existing" ]]; then
|
||||
print -r -- "The shortcut is already in the Steam library (app id $existing)"
|
||||
else
|
||||
icon=''
|
||||
for dir in /var/lib/flatpak '~/.local/share/flatpak'; do
|
||||
candidate=$dir/exports/share/icons/hicolor/256x256/apps/org.chromium.Chromium.png
|
||||
if ssh "$FRAME_ALIAS" "test -f $candidate"; then icon=${candidate/#\~/$home}; break; fi
|
||||
done
|
||||
existing=$(ssh "$FRAME_ALIAS" python3 - add ${(q)STEAM_NAME} ${(q)home}/Applications/ChromiumXR/launch.sh ${(q)home} ${(q)icon} < "$shortcuts")
|
||||
[[ "$existing" == <-> ]] || { print -u2 -r -- "Steam didn't return a shortcut app id: $existing"; exit 1; }
|
||||
print -r -- "Added $STEAM_NAME to the Steam library (shortcut app id $existing)"
|
||||
fi
|
||||
ssh "$FRAME_ALIAS" "printf '%s\n' $existing > ~/Applications/ChromiumXR/shortcut-appid"
|
||||
;;
|
||||
check)
|
||||
# DevTools listens on the Frame's loopback only; evaluate there.
|
||||
ssh "$FRAME_ALIAS" python3 - "$DEVTOOLS_PORT" <<'EOF'
|
||||
import json, sys, urllib.request, base64, os, socket, struct
|
||||
port = int(sys.argv[1])
|
||||
tabs = json.load(urllib.request.urlopen(f"http://127.0.0.1:{port}/json", timeout=10))
|
||||
page = next((t for t in tabs if t["type"] == "page"), None)
|
||||
if page is None:
|
||||
sys.exit("no open page: run 'chromium-xr.sh launch' first")
|
||||
path = page["webSocketDebuggerUrl"].split(f":{port}", 1)[1]
|
||||
s = socket.create_connection(("127.0.0.1", port), timeout=30)
|
||||
key = base64.b64encode(os.urandom(16)).decode()
|
||||
s.sendall(f"GET {path} HTTP/1.1\r\nHost: 127.0.0.1\r\nUpgrade: websocket\r\n"
|
||||
f"Connection: Upgrade\r\nSec-WebSocket-Key: {key}\r\n"
|
||||
"Sec-WebSocket-Version: 13\r\n\r\n".encode())
|
||||
s.recv(4096)
|
||||
msg = json.dumps({"id": 1, "method": "Runtime.evaluate", "params": {
|
||||
"expression": "navigator.xr ? navigator.xr.isSessionSupported('immersive-vr') : 'no navigator.xr'",
|
||||
"awaitPromise": True}}).encode()
|
||||
mask = os.urandom(4)
|
||||
hdr = bytes([0x81]) + (bytes([0x80 | len(msg)]) if len(msg) < 126
|
||||
else bytes([0x80 | 126]) + struct.pack(">H", len(msg)))
|
||||
s.sendall(hdr + mask + bytes(b ^ mask[i % 4] for i, b in enumerate(msg)))
|
||||
buf = b""
|
||||
reply = None
|
||||
while reply is None:
|
||||
chunk = s.recv(65536)
|
||||
if not chunk:
|
||||
sys.exit("DevTools closed the connection")
|
||||
buf += chunk
|
||||
# Consume every complete frame already buffered before reading again.
|
||||
while len(buf) >= 2:
|
||||
n = buf[1] & 0x7F
|
||||
off = 2
|
||||
if n == 126:
|
||||
if len(buf) < 4:
|
||||
break
|
||||
n, off = struct.unpack(">H", buf[2:4])[0], 4
|
||||
elif n == 127:
|
||||
if len(buf) < 10:
|
||||
break
|
||||
n, off = struct.unpack(">Q", buf[2:10])[0], 10
|
||||
if len(buf) < off + n:
|
||||
break
|
||||
frame, buf = buf[off:off + n], buf[off + n:]
|
||||
msg = json.loads(frame)
|
||||
if msg.get("id") == 1:
|
||||
reply = msg
|
||||
break
|
||||
print("immersive-vr supported:", reply["result"]["result"].get("value"))
|
||||
EOF
|
||||
;;
|
||||
*) sed -n '2,14p' "$0"; exit 2 ;;
|
||||
esac
|
||||
@@ -59,9 +59,13 @@ colordepth=32
|
||||
quality=9
|
||||
viewonly=0
|
||||
showcursor=1
|
||||
scale=1
|
||||
viewmode=1
|
||||
window_maximize=1
|
||||
EOF
|
||||
echo \"wrote \$d/mac-screen-sharing.remmina\"
|
||||
"
|
||||
print "On the Mac: System Settings > General > Sharing > Screen Sharing (i) >"
|
||||
print " enable 'VNC viewers may control screen with password' and set one."
|
||||
print "Remmina may ask for your Mac account name + login password instead (Apple auth)."
|
||||
fi
|
||||
Executable
+74
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env zsh
|
||||
# Mac-side: stop the Steam Frame from going to sleep while an agent works on it.
|
||||
#
|
||||
# The Frame sleeps when Steam's own idle timer runs out ("Sleep after
|
||||
# inactivity": 60 min on AC, 15 min on battery by default). SSH activity
|
||||
# doesn't count as input, and asleep the Frame is off the network. `on` sets
|
||||
# both timers to Never through Steam's UI (DevTools on 127.0.0.1:8080, via
|
||||
# ui/frame_steam.py) and holds a logind sleep inhibitor as a user unit.
|
||||
# `off` drops the inhibitor and restores the timers `on` saved.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/keep-awake.sh on
|
||||
# scripts/keep-awake.sh off
|
||||
# scripts/keep-awake.sh status
|
||||
set -euo pipefail
|
||||
|
||||
FRAME_ALIAS=${FRAME_ALIAS:-frame}
|
||||
HERE=${0:A:h}
|
||||
cmd=${1:-status}
|
||||
case $cmd in on|off|status) ;; *) echo "usage: keep-awake.sh on|off|status" >&2; exit 2 ;; esac
|
||||
|
||||
ssh -o ConnectTimeout=8 "$FRAME_ALIAS" \
|
||||
'mkdir -p ~/.cache/frame-control && cat > ~/.cache/frame-control/frame_steam.py' < "$HERE/../ui/frame_steam.py"
|
||||
|
||||
# Runs on the Frame. Verified 2026-09-28 (BUILD_ID 20260925.6191901): the
|
||||
# timers are client settings system_idle_suspend_{ac,battery}_sec (0 = Never),
|
||||
# written the way Steam's settings page does (steamui module exporting the
|
||||
# SetSetting wrapper). logind refuses an inhibitor from an SSH session
|
||||
# ("Interactive authentication required") but allows one from a user unit.
|
||||
ssh "$FRAME_ALIAS" python3 - "$cmd" <<'EOF'
|
||||
import json, os, subprocess, sys
|
||||
sys.path.insert(0, os.path.expanduser("~/.cache/frame-control"))
|
||||
from frame_steam import Page
|
||||
|
||||
cmd = sys.argv[1]
|
||||
saved_path = os.path.expanduser("~/.cache/frame-control/keep-awake.json")
|
||||
unit = "fc-keep-awake"
|
||||
keys = ("system_idle_suspend_ac_sec", "system_idle_suspend_battery_sec")
|
||||
|
||||
if cmd == "off": # release the lock first, even if Steam's UI is down
|
||||
subprocess.run(["systemctl", "--user", "stop", unit], stderr=subprocess.DEVNULL)
|
||||
page = Page()
|
||||
def read():
|
||||
return {k: page.eval(f"settingsStore.clientSettings.{k}") for k in keys}
|
||||
def write(values):
|
||||
page.eval("""(async () => { let req;
|
||||
webpackChunksteamui.push([[Symbol()], {}, r => { req = r }]);
|
||||
const mod = Object.keys(req.m).map(id => req.m[id].toString().includes("Settings.SetSetting") ? req(id) : null).find(Boolean);
|
||||
const set = Object.values(mod).find(f => typeof f == "function" && f.toString().includes("SetSetting("));
|
||||
for (const [k, v] of Object.entries(%s)) await set(k, v);
|
||||
await new Promise(r => setTimeout(r, 1000)); })()""" % json.dumps(values))
|
||||
def inhibitor():
|
||||
return subprocess.run(["systemctl", "--user", "is-active", "-q", unit]).returncode == 0
|
||||
|
||||
if cmd == "on":
|
||||
current = read()
|
||||
if not os.path.exists(saved_path):
|
||||
with open(saved_path, "w") as f:
|
||||
json.dump(current, f)
|
||||
write({k: 0 for k in keys})
|
||||
if not inhibitor():
|
||||
subprocess.run(["systemd-run", "--user", "-q", f"--unit={unit}",
|
||||
"--description=Frame Control: keep the Frame awake",
|
||||
"systemd-inhibit", "--what=sleep:idle:handle-suspend-key:handle-power-key",
|
||||
"--who=Frame Control", "--why=Keep the Frame awake while an agent works on it",
|
||||
"--mode=block", "sleep", "infinity"], check=True)
|
||||
elif cmd == "off":
|
||||
if os.path.exists(saved_path): # no backup: leave the timers as they are
|
||||
with open(saved_path) as f:
|
||||
write(json.load(f))
|
||||
os.remove(saved_path)
|
||||
|
||||
print(json.dumps({"timers": read(), "inhibitor": inhibitor()}))
|
||||
EOF
|
||||
@@ -0,0 +1,40 @@
|
||||
-- Hammerspoon: draw a ring around the Mac pointer so it shows in the VNC
|
||||
-- mirror on the Frame. macOS Screen Sharing leaves the pointer out of the
|
||||
-- framebuffer; a real on-screen window is captured like anything else.
|
||||
--
|
||||
-- Install: brew install --cask hammerspoon, then in ~/.hammerspoon/init.lua:
|
||||
-- dofile("/path/to/frame-control/scripts/mac-cursor-ring.lua")
|
||||
-- Toggle: ctrl+alt+cmd+M. Polls the pointer position, so no Accessibility
|
||||
-- permission is needed.
|
||||
|
||||
local SIZE, WIDTH = 34, 3
|
||||
local COLOR = { red = 1, green = 0.2, blue = 0.2, alpha = 0.9 }
|
||||
|
||||
local ring = hs.canvas.new({ x = 0, y = 0, w = SIZE, h = SIZE })
|
||||
ring:appendElements({
|
||||
type = "circle", action = "stroke",
|
||||
strokeColor = COLOR, strokeWidth = WIDTH,
|
||||
radius = (SIZE - WIDTH) / 2,
|
||||
})
|
||||
ring:level(hs.canvas.windowLevels.cursor)
|
||||
ring:behavior({ "canJoinAllSpaces", "stationary", "ignoresCycle" })
|
||||
|
||||
local last = {}
|
||||
local function follow()
|
||||
local p = hs.mouse.absolutePosition()
|
||||
if p.x ~= last.x or p.y ~= last.y then
|
||||
ring:topLeft({ x = p.x - SIZE / 2, y = p.y - SIZE / 2 })
|
||||
last = p
|
||||
end
|
||||
end
|
||||
|
||||
frameCursorRing = { canvas = ring, timer = hs.timer.new(1 / 60, follow) }
|
||||
|
||||
local function show() follow(); ring:show(); frameCursorRing.timer:start() end
|
||||
local function hide() frameCursorRing.timer:stop(); ring:hide() end
|
||||
|
||||
hs.hotkey.bind({ "ctrl", "alt", "cmd" }, "M", function()
|
||||
if ring:isShowing() then hide() else show() end
|
||||
end)
|
||||
|
||||
show()
|
||||
Executable
+847
@@ -0,0 +1,847 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Benchmark "Mac in the headset" on the real Frame, so every change to the
|
||||
stream is judged by numbers (docs/mac-in-headset.md, "Measuring").
|
||||
|
||||
scripts/macview-bench.py run [--scenario test,scroll,type] [--net 8] [--label x]
|
||||
scripts/macview-bench.py compare bench/results/A.json bench/results/B.json
|
||||
scripts/macview-bench.py ab --arm off:FRAME_MAC_VIEW_ADAPT=0 --arm on: --repeat 3 --scenario scroll
|
||||
|
||||
`run` uses the lab agent (mac/frame-mac-view/lab.sh serve), which has
|
||||
Screen Recording and Accessibility. It opens a viewer on the Frame the way
|
||||
Frame Control does, drives a fixed scenario, and reads the agent's per-frame
|
||||
records: when the Mac composited each frame, when it was encoded and sent,
|
||||
and when the viewer received, decoded and drew it (the viewer's clock is
|
||||
synced to the Mac's). "content" is Mac composited -> drawn in the viewer;
|
||||
what the Frame's compositor adds after that is reported ("present",
|
||||
"fps_shown") but not graded, because an unworn Frame throttles panels to
|
||||
36 fps (15 in standby) after a few seconds whatever they draw. It writes a summary to bench/results/ and exits 1 if a
|
||||
target is missed ("bad"), or if --baseline is given and a key number got
|
||||
more than 10% worse.
|
||||
|
||||
Wi-Fi changes from minute to minute, so single runs taken at different times
|
||||
can differ by more than a change does. `ab` restarts the lab agent with each
|
||||
arm's settings (environment variables read by the agent), runs the arms
|
||||
interleaved, and prints each metric's median and range per arm.
|
||||
|
||||
Nobody needs to wear the headset. The scroll and type scenarios open a
|
||||
throwaway Chrome window (its own profile in /tmp) on the Mac.
|
||||
|
||||
Network conditions come from a relay on the Mac between the agent and the
|
||||
tunnel, so they need no sudo:
|
||||
--net 8 8 Mbit/s the whole run
|
||||
--net 50@0,8@10,50@20 50 Mbit/s, 8 from 10 s, back to 50 from 20 s
|
||||
--delay 30 30 ms more one-way delay
|
||||
The relay holds at most --buffer ms of data at the current rate (default
|
||||
250 ms, like a bloated Wi-Fi queue), then pushes back on the agent.
|
||||
|
||||
Stdlib only.
|
||||
"""
|
||||
import argparse
|
||||
import asyncio
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import statistics
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from urllib.parse import quote, urlencode # %20, not +: the agent's URLComponents keeps +
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT / "ui"))
|
||||
import frame_macview # noqa: E402
|
||||
|
||||
RESULTS = ROOT / "bench" / "results"
|
||||
PAGES = ROOT / "bench" / "pages"
|
||||
CHROME = "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
|
||||
CHROME_PROFILE = "/tmp/fmv-bench-chrome"
|
||||
LAB_DIR = Path.home() / "Library" / "Caches" / "frame-mac-view-lab" # lab.sh serve
|
||||
|
||||
# The acceptance bar (the handoff's targets): (feels local, acceptable) per
|
||||
# metric; worse than acceptable is "bad". Latencies in ms.
|
||||
TARGETS = {
|
||||
"content_p50": (25, 60), "content_p95": (40, 100),
|
||||
"input_p50": (50, 100), "input_p95": (70, 150),
|
||||
"fps": (58, 45), # higher is better
|
||||
"late_pct": (1, 5), "stall_max": (100, 250),
|
||||
"adapt_s": (1, 3),
|
||||
}
|
||||
HIGHER_IS_BETTER = {"fps"}
|
||||
# Content that changes every frame, so fps and stalls mean something.
|
||||
MOVING = {"test", "scroll"}
|
||||
|
||||
|
||||
def ms(a, b):
|
||||
return (b - a) / 1000 if a and b else None
|
||||
|
||||
|
||||
def pct(values, p):
|
||||
v = sorted(x for x in values if x is not None)
|
||||
if not v:
|
||||
return None
|
||||
return round(v[min(len(v) - 1, max(0, round(p * (len(v) - 1))))], 1)
|
||||
|
||||
|
||||
def dist(values):
|
||||
v = [x for x in values if x is not None]
|
||||
return {"p50": pct(v, 0.5), "p95": pct(v, 0.95), "p99": pct(v, 0.99), "n": len(v)} if v else {"n": 0}
|
||||
|
||||
|
||||
# ---- the network relay ----
|
||||
|
||||
class Relay:
|
||||
"""Agent <-> tunnel, shaping the agent-to-Frame direction: a bottleneck
|
||||
of `rate` Mbit/s with a queue of `buffer_ms`, plus `delay` ms each way."""
|
||||
|
||||
def __init__(self, target_port, schedule, delay_ms=0, buffer_ms=250):
|
||||
self.target_port = target_port
|
||||
self.schedule = schedule # [(seconds from start, Mbit/s or None)]
|
||||
self.delay = delay_ms / 1000
|
||||
self.buffer_ms = buffer_ms
|
||||
self.rate = schedule[0][1] if schedule else None
|
||||
self.next_free = 0.0
|
||||
self.port = None
|
||||
self.loop = asyncio.new_event_loop()
|
||||
# Connections reset when a run tears the tunnel down; that's expected.
|
||||
self.loop.set_exception_handler(lambda loop, ctx: None if isinstance(ctx.get("exception"), ConnectionError)
|
||||
else loop.default_exception_handler(ctx))
|
||||
self.started = None
|
||||
self.queued_max = 0
|
||||
|
||||
def start(self):
|
||||
ready = threading.Event()
|
||||
|
||||
def run():
|
||||
asyncio.set_event_loop(self.loop)
|
||||
server = self.loop.run_until_complete(asyncio.start_server(self._client, "127.0.0.1", 0))
|
||||
self.port = server.sockets[0].getsockname()[1]
|
||||
ready.set()
|
||||
self.loop.run_forever()
|
||||
|
||||
threading.Thread(target=run, daemon=True).start()
|
||||
ready.wait(5)
|
||||
|
||||
def begin(self):
|
||||
"""Start the schedule's clock (when the measured part begins)."""
|
||||
self.started = time.monotonic()
|
||||
for at, rate in self.schedule:
|
||||
self.loop.call_soon_threadsafe(self.loop.call_later, at, self._set_rate, rate)
|
||||
|
||||
def _set_rate(self, rate):
|
||||
self.rate = rate
|
||||
|
||||
def rate_at(self, t):
|
||||
r = None
|
||||
for at, rate in self.schedule:
|
||||
if t >= at:
|
||||
r = rate
|
||||
return r
|
||||
|
||||
async def _client(self, reader, writer):
|
||||
try:
|
||||
up_r, up_w = await asyncio.open_connection("127.0.0.1", self.target_port)
|
||||
except OSError:
|
||||
writer.close()
|
||||
return
|
||||
await asyncio.gather(self._shaped(up_r, writer), self._plain(reader, up_w), return_exceptions=True)
|
||||
for w in (writer, up_w):
|
||||
w.close()
|
||||
|
||||
async def _plain(self, reader, writer): # Frame -> agent: delay only
|
||||
if not self.delay:
|
||||
while data := await reader.read(65536):
|
||||
writer.write(data)
|
||||
await writer.drain()
|
||||
writer.close()
|
||||
return
|
||||
# Each chunk leaves `delay` after it arrived, while reading goes on:
|
||||
# a delay line, not a pause (which would add up behind a busy stream).
|
||||
queue = asyncio.Queue()
|
||||
|
||||
async def send():
|
||||
while (item := await queue.get())[1] is not None:
|
||||
wait = item[0] - self.loop.time()
|
||||
if wait > 0:
|
||||
await asyncio.sleep(wait)
|
||||
writer.write(item[1])
|
||||
await writer.drain()
|
||||
writer.close()
|
||||
|
||||
async def pump():
|
||||
try:
|
||||
while data := await reader.read(65536):
|
||||
await queue.put((self.loop.time() + self.delay, data))
|
||||
finally: # EOF or a reset: either way the sender finishes and closes
|
||||
queue.put_nowait((0, None))
|
||||
|
||||
sender, pumper = asyncio.ensure_future(send()), asyncio.ensure_future(pump())
|
||||
done, _ = await asyncio.wait({sender, pumper}, return_when=asyncio.FIRST_COMPLETED)
|
||||
if sender in done: # the agent's side went away: stop reading, so the connection closes
|
||||
pumper.cancel()
|
||||
writer.close()
|
||||
return
|
||||
await sender # the viewer's side ended: deliver what's queued first
|
||||
|
||||
async def _shaped(self, reader, writer): # agent -> Frame
|
||||
queue = asyncio.Queue()
|
||||
state = {"bytes": 0}
|
||||
room = asyncio.Event()
|
||||
room.set()
|
||||
|
||||
async def send():
|
||||
while True:
|
||||
at, data = await queue.get()
|
||||
if data is None:
|
||||
return
|
||||
wait = at - self.loop.time()
|
||||
if wait > 0:
|
||||
await asyncio.sleep(wait)
|
||||
writer.write(data)
|
||||
await writer.drain()
|
||||
state["bytes"] -= len(data)
|
||||
room.set()
|
||||
|
||||
sender = asyncio.ensure_future(send())
|
||||
while data := await reader.read(16384):
|
||||
now = self.loop.time()
|
||||
depart = now
|
||||
if self.rate:
|
||||
depart = max(now, self.next_free) + len(data) * 8 / (self.rate * 1e6)
|
||||
self.next_free = depart
|
||||
state["bytes"] += len(data)
|
||||
self.queued_max = max(self.queued_max, state["bytes"])
|
||||
await queue.put((depart + self.delay, data))
|
||||
# A full queue pushes back, as a real bottleneck's buffer does.
|
||||
while self.rate and state["bytes"] > self.rate * 1e6 / 8 * self.buffer_ms / 1000:
|
||||
room.clear()
|
||||
await room.wait()
|
||||
await queue.put((0, None))
|
||||
await sender
|
||||
|
||||
|
||||
# ---- the lab agent, through Frame Control's own code ----
|
||||
|
||||
class LabView(frame_macview.MacView):
|
||||
"""MacView against the already-running lab agent, with the tunnel
|
||||
pointing at `tunnel_port` (the relay, or the agent itself)."""
|
||||
|
||||
def __init__(self, tunnel_ssh, run, frame, agent_port, token, tunnel_port):
|
||||
super().__init__(tunnel_ssh, run, frame)
|
||||
self.agent_port = agent_port
|
||||
self.token = token
|
||||
self.port = tunnel_port
|
||||
|
||||
class Alive:
|
||||
def poll(self):
|
||||
return None
|
||||
|
||||
self.agent = Alive()
|
||||
|
||||
def ensure_agent(self):
|
||||
pass
|
||||
|
||||
def call(self, path, method="GET", **query):
|
||||
url = f"http://127.0.0.1:{self.agent_port}{path}?{urlencode({**query, 'k': self.token}, quote_via=quote)}"
|
||||
req = urllib.request.Request(url, method=method, data=b"" if method == "POST" else None)
|
||||
with urllib.request.urlopen(req, timeout=15) as r:
|
||||
return json.load(r)
|
||||
|
||||
|
||||
def ssh_runner(base):
|
||||
def run(remote, stdin=None, timeout=30):
|
||||
r = subprocess.run([*base, remote], input=stdin, capture_output=True, text=True, timeout=timeout)
|
||||
if r.returncode:
|
||||
e = RuntimeError((r.stderr or r.stdout).strip())
|
||||
e.stdout = r.stdout
|
||||
raise e
|
||||
return r.stdout
|
||||
return run
|
||||
|
||||
|
||||
# ---- CPU on both ends ----
|
||||
|
||||
FRAME_CPU = r"""
|
||||
hz=$(getconf CLK_TCK)
|
||||
while :; do
|
||||
t=$(date +%s.%N)
|
||||
for g in viewer:frame-control/mac-view tailscaled:tailscaled sshd:'^sshd' gamescope:'^gamescope'; do
|
||||
name=${g%%:*} pat=${g#*:} sum=0
|
||||
for p in $(pgrep -f "$pat"); do
|
||||
s=$(awk '{print $14+$15}' /proc/$p/stat 2>/dev/null) && sum=$((sum + s))
|
||||
done
|
||||
echo "$t $name $sum $hz"
|
||||
done
|
||||
echo "$t total $(awk '/^cpu /{print $2+$3+$4+$6+$7+$8}' /proc/stat) $hz"
|
||||
sleep 2
|
||||
done
|
||||
"""
|
||||
|
||||
|
||||
class CpuSampler:
|
||||
def __init__(self, frame_ssh, agent_pid):
|
||||
self.agent_pid = agent_pid
|
||||
self.mac = [] # (t, cpu seconds)
|
||||
self.frame = {} # name -> [(t, ticks, hz)]
|
||||
self.proc = subprocess.Popen([*frame_ssh, "bash -s"], stdin=subprocess.PIPE, stdout=subprocess.PIPE,
|
||||
stderr=subprocess.DEVNULL, text=True)
|
||||
self.proc.stdin.write(FRAME_CPU)
|
||||
self.proc.stdin.close()
|
||||
threading.Thread(target=self._read, daemon=True).start()
|
||||
self.stop_flag = False
|
||||
threading.Thread(target=self._mac, daemon=True).start()
|
||||
|
||||
def _read(self):
|
||||
for line in self.proc.stdout:
|
||||
parts = line.split()
|
||||
if len(parts) == 4:
|
||||
self.frame.setdefault(parts[1], []).append((float(parts[0]), int(parts[2]), int(parts[3])))
|
||||
|
||||
def _mac(self):
|
||||
while not self.stop_flag:
|
||||
out = subprocess.run(["ps", "-o", "cputime=", "-p", str(self.agent_pid)], capture_output=True,
|
||||
text=True).stdout.strip()
|
||||
m = re.match(r"(?:(\d+):)?(\d+):(\d+(?:\.\d+)?)", out)
|
||||
if m:
|
||||
h, mnt, s = m.groups()
|
||||
self.mac.append((time.monotonic(), int(h or 0) * 3600 + int(mnt) * 60 + float(s)))
|
||||
time.sleep(2)
|
||||
|
||||
def stop(self):
|
||||
self.stop_flag = True
|
||||
self.proc.terminate()
|
||||
|
||||
def summary(self):
|
||||
def rate(samples):
|
||||
if len(samples) < 2:
|
||||
return None
|
||||
(t0, a0, *h0), (t1, a1, *_) = samples[0], samples[-1]
|
||||
hz = h0[0] if h0 else 1
|
||||
return round((a1 - a0) / hz / (t1 - t0) * 100, 1)
|
||||
out = {"mac_agent_pct": rate(self.mac)}
|
||||
for name, s in self.frame.items():
|
||||
out[f"frame_{name}_pct"] = rate(s)
|
||||
if "frame_total_pct" in out and out["frame_total_pct"] is not None:
|
||||
out["frame_total_pct"] = round(out["frame_total_pct"] / 8, 1) # 8 cores -> share of the whole
|
||||
return out
|
||||
|
||||
|
||||
# ---- scenarios ----
|
||||
|
||||
def chrome_window(page, size=(1280, 820)):
|
||||
"""Opens a page in a throwaway Chrome app window; returns (proc, title)."""
|
||||
if not Path(CHROME).exists():
|
||||
raise SystemExit("Google Chrome is needed for the scroll and type scenarios.")
|
||||
proc = subprocess.Popen([CHROME, f"--user-data-dir={CHROME_PROFILE}", "--no-first-run",
|
||||
"--no-default-browser-check", f"--window-size={size[0]},{size[1]}",
|
||||
"--window-position=80,80", f"--app=file://{PAGES / page}"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
|
||||
return proc
|
||||
|
||||
|
||||
def find_window(mv, title, timeout=15):
|
||||
end = time.time() + timeout
|
||||
while time.time() < end:
|
||||
for w in mv.call("/windows").get("windows", []):
|
||||
if title in (w.get("title") or ""):
|
||||
return w
|
||||
time.sleep(0.5)
|
||||
raise SystemExit(f"window {title!r} didn't appear")
|
||||
|
||||
|
||||
def git(*a):
|
||||
return subprocess.run(["git", *a], cwd=ROOT, capture_output=True, text=True).stdout.strip()
|
||||
|
||||
|
||||
def parse_schedule(net):
|
||||
if not net or net == "none":
|
||||
return []
|
||||
out = []
|
||||
for part in net.split(","):
|
||||
rate, _, at = part.partition("@")
|
||||
out.append((float(at or 0), None if rate in ("", "none", "0") else float(rate)))
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def run_scenario(args, scenario, agent_port, token, frame_ssh):
|
||||
schedule = parse_schedule(args.net)
|
||||
relay = None
|
||||
tunnel_port = agent_port
|
||||
if schedule or args.delay:
|
||||
relay = Relay(agent_port, schedule, args.delay, args.buffer)
|
||||
relay.start()
|
||||
tunnel_port = relay.port
|
||||
tunnel_ssh = list(frame_ssh[:-1]) + args.ssh_opt
|
||||
mv = LabView(tunnel_ssh, ssh_runner(frame_ssh), frame_ssh[-1], agent_port, token, tunnel_port)
|
||||
mv.prefer_usb = args.usb # otherwise the path is --host's (or ssh's default)
|
||||
if args.browser_flag is not None:
|
||||
mv.browser_flags = [f for f in args.browser_flag if f]
|
||||
chrome = None
|
||||
src = "test"
|
||||
try:
|
||||
if scenario in ("scroll", "type"):
|
||||
chrome = chrome_window(f"{scenario}.html")
|
||||
w = find_window(mv, f"fmv-bench {scenario}")
|
||||
src = f"{args.mode}:{w['id']}"
|
||||
t = time.time()
|
||||
shown = mv.show(src, args.quality, 1280, 820)
|
||||
show_s = round(time.time() - t, 2)
|
||||
# Wait for frames, then measure from a clean start.
|
||||
stream = None
|
||||
end = time.time() + 20
|
||||
while time.time() < end and not stream:
|
||||
for s in mv.call("/status").get("streams", []):
|
||||
if s["src"] == src and s.get("stats", {}).get("fps", 0) > 0:
|
||||
stream = s
|
||||
time.sleep(0.5)
|
||||
if not stream:
|
||||
raise SystemExit(f"{scenario}: no frames reached the viewer within 20 s")
|
||||
time.sleep(args.warmup)
|
||||
first = mv.call("/stats", id=stream["id"], settle=0)["streams"][0]
|
||||
since = max([f["s"] for f in first["frames"]] or [0])
|
||||
captured_before = first["captured"]
|
||||
start_mac_us = mv.call("/stats", id=stream["id"])["now"]
|
||||
agent_pid = int(subprocess.run(["pgrep", "-f", "Frame Mac View Lab.app/Contents/MacOS/frame-mac-view"],
|
||||
capture_output=True, text=True).stdout.split()[0])
|
||||
cpu = CpuSampler(frame_ssh, agent_pid)
|
||||
if relay:
|
||||
relay.begin()
|
||||
expected = 0 # input events the harness asked the viewer for
|
||||
if scenario == "type":
|
||||
mv.call("/bench", method="POST", src=src, action="click", x=0.5, y=0.3)
|
||||
time.sleep(0.5)
|
||||
# Steady typing, then slow typing that lets the link go idle.
|
||||
fast = "the quick brown fox jumps over the lazy dog "
|
||||
n_fast = max(10, int((args.duration * 0.6) / 0.2))
|
||||
mv.call("/bench", method="POST", src=src, action="type", text=(fast * (n_fast // len(fast) + 1))[:n_fast],
|
||||
interval=200)
|
||||
expected += n_fast
|
||||
time.sleep(n_fast * 0.2 + 0.5)
|
||||
slow_n = max(3, int(args.duration * 0.4 / 1.5))
|
||||
mv.call("/bench", method="POST", src=src, action="type", text=("idle typing test " * (slow_n // 17 + 1))[:slow_n],
|
||||
interval=1500)
|
||||
expected += slow_n
|
||||
time.sleep(slow_n * 1.5 + 0.5)
|
||||
else:
|
||||
# Clicks on the test pattern measure the input path; the scroll
|
||||
# page just scrolls.
|
||||
end = time.time() + args.duration
|
||||
while time.time() < end:
|
||||
if scenario == "test":
|
||||
mv.call("/bench", method="POST", src=src, action="click", x=0.3, y=0.5)
|
||||
expected += 1
|
||||
time.sleep(0.5)
|
||||
at_end = mv.call("/stats", id=stream["id"], since=2**32 - 1)
|
||||
end_mac_us = at_end["now"]
|
||||
captured_end = at_end["streams"][0]["captured"] if at_end["streams"] else None
|
||||
time.sleep(2) # let the viewer report the last frames
|
||||
streams = mv.call("/stats", id=stream["id"], since=since)["streams"]
|
||||
if not streams:
|
||||
raise SystemExit(f"{scenario}: the stream ended during the run (did the Frame go to sleep?)")
|
||||
data = streams[0]
|
||||
cpu.stop()
|
||||
if args.raw:
|
||||
Path(f"{args.raw}-{scenario}.json").write_text(json.dumps(data))
|
||||
result = summarize(scenario, data, start_mac_us, end_mac_us, args, relay, schedule, expected)
|
||||
result["controller"] = data.get("controller", {})
|
||||
result["events"] = [{"t": round((e["t"] - start_mac_us) / 1e6, 2), "e": e["e"]}
|
||||
for e in data.get("events", []) if e["t"] >= start_mac_us]
|
||||
result["captured"] = (captured_end if captured_end is not None else data["captured"]) - captured_before
|
||||
result["source_fps"] = round(result["captured"] / max(result["duration_s"], 1), 1)
|
||||
result["cpu"] = cpu.summary()
|
||||
result["viewer"] = data["summary"].get("decoder", "")
|
||||
result["show_s"] = show_s
|
||||
result["panel"] = shown.get("panel")
|
||||
result["src"] = src
|
||||
result["route"] = mv.route
|
||||
return result
|
||||
finally:
|
||||
try:
|
||||
mv.stop(src)
|
||||
except Exception: # noqa: BLE001 - best effort
|
||||
pass
|
||||
mv.closing = True # or its supervisor reopens the tunnel
|
||||
if mv.tunnel and mv.tunnel.poll() is None:
|
||||
mv.tunnel.terminate()
|
||||
mv.tunnel.wait(5)
|
||||
if relay:
|
||||
relay.loop.call_soon_threadsafe(relay.loop.stop)
|
||||
if chrome:
|
||||
chrome.terminate()
|
||||
time.sleep(3) # Stop ends the Frame's viewer browser 2 s later
|
||||
|
||||
|
||||
def composited(f):
|
||||
"""When the Mac had the frame: its display time, or when ScreenCaptureKit
|
||||
handed it over if that was earlier (display time can be a few ms ahead of
|
||||
delivery, which would make latency look lower than it is)."""
|
||||
return min(f["cap"], f["arr"]) if f["arr"] else f["cap"]
|
||||
|
||||
|
||||
def summarize(scenario, data, start_us, end_us, args, relay, schedule, expected=0):
|
||||
frames = [f for f in data["frames"] if start_us <= f["cap"] <= end_us]
|
||||
inputs = [i for i in data["inputs"] if i["tv"] and i["tv"] >= start_us]
|
||||
by_seq = {f["s"]: f for f in data["frames"]}
|
||||
drawn = [f for f in frames if f["drw"]]
|
||||
shown = [f for f in frames if f["vs"]]
|
||||
# The whole measured interval, so a stall at either end still counts.
|
||||
duration = (end_us - start_us) / 1e6
|
||||
stages = {
|
||||
"capture": [ms(f["cap"], f["arr"]) for f in frames],
|
||||
"queue": [ms(f["arr"], f["e0"]) for f in frames],
|
||||
"encode": [ms(f["e0"], f["e1"]) for f in frames],
|
||||
"socket": [ms(f["snd"], f["wire"]) for f in frames],
|
||||
"network": [ms(f["e1"], f["rx"]) for f in frames],
|
||||
"decode": [ms(f["rx"], f["dec"]) for f in frames],
|
||||
"draw": [ms(f["dec"], f["drw"]) for f in frames],
|
||||
# Waiting for the browser's next frame: set by the Frame's compositor,
|
||||
# which throttles panels nobody is looking at (see "Measuring").
|
||||
"present": [ms(f["drw"], f["vs"]) for f in frames],
|
||||
"content": [ms(composited(f), f["drw"]) for f in frames],
|
||||
"content_shown": [ms(composited(f), f["vs"]) for f in frames],
|
||||
}
|
||||
out = {"scenario": scenario, "frames_sent": len(frames), "frames_drawn": len(drawn),
|
||||
"frames_shown": len(shown), "duration_s": round(duration, 1)}
|
||||
out["stages_ms"] = {k: dist(v) for k, v in stages.items()}
|
||||
# Smoothness: gaps between frames reaching the viewer's canvas.
|
||||
# Smoothness counts only what was drawn inside the interval; a frame
|
||||
# captured just before the end but drawn after it still counts for latency.
|
||||
dr = [start_us] + sorted(f["drw"] for f in drawn if f["drw"] <= end_us) + [end_us]
|
||||
gaps = [(b - a) / 1000 for a, b in zip(dr, dr[1:]) if b > a]
|
||||
target_fps = frame_macview.QUALITY[args.quality]["fps"]
|
||||
out["fps"] = round(sum(f["drw"] <= end_us for f in drawn) / duration, 1) if duration else 0
|
||||
out["fps_shown"] = round(sum(0 < f["vs"] <= end_us for f in shown) / duration, 1) if duration else 0
|
||||
out["late_pct"] = round(100 * sum(g > 1.5 * 1000 / target_fps for g in gaps) / len(gaps), 2) if gaps else None
|
||||
out["stall_max"] = round(max(gaps), 1) if gaps else None
|
||||
out["stalls_over_100ms"] = sum(g > 100 for g in gaps)
|
||||
out["mbps"] = round(sum(f["b"] for f in frames) * 8 / duration / 1e6, 2) if duration else 0
|
||||
out["keyframes"] = sum(f["k"] for f in frames)
|
||||
out["size"] = f"{frames[-1]['w']}x{frames[-1]['h']}" if frames else ""
|
||||
out["captured"] = data.get("captured")
|
||||
out["viewer_never_drawn"] = len(frames) - len(drawn)
|
||||
# Input: the viewer's event -> the first frame after it drawn (and shown).
|
||||
lat, lat_shown = [], []
|
||||
parts = {"uplink": [], "mac": [], "back": []}
|
||||
for i in inputs:
|
||||
f = by_seq.get(i["frame"])
|
||||
if f and f["drw"]:
|
||||
lat.append(ms(i["tv"], f["drw"]))
|
||||
lat_shown.append(ms(i["tv"], f["vs"]))
|
||||
parts["uplink"].append(ms(i["tv"], i["inj"]))
|
||||
parts["mac"].append(ms(i["inj"], composited(f)))
|
||||
parts["back"].append(ms(composited(f), f["drw"]))
|
||||
out["input_ms"] = dist(lat)
|
||||
out["input_shown_ms"] = dist(lat_shown)
|
||||
out["input_parts_ms"] = {k: dist(v) for k, v in parts.items()}
|
||||
out["inputs"] = {"asked": expected, "sent": len(inputs), "seen": len(lat)}
|
||||
# A per-second timeline, for adaptation.
|
||||
timeline = []
|
||||
t0 = start_us
|
||||
for sec in range(int(duration) + 1):
|
||||
a, b = t0 + sec * 1_000_000, t0 + (sec + 1) * 1_000_000
|
||||
fs = [f for f in frames if a <= f["cap"] < b]
|
||||
if not fs:
|
||||
continue
|
||||
timeline.append({
|
||||
"t": sec, "fps": sum(1 for f in fs if f["drw"]), "mbps": round(sum(f["b"] for f in fs) * 8 / 1e6, 2),
|
||||
"content_p50": pct([ms(composited(f), f["drw"]) for f in fs], 0.5),
|
||||
"content_p95": pct([ms(composited(f), f["drw"]) for f in fs], 0.95),
|
||||
"bitrate": fs[-1].get("br"), "tier": fs[-1].get("tier"), "w": fs[-1]["w"],
|
||||
"net": relay.rate_at(sec) if relay else None,
|
||||
})
|
||||
out["timeline"] = timeline
|
||||
out["adapt"] = adaptation(timeline, schedule, duration)
|
||||
out["content_p50"] = out["stages_ms"]["content"].get("p50")
|
||||
out["content_p95"] = out["stages_ms"]["content"].get("p95")
|
||||
out["input_p50"] = out["input_ms"].get("p50")
|
||||
out["input_p95"] = out["input_ms"].get("p95")
|
||||
out["grades"] = grade(out, scenario, args.quality)
|
||||
return out
|
||||
|
||||
|
||||
def adaptation(timeline, schedule, duration=float("inf")):
|
||||
"""After each drop in the link's rate: how long until latency was within
|
||||
bounds again and stayed there for 3 s. The bound is the acceptable p95
|
||||
(100 ms), or 1.5 times the p95 before the drop if that was higher: on a
|
||||
slower link each frame takes longer to send, so latency can't return to
|
||||
what it was on the fast one."""
|
||||
steps = []
|
||||
for n, ((at, rate), (_, before)) in enumerate(zip(schedule[1:], schedule)):
|
||||
# Settling must happen while this rate lasts, not after the link recovers.
|
||||
until = min(schedule[n + 2][0] if n + 2 < len(schedule) else float("inf"), duration) # or the run ends
|
||||
if before is not None and (rate is None or rate >= before):
|
||||
continue
|
||||
pre = [s["content_p95"] for s in timeline if at - 4 <= s["t"] < at and s["content_p95"]]
|
||||
base = max(float(TARGETS["content_p95"][1]), 1.5 * statistics.median(pre) if pre else 0)
|
||||
after = [s for s in timeline if s["t"] >= at]
|
||||
worst = max([s["content_p95"] or 0 for s in after[:10]] or [0])
|
||||
settled = None
|
||||
for i, s in enumerate(after):
|
||||
window = after[i:i + 3]
|
||||
# Three populated seconds in a row: a second with no frames at all
|
||||
# is a stall, not a pass.
|
||||
if (len(window) == 3 and [w["t"] for w in window] == [s["t"], s["t"] + 1, s["t"] + 2] and s["t"] + 3 <= until
|
||||
and all((w["content_p95"] or 1e9) <= base for w in window)):
|
||||
settled = s["t"] - at
|
||||
break
|
||||
steps.append({"at": at, "to_mbit": rate, "p95_bound": round(base, 1), "worst_p95": worst,
|
||||
"settled_s": settled})
|
||||
return steps
|
||||
|
||||
|
||||
def grade(out, scenario, quality="balanced"):
|
||||
g = {}
|
||||
checks = ["content_p50", "content_p95"]
|
||||
asked = max(out["inputs"].get("asked", 0), out["inputs"]["sent"])
|
||||
if asked:
|
||||
checks += ["input_p50", "input_p95"]
|
||||
# Input that never reaches the Mac, or never shows up on screen, is a
|
||||
# failure, not a gap in the data.
|
||||
seen = out["inputs"]["seen"] / asked
|
||||
g["input_replies"] = "local" if seen >= 0.95 else "acceptable" if seen >= 0.8 else "bad"
|
||||
if scenario in MOVING:
|
||||
checks += ["fps", "late_pct", "stall_max"]
|
||||
for k in checks:
|
||||
v = out.get(k)
|
||||
if v is None:
|
||||
g[k] = "missing"
|
||||
continue
|
||||
good, ok = TARGETS[k]
|
||||
if k == "fps": # the targets are for 60 fps; Light and Compatible ask for fewer
|
||||
want = frame_macview.QUALITY[quality]["fps"]
|
||||
good, ok = good * want / 60, ok * want / 60
|
||||
if k in HIGHER_IS_BETTER:
|
||||
g[k] = "local" if v >= good else "acceptable" if v >= ok else "bad"
|
||||
else:
|
||||
g[k] = "local" if v <= good else "acceptable" if v <= ok else "bad"
|
||||
for step in out["adapt"]:
|
||||
s = step["settled_s"]
|
||||
good, ok = TARGETS["adapt_s"]
|
||||
g[f"adapt@{step['at']:g}s"] = "bad" if s is None or s > ok else "local" if s <= good else "acceptable"
|
||||
return g
|
||||
|
||||
|
||||
# ---- compare ----
|
||||
|
||||
KEYS = [("content_p50", "content p50 ms"), ("content_p95", "content p95 ms"), ("input_p50", "input p50 ms"),
|
||||
("input_p95", "input p95 ms"), ("fps", "fps"), ("late_pct", "late %"), ("stall_max", "max gap ms"),
|
||||
("fps_shown", "fps shown"), ("mbps", "Mbit/s")]
|
||||
STAGE_KEYS = ["capture", "queue", "encode", "socket", "network", "decode", "draw", "present"]
|
||||
|
||||
|
||||
def load(path):
|
||||
return json.loads(Path(path).read_text())
|
||||
|
||||
|
||||
def compare(a, b, threshold=0.10):
|
||||
"""Prints before/after per scenario; returns the regressions."""
|
||||
regressions = []
|
||||
print(f"A: {a['label']} ({a['commit']}, {a['date']})\nB: {b['label']} ({b['commit']}, {b['date']})")
|
||||
for sc in b["scenarios"]:
|
||||
ra = next((x for x in a["scenarios"] if x["scenario"] == sc["scenario"]), None)
|
||||
if not ra:
|
||||
continue
|
||||
print(f"\n{sc['scenario']:<10} {'A':>10} {'B':>10} {'change':>9}")
|
||||
rows = [(k, n, ra.get(k), sc.get(k)) for k, n in KEYS]
|
||||
rows += [(f"stage.{s}", f" {s} p50", ra["stages_ms"].get(s, {}).get("p50"), sc["stages_ms"].get(s, {}).get("p50"))
|
||||
for s in STAGE_KEYS]
|
||||
for k, name, va, vb in rows:
|
||||
change = ""
|
||||
if isinstance(va, (int, float)) and isinstance(vb, (int, float)) and va:
|
||||
d = (vb - va) / abs(va)
|
||||
change = f"{d:+.0%}"
|
||||
worse = d < -threshold if k == "fps" else d > threshold
|
||||
if worse and k in ("content_p50", "content_p95", "input_p50", "input_p95", "fps") and abs(vb - va) > 2:
|
||||
regressions.append(f"{sc['scenario']} {name}: {va} -> {vb}")
|
||||
change += " worse"
|
||||
print(f"{name:<18} {fmt(va):>10} {fmt(vb):>10} {change:>9}")
|
||||
return regressions
|
||||
|
||||
|
||||
def fmt(v):
|
||||
return "–" if v is None else f"{v:g}" if isinstance(v, (int, float)) else str(v)
|
||||
|
||||
|
||||
# ---- main ----
|
||||
|
||||
def add_run_args(r):
|
||||
r.add_argument("--scenario", default="test,scroll,type")
|
||||
r.add_argument("--duration", type=float, default=20)
|
||||
r.add_argument("--warmup", type=float, default=3)
|
||||
r.add_argument("--quality", default="balanced", choices=list(frame_macview.QUALITY))
|
||||
r.add_argument("--mode", default="separate", choices=["separate", "window"], help="how Mac windows are shown")
|
||||
r.add_argument("--net", default="", help="Mbit/s, or a schedule like 50@0,8@10,50@20")
|
||||
r.add_argument("--delay", type=float, default=0, help="extra one-way delay, ms")
|
||||
r.add_argument("--buffer", type=float, default=250, help="bottleneck queue, ms at the current rate")
|
||||
r.add_argument("--frame", default="frame", help="ssh host of the Frame")
|
||||
r.add_argument("--host", default="", help="connect to this address instead (e.g. the Frame's LAN IP)")
|
||||
r.add_argument("--usb", action="store_true", help="use the Frame's USB-C network when plugged in, as Frame Control does")
|
||||
r.add_argument("--ssh-opt", action="append", default=[], help="extra ssh option for the tunnel, e.g. -oIPQoS=ef")
|
||||
r.add_argument("--browser-flag", action="append", default=None,
|
||||
help="Chromium flag for the viewer (replaces Frame Control's defaults; '' for none)")
|
||||
r.add_argument("--label", default="")
|
||||
r.add_argument("--out", default="")
|
||||
r.add_argument("--raw", default="", help="also write the agent's raw records here")
|
||||
|
||||
|
||||
def lab_agent():
|
||||
try:
|
||||
agent_port = int((LAB_DIR / "port").read_text())
|
||||
token = (LAB_DIR / "token").read_text().strip()
|
||||
urllib.request.urlopen(f"http://127.0.0.1:{agent_port}/ping", timeout=3)
|
||||
return agent_port, token
|
||||
except (OSError, ValueError):
|
||||
raise SystemExit("Start the lab agent first: mac/frame-mac-view/lab.sh serve")
|
||||
|
||||
|
||||
def restart_lab(env):
|
||||
"""The lab agent again, without rebuilding, with these experiment switches."""
|
||||
subprocess.run(["sh", str(ROOT / "mac" / "frame-mac-view" / "lab.sh"), "serve-only"],
|
||||
env={**os.environ, **env}, check=True, capture_output=True)
|
||||
time.sleep(1)
|
||||
|
||||
|
||||
def frame_ssh_for(args):
|
||||
frame_ssh = ["ssh", "-o", "BatchMode=yes"]
|
||||
if args.host:
|
||||
# Another address for the same Frame: its host key must still match.
|
||||
alias = next((line.split()[1] for line in subprocess.run(["ssh", "-G", args.frame], capture_output=True,
|
||||
text=True).stdout.splitlines() if line.startswith("hostname ")), args.frame)
|
||||
frame_ssh += ["-o", f"HostName={args.host}", "-o", f"HostKeyAlias={alias}"]
|
||||
frame_ssh.append(args.frame)
|
||||
return frame_ssh
|
||||
|
||||
|
||||
def run_suite(args, frame_ssh, quiet=False):
|
||||
agent_port, token = lab_agent()
|
||||
results = []
|
||||
for sc in args.scenario.split(","):
|
||||
if not quiet:
|
||||
print(f"-- {sc} ...", flush=True)
|
||||
res = run_scenario(args, sc, agent_port, token, frame_ssh)
|
||||
if not quiet:
|
||||
print(f" content p50/p95 {res['content_p50']}/{res['content_p95']} ms, input p50/p95 "
|
||||
f"{res['input_p50']}/{res['input_p95']} ms, {res['fps']} fps (shown {res['fps_shown']}), "
|
||||
f"late {res['late_pct']}%, max gap {res['stall_max']} ms, {res['mbps']} Mbit/s, {res['size']}",
|
||||
flush=True)
|
||||
print(f" stages p50: " + ", ".join(f"{k} {v.get('p50')}" for k, v in res["stages_ms"].items()), flush=True)
|
||||
print(f" grades: {res['grades']} cpu: {res['cpu']}\n viewer: {res['viewer']}", flush=True)
|
||||
for e in res["events"][:12]:
|
||||
print(f" {e['t']:6.2f}s {e['e']}", flush=True)
|
||||
results.append(res)
|
||||
return results
|
||||
|
||||
|
||||
def document(args, frame_ssh, results, **extra):
|
||||
build = subprocess.run([*frame_ssh, "grep -m1 BUILD_ID /etc/os-release"], capture_output=True, text=True).stdout
|
||||
# Whether the headset is in standby (it throttles panels even harder then).
|
||||
standby = subprocess.run([*frame_ssh, "grep -h standby ~/.local/share/Steam/logs/vrserver.txt | tail -n 1"],
|
||||
capture_output=True, text=True).stdout.strip()
|
||||
commit = git("rev-parse", "--short", "HEAD") + ("+dirty" if git("status", "--porcelain", "--", "mac", "ui") else "")
|
||||
return {
|
||||
"label": args.label or args.cmd, "date": datetime.datetime.now().isoformat(timespec="seconds"), "commit": commit,
|
||||
"config": {"quality": args.quality, "mode": args.mode, "net": args.net or "none", "delay_ms": args.delay,
|
||||
"buffer_ms": args.buffer, "host": args.host or args.frame, "usb": args.usb, "ssh_opts": args.ssh_opt,
|
||||
"encoder": os.environ.get("FRAME_MAC_VIEW_ENCODER", ""), "duration_s": args.duration,
|
||||
"browser_flags": args.browser_flag if args.browser_flag is not None else frame_macview.BROWSER_FLAGS},
|
||||
"frame_build": build.strip().partition("=")[2], "mac": platform.mac_ver()[0], "headset": standby[-40:],
|
||||
"scenarios": results, **extra,
|
||||
}
|
||||
|
||||
|
||||
def write(doc, args):
|
||||
RESULTS.mkdir(parents=True, exist_ok=True)
|
||||
name = f"{doc['date'][:10]}-{doc['commit'].replace('+', '-')}-{re.sub(r'[^a-zA-Z0-9_.-]+', '-', doc['label'])}.json"
|
||||
out = Path(args.out) if args.out else RESULTS / name
|
||||
out.write_text(json.dumps(doc, indent=1))
|
||||
print(f"wrote {out}")
|
||||
return out
|
||||
|
||||
|
||||
AB_KEYS = ["content_p50", "content_p95", "input_p50", "input_p95", "fps", "late_pct", "stall_max", "mbps"]
|
||||
|
||||
|
||||
def ab(args, frame_ssh):
|
||||
"""Arms interleaved (A B, B A, ...) so a Wi-Fi link that changes over
|
||||
minutes affects both alike; medians and ranges per arm."""
|
||||
arms = []
|
||||
for a in args.arm:
|
||||
name, _, envs = a.partition(":")
|
||||
arms.append((name, dict(e.split("=", 1) for e in envs.split(",") if e)))
|
||||
runs = {name: [] for name, _ in arms}
|
||||
for rep in range(args.repeat):
|
||||
for name, env in (arms if rep % 2 == 0 else arms[::-1]):
|
||||
print(f"-- repeat {rep + 1}, arm {name} {env}", flush=True)
|
||||
restart_lab(env)
|
||||
runs[name].append(run_suite(args, frame_ssh, quiet=True))
|
||||
table = {}
|
||||
for sc in args.scenario.split(","):
|
||||
print(f"\n{sc:<14}" + "".join(f"{name:>26}" for name, _ in arms))
|
||||
for k in AB_KEYS:
|
||||
row = []
|
||||
for name, _ in arms:
|
||||
vals = [r[k] for suite in runs[name] for r in suite if r["scenario"] == sc and r.get(k) is not None]
|
||||
table.setdefault(sc, {}).setdefault(k, {})[name] = vals
|
||||
row.append(f"{statistics.median(vals):g} ({min(vals):g}-{max(vals):g})" if vals else "–")
|
||||
print(f"{k:<14}" + "".join(f"{c:>26}" for c in row))
|
||||
restart_lab({}) # back to the defaults
|
||||
return runs, table
|
||||
|
||||
|
||||
def main():
|
||||
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
sub = p.add_subparsers(dest="cmd", required=True)
|
||||
r = sub.add_parser("run")
|
||||
add_run_args(r)
|
||||
r.add_argument("--baseline", default="", help="fail on >10%% regressions against this result")
|
||||
a = sub.add_parser("ab", help="compare lab-agent settings, interleaved: --arm name:ENV=V,ENV2=V ...")
|
||||
add_run_args(a)
|
||||
a.add_argument("--arm", action="append", required=True)
|
||||
a.add_argument("--repeat", type=int, default=3)
|
||||
c = sub.add_parser("compare")
|
||||
c.add_argument("a")
|
||||
c.add_argument("b")
|
||||
args = p.parse_args()
|
||||
# The agent keeps the last 4096 frames (Stats.swift) and results are read
|
||||
# at the end: at 60 fps that's 68 s, less warm-up and the reporting wait.
|
||||
if args.cmd != "compare" and args.host and args.usb:
|
||||
p.error("--host and --usb pick the path two ways; use one")
|
||||
if args.cmd != "compare" and args.warmup + args.duration > 60:
|
||||
p.error("--warmup plus --duration can be at most 60 s (the agent keeps 4096 frames)")
|
||||
|
||||
if args.cmd == "compare":
|
||||
regs = compare(load(args.a), load(args.b))
|
||||
if regs:
|
||||
print("\nworse:\n " + "\n ".join(regs))
|
||||
sys.exit(1 if regs else 0)
|
||||
|
||||
lab_agent()
|
||||
frame_ssh = frame_ssh_for(args)
|
||||
# Keep the Mac's screen awake: virtual displays aren't removed while it sleeps.
|
||||
runs = 1 if args.cmd == "run" else args.repeat * len(args.arm)
|
||||
awake = subprocess.Popen(["caffeinate", "-d", "-u", "-t", str(int(runs * (args.duration * 4 + 60) + 120))])
|
||||
try:
|
||||
if args.cmd == "ab":
|
||||
arm_runs, table = ab(args, frame_ssh)
|
||||
write(document(args, frame_ssh, [], arms=args.arm, runs=arm_runs, table=table), args)
|
||||
return
|
||||
results = run_suite(args, frame_ssh)
|
||||
finally:
|
||||
awake.terminate()
|
||||
doc = document(args, frame_ssh, results)
|
||||
write(doc, args)
|
||||
bad = [f"{r['scenario']} {k}" for r in results for k, v in r["grades"].items() if v in ("bad", "missing")]
|
||||
regs = compare(load(args.baseline), doc) if args.baseline else []
|
||||
if bad:
|
||||
print("missed targets: " + ", ".join(bad))
|
||||
if regs:
|
||||
print("worse than baseline:\n " + "\n ".join(regs))
|
||||
sys.exit(1 if bad or regs else 0)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/bin/sh
|
||||
# Publish a tested draft release so running copies of Frame Control offer it
|
||||
# (docs/releasing.md). Checks every installer is attached with a SHA-256
|
||||
# digest first, since the app's updater refuses assets without one, then
|
||||
# attaches update.json, the manifest the updater reads.
|
||||
# Usage: scripts/publish-release.sh v0.4.0
|
||||
set -eu
|
||||
tag="${1:?usage: $0 vX.Y.Z}"
|
||||
repo=saphid/frame-control
|
||||
expected="Frame-Control-mac-arm64.dmg Frame-Control-mac-arm64.zip Frame-Control-Setup-x64.exe
|
||||
Frame-Control-win-x64.zip Frame-Control-linux-x86_64.AppImage Frame-Control-linux-arm64.AppImage
|
||||
Frame-Control-linux-amd64.deb Frame-Control-linux-arm64.deb"
|
||||
|
||||
info=$(gh release view "$tag" -R "$repo" --json isDraft,isPrerelease,assets)
|
||||
version=$(sed -n 's/.*"version": *"\([^"]*\)".*/\1/p' "$(dirname "$0")/../app/package.json")
|
||||
[ "v$version" = "$tag" ] || echo "note: app/package.json here says $version (the release was built from the tag)"
|
||||
|
||||
missing=""
|
||||
for name in $expected; do
|
||||
digest=$(printf '%s' "$info" | python3 -c 'import json,sys
|
||||
d=json.load(sys.stdin); n=sys.argv[1]
|
||||
print(next((a.get("digest") or "" for a in d["assets"] if a["name"]==n), "absent"))' "$name")
|
||||
case "$digest" in
|
||||
sha256:*) echo "ok $name" ;;
|
||||
absent) echo "MISSING $name"; missing=1 ;;
|
||||
*) echo "NO HASH $name"; missing=1 ;;
|
||||
esac
|
||||
done
|
||||
[ -z "$missing" ] || { echo "not publishing: fix the assets above" >&2; exit 1; }
|
||||
|
||||
# update.json: what running copies read (app/updater.js), from github.com's
|
||||
# latest/download link rather than the rate-limited REST API.
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
gh release view "$tag" -R "$repo" --json tagName,url,body,assets | python3 -c 'import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
names=set(sys.argv[1].split())
|
||||
print(json.dumps({"version": d["tagName"].lstrip("v"), "page": d["url"], "notes": d["body"][:4000],
|
||||
"assets": [{"name": a["name"], "size": a["size"], "digest": a["digest"]}
|
||||
for a in d["assets"] if a["name"] in names]}, indent=1))' "$expected" > "$tmp/update.json"
|
||||
gh release upload "$tag" -R "$repo" "$tmp/update.json" --clobber
|
||||
echo "ok update.json"
|
||||
|
||||
gh release edit "$tag" -R "$repo" --draft=false --prerelease=false --latest
|
||||
echo "published $tag; running copies will offer it at their next check"
|
||||
@@ -0,0 +1,3 @@
|
||||
node_modules/
|
||||
.wrangler/
|
||||
.dev.vars
|
||||
@@ -0,0 +1,35 @@
|
||||
# Website
|
||||
|
||||
The Frame Control website, <https://frame-control.pages.dev>, on Cloudflare Pages.
|
||||
|
||||
- `public/`: static pages. `/` is the landing page, `/feedback/` the feedback form, `/privacy/` the privacy note.
|
||||
- `functions/api/feedback.js`: `POST /api/feedback`, which turns the form into a GitHub issue labelled `feedback`.
|
||||
- `lib/feedback.js`: validation and issue formatting, tested by `test/feedback.test.mjs`.
|
||||
- `public/js/site.js`: settings, including the Ko-fi page name for the donate buttons.
|
||||
|
||||
## Feedback → GitHub issues
|
||||
|
||||
The function needs a `GITHUB_TOKEN` secret: a fine-grained token with **Issues: read and write** on
|
||||
`saphid/frame-control` only. Issues are opened as the token's owner, so they pass the contributor gate
|
||||
(`.github/workflows/issue-gate.yml`) and stay open. Without the token the form answers 503 and offers a
|
||||
prefilled GitHub issue instead.
|
||||
|
||||
```sh
|
||||
cd site
|
||||
npx wrangler pages secret put GITHUB_TOKEN --project-name frame-control
|
||||
```
|
||||
|
||||
Spam protection: a hidden honeypot field, a 3-second minimum fill time, 5 submissions per hour per IP
|
||||
(a salted hash, kept in the `FEEDBACK_RL` KV namespace for about an hour), and 100 a day in total.
|
||||
User text has `@mentions` and `#123` references broken so nobody gets pinged.
|
||||
|
||||
## Run and deploy
|
||||
|
||||
```sh
|
||||
cd site
|
||||
node --test test/*.test.mjs
|
||||
npx wrangler pages dev --port 8788 # local; put GITHUB_TOKEN/GITHUB_REPO in .dev.vars to test issues
|
||||
npx wrangler pages deploy --branch main # production
|
||||
```
|
||||
|
||||
Point `GITHUB_REPO` in `.dev.vars` at a scratch repo when testing locally so test issues don't land on the real tracker.
|
||||
@@ -0,0 +1,86 @@
|
||||
// POST /api/feedback: turns the website's feedback form into a GitHub issue.
|
||||
//
|
||||
// Environment (Cloudflare Pages → Settings → Variables and Secrets):
|
||||
// GITHUB_TOKEN secret. Fine-grained token with Issues: read and write on GITHUB_REPO only.
|
||||
// GITHUB_REPO owner/name, e.g. saphid/frame-control (wrangler.toml sets it).
|
||||
// FEEDBACK_RL KV namespace binding for rate limits (optional; without it there is no limit).
|
||||
|
||||
import { buildIssue, hashIp, validate } from "../../lib/feedback.js";
|
||||
|
||||
const PER_IP_PER_HOUR = 5;
|
||||
const TOTAL_PER_DAY = 100;
|
||||
|
||||
const json = (status, data) =>
|
||||
new Response(JSON.stringify(data), {
|
||||
status,
|
||||
headers: { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" },
|
||||
});
|
||||
|
||||
async function overLimit(kv, key, limit, ttl) {
|
||||
const count = Number(await kv.get(key)) || 0;
|
||||
if (count >= limit) return true;
|
||||
await kv.put(key, String(count + 1), { expirationTtl: ttl });
|
||||
return false;
|
||||
}
|
||||
|
||||
export async function onRequestPost({ request, env }) {
|
||||
if (!env.GITHUB_TOKEN || !env.GITHUB_REPO) {
|
||||
return json(503, { error: "Feedback isn't connected to GitHub yet. Use the GitHub link instead." });
|
||||
}
|
||||
|
||||
const origin = request.headers.get("origin");
|
||||
if (origin && new URL(origin).host !== new URL(request.url).host) {
|
||||
return json(403, { error: "Send feedback from the website's form." });
|
||||
}
|
||||
|
||||
let input;
|
||||
try {
|
||||
input = await request.json();
|
||||
} catch {
|
||||
return json(400, { error: "Send the form as JSON." });
|
||||
}
|
||||
|
||||
const checked = validate(input);
|
||||
// Bots get a success-shaped answer so they don't learn what tripped them.
|
||||
if (checked.spam) return json(200, { ok: true });
|
||||
if (checked.error) return json(400, { error: checked.error });
|
||||
|
||||
// Best effort: KV is eventually consistent, so bursts can slip past, and a
|
||||
// storage error lets the feedback through rather than losing it.
|
||||
if (env.FEEDBACK_RL) try {
|
||||
const ip = request.headers.get("cf-connecting-ip") || "unknown";
|
||||
const hour = Math.floor(Date.now() / 3600e3);
|
||||
const day = Math.floor(Date.now() / 86400e3);
|
||||
// Salted with the secret token, so the stored hashes can't be reversed by trying every IP.
|
||||
const who = await hashIp(ip, env.GITHUB_TOKEN);
|
||||
if (await overLimit(env.FEEDBACK_RL, `ip:${who}:${hour}`, PER_IP_PER_HOUR, 3900)) {
|
||||
return json(429, { error: "That's a lot of feedback in one hour. Try again later, or use GitHub." });
|
||||
}
|
||||
if (await overLimit(env.FEEDBACK_RL, `day:${day}`, TOTAL_PER_DAY, 90000)) {
|
||||
return json(429, { error: "The form has had a busy day. Try again tomorrow, or use GitHub." });
|
||||
}
|
||||
} catch (err) {
|
||||
console.log(`Rate limit check failed: ${err}`);
|
||||
}
|
||||
|
||||
const res = await fetch(`https://api.github.com/repos/${env.GITHUB_REPO}/issues`, {
|
||||
method: "POST",
|
||||
headers: {
|
||||
authorization: `Bearer ${env.GITHUB_TOKEN}`,
|
||||
accept: "application/vnd.github+json",
|
||||
"x-github-api-version": "2022-11-28",
|
||||
"user-agent": "frame-control-website",
|
||||
"content-type": "application/json",
|
||||
},
|
||||
body: JSON.stringify(buildIssue(checked.value)),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
console.log(`GitHub answered ${res.status}: ${(await res.text()).slice(0, 500)}`);
|
||||
return json(502, { error: "GitHub didn't accept it just now. Try again, or use the GitHub link." });
|
||||
}
|
||||
const issue = await res.json();
|
||||
return json(201, { ok: true, number: issue.number, url: issue.html_url });
|
||||
}
|
||||
|
||||
export const onRequest = () => json(405, { error: "POST only." });
|
||||
@@ -0,0 +1,102 @@
|
||||
// Feedback form → GitHub issue. Pure functions, so tests can run them without
|
||||
// Cloudflare or GitHub (site/test/feedback.test.mjs).
|
||||
|
||||
export const KINDS = {
|
||||
bug: { label: "bug", title: "Bug report" },
|
||||
idea: { label: "enhancement", title: "Idea" },
|
||||
question: { label: "question", title: "Question" },
|
||||
other: { label: null, title: "Other feedback" },
|
||||
};
|
||||
|
||||
export const LIMITS = { title: [5, 120], message: [10, 5000], field: 120 };
|
||||
|
||||
// Anyone who fills the form in under this many milliseconds is a script.
|
||||
export const MIN_FILL_MS = 3000;
|
||||
|
||||
const GITHUB_LOGIN = /^[A-Za-z0-9](?:[A-Za-z0-9-]{0,37}[A-Za-z0-9])?$/;
|
||||
|
||||
const oneLine = (value, max) => String(value ?? "").replace(/\s+/g, " ").trim().slice(0, max);
|
||||
|
||||
// Mentions in someone else's text would ping strangers, and issue references
|
||||
// (#1, owner/repo#1, GH-1, github.com links) would add backlinks to other
|
||||
// people's issues, so break them all with a zero-width space. Escaping & first
|
||||
// stops @ and # from turning back into @ and # when GitHub renders,
|
||||
// escaping < keeps out raw HTML such as an unclosed <!-- comment, and doubling
|
||||
// backslashes stops GH\-1 or github\.com from being unescaped back into references.
|
||||
const ZWSP = "\u200b";
|
||||
export function defang(text) {
|
||||
return text
|
||||
.replace(/\\/g, "\\\\")
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/@(?=[A-Za-z0-9])/g, `@${ZWSP}`)
|
||||
.replace(/#(?=\d)/g, `#${ZWSP}`)
|
||||
.replace(/\b(GH)-(?=\d)/gi, `$1${ZWSP}-`)
|
||||
.replace(/\b(github)\.com/gi, `$1${ZWSP}.com`);
|
||||
}
|
||||
|
||||
// Returns { error } or { value } with every field trimmed and bounded.
|
||||
export function validate(input) {
|
||||
if (!input || typeof input !== "object") return { error: "Send the form as JSON." };
|
||||
if (oneLine(input.website, 200)) return { spam: true };
|
||||
// Measured in the browser with a monotonic clock, so clock skew doesn't matter.
|
||||
const elapsed = Number(input.elapsed);
|
||||
if (!Number.isFinite(elapsed)) return { spam: true };
|
||||
// The page waits this long before sending, so only scripts get here; say so anyway.
|
||||
if (elapsed < MIN_FILL_MS) return { error: "That was quick. Send it again in a moment." };
|
||||
|
||||
const kind = Object.hasOwn(KINDS, input.kind) ? input.kind : "other";
|
||||
const title = oneLine(input.title, LIMITS.title[1]);
|
||||
const message = String(input.message ?? "").replace(/\r\n?/g, "\n").trim();
|
||||
if (title.length < LIMITS.title[0]) return { error: "Give it a short title (at least 5 characters)." };
|
||||
if (message.length < LIMITS.message[0]) return { error: "Tell us a little more (at least 10 characters)." };
|
||||
if (message.length > LIMITS.message[1]) return { error: `Keep it under ${LIMITS.message[1]} characters.` };
|
||||
|
||||
const github = oneLine(input.github, 40).replace(/^@/, "");
|
||||
if (github && !GITHUB_LOGIN.test(github)) return { error: "That doesn't look like a GitHub username." };
|
||||
|
||||
return {
|
||||
value: {
|
||||
kind,
|
||||
title,
|
||||
message,
|
||||
github,
|
||||
version: oneLine(input.version, LIMITS.field),
|
||||
os: oneLine(input.os, LIMITS.field),
|
||||
steamos: oneLine(input.steamos, LIMITS.field),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export function buildIssue(value) {
|
||||
const kind = KINDS[value.kind];
|
||||
const details = [
|
||||
["Frame Control version", value.version],
|
||||
["Computer", value.os],
|
||||
["SteamOS build", value.steamos],
|
||||
].filter(([, v]) => v);
|
||||
|
||||
// Our own lines go first, so nothing in the sender's text can hide them.
|
||||
const lines = [
|
||||
value.github
|
||||
? `> Sent from the website feedback form by @${value.github}.`
|
||||
: "> Sent from the website feedback form. The sender left no GitHub username, so they won't see replies here.",
|
||||
"",
|
||||
];
|
||||
if (details.length) {
|
||||
lines.push("| | |", "|---|---|", ...details.map(([k, v]) => `| ${k} | ${defang(v).replace(/\|/g, "\\|")} |`), "");
|
||||
}
|
||||
lines.push(defang(value.message));
|
||||
|
||||
return {
|
||||
title: `${kind.title}: ${value.title}`,
|
||||
body: lines.join("\n"),
|
||||
labels: ["feedback", ...(kind.label ? [kind.label] : [])],
|
||||
};
|
||||
}
|
||||
|
||||
export async function hashIp(ip, salt) {
|
||||
const bytes = new TextEncoder().encode(`${salt}:${ip}`);
|
||||
const digest = await crypto.subtle.digest("SHA-256", bytes);
|
||||
return [...new Uint8Array(digest)].slice(0, 12).map((b) => b.toString(16).padStart(2, "0")).join("");
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
{ "type": "module", "private": true }
|
||||
@@ -0,0 +1,60 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Not found · Frame Control</title>
|
||||
<meta name="description" content="Page not found.">
|
||||
<meta name="theme-color" content="#0e141b">
|
||||
<link rel="icon" href="/favicon.png">
|
||||
<link rel="apple-touch-icon" href="/apple-touch-icon.png">
|
||||
<link rel="stylesheet" href="/css/site.css">
|
||||
<script src="/js/site.js" defer></script>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
<header class="top">
|
||||
<div class="wrap">
|
||||
<a class="brand" href="/"><img src="/img/icon.png" alt="">Frame Control</a>
|
||||
<nav aria-label="Sections">
|
||||
<a href="/#features">Features</a>
|
||||
<a href="/#setup">Setup</a>
|
||||
<a href="/#download">Download</a>
|
||||
<a href="/#faq">FAQ</a>
|
||||
<a href="/feedback/">Feedback</a>
|
||||
</nav>
|
||||
<div class="end">
|
||||
<a class="btn small ghost" href="https://github.com/saphid/frame-control">
|
||||
<svg viewBox="0 0 16 16" fill="currentColor" aria-hidden="true"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg>
|
||||
GitHub
|
||||
</a>
|
||||
<a class="btn small coffee" data-kofi href="#" target="_blank" rel="noopener">Support</a>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main class="page">
|
||||
<div class="wrap" style="text-align:center;padding:80px 24px">
|
||||
<span class="kicker">404</span>
|
||||
<h1>That page isn't here</h1>
|
||||
<p style="color:var(--muted);margin-top:14px">It may have moved. Try the home page, or tell us what you were looking for.</p>
|
||||
<div class="cta"><a class="btn primary" href="/">Home</a><a class="btn ghost" href="/feedback/">Send feedback</a></div>
|
||||
</div>
|
||||
</main>
|
||||
|
||||
<footer>
|
||||
<div class="wrap">
|
||||
<a class="brand" href="/"><img src="/img/icon.png" alt="">Frame Control</a>
|
||||
<div class="cols">
|
||||
<a href="https://github.com/saphid/frame-control">GitHub</a>
|
||||
<a href="https://github.com/saphid/frame-control/releases">Releases</a>
|
||||
<a href="https://github.com/saphid/frame-control/blob/main/docs/frame-control.md">Docs</a>
|
||||
<a href="/feedback/">Feedback</a>
|
||||
<a href="https://github.com/saphid/frame-control/blob/main/CONTRIBUTING.md">Contributing</a>
|
||||
<a href="/privacy/">Privacy</a>
|
||||
</div>
|
||||
<p class="legal">© <span data-year>2026</span> saphid · MIT licence. Unofficial and not affiliated with or endorsed by Valve. Steam, Steam Frame and SteamVR are trademarks of Valve Corporation.</p>
|
||||
</div>
|
||||
</footer>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,15 @@
|
||||
/*
|
||||
X-Content-Type-Options: nosniff
|
||||
Referrer-Policy: strict-origin-when-cross-origin
|
||||
X-Frame-Options: DENY
|
||||
Permissions-Policy: camera=(), microphone=(), geolocation=()
|
||||
Content-Security-Policy: default-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; media-src 'self'; connect-src 'self' https://api.github.com; frame-ancestors 'none'; base-uri 'self'; form-action 'self'
|
||||
|
||||
/css/*
|
||||
Cache-Control: public, max-age=3600
|
||||
/js/*
|
||||
Cache-Control: public, max-age=3600
|
||||
/img/*
|
||||
Cache-Control: public, max-age=86400
|
||||
/media/*
|
||||
Cache-Control: public, max-age=604800
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
@@ -0,0 +1,201 @@
|
||||
/* Frame Control website. Colours follow the app (ui/index.html): Steam's navy and blue. */
|
||||
:root {
|
||||
--bg: #0e141b; --bg-2: #131c26; --panel: #17222e; --panel-2: #1d2a38; --line: rgba(143, 152, 160, .16);
|
||||
--text: #c7d0d8; --bright: #fff; --muted: #8f98a0; --dim: #5e6873;
|
||||
--blue: #1a9fff; --link: #66c0f4; --green: #75b022; --warn: #d9a23a; --bad: #e0573c;
|
||||
--action: linear-gradient(to right, #47bfff 5%, #1a44c2 95%);
|
||||
--action-hi: linear-gradient(to right, #5fcaff 5%, #2458d6 95%);
|
||||
--radius: 14px; --wrap: 1160px;
|
||||
--font: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, Arial, sans-serif;
|
||||
color-scheme: dark;
|
||||
}
|
||||
* { box-sizing: border-box; }
|
||||
html { scroll-behavior: smooth; scroll-padding-top: 80px; }
|
||||
body { margin: 0; background: var(--bg); color: var(--text); font: 16px/1.6 var(--font); -webkit-font-smoothing: antialiased; }
|
||||
img, video { max-width: 100%; height: auto; display: block; }
|
||||
a { color: var(--link); text-decoration: none; }
|
||||
a:hover { color: var(--bright); }
|
||||
::selection { background: var(--blue); color: #fff; }
|
||||
:focus-visible { outline: 2px solid var(--link); outline-offset: 3px; border-radius: 4px; }
|
||||
h1, h2, h3 { color: var(--bright); line-height: 1.15; margin: 0; letter-spacing: -.02em; }
|
||||
h1 { font-size: clamp(38px, 6vw, 64px); font-weight: 800; }
|
||||
h2 { font-size: clamp(28px, 3.6vw, 40px); font-weight: 750; }
|
||||
h3 { font-size: 18px; font-weight: 650; letter-spacing: -.01em; }
|
||||
p { margin: 0; }
|
||||
code { font: 14px ui-monospace, SFMono-Regular, Menlo, monospace; background: rgba(255,255,255,.06); padding: 2px 6px; border-radius: 5px; color: var(--bright); }
|
||||
.wrap { max-width: var(--wrap); margin: 0 auto; padding: 0 24px; }
|
||||
.sr-only { position: absolute; width: 1px; height: 1px; overflow: hidden; clip: rect(0 0 0 0); white-space: nowrap; }
|
||||
[hidden] { display: none !important; }
|
||||
|
||||
/* ---- header ---- */
|
||||
.top { position: sticky; top: 0; z-index: 20; background: rgba(14, 20, 27, .78); backdrop-filter: saturate(160%) blur(14px);
|
||||
-webkit-backdrop-filter: saturate(160%) blur(14px); border-bottom: 1px solid var(--line); }
|
||||
.top .wrap { display: flex; align-items: center; gap: 28px; height: 64px; }
|
||||
.brand { display: flex; align-items: center; gap: 10px; color: var(--bright); font-weight: 700; letter-spacing: 2.2px; font-size: 14px; text-transform: uppercase; }
|
||||
.brand { white-space: nowrap; }
|
||||
.brand img { width: 30px; height: 30px; }
|
||||
.top nav { display: flex; gap: 22px; margin-left: 8px; }
|
||||
.top nav a { color: var(--muted); font-size: 14.5px; font-weight: 500; }
|
||||
.top nav a:hover, .top nav a[aria-current] { color: var(--bright); }
|
||||
.top .end { margin-left: auto; display: flex; gap: 10px; align-items: center; }
|
||||
@media (max-width: 880px) { .top nav { display: none; } }
|
||||
@media (max-width: 480px) { .top .end .ghost { display: none; } }
|
||||
|
||||
/* ---- buttons ---- */
|
||||
.btn { display: inline-flex; align-items: center; justify-content: center; gap: 9px; height: 46px; padding: 0 22px; border-radius: 10px;
|
||||
font: 600 15.5px var(--font); color: var(--bright); background: rgba(103, 112, 123, .22); border: 1px solid transparent;
|
||||
cursor: pointer; transition: background .15s, transform .15s, box-shadow .15s; white-space: nowrap; }
|
||||
.btn:hover { background: rgba(103, 112, 123, .4); color: var(--bright); }
|
||||
.btn.primary { background: var(--action); box-shadow: 0 8px 28px rgba(26, 159, 255, .28); }
|
||||
.btn.primary:hover { background: var(--action-hi); transform: translateY(-1px); }
|
||||
/* The background shorthand resets this; without it the gradient repeats under the transparent border. */
|
||||
.btn.primary, .btn.primary:hover { background-origin: border-box; }
|
||||
.btn.ghost { background: transparent; border-color: var(--line); }
|
||||
.btn.ghost:hover { border-color: rgba(143,152,160,.4); background: rgba(255,255,255,.03); }
|
||||
.btn.small { height: 36px; padding: 0 14px; font-size: 14px; border-radius: 8px; }
|
||||
.btn.coffee { background: #ff5e5b; box-shadow: 0 8px 24px rgba(255, 94, 91, .22); }
|
||||
.btn.coffee:hover { background: #ff7471; }
|
||||
.btn svg { width: 18px; height: 18px; flex: none; }
|
||||
.btn[disabled] { opacity: .6; cursor: progress; transform: none; }
|
||||
|
||||
/* ---- hero ---- */
|
||||
.hero { position: relative; padding: 88px 0 40px; overflow: hidden; text-align: center; }
|
||||
.hero::before { content: ""; position: absolute; inset: -30% -10% auto; height: 900px; pointer-events: none;
|
||||
background: radial-gradient(600px 380px at 30% 30%, rgba(26,159,255,.22), transparent 70%),
|
||||
radial-gradient(520px 360px at 72% 20%, rgba(111,66,193,.2), transparent 70%); }
|
||||
.hero > * { position: relative; }
|
||||
.eyebrow { display: inline-block; max-width: 100%; padding: 6px 14px; border-radius: 999px; font-size: 13.5px;
|
||||
color: var(--link); background: rgba(26,159,255,.1); border: 1px solid rgba(102,192,244,.22); margin-bottom: 26px; }
|
||||
.eyebrow b { color: var(--bright); font-weight: 600; }
|
||||
.eyebrow .plats { white-space: nowrap; }
|
||||
@media (max-width: 520px) { .eyebrow { border-radius: 16px; } .eyebrow .sep { display: none; } .eyebrow .plats { display: block; white-space: normal; } }
|
||||
.hero h1 { max-width: 880px; margin: 0 auto; }
|
||||
.hero h1 span { background: linear-gradient(90deg, #66c0f4, #1a9fff 45%, #8a6cff); -webkit-background-clip: text; background-clip: text; color: transparent; }
|
||||
.lede { max-width: 680px; margin: 22px auto 0; font-size: 19px; color: var(--text); }
|
||||
.cta { display: flex; flex-wrap: wrap; justify-content: center; gap: 12px; margin-top: 34px; }
|
||||
.fine { margin-top: 16px; font-size: 13.5px; color: var(--muted); }
|
||||
.fine a { color: var(--muted); text-decoration: underline; text-underline-offset: 3px; }
|
||||
.shot { margin: 64px auto 0; max-width: 1080px; border-radius: var(--radius); overflow: hidden; border: 1px solid var(--line);
|
||||
box-shadow: 0 50px 120px rgba(0,0,0,.55), 0 0 0 1px rgba(255,255,255,.02), 0 0 120px rgba(26,159,255,.12); }
|
||||
.shot .bar { display: flex; gap: 7px; padding: 12px 14px; background: #10171f; border-bottom: 1px solid var(--line); }
|
||||
.shot .bar i { width: 11px; height: 11px; border-radius: 50%; background: #2a3440; }
|
||||
|
||||
/* ---- sections ---- */
|
||||
section { padding: 96px 0; }
|
||||
section.alt { background: var(--bg-2); border-block: 1px solid var(--line); }
|
||||
.head { max-width: 700px; margin: 0 auto 52px; text-align: center; }
|
||||
.kicker { display: block; color: var(--link); font-size: 13px; font-weight: 650; letter-spacing: 2px; text-transform: uppercase; margin-bottom: 12px; }
|
||||
.head p { margin-top: 14px; font-size: 17.5px; color: var(--muted); }
|
||||
|
||||
.video { max-width: 1000px; margin: 0 auto; border-radius: var(--radius); overflow: hidden; border: 1px solid var(--line); background: #000;
|
||||
box-shadow: 0 40px 100px rgba(0,0,0,.5); }
|
||||
.video video { width: 100%; aspect-ratio: 16 / 9; }
|
||||
|
||||
.grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(250px, 1fr)); gap: 16px; }
|
||||
.card { background: var(--panel); border: 1px solid var(--line); border-radius: var(--radius); padding: 24px; transition: border-color .2s, transform .2s; }
|
||||
.card:hover { border-color: rgba(102,192,244,.3); transform: translateY(-2px); }
|
||||
.card .ico { width: 42px; height: 42px; border-radius: 10px; display: grid; place-items: center; margin-bottom: 16px;
|
||||
background: rgba(26,159,255,.12); color: var(--link); }
|
||||
.card .ico svg { width: 22px; height: 22px; }
|
||||
.card p { margin-top: 8px; color: var(--muted); font-size: 15px; }
|
||||
|
||||
.feature { display: grid; grid-template-columns: 1fr 1.35fr; gap: 56px; align-items: center; }
|
||||
.feature + .feature { margin-top: 96px; }
|
||||
.feature.flip { grid-template-columns: 1.35fr 1fr; }
|
||||
.feature.flip .copy { order: 2; }
|
||||
.feature .copy h2 { margin-top: 0; }
|
||||
.feature .copy p { margin-top: 16px; color: var(--muted); font-size: 17px; }
|
||||
.feature ul { margin: 20px 0 0; padding: 0; list-style: none; display: grid; gap: 10px; }
|
||||
.feature li { padding-left: 28px; position: relative; color: var(--text); }
|
||||
.feature li::before { content: ""; position: absolute; left: 2px; top: 7px; width: 14px; height: 8px; border-left: 2px solid var(--link);
|
||||
border-bottom: 2px solid var(--link); transform: rotate(-45deg); }
|
||||
.feature img { border-radius: 12px; border: 1px solid var(--line); box-shadow: 0 30px 80px rgba(0,0,0,.45); }
|
||||
@media (max-width: 880px) { .feature, .feature.flip { grid-template-columns: 1fr; gap: 28px; } .feature.flip .copy { order: 0; } }
|
||||
|
||||
.phones { display: flex; justify-content: center; gap: 28px; margin-top: 8px; }
|
||||
.phones img { width: 260px; border-radius: 34px; border: 8px solid #0a0f15; box-shadow: 0 30px 80px rgba(0,0,0,.5), 0 0 0 1px var(--line); }
|
||||
.phones img:last-child { transform: translateY(40px); }
|
||||
@media (max-width: 600px) { .phones img { width: 44%; border-width: 5px; border-radius: 22px; } }
|
||||
|
||||
.steps { display: grid; grid-template-columns: repeat(3, 1fr); gap: 16px; counter-reset: step; }
|
||||
.steps .card { position: relative; padding-top: 64px; }
|
||||
.steps .card::before { counter-increment: step; content: counter(step); position: absolute; top: 22px; left: 24px; width: 30px; height: 30px;
|
||||
border-radius: 50%; display: grid; place-items: center; font-weight: 700; font-size: 14px; color: var(--bright); background: var(--action); }
|
||||
@media (max-width: 880px) { .steps { grid-template-columns: 1fr; } }
|
||||
|
||||
.downloads { display: grid; grid-template-columns: repeat(auto-fit, minmax(240px, 1fr)); gap: 16px; }
|
||||
.dl { display: flex; flex-direction: column; gap: 6px; background: var(--panel); border: 1px solid var(--line); border-radius: var(--radius); padding: 26px; }
|
||||
.dl.mine { border-color: rgba(26,159,255,.55); box-shadow: 0 0 0 1px rgba(26,159,255,.25), 0 20px 50px rgba(26,159,255,.1); }
|
||||
.dl .os { display: flex; align-items: center; gap: 10px; }
|
||||
.dl .os svg { width: 24px; height: 24px; color: var(--bright); }
|
||||
.dl .need { color: var(--muted); font-size: 14px; }
|
||||
.dl .links { display: flex; flex-direction: column; gap: 8px; margin-top: 16px; }
|
||||
.dl .links .btn { width: 100%; }
|
||||
.dl .tag { margin-left: auto; font-size: 11.5px; font-weight: 650; letter-spacing: 1px; text-transform: uppercase; color: var(--link); }
|
||||
|
||||
.faq { max-width: 780px; margin: 0 auto; display: grid; gap: 10px; }
|
||||
.faq details { background: var(--panel); border: 1px solid var(--line); border-radius: 12px; padding: 0 22px; }
|
||||
.faq summary { cursor: pointer; list-style: none; padding: 18px 0; color: var(--bright); font-weight: 600; display: flex; justify-content: space-between; gap: 16px; }
|
||||
.faq summary::-webkit-details-marker { display: none; }
|
||||
.faq summary::after { content: "+"; color: var(--muted); font-size: 22px; line-height: 1; transition: transform .2s; }
|
||||
.faq details[open] summary::after { transform: rotate(45deg); }
|
||||
.faq details > div { padding: 0 0 20px; color: var(--muted); }
|
||||
.faq details > div p + p { margin-top: 10px; }
|
||||
.faq.notes section { background: var(--panel); border: 1px solid var(--line); border-radius: 12px; padding: 18px 22px 20px; color: var(--muted); }
|
||||
.faq.notes h2 { font-size: 16px; font-weight: 600; letter-spacing: 0; color: var(--bright); margin: 0 0 10px; }
|
||||
|
||||
.split { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; }
|
||||
.split .card { padding: 36px; }
|
||||
.split .card h3 { font-size: 24px; }
|
||||
.split .card p { font-size: 16px; margin: 12px 0 24px; }
|
||||
@media (max-width: 760px) { .split { grid-template-columns: 1fr; } }
|
||||
|
||||
/* ---- footer ---- */
|
||||
footer { border-top: 1px solid var(--line); padding: 48px 0 56px; color: var(--dim); font-size: 14px; }
|
||||
footer .wrap { display: flex; flex-wrap: wrap; gap: 24px 48px; justify-content: space-between; }
|
||||
footer .cols { display: flex; gap: 12px 28px; flex-wrap: wrap; }
|
||||
footer a { color: var(--muted); }
|
||||
footer .legal { flex-basis: 100%; font-size: 13px; }
|
||||
|
||||
/* ---- feedback page ---- */
|
||||
.page { padding: 72px 0 96px; }
|
||||
.page .head { text-align: left; margin: 0 0 36px; max-width: none; }
|
||||
.form-wrap { display: grid; grid-template-columns: 1fr 320px; gap: 28px; align-items: start; }
|
||||
@media (max-width: 900px) { .form-wrap { grid-template-columns: 1fr; } }
|
||||
.panel { background: var(--panel); border: 1px solid var(--line); border-radius: var(--radius); padding: 30px; }
|
||||
aside.panel h3 { margin-bottom: 10px; }
|
||||
aside.panel p { color: var(--muted); font-size: 15px; }
|
||||
aside.panel p + h3 { margin-top: 26px; }
|
||||
.field { display: grid; gap: 8px; margin-bottom: 22px; }
|
||||
.field > label, .field > legend { color: var(--bright); font-weight: 600; font-size: 14.5px; padding: 0; }
|
||||
.field > legend { margin-bottom: 10px; } /* fieldset grids ignore gap for the legend */
|
||||
.field small { color: var(--muted); font-size: 13px; font-weight: 400; }
|
||||
.row3 { display: grid; grid-template-columns: repeat(3, 1fr); gap: 14px; }
|
||||
@media (max-width: 640px) { .row3 { grid-template-columns: 1fr; } }
|
||||
fieldset { border: 0; padding: 0; margin: 0 0 22px; min-width: 0; }
|
||||
input[type=text], textarea { width: 100%; font: 15.5px/1.5 var(--font); color: var(--bright); background: #0f161e; border: 1px solid rgba(143,152,160,.22);
|
||||
border-radius: 9px; padding: 11px 13px; transition: border-color .15s, box-shadow .15s; }
|
||||
input[type=text]::placeholder, textarea::placeholder { color: var(--dim); }
|
||||
input[type=text]:focus, textarea:focus { outline: none; border-color: var(--blue); box-shadow: 0 0 0 3px rgba(26,159,255,.2); }
|
||||
textarea { min-height: 180px; resize: vertical; }
|
||||
.kinds { display: grid; grid-template-columns: repeat(4, 1fr); gap: 8px; }
|
||||
@media (max-width: 640px) { .kinds { grid-template-columns: repeat(2, 1fr); } }
|
||||
.kinds label { position: relative; cursor: pointer; }
|
||||
.kinds input { position: absolute; opacity: 0; inset: 0; pointer-events: none; }
|
||||
.kinds span { display: flex; flex-direction: column; align-items: center; gap: 6px; padding: 14px 8px; border-radius: 10px; font-size: 14px; font-weight: 600;
|
||||
color: var(--muted); background: #0f161e; border: 1px solid rgba(143,152,160,.18); transition: all .15s; }
|
||||
.kinds span svg { width: 20px; height: 20px; }
|
||||
.kinds input:checked + span { color: var(--bright); border-color: var(--blue); background: rgba(26,159,255,.1); }
|
||||
.kinds input:focus-visible + span { outline: 2px solid var(--link); outline-offset: 2px; }
|
||||
.trap { position: absolute; left: -9999px; width: 1px; height: 1px; overflow: hidden; }
|
||||
.actions { display: flex; flex-wrap: wrap; align-items: center; gap: 14px; margin-top: 8px; }
|
||||
.note { color: var(--muted); font-size: 13.5px; }
|
||||
.alert { border-radius: 10px; padding: 14px 16px; margin-bottom: 20px; font-size: 15px; }
|
||||
.alert.err { background: rgba(224,87,60,.12); border: 1px solid rgba(224,87,60,.4); color: #ffb4a6; }
|
||||
.alert.err a { color: #ffd2c9; text-decoration: underline; }
|
||||
.done { text-align: center; padding: 40px 10px; }
|
||||
.done .tick { width: 64px; height: 64px; border-radius: 50%; margin: 0 auto 20px; display: grid; place-items: center; background: rgba(117,176,34,.15); color: #a4d007; }
|
||||
.done .tick svg { width: 30px; height: 30px; }
|
||||
.done p { color: var(--muted); margin: 10px auto 24px; max-width: 460px; }
|
||||
.done .cta { margin-top: 0; }
|
||||
.page h1 { font-size: clamp(34px, 5vw, 48px); }
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 4.9 KiB |
Loaded 100 of 150 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user