Merge pull request #17 from saphid/analytics-and-updates

Analytics, self-update and Report a problem

docs/frame-control.md

@@ -56,9 +56,11 @@ counts them while they run.
   Steam library), then launch, stop, test or remove it. **Report an APK** records
   whether any APK worked (F-Droid or not: pick a file, type a package, or use an
   installed app). Your reports are saved on your computer and change the verdicts
-  you see. They aren't uploaded anywhere: the shared database is maintainer-only
-  for now (see [compat-db/README.md](../compat-db/README.md)). Uses the app's bundled
-  `adb`, or yours if you have one.
+  you see. With **Share compatibility results** on (Privacy & updates), they also
+  go to the shared database ([privacy.md](privacy.md),
+  [compat-db/README.md](../compat-db/README.md)). A failed install records
+  itself when the APK was the problem, and after an install the app offers a
+  20-second test. Uses the app's bundled `adb`, or yours if you have one.
 - **Android display**: pick a running Lepton instance (by the app in it) and set
   its resolution (Native 1920×1080, or Sharp 2560×1440 with density scaled to
   match), UI scale (Smaller / Default / Larger, or an exact dpi) and text size
@@ -148,8 +150,9 @@ npm run dist:win       # Windows: installer and .zip
 npm run dist:linux     # Linux: AppImage and .deb, x64 and arm64
 ```
 
-Pushing a `v*` tag builds all three in GitHub Actions and attaches them to the
-release (`.github/workflows/release.yml`).
+Pushing a `v*` tag builds all three in GitHub Actions and attaches them to a
+draft release (`.github/workflows/release.yml`). Running copies are offered it
+once you publish it: see [releasing.md](releasing.md).
 
 ## AI agents and assistant
 

docs/privacy.md

@@ -0,0 +1,143 @@
+# Privacy and analytics
+
+Frame Control sends anonymous analytics to [PostHog](https://posthog.com)
+(US cloud) so the maintainer can see how many people use it, which features
+matter and where installs fail. You choose how much in **Privacy & updates**,
+the last panel on the page. `ui/frame_telemetry.py` is the whole
+implementation.
+
+## The three levels
+
+| Level | Default | What it sends |
+|---|---|---|
+| Anonymous usage statistics | On, after a notice on first run | The events in the table below |
+| Share compatibility results | Off | Your Android compatibility reports and tests |
+| Send error details | Off | Scrubbed error messages and tracebacks |
+
+Nothing is sent until the first-run notice has been shown. The notice's
+**Share more to help fix problems** button turns on the second and third
+levels together. Either can be turned off later. Turning a level off
+deletes that level's events that haven't been sent yet.
+
+**Show what's been sent** in the panel lists the last 50 events that left your
+computer, exactly as they were sent.
+
+## Anonymous
+
+- Events carry a random id, made when Frame Control first runs and kept in
+  its data folder (`telemetry/settings.json`). It isn't derived from your
+  computer, account or network. To get a new one, delete that file.
+- Events are sent without person profiles (`$process_person_profile: false`)
+  and without location lookup (`$geoip_disable: true`). Each carries a
+  placeholder address (`$ip: 0.0.0.0`), so PostHog stores that instead of
+  yours.
+- Every event includes the app version, OS name (macOS, Windows or Linux),
+  CPU architecture and Python version.
+
+## Usage events
+
+| Event | When | Properties besides the common ones |
+|---|---|---|
+| `app_installed` | First run | |
+| `app_updated` | First run of a new version | `from_version` |
+| `app_opened` | At most once a day | |
+| `frame_connected` | The first time a SteamOS build is seen | `steamos_build`, `steamos_version` |
+| `tab_viewed` | The first click on each tab in a session | `tab` |
+| `install_finished` | Any install finishes, working or not | `kind` (apk, flatpak, steam, title, web), `ok`, `seconds`, `error_category`, `installer_code`, and see below |
+| `update_offered`, `update_started`, `update_failed` | The update banner | `to_version`, `error_category` |
+
+`install_finished` never includes a file name, path or error message. An
+error becomes one category from a fixed list (for example `apk_wrong_abi` or
+`frame_unreachable`), plus Android's own `INSTALL_FAILED_…` code when there
+is one. It names what was installed only when that's already public:
+
+- F-Droid catalogue apps: `package`. Never the version, since a local build can reuse a
+  catalogue app's package name
+- Flathub apps: `flatpak_id`
+- Steam games: `steam_appid`
+- A sideloaded title: only its runtime (Proton or Linux)
+
+Any other APK is sent as `catalog: false`, with no name.
+
+## Compatibility results (opt-in)
+
+Each report becomes a `compat_report` event with the fields the Report dialog
+shows: package, version, result or rating, your notes, how it was run, and the
+SteamOS and Lepton builds. Before sending:
+
+- the notes, app name and version are scrubbed like error messages (see
+  below)
+- the APK's source is kept only if it's `F-Droid` or the public host name of
+  a download link (`https://example.com/…`). File names, user names,
+  passwords, ports, paths, IP addresses and local host names are dropped
+
+When you turn this on, reports you made earlier on this computer are shared
+too.
+
+The maintainer's `python3 ui/frame_compat_db.py sync` copies these events
+into the compatibility database, marked `via=community…`. It takes at most
+30 per reporter per day.
+
+## Error details (opt-in)
+
+`$exception` events carry an error message, the Frame Control file, line and
+function it came from, and the request that failed (for example
+`POST /api/android install`). Before anything is sent, the message is
+scrubbed:
+
+- your home folder becomes `~`, and any user name becomes `<user>`
+- IP and MAC addresses, email addresses, `.local`, `.lan` and Tailscale host
+  names, Steam ids, SSH and PEM keys, API tokens and long hex strings are
+  replaced
+- URLs are cut down to their scheme and a public host name, or `<url>`. User
+  names, passwords, ports, paths and queries are dropped
+- `token=`, `key=`, `password=` and similar values are replaced
+
+The same error is sent at most once every 10 minutes.
+
+## Report a problem
+
+**Report a problem** is the warning-sign button in the header, also in the
+Privacy panel and under **Help → Report a Problem…**. It sends the report
+privately to Frame Control's PostHog project as a `problem_report` event, the
+same way as the analytics above, so only the maintainer can read it and
+nothing is published. It works whatever the analytics settings are, because
+the person sends it deliberately. The report has the kind, title and text you
+wrote, how to reach you if you gave it, a short reference shown after sending,
+and the diagnostics below. It has its own random id, so it isn't linked to
+your analytics events.
+
+With **Include diagnostics** ticked (the default), the report adds:
+
+- the app version and whether it's a built app
+- the OS, its release and CPU, and the Python version
+- the Frame's SteamOS build, if it has connected since the app started
+- which analytics levels are on
+
+**Also include recent activity and the server log** is off by default,
+because those lines can name files and apps. When ticked, it adds the newest
+Activity lines and server log lines, without the request lines.
+
+Everything is scrubbed like error details and limited to what fits in the
+report. Environment details are kept first, then the newest lines. **Show
+exactly what's included** shows the snapshot that will be sent, and later
+activity isn't added to it. If PostHog can't be reached, **Copy report** puts
+the whole report on the clipboard.
+
+The maintainer reads reports on the Frame Control dashboard in PostHog, or
+with `python3 ui/frame_report.py inbox [days]`, which uses the same personal
+API key as `frame_compat_db.py sync`.
+
+## Turning it all off
+
+Untick the boxes, or set `DO_NOT_TRACK=1` or `FRAME_CONTROL_TELEMETRY=0` in
+the environment that starts Frame Control. A copy run from a source checkout
+never sends anything unless `FRAME_CONTROL_TELEMETRY=1` is set.
+
+## Update checks
+
+The desktop app asks GitHub for the latest release shortly after starting,
+then every 6 hours: the latest release's `update.json` on GitHub, or
+`api.github.com/repos/saphid/frame-control/releases/latest` if that fails.
+Those requests carry no id. To stop it, set
+`FRAME_CONTROL_NO_UPDATE_CHECK=1`. See [releasing.md](releasing.md).

docs/releasing.md

@@ -0,0 +1,59 @@
+# Releasing and updates
+
+Frame Control checks for updates itself. The desktop app offers a new version
+only once it's GitHub's **latest release**, and drafts and pre-releases never
+count. So a build reaches people only when you publish it, after testing it.
+
+## Steps
+
+1. Bump `version` in `app/package.json`, commit, and push a tag:
+
+   ```sh
+   git tag v0.4.0 && git push origin v0.4.0
+   ```
+
+   `.github/workflows/release.yml` builds macOS, Windows and Linux, and
+   attaches everything to a **draft** release for that tag. Nobody is
+   offered a draft.
+
+2. Download the draft's installers and test them. An installed copy of the
+   previous version won't offer the draft, so install it directly.
+
+3. Write the release notes on the draft. The update banner links to them.
+
+4. Publish:
+
+   ```sh
+   scripts/publish-release.sh v0.4.0
+   ```
+
+   The script checks that all eight installers are attached, each with the
+   SHA-256 digest GitHub records. It attaches `update.json` (the version, the
+   notes and each installer's digest), then publishes the release and marks it
+   latest. From then on, running copies see the update. They check about 8
+   seconds after starting, then every 6 hours, and anyone can use **Check for
+   Updates…** (the app menu on macOS, the Help menu elsewhere).
+
+To pull a bad release, mark the previous one as latest
+(`gh release edit v0.3.9 --latest`) or turn the bad one back into a draft.
+Copies that already updated stay on it. Nothing downgrades them.
+
+## How a copy updates itself
+
+`app/updater.js` reads `update.json` from
+`github.com/saphid/frame-control/releases/latest/download/`. It falls back to the
+REST API only when a release has no manifest, because the API allows just 60
+unauthenticated requests an hour per IP address, shared by a whole household.
+Then it downloads the installer for its platform and checks it
+against the SHA-256 digest GitHub publishes for the asset. It refuses if the
+digest is missing or doesn't match. Then:
+
+| Installed from | Update |
+|---|---|
+| macOS `.dmg`, app in a writable folder such as Applications | The `.zip` is unpacked next to the app and its version checked. After the app quits, a small script swaps the new app in, putting the old one back if that fails, and reopens it. Updates don't get the download quarantine, so there's no `xattr` step. |
+| Windows installer | The new `Setup` runs silently over the install (`/S --force-run`) and reopens the app. |
+| Linux AppImage | The new AppImage replaces the old file and is started. |
+| macOS app still on the disk image or translocated, Windows `.zip`, Linux `.deb` | The banner opens the release page instead. |
+
+Version 0.3.1 and earlier have no updater, so people on them have to download
+the new version once by hand.