Send analytics to the existing PostHog project; make bug reports private

- Analytics go to the maintainer's PostHog US project 343535, tagged
  $lib = frame-control. Every event carries $ip 0.0.0.0, since PostHog
  stores the sender's address otherwise (checked live), including events
  queued by earlier versions.
- Report a problem sends a private problem_report event to PostHog instead
  of a public GitHub issue, with its own random id so a contact address
  can't be linked to analytics. The dialog asks how to reach the person and
  shows a reference. Maintainers read reports on the PostHog dashboard or
  with `python3 ui/frame_report.py inbox`.
- Community sync pages by timestamp in UTC: PostHog refuses OFFSET for
  personal API keys.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
saphidandClaude Opus 5.5 committed 2026-09-28 20:14:36 +10:00
1 parent e67802f15d
commit f076527722
8 files changed
+219 -132

No files matched your search

+1 -1
View File
@@ -182,7 +182,7 @@ This is a first public test, so reports are really useful, especially from
Windows and Linux. The quickest way is **Report a problem** in the app (the Windows and Linux. The quickest way is **Report a problem** in the app (the
warning-sign button at the top, or **Help → Report a Problem…**). It adds warning-sign button at the top, or **Help → Report a Problem…**). It adds
diagnostics with personal details removed, shows you exactly what's included, diagnostics with personal details removed, shows you exactly what's included,
and opens an issue here. You don't need a GitHub account. Without the app, and sends it privately to the maintainer; nothing is published. Without the app,
use the [feedback form](https://frame-control.pages.dev/feedback/). Please include: use the [feedback form](https://frame-control.pages.dev/feedback/). Please include:
- what you tried and what happened - what you tried and what happened
+22 -12
View File
@@ -1,7 +1,7 @@
# Privacy and analytics # Privacy and analytics
Frame Control sends anonymous analytics to [PostHog](https://posthog.com) Frame Control sends anonymous analytics to [PostHog](https://posthog.com)
(EU cloud) so the maintainer can see how many people use it, which features (US cloud) so the maintainer can see how many people use it, which features
matter and where installs fail. You choose how much in **Privacy & updates**, matter and where installs fail. You choose how much in **Privacy & updates**,
the last panel on the page. `ui/frame_telemetry.py` is the whole the last panel on the page. `ui/frame_telemetry.py` is the whole
implementation. implementation.
@@ -28,8 +28,9 @@ computer, exactly as they were sent.
its data folder (`telemetry/settings.json`). It isn't derived from your its data folder (`telemetry/settings.json`). It isn't derived from your
computer, account or network. To get a new one, delete that file. computer, account or network. To get a new one, delete that file.
- Events are sent without person profiles (`$process_person_profile: false`) - Events are sent without person profiles (`$process_person_profile: false`)
and without location lookup (`$geoip_disable: true`). The PostHog project and without location lookup (`$geoip_disable: true`). Each carries a
is also set to discard client IP addresses. placeholder address (`$ip: 0.0.0.0`), so PostHog stores that instead of
yours.
- Every event includes the app version, OS name (macOS, Windows or Linux), - Every event includes the app version, OS name (macOS, Windows or Linux),
CPU architecture and Python version. CPU architecture and Python version.
@@ -97,10 +98,14 @@ The same error is sent at most once every 10 minutes.
## Report a problem ## Report a problem
**Report a problem** is the warning-sign button in the header, also in the **Report a problem** is the warning-sign button in the header, also in the
Privacy panel and under **Help → Report a Problem…**. It files a public issue Privacy panel and under **Help → Report a Problem…**. It sends the report
on [GitHub](https://github.com/saphid/frame-control/issues) through the privately to Frame Control's PostHog project as a `problem_report` event, the
website's feedback service, so no GitHub account is needed. It works whatever same way as the analytics above, so only the maintainer can read it and
the analytics settings are, because the person sends it deliberately. nothing is published. It works whatever the analytics settings are, because
the person sends it deliberately. The report has the kind, title and text you
wrote, how to reach you if you gave it, a short reference shown after sending,
and the diagnostics below. It has its own random id, so it isn't linked to
your analytics events.
With **Include diagnostics** ticked (the default), the report adds: With **Include diagnostics** ticked (the default), the report adds:
@@ -114,10 +119,14 @@ because those lines can name files and apps. When ticked, it adds the newest
Activity lines and server log lines, without the request lines. Activity lines and server log lines, without the request lines.
Everything is scrubbed like error details and limited to what fits in the Everything is scrubbed like error details and limited to what fits in the
issue. Environment details are kept first, then the newest lines. **Show report. Environment details are kept first, then the newest lines. **Show
exactly what's included** shows the snapshot that will be sent, and later exactly what's included** shows the snapshot that will be sent, and later
activity isn't added to it. If the service can't be reached, the dialog offers activity isn't added to it. If PostHog can't be reached, **Copy report** puts
**Copy report** and **Open on GitHub instead**, a prefilled issue. the whole report on the clipboard.
The maintainer reads reports on the Frame Control dashboard in PostHog, or
with `python3 ui/frame_report.py inbox [days]`, which uses the same personal
API key as `frame_compat_db.py sync`.
## Turning it all off ## Turning it all off
@@ -128,6 +137,7 @@ never sends anything unless `FRAME_CONTROL_TELEMETRY=1` is set.
## Update checks ## Update checks
The desktop app asks GitHub for the latest release shortly after starting, The desktop app asks GitHub for the latest release shortly after starting,
then every 6 hours (`api.github.com/repos/saphid/frame-control/releases/latest`). then every 6 hours: the latest release's `update.json` on GitHub, or
That request carries no id. To stop it, set `api.github.com/repos/saphid/frame-control/releases/latest` if that fails.
Those requests carry no id. To stop it, set
`FRAME_CONTROL_NO_UPDATE_CHECK=1`. See [releasing.md](releasing.md). `FRAME_CONTROL_NO_UPDATE_CHECK=1`. See [releasing.md](releasing.md).
+67 -25
View File
@@ -322,18 +322,17 @@ class Regressions(Base):
class ReportProblem(Base): class ReportProblem(Base):
"""Report a problem: diagnostics are scrubbed and fit the feedback API; sending goes to it.""" """Report a problem: diagnostics are scrubbed and bounded; the report goes privately to PostHog."""
def serve(self, status=201, reply=None): def serve(self, status=200):
got = [] got = []
class H(BaseHTTPRequestHandler): class H(BaseHTTPRequestHandler):
def do_POST(self): def do_POST(self):
got.append(json.loads(self.rfile.read(int(self.headers["Content-Length"])))) got.append((self.path, json.loads(self.rfile.read(int(self.headers["Content-Length"])))))
self.send_response(status) self.send_response(status)
self.end_headers() self.end_headers()
self.wfile.write(json.dumps(reply or {"ok": True, "number": 42, self.wfile.write(b'{"status":"Ok"}')
"url": "https://github.com/saphid/frame-control/issues/42"}).encode())
def log_message(self, *a): def log_message(self, *a):
pass pass
@@ -342,7 +341,7 @@ class ReportProblem(Base):
threading.Thread(target=httpd.serve_forever, daemon=True).start() threading.Thread(target=httpd.serve_forever, daemon=True).start()
self.addCleanup(httpd.server_close) self.addCleanup(httpd.server_close)
self.addCleanup(httpd.shutdown) self.addCleanup(httpd.shutdown)
p = mock.patch.object(fr, "FEEDBACK_URL", f"http://127.0.0.1:{httpd.server_port}/api/feedback") p = mock.patch.dict(os.environ, {"FRAME_CONTROL_POSTHOG_HOST": f"http://127.0.0.1:{httpd.server_port}"})
p.start() p.start()
self.addCleanup(p.stop) self.addCleanup(p.stop)
return got return got
@@ -358,11 +357,12 @@ class ReportProblem(Base):
for leaked in ("alice", "192.168.1.9", str(Path.home()), "bob", "pw@"): for leaked in ("alice", "192.168.1.9", str(Path.home()), "bob", "pw@"):
self.assertNotIn(leaked, text) self.assertNotIn(leaked, text)
def test_a_report_fits_the_api_limit_in_utf16_units(self): def test_a_report_is_bounded_in_utf16_units(self):
body = {"title": "Live view stops", "message": "It stops 😀 " * 600, "diagnostics": "log 😀 line\n" * 2000} body = {"title": "Live view stops", "message": "It stops 😀 " * 800, "diagnostics": "log 😀 line\n" * 2000}
title, message = fr.compose(body) title, text, diag = fr.compose(body)
self.assertLessEqual(fr.u16(message), fr.MESSAGE_MAX) self.assertLessEqual(fr.u16(text), fr.TEXT_MAX)
self.assertTrue(message.startswith("It stops")) self.assertLessEqual(fr.u16(diag), fr.DIAG_MAX)
self.assertTrue(text.startswith("It stops"))
with self.assertRaises(ValueError): with self.assertRaises(ValueError):
fr.compose({"title": "hi", "message": "It stops after a minute."}) fr.compose({"title": "hi", "message": "It stops after a minute."})
@@ -381,24 +381,66 @@ class ReportProblem(Base):
def test_the_previewed_diagnostics_are_what_is_sent(self): def test_the_previewed_diagnostics_are_what_is_sent(self):
got = self.serve() got = self.serve()
fr.send({"title": "Live view stops", "message": "It stops after a minute.", "elapsed": 9000, fr.send({"title": "Live view stops", "message": "It stops after a minute.",
"diagnostics": "Frame Control 9.9.9\nssh janes-mac.tail12345.ts.net failed"}) "diagnostics": "Frame Control 9.9.9\nssh janes-mac.tail12345.ts.net failed"})
self.assertIn("Frame Control 9.9.9", got[0]["message"]) diag = got[0][1]["batch"][0]["properties"]["diagnostics"]
self.assertNotIn("janes-mac", got[0]["message"]) self.assertIn("Frame Control 9.9.9", diag)
self.assertNotIn("janes-mac", diag)
def test_send_files_an_issue_and_reports_its_number(self): def test_send_is_a_private_posthog_event_whatever_the_settings(self):
got = self.serve() got = self.serve()
res = fr.send({"kind": "bug", "title": "Live view stops", "message": "It stops after a minute.", tm.update_settings({"usage": False}) # analytics off: a deliberate report still goes
"elapsed": 9000, "github": "@someone"}) res = fr.send({"kind": "idea", "title": "Live view stops", "message": "It stops after a minute.",
self.assertEqual((res["number"], res["url"]), (42, "https://github.com/saphid/frame-control/issues/42")) "contact": "me@example.com"})
sent = got[0] path, body = got[0]
self.assertEqual((sent["kind"], sent["github"], sent["website"], sent["elapsed"]), ("bug", "someone", "", 9000)) event = body["batch"][0]
self.assertEqual(sent["message"], "It stops after a minute.") self.assertEqual((path, body["api_key"], event["event"]), ("/batch/", "phc_test", "problem_report"))
props = event["properties"]
self.assertEqual((props["kind"], props["title"], props["message"], props["contact"], props["report_id"]),
("idea", "Live view stops", "It stops after a minute.", "me@example.com", res["id"]))
self.assertEqual((props["$process_person_profile"], props["$geoip_disable"]), (False, True))
self.assertNotEqual(event["distinct_id"], tm.settings()["id"]) # not linked to the analytics
self.assertIn(res["id"], res["message"])
self.assertEqual([e["event"] for e in tm._read_lines(tm.SENT)], ["problem_report"])
def test_the_services_error_is_passed_on(self): def test_a_sent_report_is_not_an_error_if_the_local_log_fails(self):
self.serve(status=429, reply={"error": "That's a lot of feedback in one hour."}) self.serve()
with self.assertRaisesRegex(fr.ReportError, "a lot of feedback"): with mock.patch.object(tm, "record_sent", side_effect=OSError("disk full")):
fr.send({"title": "Live view stops", "message": "It stops after a minute.", "elapsed": 9000}) res = fr.send({"title": "Live view stops", "message": "It stops after a minute."})
self.assertTrue(res["id"])
def test_events_queued_by_older_versions_get_the_placeholder_address(self):
got = self.serve()
tm.update_settings({"noticeShown": True})
tm._write_lines(tm.OUTBOX, [{"event": "app_opened", "distinct_id": "x", "uuid": "u1",
"properties": {"level": "usage"}}])
self.assertEqual(tm.flush(), 1)
self.assertEqual(got[0][1]["batch"][0]["properties"]["$ip"], "0.0.0.0")
self.assertEqual(tm._read_lines(tm.SENT)[0]["properties"]["$ip"], "0.0.0.0")
def test_the_inbox_skips_malformed_reports(self):
good = ["2026-09-28T09:50:00Z", "AB12CD34", "bug", "Live view stops", "It stops.", None,
"0.4.0", "macOS", "", ""]
rows = [["2026-09-28T10:00:00Z", "X", "bug", "Hand-made", None, None, None, None, None, None], ["short"], good]
with mock.patch.object(db, "_posthog_query", return_value={"results": rows}), \
mock.patch.object(sys, "argv", ["frame_report.py", "inbox"]), \
mock.patch("builtins.print") as out:
fr.main()
printed = " ".join(str(c.args[0]) for c in out.call_args_list if c.args)
self.assertIn("AB12CD34", printed)
self.assertIn("Hand-made", printed)
def test_a_refused_report_is_an_error(self):
self.serve(status=401)
with self.assertRaisesRegex(fr.ReportError, "HTTP 401"):
fr.send({"title": "Live view stops", "message": "It stops after a minute."})
self.assertEqual(tm._read_lines(tm.SENT), [])
def test_no_key_means_no_report(self):
with mock.patch.dict(os.environ, {"FRAME_CONTROL_POSTHOG_KEY": ""}), \
mock.patch.object(tm, "HERE", tm.STATE):
with self.assertRaisesRegex(fr.ReportError, "no PostHog project key"):
fr.send({"title": "Live view stops", "message": "It stops after a minute."})
if __name__ == "__main__": if __name__ == "__main__":
+13 -7
View File
@@ -267,7 +267,7 @@ def _posthog_query(sql):
project = os.environ.get('FRAME_CONTROL_POSTHOG_PROJECT') or cfg.get('project') project = os.environ.get('FRAME_CONTROL_POSTHOG_PROJECT') or cfg.get('project')
if not project: if not project:
raise DBError('No PostHog project id (ui/telemetry.json "project", or FRAME_CONTROL_POSTHOG_PROJECT)') raise DBError('No PostHog project id (ui/telemetry.json "project", or FRAME_CONTROL_POSTHOG_PROJECT)')
# The query API lives on the app host (eu.posthog.com), not the ingestion host (eu.i.posthog.com). # The query API lives on the app host (us.posthog.com), not the ingestion host (us.i.posthog.com).
host = cfg['host'].replace('.i.posthog.com', '.posthog.com') host = cfg['host'].replace('.i.posthog.com', '.posthog.com')
req = urllib.request.Request(f'{host}/api/projects/{urllib.parse.quote(str(project))}/query/', method='POST', req = urllib.request.Request(f'{host}/api/projects/{urllib.parse.quote(str(project))}/query/', method='POST',
data=json.dumps({'query': {'kind': 'HogQLQuery', 'query': sql}}).encode(), data=json.dumps({'query': {'kind': 'HogQLQuery', 'query': sql}}).encode(),
@@ -352,15 +352,21 @@ def sync(dry_run=False):
key() # the maintainer's copy only key() # the maintainer's copy only
state = _sync_state() state = _sync_state()
since = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(time.time() - SYNC_OVERLAP_DAYS * 86400)) since = time.strftime('%Y-%m-%d %H:%M:%S', time.gmtime(time.time() - SYNC_OVERLAP_DAYS * 86400))
events = [] events, after = [], f"timestamp >= toDateTime('{since}', 'UTC')"
for page in range(40): for _ in range(40):
res = _posthog_query("SELECT properties, distinct_id, timestamp FROM events " # Keyset paging: PostHog refuses OFFSET with a personal API key. The cursor is in UTC,
f"WHERE event = 'compat_report' AND timestamp >= toDateTime('{since}') " # since a local time is ambiguous in the hour clocks go back.
f"ORDER BY timestamp, uuid LIMIT {SYNC_PAGE} OFFSET {page * SYNC_PAGE}") res = _posthog_query("SELECT properties, distinct_id, timestamp, toString(uuid), "
"formatDateTime(timestamp, '%Y-%m-%d %H:%i:%S.%f', 'UTC') FROM events "
f"WHERE event = 'compat_report' AND {after} "
f"ORDER BY timestamp, toString(uuid) LIMIT {SYNC_PAGE}")
rows = res.get('results') or [] rows = res.get('results') or []
events += rows events += [row[:3] for row in rows]
if len(rows) < SYNC_PAGE: if len(rows) < SYNC_PAGE:
break break
last_uuid, last_ts = rows[-1][3], rows[-1][4]
after = (f"(timestamp > toDateTime64('{last_ts}', 6, 'UTC') OR "
f"(timestamp = toDateTime64('{last_ts}', 6, 'UTC') AND toString(uuid) > '{last_uuid}'))")
rows, skipped = community_rows(events, state) rows, skipped = community_rows(events, state)
if dry_run: if dry_run:
return rows, skipped return rows, skipped
+61 -47
View File
@@ -1,32 +1,26 @@
"""Report a problem from inside Frame Control. Python stdlib only. """Report a problem from inside Frame Control. Python stdlib only.
The page's Report a problem dialog shows the diagnostics below before anything The page's Report a problem dialog shows the diagnostics below before anything
is sent, then this sends the report to the website's feedback API is sent, then this sends the report privately to Frame Control's PostHog
(site/functions/api/feedback.js), which files it as a GitHub issue. The user project as a `problem_report` event: only the maintainer can read it, and
needs no GitHub account. Everything collected is scrubbed first nothing is published. It is sent whatever the analytics settings are, because
(frame_telemetry.scrub), since the issue is public. the person sends it deliberately. Diagnostics are scrubbed first
(frame_telemetry.scrub); the person's own words are sent as written.
""" """
import json
import os import os
import platform import platform
import sys import sys
import time import time
import urllib.error import uuid
import urllib.request
import frame_host import frame_host
import frame_telemetry import frame_telemetry
FEEDBACK_URL = os.environ.get('FRAME_CONTROL_FEEDBACK_URL', 'https://frame-control.pages.dev/api/feedback')
ISSUES_URL = 'https://github.com/saphid/frame-control/issues/new'
KINDS = ('bug', 'idea', 'question', 'other') KINDS = ('bug', 'idea', 'question', 'other')
MESSAGE_MAX = 5000 # the feedback API's limit, in JavaScript (UTF-16) units TEXT_MAX = 5000 # the person's own text, in JavaScript (UTF-16) units like the page's maxlength
TEXT_MAX = 3500 # the person's own text DIAG_MAX = 8000 # the diagnostics block
DIAG_MAX = 1300 # the diagnostics block, so text + diagnostics always fit MESSAGE_MAX LOG_LINES = 60
LOG_LINES = 40
ACTIVITY_LINES = 25 ACTIVITY_LINES = 25
HEAD = '\n\n---\nDiagnostics from Frame Control (personal details removed):\n```\n'
TAIL = '\n```'
frame = {} # the Frame's last known SteamOS build, set by server.status() frame = {} # the Frame's last known SteamOS build, set by server.status()
@@ -96,52 +90,72 @@ def diagnostics(activity=(), include_logs=False, limit=DIAG_MAX):
def compose(body): def compose(body):
"""(title, message) for the feedback API: the person's text, then the diagnostics exactly as """(title, text, diagnostics): the diagnostics exactly as the dialog previewed them (passed
the dialog previewed them (passed back, scrubbed again and bounded here).""" back, scrubbed again and bounded here)."""
title = ' '.join(str(body.get('title') or '').split()) title = ' '.join(str(body.get('title') or '').split())
text = str(body.get('message') or '').strip() text = str(body.get('message') or '').strip()
if len(title) < 5: if len(title) < 5:
raise ValueError('give it a short title (at least 5 characters)') raise ValueError('give it a short title (at least 5 characters)')
if len(text) < 10: if len(text) < 10:
raise ValueError('say a little more about what happened (at least 10 characters)') raise ValueError('say a little more about what happened (at least 10 characters)')
title, text = cut(title, 120), cut(text, TEXT_MAX)
diag = body.get('diagnostics') diag = body.get('diagnostics')
if not isinstance(diag, str) or not diag.strip(): diag = cut(frame_telemetry.scrub(diag, 40000), DIAG_MAX) if isinstance(diag, str) and diag.strip() else ''
return title, text return cut(title, 120), cut(text, TEXT_MAX), diag
diag = cut(frame_telemetry.scrub(diag, 20000), DIAG_MAX)
return title, f'{text}{HEAD}{diag}{TAIL}'
def send(body): def send(body):
"""File the report. Returns {"number", "url"} of the new issue; raises ReportError.""" """Send the report to PostHog. Returns {"id", "message"}; raises ReportError."""
kind = body.get('kind') if body.get('kind') in KINDS else 'bug' kind = body.get('kind') if body.get('kind') in KINDS else 'bug'
title, message = compose(body) title, text, diag = compose(body)
payload = {'kind': kind, 'title': title, 'message': message, 'website': '', ref = uuid.uuid4().hex[:8].upper()
'github': str(body.get('github') or '').strip().lstrip('@')[:40], props = {**frame_telemetry.common(), 'kind': kind, 'title': title, 'message': text,
'version': frame_telemetry.app_version(), 'os': f'{frame_host.NAME} {platform.machine()}', 'contact': str(body.get('contact') or '').strip()[:120], 'diagnostics': diag,
'steamos': str(frame.get('build') or '')[:120], 'report_id': ref, 'steamos': str(frame.get('build') or '')[:120], 'level': 'report'}
# How long the dialog was open; the API treats anything under 3 s as a script. # Its own random id: a report can carry contact details, so it isn't linked to this copy's analytics.
'elapsed': max(0, int(body.get('elapsed') or 0))} event = {'event': 'problem_report', 'distinct_id': str(uuid.uuid4()), 'uuid': str(uuid.uuid4()),
req = urllib.request.Request(FEEDBACK_URL, data=json.dumps(payload).encode(), method='POST', 'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'properties': props}
headers={'content-type': 'application/json',
'user-agent': f'FrameControl/{frame_telemetry.app_version()}'})
try: try:
with urllib.request.urlopen(req, timeout=30) as r: frame_telemetry.post([event], timeout=30)
res = json.loads(r.read() or b'{}') except frame_telemetry.SendError as e:
except urllib.error.HTTPError as e: raise ReportError(str(e))
try: try:
why = json.loads(e.read() or b'{}').get('error') frame_telemetry.record_sent([event])
except ValueError: except OSError:
why = None pass # it was sent; failing to log it here mustn't make the person send it again
e.close() return {'id': ref, 'message': f'Sent privately to the Frame Control developer (report {ref}).'}
raise ReportError(why or f'the feedback service said HTTP {e.code}')
except (urllib.error.URLError, OSError, ValueError) as e:
raise ReportError(f"couldn't reach the feedback service: {e}")
if not res.get('url'):
raise ReportError("the feedback service didn't file it; try again in a moment")
frame_telemetry.capture('problem_reported', {'kind': kind, 'with_diagnostics': bool(body.get('diagnostics'))})
return {'number': res.get('number'), 'url': res['url'], 'message': f"Sent. It's issue #{res.get('number')} on GitHub."}
class ReportError(RuntimeError): class ReportError(RuntimeError):
pass pass
def inbox(days=30):
"""The maintainer's recent reports from PostHog, newest first (needs the personal API key
frame_compat_db.sync uses)."""
import frame_compat_db
res = frame_compat_db._posthog_query(
"SELECT timestamp, properties.report_id, properties.kind, properties.title, properties.message, "
"properties.contact, properties.app_version, properties.os, properties.steamos, properties.diagnostics "
f"FROM events WHERE event = 'problem_report' AND timestamp > now() - INTERVAL {int(days)} DAY "
"ORDER BY timestamp DESC LIMIT 200")
return res.get('results') or []
def main():
cmd, *args = sys.argv[1:] or ['inbox']
if cmd != 'inbox':
sys.exit('usage: frame_report.py inbox [days]')
for row in inbox(*(args[:1] or [30])):
if not isinstance(row, list) or len(row) != 10:
continue
ts, ref, kind, title, text, contact, version, osname, steamos, diag = (str(v or '') for v in row)
print(f"== {ts[:16].replace('T', ' ')} {ref} [{kind}] {title}")
print(f" {version} on {osname}, SteamOS {steamos or 'unknown'}{', reply to ' + contact if contact else ''}")
print(' ' + text.replace('\n', '\n '))
if diag:
print(' --- diagnostics\n ' + diag.replace('\n', '\n '))
print()
if __name__ == '__main__':
main()
+41 -18
View File
@@ -14,8 +14,8 @@ every event and property):
home folders, user names, addresses and keys. home folders, user names, addresses and keys.
The first-run notice offers compat and diagnostics together, and the page's The first-run notice offers compat and diagnostics together, and the page's
Report a problem dialog (frame_report.py) files bug reports whatever is chosen Report a problem dialog (frame_report.py) sends bug reports privately to the
here. same project whatever is chosen here.
Events are identified by a random id made on first run, not by the person or Events are identified by a random id made on first run, not by the person or
computer, and sent without person profiles or GeoIP. Nothing is sent without a computer, and sent without person profiles or GeoIP. Nothing is sent without a
@@ -53,7 +53,7 @@ SENT_KEEP = 200
OUTBOX_MAX = 2000 # events kept while offline; the oldest go first OUTBOX_MAX = 2000 # events kept while offline; the oldest go first
FLUSH_EVERY = 60 FLUSH_EVERY = 60
REPEAT_WINDOW = 600 # the same diagnostic error is sent at most once in this many seconds REPEAT_WINDOW = 600 # the same diagnostic error is sent at most once in this many seconds
DEFAULT_HOST = 'https://eu.i.posthog.com' DEFAULT_HOST = 'https://us.i.posthog.com'
LEVELS = ('usage', 'compat', 'diagnostics') LEVELS = ('usage', 'compat', 'diagnostics')
# Events the page may send through /api/telemetry, and the properties each may carry. # Events the page may send through /api/telemetry, and the properties each may carry.
@@ -270,11 +270,12 @@ def categorize(message):
# ---- capturing ------------------------------------------------------------------ # ---- capturing ------------------------------------------------------------------
def _common(): def common():
return {'app_version': app_version(), 'os': frame_host.NAME, 'arch': platform.machine().lower(), return {'app_version': app_version(), 'os': frame_host.NAME, 'arch': platform.machine().lower(),
'python': '%d.%d' % sys.version_info[:2], '$lib': 'frame-control', 'python': '%d.%d' % sys.version_info[:2], '$lib': 'frame-control',
# Anonymous events: no person profile, no location lookup. # Anonymous events: no person profile, no location lookup, and a placeholder address,
'$process_person_profile': False, '$geoip_disable': True} # since PostHog stores the sender's IP unless an event gives one.
'$process_person_profile': False, '$geoip_disable': True, '$ip': '0.0.0.0'}
def app_version(): def app_version():
@@ -296,7 +297,7 @@ def capture(event, props=None, level='usage'):
s = settings() s = settings()
e = {'event': event, 'distinct_id': s['id'], 'uuid': str(uuid.uuid4()), e = {'event': event, 'distinct_id': s['id'], 'uuid': str(uuid.uuid4()),
'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()), 'timestamp': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()),
'properties': {**_common(), **(props or {}), 'level': level}} 'properties': {**common(), **(props or {}), 'level': level}}
with _lock: with _lock:
lines = _read_lines(OUTBOX) + [e] lines = _read_lines(OUTBOX) + [e]
_write_lines(OUTBOX, lines[-OUTBOX_MAX:]) _write_lines(OUTBOX, lines[-OUTBOX_MAX:])
@@ -459,17 +460,13 @@ def _drop_unwanted(s):
_write_lines(OUTBOX, kept) _write_lines(OUTBOX, kept)
def flush(timeout=20): def post(batch, timeout=20):
"""Send what's queued. Returns how many were sent; on failure they stay queued.""" """Send events to PostHog now. Raises SendError if they weren't accepted."""
with _send_lock:
if blocked() or not settings()['notice_shown']:
return 0
cfg = config() cfg = config()
with _lock: if not cfg['key']:
_drop_unwanted(settings()) raise SendError('no PostHog project key in this build')
batch = _read_lines(OUTBOX)[:100] for e in batch: # also events queued by versions that didn't add the placeholder address
if not batch: e.setdefault('properties', {})['$ip'] = '0.0.0.0'
return 0
body = json.dumps({'api_key': cfg['key'], 'batch': batch}).encode() body = json.dumps({'api_key': cfg['key'], 'batch': batch}).encode()
req = urllib.request.Request(cfg['host'] + '/batch/', data=body, method='POST', req = urllib.request.Request(cfg['host'] + '/batch/', data=body, method='POST',
headers={'content-type': 'application/json', headers={'content-type': 'application/json',
@@ -479,8 +476,34 @@ def flush(timeout=20):
r.read() r.read()
except urllib.error.HTTPError as e: except urllib.error.HTTPError as e:
e.close() e.close()
raise SendError(f'PostHog said HTTP {e.code}')
except (urllib.error.URLError, OSError, ValueError) as e:
raise SendError(f"couldn't reach PostHog: {e}")
def record_sent(events):
"""Add events sent outside the outbox to the log the page shows."""
with _lock:
_write_lines(SENT, (_read_lines(SENT) + list(events))[-SENT_KEEP:])
class SendError(RuntimeError):
pass
def flush(timeout=20):
"""Send what's queued. Returns how many were sent; on failure they stay queued."""
with _send_lock:
if blocked() or not settings()['notice_shown']:
return 0 return 0
except (urllib.error.URLError, OSError, ValueError): with _lock:
_drop_unwanted(settings())
batch = _read_lines(OUTBOX)[:100]
if not batch:
return 0
try:
post(batch, timeout)
except SendError:
return 0 return 0
sent_ids = {e['uuid'] for e in batch} sent_ids = {e['uuid'] for e in batch}
with _lock: with _lock:
+11 -19
View File
@@ -759,19 +759,18 @@
<option value="question">A question</option><option value="other">Something else</option></select></label> <option value="question">A question</option><option value="other">Something else</option></select></label>
<label class="field">Title<input type="text" id="bugTitleIn" maxlength="120" required minlength="5" <label class="field">Title<input type="text" id="bugTitleIn" maxlength="120" required minlength="5"
placeholder="e.g. Installing an APK stops at 'copying to the Frame'"></label> placeholder="e.g. Installing an APK stops at 'copying to the Frame'"></label>
<label class="field">What happened?<textarea id="bugText" maxlength="3500" required minlength="10" <label class="field">What happened?<textarea id="bugText" maxlength="5000" required minlength="10"
placeholder="What you did, what happened, and what you expected."></textarea></label> placeholder="What you did, what happened, and what you expected."></textarea></label>
<label class="field">GitHub username (optional, so you can follow replies)<input type="text" id="bugGithub" maxlength="40" placeholder="@yourname"></label> <label class="field">How can we reach you? (optional, for a reply)<input type="text" id="bugContact" maxlength="120" placeholder="Email, GitHub or Discord name"></label>
<label class="popt"><input type="checkbox" id="bugDiag" checked><b>Include diagnostics</b> <label class="popt"><input type="checkbox" id="bugDiag" checked><b>Include diagnostics</b>
<span class="sub">Frame Control's version, your OS and the Frame's SteamOS build.</span></label> <span class="sub">Frame Control's version, your OS and the Frame's SteamOS build.</span></label>
<label class="popt"><input type="checkbox" id="bugLogs"><b>Also include recent activity and the server log</b> <label class="popt"><input type="checkbox" id="bugLogs"><b>Also include recent activity and the server log</b>
<span class="sub">Often shows what went wrong, but can contain file and app names. Check it below before sending.</span></label> <span class="sub">Often shows what went wrong, but can contain file and app names. Check it below before sending.</span></label>
<details id="bugDiagBox"><summary>Show exactly what's included</summary><div class="sentlog" id="bugDiagText">Loading…</div></details> <details id="bugDiagBox"><summary>Show exactly what's included</summary><div class="sentlog" id="bugDiagText">Loading…</div></details>
<p id="bugWarn">This becomes a public issue on GitHub (saphid/frame-control). No GitHub account is needed.</p> <p id="bugWarn">Sent privately to the Frame Control developer. Nothing is published.</p>
<div class="row rep-actions"><span class="sub" id="bugMsg"></span><span class="spacer"></span> <div class="row rep-actions"><span class="sub" id="bugMsg"></span><span class="spacer"></span>
<button type="button" class="small" id="bugCancel">Cancel</button> <button type="button" class="small" id="bugCancel">Cancel</button>
<button type="button" class="small" id="bugCopy">Copy report</button> <button type="button" class="small" id="bugCopy">Copy report</button>
<a class="small" id="bugGithubLink" href="#" target="_blank" hidden>Open on GitHub instead</a>
<button type="submit" class="action small" id="bugSend">Send report</button></div> <button type="submit" class="action small" id="bugSend">Send report</button></div>
</form> </form>
</dialog> </dialog>
@@ -2331,18 +2330,14 @@ document.querySelectorAll("nav a").forEach(a => a.addEventListener("click", () =
if (!tabsSeen.has(tab)) { tabsSeen.add(tab); pageEvent("tab_viewed", { tab }); } if (!tabsSeen.has(tab)) { tabsSeen.add(tab); pageEvent("tab_viewed", { tab }); }
})); }));
// ---- report a problem (ui/frame_report.py): a GitHub issue via the website, with diagnostics ---- // ---- report a problem (ui/frame_report.py): sent privately to PostHog, with diagnostics ----
const bug = { opened: 0, preview: "" }; const bug = { preview: "" };
const activityLines = () => [...$("log").children].slice(0, 25).map(el => el.textContent.trim()); const activityLines = () => [...$("log").children].slice(0, 25).map(el => el.textContent.trim());
function bugReportText() { function bugReportText() {
const body = `${$("bugText").value.trim()}${bug.preview ? "\n\n---\nDiagnostics:\n```\n" + bug.preview + "\n```" : ""}`; const contact = $("bugContact").value.trim();
const body = `Kind: ${$("bugKind").value}${contact ? `\nContact: ${contact}` : ""}\n\n${$("bugText").value.trim()}${bug.preview ? "\n\n---\nDiagnostics:\n```\n" + bug.preview + "\n```" : ""}`;
return { title: $("bugTitleIn").value.trim(), body }; return { title: $("bugTitleIn").value.trim(), body };
} }
function bugGithubUrl() {
const { title, body } = bugReportText();
return "https://github.com/saphid/frame-control/issues/new?labels=feedback&title=" + encodeURIComponent(title)
+ "&body=" + encodeURIComponent(body.slice(0, 6000));
}
// The preview is a snapshot: exactly this text is sent, even if more activity happens meanwhile. // The preview is a snapshot: exactly this text is sent, even if more activity happens meanwhile.
async function loadBugPreview() { async function loadBugPreview() {
if (!$("bugDiag").checked) { bug.preview = ""; $("bugDiagText").textContent = "Nothing: diagnostics are off."; return; } if (!$("bugDiag").checked) { bug.preview = ""; $("bugDiagText").textContent = "Nothing: diagnostics are off."; return; }
@@ -2353,10 +2348,9 @@ async function loadBugPreview() {
} }
function openBugReport() { function openBugReport() {
$("bugForm").reset(); $("bugForm").reset();
$("bugMsg").textContent = ""; $("bugGithubLink").hidden = true; $("bugSend").disabled = false; $("bugMsg").textContent = ""; $("bugSend").disabled = false;
$("bugCancel").textContent = "Cancel"; $("bugCancel").textContent = "Cancel";
$("bugDiagBox").open = false; $("bugLogs").disabled = false; $("bugDiagBox").open = false; $("bugLogs").disabled = false;
bug.opened = performance.now();
$("bugDlg").showModal(); $("bugDlg").showModal();
loadBugPreview(); loadBugPreview();
} }
@@ -2376,14 +2370,12 @@ $("bugForm").onsubmit = async e => {
try { try {
const res = await api("/api/report", { const res = await api("/api/report", {
kind: $("bugKind").value, title: $("bugTitleIn").value, message: $("bugText").value, kind: $("bugKind").value, title: $("bugTitleIn").value, message: $("bugText").value,
github: $("bugGithub").value, diagnostics: $("bugDiag").checked ? bug.preview : "", contact: $("bugContact").value, diagnostics: $("bugDiag").checked ? bug.preview : "" });
elapsed: Math.round(performance.now() - bug.opened) }); $("bugMsg").textContent = `Sent, thank you. Your reference is ${res.id}.`;
$("bugMsg").innerHTML = `Sent. <a href="${esc(res.url)}" target="_blank">Issue #${esc(String(res.number))}</a> on GitHub.`;
$("bugCancel").textContent = "Close"; $("bugCancel").textContent = "Close";
log(res.message, "ok"); log(res.message, "ok");
} catch (err) { } catch (err) {
$("bugMsg").textContent = `Couldn't send it: ${err.message}.`; $("bugMsg").textContent = `Couldn't send it: ${err.message}. Try again later, or use Copy report.`;
$("bugGithubLink").href = bugGithubUrl(); $("bugGithubLink").hidden = false;
$("bugSend").disabled = false; $("bugSend").disabled = false;
} }
}; };
+3 -3
View File
@@ -1,5 +1,5 @@
{ {
"host": "https://eu.i.posthog.com", "host": "https://us.i.posthog.com",
"key": "", "key": "phc_qkmbgQBvl2oBXGUVzfV6gG52EpmJdeaQyaRIxHRoQoL",
"project": "" "project": "343535"
} }