Add unified APK source search and source management

Co-Authored-By: GPT-6 Astra <noreply@openai.com>

.claude/NOTES-apk-search.md

@@ -0,0 +1,32 @@
+# APK search
+
+- Scope: dynamic source aggregator, additive server APIs, Android search view; no device calls.
+- Source metadata stays unknown when missing. Unknown-package names only group with other unknown packages (avoids ambiguous package attribution).
+- Queries use daemon workers with at most one in-flight request per source; deadlines don't wait for stuck workers.
+- Independent review delegated to parent per task brief; no workers launched.
+
+## Implementation
+
+- `ui/apk_sources/search.py`: discovers KIND modules, parallel daemon queries (12 s deadline), bounded per-source workers, grouped offers, fit/rank/filter logic, persisted source enablement, repo adapter and install adapter.
+- `ui/server.py`: GET `/api/sources`, GET `/api/search?q=&vr=true|false&installable=true|false&source=`, POST `/api/sources/install` and POST `/api/sources` with actions `add`, `remove`, `enable`.
+- `ui/index.html`: unified Find apps in Android tab, source/VR/Flat/Installable chips, offer picker, metadata, jobs, Sources panel. Old catalogue DOM remains hidden for existing report/icon code; only the unified search is visible.
+- `_demo.py` is discovered only with `FRAME_APK_SEARCH_DEMO=1`; it cannot download. `tests/search_preview.py` serves the real UI/search endpoints and rejects all device endpoints and writes.
+
+## Evidence
+
+- `python3 -m unittest discover -s tests`: 179 tests passed on Python 3.9.6, exit 0. Log `/tmp/apk-search-suite.log`.
+- `node --check /tmp/apk-search-evidence/ui.js`: exit 0; script extracted from index.html.
+- `git diff --check`: exit 0.
+- Python 3.9 grammar parse: search.py, _demo.py, server.py, test_apk_search.py and search_preview.py passed; actual suite interpreter also Python 3.9.6.
+- Started `FRAME_APK_SEARCH_DEMO=1 python3 tests/search_preview.py` locally; used T3 preview at http://127.0.0.1:8795/#android (1280x800).
+- Browser assertions: two grouped apps, source picker updates install choice, VR excludes flat, Flat excludes VR, source filter keeps one offer, search query narrows results, pending install disables its button after re-render. Passed.
+- Inspected screenshots: `/tmp/apk-search-evidence/search.png`, `/tmp/apk-search-evidence/sources.png`. Other panels show intentional device-access errors in the preview.
+- Tests cover dynamic module discovery, grouping (including ambiguous names), ranking, fit, timeout/failure isolation, persistent enablement, endpoint validation, repo callbacks, background install with mocked frame_android.install, conditional artwork, and OBB support/refusal.
+
+## Parent integration / unverified
+
+- No source modules from sibling branches are present yet. Real network listings/downloads, signatures, merged repo persistence and physical installation were not exercised. No SSH/device installation performed.
+- Assumes future `frame_android.install_obb(package, paths)`; reconcile with vr-library worker's actual signature. Artwork is passed only when install explicitly declares that parameter. OBB-required apps fail before APK installation when helper is absent.
+- `app/package.json` currently copies ui with only `*.py` / `*.html`; parent must include `apk_sources/**/*.py` in packaged resources when integrating source workers. Kept package config outside this worker's scope.
+- No cross-provider reviewer launched: task explicitly forbids delegation and assigns integration/review to the parent.
+- Source metadata unknowns do not imply compatibility or verification. Unknown-package entries only group with other unknown-package entries with the same normalized name.

tests/fixtures/apk-search/entries.json

@@ -0,0 +1,7 @@
+[
+ {"source":"one","id":"brush","package":"org.brush","name":"Open Brush","free":true,"downloadable":true,"verified":true,"version":"1","version_code":1,"min_sdk":29,"abis":["arm64-v8a"],"vr":true,"updated":"2025-01-01"},
+ {"source":"two","id":"brush2","package":"org.brush","name":"Open Brush","free":true,"downloadable":true,"verified":false,"version":"2","version_code":2,"min_sdk":29,"abis":["arm64-v8a"],"vr":true,"updated":"2026-01-01"},
+ {"source":"one","id":"other","package":"org.other","name":"Open Brush","free":true,"downloadable":true,"min_sdk":31,"abis":["arm64-v8a"],"vr":true},
+ {"source":"one","id":"unknown","package":null,"name":"Pocket Radio!","free":true,"downloadable":false,"vr":false},
+ {"source":"two","id":"unknown2","package":null,"name":"pocket  radio","free":true,"downloadable":false,"vr":false}
+]

tests/search_preview.py

@@ -0,0 +1,27 @@
+"""Local demo server for UI checks; rejects every device endpoint.
+
+FRAME_APK_SEARCH_DEMO=1 python3 tests/search_preview.py
+"""
+import os
+from pathlib import Path
+import sys
+from urllib.parse import urlparse
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
+import server
+
+
+class Preview(server.Handler):
+    def do_GET(self):
+        if urlparse(self.path).path not in ('/', '/index.html', '/api/search', '/api/sources', '/api/job', '/api/host'):
+            self.send_json({'error': 'Device access disabled in search preview'}, 503)
+            return
+        super().do_GET()
+
+    def do_POST(self):
+        self.send_json({'error': 'Writes disabled in search preview'}, 403)
+
+
+if __name__ == '__main__':
+    if os.environ.get('FRAME_APK_SEARCH_DEMO') != '1':
+        sys.exit('Set FRAME_APK_SEARCH_DEMO=1')
+    server.ThreadingHTTPServer(('127.0.0.1', 8795), Preview).serve_forever()

tests/test_apk_search.py

@@ -0,0 +1,195 @@
+import http.client
+import json
+from pathlib import Path
+import sys
+import tempfile
+import threading
+import time
+import types
+import unittest
+from unittest.mock import Mock, patch
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / 'ui'))
+from apk_sources import search, SourceError
+import server
+
+ENTRIES = json.loads((Path(__file__).parent / 'fixtures/apk-search/entries.json').read_text())
+
+
+def fake(source_id='one', fn=None):
+    return types.SimpleNamespace(KIND=source_id, sources=lambda: [dict(id=source_id, name=source_id,
+                                 enabled=True, trust='official', builtin=True)],
+                                 search=fn or (lambda s, q, limit=50: [e for e in ENTRIES if e['source'] == source_id]),
+                                 details=lambda s, i: ENTRIES[0],
+                                 download=Mock(return_value={'apk': '/fake.apk', 'obb': []}))
+
+
+class SettingsTest(unittest.TestCase):
+    def setUp(self):
+        self.tmp = tempfile.TemporaryDirectory()
+        self.addCleanup(self.tmp.cleanup)
+        p = patch.object(search, 'settings_path', return_value=Path(self.tmp.name) / 'enabled.json')
+        p.start()
+        self.addCleanup(p.stop)
+        search._running.clear()
+        search._status.clear()
+
+
+class SearchTests(SettingsTest):
+    def test_group_does_not_merge_distinct_or_unknown_packages(self):
+        result = search.group(ENTRIES)
+        self.assertEqual(len(result), 3)
+        self.assertEqual(sorted(len(a['offers']) for a in result), [1, 2, 2])
+        same_name = dict(ENTRIES[0], package=None)
+        self.assertEqual(len(search.group(ENTRIES[:1] + [same_name])), 2)
+
+    def test_rank_exact_and_installable_and_verified(self):
+        result = search.group(ENTRIES, 'Open Brush')
+        self.assertEqual(result[0]['package'], 'org.brush')
+        self.assertEqual(result[0]['offers'][0]['source'], 'one')
+        self.assertEqual(result[1]['package'], 'org.other')
+        self.assertEqual(len(search.group(ENTRIES, vr=False)), 1)
+        self.assertEqual(len(search.group(ENTRIES, installable=True)), 1)
+
+    def test_fit_unknown_and_native_free_and_vr_hints(self):
+        self.assertIsNone(search.fit({})['installable'])
+        self.assertTrue(search.fit({'min_sdk': 23, 'abis': []})['installable'])
+        self.assertFalse(search.fit({'min_sdk': 23, 'abis': ['x86']})['installable'])
+        self.assertIn('Legacy VrApi', search.fit({'engine': 'VrApi'})['reasons'][0])
+
+    def test_timeout_and_failure_leave_other_results(self):
+        release = threading.Event()
+        calls = []
+        def slow(s, q, limit=50):
+            calls.append(q)
+            release.wait(2)
+            return []
+        mods = [fake(), fake('slow', slow), fake('broken', Mock(side_effect=SourceError('offline')))]
+        try:
+            with patch.object(search, 'modules', return_value=(mods, [])):
+                started = time.monotonic()
+                result = search.search(timeout=.03)
+                self.assertLess(time.monotonic() - started, .3)
+                self.assertTrue(result['apps'])
+                self.assertEqual([s['status'] for s in result['sources']], ['ok', 'timed out', 'error'])
+                search.search('other', timeout=.03)
+                self.assertEqual(calls, [''])
+        finally:
+            release.set()
+
+    def test_disable_persists_and_prevents_queries_and_installs(self):
+        mod = fake()
+        with patch.object(search, 'modules', return_value=([mod], [])):
+            search.set_enabled('one', False)
+            self.assertFalse(search.sources()[0]['enabled'])
+            self.assertEqual(search.search()['apps'], [])
+            with self.assertRaisesRegex(SourceError, 'disabled'):
+                search.install('one', 'brush')
+
+    def test_install_passes_metadata_artwork_and_obb(self):
+        mod = fake()
+        mod.download.return_value.update(obb=['main.obb'], artwork={'hero': '/hero.png'}, icon_png=b'png')
+        def install(apk, name=None, icon_png=None, source=None, artwork=None):
+            self.assertEqual((apk, name, icon_png, source, artwork),
+                             ('/fake.apk', 'Open Brush', b'png', 'one', {'hero': '/hero.png'}))
+            return {'package': 'org.brush'}
+        with patch.object(search, 'modules', return_value=([mod], [])), \
+             patch.object(server.frame_android, 'install_obb', create=True) as obb:
+            # An actual function exposes the future signature for inspection.
+            with patch.object(server.frame_android, 'install', install):
+                search.install('one', 'brush', 1)
+            obb.assert_called_once_with('org.brush', ['main.obb'])
+            mod.download.assert_called_once_with(mod.sources()[0] | {'status': 'not searched'}, 'brush', version_code=1)
+
+    def test_discovery_and_demo_are_opt_in(self):
+        module = fake()
+        with patch.object(search.pkgutil, 'iter_modules', return_value=[types.SimpleNamespace(name='example')]), \
+             patch.object(search.importlib, 'import_module', return_value=module), \
+             patch.dict(search.os.environ, {'FRAME_APK_SEARCH_DEMO': '0'}):
+            self.assertEqual(search.modules(), ([module], []))
+        with patch.object(search.pkgutil, 'iter_modules', return_value=[]), \
+             patch.dict(search.os.environ, {'FRAME_APK_SEARCH_DEMO': '0'}):
+            self.assertEqual(search.modules(), ([], []))
+
+    def test_newest_compatible_then_official_offer(self):
+        first = dict(ENTRIES[0], verified=False, trust='community')
+        newer = dict(first, source='new', version_code=2, updated='2026-01-01')
+        official = dict(newer, source='official', trust='official')
+        incompatible = dict(newer, source='blocked', verified=True, min_sdk=40)
+        offers = search.group([first, incompatible, newer, official])[0]['offers']
+        self.assertEqual([e['source'] for e in offers], ['official', 'new', 'one', 'blocked'])
+
+    def test_missing_obb_support_stops_before_install(self):
+        mod = fake()
+        mod.download.return_value['obb'] = ['main.obb']
+        with patch.object(search, 'modules', return_value=([mod], [])), \
+             patch.object(server.frame_android, 'install') as install, \
+             patch.dict(server.frame_android.__dict__):
+            server.frame_android.__dict__.pop('install_obb', None)
+            with self.assertRaisesRegex(SourceError, 'OBB'):
+                search.install('one', 'brush')
+            install.assert_not_called()
+
+    def test_listing_cannot_download(self):
+        mod = fake()
+        mod.details = lambda s, i: dict(ENTRIES[0], downloadable=False)
+        with patch.object(search, 'modules', return_value=([mod], [])):
+            with self.assertRaisesRegex(SourceError, 'developer page'):
+                search.install('one', 'brush')
+            mod.download.assert_not_called()
+
+
+class EndpointTests(SettingsTest):
+    def setUp(self):
+        super().setUp()
+        self.mod = fake()
+        p = patch.object(search, 'modules', return_value=([self.mod], []))
+        p.start()
+        self.addCleanup(p.stop)
+        self.httpd = server.ThreadingHTTPServer(('127.0.0.1', 0), server.Handler)
+        threading.Thread(target=self.httpd.serve_forever, daemon=True).start()
+        self.addCleanup(self.httpd.server_close)
+        self.addCleanup(self.httpd.shutdown)
+
+    def request(self, method, path, body=None):
+        c = http.client.HTTPConnection('127.0.0.1', self.httpd.server_port)
+        c.request(method, path, json.dumps(body) if body is not None else None,
+                  {'X-Frame-UI': '1', 'Content-Type': 'application/json'})
+        r = c.getresponse()
+        result = r.status, json.loads(r.read())
+        c.close()
+        return result
+
+    def test_http_search_and_validation(self):
+        self.assertEqual(self.request('GET', '/api/sources')[1]['sources'][0]['id'], 'one')
+        self.assertTrue(self.request('GET', '/api/search?q=Brush&vr=true')[1]['apps'])
+        self.assertEqual(self.request('GET', '/api/search?vr=invalid')[0], 400)
+        self.assertEqual(self.request('GET', '/api/search?source=missing')[0], 400)
+        for body in ({'source': 'one'}, {'source': 'one', 'id': 'brush', 'version_code': True}):
+            self.assertEqual(self.request('POST', '/api/sources/install', body)[0], 400)
+
+    def test_http_install_background_job(self):
+        with patch.object(server.frame_android, 'install', return_value={'package': 'org.brush'}) as install:
+            status, reply = self.request('POST', '/api/sources/install', {'source': 'one', 'id': 'brush'})
+            self.assertEqual(status, 200)
+            for _ in range(100):
+                job = self.request('GET', '/api/job?id=' + reply['job'])[1]
+                if job['done']:
+                    break
+                time.sleep(.01)
+            self.assertTrue(job['done'])
+            self.assertIsNone(job['error'])
+            install.assert_called_once_with('/fake.apk', name='Open Brush', icon_png=None, source='one')
+
+    def test_http_repository_management(self):
+        self.assertEqual(self.request('POST', '/api/sources', {'action': 'enable', 'source': 'one', 'enabled': False})[0], 200)
+        code, reply = self.request('POST', '/api/sources', {'action': 'add', 'url': 'https://repo.example/repo'})
+        self.assertEqual(code, 400)
+        self.assertIn('not available', reply['error'])
+        mod = fake('fdroid')
+        mod.add_repo, mod.remove_repo, mod.set_enabled = Mock(), Mock(), Mock()
+        with patch.object(search, 'modules', return_value=([mod], [])):
+            self.assertEqual(self.request('POST', '/api/sources', {'action': 'add', 'url': 'https://repo.example/repo'})[0], 200)
+            mod.add_repo.assert_called_once_with(url='https://repo.example/repo', fingerprint=None, name=None)
+            self.assertEqual(self.request('POST', '/api/sources', {'action': 'remove', 'source': 'fdroid'})[0], 200)
+            mod.remove_repo.assert_called_once_with(source_id='fdroid')

ui/apk_sources/_demo.py

@@ -0,0 +1,34 @@
+"""Opt-in local UI fixtures: FRAME_APK_SEARCH_DEMO=1. Never downloads."""
+from apk_sources import SourceError
+
+KIND = 'demo'
+
+
+def sources():
+    return [{'id': 'demo-' + key, 'kind': KIND, 'name': name + ' (demo)', 'enabled': True,
+             'builtin': True, 'trust': trust, 'url': 'https://example.invalid'}
+            for key, name, trust in [('fdroid', 'F-Droid', 'community'),
+                                     ('sidequest', 'SideQuest', 'official'), ('itch', 'itch.io', 'community')]]
+
+
+def search(source, query, limit=50):
+    if source['id'] == 'demo-itch':
+        raise SourceError('Demo: source temporarily unavailable')
+    entries = [{'id': 'brush', 'package': 'org.demo.brush', 'name': 'Open Brush',
+                'summary': 'Paint in VR — demonstration listing only', 'version': '2.32',
+                'version_code': 232, 'min_sdk': 29, 'abis': ['arm64-v8a'], 'vr': True,
+                'engine': 'Unity OpenXR', 'size': 125000000, 'updated': '2026-09-25',
+                'verified': source['id'] == 'demo-fdroid', 'free': True, 'downloadable': True},
+               {'id': 'radio', 'package': 'org.demo.radio', 'name': 'Pocket Radio',
+                'summary': 'Listen in a flat Android panel', 'version': '1.0', 'version_code': 1,
+                'min_sdk': 24, 'abis': [], 'vr': False, 'size': 3000000, 'free': True,
+                'downloadable': False, 'page': 'https://example.invalid/radio'}]
+    return [e for e in entries if query.lower() in e['name'].lower()][:limit]
+
+
+def details(source, entry_id):
+    return next(e for e in search(source, '') if e['id'] == entry_id)
+
+
+def download(source, entry_id, version_code=None):
+    raise SourceError('Demo sources cannot download or install apps')

ui/apk_sources/search.py

@@ -0,0 +1,226 @@
+"""Parallel APK search and source selection. No device access during searches."""
+import importlib
+import inspect
+import json
+import os
+import pkgutil
+import re
+import threading
+import time
+import unicodedata
+
+import frame_host
+from apk_sources import SourceError
+
+_lock = threading.RLock()
+_running = {}
+_status = {}
+TIMEOUT = 12
+
+
+def modules():
+    import apk_sources
+    found, errors = [], []
+    for item in pkgutil.iter_modules(apk_sources.__path__):
+        if item.name == 'search' or item.name.startswith('_'):
+            continue
+        try:
+            module = importlib.import_module('apk_sources.' + item.name)
+            if getattr(module, 'KIND', None):
+                found.append(module)
+        except Exception as e:
+            errors.append({'id': item.name, 'name': item.name, 'enabled': False,
+                           'trust': 'unknown', 'status': 'error', 'error': str(e)})
+    if os.environ.get('FRAME_APK_SEARCH_DEMO') == '1':
+        found.append(importlib.import_module('apk_sources._demo'))
+    return found, errors
+
+
+def settings_path():
+    return frame_host.data_dir('apk-sources', 'enabled.json')
+
+
+def overrides():
+    try:
+        return json.loads(settings_path().read_text())
+    except FileNotFoundError:
+        return {}
+
+
+def registry():
+    result = []
+    mods, errors = modules()
+    with _lock:
+        enabled = overrides()
+    for module in mods:
+        try:
+            for source in module.sources():
+                source = dict(source)
+                source['enabled'] = enabled.get(source['id'], source.get('enabled', True))
+                source.update(_status.get(source['id'], {'status': 'not searched'}))
+                result.append((module, source))
+        except Exception as e:
+            errors.append({'id': module.KIND, 'name': module.KIND, 'enabled': False,
+                           'trust': 'unknown', 'status': 'error', 'error': str(e)})
+    return result, errors
+
+
+def sources():
+    items, errors = registry()
+    return [s for _, s in items] + errors
+
+
+def resolve(source_id):
+    for module, source in registry()[0]:
+        if source['id'] == source_id:
+            return module, source
+    raise SourceError('Unknown source')
+
+
+def set_enabled(source_id, enabled):
+    module, source = resolve(source_id)
+    with _lock:
+        if hasattr(module, 'set_enabled'):
+            module.set_enabled(source_id, enabled)
+        values = overrides()
+        values[source_id] = enabled
+        path = settings_path()
+        path.parent.mkdir(parents=True, exist_ok=True)
+        tmp = path.with_suffix('.tmp')
+        tmp.write_text(json.dumps(values))
+        tmp.replace(path)
+    return {'message': source['name'] + (' enabled' if enabled else ' disabled')}
+
+
+def manage_repo(action, **kwargs):
+    mods, _ = modules()
+    module = next((m for m in mods if m.KIND == 'fdroid'), None)
+    if module is None or not hasattr(module, action):
+        raise SourceError('User repositories are not available in this build')
+    return getattr(module, action)(**kwargs)
+
+
+def normalise(name):
+    return ' '.join(re.findall(r'\w+', unicodedata.normalize('NFKC', name or '').casefold()))
+
+
+def fit(entry):
+    sdk, abis = entry.get('min_sdk'), entry.get('abis')
+    reasons = []
+    blocked = (sdk is not None and sdk > 30) or (abis is not None and bool(abis) and 'arm64-v8a' not in abis)
+    if sdk is not None and sdk > 30:
+        reasons.append('Needs Android API %s; Lepton supports 30' % sdk)
+    if abis and 'arm64-v8a' not in abis:
+        reasons.append('No arm64-v8a build')
+    known = sdk is not None and abis is not None
+    hints = ' '.join(str(entry.get(k) or '') for k in ('engine', 'vr_engine', 'vr_hints', 'vr_issues')).lower()
+    if 'vrapi' in hints:
+        reasons.append('Legacy VrApi requires a translator')
+    if 'godot' in hints:
+        reasons.append('Older Godot builds can crash on the missing clipboard service')
+    if 'openxr' in hints:
+        reasons.append('OpenXR candidate; required extensions still need checking')
+    if entry.get('vr'):
+        reasons.append('VR runtime compatibility is not guaranteed')
+    return {'installable': False if blocked else True if known else None,
+            'verdict': "Won't install" if blocked else 'Installable' if known else 'Compatibility unknown',
+            'reasons': reasons}
+
+
+def offer_rank(entry):
+    compatible = entry['fit']['installable'] is True
+    return (entry.get('downloadable') is True and entry['fit']['installable'] is not False,
+            entry.get('verified') is True, compatible,
+            str(entry.get('updated') or '') if compatible else '',
+            (entry.get('version_code') or 0) if compatible else 0,
+            entry.get('trust') == 'official')
+
+
+def group(entries, query='', vr=None, installable=False):
+    groups = {}
+    for entry in entries:
+        entry = dict(entry, fit=fit(entry))
+        if vr is not None and entry.get('vr') is not vr:
+            continue
+        if installable and entry['fit']['installable'] is not True:
+            continue
+        key = ('package', entry['package']) if entry.get('package') else ('name', normalise(entry.get('name')))
+        if not key[1]:
+            key = ('id', entry['source'], entry['id'])
+        groups.setdefault(key, []).append(entry)
+    result = []
+    for offers in groups.values():
+        offers.sort(key=offer_rank, reverse=True)
+        best = offers[0]
+        result.append({'name': best.get('name'), 'package': best.get('package'),
+                       'summary': best.get('summary'), 'offers': offers})
+    q = normalise(query)
+    result.sort(key=lambda a: (not any(normalise(o.get('name')) == q for o in a['offers']),
+                               not any(o['fit']['installable'] is True for o in a['offers']),
+                               not any(normalise(o.get('name')).startswith(q) for o in a['offers']),
+                               normalise(a['name'])))
+    return result
+
+
+def _launch(module, source, query, limit):
+    key = source['id']
+    with _lock:
+        old = _running.get(key)
+        if old and not old['event'].is_set():
+            return old if old['query'] == query else None
+        task = {'event': threading.Event(), 'query': query, 'started': time.monotonic()}
+        _running[key] = task
+    def run():
+        try:
+            task['entries'] = [dict(e, source=key, source_name=source['name'], trust=source.get('trust'))
+                               for e in module.search(source, query, limit=limit) if e.get('free') is True]
+        except Exception as e:
+            task['error'] = str(e)
+        finally:
+            task['event'].set()
+    threading.Thread(target=run, daemon=True).start()
+    return task
+
+
+def search(query='', vr=None, source=None, installable=False, timeout=TIMEOUT, limit=50):
+    items, errors = registry()
+    if source and source not in [s['id'] for _, s in items]:
+        raise SourceError('Unknown source')
+    tasks = [(s, _launch(m, s, query, limit)) for m, s in items
+             if s['enabled'] and (not source or s['id'] == source)]
+    entries, statuses = [], list(errors)
+    for s, task in tasks:
+        status = {'id': s['id'], 'name': s['name']}
+        if task is None or not task['event'].wait(max(0, task['started'] + timeout - time.monotonic())):
+            status.update(status='timed out')
+        elif 'error' in task:
+            status.update(status='error', error=task['error'])
+        else:
+            status.update(status='ok')
+            entries.extend(task['entries'])
+        statuses.append(status)
+        with _lock:
+            _status[s['id']] = {k: v for k, v in status.items() if k not in ('id', 'name')}
+    return {'apps': group(entries, query, vr, installable), 'sources': statuses}
+
+
+def install(source_id, entry_id, version_code=None):
+    import frame_android
+    module, source = resolve(source_id)
+    if not source['enabled']:
+        raise SourceError('This source is disabled')
+    entry = module.details(source, entry_id)
+    if entry.get('free') is not True or entry.get('downloadable') is not True:
+        raise SourceError('This app must be obtained from its developer page')
+    downloaded = module.download(source, entry_id, version_code=version_code)
+    obb = downloaded.get('obb') or entry.get('obb') or []
+    if obb and not hasattr(frame_android, 'install_obb'):
+        raise SourceError('This app needs OBB data; this build cannot install it yet')
+    kwargs = {'name': entry.get('name'), 'icon_png': downloaded.get('icon_png') or entry.get('icon_png'),
+              'source': source['name']}
+    if 'artwork' in inspect.signature(frame_android.install).parameters:
+        kwargs['artwork'] = downloaded.get('artwork') or entry.get('artwork')
+    result = frame_android.install(downloaded['apk'], **kwargs)
+    if obb:
+        frame_android.install_obb(result['package'], obb)
+    return result

ui/index.html

@@ -70,6 +70,15 @@
   body.mobile .desk-only { display: none; }
 
 
+  .source-repo-form { display:grid; gap:10px; margin-top:14px; }
+  .source-repo-form input { display:block; width:100%; margin-top:5px; }
+  .source-offer { border-top:1px solid var(--line); padding-top:10px; margin-top:10px; }
+  #searchGrid .source-offer select { display:block; max-width:100%; width:100%; margin-top:5px; }
+  #searchFilters [aria-pressed=true], #sourceFilters [aria-pressed=true] { box-shadow:inset 0 0 0 1px var(--link); color:var(--link); }
+  #searchGrid h3, #searchGrid p { margin:4px 0; }
+  #searchGrid .row { margin-top:0; }
+  #searchGrid .source-offer { margin-top:auto; }
+  #sourceStatus { margin-bottom:10px; }
   /* ---- pages: one per tab; the header nav switches between them ---- */
   .page { display: flex; flex-direction: column; gap: 26px; }
   .page:not(.on) { display: none; }
@@ -121,7 +130,7 @@
   .seg { display: inline-flex; background: rgba(0,0,0,.3); border-radius: 3px; padding: 2px; }
   .seg button { background: transparent; height: 28px; font-size: 12.5px; letter-spacing: .6px; text-transform: uppercase; }
   .seg button.on { background: var(--btn-hi); color: var(--bright); }
-  input[type=text], input[type=password], textarea { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent;
+  input[type=text], input[type=search], input[type=url], input[type=password], textarea { width: 100%; background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent;
                                border-radius: 3px; padding: 9px 11px; font: inherit; }
   textarea { resize: vertical; min-height: 76px; }
   input:focus, textarea:focus { outline: none; border-color: var(--blue); background: rgba(0,0,0,.4); }
@@ -282,8 +291,8 @@
   .and-btns { flex-basis: 100%; padding-left: 46px; }
   .and-icon { width: 36px; height: 36px; border-radius: 8px; flex: none; background: rgba(0,0,0,.25); object-fit: cover; }
   .cat-tools { display: flex; gap: 8px; flex-wrap: wrap; align-items: center; }
-  .cat-tools input[type=text] { flex: 1 1 240px; width: auto; }
-  .cat-tools select { background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px;
+  .cat-tools input[type=text], .cat-tools input[type=search] { flex: 1 1 240px; width: auto; }
+  .cat-tools select, .source-offer select { background: rgba(0,0,0,.28); color: var(--text); border: 1px solid transparent; border-radius: 3px;
                       padding: 9px 10px; font: inherit; font-size: 13px; max-width: 220px; }
   .vchips { display: flex; gap: 6px; flex-wrap: wrap; margin: 10px 0 14px; }
   .vchip { height: 26px; font-size: 12px; padding: 0 10px; opacity: .55; }
@@ -357,7 +366,7 @@
     button { height: 38px; }
     button.small { height: 32px; }
     .actions button { height: 46px; }
-    .cat-tools select { max-width: none; flex: 1 1 100%; }
+    .cat-tools select, .source-offer select { max-width: none; flex: 1 1 100%; }
     /* The on-screen keyboard covers the bottom of the screen; the tab bar would ride on top of it. */
     body.typing nav { display: none; }
   }
@@ -575,7 +584,22 @@
       </div>
       </div>
       <div class="panel">
-        <div class="shelf-head"><h2>Find apps</h2><span class="count" id="catCount"></span></div>
+        <div class="shelf-head"><h2>Find apps</h2><span class="count" id="searchCount"></span></div>
+        <form id="sourceSearch" class="cat-tools"><input id="sourceQ" type="search" aria-label="Search every APK source" placeholder="Search every APK source…"><button class="small">Search</button></form>
+        <div class="vchips" id="searchFilters"><button class="chip" data-filter="vr" aria-pressed="false">VR</button><button class="chip" data-filter="flat" aria-pressed="false">Flat</button><button class="chip" data-filter="installable" aria-pressed="false">Installable</button></div>
+        <div class="vchips" id="sourceFilters"></div>
+        <div class="hint" id="sourceStatus" role="status" aria-live="polite"></div>
+        <div class="cat-grid" id="searchGrid"></div>
+        <details style="margin-top:20px" id="sourcesPanel"><summary>Sources</summary>
+          <div class="list" id="sourcesList"></div>
+          <form id="addSource" class="source-repo-form">
+            <label>Repository URL<input name="url" type="url" required placeholder="https://example.org/fdroid/repo"></label>
+            <label>Fingerprint (optional)<input name="fingerprint" type="text" autocomplete="off"></label>
+            <button class="small">Add repository</button>
+          </form><div class="hint" id="sourceError" role="status"></div>
+        </details>
+        <div class="hint">Installable means Android 11 and arm64 requirements match. It does not guarantee the app runs. Unknown facts stay unknown.</div>
+<div hidden>        <div class="shelf-head"><h2>Find apps</h2><span class="count" id="catCount"></span></div>
         <div class="cat-tools">
           <input type="text" id="catQ" placeholder="Search F-Droid apps rated for the Frame…">
           <select id="catCat" aria-label="Category"><option value="">All categories</option></select>
@@ -584,7 +608,7 @@
         <div class="cat-grid" id="catGrid"><div class="sub">Loading the catalogue…</div></div>
         <div class="row" style="margin-top:12px"><button class="small" id="catMore" hidden>Show more</button></div>
         <div class="hint">Verdicts come from checks of each APK against what Lepton supports (Android 11,
-          64-bit ARM, no clipboard or Play Services) and from reports made on a Frame. See docs/apks.md.</div>
+          64-bit ARM, no clipboard or Play Services) and from reports made on a Frame. See docs/apks.md.</div></div>
       </div>
     </div>
   </section>
@@ -1780,6 +1804,66 @@ document.body.addEventListener("click", async e => {
 });
 loadOwned();
 
+// ---- All APK sources ----
+const sourceState = { vr: null, installable: false, source: "", apps: [], request: 0 };
+function sourcePage(url) { try { const u = new URL(url); return ["https:", "http:"].includes(u.protocol) ? u.href : ""; } catch { return ""; } }
+async function loadSources() {
+  try {
+    const { sources } = await api("/api/sources");
+    $("sourceFilters").innerHTML = `<button class="chip" data-source="" aria-pressed="${!sourceState.source}">All sources</button>` + sources.filter(s => s.enabled).map(s => `<button class="chip" data-source="${esc(s.id)}" aria-pressed="${sourceState.source === s.id}">${esc(s.name)}</button>`).join("");
+    $("sourcesList").innerHTML = sources.map(s => `<div class="item"><label class="grow"><input type="checkbox" data-enable="${esc(s.id)}" ${s.enabled ? "checked" : ""}> ${esc(s.name)} <span class="sub">${esc(s.trust || "unknown")} · ${esc(s.status || "not searched")}</span></label>${s.builtin === false ? `<button class="small" data-remove="${esc(s.id)}">Remove</button>` : ""}</div>`).join("") || `<div class="sub">No source modules are available in this build.</div>`;
+  } catch (e) { $("sourceError").textContent = e.message; }
+}
+async function searchSources() {
+  const request = ++sourceState.request;
+  $("sourceStatus").textContent = "Searching enabled sources…";
+  const params = new URLSearchParams({ q: $("sourceQ").value, source: sourceState.source, installable: String(sourceState.installable) });
+  if (sourceState.vr !== null) params.set("vr", String(sourceState.vr));
+  try {
+    const result = await api("/api/search?" + params);
+    if (request !== sourceState.request) return;
+    sourceState.apps = result.apps;
+    $("searchCount").textContent = `${result.apps.length} apps`;
+    $("sourceStatus").textContent = result.sources.map(s => `${s.name}: ${s.status === "error" ? s.error : s.status}`).join(" · ") || "No enabled sources";
+    $("searchGrid").innerHTML = result.apps.map((a, i) => `<article class="cat-card"><h3>${esc(a.name || "Unnamed app")}</h3><div class="sub">${esc(a.package || "Package unknown")}</div><p>${esc(a.summary || "")}</p><div class="row">${[...new Set(a.offers.map(o => o.source_name))].map(n => `<span class="tag">${esc(n)}</span>`).join(" ")}</div><div class="source-offer"><label>Get from <select aria-label="Source for ${esc(a.name)}" data-offer="${i}">${a.offers.map((o, j) => `<option value="${j}">${esc(o.source_name)} · ${esc(o.version || "Version unknown")} · ${o.verified === true ? "Verified" : "Unverified"}</option>`).join("")}</select></label><div id="offerInfo${i}"></div></div></article>`).join("") || `<div class="sub">No matching apps. Try another search or enable more sources.</div>`;
+    result.apps.forEach((_, i) => renderOffer(i, 0));
+    await loadSources();
+  } catch (e) { if (request === sourceState.request) $("sourceStatus").textContent = e.message; }
+}
+function renderOffer(i, j) {
+  const o = sourceState.apps[i].offers[j], page = sourcePage(o.page), busy = installing.has(`${o.source}:${o.id}`);
+  $("offerInfo" + i).innerHTML = `<p class="sub">${esc(o.fit.verdict)} · API ${esc(o.min_sdk ?? "unknown")} · ${esc(o.abis == null ? "ABI unknown" : o.abis.join(", ") || "No native code")}<br>Version code ${esc(o.version_code ?? "unknown")} · ${o.size == null ? "Size unknown" : (o.size / 1048576).toFixed(1) + " MB"} · Updated ${esc(o.updated || "unknown")}</p><p class="hint">${o.fit.reasons.map(esc).join(" · ")}</p><div class="row">${o.downloadable && o.fit.installable !== false ? `<button class="small action" data-source-install="${i}" data-choice="${j}" ${busy ? "disabled" : ""}>${busy ? "Downloading and installing…" : "Install"}</button>` : ""}${page ? `<a href="${esc(page)}" target="_blank" rel="noopener noreferrer">Developer page</a>` : ""}</div>`;
+}
+$("sourceSearch").onsubmit = e => { e.preventDefault(); searchSources(); };
+$("searchFilters").onclick = e => {
+  const b = e.target.closest("[data-filter]"); if (!b) return;
+  if (b.dataset.filter === "installable") sourceState.installable = !sourceState.installable;
+  else { const value = b.dataset.filter === "vr"; sourceState.vr = sourceState.vr === value ? null : value; }
+  $("searchFilters").querySelectorAll("button").forEach(b => b.setAttribute("aria-pressed", b.dataset.filter === "installable" ? sourceState.installable : sourceState.vr === (b.dataset.filter === "vr")));
+  searchSources();
+};
+$("sourceFilters").onclick = e => { const b = e.target.closest("[data-source]"); if (b) { sourceState.source = b.dataset.source; searchSources(); } };
+$("searchGrid").onchange = e => { if (e.target.matches("[data-offer]")) renderOffer(+e.target.dataset.offer, +e.target.value); };
+$("searchGrid").onclick = async e => {
+  const b = e.target.closest("[data-source-install]"); if (!b) return;
+  const o = sourceState.apps[+b.dataset.sourceInstall].offers[+b.dataset.choice];
+  if (installing.has(`${o.source}:${o.id}`)) return;
+  b.disabled = true; b.textContent = "Downloading and installing…";
+  const result = await runJob(`Install ${o.name}`, `${o.source}:${o.id}`, () => api("/api/sources/install", {source: o.source, id: o.id, version_code: o.version_code ?? null}));
+  b.disabled = false; b.textContent = result ? "Installed" : "Retry install";
+  if (result) loadAndroid();
+  if (!b.isConnected) searchSources();
+};
+async function changeSource(body) {
+  $("sourceError").textContent = "Saving…";
+  try { const result = await api("/api/sources", body); $("sourceError").textContent = result.message; sourceState.source = ""; await loadSources(); await searchSources(); }
+  catch (e) { $("sourceError").textContent = e.message; await loadSources(); }
+}
+$("sourcesList").onchange = e => { if (e.target.matches("[data-enable]")) changeSource({action: "enable", source: e.target.dataset.enable, enabled: e.target.checked}); };
+$("sourcesList").onclick = e => { const b = e.target.closest("[data-remove]"); if (b) changeSource({action: "remove", source: b.dataset.remove}); };
+$("addSource").onsubmit = e => { e.preventDefault(); const data = new FormData(e.target); changeSource({action: "add", url: data.get("url"), fingerprint: data.get("fingerprint")}); };
+loadSources().then(searchSources);
+
 // ---- Android apps (own Lepton instance each) + catalogue ----
 const VLABEL = { works: "Works on Frame", likely: "Should work", maybe: "Might work", unlikely: "Probably crashes", no: "Won't work" };
 const cat = { apps: null, q: "", cat: "", v: new Set(["works", "likely"]), shown: 0, results: [] };

ui/server.py

@@ -37,6 +37,7 @@ from urllib.parse import parse_qs, unquote, urlparse
 sys.path.insert(0, str(Path(__file__).resolve().parent))
 
 import frame_android  # noqa: E402
+from apk_sources import search as apk_search, SourceError  # noqa: E402
 import frame_apk_versions  # noqa: E402
 import frame_catalog  # noqa: E402
 import frame_host  # noqa: E402
@@ -1227,7 +1228,66 @@ def _sweep_one(prefix, d):
         pass
 
 
-POST = {"/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
+def source_text(body, key, optional=False):
+    value = body.get(key)
+    if optional and value in (None, ''):
+        return None
+    if not isinstance(value, str) or not value.strip() or len(value) > 2000:
+        raise Failure('Provide a valid ' + key, 400)
+    return value.strip()
+
+
+def source_search(query):
+    args = parse_qs(query)
+    q = args.get('q', [''])[0]
+    vr = args.get('vr', [''])[0]
+    installable = args.get('installable', [''])[0]
+    if len(q) > 500 or vr not in ('', 'true', 'false', '1', '0') or installable not in ('', 'true', 'false', '1', '0'):
+        raise Failure('Invalid search filters', 400)
+    try:
+        return apk_search.search(q, vr=None if not vr else vr in ('true', '1'),
+                                 source=args.get('source', [None])[0], installable=installable in ('true', '1'))
+    except SourceError as e:
+        raise Failure(str(e), 400)
+
+
+def source_install(body):
+    source, entry = source_text(body, 'source'), source_text(body, 'id')
+    version = body.get('version_code')
+    if version is not None and (type(version) is not int or version < 0):
+        raise Failure('version_code must be a non-negative integer', 400)
+    try:
+        _, selected = apk_search.resolve(source)
+        if not selected['enabled']:
+            raise SourceError('This source is disabled')
+    except SourceError as e:
+        raise Failure(str(e), 400)
+    return start_job('Install ' + entry, lambda: apk_search.install(source, entry, version))
+
+
+def source_manage(body):
+    action = body.get('action')
+    try:
+        if action == 'enable':
+            if type(body.get('enabled')) is not bool:
+                raise Failure('enabled must be true or false', 400)
+            return apk_search.set_enabled(source_text(body, 'source'), body['enabled'])
+        if action == 'add':
+            url = source_text(body, 'url')
+            if urlparse(url).scheme != 'https' or not urlparse(url).hostname or urlparse(url).username:
+                raise Failure('Use an HTTPS repository URL without credentials', 400)
+            apk_search.manage_repo('add_repo', url=url, fingerprint=source_text(body, 'fingerprint', True),
+                                   name=source_text(body, 'name', True))
+            return {'message': 'Repository added'}
+        if action == 'remove':
+            apk_search.manage_repo('remove_repo', source_id=source_text(body, 'source'))
+            return {'message': 'Repository removed'}
+        raise Failure('Unknown source action', 400)
+    except SourceError as e:
+        raise Failure(str(e), 400)
+
+
+POST = {"/api/sources": source_manage, "/api/sources/install": source_install, "/api/android/display": android_display, "/api/android": android, "/api/titles": titles, "/api/launch": launch, "/api/steam": steam, "/api/volume": set_volume, "/api/clipboard": clipboard,
         "/api/flatpak": flatpak, "/api/open": open_thing, "/api/shots/save": save_shots,
         "/api/webinstall/check": webinstall_check, "/api/webinstall/start": webinstall_start,
         "/api/webinstall/cancel": webinstall_cancel}
@@ -1335,6 +1395,10 @@ class Handler(BaseHTTPRequestHandler):
                 self.send_json({"os": "SteamOS", "fileManager": None, "computer": DEVICE, "mobile": True} if LOCAL else
                                {"os": frame_host.NAME, "fileManager": frame_host.FILE_MANAGER,
                                 "computer": "Mac" if frame_host.MAC else "PC"})
+            elif path == "/api/sources":
+                self.send_json({"sources": apk_search.sources()})
+            elif path == "/api/search":
+                self.send_json(source_search(url.query))
             elif path == "/api/apk-versions":
                 self.send_json(apk_versions(url.query))
             elif path == "/api/android":