mirror of
https://github.com/DeeJanuz/frametop.git
synced 2026-10-06 05:00:08 +02:00
hand recorder: log in from the Upload page, three-step page, no terminal
The Upload page is now three numbered steps: choose the export, log in to Hugging Face, upload. Log in runs hub.py login, huggingface_hub's browser login (OAuth device code, as hf auth login does): the link opens in the browser and the page shows the code to enter, with Copy code and Cancel. hub.py saves the token; the window never sees one, and nobody pastes one. The terminal upload and the login command are gone from the page, and UPLOAD.md is now a short 'About uploading' under the steps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
DeeJanuzandClaude Opus 5.5 committed
1 parent
e62ebb8669
commit
ea483f7ce3
6 files changed
+440
-335
No files matched your search
+12
-4
@@ -313,7 +313,7 @@ On 2026-10-02 a whole session showed "I can't see your hands": after the headset
|
||||
- `SHA256SUMS`.
|
||||
|
||||
Compression runs at nice 19. While it runs with the headset worn, the window notes that VR may stutter a little (exports are done in the headset). Worn is judged the way `frame-job` does: `vrcompositor` runs and a `/sys/class/backlight/*/brightness` reads over 0 (SteamVR turns the panel off 5 s after the headset comes off). CPU work while in VR causes stutter. The proximity sensor is no use here: it read 9-43 with the headset sitting unworn.
|
||||
- **Upload.** The Upload page uploads an export from the window. It also keeps the manual way as a fallback: `UPLOAD.md` with the export's path and size filled in, plus the copyable command (`validate.py`, then `hf upload ... --create-pr`).
|
||||
- **Upload.** The Upload page uploads an export from the window, logging in included: no terminal. Below its steps it shows `UPLOAD.md` ("About uploading"), with the export's path, size and contributor id filled in.
|
||||
- **`hands/rec/validate.py`** (standard library) checks an export. The window runs it before an upload, and the maintainer runs it on each submission (`validate.py DIR [--json]`, exit status 1 on errors). It checks:
|
||||
- `SHA256SUMS`: every file listed and matching.
|
||||
- An allow-list: `manifest.json`, `calibration.json`, `device.json` and `SHA256SUMS` at the top, and `prompts.jsonl`, `poses.jsonl`, `take.json` and `sets.bin.zst` in `takes/<NN>-<section>/`. Anything else is an error, and so is a symlink.
|
||||
@@ -329,7 +329,10 @@ On 2026-10-02 a whole session showed "I can't see your hands": after the headset
|
||||
Warnings cover notes kept in the export, a home folder path in the manifest, and missing `poses.jsonl` files.
|
||||
|
||||
It runs on Linux and on Windows (the maintainer's PC) with Python 3.12 or later. It decompresses with Python 3.14's `compression.zstd`, else the `zstandard` package, else the `zstd` program, and handles several zstd frames in a row.
|
||||
- **`hands/rec/hub.py`** does the upload. It runs as a child process of the window (Cancel ends it), or from the command line (`hub.py [--base DIR] whoami | upload SESSION [--dry-run] [--again] [--json]`). It uses `huggingface_hub` (`python3-huggingface-hub` in the dev container) with the token `hf auth login` saved, and never handles a token itself. An upload goes through these steps:
|
||||
- **`hands/rec/hub.py`** does the upload. It runs as a child process of the window (Cancel ends it), or from the command line (`hub.py [--base DIR] whoami | login | upload SESSION [--dry-run] [--again] [--json]`). It uses `huggingface_hub` (`python3-huggingface-hub` in the dev container) with the login saved in huggingface_hub's token file.
|
||||
- **`login`** is huggingface_hub's browser login (OAuth device code), the same as `hf auth login`'s default since huggingface_hub 1.x. It asks Hugging Face for a link (`https://hf.co/oauth/device`) and a short code, prints them (`{"phase": "code", "url", "code", "expires_in"}`), and waits while the person enters the code in their browser and approves. Then it saves the token, which can refresh itself. Nobody types or pastes a token, and the window never sees one. It uses huggingface_hub's own helpers (`request_device_code`, `poll_device_token`, `_save_oauth_token`), as there's no public call that hands back the code. On 2026-10-03 the code lasted 5 minutes and wasn't filled into the link. The consent screen lists the person's organizations: none are needed, as a pull request on a public dataset comes from the person's own account.
|
||||
|
||||
An upload goes through these steps: An upload goes through these steps:
|
||||
1. Validate, and stop on errors.
|
||||
2. Stop if the same export was uploaded before (same `SHA256SUMS`), unless asked again.
|
||||
3. Stop while the texts are drafts, unless `FT_HANDREC_ALLOW_UPLOAD=1`.
|
||||
@@ -339,8 +342,13 @@ On 2026-10-02 a whole session showed "I can't see your hands": after the headset
|
||||
7. `upload_folder(repo_id=HF_DATASET, repo_type="dataset", folder_path=EXPORT, path_in_repo="contributions/<contributor>/<session>", revision="refs/pr/N", commit_message=..., commit_description=...)`. The description summarizes the manifest: takes, minutes, sets, lighting, objects, controllers, the consent and tool versions, the size, and validate's warnings. Then mark the pull request open if it's still a draft (the maintainer's `list` shows open ones, so drafts are uploads still in progress).
|
||||
8. Mark the record `"status": "done"` with `uploaded`. `export_sha` is the SHA256 of `SHA256SUMS`. The file keeps its modification time, so the export doesn't count as out of date. Only finished records count as "uploaded before" (records without a status are from before 2026-10-03 and finished).
|
||||
|
||||
Errors get a plain explanation: not logged in, a token Hugging Face rejects (401), a token that can't open a pull request (403), terms not accepted, dataset not found, network errors. `--dry-run` does everything except the network calls and the record, and lists what it would upload.
|
||||
- **The page** shows the login (`whoami`, with "Check again"). If nobody is logged in, it explains how to run `XDG_RUNTIME_DIR=/run/user/$(id -u) distrobox enter dev -- hf auth login` in Konsole (Frametop's Konsole has its own runtime folder, where podman can't find the container) with a write token: the token goes only into that terminal. The page then has Upload and Cancel, the phase with a progress bar (a share while the export is checked, a sweep while it's sent, as `huggingface_hub` reports no progress), the pull request's link once it's open, with "plug in the headset and leave it plugged in until this says Uploaded", and Uploaded at the end. If this export was uploaded before, the page says so, and uploading it again asks first. A stale export can't be uploaded.
|
||||
Errors get a plain explanation: not logged in, a login Hugging Face rejects (401), a login that can't open a pull request (403), terms not accepted, dataset not found, network errors. `--dry-run` does everything except the network calls and the record, and lists what it would upload.
|
||||
- **The page** has three numbered steps:
|
||||
1. Choose the export, with its size.
|
||||
2. Log in to Hugging Face. The page shows whether someone is logged in (`whoami`). Log in runs `hub.py login`, opens the link in the browser, and shows the code large, with Copy code and Cancel. "Use another account" logs in again. A classic read-only token counts as not logged in.
|
||||
3. Upload, with a note to accept the dataset's terms the first time.
|
||||
|
||||
Upload has Cancel, the phase with a progress bar (a share while the export is checked, a sweep while it's sent, as `huggingface_hub` reports no progress), the pull request's link once it's open, with "plug in the headset and leave it plugged in until this says Uploaded", and Uploaded at the end. If this export was uploaded before, the page says so, and uploading it again asks first. A stale export can't be uploaded.
|
||||
- **Staying awake:** while an export or an upload runs, the window holds a host unit, `frametop-handrec-awake.service`, running `systemd-inhibit --what=sleep:idle --mode=block ... sleep infinity`, and stops it when the last of them ends (or on quit). It's meant to keep Steam from putting the Frame to sleep with the headset off; whether Steam's sleep honours a logind block inhibitor is still to be checked on the device. Exports and uploads are done in the headset: the export page only notes that VR may stutter a little while it compresses.
|
||||
- **While the texts are drafts**, Upload stays off unless `FT_HANDREC_ALLOW_UPLOAD=1`, so the maintainer can rehearse against a private test repo. `FT_HANDREC_DATASET` overrides `HF_DATASET`. `ft-handrec --hub-dry-run` makes Upload a dry run: no network, so it isn't held back by the drafts.
|
||||
- **Rehearsal: `hands/rec/rehearse.sh [--repo ID]`** runs it all without the headset, in the dev container, in one `frame-job --local` scope when frame-job is installed. `ft-ringplay` plays 30 s of a recording into a ring in `/run/user/UID`. A tracking ft-hands that's already running is used, or one is started on that ring. `ft-handpanel --no-vr` stands in for the panel. `session.py --no-start --next-after 0.3` records a three-section test script (a prompt, a two-cue sweep, no hands) in step mode, three parts of 6 s (countdown and hold), about 360 MB once exported. Then `takes.py` exports, `validate.py` checks, and `hub.py` uploads: a dry run by default, or for real to `--repo ID` with `FT_HANDREC_ALLOW_UPLOAD=1`. It prints a summary, deletes its temporary folders (camera images of a room) and stops everything it started, Ctrl+C included. The `--no-vr` panel logs no poses, so `poses.jsonl` is missing there (a warning).
|
||||
|
||||
+8
-60
@@ -1,63 +1,11 @@
|
||||
**DRAFT: contributions aren't open yet. Please don't upload until this banner is gone.**
|
||||
|
||||
# Uploading your recordings
|
||||
### About uploading
|
||||
|
||||
Your export is ready:
|
||||
|
||||
- Folder: `@EXPORT_PATH@`
|
||||
- Size: @EXPORT_SIZE@
|
||||
- Contributor id: `@CONTRIBUTOR@`
|
||||
|
||||
Uploads go to the Hugging Face dataset [@DATASET@](https://huggingface.co/datasets/@DATASET@), from your own Hugging Face account. Nothing is uploaded until you press Upload (or run the command below). The upload opens a pull request, so nothing is published until the maintainer has looked at it.
|
||||
|
||||
You can do every step below in the headset: the web pages in a browser window, the login in a terminal window. Once your pull request is open, you plug the headset in and leave it to finish.
|
||||
|
||||
## 1. Make a Hugging Face account
|
||||
|
||||
Sign up at <https://huggingface.co/join>, if you don't have an account. Your username shows on your pull request, but the dataset credits your contributor id, not your name.
|
||||
|
||||
## 2. Accept the dataset's terms
|
||||
|
||||
Open <https://huggingface.co/datasets/@DATASET@>, read the terms and accept them. They're the same as the consent you agreed to in the hand recorder. Until you've accepted them, the upload stops with "accept the dataset's terms first".
|
||||
|
||||
## 3. Create a write token
|
||||
|
||||
Go to <https://huggingface.co/settings/tokens>, press "Create new token", choose "Write" and give it a name like "frametop-hands". Copy the token.
|
||||
|
||||
The token lets anyone who has it change things in your account. Paste it only into your own terminal in the next step: never into a chat, a website, an issue or this window. The hand recorder never asks for it.
|
||||
|
||||
## 4. Log in, in a terminal
|
||||
|
||||
Open a terminal (Konsole) and run:
|
||||
|
||||
```
|
||||
@LOGIN@
|
||||
```
|
||||
|
||||
It asks for the token: paste it there (it doesn't show as you paste) and press Enter. The token is saved in your home folder, in `~/.cache/huggingface/token`, where the hand recorder's upload finds it. Then press "Check again" on this page: it should say you're logged in.
|
||||
|
||||
If the page says `huggingface_hub` isn't installed, update the dev container first: run `setup/dev-container.sh` from the Frametop folder.
|
||||
|
||||
## 5. Upload
|
||||
|
||||
Press **Upload** on this page. It first checks the export (that every file is complete and matches its checksum, and that nothing identifying is left in). Then it opens your pull request and shows its link, and starts uploading the files to it, into `contributions/@CONTRIBUTOR@/@SESSION@` in the dataset.
|
||||
|
||||
**Once the link shows, plug in the headset and leave it plugged in until the page says Uploaded.** A round is several gigabytes, so this can take a while. You can take the headset off: the Hand Recorder keeps it awake until the upload is done. Keep the Hand Recorder open, since closing it stops the upload.
|
||||
|
||||
If it stops partway (Cancel, or the network drops), press Upload again: it carries on in the same pull request, and files already sent aren't sent twice.
|
||||
|
||||
### Or upload from a terminal
|
||||
|
||||
If the Upload button doesn't work for you, run this in the dev container (`distrobox enter dev`) after step 4. It checks the export the same way, then uploads it:
|
||||
|
||||
```
|
||||
@COMMAND@
|
||||
```
|
||||
|
||||
## 6. The maintainer reviews it
|
||||
|
||||
The maintainer checks your pull request (that the files are complete, and that nobody else and nothing private is in view) before merging it into the dataset. You can follow it, and answer questions, on the pull request's page.
|
||||
|
||||
When it's merged, you can delete the session and its export on your headset to free the space.
|
||||
|
||||
To withdraw a contribution later, see "Withdrawing" in the consent text: you'll need your contributor id, `@CONTRIBUTOR@`.
|
||||
- **What's sent:** the export in `@EXPORT_PATH@` (@EXPORT_SIZE@), into `contributions/@CONTRIBUTOR@/@SESSION@` in [@DATASET@](https://huggingface.co/datasets/@DATASET@). Upload first checks that every file is complete and matches its checksum, and that nothing identifying is left in.
|
||||
- **Your account:** the pull request comes from your Hugging Face account, and your username shows on it. The dataset credits your contributor id, `@CONTRIBUTOR@`, not your name. The login stays saved on this headset, so you only log in once.
|
||||
- **The dataset's terms** are the same as the consent you agreed to in the hand recorder. Until you've accepted them on the dataset's page, Upload stops with "accept the dataset's terms first".
|
||||
- **Once your pull request's link shows, plug in the headset and leave it plugged in until this page says Uploaded.** A round is several gigabytes, so this can take a while. You can take the headset off: the Hand Recorder keeps it awake until the upload is done. Keep the Hand Recorder open, since closing it stops the upload.
|
||||
- **If it stops partway** (Cancel, or the network drops), press Upload again. It carries on in the same pull request, and files already sent aren't sent twice.
|
||||
- **The maintainer's review:** they check that the files are complete, and that nobody else and nothing private is in view, before merging. You can follow it and answer questions on the pull request's page. Once it's merged, you can delete the session and its export here to free the space.
|
||||
- **Withdrawing:** see "Withdrawing" in the consent text. You'll need your contributor id, `@CONTRIBUTOR@`.
|
||||
+75
-30
@@ -31,7 +31,6 @@ import datetime
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shlex
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -40,7 +39,7 @@ import time
|
||||
import uuid
|
||||
|
||||
from PySide6.QtCore import Property, QObject, Qt, QTimer, QUrl, Signal, Slot
|
||||
from PySide6.QtGui import QColor, QFont, QGuiApplication, QIcon, QImage, QPainter, QPalette
|
||||
from PySide6.QtGui import QColor, QDesktopServices, QFont, QGuiApplication, QIcon, QImage, QPainter, QPalette
|
||||
from PySide6.QtQml import QQmlApplicationEngine
|
||||
from PySide6.QtQuick import QQuickImageProvider
|
||||
from PySide6.QtQuickControls2 import QQuickStyle
|
||||
@@ -57,10 +56,6 @@ UPLOAD_PATH = hub.UPLOAD_PATH
|
||||
HUB_PATH = os.path.join(HERE, "hub.py")
|
||||
VALIDATE_PATH = os.path.join(HERE, "validate.py")
|
||||
AWAKE_UNIT = "frametop-handrec-awake.service"
|
||||
# What `hf auth login` needs: run in a terminal, where the token is typed (never in this window).
|
||||
# Frametop's Konsole has XDG_RUNTIME_DIR=/run/user/UID/frametop, where podman finds no container
|
||||
# state ("crun: error opening file .../status"), so the command sets the real one.
|
||||
LOGIN_COMMAND = "XDG_RUNTIME_DIR=/run/user/$(id -u) distrobox enter dev -- hf auth login"
|
||||
SCRIPT_PATH = os.path.join(HERE, "script.json")
|
||||
# The headset counts as worn while vrcompositor runs and a display panel is lit: SteamVR turns
|
||||
# the panels off 5 s after the headset comes off (frame-job's check; the proximity sensor's
|
||||
@@ -220,6 +215,7 @@ class Backend(QObject):
|
||||
loginChanged = Signal()
|
||||
uploadChanged = Signal()
|
||||
_loginArrived = Signal(dict)
|
||||
_loginLine = Signal(dict)
|
||||
_uploadLine = Signal(dict)
|
||||
_uploadFinished = Signal(dict)
|
||||
cameraChanged = Signal()
|
||||
@@ -232,6 +228,8 @@ class Backend(QObject):
|
||||
self._hub_dry_run = hub_dry_run
|
||||
self._login = {"state": "dry" if hub_dry_run else "unknown"}
|
||||
self._login_busy = False
|
||||
self._login_proc = None # hub.py login: waiting for the browser
|
||||
self._login_cancelled = False
|
||||
self._upload_proc = None
|
||||
self._upload_thread = None
|
||||
self._upload_cancelled = False
|
||||
@@ -263,6 +261,7 @@ class Backend(QObject):
|
||||
self._exportProgress.connect(self._on_export_progress)
|
||||
self._exportFinished.connect(self._on_export_finished)
|
||||
self._loginArrived.connect(self._on_login)
|
||||
self._loginLine.connect(self._on_login_line)
|
||||
self._uploadLine.connect(self._on_upload_line)
|
||||
self._uploadFinished.connect(self._on_upload_finished)
|
||||
self.disk_timer = QTimer(interval=30000, timeout=self.diskChanged.emit)
|
||||
@@ -646,6 +645,7 @@ class Backend(QObject):
|
||||
if self._export_cancel:
|
||||
self._export_cancel.set() # and wait, so export can remove its half-written copy
|
||||
self._export_thread.join(15)
|
||||
self.cancelLogin()
|
||||
if self._upload_proc:
|
||||
self.cancelUpload()
|
||||
self._upload_thread.join(10)
|
||||
@@ -901,17 +901,14 @@ class Backend(QObject):
|
||||
def allowUploadSet(self):
|
||||
return os.environ.get(hub.ALLOW_ENV) == "1"
|
||||
|
||||
@Property(str, constant=True)
|
||||
def loginCommand(self):
|
||||
return LOGIN_COMMAND
|
||||
|
||||
def _hub_argv(self, *args):
|
||||
return [sys.executable, HUB_PATH, "--base", self.store.base, *args]
|
||||
|
||||
# The login: hub.py whoami in a child process (it asks huggingface.co)
|
||||
@Property("QVariantMap", notify=loginChanged)
|
||||
def login(self):
|
||||
"""{"state": unknown|checking|ok|none|read|error|missing|dry, "name", "role", "text", "link"}."""
|
||||
"""{"state": unknown|checking|ok|none|read|error|missing|dry|starting|waiting, "name", "role",
|
||||
"text", "link"}; while waiting, "url" and "code" (the browser login's)."""
|
||||
return self._login
|
||||
|
||||
@Slot()
|
||||
@@ -946,13 +943,18 @@ class Backend(QObject):
|
||||
|
||||
def _on_login(self, result):
|
||||
self._login_busy = False
|
||||
self._login_proc = None
|
||||
if result.get("cancelled"):
|
||||
self._login = {"state": "unknown"}
|
||||
self.checkLogin() # whatever was saved before is still there
|
||||
return
|
||||
if "done" in result:
|
||||
who = result["done"]
|
||||
role = who.get("role", "")
|
||||
if role == "read":
|
||||
self._login = {"state": "read", "name": who.get("name", ""), "role": role, "link": hub.TOKENS_URL,
|
||||
"text": f"Logged in as {who.get('name', '')}, but with a read-only token: uploads "
|
||||
"need a token with the Write role. Log in again with one."}
|
||||
"text": f"Logged in as {who.get('name', '')}, but with a read-only token, which "
|
||||
"can't upload. Log in again."}
|
||||
else:
|
||||
self._login = {"state": "ok", "name": who.get("name", ""), "role": role,
|
||||
"text": f"Logged in as {who.get('name', '')}"}
|
||||
@@ -962,6 +964,62 @@ class Backend(QObject):
|
||||
self._login = {"state": state, "text": e.get("text", "The login check failed"), "link": e.get("link", "")}
|
||||
self.loginChanged.emit()
|
||||
|
||||
# The browser login: hub.py login --json in a child process. It gets a link and a short code;
|
||||
# the link opens in the browser, the person types the code there (Hugging Face doesn't fill it
|
||||
# in, 2026-10-03) and approves, and hub.py saves the token. The token never reaches this process.
|
||||
@Slot()
|
||||
def logIn(self):
|
||||
if self._hub_dry_run or self._login_proc or self._login_busy:
|
||||
return
|
||||
try:
|
||||
proc = subprocess.Popen(self._hub_argv("login", "--json"), stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, bufsize=1)
|
||||
except OSError as e:
|
||||
self._on_login({"error": {"kind": "hub", "text": f"Couldn't start the login: {e}"}})
|
||||
return
|
||||
self._login_proc = proc
|
||||
self._login_cancelled = False
|
||||
self._login_busy = True
|
||||
self._login = {"state": "starting", "text": "Getting a login link from Hugging Face"}
|
||||
self.loginChanged.emit()
|
||||
|
||||
def run():
|
||||
last = {}
|
||||
for line in proc.stdout:
|
||||
try:
|
||||
obj = json.loads(line)
|
||||
except ValueError:
|
||||
continue
|
||||
if isinstance(obj, dict):
|
||||
if "done" in obj or "error" in obj:
|
||||
last = obj
|
||||
else:
|
||||
self._loginLine.emit(obj)
|
||||
err = proc.stderr.read()
|
||||
proc.wait()
|
||||
proc.stdout.close()
|
||||
proc.stderr.close()
|
||||
if self._login_cancelled:
|
||||
last = {"cancelled": True}
|
||||
elif not last:
|
||||
tail = (err.strip().splitlines() or [f"exit status {proc.returncode}"])[-1]
|
||||
last = {"error": {"kind": "hub", "text": f"The login stopped: {tail}"}}
|
||||
self._loginArrived.emit(last)
|
||||
self._thread(run)
|
||||
|
||||
@Slot()
|
||||
def cancelLogin(self):
|
||||
if self._login_proc and self._login_proc.poll() is None:
|
||||
self._login_cancelled = True
|
||||
self._login_proc.terminate()
|
||||
|
||||
def _on_login_line(self, obj):
|
||||
if obj.get("phase") == "code" and obj.get("url"):
|
||||
self._login = {"state": "waiting", "url": obj["url"], "code": obj.get("code", ""),
|
||||
"expires_in": obj.get("expires_in", 0), "text": obj.get("text", "")}
|
||||
self.loginChanged.emit()
|
||||
QDesktopServices.openUrl(QUrl(obj["url"]))
|
||||
|
||||
# The upload: hub.py upload --json in a child process; its lines say how it goes
|
||||
@Property(bool, notify=uploadChanged)
|
||||
def uploading(self):
|
||||
@@ -1073,31 +1131,18 @@ class Backend(QObject):
|
||||
self.uploadChanged.emit()
|
||||
self.sessionsChanged.emit()
|
||||
|
||||
# The manual way, for a terminal (UPLOAD.md)
|
||||
@Slot(str, result=str)
|
||||
def uploadCommand(self, session):
|
||||
try:
|
||||
path = self.store.export_dir(session)
|
||||
except ValueError:
|
||||
return ""
|
||||
contributor = self.contributor or "CONTRIBUTOR"
|
||||
check = " ".join(["python3", shlex.quote(VALIDATE_PATH), shlex.quote(path)])
|
||||
upload = " ".join(["hf", "upload", self.dataset, shlex.quote(path), f"contributions/{contributor}/{session}",
|
||||
"--repo-type", "dataset", "--create-pr",
|
||||
"--commit-message", shlex.quote(f"Hands: session {session} from {contributor}")])
|
||||
return check + " && " + upload
|
||||
|
||||
@Slot(str, result=str)
|
||||
def uploadText(self, session):
|
||||
"""UPLOAD.md with this export's path, size and command filled in."""
|
||||
"""UPLOAD.md with this export's path and size filled in."""
|
||||
try:
|
||||
path = self.store.export_dir(session)
|
||||
except ValueError:
|
||||
return ""
|
||||
values = {"EXPORT_PATH": path, "EXPORT_SIZE": takes.human_bytes(takes.tree_bytes(path)),
|
||||
"CONTRIBUTOR": self.contributor or "CONTRIBUTOR", "SESSION": session, "DATASET": self.dataset,
|
||||
"COMMAND": self.uploadCommand(session), "LOGIN": LOGIN_COMMAND}
|
||||
"CONTRIBUTOR": self.contributor or "CONTRIBUTOR", "SESSION": session, "DATASET": self.dataset}
|
||||
text = read_text(UPLOAD_PATH) or "UPLOAD.md is missing."
|
||||
if hub.is_draft(UPLOAD_PATH):
|
||||
text = text.split("\n", 1)[-1] # the page's banner says it already
|
||||
return re.sub(r"@([A-Z_]+)@", lambda m: values.get(m.group(1), m.group(0)), text)
|
||||
|
||||
@Slot(QColor)
|
||||
|
||||
+56
-12
@@ -3,8 +3,13 @@
|
||||
|
||||
The window runs this as a child process (so Cancel can end it, and huggingface_hub stays out
|
||||
of the window's process); it also runs from the command line. It uses huggingface_hub (in the
|
||||
dev container: python3-huggingface-hub, from setup/dev-container.sh) with the token that
|
||||
`hf auth login` saved. It never asks for or handles a token itself.
|
||||
dev container: python3-huggingface-hub, from setup/dev-container.sh) with the login that
|
||||
`hub.py login` (the window's Log in) or `hf auth login` saved. Nobody types a token anywhere.
|
||||
|
||||
`login` is huggingface_hub's browser login (OAuth device code, as `hf auth login` does): it gets a
|
||||
link and a short code, the person enters the code in their browser and approves, and the token goes
|
||||
straight from Hugging Face into huggingface_hub's token file. This process saves it; it never
|
||||
prints it, and the window never sees it.
|
||||
|
||||
An upload:
|
||||
1. checks the export with validate.py, and stops on errors;
|
||||
@@ -23,10 +28,12 @@ it would upload.
|
||||
The dataset is HF_DATASET; FT_HANDREC_DATASET overrides it (a test repo, for rehearsals).
|
||||
|
||||
usage: hub.py [--base DIR] whoami [--json]
|
||||
hub.py [--base DIR] login [--json]
|
||||
hub.py [--base DIR] upload SESSION [--dry-run] [--again] [--json]
|
||||
With --json, each line of output is one JSON object: {"phase", "text", "fraction"} as it goes,
|
||||
with {"phase": "opened", "pr_url"} once the pull request exists, then {"done": {...}} or
|
||||
{"error": {"kind", "text", "link", "errors"}}. Exit status 0: done.
|
||||
{"error": {"kind", "text", "link", "errors"}}. login says {"phase": "code", "url", "code",
|
||||
"expires_in"} once it has the link, then {"done": {"name", "role"}}. Exit status 0: done.
|
||||
"""
|
||||
import argparse
|
||||
import datetime
|
||||
@@ -204,7 +211,7 @@ def explain(e, repo):
|
||||
url = dataset_url(repo)
|
||||
if hf is not None:
|
||||
if isinstance(e, hf.LocalTokenNotFoundError):
|
||||
return HubError("login", "You're not logged in to Hugging Face. Log in with hf auth login (see below).")
|
||||
return HubError("login", "You're not logged in to Hugging Face. Press Log in.")
|
||||
if isinstance(e, hf.GatedRepoError):
|
||||
return HubError("terms", f"Accept the dataset's terms first: open {url}, read them and accept them, "
|
||||
"then try again.", url)
|
||||
@@ -215,12 +222,11 @@ def explain(e, repo):
|
||||
status = getattr(getattr(e, "response", None), "status_code", None)
|
||||
first = str(e).strip().splitlines()[0] if str(e).strip() else type(e).__name__
|
||||
if status == 401:
|
||||
return HubError("login", "Hugging Face didn't accept your token (it may have been deleted or have "
|
||||
"expired). Log in again with hf auth login.", TOKENS_URL)
|
||||
return HubError("login", "Hugging Face didn't accept your login (it may have expired or been "
|
||||
"revoked). Log in again.")
|
||||
if status == 403:
|
||||
return HubError("permission", "Your token isn't allowed to open a pull request. Create a token with "
|
||||
"the Write role and log in again with it: hf auth login --force.",
|
||||
TOKENS_URL)
|
||||
return HubError("permission", "Your login isn't allowed to open a pull request. Log in again, "
|
||||
"and allow everything the Hugging Face page asks for.")
|
||||
return HubError("hub", f"Hugging Face refused the upload ({status or 'no status'}): {first}")
|
||||
try:
|
||||
import httpx
|
||||
@@ -245,6 +251,31 @@ def whoami():
|
||||
return {"name": info.get("name", ""), "role": token.get("role", "")}
|
||||
|
||||
|
||||
def login(progress=None):
|
||||
"""The browser login: progress("code", text, url=, code=, expires_in=) once the link is ready,
|
||||
then wait (up to the code's expiry: 5 minutes on 2026-10-03) for the person to approve it, and save
|
||||
the token. Returns whoami(). Uses huggingface_hub's own device code helpers (1.x)."""
|
||||
tell = progress or (lambda phase, text, **extra: None)
|
||||
_hf()
|
||||
try:
|
||||
from huggingface_hub._login import _save_oauth_token
|
||||
from huggingface_hub.errors import DeviceCodeError
|
||||
from huggingface_hub.utils._oauth_device import poll_device_token, request_device_code
|
||||
except ImportError:
|
||||
raise HubError("missing", "This huggingface_hub has no browser login: update the dev container "
|
||||
"(setup/dev-container.sh).") from None
|
||||
try:
|
||||
info = request_device_code()
|
||||
tell("code", "Approve the login in your browser", url=info["verification_uri_complete"],
|
||||
code=info["user_code"], expires_in=info["expires_in"])
|
||||
_save_oauth_token(poll_device_token(info))
|
||||
except DeviceCodeError as e:
|
||||
raise HubError("login", f"The login didn't go through: {e}. Press Log in to try again.") from None
|
||||
except Exception as e:
|
||||
raise explain(e, dataset_id()) from None
|
||||
return whoami()
|
||||
|
||||
|
||||
def upload(store, session, dry_run=False, again=False, progress=None, log=None):
|
||||
"""Upload exports/<session> (the steps in this file's docstring). progress(phase, text,
|
||||
fraction or None); log(text) for the dry run's account. Returns the result; raises HubError."""
|
||||
@@ -305,9 +336,8 @@ def upload(store, session, dry_run=False, again=False, progress=None, log=None):
|
||||
raise explain(e, repo) from None
|
||||
role = ((who.get("auth") or {}).get("accessToken") or {}).get("role", "")
|
||||
if role == "read":
|
||||
raise HubError("read-token", "Your saved token can only read, so it can't open a pull request. Create one "
|
||||
"with the Write role and log in again with it: hf auth login --force.",
|
||||
TOKENS_URL)
|
||||
raise HubError("read-token", "Your saved login is a read-only token, so it can't open a pull request. "
|
||||
"Log in again.")
|
||||
tell("access", f"Checking access to {repo}", None)
|
||||
try:
|
||||
api.auth_check(repo, repo_type="dataset")
|
||||
@@ -360,6 +390,8 @@ def main():
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
w = sub.add_parser("whoami", help="show the saved Hugging Face login")
|
||||
w.add_argument("--json", action="store_true")
|
||||
li = sub.add_parser("login", help="log in to Hugging Face in the browser")
|
||||
li.add_argument("--json", action="store_true")
|
||||
u = sub.add_parser("upload", help="upload exports/SESSION as a pull request")
|
||||
u.add_argument("session")
|
||||
u.add_argument("--dry-run", action="store_true", help="no network: say what would be uploaded")
|
||||
@@ -378,6 +410,18 @@ def main():
|
||||
else:
|
||||
print(f"logged in as {who['name']} (token role: {who['role'] or 'unknown'})")
|
||||
return 0
|
||||
if a.cmd == "login":
|
||||
def code(phase, text, **extra):
|
||||
if a.json:
|
||||
emit(dict(extra, phase=phase, text=text))
|
||||
else:
|
||||
print(f"Open {extra['url']} and approve the code {extra['code']}. Waiting...", flush=True)
|
||||
who = login(code)
|
||||
if a.json:
|
||||
emit({"done": who})
|
||||
else:
|
||||
print(f"logged in as {who['name']}")
|
||||
return 0
|
||||
if a.json:
|
||||
def progress(phase, text, fraction=None, **extra):
|
||||
emit(dict(extra, phase=phase, text=text, fraction=fraction))
|
||||
|
||||
+252
-229
@@ -1494,7 +1494,7 @@ Kirigami.ApplicationWindow {
|
||||
if (backend.uploading) return ""
|
||||
if (backend.hubDryRun) return ""
|
||||
if (!backend.uploadAllowed) return "Contributions aren't open yet"
|
||||
if (!loggedIn) return "Log in to Hugging Face first (below)"
|
||||
if (!loggedIn) return "Log in first (step 2)"
|
||||
return ""
|
||||
}
|
||||
|
||||
@@ -1555,263 +1555,286 @@ Kirigami.ApplicationWindow {
|
||||
}
|
||||
}
|
||||
|
||||
Kirigami.FormLayout {
|
||||
ColumnLayout {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.exported.length > 0
|
||||
Controls.ComboBox {
|
||||
id: exportBox
|
||||
Kirigami.FormData.label: "Export:"
|
||||
model: uploadView.exported
|
||||
textRole: "label"
|
||||
valueRole: "id"
|
||||
enabled: !backend.uploading
|
||||
Component.onCompleted: currentIndex = Math.max(0, indexOfValue(root.chosenSession))
|
||||
onActivated: root.chosenSession = currentValue
|
||||
}
|
||||
spacing: Kirigami.Units.largeSpacing
|
||||
|
||||
Controls.Label {
|
||||
Kirigami.FormData.label: "Size:"
|
||||
text: uploadView.chosen ? uploadView.chosen.exportText : ""
|
||||
}
|
||||
Controls.Label {
|
||||
Kirigami.FormData.label: "Goes to:"
|
||||
Layout.fillWidth: true
|
||||
wrapMode: Text.Wrap
|
||||
textFormat: Text.StyledText
|
||||
text: "<a href=\"" + backend.datasetUrl + "\">" + backend.dataset + "</a>, as a pull request"
|
||||
text: "Your export goes to <a href=\"" + backend.datasetUrl + "\">" + backend.dataset + "</a> as a "
|
||||
+ "pull request from your own Hugging Face account. Nothing is published until the "
|
||||
+ "maintainer has checked it."
|
||||
onLinkActivated: link => Qt.openUrlExternally(link)
|
||||
}
|
||||
|
||||
// 1. The export
|
||||
Kirigami.Heading { level: 3; text: "1. Choose the export" }
|
||||
RowLayout {
|
||||
Kirigami.FormData.label: "Hugging Face:"
|
||||
Controls.Label {
|
||||
Layout.fillWidth: true
|
||||
Controls.ComboBox {
|
||||
id: exportBox
|
||||
Layout.fillWidth: true
|
||||
Layout.maximumWidth: Kirigami.Units.gridUnit * 22
|
||||
model: uploadView.exported
|
||||
textRole: "label"
|
||||
valueRole: "id"
|
||||
enabled: !backend.uploading
|
||||
Component.onCompleted: currentIndex = Math.max(0, indexOfValue(root.chosenSession))
|
||||
onActivated: root.chosenSession = currentValue
|
||||
}
|
||||
Controls.Label {
|
||||
opacity: 0.7
|
||||
text: uploadView.chosen ? uploadView.chosen.exportText : ""
|
||||
}
|
||||
}
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.chosen !== null && uploadView.chosen.export_stale
|
||||
type: Kirigami.MessageType.Warning
|
||||
text: "This session changed since it was exported. Export it again so the upload has your latest deletions."
|
||||
}
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== "" && uploadView.info.previous.export_sha !== undefined
|
||||
type: Kirigami.MessageType.Information
|
||||
text: "This export was uploaded on " + (uploadView.info.previous.uploaded || "") + ": "
|
||||
+ (uploadView.info.previous.pr_url || "") + ". There's no need to upload it again."
|
||||
}
|
||||
|
||||
// 2. The login: Log in opens Hugging Face in the browser, where the person approves
|
||||
// it. The token goes from there to hub.py, never into this window.
|
||||
Kirigami.Heading { level: 3; text: "2. Log in to Hugging Face" }
|
||||
RowLayout {
|
||||
Layout.fillWidth: true
|
||||
visible: loginState !== "waiting"
|
||||
readonly property string loginState: backend.login.state
|
||||
Kirigami.Icon {
|
||||
visible: parent.loginState === "ok"
|
||||
source: "checkmark"
|
||||
implicitWidth: Kirigami.Units.iconSizes.small
|
||||
implicitHeight: implicitWidth
|
||||
}
|
||||
Controls.BusyIndicator {
|
||||
visible: ["checking", "starting", "unknown"].indexOf(parent.loginState) >= 0
|
||||
running: visible
|
||||
implicitWidth: Kirigami.Units.gridUnit * 1.5
|
||||
implicitHeight: implicitWidth
|
||||
}
|
||||
Controls.Label {
|
||||
Layout.fillWidth: true
|
||||
visible: parent.loginState !== "waiting"
|
||||
wrapMode: Text.Wrap
|
||||
text: backend.login.state === "dry" ? "not checked in a dry run"
|
||||
: backend.login.state === "unknown" ? "not checked yet"
|
||||
: backend.login.state === "none" ? "not logged in"
|
||||
text: parent.loginState === "dry" ? "Not needed for a dry run."
|
||||
: parent.loginState === "unknown" ? "Checking your login"
|
||||
: parent.loginState === "none"
|
||||
? "Press Log in. Hugging Face opens in your browser: sign in, or make a free "
|
||||
+ "account, then enter the code this page shows."
|
||||
: backend.login.text || ""
|
||||
}
|
||||
Controls.Button {
|
||||
visible: backend.login.state !== "dry"
|
||||
visible: ["none", "read", "error", "ok"].indexOf(parent.loginState) >= 0
|
||||
text: parent.loginState === "ok" ? "Use another account" : "Log in"
|
||||
icon.name: parent.loginState === "ok" ? "system-switch-user" : "im-user-online"
|
||||
flat: parent.loginState === "ok"
|
||||
enabled: !backend.uploading
|
||||
onClicked: backend.logIn()
|
||||
}
|
||||
Controls.Button {
|
||||
visible: parent.loginState === "error"
|
||||
text: "Check again"
|
||||
icon.name: "view-refresh"
|
||||
enabled: backend.login.state !== "checking"
|
||||
onClicked: backend.checkLogin()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.chosen !== null && uploadView.chosen.export_stale
|
||||
type: Kirigami.MessageType.Warning
|
||||
text: "This session changed since it was exported. Export it again so the upload has your latest deletions."
|
||||
}
|
||||
|
||||
// Not logged in: how to log in, in a terminal. The token never goes into this window.
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== "" && ["none", "read", "error", "missing"].indexOf(backend.login.state) >= 0
|
||||
type: backend.login.state === "error" ? Kirigami.MessageType.Error : Kirigami.MessageType.Warning
|
||||
text: backend.login.state === "none"
|
||||
? "You're not logged in to Hugging Face. Open a terminal (Konsole) and run the command below. "
|
||||
+ "Paste a token with the Write role when it asks (steps 1 to 3 below say how to make one). "
|
||||
+ "The token goes only into that terminal, never into this window. Then press Check again."
|
||||
: backend.login.text || ""
|
||||
actions: backend.login.link ? [linkAction] : []
|
||||
Kirigami.Action {
|
||||
id: linkAction
|
||||
text: "Open"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(backend.login.link)
|
||||
}
|
||||
}
|
||||
RowLayout {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== "" && ["none", "read"].indexOf(backend.login.state) >= 0
|
||||
Controls.TextField {
|
||||
id: loginCommand
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
readOnly: true
|
||||
font.family: "monospace"
|
||||
text: backend.loginCommand + (backend.login.state === "read" ? " --force" : "")
|
||||
}
|
||||
Controls.Button {
|
||||
text: "Copy command"
|
||||
icon.name: "edit-copy"
|
||||
onClicked: backend.copy(loginCommand.text)
|
||||
}
|
||||
}
|
||||
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== "" && uploadView.info.previous.export_sha !== undefined
|
||||
type: Kirigami.MessageType.Information
|
||||
text: "This export was uploaded on " + (uploadView.info.previous.uploaded || "") + ": "
|
||||
+ (uploadView.info.previous.pr_url || "") + ". There's no need to upload it again."
|
||||
}
|
||||
|
||||
RowLayout {
|
||||
visible: uploadView.session !== ""
|
||||
Controls.Button {
|
||||
text: backend.uploading ? "Uploading…" : backend.hubDryRun ? "Upload (dry run)" : "Upload"
|
||||
icon.name: "cloud-upload"
|
||||
enabled: uploadView.blocked === "" && !backend.uploading
|
||||
onClicked: uploadView.startUpload()
|
||||
}
|
||||
Controls.BusyIndicator {
|
||||
visible: backend.uploading
|
||||
running: visible
|
||||
implicitWidth: Kirigami.Units.gridUnit * 1.5
|
||||
implicitHeight: implicitWidth
|
||||
}
|
||||
Controls.Button {
|
||||
visible: backend.uploading
|
||||
text: "Cancel"
|
||||
icon.name: "dialog-cancel"
|
||||
onClicked: backend.cancelUpload()
|
||||
}
|
||||
Controls.Label {
|
||||
visible: !backend.uploading && uploadView.blocked !== ""
|
||||
opacity: 0.7
|
||||
text: uploadView.blocked
|
||||
}
|
||||
}
|
||||
|
||||
// Progress: a share while the export is checked, a sweeping bar while it's sent
|
||||
// (huggingface_hub doesn't report progress). Drawn here, as on the Export page.
|
||||
Rectangle {
|
||||
id: track
|
||||
Layout.preferredWidth: Kirigami.Units.gridUnit * 20
|
||||
implicitHeight: Kirigami.Units.smallSpacing * 2
|
||||
visible: backend.uploading
|
||||
radius: height / 2
|
||||
clip: true
|
||||
color: Qt.rgba(Kirigami.Theme.textColor.r, Kirigami.Theme.textColor.g, Kirigami.Theme.textColor.b, 0.15)
|
||||
readonly property bool unknown: uploadView.up.fraction === undefined || uploadView.up.fraction < 0
|
||||
Rectangle {
|
||||
id: fill
|
||||
height: parent.height
|
||||
radius: parent.radius
|
||||
color: Kirigami.Theme.highlightColor
|
||||
width: track.unknown ? parent.width / 4 : parent.width * Math.max(0, Math.min(1, uploadView.up.fraction))
|
||||
x: 0
|
||||
SequentialAnimation on x {
|
||||
running: track.visible && track.unknown
|
||||
loops: Animation.Infinite
|
||||
onRunningChanged: if (!running) fill.x = 0
|
||||
NumberAnimation { from: -fill.width; to: track.width; duration: 1600 }
|
||||
visible: backend.login.state === "waiting"
|
||||
type: Kirigami.MessageType.Information
|
||||
text: "Hugging Face is open in your browser (" + (backend.login.url || "") + "). Sign in, or "
|
||||
+ "make a free account, enter the code below and approve the login. If it asks about "
|
||||
+ "organizations, leave them unticked. The code works for "
|
||||
+ Math.round((backend.login.expires_in || 300) / 60) + " minutes; this page carries on "
|
||||
+ "by itself once you've approved."
|
||||
actions: [openLogin, copyLogin, cancelLogin]
|
||||
Kirigami.Action {
|
||||
id: openLogin
|
||||
text: "Open again"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(backend.login.url)
|
||||
}
|
||||
Kirigami.Action {
|
||||
id: copyLogin
|
||||
text: "Copy code"
|
||||
icon.name: "edit-copy"
|
||||
onTriggered: backend.copy(backend.login.code)
|
||||
}
|
||||
Kirigami.Action {
|
||||
id: cancelLogin
|
||||
text: "Cancel"
|
||||
icon.name: "dialog-cancel"
|
||||
onTriggered: backend.cancelLogin()
|
||||
}
|
||||
}
|
||||
}
|
||||
Controls.Label {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.mine && (uploadView.up.text || "") !== ""
|
||||
&& (backend.uploading || uploadView.up.phase === "failed" && uploadView.upError.kind === "cancelled")
|
||||
wrapMode: Text.Wrap
|
||||
text: uploadView.up.text || ""
|
||||
}
|
||||
Controls.Label {
|
||||
visible: backend.login.state === "waiting"
|
||||
Layout.leftMargin: Kirigami.Units.gridUnit
|
||||
text: backend.login.code || ""
|
||||
font.family: "monospace"
|
||||
font.pointSize: Kirigami.Theme.defaultFont.pointSize * 2.5
|
||||
font.letterSpacing: 4
|
||||
}
|
||||
|
||||
// The pull request is open and the files are on their way: time to plug in.
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: backend.uploading && uploadView.mine && (uploadView.up.pr_url || "") !== ""
|
||||
type: Kirigami.MessageType.Positive
|
||||
text: "Your pull request is open: " + (uploadView.up.pr_url || "") + ". The files are uploading "
|
||||
+ "to it now, which can take a while. Plug in the headset and leave it plugged in until "
|
||||
+ "this page says Uploaded. You can take the headset off: it stays awake until the upload "
|
||||
+ "is done. Keep the Hand Recorder open."
|
||||
actions: [openOpenedPr, copyOpenedPr]
|
||||
Kirigami.Action {
|
||||
id: openOpenedPr
|
||||
text: "Open"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(uploadView.up.pr_url)
|
||||
}
|
||||
Kirigami.Action {
|
||||
id: copyOpenedPr
|
||||
text: "Copy link"
|
||||
icon.name: "edit-copy"
|
||||
onTriggered: backend.copy(uploadView.up.pr_url)
|
||||
}
|
||||
}
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: !backend.uploading && uploadView.mine && uploadView.up.phase === "done"
|
||||
type: Kirigami.MessageType.Positive
|
||||
text: uploadView.upResult.dry_run
|
||||
? "Dry run: the export passed its checks. It would go to " + uploadView.upResult.repo + "/"
|
||||
+ uploadView.upResult.path_in_repo + " (" + uploadView.upResult.files + " files). Nothing was sent."
|
||||
: "Uploaded. Your pull request: " + (uploadView.upResult.pr_url || "")
|
||||
+ ". The maintainer reviews it before it joins the dataset."
|
||||
actions: uploadView.upResult.pr_url ? [openPr, copyPr] : []
|
||||
Kirigami.Action {
|
||||
id: openPr
|
||||
text: "Open"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(uploadView.upResult.pr_url)
|
||||
}
|
||||
Kirigami.Action {
|
||||
id: copyPr
|
||||
text: "Copy link"
|
||||
icon.name: "edit-copy"
|
||||
onTriggered: backend.copy(uploadView.upResult.pr_url)
|
||||
}
|
||||
}
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: !backend.uploading && uploadView.mine && uploadView.up.phase === "failed"
|
||||
&& uploadView.upError.kind !== "cancelled"
|
||||
type: Kirigami.MessageType.Error
|
||||
text: (uploadView.upError.text || "")
|
||||
+ ((uploadView.upError.errors || []).length ? "\n\n• " + uploadView.upError.errors.join("\n• ") : "")
|
||||
actions: uploadView.upError.link ? [openErrorLink] : []
|
||||
Kirigami.Action {
|
||||
id: openErrorLink
|
||||
text: "Open"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(uploadView.upError.link)
|
||||
}
|
||||
}
|
||||
Controls.TextArea {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.mine && (uploadView.up.log || "") !== ""
|
||||
readOnly: true
|
||||
selectByMouse: true
|
||||
wrapMode: Text.WrapAnywhere
|
||||
font.family: "monospace"
|
||||
text: uploadView.up.log || ""
|
||||
}
|
||||
|
||||
Kirigami.Separator {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== ""
|
||||
}
|
||||
Kirigami.Heading {
|
||||
level: 3
|
||||
visible: uploadView.session !== ""
|
||||
text: "Or upload from a terminal"
|
||||
}
|
||||
RowLayout {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== ""
|
||||
Controls.TextArea {
|
||||
id: command
|
||||
// 3. The upload
|
||||
Kirigami.Heading { level: 3; text: "3. Upload" }
|
||||
Controls.Label {
|
||||
Layout.fillWidth: true
|
||||
wrapMode: Text.Wrap
|
||||
textFormat: Text.StyledText
|
||||
text: "The first time, accept the dataset's terms on <a href=\"" + backend.datasetUrl
|
||||
+ "\">its page</a>. Upload checks the export, opens your pull request, then sends the files."
|
||||
onLinkActivated: link => Qt.openUrlExternally(link)
|
||||
}
|
||||
RowLayout {
|
||||
Controls.Button {
|
||||
text: backend.uploading ? "Uploading…" : backend.hubDryRun ? "Upload (dry run)" : "Upload"
|
||||
icon.name: "cloud-upload"
|
||||
enabled: uploadView.blocked === "" && !backend.uploading
|
||||
onClicked: uploadView.startUpload()
|
||||
}
|
||||
Controls.BusyIndicator {
|
||||
visible: backend.uploading
|
||||
running: visible
|
||||
implicitWidth: Kirigami.Units.gridUnit * 1.5
|
||||
implicitHeight: implicitWidth
|
||||
}
|
||||
Controls.Button {
|
||||
visible: backend.uploading
|
||||
text: "Cancel"
|
||||
icon.name: "dialog-cancel"
|
||||
onClicked: backend.cancelUpload()
|
||||
}
|
||||
Controls.Label {
|
||||
visible: !backend.uploading && uploadView.blocked !== ""
|
||||
opacity: 0.7
|
||||
text: uploadView.blocked
|
||||
}
|
||||
}
|
||||
|
||||
// Progress: a share while the export is checked, a sweeping bar while it's sent
|
||||
// (huggingface_hub doesn't report progress). Drawn here, as on the Export page.
|
||||
Rectangle {
|
||||
id: track
|
||||
Layout.preferredWidth: Kirigami.Units.gridUnit * 20
|
||||
implicitHeight: Kirigami.Units.smallSpacing * 2
|
||||
visible: backend.uploading
|
||||
radius: height / 2
|
||||
clip: true
|
||||
color: Qt.rgba(Kirigami.Theme.textColor.r, Kirigami.Theme.textColor.g, Kirigami.Theme.textColor.b, 0.15)
|
||||
readonly property bool unknown: uploadView.up.fraction === undefined || uploadView.up.fraction < 0
|
||||
Rectangle {
|
||||
id: fill
|
||||
height: parent.height
|
||||
radius: parent.radius
|
||||
color: Kirigami.Theme.highlightColor
|
||||
width: track.unknown ? parent.width / 4 : parent.width * Math.max(0, Math.min(1, uploadView.up.fraction))
|
||||
x: 0
|
||||
SequentialAnimation on x {
|
||||
running: track.visible && track.unknown
|
||||
loops: Animation.Infinite
|
||||
onRunningChanged: if (!running) fill.x = 0
|
||||
NumberAnimation { from: -fill.width; to: track.width; duration: 1600 }
|
||||
}
|
||||
}
|
||||
}
|
||||
Controls.Label {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.mine && (uploadView.up.text || "") !== ""
|
||||
&& (backend.uploading || uploadView.up.phase === "failed" && uploadView.upError.kind === "cancelled")
|
||||
wrapMode: Text.Wrap
|
||||
text: uploadView.up.text || ""
|
||||
}
|
||||
|
||||
// The pull request is open and the files are on their way: time to plug in.
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: backend.uploading && uploadView.mine && (uploadView.up.pr_url || "") !== ""
|
||||
type: Kirigami.MessageType.Positive
|
||||
text: "Your pull request is open: " + (uploadView.up.pr_url || "") + ". The files are uploading "
|
||||
+ "to it now, which can take a while. Plug in the headset and leave it plugged in until "
|
||||
+ "this page says Uploaded. You can take the headset off: it stays awake until the upload "
|
||||
+ "is done. Keep the Hand Recorder open."
|
||||
actions: [openOpenedPr, copyOpenedPr]
|
||||
Kirigami.Action {
|
||||
id: openOpenedPr
|
||||
text: "Open"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(uploadView.up.pr_url)
|
||||
}
|
||||
Kirigami.Action {
|
||||
id: copyOpenedPr
|
||||
text: "Copy link"
|
||||
icon.name: "edit-copy"
|
||||
onTriggered: backend.copy(uploadView.up.pr_url)
|
||||
}
|
||||
}
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: !backend.uploading && uploadView.mine && uploadView.up.phase === "done"
|
||||
type: Kirigami.MessageType.Positive
|
||||
text: uploadView.upResult.dry_run
|
||||
? "Dry run: the export passed its checks. It would go to " + uploadView.upResult.repo + "/"
|
||||
+ uploadView.upResult.path_in_repo + " (" + uploadView.upResult.files + " files). Nothing was sent."
|
||||
: "Uploaded. Your pull request: " + (uploadView.upResult.pr_url || "")
|
||||
+ ". The maintainer reviews it before it joins the dataset."
|
||||
actions: uploadView.upResult.pr_url ? [openPr, copyPr] : []
|
||||
Kirigami.Action {
|
||||
id: openPr
|
||||
text: "Open"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(uploadView.upResult.pr_url)
|
||||
}
|
||||
Kirigami.Action {
|
||||
id: copyPr
|
||||
text: "Copy link"
|
||||
icon.name: "edit-copy"
|
||||
onTriggered: backend.copy(uploadView.upResult.pr_url)
|
||||
}
|
||||
}
|
||||
Kirigami.InlineMessage {
|
||||
Layout.fillWidth: true
|
||||
visible: !backend.uploading && uploadView.mine && uploadView.up.phase === "failed"
|
||||
&& uploadView.upError.kind !== "cancelled"
|
||||
type: Kirigami.MessageType.Error
|
||||
text: (uploadView.upError.text || "")
|
||||
+ ((uploadView.upError.errors || []).length ? "\n\n• " + uploadView.upError.errors.join("\n• ") : "")
|
||||
actions: uploadView.upError.link ? [openErrorLink] : []
|
||||
Kirigami.Action {
|
||||
id: openErrorLink
|
||||
text: "Open"
|
||||
icon.name: "internet-services"
|
||||
onTriggered: Qt.openUrlExternally(uploadView.upError.link)
|
||||
}
|
||||
}
|
||||
Controls.TextArea {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.mine && (uploadView.up.log || "") !== ""
|
||||
readOnly: true
|
||||
selectByMouse: true
|
||||
wrapMode: Text.WrapAnywhere
|
||||
font.family: "monospace"
|
||||
text: backend.uploadCommand(uploadView.session)
|
||||
text: uploadView.up.log || ""
|
||||
}
|
||||
Controls.Button {
|
||||
text: "Copy command"
|
||||
icon.name: "edit-copy"
|
||||
onClicked: backend.copy(command.text)
|
||||
|
||||
Kirigami.Separator { Layout.fillWidth: true; visible: uploadView.session !== "" }
|
||||
MarkdownText {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== ""
|
||||
markdown: backend.uploadText(uploadView.session)
|
||||
}
|
||||
}
|
||||
MarkdownText {
|
||||
Layout.fillWidth: true
|
||||
visible: uploadView.session !== ""
|
||||
markdown: backend.uploadText(uploadView.session)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -315,6 +315,43 @@ class SessionFilesTest(unittest.TestCase):
|
||||
self.assertEqual(validate.validate(path).errors, [])
|
||||
|
||||
|
||||
class LoginTest(unittest.TestCase):
|
||||
"""hub.login: the link and code go out first, the token is saved by huggingface_hub and never
|
||||
passed on, and a refused login is a "login" error. huggingface_hub's helpers are faked."""
|
||||
|
||||
def setUp(self):
|
||||
try:
|
||||
import huggingface_hub._login
|
||||
import huggingface_hub.utils._oauth_device
|
||||
except ImportError as e:
|
||||
self.skipTest(f"no huggingface_hub browser login: {e}")
|
||||
self.login_mod = huggingface_hub._login
|
||||
self.device = huggingface_hub.utils._oauth_device
|
||||
|
||||
def test_code_then_saved(self):
|
||||
told, saved = [], []
|
||||
info = {"verification_uri_complete": "https://hf.co/oauth/device", "user_code": "ABCD-1234", "expires_in": 300}
|
||||
with mock.patch.object(self.device, "request_device_code", return_value=info), \
|
||||
mock.patch.object(self.device, "poll_device_token", return_value={"access_token": "secret"}), \
|
||||
mock.patch.object(self.login_mod, "_save_oauth_token", side_effect=saved.append), \
|
||||
mock.patch.object(hub, "whoami", return_value={"name": "someone", "role": ""}):
|
||||
who = hub.login(lambda phase, text, **extra: told.append(dict(extra, phase=phase)))
|
||||
self.assertEqual(told, [{"phase": "code", "url": info["verification_uri_complete"], "code": "ABCD-1234",
|
||||
"expires_in": 300}])
|
||||
self.assertEqual(saved, [{"access_token": "secret"}])
|
||||
self.assertEqual(who["name"], "someone")
|
||||
self.assertNotIn("secret", json.dumps(told) + json.dumps(who))
|
||||
|
||||
def test_refused(self):
|
||||
from huggingface_hub.errors import DeviceCodeError
|
||||
info = {"verification_uri_complete": "u", "user_code": "c", "expires_in": 300}
|
||||
with mock.patch.object(self.device, "request_device_code", return_value=info), \
|
||||
mock.patch.object(self.device, "poll_device_token", side_effect=DeviceCodeError("access_denied")):
|
||||
with self.assertRaises(hub.HubError) as cm:
|
||||
hub.login()
|
||||
self.assertEqual(cm.exception.kind, "login")
|
||||
|
||||
|
||||
class HubTest(unittest.TestCase):
|
||||
"""hub.py without the network: the dry run, the draft gate, upload records."""
|
||||
|
||||
|
||||
Reference in new issue
Block a user