chore: nightly index update [skip ci]

.env.example

@@ -13,11 +13,6 @@ PINKCUFFS_USERNAME=your-email-or-username
 PINKCUFFS_PASSWORD=your-password
 PINKCUFFS_LOGIN_COOKIE=wordpress_logged_in_<hash>=<value>
 
-# femuniverse.com credentials (separate membership)
-FEMUNIVERSE_USERNAME=your-email-or-username
-FEMUNIVERSE_PASSWORD=your-password
-FEMUNIVERSE_LOGIN_COOKIE=wordpress_logged_in_<hash>=<value>
-
 # PeerTube upload target
 PEERTUBE_URL=https://your-peertube-instance.example
 PEERTUBE_USER=admin

.github/instructions/snyk_rules.instructions.md

@@ -0,0 +1,14 @@
+---
+alwaysApply: true
+always_on: true
+trigger: always_on
+applyTo: "**"
+description: Snyk Security At Inception
+---
+
+# Project security best practices
+
+- Always run snyk_code_scan tool for new first party code that is generated in a Snyk-supported language.
+- If any security issues are found based on newly introduced or modified code or dependencies, attempt to fix the issues using the results context from Snyk.
+- Rescan the code after fixing the issues to ensure that the issues were fixed and that there are no newly introduced issues.
+- Repeat this process until no new issues are found.

.github/workflows/nightly-index.yml

@@ -38,8 +38,6 @@ jobs:
           JAILBIRDZ_PASSWORD: ${{ secrets.JAILBIRDZ_PASSWORD }}
           PINKCUFFS_USERNAME: ${{ secrets.PINKCUFFS_USERNAME }}
           PINKCUFFS_PASSWORD: ${{ secrets.PINKCUFFS_PASSWORD }}
-          FEMUNIVERSE_USERNAME: ${{ secrets.FEMUNIVERSE_USERNAME }}
-          FEMUNIVERSE_PASSWORD: ${{ secrets.FEMUNIVERSE_PASSWORD }}
 
       - name: Commit updated video_map.json
         if: always()  # save progress even if main.py crashed or timed out

.gitignore

@@ -1,6 +1,3 @@
-# Local Claude instructions
-.claude/settings.local.json
-
 # Temporary cache
 __pycache__/
 .ruff_cache/

README.md

@@ -1,6 +1,6 @@
 # 𝒥𝒶𝒾𝓁𝒷𝒾𝓇𝒹𝓏-𝒹𝓁
 
-Jailbirdz.com, Pinkcuffs.com, and Femuniverse.com are Arizona-based subscription video sites publishing arrest and jail roleplay scenarios featuring women. This tool scrapes the member area of any combination of these sites, downloads the videos, and re-hosts them on a self-owned PeerTube instance.
+Jailbirdz.com and Pinkcuffs.com are Arizona-based subscription video sites publishing arrest and jail roleplay scenarios featuring women. This tool scrapes the member area of one or both sites, downloads the videos, and re-hosts them on a self-owned PeerTube instance.
 
 > [!NOTE]  
 > This tool does not bypass authentication, modify the site, or intercept anything it isn't entitled to. A valid, paid membership is required. The scraper authenticates using your own session cookie and accesses only content your account can already view in a browser.
@@ -23,9 +23,9 @@ cp .env.example .env
 
 Set credentials for whichever sites you have a membership on. You don't need both.
 
-**Option A — credentials (recommended):** set `JAILBIRDZ_USERNAME` + `JAILBIRDZ_PASSWORD` (and/or the `PINKCUFFS_*` / `FEMUNIVERSE_*` equivalents) in `.env`. `main.py` logs in automatically on startup.
+**Option A — credentials (recommended):** set `JAILBIRDZ_USERNAME` + `JAILBIRDZ_PASSWORD` (and/or the `PINKCUFFS_*` equivalents) in `.env`. `main.py` logs in automatically on startup.
 
-**Option B — manual cookie:** set `JAILBIRDZ_LOGIN_COOKIE` (and/or `PINKCUFFS_LOGIN_COOKIE` / `FEMUNIVERSE_LOGIN_COOKIE`) yourself. Get the value from browser DevTools → Storage → Cookies — copy the full `name=value` of the `wordpress_logged_in_*` cookie.
+**Option B — manual cookie:** set `JAILBIRDZ_LOGIN_COOKIE` (and/or `PINKCUFFS_LOGIN_COOKIE`) yourself. Get the value from browser DevTools → Storage → Cookies — copy the full `name=value` of the `wordpress_logged_in_*` cookie.
 
 Sites with no credentials are skipped automatically when running `python main.py`.
 
@@ -35,8 +35,6 @@ Sites with no credentials are skipped automatically when running `python main.py
 - `JAILBIRDZ_LOGIN_COOKIE` — jailbirdz.com session cookie (fallback).
 - `PINKCUFFS_USERNAME` / `PINKCUFFS_PASSWORD` — pinkcuffs.com login.
 - `PINKCUFFS_LOGIN_COOKIE` — pinkcuffs.com session cookie (fallback).
-- `FEMUNIVERSE_USERNAME` / `FEMUNIVERSE_PASSWORD` — femuniverse.com login.
-- `FEMUNIVERSE_LOGIN_COOKIE` — femuniverse.com session cookie (fallback).
 - `PEERTUBE_URL` — base URL of your PeerTube instance.
 - `PEERTUBE_USER` — PeerTube username.
 - `PEERTUBE_CHANNEL` — channel to upload to.
@@ -46,13 +44,12 @@ Sites with no credentials are skipped automatically when running `python main.py
 
 ### 1. Scrape
 
-Discovers all post URLs via the WordPress REST API, then visits each page with a headless Firefox browser to intercept video network requests (MP4, MOV, WebM, AVI, M4V, HLS/M3U8).
+Discovers all post URLs via the WordPress REST API, then visits each page with a headless Firefox browser to intercept video network requests (MP4, MOV, WebM, AVI, M4V).
 
 ```bash
 python main.py                    # scrape all sites you have credentials for
 python main.py --site jailbirdz   # scrape one site only
 python main.py --site pinkcuffs --site jailbirdz  # explicit multi-site
-python main.py --site femuniverse                 # femuniverse only
 ```
 
 Results are written to `video_map.json`. Safe to re-run — already-scraped posts are skipped.
@@ -65,11 +62,11 @@ python download.py [options]
 Options:
   -o, --output DIR      Download directory (default: downloads)
   -t, --titles          Name files by post title
-      --original        Name files by original filename derived from the video URL (default)
+      --original        Name files by original CloudFront filename (default)
       --reorganize      Rename existing files to match current naming mode
   -w, --workers N       Concurrent downloads (default: 4)
   -n, --dry-run         Print what would be downloaded
-      --site SITE       Limit to one site (jailbirdz, pinkcuffs, or femuniverse); repeatable
+      --site SITE       Limit to one site (jailbirdz or pinkcuffs); repeatable
 ```
 
 Resumes partial downloads. The chosen naming mode is saved to `.naming_mode` inside the output directory and persists across runs. Filenames that would clash are placed into subfolders.
@@ -89,13 +86,10 @@ Options:
       --poll-interval SECS  State poll interval in seconds (default: 30)
       --skip-wait           Upload without waiting for transcoding
       --nsfw                Mark videos as NSFW
-  -n, --dry-run             Print what would be uploaded (checks the channel, uploads nothing)
-      --scan-all            Scan every subdirectory of --input, not just the site folders
+  -n, --dry-run             Print what would be uploaded
 ```
 
-Uploads in resumable 10 MB chunks. After each batch, waits for transcoding and object storage to complete before uploading the next batch — this prevents disk exhaustion on the PeerTube server. Videos already present on the channel (matched by name) are skipped. If `--input` contains any of the configured site folders (`jailbirdz`, `pinkcuffs`, `femuniverse`), only those are scanned — so the input directory can be a shared drive holding unrelated purchases alongside them. Anything skipped is listed on startup; `--scan-all` includes it. A directory with no site folders is walked in full.
-
-Progress is tracked in `.uploaded` inside the input directory, so a new `--input` directory starts with an empty ledger — the channel itself is reconciled against on every run (dry runs included) to fill it back in.
+Uploads in resumable 10 MB chunks. After each batch, waits for transcoding and object storage to complete before uploading the next batch — this prevents disk exhaustion on the PeerTube server. Videos already present on the channel (matched by name) are skipped. Progress is tracked in `.uploaded` inside the input directory.
 
 ## CI / Nightly Indexing
 
@@ -111,10 +105,6 @@ gh secret set JAILBIRDZ_PASSWORD
 # pinkcuffs (if you have a membership)
 gh secret set PINKCUFFS_USERNAME
 gh secret set PINKCUFFS_PASSWORD
-
-# femuniverse (if you have a membership)
-gh secret set FEMUNIVERSE_USERNAME
-gh secret set FEMUNIVERSE_PASSWORD
 ```
 
 **Seed CI with your current progress before the first run:**

check_clashes.py

@@ -121,21 +121,17 @@ def save_video_map(
 
 
 def build_url_referers(video_map: dict[str, Any]) -> dict[str, str]:
-    """Pure function: return {cdn_video_url: referer} from a flat video map.
+    """Pure function: return {cdn_video_url: site_referer} from a flat video map.
 
-    Bunny.net CDN URLs require https://player.mediadelivery.net/ as referer.
-    All other URLs use the scheme+netloc of the page they were found on.
+    The flat video map has page URLs as keys; the scheme+netloc of each page URL
+    is used as the Referer for all CDN video URLs found in that entry.
     """
     result: dict[str, str] = {}
     for page_url, entry in video_map.items():
         parsed = urlparse(page_url)
-        site_referer = f"{parsed.scheme}://{parsed.netloc}/"
+        referer = f"{parsed.scheme}://{parsed.netloc}/"
         for vid in cast(dict[str, Any], entry).get("videos", []):
-            vid_url = vid["url"]
-            if urlparse(vid_url).netloc.endswith(".b-cdn.net"):
-                result.setdefault(vid_url, "https://player.mediadelivery.net/")
-            else:
-                result.setdefault(vid_url, site_referer)
+            result.setdefault(vid["url"], referer)
     return result
 
 
@@ -151,17 +147,8 @@ def fmt_size(b: float | int) -> str:
     return f"{b:.1f} TB"
 
 
-def is_hls_url(url: str) -> bool:
-    """True if url is an HLS master playlist (.m3u8)."""
-    return urlparse(url).path.endswith(".m3u8")
-
-
 def url_to_filename(url: str) -> str:
-    path = PurePosixPath(urlparse(url).path)
-    # Bunny.net HLS: .../guid/playlist.m3u8 → guid.mp4
-    if path.name == "playlist.m3u8":
-        return unquote(path.parent.name) + ".mp4"
-    return unquote(path.name)
+    return unquote(PurePosixPath(urlparse(url).path).name)
 
 
 def find_clashes(urls: list[str]) -> dict[str, list[str]]:
@@ -178,72 +165,27 @@ def find_clashes(urls: list[str]) -> dict[str, list[str]]:
     }
 
 
-def _path_folders(url: str) -> list[str]:
-    """Decoded URL path segments above the filename (filename excluded)."""
-    parts = [unquote(p) for p in urlparse(url).path.split("/") if p]
-    return parts[:-1]
-
-
-def _disambiguate_group(group: list[str]) -> dict[str, tuple[str, ...]]:
-    """Find the smallest depth of trailing folder segments that gives every URL in the group
-    a unique subfolder path. Returns {url: subfolder_segments}.
-
-    Comparison is case-insensitive so the result is safe on NTFS/APFS as well as ext4.
-    """
-    folders = {u: _path_folders(u) for u in group}
-    max_depth = max((len(f) for f in folders.values()), default=0)
-
-    for depth in range(1, max_depth + 1):
-        keys = {u: tuple(p.lower() for p in folders[u][-depth:]) for u in group}
-        if len(set(keys.values())) == len(group):
-            return {u: tuple(folders[u][-depth:]) for u in group}
-
-    raise RuntimeError(
-        f"Cannot disambiguate URL group sharing filename and full parent path: {group}"
-    )
+def _clash_subfolder(url: str) -> str:
+    """Parent path segment used as disambiguator for clashing filenames."""
+    parts = urlparse(url).path.rstrip("/").split("/")
+    return unquote(parts[-2]) if len(parts) >= 2 else "unknown"
 
 
 def build_download_paths(
     urls: list[str],
     output_dir: str | Path,
 ) -> dict[str, Path]:
-    """Map each URL to a unique local file path.
+    """Map each URL to a local file path. Flat layout; clashing names get a subfolder."""
+    clashes = find_clashes(urls)
+    clash_lower = {name.lower() for name in clashes}
 
-    Unique filenames go directly under output_dir. Filenames that clash
-    (case-insensitively) get the smallest tail of their URL path prepended
-    that makes every URL in the clashing group unique — e.g. /2018/Daisy/foo.mp4
-    and /2023/Daisy/foo.mp4 land at 2018/Daisy/foo.mp4 and 2023/Daisy/foo.mp4
-    rather than colliding at Daisy/foo.mp4.
-    """
-    by_lower: defaultdict[str, list[str]] = defaultdict(list)
+    paths = {}
     for url in urls:
-        by_lower[url_to_filename(url).lower()].append(url)
-
-    base = Path(output_dir)
-    paths: dict[str, Path] = {}
-
-    for group in by_lower.values():
-        if len(group) == 1:
-            url = group[0]
-            paths[url] = base / url_to_filename(url)
-            continue
-        subfolders = _disambiguate_group(group)
-        for url in group:
-            paths[url] = base.joinpath(*subfolders[url]) / url_to_filename(url)
-
-    # Defensive: every URL must map to a distinct destination path.
-    # Case-fold the comparison since callers commonly run on NTFS/APFS where
-    # "Daisy/foo" and "daisy/foo" are the same file on disk.
-    seen: dict[str, str] = {}
-    for url, p in paths.items():
-        key = str(p).lower()
-        if key in seen:
-            raise RuntimeError(
-                f"Path collision after disambiguation: {url!r} and {seen[key]!r} "
-                f"both map to {p}"
-            )
-        seen[key] = url
-
+        filename = url_to_filename(url)
+        if filename.lower() in clash_lower:
+            paths[url] = Path(output_dir) / _clash_subfolder(url) / filename
+        else:
+            paths[url] = Path(output_dir) / filename
     return paths
 
 

config.py

@@ -15,9 +15,4 @@ SITES: Final[dict[str, dict[str, str]]] = {
         "cookie_domain": "pinkcuffs.com",
         "env_prefix": "PINKCUFFS",
     },
-    "femuniverse": {
-        "base_url": "https://www.femuniverse.com",
-        "cookie_domain": "femuniverse.com",
-        "env_prefix": "FEMUNIVERSE",
-    },
 }

download.py

@@ -14,8 +14,6 @@ import argparse
 from pathlib import Path
 import re
 import shutil
-import subprocess
-import sys
 from collections import defaultdict
 from concurrent.futures import ThreadPoolExecutor, as_completed
 from typing import Any
@@ -33,7 +31,6 @@ from check_clashes import (
     load_video_map,
     save_video_map,
     is_valid_url,
-    is_hls_url,
     VIDEO_MAP_FILE,
 )
 from config import SITES
@@ -209,30 +206,6 @@ def reorganize(
 # ── Download ─────────────────────────────────────────────────────────
 
 
-def download_hls(url: str, dest: Path, referer: str = "") -> tuple[str, int]:
-    """Download an HLS stream via yt-dlp. Returns (status, bytes_written)."""
-    dest.parent.mkdir(parents=True, exist_ok=True)
-    if dest.exists():
-        return "ok", 0
-    cmd = [
-        sys.executable, "-m", "yt_dlp",
-        "--quiet", "--no-warnings",
-        "--referer", referer or "https://player.mediadelivery.net/",
-        "-o", str(dest),
-        url,
-    ]
-    try:
-        proc = subprocess.run(cmd, capture_output=True, text=True)
-        if proc.returncode != 0:
-            lines = (proc.stderr or proc.stdout).strip().splitlines()
-            return f"error: {lines[-1] if lines else 'yt-dlp failed'}", 0
-        if not dest.exists():
-            return "error: output file missing after yt-dlp", 0
-        return "ok", dest.stat().st_size
-    except Exception as e:
-        return f"error: {e}", 0
-
-
 def download_one(
     session: requests.Session,
     url: str,
@@ -411,7 +384,7 @@ def main() -> None:
     saved = read_mode(args.output)
     mode_changed = saved is not None and saved != mode
 
-    print(f"[+] {len(urls)} video URLs from {VIDEO_MAP_FILE}")
+    print(f"[+] {len(urls)} MP4 URLs from {VIDEO_MAP_FILE}")
     print(f"[+] Naming mode: {mode}" + (" (changed!)" if mode_changed else ""))
 
     # Handle reorganize
@@ -472,7 +445,7 @@ def main() -> None:
     }
 
     newly_fetched: dict[str, int | None] = {}
-    uncached_pending = [u for u in pending if u not in cached_sizes and not is_hls_url(u)]
+    uncached_pending = [u for u in pending if u not in cached_sizes]
     session = make_session()
     if uncached_pending:
         print(
@@ -489,19 +462,17 @@ def main() -> None:
     total_bytes = sum(sized.values())
     print(f"[+] Download size: {fmt_size(total_bytes)} across {len(pending)} files")
 
-    already_sizes: dict[str, int | None] = {}
-    already_to_verify = [u for u in already if not is_hls_url(u)]
-    if already_to_verify:
-        uncached_already = [u for u in already_to_verify if u not in cached_sizes]
+    if already:
+        uncached_already = [u for u in already if u not in cached_sizes]
         if uncached_already:
             print(
-                f"[+] Verifying {len(already_to_verify)} existing files ({len(uncached_already)} uncached)…"
+                f"[+] Verifying {len(already)} existing files ({len(uncached_already)} uncached)…"
             )
             fetched_already = fetch_sizes(uncached_already, workers=20, url_referers=url_referers)
             newly_fetched.update(fetched_already)
-            already_sizes = {**cached_sizes, **fetched_already}
+            already_sizes: dict[str, int | None] = {**cached_sizes, **fetched_already}
         else:
-            print(f"[+] Verifying {len(already_to_verify)} existing files (all sizes cached)…")
+            print(f"[+] Verifying {len(already)} existing files (all sizes cached)…")
             already_sizes = dict(cached_sizes)
 
     mismatched = 0
@@ -534,8 +505,6 @@ def main() -> None:
 
     def do_download(url: str) -> tuple[str, tuple[str, int]]:
         dest = paths[url]
-        if is_hls_url(url):
-            return url, download_hls(url, dest, url_referers.get(url, ""))
         expected = remote_sizes.get(url)
         return url, download_one(
             session, url, dest, expected, url_referers.get(url, "")

main.py

@@ -335,46 +335,6 @@ def extract_title_from_html(html: str) -> str | None:
     return None
 
 
-def _is_bunny_playlist(url: str) -> bool:
-    """True if url is the root Bunny.net HLS playlist (not a sub-playlist)."""
-    parsed = urlparse(url)
-    return (
-        parsed.netloc.endswith(".b-cdn.net")
-        and parsed.path.endswith("/playlist.m3u8")
-    )
-
-
-def _is_bunny_junk(url: str) -> bool:
-    """True if url is a Bunny.net CDN init segment (not a usable video URL)."""
-    parsed = urlparse(url)
-    return parsed.netloc.endswith(".b-cdn.net") and PurePosixPath(
-        parsed.path
-    ).name in {"init.mp4", "init.dmp4"}
-
-
-def extract_bunny_embed_url(html: str) -> str | None:
-    """Return a tokenless Bunny.net embed URL found in an iframe, or None."""
-    m = re.search(
-        r'<iframe[^>]+src="(https://player\.mediadelivery\.net/embed/[^"?]+)',
-        html,
-    )
-    return m.group(1) if m else None
-
-
-def _clear_junk_video_entries(video_map: dict[str, Any], site_key: str) -> int:
-    """Reset entries whose only stored videos are CDN init segments. Returns count fixed."""
-    cleared = 0
-    for entry in video_map.values():
-        videos = entry.get("videos", [])
-        if videos and all(_is_bunny_junk(v["url"]) for v in videos):
-            entry["videos"] = []
-            entry.pop("scraped_at", None)
-            cleared += 1
-    if cleared:
-        save_video_map(video_map, site_key)
-    return cleared
-
-
 MAX_RETRIES = 2
 
 
@@ -401,9 +361,7 @@ async def worker(
 
     page.on(
         "response",
-        lambda resp: video_hits.add(resp.url)
-        if (_is_video_url(resp.url) or _is_bunny_playlist(resp.url))
-        else None,
+        lambda resp: video_hits.add(resp.url) if _is_video_url(resp.url) else None,
     )
 
     try:
@@ -418,7 +376,7 @@ async def worker(
             print(f"[W{worker_id}] ({idx + 1}/{total}) {url}{label}")
 
             try:
-                await page.goto(url, wait_until="load", timeout=60000)
+                await page.goto(url, wait_until="networkidle", timeout=60000)
             except Exception as e:
                 print(f"[W{worker_id}] Navigation error: {e}")
                 if expects_video(url) and attempt < MAX_RETRIES:
@@ -493,29 +451,18 @@ async def worker(
             found = set(html_videos) | set(video_hits)
             video_hits.clear()
 
-            print(f"[W{worker_id}] network hits raw: {found or '(empty)'}")
-
             all_videos = [
                 m
                 for m in found
                 if is_valid_url(m)
-                and not _is_bunny_junk(m)
                 and m
                 not in (
                     f"{base_url}/wp-content/plugins/easy-video-player/lib/blank.mp4",
                 )
             ]
 
-            if not all_videos:
-                embed_url = extract_bunny_embed_url(html)
-                if embed_url:
-                    print(
-                        f"[W{worker_id}] No network hit — iframe fallback: {embed_url}"
-                    )
-                    all_videos = [embed_url]
-
             async with map_lock:
-                new_found = set(all_videos) - known
+                new_found = found - known
                 if new_found:
                     print(f"[W{worker_id}] Found {len(new_found)} new video URLs")
                     known.update(new_found)
@@ -572,12 +519,6 @@ async def run_for_site(
     urls = load_post_urls(site_key, base_url, wp_api, req_headers)
 
     video_map = load_video_map(site_key)
-    junk_cleared = _clear_junk_video_entries(video_map, site_key)
-    if junk_cleared:
-        print(
-            f"[{site_key}] Cleared {junk_cleared} entries with junk CDN init segments — will re-scrape."
-        )
-
     if any(
         u not in video_map
         or not video_map[u].get("title")

requirements.txt

@@ -1,5 +1,3 @@
 playwright==1.58.0
 python-dotenv==1.2.1
 Requests==2.32.5
-yt-dlp>=2026.3.17
-pycryptodomex>=3.23.0

upload.py

@@ -8,7 +8,7 @@ Usage:
     python upload.py                      # upload from ./downloads
     python upload.py -i /mnt/vol/dl      # custom input dir
     python upload.py --batch-size 2      # upload 2, wait, repeat
-    python upload.py --dry-run           # preview (checks the channel, uploads nothing)
+    python upload.py --dry-run           # preview without uploading
     python upload.py --skip-wait         # upload without waiting
 
 Required (CLI flag or env var):
@@ -32,7 +32,6 @@ import requests
 from dotenv import load_dotenv
 
 from check_clashes import fmt_size, url_to_filename, VIDEO_EXTS, load_video_map
-from config import SITES
 from download import (
     collect_urls,
     get_paths_for_mode,
@@ -50,7 +49,6 @@ DEFAULT_BATCH_SIZE = 1
 DEFAULT_POLL = 30
 UPLOADED_FILE = ".uploaded"
 PT_NAME_MAX = 120
-PT_DESC_MIN = 3  # PeerTube rejects descriptions shorter than this
 
 
 # ── Text helpers ─────────────────────────────────────────────────────
@@ -64,11 +62,6 @@ def clean_description(raw: str) -> str:
     text = re.sub(r"<[^>]+>", "", text)
     text = html.unescape(text)
     text = re.sub(r"\n{3,}", "\n\n", text).strip()
-    # PeerTube enforces a 3-char minimum on descriptions; a sub-minimum
-    # description (e.g. a stray ".") makes the upload-init 400.  Drop it so
-    # it's omitted from the request rather than rejected.
-    if len(text) < PT_DESC_MIN:
-        return ""
     return text[:10000]
 
 
@@ -132,12 +125,11 @@ def get_channel_id(base: str, token: str, channel_name: str) -> int:
 def get_channel_video_names(base: str, token: str, channel_name: str) -> Counter[str]:
     """Paginate through the channel and return a Counter of video names."""
     counts: Counter[str] = Counter()
-    page_size = 25
     start = 0
     while True:
         r = requests.get(
             f"{base}/api/v1/video-channels/{channel_name}/videos",
-            params={"start": start, "count": page_size},
+            params={"start": start, "count": 100},
             headers=api_headers(token),
             timeout=30,
         )
@@ -145,7 +137,7 @@ def get_channel_video_names(base: str, token: str, channel_name: str) -> Counter
         data = r.json()
         for v in data.get("data", []):
             counts[v["name"]] += 1
-        start += page_size
+        start += 100
         if start >= data.get("total", 0):
             break
     return counts
@@ -449,43 +441,14 @@ def build_path_to_meta(
     return result
 
 
-def site_dirs_in(input_dir: str) -> set[str]:
-    """Names of the per-site subdirectories that actually exist under input_dir."""
-    root = Path(input_dir)
-    if not root.is_dir():
-        return set()
-    return {site for site in SITES if (root / site).is_dir()}
-
-
-def unmanaged_dirs_in(input_dir: str, site_dirs: set[str]) -> list[str]:
-    """Top-level subdirectories of input_dir that belong to no configured site."""
-    root = Path(input_dir)
-    if not root.is_dir():
-        return []
-    return sorted(
-        d.name
-        for d in root.iterdir()
-        if d.is_dir() and not d.name.startswith(".") and d.name not in site_dirs
-    )
-
-
-def find_videos(input_dir: str, only_dirs: set[str] | None = None) -> set[Path]:
-    """Walk input_dir and return a set of relative paths for all video files.
-
-    When *only_dirs* is given, just those top-level directories are descended
-    into; loose files sitting directly in input_dir are always included.  That
-    keeps a shared parent directory — an external drive holding other purchases
-    next to the site folders — from dragging in unrelated albums.
-    """
-    top = Path(input_dir)
+def find_videos(input_dir: str) -> set[Path]:
+    """Walk input_dir and return a set of relative paths for all video files."""
     found = set()
-    for cur, dirs, files in os.walk(str(top)):
+    for root, dirs, files in os.walk(input_dir):
         dirs[:] = [d for d in dirs if not d.startswith(".")]
-        if only_dirs is not None and Path(cur) == top:
-            dirs[:] = [d for d in dirs if d in only_dirs]
         for f in files:
             if Path(f).suffix.lower() in VIDEO_EXTS:
-                found.add((Path(cur) / f).relative_to(top))
+                found.add((Path(root) / f).relative_to(input_dir))
     return found
 
 
@@ -562,11 +525,6 @@ def main() -> None:
     ap.add_argument(
         "--dry-run", "-n", action="store_true", help="Preview what would be uploaded"
     )
-    ap.add_argument(
-        "--scan-all",
-        action="store_true",
-        help="Scan every subdirectory of --input, not just the configured site folders",
-    )
     args = ap.parse_args()
 
     url = args.url or os.environ.get("PEERTUBE_URL")
@@ -574,42 +532,26 @@ def main() -> None:
     channel = args.channel or os.environ.get("PEERTUBE_CHANNEL")
     password = args.password or os.environ.get("PEERTUBE_PASSWORD")
 
-    missing = [
-        label
-        for label, val in [
-            ("--url / PEERTUBE_URL", url),
-            ("--username / PEERTUBE_USER", username),
-            ("--channel / PEERTUBE_CHANNEL", channel),
-            ("--password / PEERTUBE_PASSWORD", password),
+    if not args.dry_run:
+        missing = [
+            label
+            for label, val in [
+                ("--url / PEERTUBE_URL", url),
+                ("--username / PEERTUBE_USER", username),
+                ("--channel / PEERTUBE_CHANNEL", channel),
+                ("--password / PEERTUBE_PASSWORD", password),
+            ]
+            if not val
         ]
-        if not val
-    ]
-    have_creds = not missing
-    if missing and not args.dry_run:
-        for label in missing:
-            print(f"[!] Required: {label}")
-        sys.exit(1)
+        if missing:
+            for label in missing:
+                print(f"[!] Required: {label}")
+            sys.exit(1)
 
     # ── load metadata & scan disk ──
     video_map = load_video_map()
     path_meta = build_path_to_meta(video_map, args.input)
-
-    # --input may be a shared directory that holds unrelated purchases beside
-    # the per-site folders.  Once any site folder is present, treat the layout
-    # as site-partitioned and ignore everything else; a flat directory (no site
-    # folders at all) is still walked in full.
-    site_dirs = set() if args.scan_all else site_dirs_in(args.input)
-    if site_dirs:
-        skipped = unmanaged_dirs_in(args.input, site_dirs)
-        if skipped:
-            print(
-                f"[!] Ignoring {len(skipped)} unmanaged folder(s) in {args.input}: "
-                f"{', '.join(skipped)}"
-            )
-            print("    (pass --scan-all to upload those too)")
-        on_disk = find_videos(args.input, site_dirs)
-    else:
-        on_disk = find_videos(args.input)
+    on_disk = find_videos(args.input)
 
     unmatched = on_disk - set(path_meta.keys())
     if unmatched:
@@ -619,12 +561,11 @@ def main() -> None:
         for rel in unmatched:
             path_meta[rel] = {"title": "", "description": ""}
 
-    ledger = Path(args.input) / UPLOADED_FILE
     uploaded = load_uploaded(args.input)
     pending = sorted(rel for rel in on_disk if rel not in uploaded)
 
     print(f"[+] {len(on_disk)} video files in {args.input}/")
-    print(f"[+] {len(uploaded)} already uploaded (per {ledger})")
+    print(f"[+] {len(uploaded)} already uploaded")
     print(f"[+] {len(pending)} pending")
     print(f"[+] Batch size: {args.batch_size}")
 
@@ -632,79 +573,6 @@ def main() -> None:
         print("\nAll videos already uploaded.")
         return
 
-    # ── connect to the channel ────────────────────────────────────────
-    # The .uploaded ledger lives inside --input, so pointing at a fresh
-    # directory (an external drive, say) starts from an empty one.  The channel
-    # is the real source of truth, so reconcile against it before deciding what
-    # is pending — dry runs included, whenever credentials are available.
-    base = ""
-    token = ""
-    channel_id = 0
-    existing: set[str] = set()
-
-    if have_creds:
-        assert url is not None
-        assert username is not None
-        assert channel is not None
-        assert password is not None
-
-        base = url.rstrip("/")
-        if not base.startswith("http"):
-            base = "https://" + base
-
-        print(f"\n[+] Authenticating with {base} ...")
-        token = get_oauth_token(base, username, password)
-        print(f"[+] Authenticated as {username}")
-
-        channel_id = get_channel_id(base, token, channel)
-        print(f"[+] Channel: {channel} (id {channel_id})")
-
-        name_counts = get_channel_video_names(base, token, channel)
-        existing = set(name_counts)
-        total = sum(name_counts.values())
-        print(
-            f"[+] Found {total} video(s) on channel ({len(name_counts)} unique name(s))"
-        )
-
-        dupes = {name: count for name, count in name_counts.items() if count > 1}
-        if dupes:
-            print(f"[!] {len(dupes)} duplicate name(s) detected on channel:")
-            for name, count in sorted(dupes.items()):
-                print(f"    x{count}  {name}")
-    else:
-        print(
-            "\n[!] No PeerTube credentials given — skipping the channel check; "
-            "everything below is reported as pending."
-        )
-
-    # ── pre-reconcile: sweep all pending against channel names ────────
-    # The main upload loop discovers already-uploaded videos lazily as it
-    # walks the sorted pending list — meaning on a fresh run (no .uploaded
-    # file) you won't know how many files are genuinely new until the loop
-    # has processed everything.  Doing a full sweep here, before any
-    # upload starts, gives an accurate count up-front and pre-populates
-    # .uploaded so that interrupted/re-run sessions skip them instantly
-    # without re-checking each time.  A dry run sweeps but writes nothing.
-    if existing:
-        pre_matched = [
-            rel for rel in pending if _channel_match(rel, path_meta, existing)[0]
-        ]
-        if pre_matched:
-            suffix = "" if args.dry_run else " — marking uploaded"
-            print(
-                f"\n[+] Pre-sweep: {len(pre_matched)} local file(s) already on channel{suffix}"
-            )
-            if not args.dry_run:
-                for rel in pre_matched:
-                    mark_uploaded(args.input, rel)
-            matched = set(pre_matched)
-            pending = [rel for rel in pending if rel not in matched]
-            print(f"[+] {len(pending)} left to upload\n")
-
-    if not pending:
-        print("All videos already uploaded.")
-        return
-
     # ── dry run ──
     if args.dry_run:
         total_bytes = 0
@@ -717,6 +585,55 @@ def main() -> None:
         print(f"\n  Total: {fmt_size(total_bytes)} across {len(pending)} videos")
         return
 
+    assert url is not None
+    assert username is not None
+    assert channel is not None
+    assert password is not None
+
+    # ── authenticate ──
+    base = url.rstrip("/")
+    if not base.startswith("http"):
+        base = "https://" + base
+
+    print(f"\n[+] Authenticating with {base} ...")
+    token = get_oauth_token(base, username, password)
+    print(f"[+] Authenticated as {username}")
+
+    channel_id = get_channel_id(base, token, channel)
+    print(f"[+] Channel: {channel} (id {channel_id})")
+
+    name_counts = get_channel_video_names(base, token, channel)
+    existing = set(name_counts)
+    total = sum(name_counts.values())
+    print(f"[+] Found {total} video(s) on channel ({len(name_counts)} unique name(s))")
+
+    dupes = {name: count for name, count in name_counts.items() if count > 1}
+    if dupes:
+        print(f"[!] {len(dupes)} duplicate name(s) detected on channel:")
+        for name, count in sorted(dupes.items()):
+            print(f"    x{count}  {name}")
+
+    # ── pre-reconcile: sweep all pending against channel names ────────
+    # The main upload loop discovers already-uploaded videos lazily as it
+    # walks the sorted pending list — meaning on a fresh run (no .uploaded
+    # file) you won't know how many files are genuinely new until the loop
+    # has processed everything.  Doing a full sweep here, before any
+    # upload starts, gives an accurate count up-front and pre-populates
+    # .uploaded so that interrupted/re-run sessions skip them instantly
+    # without re-checking each time.
+    pre_matched = []
+    for rel in pending:
+        if _channel_match(rel, path_meta, existing)[0]:
+            pre_matched.append(rel)
+    if pre_matched:
+        print(
+            f"\n[+] Pre-sweep: {len(pre_matched)} local file(s) already on channel — marking uploaded"
+        )
+        for rel in pre_matched:
+            mark_uploaded(args.input, rel)
+        pending = [rel for rel in pending if rel not in set(pre_matched)]
+        print(f"[+] {len(pending)} left to upload\n")
+
     nsfw = args.nsfw
     total_up = 0
     batch: list[tuple[str, str]] = []  # [(uuid, name), ...]
@@ -779,10 +696,4 @@ def main() -> None:
 
 
 if __name__ == "__main__":
-    try:
-        main()
-    except BrokenPipeError:
-        # Downstream reader closed the pipe (e.g. `| head`).  Point stdout at
-        # devnull so interpreter shutdown does not raise a second time.
-        os.dup2(os.open(os.devnull, os.O_WRONLY), sys.stdout.fileno())
-        sys.exit(0)
+    main()