Compare commits
30 Commits
cdde5e3d6a
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 32859a5df0 | |||
| 1c1dbf7f21 | |||
| fbd1cee0ba | |||
| bf1361262a | |||
| e85fce18c6 | |||
| 9752eef0b0 | |||
| d858d7e40b | |||
| bb4700f4a8 | |||
| 3a62d700bd | |||
| d5ec264e8b | |||
| ff6fcfc721 | |||
| 48ab1db956 | |||
| ffe0377f4c | |||
| 316a7db1a8 | |||
| 8f3b815f16 | |||
| 7b98d2ed25 | |||
| 9ea3753069 | |||
| c4a85c49d0 | |||
| 22593678c2 | |||
| 1ee7d7210a | |||
| c718a80651 | |||
| 66d085ad96 | |||
| 9bd45977e2 | |||
| 76ade4f768 | |||
| 2639829e43 | |||
| 2c0d01704f | |||
| 149b818036 | |||
| 9963dc0327 | |||
| b7e3031a3d | |||
| 83779b6906 |
21
AGENTS.md
21
AGENTS.md
@@ -50,8 +50,29 @@ sleep ──→ idlecheck_caffeinatestuck
|
||||
|
||||
pull-all ──→ iterates repos in ~/git/ matching *schmeeve* remote
|
||||
checkin-all ──→ iterates repos in ~/git/ matching *schmeeve* remote
|
||||
|
||||
comfyui-sync (systemd timer on nimo.loc, ~/git checkout)
|
||||
├── neatcli organize --by-type (drops → Pictures/Images/)
|
||||
├── rename-ai-snaps . -n (rename in place, top-level only)
|
||||
└── organize-images -e (sort into AI/{model}/{lora}/)
|
||||
```
|
||||
|
||||
### Image pipeline (ComfyUI)
|
||||
|
||||
Strict separation of concerns — do NOT re-merge them:
|
||||
|
||||
| Script | Role |
|
||||
|---|---|
|
||||
| `rename-ai-snaps` | Rename ONLY, in place, to `schmeeve-AI-{keywords}.png`. Skips `schmeeve-AI-*` unless `--include-renamed`; `--only-generic` limits to `ComfyUI_*` names. PNG only. |
|
||||
| `organize-images` | ALL sorting/moving. Default: top-level → `AI/ Photos/ Screenshots/ Unsorted/`. `--resort <Images root>`: recursive canonical re-sort into `AI/{model}/{lora}/` (never renames files). |
|
||||
| `extract-ai-meta` | Shared metadata engine. JSON manifest `relpath → {model, loras, dest}`; `dest` is the canonical sanitized folder (`+`-joined LoRAs, capped 200 chars). Presence in manifest = is an AI image. |
|
||||
|
||||
Canonical folder sanitizer lives in `extract-ai-meta` (`sanitize_dir_name`):
|
||||
path prefixes stripped (`\`→`/`, basename), spaces/specials → `_`. Anything else
|
||||
creating `AI/` subfolders must match it exactly.
|
||||
|
||||
`sync-comfyui-snaps` is the dormant cron predecessor of `comfyui-sync`.
|
||||
|
||||
Sub-scripts are invoked via `/bin/bash` or `/bin/sh`, never sourced.
|
||||
|
||||
### Trigger chain
|
||||
|
||||
110
FIX-AI-FOLDERS.md
Normal file
110
FIX-AI-FOLDERS.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# FIX-AI-FOLDERS — Re-sort plan
|
||||
|
||||
## Problem
|
||||
|
||||
~17K files under `/Volumes/miniShare1/Pictures/Images/AI/` were sorted by an older
|
||||
scheme. Two scripts currently create `AI/{model}/{lora}/` folders with **different
|
||||
sanitizers**, fragmenting the tree:
|
||||
|
||||
| | `organize-images` (`clean()`) | `rename-ai-snaps` (`sanitize_dir_name`) |
|
||||
|---|---|---|
|
||||
| `Pony XL\trserkanWildpony_v20.safetensors` | `Pony XL\trserkanWildpony_v20/` or `Pony XL-trserkanWildpony_v20/` (its `\\`→`-` fix only catches *double* backslashes) | `trserkanWildpony_v20/` (strips path prefix, `_`-sanitizes) |
|
||||
| spaces | kept (`FLUX Handsome Male Model_1.0/`) | → underscores |
|
||||
|
||||
Legacy artifacts also exist: `None/` and `unknown/` folders (at Images root and inside
|
||||
`AI/`), plus loose files at the `AI/` root.
|
||||
|
||||
Additional blockers:
|
||||
|
||||
- `rename-ai-snaps` **skips all `schmeeve-AI-*` files** — 17,178 of 17,255 files in
|
||||
`AI/`. As-is it would touch almost nothing.
|
||||
- `rename-ai-snaps` is PNG-only (fine: `AI/` is 99.97% PNG; 5 jpgs reported, not moved).
|
||||
- All 30 sampled `AI/` files still carry full ComfyUI `prompt` metadata — a
|
||||
metadata-driven re-sort is viable.
|
||||
- `Unsorted/` sweep: 0 of 120 sampled files (jpg/png/webp) have ComfyUI metadata —
|
||||
they are web downloads, not ComfyUI outputs. Only files with real `prompt`
|
||||
metadata qualify for moving.
|
||||
- `nas_mount_base()` does not know `/Volumes/miniShare1`.
|
||||
- `~/.local/bin/organize-images` is stale (older than repo copy).
|
||||
|
||||
## Requirements (user decisions)
|
||||
|
||||
1. **Scope:** Everything — `AI/`, `None/`, `unknown/`, plus sweep `Unsorted/` and
|
||||
`Screenshots/` for PNGs with real ComfyUI metadata.
|
||||
2. **Filenames:** Keep existing names, **except** generic names (`ComfyUI_*` etc.)
|
||||
which get a proper keyword-based rename.
|
||||
3. **Architecture:** Re-separate concerns — `rename-ai-snaps` renames only (the
|
||||
folder-moving added in `e85fce1 "model first"` was a mistake), `organize-images`
|
||||
does all sorting. Fix its sanitizer; sync the stale `~/.local/bin` copy.
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. `rename-ai-snaps` → rename-only
|
||||
|
||||
- Strip the move/folder logic (`output_root`, `mkdir`, `shutil.move`, model/lora
|
||||
folder computation). Keep metadata extraction, keyword scoring, and
|
||||
interactive/batch renaming **in place** (filename changes only, never directory
|
||||
changes).
|
||||
- Default: skip `schmeeve-AI-*` as today. New flag `--include-renamed` for
|
||||
corpus-wide renames later.
|
||||
|
||||
### 2. `extract-ai-meta` → shared, recursive metadata engine
|
||||
|
||||
- Add `-r` recursive walk; relative-path keys instead of bare basenames.
|
||||
- Port the robust extraction from `rename-ai-snaps` (`_strip_model_name`,
|
||||
`UNETLoader`/extra checkpoint classes, generic `*lora_name*` fields). It currently
|
||||
only knows `CheckpointLoaderSimple` and keeps backslash path prefixes — the cause
|
||||
of `Pony XL\…` folders.
|
||||
- Keep its connected-LoRA check (excludes disconnected LoRA nodes).
|
||||
- Add a `prompt_present` flag per file so `organize-images` can classify AI files
|
||||
**without** `mdls`, dimension heuristics, or filename guesses.
|
||||
|
||||
### 3. `organize-images` → all sorting, canonical scheme
|
||||
|
||||
- **Canonical sanitizer** (single implementation; fixes the `\\` bug and space
|
||||
handling): strip path prefix → spaces → `_` → non `[a-zA-Z0-9_.-]` → `_` →
|
||||
collapse. `Pony XL\trserkanWildpony_v20.safetensors` → `trserkanWildpony_v20`
|
||||
everywhere.
|
||||
- **New `--resort` mode:** recursively walk `AI/` (including `schmeeve-AI-*` files),
|
||||
compute canonical `AI/{model}/{lora}/` per file, move only what's misplaced.
|
||||
**Filenames never touched.** Files already correct are skipped (idempotent).
|
||||
Missing model → `unknown/`; no loras → `no-lora/`.
|
||||
- **Sort `None/` and `unknown/`:** pass as targets; files with metadata route into
|
||||
`AI/{model}/{lora}/`, files without go to `AI/unknown/`.
|
||||
- **Sweep `Unsorted/` + `Screenshots/`:** only files with actual ComfyUI `prompt`
|
||||
metadata move into `AI/{model}/{lora}/`.
|
||||
- **Generic-name hook:** before moving files matching `ComfyUI_*`/hash-style names,
|
||||
invoke `rename-ai-snaps -n` on them (its skip-rule already protects
|
||||
`schmeeve-AI-*`).
|
||||
- **Collision policy:** destination exists → append `_1`, `_2`… (never overwrite,
|
||||
never delete).
|
||||
- **Performance:** one metadata scan → JSON manifest in a temp file; move logic
|
||||
reads the manifest (no per-file `mdls`/`sips` subprocess storms).
|
||||
- Keep dry-run default, `-e` to execute, `--limit` for batching.
|
||||
|
||||
### 4. Verification
|
||||
|
||||
- Full dry-run first: summary counts per destination folder + sample moves.
|
||||
- Spot-check that `Pony XL\…`, `Pony XL-…`, `None/`, loose-root files land
|
||||
correctly; confirm already-correct files are untouched; re-run dry-run
|
||||
post-execute to confirm ~0 remaining moves (idempotency proof).
|
||||
|
||||
### 5. Deploy + docs
|
||||
|
||||
- Copy updated scripts to `~/.local/bin/` (fixes stale `organize-images`) and
|
||||
`~/Dropbox/bin/`.
|
||||
- Add a short "Image pipeline" section to `AGENTS.md` documenting the restored
|
||||
separation: `rename-ai-snaps` = rename, `organize-images` = sort,
|
||||
`extract-ai-meta` = shared engine.
|
||||
|
||||
## Untouched
|
||||
|
||||
- `Unsorted/` web downloads (no metadata = no move), `Screenshots/` non-AI files,
|
||||
non-PNGs in `AI/` (5 jpgs — reported, not moved).
|
||||
- Nothing is ever deleted; worst case is a `_1` suffix.
|
||||
|
||||
## Note
|
||||
|
||||
With `schmeeve-AI-*` names kept as-is, a handful may not match what today's keyword
|
||||
scorer would produce — but placement will be correct, and any folder can be
|
||||
re-renamed later with `rename-ai-snaps --include-renamed`.
|
||||
180
arc-export/.gitignore
vendored
Normal file
180
arc-export/.gitignore
vendored
Normal file
@@ -0,0 +1,180 @@
|
||||
# General
|
||||
.DS_Store
|
||||
.Trashes
|
||||
.Spotlight-V100
|
||||
Thumbs.db
|
||||
$RECYCLE.BIN/
|
||||
|
||||
# Bookmarks
|
||||
bookmarks/
|
||||
arc_bookmarks_*
|
||||
|
||||
# VsCode
|
||||
.vscode
|
||||
|
||||
# Idea
|
||||
.idea
|
||||
|
||||
# Project
|
||||
poetry.lock
|
||||
|
||||
# Byte-compiled / optimized / DLL files
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
develop-eggs/
|
||||
dist/
|
||||
downloads/
|
||||
eggs/
|
||||
.eggs/
|
||||
lib/
|
||||
lib64/
|
||||
parts/
|
||||
sdist/
|
||||
var/
|
||||
wheels/
|
||||
share/python-wheels/
|
||||
*.egg-info/
|
||||
.installed.cfg
|
||||
*.egg
|
||||
MANIFEST
|
||||
|
||||
# PyInstaller
|
||||
# Usually these files are written by a python script from a template
|
||||
# before PyInstaller builds the exe, so as to inject date/other infos into it.
|
||||
*.manifest
|
||||
*.spec
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
pip-delete-this-directory.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
htmlcov/
|
||||
.tox/
|
||||
.nox/
|
||||
.coverage
|
||||
.coverage.*
|
||||
.cache
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
*.cover
|
||||
*.py,cover
|
||||
.hypothesis/
|
||||
.pytest_cache/
|
||||
cover/
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Django stuff:
|
||||
*.log
|
||||
local_settings.py
|
||||
db.sqlite3
|
||||
db.sqlite3-journal
|
||||
|
||||
# Flask stuff:
|
||||
instance/
|
||||
.webassets-cache
|
||||
|
||||
# Scrapy stuff:
|
||||
.scrapy
|
||||
|
||||
# Sphinx documentation
|
||||
docs/_build/
|
||||
|
||||
# PyBuilder
|
||||
.pybuilder/
|
||||
target/
|
||||
|
||||
# Jupyter Notebook
|
||||
.ipynb_checkpoints
|
||||
|
||||
# IPython
|
||||
profile_default/
|
||||
ipython_config.py
|
||||
|
||||
# pyenv
|
||||
# For a library or package, you might want to ignore these files since the code is
|
||||
# intended to run in multiple environments; otherwise, check them in:
|
||||
# .python-version
|
||||
|
||||
# pipenv
|
||||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
||||
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
||||
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
||||
# install all needed dependencies.
|
||||
#Pipfile.lock
|
||||
|
||||
# poetry
|
||||
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
|
||||
# This is especially recommended for binary packages to ensure reproducibility, and is more
|
||||
# commonly ignored for libraries.
|
||||
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
|
||||
#poetry.lock
|
||||
|
||||
# pdm
|
||||
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
|
||||
#pdm.lock
|
||||
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
|
||||
# in version control.
|
||||
# https://pdm.fming.dev/#use-with-ide
|
||||
.pdm.toml
|
||||
|
||||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
|
||||
__pypackages__/
|
||||
|
||||
# Celery stuff
|
||||
celerybeat-schedule
|
||||
celerybeat.pid
|
||||
|
||||
# SageMath parsed files
|
||||
*.sage.py
|
||||
|
||||
# Environments
|
||||
.env
|
||||
.venv
|
||||
env/
|
||||
venv/
|
||||
ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# Spyder project settings
|
||||
.spyderproject
|
||||
.spyproject
|
||||
|
||||
# Rope project settings
|
||||
.ropeproject
|
||||
|
||||
# mkdocs documentation
|
||||
/site
|
||||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
.dmypy.json
|
||||
dmypy.json
|
||||
|
||||
# Pyre type checker
|
||||
.pyre/
|
||||
|
||||
# pytype static type analyzer
|
||||
.pytype/
|
||||
|
||||
# Cython debug symbols
|
||||
cython_debug/
|
||||
|
||||
# PyCharm
|
||||
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
|
||||
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
|
||||
# and can be added to the global gitignore or merged into this file. For a more nuclear
|
||||
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
|
||||
#.idea/
|
||||
7
build-lora-sheet
Executable file
7
build-lora-sheet
Executable file
@@ -0,0 +1,7 @@
|
||||
#!/bin/bash
|
||||
# Generate LoRA trigger word sheet to ~/git/ai/comfyui/lora-info-sheet.html
|
||||
|
||||
DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
"$DIR/lora-trigger-sheet" \
|
||||
"$HOME/mnt/nimo.loc/ComfyUI/models/loras" \
|
||||
"$HOME/git/ai/comfyui/lora-info-sheet.html"
|
||||
58
comfyui-sync
58
comfyui-sync
@@ -1,27 +1,43 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
SOURCE="$HOME/ComfyUI/output"
|
||||
SHARE_DIR="/mini.nas/miniShare1/Pictures"
|
||||
OUTPUT_SOURCE="$HOME/ComfyUI/output"
|
||||
VIDEO_SOURCE="$HOME/ComfyUI/videos"
|
||||
AUDIO_SOURCE="$HOME/ComfyUI/output/audio"
|
||||
NAS_MOUNT="/mini.nas/miniShare1"
|
||||
SHARE_DIR="$NAS_MOUNT/Pictures"
|
||||
STATE_FILE="$HOME/.local/state/comfyui-sync.state"
|
||||
NEATCLI="$(command -v neatcli || true)"
|
||||
NEATCLI="$(command -v /home/linuxbrew/.linuxbrew/bin/neatcli || true)"
|
||||
RENAME_SCRIPT="$HOME/git/schmeeve-toolz/rename-ai-snaps"
|
||||
ORGANIZE_SCRIPT="$HOME/git/schmeeve-toolz/organize-images"
|
||||
|
||||
mkdir -p "$(dirname "$STATE_FILE")"
|
||||
|
||||
if [ ! -d "$SOURCE" ]; then
|
||||
echo "[comfyui-sync] ERROR: Source $SOURCE does not exist"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -d "$SHARE_DIR" ]; then
|
||||
echo "[comfyui-sync] ERROR: Share dir $SHARE_DIR does not exist"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! mountpoint -q "$NAS_MOUNT"; then
|
||||
echo "[comfyui-sync] $NAS_MOUNT is not mounted, attempting to mount..."
|
||||
if ! sudo -n mount "$NAS_MOUNT" 2>&1; then
|
||||
echo "[comfyui-sync] ERROR: mount attempt failed"
|
||||
exit 1
|
||||
fi
|
||||
if ! mountpoint -q "$NAS_MOUNT"; then
|
||||
echo "[comfyui-sync] ERROR: $NAS_MOUNT still not mounted after mount attempt, refusing to copy locally"
|
||||
exit 1
|
||||
fi
|
||||
echo "[comfyui-sync] Mount succeeded."
|
||||
fi
|
||||
|
||||
echo "[comfyui-sync] Starting at $(date)"
|
||||
|
||||
current_files=$(ls "$SOURCE" | sort)
|
||||
current_files=$( {
|
||||
find "$OUTPUT_SOURCE" -maxdepth 1 -type f -printf '%f\n' 2>/dev/null
|
||||
find "$VIDEO_SOURCE" -maxdepth 1 -type f -printf '%f\n' 2>/dev/null
|
||||
find "$AUDIO_SOURCE" -maxdepth 1 -type f -printf '%f\n' 2>/dev/null
|
||||
} | sort)
|
||||
|
||||
if [ -f "$STATE_FILE" ]; then
|
||||
new_files=$(comm -13 <(sort "$STATE_FILE") <(echo "$current_files"))
|
||||
@@ -32,19 +48,24 @@ fi
|
||||
|
||||
new_count=$(echo "$new_files" | grep -c '[^[:space:]]' || true)
|
||||
|
||||
if [ "$new_count" -eq 0 ]; then
|
||||
echo "[comfyui-sync] No new files to copy."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$new_count" -gt 0 ]; then
|
||||
echo "[comfyui-sync] Copying $new_count new file(s)..."
|
||||
while IFS= read -r f; do
|
||||
[ -z "$f" ] && continue
|
||||
cp "$SOURCE/$f" "$SHARE_DIR/"
|
||||
if [ -f "$OUTPUT_SOURCE/$f" ]; then
|
||||
cp "$OUTPUT_SOURCE/$f" "$SHARE_DIR/"
|
||||
elif [ -f "$VIDEO_SOURCE/$f" ]; then
|
||||
cp "$VIDEO_SOURCE/$f" "$SHARE_DIR/"
|
||||
elif [ -f "$AUDIO_SOURCE/$f" ]; then
|
||||
cp "$AUDIO_SOURCE/$f" "$SHARE_DIR/"
|
||||
fi
|
||||
done <<<"$new_files"
|
||||
|
||||
echo "$current_files" >"$STATE_FILE"
|
||||
echo "[comfyui-sync] State file updated."
|
||||
else
|
||||
echo "[comfyui-sync] No new files to copy."
|
||||
fi
|
||||
|
||||
if [ -n "$NEATCLI" ]; then
|
||||
echo "[comfyui-sync] Running neatcli organize --by-type -e $SHARE_DIR"
|
||||
@@ -61,4 +82,11 @@ elif [ ! -x "$RENAME_SCRIPT" ]; then
|
||||
echo "[comfyui-sync] WARNING: rename-ai-snaps not found, skipping rename step"
|
||||
fi
|
||||
|
||||
if [ -d "$IMAGES_DIR" ] && [ -x "$ORGANIZE_SCRIPT" ]; then
|
||||
echo "[comfyui-sync] Running organize-images on $IMAGES_DIR"
|
||||
"$ORGANIZE_SCRIPT" "$IMAGES_DIR" --execute
|
||||
elif [ ! -x "$ORGANIZE_SCRIPT" ]; then
|
||||
echo "[comfyui-sync] WARNING: organize-images not found, skipping organize step"
|
||||
fi
|
||||
|
||||
echo "[comfyui-sync] Done at $(date)"
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
146
extract-ai-meta
146
extract-ai-meta
@@ -1,13 +1,22 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract model and LoRA metadata from ComfyUI-generated images.
|
||||
|
||||
Outputs a JSON mapping of filename → {"model": "...", "loras": ["...", ...]}.
|
||||
Outputs a JSON mapping of path (relative to target) →
|
||||
{"model": "...", "loras": ["...", ...], "dest": "{model_dir}/{lora_dir}"}
|
||||
|
||||
Only files with parseable ComfyUI 'prompt' metadata are included, so
|
||||
presence in the output doubles as "is an AI-generated image" for
|
||||
organize-images. 'dest' is the canonical folder path (sans leading "AI/"):
|
||||
model/LoRA names are path-stripped and sanitized (spaces/specials → _),
|
||||
LoRAs joined with '+', missing model → "unknown", no LoRAs → "no-lora".
|
||||
|
||||
Usage: extract-ai-meta [path] [-r|--recursive]
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from PIL import Image
|
||||
@@ -17,6 +26,43 @@ except ImportError:
|
||||
|
||||
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
|
||||
CHECKPOINT_CLASS_TYPES = {
|
||||
"CheckpointLoaderSimple",
|
||||
"Checkpoint Loader with Name (Image Saver)",
|
||||
"CheckpointLoader",
|
||||
}
|
||||
|
||||
MODEL_CLASS_TYPES = {
|
||||
"UNETLoader",
|
||||
}
|
||||
|
||||
LORA_CLASS_TYPES = {
|
||||
"LoraLoader",
|
||||
"Power Lora Loader (rgthree)",
|
||||
}
|
||||
|
||||
# Filesystem-safe cap for the joined LoRA combo directory name
|
||||
MAX_LORA_DIR_LEN = 200
|
||||
|
||||
|
||||
def _strip_model_name(raw):
|
||||
"""Strip path prefix and .safetensors extension from a model/LoRA name."""
|
||||
name = raw.strip()
|
||||
# Handle backslash/forward-slash paths: "Pony\model.safetensors" or "Pony/model.safetensors"
|
||||
name = name.replace("\\", "/")
|
||||
name = os.path.basename(name)
|
||||
if name.endswith(".safetensors"):
|
||||
name = name[:-len(".safetensors")]
|
||||
return name
|
||||
|
||||
|
||||
def sanitize_dir_name(name):
|
||||
"""Canonical directory-name sanitizer (spaces/specials → _)."""
|
||||
name = name.strip().replace(" ", "_")
|
||||
name = re.sub(r"[^a-zA-Z0-9_.-]", "_", name)
|
||||
name = re.sub(r"_+", "_", name).strip("_")
|
||||
return name
|
||||
|
||||
|
||||
def is_lora_connected(node_id, data):
|
||||
"""Check if a LoRA node's output is consumed by any downstream node."""
|
||||
@@ -30,19 +76,23 @@ def is_lora_connected(node_id, data):
|
||||
|
||||
|
||||
def extract_ai_meta(filepath):
|
||||
"""Return (model, loras) or (None, []) if no ComfyUI metadata found."""
|
||||
"""Return (model, loras, has_prompt).
|
||||
|
||||
has_prompt is True whenever the file carries parseable ComfyUI 'prompt'
|
||||
metadata, even if no model/LoRA nodes were found in it.
|
||||
"""
|
||||
try:
|
||||
img = Image.open(filepath)
|
||||
except Exception:
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
if "prompt" not in img.info:
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
try:
|
||||
data = json.loads(img.info["prompt"])
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
model = None
|
||||
loras = []
|
||||
@@ -51,24 +101,59 @@ def extract_ai_meta(filepath):
|
||||
cls = node.get("class_type", "")
|
||||
inputs = node.get("inputs", {})
|
||||
|
||||
if cls == "CheckpointLoaderSimple":
|
||||
ckpt = inputs.get("ckpt_name", "")
|
||||
if ckpt:
|
||||
model = ckpt.rsplit(".", 1)[0]
|
||||
if model is None:
|
||||
if cls in CHECKPOINT_CLASS_TYPES and inputs.get("ckpt_name"):
|
||||
model = _strip_model_name(inputs["ckpt_name"])
|
||||
elif cls in MODEL_CLASS_TYPES and inputs.get("unet_name"):
|
||||
model = _strip_model_name(inputs["unet_name"])
|
||||
|
||||
if "lora" in cls.lower():
|
||||
lora = inputs.get("lora_name", "")
|
||||
if lora:
|
||||
if cls in LORA_CLASS_TYPES and inputs.get("lora_name"):
|
||||
# Only include LoRAs whose output is actually consumed
|
||||
if is_lora_connected(nid, data):
|
||||
loras.append(lora.rsplit(".", 1)[0])
|
||||
loras.append(_strip_model_name(inputs["lora_name"]))
|
||||
else:
|
||||
# Some custom nodes use fields ending in 'lora'/'lora_name'
|
||||
for field, val in inputs.items():
|
||||
if isinstance(val, str) and val.endswith(".safetensors") and "lora" in field.lower():
|
||||
name = _strip_model_name(val)
|
||||
if name:
|
||||
loras.append(name)
|
||||
|
||||
loras.sort()
|
||||
return model, loras
|
||||
loras = sorted(set(loras))
|
||||
return model, loras, True
|
||||
|
||||
|
||||
def dest_for(model, loras):
|
||||
"""Canonical relative destination dir (sans 'AI/' prefix)."""
|
||||
model_dir = sanitize_dir_name(model) if model else "unknown"
|
||||
if loras:
|
||||
lora_dir = "+".join(sanitize_dir_name(l) for l in loras if l)
|
||||
if len(lora_dir) > MAX_LORA_DIR_LEN:
|
||||
lora_dir = lora_dir[:MAX_LORA_DIR_LEN].rstrip("+_")
|
||||
if not lora_dir:
|
||||
lora_dir = "no-lora"
|
||||
else:
|
||||
lora_dir = "no-lora"
|
||||
return f"{model_dir}/{lora_dir}"
|
||||
|
||||
|
||||
def iter_images(target, recursive):
|
||||
if recursive:
|
||||
for dirpath, _dirnames, filenames in os.walk(target):
|
||||
for fn in sorted(filenames):
|
||||
if os.path.splitext(fn)[1].lower() in IMAGE_EXTS:
|
||||
yield os.path.join(dirpath, fn)
|
||||
else:
|
||||
for entry in sorted(os.listdir(target)):
|
||||
fpath = os.path.join(target, entry)
|
||||
if os.path.isfile(fpath) and os.path.splitext(entry)[1].lower() in IMAGE_EXTS:
|
||||
yield fpath
|
||||
|
||||
|
||||
def main():
|
||||
target = sys.argv[1] if len(sys.argv) > 1 else os.path.expanduser("~/Pictures")
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("-")]
|
||||
recursive = any(a in ("-r", "--recursive") for a in sys.argv[1:])
|
||||
target = args[0] if args else os.path.expanduser("~/Pictures")
|
||||
target = os.path.abspath(target)
|
||||
|
||||
if not os.path.isdir(target):
|
||||
@@ -76,18 +161,25 @@ def main():
|
||||
sys.exit(1)
|
||||
|
||||
meta = {}
|
||||
|
||||
for entry in sorted(os.listdir(target)):
|
||||
fpath = os.path.join(target, entry)
|
||||
if not os.path.isfile(fpath):
|
||||
continue
|
||||
ext = os.path.splitext(entry)[1].lower()
|
||||
if ext not in IMAGE_EXTS:
|
||||
scanned = 0
|
||||
for fpath in iter_images(target, recursive):
|
||||
scanned += 1
|
||||
if scanned % 200 == 0:
|
||||
print(f"\r …scanned {scanned} images ({len(meta)} with AI metadata)",
|
||||
end="", file=sys.stderr, flush=True)
|
||||
model, loras, has_prompt = extract_ai_meta(fpath)
|
||||
if not has_prompt:
|
||||
continue
|
||||
rel = os.path.relpath(fpath, target)
|
||||
meta[rel] = {
|
||||
"model": model,
|
||||
"loras": loras,
|
||||
"dest": dest_for(model, loras),
|
||||
}
|
||||
|
||||
model, loras = extract_ai_meta(fpath)
|
||||
if model or loras:
|
||||
meta[entry] = {"model": model, "loras": loras}
|
||||
if scanned:
|
||||
print(f"\r Scanned {scanned} images, {len(meta)} with AI metadata. ",
|
||||
file=sys.stderr, flush=True)
|
||||
|
||||
print(json.dumps(meta, indent=2))
|
||||
|
||||
|
||||
BIN
img/ai-technology.png
Normal file
BIN
img/ai-technology.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 31 KiB |
BIN
img/nude.png
Normal file
BIN
img/nude.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 24 KiB |
1036
lora-info-sheet.html
Normal file
1036
lora-info-sheet.html
Normal file
File diff suppressed because one or more lines are too long
629
lora-trigger-sheet
Executable file
629
lora-trigger-sheet
Executable file
@@ -0,0 +1,629 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Generate a LoRA trigger word dictionary HTML page from .safetensors metadata.
|
||||
|
||||
Usage:
|
||||
lora-trigger-sheet # default: ~/mnt/nimo.loc/ComfyUI/models/loras
|
||||
lora-trigger-sheet /path/to/loras # specify directory
|
||||
lora-trigger-sheet . # current directory
|
||||
lora-trigger-sheet /path/to/loras /path/to/output # specify both input and output
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import hashlib
|
||||
import time
|
||||
from urllib.error import HTTPError, URLError
|
||||
from urllib.parse import quote
|
||||
from urllib.request import urlopen, Request
|
||||
|
||||
|
||||
def read_safetensors_meta(filepath):
|
||||
try:
|
||||
with open(filepath, "rb") as f:
|
||||
header_len = struct.unpack("<Q", f.read(8))[0]
|
||||
raw = f.read(header_len)
|
||||
parsed = json.loads(raw)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
# Metadata is inside "__metadata__" key
|
||||
if "__metadata__" in parsed:
|
||||
return parsed["__metadata__"]
|
||||
return parsed
|
||||
|
||||
|
||||
def get_tool(meta):
|
||||
if meta is None:
|
||||
return "error"
|
||||
|
||||
if meta.get("ss_sd_scripts_commit_hash"):
|
||||
return "sd-scripts"
|
||||
|
||||
net = (meta.get("ss_training_network") or meta.get("ss_network_module") or "").lower()
|
||||
if "kohya" in net:
|
||||
return "kohya"
|
||||
if "sd-scripts" in net or "sd_scripts" in net:
|
||||
return "sd-scripts"
|
||||
|
||||
if meta.get("ss_learning_rate"):
|
||||
return "sd-scripts"
|
||||
|
||||
if net:
|
||||
return net
|
||||
|
||||
return "unknown"
|
||||
|
||||
|
||||
def get_base_model(meta):
|
||||
if meta is None:
|
||||
return "error"
|
||||
|
||||
base = (meta.get("ss_base_model_version")
|
||||
or meta.get("ss_base_model")
|
||||
or meta.get("modelspec.architecture", "")
|
||||
.replace("/lora", "")
|
||||
or "")
|
||||
if base:
|
||||
return base
|
||||
|
||||
return "unknown"
|
||||
|
||||
|
||||
def get_tags(meta):
|
||||
if meta is None:
|
||||
return None, 0
|
||||
|
||||
raw = meta.get("ss_tag_frequency")
|
||||
if not raw:
|
||||
return None, 0
|
||||
|
||||
# ss_tag_frequency is often a JSON string; parse if needed
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
raw = json.loads(raw)
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None, 0
|
||||
|
||||
tags = {}
|
||||
total = 0
|
||||
|
||||
if isinstance(raw, dict):
|
||||
for category, tag_dict in raw.items():
|
||||
if isinstance(tag_dict, dict):
|
||||
for tag, count in tag_dict.items():
|
||||
total += count
|
||||
tags[tag] = tags.get(tag, 0) + count
|
||||
elif isinstance(tag_dict, (int, float)):
|
||||
total += tag_dict
|
||||
|
||||
if not tags:
|
||||
if all(isinstance(v, (int, float)) for v in raw.values()):
|
||||
tags = raw
|
||||
total = sum(raw.values())
|
||||
else:
|
||||
return None, 0
|
||||
|
||||
return tags, total
|
||||
|
||||
|
||||
def shorten_base_model(name):
|
||||
name = re.sub(r"[-_][Vv]\d+(?:[._-]\d+)?$", "", name)
|
||||
name = re.sub(r"[-_][Vv]\d+[a-z]?\d*$", "", name)
|
||||
replacements = {
|
||||
"stable-diffusion-xl-v1-base": "sdxl_base_v1-0",
|
||||
"stable diffusion xl base v1.0": "sdxl_base_v1-0",
|
||||
"sdxl base v1.0": "sdxl_base_v1-0",
|
||||
"sdxl base v0.9": "sdxl_base_v0-9",
|
||||
"flux.1": "flux1",
|
||||
"flux": "flux1",
|
||||
}
|
||||
for old, new in replacements.items():
|
||||
if name.lower() == old.lower() or name.lower().startswith(old.lower()):
|
||||
return new
|
||||
return name
|
||||
|
||||
|
||||
def html_id():
|
||||
import hashlib, time
|
||||
return hashlib.md5(str(time.monotonic_ns()).encode()).hexdigest()[:8]
|
||||
|
||||
|
||||
# --- Source URL lookup cache ---
|
||||
|
||||
CACHE_DIR = os.path.expanduser("~/.cache")
|
||||
CACHE_FILE = os.path.join(CACHE_DIR, "lora-sources.json")
|
||||
|
||||
|
||||
def load_source_cache():
|
||||
if os.path.exists(CACHE_FILE):
|
||||
try:
|
||||
with open(CACHE_FILE) as f:
|
||||
return json.load(f)
|
||||
except (json.JSONDecodeError, IOError):
|
||||
return {}
|
||||
return {}
|
||||
|
||||
|
||||
def save_source_cache(cache):
|
||||
os.makedirs(CACHE_DIR, exist_ok=True)
|
||||
with open(CACHE_FILE, "w") as f:
|
||||
json.dump(cache, f, indent=2)
|
||||
|
||||
|
||||
def file_blake2b_hash(filepath):
|
||||
h = hashlib.blake2b(digest_size=32)
|
||||
with open(filepath, "rb") as f:
|
||||
while True:
|
||||
chunk = f.read(65536)
|
||||
if not chunk:
|
||||
break
|
||||
h.update(chunk)
|
||||
return h.hexdigest()
|
||||
|
||||
|
||||
def _fetch_json(url, timeout=10):
|
||||
req = Request(url, headers={"User-Agent": "lora-trigger-sheet/1.0"})
|
||||
with urlopen(req, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode())
|
||||
|
||||
|
||||
# URL pattern for extracting embedded links from free-text fields
|
||||
_URL_RE = re.compile(r'https?://\S+', re.IGNORECASE)
|
||||
|
||||
# Metadata fields that may contain a source URL, checked in order
|
||||
_META_URL_FIELDS = [
|
||||
"modelspec.source",
|
||||
"source",
|
||||
"url",
|
||||
"model_url",
|
||||
"ss_training_comment",
|
||||
"note",
|
||||
"description",
|
||||
"comment",
|
||||
]
|
||||
|
||||
# Map hostname → display label
|
||||
_SOURCE_LABELS = {
|
||||
"civitai.com": "CivitAI",
|
||||
"huggingface.co": "HuggingFace",
|
||||
"modelscope.cn": "ModelScope",
|
||||
"tensor.art": "Tensor.Art",
|
||||
}
|
||||
|
||||
|
||||
def _label_for_url(url):
|
||||
from urllib.parse import urlparse
|
||||
host = urlparse(url).netloc.lstrip("www.")
|
||||
for domain, label in _SOURCE_LABELS.items():
|
||||
if host == domain or host.endswith("." + domain):
|
||||
return label
|
||||
return host or "Link"
|
||||
|
||||
|
||||
def _extract_meta_url(meta):
|
||||
"""Return (url, label) from embedded metadata fields, or (None, None)."""
|
||||
if not meta:
|
||||
return None, None
|
||||
for field in _META_URL_FIELDS:
|
||||
val = meta.get(field)
|
||||
if not val or not isinstance(val, str):
|
||||
continue
|
||||
# Field may be a bare URL or free text containing one
|
||||
m = _URL_RE.search(val.strip())
|
||||
if m:
|
||||
url = m.group(0).rstrip(".,;)")
|
||||
return url, _label_for_url(url)
|
||||
return None, None
|
||||
|
||||
|
||||
def lookup_source(filepath, filename, cache, meta=None):
|
||||
now = int(time.time())
|
||||
cached = cache.get(filename)
|
||||
if cached:
|
||||
url = cached.get("url")
|
||||
label = cached.get("label")
|
||||
last = cached.get("last_checked", 0)
|
||||
if (now - last) < 86400 * 30:
|
||||
return (url, label) if url else (None, None)
|
||||
|
||||
# 0. Embedded URL in safetensors __metadata__ (free, no network call)
|
||||
url, label = _extract_meta_url(meta)
|
||||
if url:
|
||||
cache[filename] = {"url": url, "label": label, "last_checked": now}
|
||||
return url, label
|
||||
|
||||
# CivitAI model-version lookup by BLAKE2b file hash
|
||||
try:
|
||||
h = file_blake2b_hash(filepath)
|
||||
data = _fetch_json(
|
||||
f"https://civitai.com/api/v1/model-versions/by-hash/{h}"
|
||||
)
|
||||
if isinstance(data, dict):
|
||||
model_id = data.get("modelId")
|
||||
version_id = data.get("id")
|
||||
if model_id:
|
||||
url = f"https://civitai.com/models/{model_id}"
|
||||
if version_id:
|
||||
url += f"?modelVersionId={version_id}"
|
||||
cache[filename] = {
|
||||
"url": url, "label": "CivitAI",
|
||||
"model_name": data.get("name", ""),
|
||||
"last_checked": now,
|
||||
}
|
||||
return url, "CivitAI"
|
||||
except (HTTPError, URLError, json.JSONDecodeError, OSError):
|
||||
pass
|
||||
|
||||
# Fallback: HuggingFace model search by filename
|
||||
try:
|
||||
q = (
|
||||
filename.replace(".safetensors", "")
|
||||
.replace("_", " ")
|
||||
.replace("-", " ")
|
||||
)
|
||||
data = _fetch_json(
|
||||
"https://huggingface.co/api/models"
|
||||
f"?search={quote(q)}&sort=downloads&direction=-1&limit=3"
|
||||
)
|
||||
if isinstance(data, list) and len(data) > 0:
|
||||
model_id = data[0].get("modelId", "")
|
||||
if model_id:
|
||||
url = f"https://huggingface.co/{model_id}"
|
||||
cache[filename] = {
|
||||
"url": url, "label": "HuggingFace",
|
||||
"model_name": model_id, "last_checked": now,
|
||||
}
|
||||
return url, "HuggingFace"
|
||||
except (HTTPError, URLError, json.JSONDecodeError, OSError):
|
||||
pass
|
||||
|
||||
# Fallback: ModelScope model search by filename
|
||||
try:
|
||||
q = (
|
||||
filename.replace(".safetensors", "")
|
||||
.replace("_", " ")
|
||||
.replace("-", " ")
|
||||
)
|
||||
data = _fetch_json(
|
||||
f"https://www.modelscope.cn/api/v1/models?Name={quote(q)}&PageSize=3&PageNumber=1"
|
||||
)
|
||||
models = (data or {}).get("Data", {}).get("Models") or []
|
||||
if models:
|
||||
m = models[0]
|
||||
model_id = m.get("Path") or m.get("Name", "")
|
||||
if model_id:
|
||||
url = f"https://www.modelscope.cn/models/{model_id}"
|
||||
cache[filename] = {
|
||||
"url": url, "label": "ModelScope",
|
||||
"model_name": model_id, "last_checked": now,
|
||||
}
|
||||
return url, "ModelScope"
|
||||
except (HTTPError, URLError, json.JSONDecodeError, OSError):
|
||||
pass
|
||||
|
||||
cache[filename] = {"url": None, "label": None, "last_checked": now}
|
||||
return None, None
|
||||
|
||||
|
||||
def generate_html(loras, sources=None):
|
||||
if sources is None:
|
||||
sources = {}
|
||||
rows = []
|
||||
no_trigger_files = []
|
||||
|
||||
for fname, meta in loras:
|
||||
tool = get_tool(meta)
|
||||
base = shorten_base_model(get_base_model(meta))
|
||||
tags, total = get_tags(meta)
|
||||
|
||||
fname_no_ext = re.sub(r"\.safetensors$", "", fname)
|
||||
|
||||
if tags:
|
||||
sorted_tags = sorted(tags.items(), key=lambda x: -x[1])
|
||||
top8 = sorted_tags[:8]
|
||||
rest = sorted_tags[8:]
|
||||
has_rest = len(rest) > 0
|
||||
|
||||
top8_html = "".join(
|
||||
f"<span class=\"tg\"><code>{_e(t)}</code> <span class=\"c\">({c}x)</span></span>"
|
||||
for t, c in top8
|
||||
)
|
||||
rest_html = ""
|
||||
more_id = ""
|
||||
if has_rest:
|
||||
more_id = "m_" + html_id()
|
||||
more_count = sum(c for _, c in rest)
|
||||
rest_html = " <span class=\"ml\" onclick=\"t('" + more_id + "',this)\" data-n=\"" + str(more_count) + "\">" + str(more_count) + " more…</span>"
|
||||
rest_list = "".join(
|
||||
f"<span class=\"tg\"><code>{_e(t)}</code> <span class=\"c\">({c}x)</span></span>"
|
||||
for t, c in rest
|
||||
)
|
||||
rest_html += "\n <span class=\"ht\" id=\"" + more_id + "\">" + rest_list + "</span>"
|
||||
|
||||
triggers = top8_html + rest_html
|
||||
else:
|
||||
no_trigger_files.append((fname_no_ext, tool, base))
|
||||
triggers = "<span class=\"n\">(none)</span>"
|
||||
|
||||
src_url, src_label = sources.get(fname, (None, None))
|
||||
rows.append((fname_no_ext, tool, base, triggers, src_url or "", src_label or ""))
|
||||
|
||||
num_files = len(rows)
|
||||
|
||||
table_rows = ""
|
||||
for fname, tool, base, triggers, src_url, src_label in rows:
|
||||
if src_url:
|
||||
src_cell = f'<a href="{_e(src_url)}" target="_blank" rel="noopener">{_e(src_label)}</a>'
|
||||
else:
|
||||
src_cell = '<span class="n">—</span>'
|
||||
table_rows += (
|
||||
f" <tr>\n"
|
||||
f" <td class=\"fn\">{_e(fname)}</td>\n"
|
||||
f" <td class=\"tl\">{_e(tool)}</td>\n"
|
||||
f" <td class=\"bm\">{_e(base)}</td>\n"
|
||||
f" <td class=\"tr\">{triggers}</td>\n"
|
||||
f" <td class=\"sr\">{src_cell}</td>\n"
|
||||
f" </tr>\n"
|
||||
)
|
||||
|
||||
html = f"""<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head><meta charset="UTF-8"><meta name="viewport" content="width=device-width,initial-scale=1.0">
|
||||
<title>LoRA Info Sheet</title>
|
||||
<link rel="icon" type="image/png" href="@img/nude.png">
|
||||
<style>
|
||||
*,*::before,*::after{{box-sizing:border-box;margin:0;padding:0}}
|
||||
body{{font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;background:#0d1117;color:#c9d1d9;padding:24px;line-height:1.5;font-size:14px}}
|
||||
h1{{font-size:1.5rem;margin-bottom:4px;color:#f0f6fc}}
|
||||
.sub{{color:#8b949e;margin-bottom:16px;font-size:0.85rem}}
|
||||
.bar{{display:flex;gap:12px;margin-bottom:12px;flex-wrap:wrap;align-items:center}}
|
||||
#q{{flex:1;min-width:200px;padding:8px 12px;border:1px solid #30363d;border-radius:6px;background:#161b22;color:#c9d1d9;font-size:0.85rem;outline:none}}
|
||||
#q:focus{{border-color:#58a6ff}}
|
||||
#cnt{{color:#8b949e;font-size:0.85rem}}
|
||||
table{{width:100%;border-collapse:collapse;font-size:0.82rem}}
|
||||
th{{text-align:left;padding:9px 10px;border-bottom:2px solid #30363d;color:#f0f6fc;font-weight:600;cursor:pointer;user-select:none;white-space:nowrap;background:#161b22;position:sticky;top:0;z-index:1}}
|
||||
th:hover{{color:#58a6ff}}
|
||||
th .ar{{margin-left:3px;color:#58a6ff}}
|
||||
td{{padding:7px 10px;border-bottom:1px solid #21262d;vertical-align:top;word-break:break-word}}
|
||||
tr:hover td{{background:#161b22}}
|
||||
.fn{{color:#58a6ff;font-weight:500;white-space:nowrap}}
|
||||
.tl,.bm{{color:#8b949e}}
|
||||
.tr code{{background:#1f2937;padding:1px 5px;border-radius:3px;font-size:0.78rem;color:#f0c674;word-break:break-word}}
|
||||
.c{{color:#8b949e;font-size:0.72rem;margin-right:3px}}
|
||||
.n{{color:#484f58;font-style:italic}}
|
||||
.ml{{color:#58a6ff;cursor:pointer;font-size:0.78rem;text-decoration:none;border-bottom:1px dotted #58a6ff}}
|
||||
.ml:hover{{border-bottom:1px solid #58a6ff}}
|
||||
.ht{{display:none}}
|
||||
.dr td{{background:#161b22;padding:10px 14px;border-bottom:2px solid #30363d}}
|
||||
.dr code{{background:#1f2937;padding:1px 5px;border-radius:3px;font-size:0.78rem;color:#f0c674}}
|
||||
.tg{{display:inline-block;margin:1px 6px 1px 0}}
|
||||
mark.hl{{background:#264f78;color:#f0f6fc;border-radius:2px;padding:0 2px}}
|
||||
.sr a{{color:#58a6ff;text-decoration:none;font-size:0.78rem}}
|
||||
.c0{{width:24%}}.c1{{width:8%}}.c2{{width:12%}}.c3{{width:48%}}.c4{{width:8%}}
|
||||
@media(max-width:768px){{.c1,.c2,.c4,td:nth-child(2),td:nth-child(3),td:nth-child(5){{display:none}}.c0{{width:30%}}.c3{{width:70%}}}}
|
||||
.foot{{margin-top:20px;color:#8b949e}}
|
||||
.foot h2{{font-size:1rem;color:#f0f6fc;margin-bottom:6px}}
|
||||
.foot ul{{list-style:none;columns:3;font-size:0.82rem}}
|
||||
.foot li{{padding:1px 0}}
|
||||
</style></head>
|
||||
<body>
|
||||
<h1>LoRA Info Sheet</h1>
|
||||
<p class="sub">{num_files} LoRA files · <span id="vc">{num_files}</span> visible</p>
|
||||
<div class="bar"><input type="text" id="q" placeholder="Search filename, trigger, base model..." autofocus> <span id="cnt">{num_files} / {num_files}</span></div>
|
||||
<table><thead><tr>
|
||||
<th class="c0" onclick="s(0)">File <span class="ar">▲</span></th>
|
||||
<th class="c1" onclick="s(1)">Tool <span class="ar"></span></th>
|
||||
<th class="c2" onclick="s(2)">Base <span class="ar"></span></th>
|
||||
<th class="c3" onclick="s(3)">Trigger(s) <span class="ar"></span></th>
|
||||
<th class="c4" onclick="s(4)">Source <span class="ar"></span></th>
|
||||
</tr></thead><tbody id="b">
|
||||
{table_rows}</tbody></table>
|
||||
|
||||
<script>
|
||||
let col = 0, desc = true;
|
||||
const b = document.getElementById('b');
|
||||
const q = document.getElementById('q');
|
||||
const cnt = document.getElementById('cnt');
|
||||
const vc = document.getElementById('vc');
|
||||
|
||||
function s(i){{
|
||||
document.querySelectorAll('.dr').forEach(r => r.remove());
|
||||
document.querySelectorAll('.ml').forEach(el => el.innerHTML = el.getAttribute('data-n') + ' more…');
|
||||
if (col == i) desc = !desc; else {{ col = i; desc = i == 0; }}
|
||||
let rows = [...b.children];
|
||||
rows.sort((a,c)=>{{
|
||||
let av = a.children[col].textContent.trim().toLowerCase();
|
||||
let cv = c.children[col].textContent.trim().toLowerCase();
|
||||
if (col == 0 || col == 3) {{
|
||||
let am = parseInt(av.match(/\\d+/)?.[0]||'0');
|
||||
let cm = parseInt(cv.match(/\\d+/)?.[0]||'0');
|
||||
if (!isNaN(am) && !isNaN(cm)) return desc ? cm-am : am-cm;
|
||||
}}
|
||||
return desc ? av.localeCompare(cv) : cv.localeCompare(av);
|
||||
}});
|
||||
rows.forEach(r => b.appendChild(r));
|
||||
let arrows = document.querySelectorAll('.ar');
|
||||
arrows.forEach(a => a.innerHTML = '');
|
||||
document.querySelectorAll('th')[col+1].querySelector('.ar').innerHTML = desc ? '▲' : '▼';
|
||||
}}
|
||||
|
||||
function filterTags(el, v) {{
|
||||
let tgs = el.querySelectorAll('.tg');
|
||||
if (!v || tgs.length === 0) {{
|
||||
tgs.forEach(tg => tg.style.display = '');
|
||||
return;
|
||||
}}
|
||||
let match = false;
|
||||
tgs.forEach(tg => {{
|
||||
if (tg.textContent.toLowerCase().includes(v)) {{
|
||||
tg.style.display = '';
|
||||
match = true;
|
||||
}} else {{
|
||||
tg.style.display = 'none';
|
||||
}}
|
||||
}});
|
||||
if (!match) tgs.forEach(tg => tg.style.display = '');
|
||||
}}
|
||||
|
||||
function t(id,el){{
|
||||
let sp = document.getElementById(id);
|
||||
let tr = el.closest('tr');
|
||||
let nxt = tr.nextElementSibling;
|
||||
if (nxt && nxt.classList.contains('dr')){{
|
||||
nxt.remove();
|
||||
el.innerHTML = el.getAttribute('data-n') + ' more…';
|
||||
}} else {{
|
||||
let dr = document.createElement('tr');
|
||||
dr.className = 'dr';
|
||||
let td = document.createElement('td');
|
||||
td.colSpan = 5;
|
||||
td.innerHTML = sp.innerHTML;
|
||||
dr.appendChild(td);
|
||||
tr.after(dr);
|
||||
el.innerHTML = 'close ×';
|
||||
}}
|
||||
}}
|
||||
|
||||
function highlight(el, str) {{
|
||||
if (!str) {{
|
||||
el.childNodes.forEach(c => {{
|
||||
if (c.nodeType === 3) return;
|
||||
if (c.classList) c.classList.remove('hl');
|
||||
if (c.querySelectorAll) c.querySelectorAll('.hl').forEach(h => h.classList.remove('hl'));
|
||||
}});
|
||||
return;
|
||||
}}
|
||||
el.childNodes.forEach(c => {{
|
||||
if (c.nodeType === 3 && c.textContent.trim()) {{
|
||||
let idx = c.textContent.toLowerCase().indexOf(str);
|
||||
if (idx === -1) return;
|
||||
let span = document.createElement('span');
|
||||
let before = c.textContent.slice(0, idx);
|
||||
let match = c.textContent.slice(idx, idx + str.length);
|
||||
let after = c.textContent.slice(idx + str.length);
|
||||
let mk = document.createElement('mark');
|
||||
mk.className = 'hl';
|
||||
mk.textContent = match;
|
||||
if (before) span.appendChild(document.createTextNode(before));
|
||||
span.appendChild(mk);
|
||||
if (after) span.appendChild(document.createTextNode(after));
|
||||
el.replaceChild(span, c);
|
||||
}} else if (c.nodeType === 1) {{
|
||||
highlight(c, str);
|
||||
}}
|
||||
}});
|
||||
}}
|
||||
|
||||
let _qt;
|
||||
q.addEventListener('input', ()=>{{
|
||||
clearTimeout(_qt);
|
||||
_qt = setTimeout(()=>{{
|
||||
let v = q.value.toLowerCase();
|
||||
let rows = b.children;
|
||||
let vis = 0;
|
||||
document.querySelectorAll('.hl').forEach(h => {{
|
||||
let p = h.parentNode;
|
||||
let txt = document.createTextNode(h.textContent);
|
||||
p.replaceChild(txt, h);
|
||||
p.normalize();
|
||||
}});
|
||||
for (let r of rows) {{
|
||||
if (r.classList.contains('dr')) continue;
|
||||
let txt = r.textContent.toLowerCase();
|
||||
let ht = r.querySelector('.ht');
|
||||
if (ht) txt += ' ' + ht.textContent.toLowerCase();
|
||||
let ok = txt.includes(v);
|
||||
r.style.display = ok ? '' : 'none';
|
||||
filterTags(r, v);
|
||||
let nxt = r.nextElementSibling;
|
||||
if (nxt && nxt.classList.contains('dr')) {{
|
||||
nxt.style.display = ok ? '' : 'none';
|
||||
filterTags(nxt, v);
|
||||
}}
|
||||
if (ok && v) highlight(r, v);
|
||||
if (ok) vis++;
|
||||
}}
|
||||
cnt.textContent = vis + ' / ' + rows.length;
|
||||
vc.textContent = vis;
|
||||
}}, 700);
|
||||
}});
|
||||
</script>
|
||||
</body></html>
|
||||
"""
|
||||
|
||||
return html
|
||||
|
||||
|
||||
def _e(s):
|
||||
"""Escape HTML entities."""
|
||||
s = str(s)
|
||||
s = s.replace("&", "&")
|
||||
s = s.replace("<", "<")
|
||||
s = s.replace(">", ">")
|
||||
s = s.replace('"', """)
|
||||
return s
|
||||
|
||||
|
||||
def main():
|
||||
args = sys.argv[1:]
|
||||
|
||||
if len(args) >= 1:
|
||||
target = args[0]
|
||||
else:
|
||||
target = os.path.expanduser("~/mnt/nimo.loc/ComfyUI/models/loras")
|
||||
|
||||
target = os.path.abspath(os.path.expanduser(target))
|
||||
|
||||
if not os.path.isdir(target):
|
||||
print(f"Error: {target} is not a directory", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
safetensors = sorted([
|
||||
f for f in os.listdir(target)
|
||||
if f.endswith(".safetensors") and not f.startswith("._")
|
||||
])
|
||||
|
||||
if not safetensors:
|
||||
print(f"No .safetensors files found in {target}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
loras = []
|
||||
for fname in safetensors:
|
||||
fpath = os.path.join(target, fname)
|
||||
meta = read_safetensors_meta(fpath)
|
||||
loras.append((fname, meta))
|
||||
|
||||
# Look up source URLs with disk-backed cache
|
||||
cache = load_source_cache()
|
||||
sources = {}
|
||||
total = len(loras)
|
||||
for i, (fname, meta) in enumerate(loras, 1):
|
||||
fpath = os.path.join(target, fname)
|
||||
url, label = lookup_source(fpath, fname, cache, meta)
|
||||
if url:
|
||||
sources[fname] = (url, label)
|
||||
if i % 10 == 0 or i == total:
|
||||
print(f" source lookup: {i}/{total}", file=sys.stderr)
|
||||
save_source_cache(cache)
|
||||
|
||||
html = generate_html(loras, sources)
|
||||
|
||||
if len(args) >= 2:
|
||||
out_path = os.path.abspath(os.path.expanduser(args[1]))
|
||||
else:
|
||||
out_path = os.path.join(os.path.dirname(os.path.abspath(__file__)), "lora-info-sheet.html")
|
||||
|
||||
with open(out_path, "w") as f:
|
||||
f.write(html)
|
||||
|
||||
out_count = sum(1 for v in sources.values() if v[0])
|
||||
print(f"Generated {out_path} with {len(loras)} LoRA entries ({out_count} with source links)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
294
organize-images
294
organize-images
@@ -1,23 +1,34 @@
|
||||
#!/bin/bash
|
||||
# organize-images -- Sort images into AI/, Photos/, Screenshots/, Unsorted/
|
||||
# Default: sub-categorizes AI by model + LoRA combo (uses extract-ai-meta).
|
||||
# Use --flat for one flat AI/ folder.
|
||||
# AI files route into AI/{model}/{lora}/ via extract-ai-meta (canonical scheme).
|
||||
#
|
||||
# Modes:
|
||||
# (default) Sort top-level files of [path] into the 4 buckets.
|
||||
# --resort Recursively re-sort [path] (the Images root containing AI/),
|
||||
# moving any file with ComfyUI metadata to its canonical
|
||||
# AI/{model}/{lora}/ location. Filenames are never changed.
|
||||
#
|
||||
# Generic ComfyUI_* filenames are renamed first via rename-ai-snaps
|
||||
# (--only-generic) before sorting, when that script is available.
|
||||
|
||||
TARGET=""
|
||||
DRY_RUN=true
|
||||
VERBOSE=false
|
||||
INSTALL=false
|
||||
FLAT=false
|
||||
RESORT=false
|
||||
LIMIT=0
|
||||
|
||||
usage() {
|
||||
echo "Usage: $(basename "$0") [path] [-e|--execute] [-v|--verbose] [-i|--install] [--flat]"
|
||||
echo "Usage: $(basename "$0") [path] [-e|--execute] [-v|--verbose] [-i|--install] [--flat] [--resort] [--limit N]"
|
||||
echo " path Directory to organize (default: ~/Pictures)"
|
||||
echo " -e Execute moves (default: dry-run only)"
|
||||
echo " -v Verbose: show per-file classification"
|
||||
echo " -v Verbose: show per-file classification/moves"
|
||||
echo " -i Install this script to ~/.local/bin/"
|
||||
echo " --flat Don't sub-categorize by model/LoRA (flat AI/ folder)"
|
||||
echo " --limit N Only process N files (for batching)"
|
||||
echo " --resort Recursively re-sort [path] into canonical AI/{model}/{lora}/"
|
||||
echo " ([path] must be the Images root containing AI/)"
|
||||
echo " --limit N Only process N files (for batching; default mode only)"
|
||||
exit 0
|
||||
}
|
||||
|
||||
@@ -34,6 +45,7 @@ for arg in "$@"; do
|
||||
-v|--verbose) VERBOSE=true ;;
|
||||
-i|--install) INSTALL=true ;;
|
||||
--flat) FLAT=true ;;
|
||||
--resort) RESORT=true ;;
|
||||
--limit) LIMIT_NEXT=true ;;
|
||||
--limit=*) LIMIT="${arg#*=}" ;;
|
||||
-h|--help) usage ;;
|
||||
@@ -63,34 +75,223 @@ if [ ! -d "$TARGET" ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "organize-images — Classifying images in: $TARGET"
|
||||
[ "$DRY_RUN" = true ] && echo " DRY RUN (use -e to execute)" || echo " EXECUTING"
|
||||
[ "$LIMIT" -gt 0 ] && echo " Limit: first $LIMIT files"
|
||||
# --- Resolve helper scripts (same dir as this script, else PATH) ---
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
find_tool() {
|
||||
local n="$1"
|
||||
if [ -x "$SCRIPT_DIR/$n" ]; then
|
||||
echo "$SCRIPT_DIR/$n"
|
||||
elif command -v "$n" >/dev/null 2>&1; then
|
||||
command -v "$n"
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# --- Pre-scan: extract AI metadata for model/LoRA routing ---
|
||||
META_SCRIPT="$(find_tool extract-ai-meta)" || {
|
||||
echo "Error: extract-ai-meta not found (looked next to $0 and in PATH)" >&2
|
||||
exit 1
|
||||
}
|
||||
RENAME_SCRIPT="$(find_tool rename-ai-snaps || true)"
|
||||
|
||||
echo "organize-images — $([ "$RESORT" = true ] && echo "Re-sorting" || echo "Classifying") images in: $TARGET"
|
||||
[ "$DRY_RUN" = true ] && echo " DRY RUN (use -e to execute)" || echo " EXECUTING"
|
||||
[ "$LIMIT" -gt 0 ] && [ "$RESORT" = false ] && echo " Limit: first $LIMIT files"
|
||||
|
||||
# --- Resort mode: recursive canonical re-sort of an existing tree ---
|
||||
if [ "$RESORT" = true ]; then
|
||||
if [ ! -d "$TARGET/AI" ]; then
|
||||
echo "Error: --resort expects the Images root (a directory containing AI/)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Generic-name hook: rename ComfyUI_* files in place before sorting
|
||||
GENERIC_COUNT=$(find "$TARGET" -type f -name 'ComfyUI*.png' 2>/dev/null | wc -l | tr -d ' ')
|
||||
if [ "$GENERIC_COUNT" -gt 0 ]; then
|
||||
if [ -n "$RENAME_SCRIPT" ]; then
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
echo " $GENERIC_COUNT generic ComfyUI_*.png file(s) would be renamed first (rename-ai-snaps --only-generic)"
|
||||
else
|
||||
echo " Renaming $GENERIC_COUNT generic ComfyUI_*.png file(s) first..."
|
||||
"$RENAME_SCRIPT" "$TARGET" -r -n --only-generic
|
||||
fi
|
||||
else
|
||||
echo " Note: $GENERIC_COUNT generic ComfyUI_*.png file(s) found; rename-ai-snaps not available, keeping names"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Metadata manifest (recursive)
|
||||
echo " Scanning AI metadata (recursive)..."
|
||||
AI_META=$(mktemp)
|
||||
if ! "$META_SCRIPT" "$TARGET" -r > "$AI_META"; then
|
||||
echo "Error: AI metadata scan failed" >&2
|
||||
rm -f "$AI_META"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 - "$TARGET" "$AI_META" "$DRY_RUN" "$VERBOSE" <<'PYEOF'
|
||||
import json, os, shutil, sys
|
||||
|
||||
root = os.path.abspath(sys.argv[1])
|
||||
meta = json.load(open(sys.argv[2]))
|
||||
dry = sys.argv[3] == "true"
|
||||
verbose = sys.argv[4] == "true"
|
||||
|
||||
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
LEGACY_DIRS = {"None", "unknown"}
|
||||
|
||||
moved = ok = nometa = collisions = errors = 0
|
||||
samples = []
|
||||
legacy_no_meta = []
|
||||
planned = set()
|
||||
|
||||
for dirpath, dirnames, filenames in os.walk(root):
|
||||
dirnames.sort()
|
||||
for fn in sorted(filenames):
|
||||
if os.path.splitext(fn)[1].lower() not in IMAGE_EXTS:
|
||||
continue
|
||||
src = os.path.join(dirpath, fn)
|
||||
rel = os.path.relpath(src, root)
|
||||
entry = meta.get(rel)
|
||||
if not entry:
|
||||
nometa += 1
|
||||
top = rel.split(os.sep)[0]
|
||||
if top in LEGACY_DIRS:
|
||||
legacy_no_meta.append(rel)
|
||||
continue
|
||||
|
||||
dest_dir = os.path.join(root, "AI", entry["dest"])
|
||||
if os.path.abspath(dest_dir) == os.path.abspath(dirpath):
|
||||
ok += 1
|
||||
continue
|
||||
|
||||
dest = os.path.join(dest_dir, fn)
|
||||
taken = planned if dry else None
|
||||
if os.path.exists(dest) or (taken is not None and dest in taken):
|
||||
stem, ext = os.path.splitext(fn)
|
||||
c = 1
|
||||
while os.path.exists(os.path.join(dest_dir, f"{stem}_{c}{ext}")) or \
|
||||
(taken is not None and os.path.join(dest_dir, f"{stem}_{c}{ext}") in taken):
|
||||
c += 1
|
||||
dest = os.path.join(dest_dir, f"{stem}_{c}{ext}")
|
||||
collisions += 1
|
||||
if taken is not None:
|
||||
taken.add(dest)
|
||||
|
||||
dest_rel = os.path.relpath(dest, root)
|
||||
if dry:
|
||||
moved += 1
|
||||
if len(samples) < 20:
|
||||
samples.append((rel, dest_rel))
|
||||
if verbose:
|
||||
print(f" WOULD MOVE {rel} → {dest_rel}")
|
||||
else:
|
||||
try:
|
||||
os.makedirs(dest_dir, exist_ok=True)
|
||||
shutil.move(src, dest)
|
||||
moved += 1
|
||||
if verbose:
|
||||
print(f" MOVED {rel} → {dest_rel}")
|
||||
elif moved % 200 == 0:
|
||||
print(f"\r …moved {moved}", end="", flush=True)
|
||||
except OSError as e:
|
||||
errors += 1
|
||||
print(f" SKIPPED ({e}): {rel}")
|
||||
|
||||
if not verbose and not dry and moved:
|
||||
print(f"\r Moved {moved} file(s). ")
|
||||
|
||||
# Prune dirs left empty by the moves (keep top-level buckets)
|
||||
pruned = 0
|
||||
if not dry and moved:
|
||||
keep = {"AI", "Photos", "Screenshots", "Unsorted"}
|
||||
for dirpath, dirnames, filenames in os.walk(root, topdown=False):
|
||||
if dirpath == root:
|
||||
continue
|
||||
if os.path.relpath(dirpath, root).split(os.sep)[0] in keep and \
|
||||
os.path.dirname(dirpath) == root:
|
||||
continue
|
||||
try:
|
||||
os.rmdir(dirpath) # only succeeds when empty
|
||||
pruned += 1
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
print()
|
||||
print(f" {'DRY RUN — would move' if dry else 'Moved'}: {moved}")
|
||||
print(f" Already correctly placed: {ok}")
|
||||
print(f" Collisions (renamed with _N suffix): {collisions}")
|
||||
if errors:
|
||||
print(f" Errors: {errors}")
|
||||
print(f" No AI metadata (left in place): {nometa}")
|
||||
if pruned:
|
||||
print(f" Empty folders pruned: {pruned}")
|
||||
|
||||
if samples:
|
||||
print()
|
||||
print(" Sample moves:")
|
||||
for s, d in samples:
|
||||
print(f" {s}")
|
||||
print(f" → {d}")
|
||||
|
||||
if legacy_no_meta:
|
||||
print()
|
||||
print(f" Note: {len(legacy_no_meta)} file(s) in None//unknown/ have no AI metadata")
|
||||
print(" and were left in place for manual triage (not all are AI images).")
|
||||
for rel in legacy_no_meta[:10]:
|
||||
print(f" {rel}")
|
||||
if len(legacy_no_meta) > 10:
|
||||
print(f" … and {len(legacy_no_meta) - 10} more")
|
||||
print()
|
||||
PYEOF
|
||||
STATUS=$?
|
||||
rm -f "$AI_META"
|
||||
exit $STATUS
|
||||
fi
|
||||
|
||||
# --- Default mode: classify top-level files into AI/Photos/Screenshots/Unsorted ---
|
||||
|
||||
# Generic-name hook (non-recursive)
|
||||
GENERIC_COUNT=$(find "$TARGET" -maxdepth 1 -type f -name 'ComfyUI*.png' 2>/dev/null | wc -l | tr -d ' ')
|
||||
if [ "$GENERIC_COUNT" -gt 0 ] && [ -n "$RENAME_SCRIPT" ]; then
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
echo " $GENERIC_COUNT generic ComfyUI_*.png file(s) would be renamed first (rename-ai-snaps --only-generic)"
|
||||
else
|
||||
echo " Renaming $GENERIC_COUNT generic ComfyUI_*.png file(s) first..."
|
||||
"$RENAME_SCRIPT" "$TARGET" -n --only-generic
|
||||
fi
|
||||
fi
|
||||
|
||||
# Pre-scan: extract AI metadata for model/LoRA routing
|
||||
AI_META=""
|
||||
if [ "$FLAT" = false ]; then
|
||||
META_SCRIPT="$(cd "$(dirname "$0")" && pwd -P)/extract-ai-meta"
|
||||
if [ -x "$META_SCRIPT" ]; then
|
||||
echo " Scanning AI metadata for model/LoRA routing..."
|
||||
AI_META=$(mktemp)
|
||||
if ! "$META_SCRIPT" "$TARGET" > "$AI_META" 2>/dev/null; then
|
||||
echo " Warning: AI metadata scan failed, falling back to flat AI/"
|
||||
FLAT=true
|
||||
rm -f "$AI_META"
|
||||
AI_META=""
|
||||
elif [ ! -s "$AI_META" ]; then
|
||||
FLAT=true
|
||||
rm -f "$AI_META"
|
||||
fi
|
||||
else
|
||||
echo " Warning: extract-ai-meta not found, falling back to flat AI/"
|
||||
FLAT=true
|
||||
AI_META=""
|
||||
fi
|
||||
fi
|
||||
|
||||
[ "$FLAT" = false ] && echo " AI sub-categorization: by model/LoRA (use --flat to disable)"
|
||||
echo
|
||||
|
||||
# Basenames of files with AI metadata (manifest presence = is an AI image)
|
||||
AI_KEYS=""
|
||||
if [ -n "$AI_META" ]; then
|
||||
AI_KEYS=$(mktemp)
|
||||
python3 -c "
|
||||
import json, sys
|
||||
for k in json.load(open('$AI_META')):
|
||||
print(k)
|
||||
" > "$AI_KEYS" 2>/dev/null
|
||||
fi
|
||||
|
||||
classify() {
|
||||
local f="$1"
|
||||
|
||||
@@ -110,6 +311,11 @@ classify() {
|
||||
ComfyUI_*|*_DEFI_*|R.*.png|schmeeve-AI-*) echo "AI"; return ;;
|
||||
esac
|
||||
|
||||
# Files with ComfyUI metadata are AI regardless of name
|
||||
if [ -n "$AI_KEYS" ] && grep -Fxq "$name" "$AI_KEYS" 2>/dev/null; then
|
||||
echo "AI"; return
|
||||
fi
|
||||
|
||||
if mdls -name kMDItemAcquisitionMake "$f" 2>/dev/null | grep -q '= "'; then
|
||||
echo "Photos"; return
|
||||
fi
|
||||
@@ -121,24 +327,10 @@ classify() {
|
||||
echo "Photos"; return ;;
|
||||
esac
|
||||
|
||||
if [ "$ext" = "png" ]; then
|
||||
local dims w h area ratio
|
||||
dims=$(sips --getProperty pixelWidth --getProperty pixelHeight "$f" 2>/dev/null | grep -E 'pixel(Width|Height)' | awk '{print $NF}')
|
||||
w=$(echo "$dims" | head -1)
|
||||
h=$(echo "$dims" | tail -1)
|
||||
if [ -n "$w" ] && [ -n "$h" ] && [ "$w" -gt 0 ] && [ "$h" -gt 0 ]; then
|
||||
area=$((w * h))
|
||||
ratio=$(echo "scale=4; $w / $h" | bc 2>/dev/null || echo 1)
|
||||
if [ "$area" -gt 1048576 ] && [ "$(echo "$ratio > 0.75" | bc -l)" = 1 ] && [ "$(echo "$ratio < 1.33" | bc -l)" = 1 ]; then
|
||||
echo "AI"; return
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Unsorted"
|
||||
}
|
||||
|
||||
# --- Pre-scan: collect all image files ---
|
||||
# --- Collect top-level image files ---
|
||||
FILES=()
|
||||
for f in "$TARGET"/*; do
|
||||
[ -f "$f" ] || continue
|
||||
@@ -155,6 +347,8 @@ if [ "$LIMIT" -gt 0 ] && [ "$LIMIT" -lt "$TOTAL" ]; then
|
||||
fi
|
||||
if [ "$TOTAL" -eq 0 ]; then
|
||||
echo " No image files found."
|
||||
[ -n "$AI_META" ] && rm -f "$AI_META"
|
||||
[ -n "$AI_KEYS" ] && rm -f "$AI_KEYS"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -174,27 +368,16 @@ progress_bar() {
|
||||
printf "] %3d%% (%d/%d)" "$pct" "$current" "$total"
|
||||
}
|
||||
|
||||
# Destination for an AI file: canonical AI/{model}/{lora}/ from the manifest
|
||||
ai_dest() {
|
||||
python3 -c "
|
||||
import json, sys
|
||||
m = json.load(open('$AI_META'))
|
||||
AI_META_PATH="$AI_META" python3 -c "
|
||||
import json, os, sys
|
||||
m = json.load(open(os.environ['AI_META_PATH']))
|
||||
entry = m.get(sys.argv[1])
|
||||
if not entry or not isinstance(entry, dict):
|
||||
if not entry or not isinstance(entry, dict) or not entry.get('dest'):
|
||||
print('AI/unknown')
|
||||
sys.exit(0)
|
||||
def clean(s):
|
||||
if not s:
|
||||
return 'unknown'
|
||||
return s.replace('/', '-').replace('\\\\', '-')
|
||||
|
||||
model = clean(entry.get('model'))
|
||||
loras = entry.get('loras') or []
|
||||
if loras:
|
||||
combo = '+'.join(clean(l.strip()) for l in loras if l)
|
||||
combo = (combo[:200]).rstrip('+')
|
||||
print(f'AI/{model}/{combo}')
|
||||
else:
|
||||
print(f'AI/{model}/no-lora')
|
||||
print('AI/' + entry['dest'])
|
||||
" "$1"
|
||||
}
|
||||
|
||||
@@ -224,7 +407,15 @@ for f in "${FILES[@]}"; do
|
||||
|
||||
if [ "$DRY_RUN" = false ]; then
|
||||
mkdir -p "$dest"
|
||||
mv -f "$f" "$dest"
|
||||
base=$(basename "$f")
|
||||
final="$dest/$base"
|
||||
if [ -e "$final" ]; then
|
||||
stem="${base%.*}"; fext="${base##*.}"
|
||||
n=1
|
||||
while [ -e "$dest/${stem}_${n}.${fext}" ]; do n=$((n + 1)); done
|
||||
final="$dest/${stem}_${n}.${fext}"
|
||||
fi
|
||||
mv "$f" "$final"
|
||||
fi
|
||||
|
||||
case "$cat" in
|
||||
@@ -235,13 +426,16 @@ for f in "${FILES[@]}"; do
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "$AI_META" ] && rm -f "$AI_META"
|
||||
[ -n "$AI_KEYS" ] && rm -f "$AI_KEYS"
|
||||
|
||||
if [ "$VERBOSE" = false ]; then
|
||||
printf "\r %-60s\n" "Done."
|
||||
fi
|
||||
|
||||
echo
|
||||
echo " Results:"
|
||||
if [ "$FLAT" = false ] && [ -n "$AI_META" ]; then
|
||||
if [ "$FLAT" = false ] && [ "$cAI" -gt 0 ]; then
|
||||
echo " AI (by model/LoRA): $cAI"
|
||||
else
|
||||
echo " AI: $cAI"
|
||||
|
||||
@@ -105,7 +105,7 @@ done
|
||||
echo -e "\n\n\n"
|
||||
echo "** Quitting Applications **"
|
||||
|
||||
prg=( "Zen Browser" "Opera" "Opera" "Opera" "Opera" "Opera" "MenubarX" "App Store" "WhatsApp" ".Agenda" "Steam" ".Loopback" ".Slack" "Boom 3D" "Paw" "DYMO Label" "QuickTime Player" "System Information" "Permute" "Activity Monitor" ".Transmit" ".Mail" ".Edison Mail" ".Airmail" "Setapp" ".Messages" "News" "zoom.us" "Preview" "Arq" "Carbon Copy Cloner" "Safari" "Vivaldi" "Firefox" "firefox" "Music" "Microsoft Edge" ".Parallels Desktop" ".Paw" ".Creative Cloud" "Things" ".Microsoft Word" ".Microsoft Excel" ".Microsoft PowerPoint" ".Xcode" "Simulator" "Audio Hijack" "Microsoft AutoUpdate" ".ToothFairy" "Creative Cloud" "Arc" "Brave Browser" "Safari Technology Preview" "YouTube" "SiriusXM" "Xcode.app" ".Xcode-beta" "Simulator" "Podcasts" "Microsoft Remote Desktop" "zen" "RustDesk")
|
||||
prg=( "Zen Browser" "Opera" "Opera" "Opera" "Opera" "Opera" "Brave Browser" "MenubarX" "App Store" "WhatsApp" ".Agenda" "Steam" ".Loopback" ".Slack" "Boom 3D" "Paw" "DYMO Label" "QuickTime Player" "System Information" "Permute" "Activity Monitor" ".Transmit" ".Mail" ".Edison Mail" ".Airmail" "Setapp" ".Messages" "News" "zoom.us" "Preview" "Arq" "Carbon Copy Cloner" "Safari" "Vivaldi" "Firefox" "firefox" "Music" "Microsoft Edge" ".Parallels Desktop" ".Paw" ".Creative Cloud" "Things" ".Microsoft Word" ".Microsoft Excel" ".Microsoft PowerPoint" ".Xcode" "Simulator" "Audio Hijack" "Microsoft AutoUpdate" ".ToothFairy" "Creative Cloud" "Arc" "Brave Browser" "Safari Technology Preview" "YouTube" "SiriusXM" "Xcode.app" ".Xcode-beta" "Simulator" "Podcasts" "Microsoft Remote Desktop" "zen" "RustDesk")
|
||||
#prg=( "Safari Technology Preview" )
|
||||
killprg=( "runningboardd" "Brave Browser" "Microsoft Edge" )
|
||||
for app in "${prg[@]}"
|
||||
|
||||
9
push-newer-pics
Executable file
9
push-newer-pics
Executable file
@@ -0,0 +1,9 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# push-newer-pics — Wrapper for sync-newer-images with default paths.
|
||||
# Copies new images/videos from ~/Pictures → ~/mnt/mini.nas/miniShare1/Pictures
|
||||
#
|
||||
# Usage: push-newer-pics [--help] [-u|--update]
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
exec "${SCRIPT_DIR}/sync-newer-images" "$@" "$HOME/mnt/mini.nas/miniShare1/Pictures" "$HOME/Pictures"
|
||||
5
quit-app.app/Contents/Resources/description.rtfd/TXT.rtf
Normal file
5
quit-app.app/Contents/Resources/description.rtfd/TXT.rtf
Normal file
@@ -0,0 +1,5 @@
|
||||
{\rtf1\ansi\ansicpg1252\cocoartf2709
|
||||
\cocoatextscaling0\cocoaplatform0{\fonttbl}
|
||||
{\colortbl;\red255\green255\blue255;}
|
||||
{\*\expandedcolortbl;;}
|
||||
}
|
||||
257
rename-ai-snaps
257
rename-ai-snaps
@@ -1,13 +1,15 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
rename-ai-snaps — Scan PNGs for AI prompt metadata and propose descriptive filenames.
|
||||
rename-ai-snaps — Scan PNGs for AI prompt metadata and rename descriptively.
|
||||
|
||||
Usage:
|
||||
./rename-ai-snaps [path] [--no-interactive]
|
||||
./rename-ai-snaps [path] [options]
|
||||
|
||||
Scans directory (default ~/Pictures) for ComfyUI PNGs, reads their embedded prompt,
|
||||
extracts short keyword descriptions, and interactively renames them to:
|
||||
Scans for ComfyUI PNGs, reads embedded prompt metadata, extracts keywords,
|
||||
and renames files IN PLACE to:
|
||||
schmeeve-AI-{keywords}.png
|
||||
|
||||
Renaming only — use organize-images to sort files into model/lora folders.
|
||||
"""
|
||||
|
||||
import json
|
||||
@@ -84,7 +86,10 @@ NEGATIVE_INDICATORS = {
|
||||
"body out of frame", "bad hands", "bad face", "blurry",
|
||||
}
|
||||
|
||||
# Generic ComfyUI output filenames (ComfyUI_00042_.png, ComfyUI_temp_….png)
|
||||
GENERIC_RE = re.compile(r"^ComfyUI", re.IGNORECASE)
|
||||
|
||||
# ── Helpers ────────────────────────────────────────────────────────────────
|
||||
|
||||
def spinner():
|
||||
chars = "⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏"
|
||||
@@ -95,7 +100,6 @@ def spinner():
|
||||
|
||||
|
||||
def is_negative_text(text):
|
||||
"""Heuristic: is this text block a negative prompt?"""
|
||||
lower = text.lower()
|
||||
score = 0
|
||||
for ind in NEGATIVE_INDICATORS:
|
||||
@@ -105,9 +109,7 @@ def is_negative_text(text):
|
||||
|
||||
|
||||
def is_quality_only(text):
|
||||
"""Heuristic: does this text block contain only quality/technical tags?"""
|
||||
lower = text.lower()
|
||||
# Split into words, strip weighting syntax
|
||||
words = re.findall(r"[a-z_]+", lower)
|
||||
if not words:
|
||||
return False
|
||||
@@ -115,39 +117,42 @@ def is_quality_only(text):
|
||||
return meaningful == 0
|
||||
|
||||
|
||||
def extract_prompts(filepath):
|
||||
"""Return (positive_prompt, negative_prompt) from a ComfyUI PNG."""
|
||||
# ── Metadata extraction from ComfyUI prompt JSON ───────────────────────────
|
||||
|
||||
def load_prompt_data(filepath):
|
||||
"""Return parsed ComfyUI prompt JSON from a PNG, or None."""
|
||||
try:
|
||||
img = Image.open(filepath)
|
||||
except Exception:
|
||||
return None, None
|
||||
return None
|
||||
|
||||
if "prompt" not in img.info:
|
||||
return None, None
|
||||
return None
|
||||
|
||||
try:
|
||||
data = json.loads(img.info["prompt"])
|
||||
return json.loads(img.info["prompt"])
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None, None
|
||||
return None
|
||||
|
||||
|
||||
def extract_prompts(data):
|
||||
"""Return (positive_prompt, negative_prompt) from parsed prompt JSON."""
|
||||
# Gather all text fields from all nodes
|
||||
candidates = []
|
||||
for node in data.values():
|
||||
inputs = node.get("inputs", {})
|
||||
# Check all common text-carrying fields
|
||||
for field in ("text", "prompt", "positive", "negative", "value", "string"):
|
||||
val = inputs.get(field, "")
|
||||
if isinstance(val, str) and len(val.strip()) > 3:
|
||||
candidates.append(val.strip())
|
||||
|
||||
# Second pass: resolve node references (e.g. ["node_id", 0])
|
||||
# Second pass: resolve node references
|
||||
for node in data.values():
|
||||
inputs = node.get("inputs", {})
|
||||
for field in ("text", "prompt", "positive", "negative", "value", "string"):
|
||||
val = inputs.get(field)
|
||||
if isinstance(val, list) and len(val) == 2 and isinstance(val[0], str):
|
||||
ref_node = data.get(val[0], {})
|
||||
# Check if the referenced node has a 'value' or 'text' field
|
||||
ref_inputs = ref_node.get("inputs", {})
|
||||
for rf in ("value", "text", "string"):
|
||||
rv = ref_inputs.get(rf, "")
|
||||
@@ -158,11 +163,9 @@ def extract_prompts(filepath):
|
||||
if not candidates:
|
||||
return None, None
|
||||
|
||||
# Split into positive vs negative
|
||||
positives = [c for c in candidates if not is_negative_text(c)]
|
||||
negatives = [c for c in candidates if is_negative_text(c)]
|
||||
|
||||
# Further filter: quality-only texts are not useful for naming
|
||||
positives = [c for c in positives if not is_quality_only(c)]
|
||||
|
||||
pos = max(positives, key=len) if positives else None
|
||||
@@ -176,14 +179,9 @@ def extract_all_candidates(text):
|
||||
return []
|
||||
|
||||
text_lower = text.lower()
|
||||
|
||||
# Strip weighting/parenthetical syntax: (word:1.2), [word], {word}, (word)
|
||||
cleaned = re.sub(r"[\[\(\{][^\]\)\}]*[\]\)\}]", "", text_lower)
|
||||
|
||||
# Split on commas, periods, semicolons, exclamation/question marks
|
||||
segments = re.split(r"[,.;:!?]+", cleaned)
|
||||
|
||||
# Collect meaningful keywords, preserving position order
|
||||
seen = set()
|
||||
candidates = []
|
||||
position = 0
|
||||
@@ -193,12 +191,10 @@ def extract_all_candidates(text):
|
||||
if not seg or len(seg) < 4:
|
||||
continue
|
||||
|
||||
# Extract individual words from segment
|
||||
words = re.findall(r"[a-zA-Z_]+", seg)
|
||||
good = []
|
||||
for w in words:
|
||||
wl = w.lower().strip("_")
|
||||
# Skip short words, stop words, quality tags, negative indicators
|
||||
if len(wl) < 3:
|
||||
continue
|
||||
if wl in STOP_WORDS or wl in GENERIC_WORDS:
|
||||
@@ -216,7 +212,6 @@ def extract_all_candidates(text):
|
||||
good.append(wl)
|
||||
|
||||
if good:
|
||||
# Take up to 2 unseen keywords from this segment
|
||||
taken = 0
|
||||
for g in good:
|
||||
if g not in seen and taken < 2:
|
||||
@@ -229,12 +224,6 @@ def extract_all_candidates(text):
|
||||
|
||||
|
||||
def select_best_keywords(candidates, freq_map, total_images, max_keywords=5):
|
||||
"""Select most discriminating keywords by rarity and position.
|
||||
|
||||
Keywords that appear in fewer images across the batch score higher.
|
||||
Later-position keywords get a small bonus for tiebreaking.
|
||||
When total_images <= 1, falls back to first-N selection.
|
||||
"""
|
||||
if not candidates:
|
||||
return []
|
||||
|
||||
@@ -244,15 +233,12 @@ def select_best_keywords(candidates, freq_map, total_images, max_keywords=5):
|
||||
scored = []
|
||||
for kw, pos in candidates:
|
||||
freq = freq_map.get(kw, 1)
|
||||
# Rarity: 1.0 = appears in 1 image, 0.0 = appears in all images
|
||||
rarity = 1.0 - ((freq - 1) / max(total_images - 1, 1))
|
||||
# Position bonus: slight preference for later words (up to 0.3)
|
||||
max_pos = max(len(candidates), 1)
|
||||
pos_bonus = (pos / max_pos) * 0.3
|
||||
score = rarity + pos_bonus
|
||||
scored.append((score, kw, pos))
|
||||
|
||||
# Sort by score desc, then by original position asc for stability
|
||||
scored.sort(key=lambda x: (-x[0], x[2]))
|
||||
|
||||
selected = []
|
||||
@@ -268,7 +254,6 @@ def select_best_keywords(candidates, freq_map, total_images, max_keywords=5):
|
||||
|
||||
|
||||
def keywords_to_name(keywords):
|
||||
"""Convert keyword list to 'schmeeve-AI-{keywords}.png' name."""
|
||||
if not keywords:
|
||||
return None
|
||||
|
||||
@@ -276,7 +261,6 @@ def keywords_to_name(keywords):
|
||||
desc = re.sub(r"[^a-z0-9-]", "-", desc)
|
||||
desc = re.sub(r"-+", "-", desc).strip("-")
|
||||
|
||||
# Truncate at 40 chars, breaking at a word boundary
|
||||
if len(desc) > 40:
|
||||
desc = desc[:40].rstrip("-")
|
||||
if "-" in desc:
|
||||
@@ -287,36 +271,14 @@ def keywords_to_name(keywords):
|
||||
return f"schmeeve-AI-{desc}.png" if desc and len(desc) > 3 else None
|
||||
|
||||
|
||||
def propose_name(filepath, freq_map=None, total_images=1):
|
||||
"""Propose 'schmeeve-AI-{keywords}.png' or None.
|
||||
|
||||
When freq_map and total_images > 1 are provided, uses frequency-aware
|
||||
keyword selection that favors rare (discriminating) words.
|
||||
"""
|
||||
pos, neg = extract_prompts(filepath)
|
||||
source = pos or neg
|
||||
if not source:
|
||||
return None
|
||||
|
||||
candidates = extract_all_candidates(source)
|
||||
if not candidates:
|
||||
return None
|
||||
|
||||
if freq_map and total_images > 1:
|
||||
keywords = select_best_keywords(candidates, freq_map, total_images)
|
||||
else:
|
||||
keywords = [kw for kw, _ in candidates[:5]]
|
||||
|
||||
return keywords_to_name(keywords)
|
||||
|
||||
|
||||
# ── main ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Rename AI-generated PNGs based on embedded prompt metadata.",
|
||||
description="Scan PNGs for AI prompt metadata and rename in place. "
|
||||
"Does not move files — use organize-images for sorting.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"path", nargs="?", default=os.path.expanduser("~/Pictures"),
|
||||
@@ -326,6 +288,23 @@ def main():
|
||||
"-n", "--no-interactive", action="store_true",
|
||||
help="Auto-rename without prompting",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-r", "--recursive", action="store_true",
|
||||
help="Search directories recursively (default: off)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-p", "--preview", "--dry-run", action="store_true",
|
||||
dest="dry_run",
|
||||
help="Show proposed changes without making any",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--include-renamed", action="store_true",
|
||||
help="Also process files already named schmeeve-AI-* (default: skip them)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--only-generic", action="store_true",
|
||||
help="Only rename generic ComfyUI_* filenames",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
scan_dir = Path(args.path).expanduser().resolve()
|
||||
@@ -333,7 +312,14 @@ def main():
|
||||
print(f"Error: {scan_dir} is not a directory")
|
||||
sys.exit(1)
|
||||
|
||||
pngs = sorted(scan_dir.glob("*.png"))
|
||||
# Gather PNGs
|
||||
glob_pattern = "**/*.png" if args.recursive else "*.png"
|
||||
pngs = sorted(scan_dir.glob(glob_pattern))
|
||||
|
||||
if not pngs:
|
||||
print(f" No PNG files found in {scan_dir}" +
|
||||
(" (recursive search)" if args.recursive else ""))
|
||||
return
|
||||
|
||||
# ── Phase 1: Collect candidates and build frequency map ──
|
||||
sys.stdout.write(" Analyzing PNGs")
|
||||
@@ -344,10 +330,15 @@ def main():
|
||||
for p in pngs:
|
||||
sys.stdout.write(f"\r {next(spin)} Analyzing PNGs")
|
||||
sys.stdout.flush()
|
||||
# Skip already-renamed files
|
||||
if p.name.startswith("schmeeve-AI-"):
|
||||
# Skip already-renamed files (schmeeve-AI-*) unless requested
|
||||
if not args.include_renamed and p.name.startswith("schmeeve-AI-"):
|
||||
continue
|
||||
pos, neg = extract_prompts(str(p))
|
||||
if args.only_generic and not GENERIC_RE.match(p.name):
|
||||
continue
|
||||
data = load_prompt_data(str(p))
|
||||
if not data:
|
||||
continue
|
||||
pos, neg = extract_prompts(data)
|
||||
source = pos or neg
|
||||
if source:
|
||||
candidates = extract_all_candidates(source)
|
||||
@@ -357,12 +348,11 @@ def main():
|
||||
if kw not in word_images:
|
||||
word_images[kw] = set()
|
||||
word_images[kw].add(p)
|
||||
time.sleep(0.02)
|
||||
|
||||
freq_map = {kw: len(images) for kw, images in word_images.items()}
|
||||
total_images = len(image_data)
|
||||
|
||||
# ── Phase 1b: Propose names with frequency-aware selection ──
|
||||
# ── Phase 1b: Propose new names (in place — same directory) ──
|
||||
raw_proposals = {}
|
||||
for p, candidates in image_data.items():
|
||||
if total_images > 1:
|
||||
@@ -370,59 +360,79 @@ def main():
|
||||
else:
|
||||
keywords = [kw for kw, _ in candidates[:5]]
|
||||
name = keywords_to_name(keywords)
|
||||
if name:
|
||||
if not name or name == p.name:
|
||||
continue
|
||||
raw_proposals[p] = name
|
||||
|
||||
# Clear the spinner line
|
||||
sys.stdout.write("\r" + " " * 60 + "\r")
|
||||
sys.stdout.flush()
|
||||
|
||||
# Deduplicate proposed names; use short mtime tag for collisions
|
||||
# Deduplicate file names within their directories
|
||||
proposals = {}
|
||||
name_counts = {}
|
||||
used_paths = {}
|
||||
for p, name in raw_proposals.items():
|
||||
base = name
|
||||
if base in name_counts:
|
||||
name_counts[base] += 1
|
||||
else:
|
||||
name_counts[base] = 1
|
||||
proposals[p] = name
|
||||
|
||||
for p, name in list(proposals.items()):
|
||||
if name_counts[name] > 1:
|
||||
new_path = p.with_name(name)
|
||||
key = str(new_path)
|
||||
counter = 0
|
||||
while key in used_paths:
|
||||
counter += 1
|
||||
ts = time.strftime("%m%d-%H%M%S", time.localtime(p.stat().st_mtime))
|
||||
stem = name.rsplit(".", 1)[0]
|
||||
proposals[p] = f"{stem}-{ts}.png"
|
||||
stem = new_path.stem
|
||||
ext = new_path.suffix
|
||||
new_path = new_path.with_name(f"{stem}-{ts}{'_' + str(counter) if counter > 1 else ''}{ext}")
|
||||
key = str(new_path)
|
||||
used_paths[key] = True
|
||||
proposals[p] = new_path
|
||||
|
||||
# ── Phase 2: Display proposed names ──
|
||||
# ── Phase 2: Display proposed changes ──
|
||||
if not proposals:
|
||||
print(" No renamable PNGs found.")
|
||||
return
|
||||
|
||||
for i, (old_path, new_name) in enumerate(proposals.items(), 1):
|
||||
old = old_path.name
|
||||
stem = old_path.stem
|
||||
ext = old_path.suffix
|
||||
# Truncate old name for display
|
||||
old_display = old if len(old) < 50 else old[:22] + "…" + old[-25:]
|
||||
print(f" {i:>3}. {old_display}")
|
||||
print(f" → {new_name}")
|
||||
print(f"\n {'DRY RUN' if args.dry_run else 'PROPOSED'} — "
|
||||
f"{len(proposals)} file(s) to rename in place\n")
|
||||
|
||||
for i, (old_path, new_path) in enumerate(proposals.items(), 1):
|
||||
try:
|
||||
src_display = str(old_path.relative_to(scan_dir))
|
||||
except ValueError:
|
||||
src_display = str(old_path)
|
||||
if len(src_display) > 55:
|
||||
src_display = src_display[:25] + "…" + src_display[-27:]
|
||||
print(f" {i:>3}. {src_display}")
|
||||
print(f" → {new_path.name}")
|
||||
|
||||
print()
|
||||
print(f" {len(proposals)} file(s) to rename.\n")
|
||||
|
||||
if args.dry_run:
|
||||
print(f" Dry-run complete. No files were changed.\n")
|
||||
return
|
||||
|
||||
def do_rename(old_path, new_path):
|
||||
"""Rename with collision suffix. Returns final path or None on failure."""
|
||||
if old_path == new_path:
|
||||
return new_path
|
||||
if new_path.exists():
|
||||
stem = new_path.stem
|
||||
ext = new_path.suffix
|
||||
counter = 1
|
||||
while new_path.exists():
|
||||
new_path = new_path.with_name(f"{stem}_{counter}{ext}")
|
||||
counter += 1
|
||||
print(f" (name existed, using {new_path.name})")
|
||||
try:
|
||||
old_path.rename(new_path)
|
||||
return new_path
|
||||
except (FileNotFoundError, OSError) as e:
|
||||
print(f" SKIPPED ({e}): {old_path.name}")
|
||||
return None
|
||||
|
||||
# ── Phase 3: Rename (interactive or auto) ──
|
||||
if args.no_interactive:
|
||||
renamed = 0
|
||||
for old_path, new_name in proposals.items():
|
||||
new_path = old_path.with_name(new_name)
|
||||
if new_path.exists():
|
||||
stem = new_path.stem
|
||||
counter = 1
|
||||
while new_path.exists():
|
||||
new_path = old_path.with_name(f"{stem}_{counter}{old_path.suffix}")
|
||||
counter += 1
|
||||
old_path.rename(new_path)
|
||||
for old_path, new_path in proposals.items():
|
||||
if do_rename(old_path, new_path):
|
||||
renamed += 1
|
||||
print(f" Renamed {renamed} file(s).")
|
||||
else:
|
||||
@@ -431,13 +441,11 @@ def main():
|
||||
items = list(proposals.items())
|
||||
i = 0
|
||||
while i < len(items):
|
||||
old_path, new_name = items[i]
|
||||
old = old_path.name
|
||||
new = new_name
|
||||
old_path, new_path = items[i]
|
||||
|
||||
print(f"\n [{i+1}/{len(items)}]")
|
||||
print(f" Current: {old}")
|
||||
print(f" New: {new}")
|
||||
print(f" From: {old_path.name}")
|
||||
print(f" To: {new_path.name}")
|
||||
remaining = len(items) - i - 1
|
||||
rlabel = f"rename all {remaining}" if remaining else ""
|
||||
sys.stdout.write(f" [Enter]=rename [e]=edit [s]=skip{' [a]=' + rlabel if rlabel else ''} [q]=quit: ")
|
||||
@@ -454,41 +462,30 @@ def main():
|
||||
i += 1
|
||||
continue
|
||||
elif choice == "e":
|
||||
sys.stdout.write(f" Edit name (will be prefixed 'schmeeve-AI-'): ")
|
||||
sys.stdout.write(f" Edit keywords (name becomes 'schmeeve-AI-...'): ")
|
||||
sys.stdout.flush()
|
||||
custom = sys.stdin.readline().strip()
|
||||
if custom:
|
||||
# Sanitize
|
||||
custom_desc = re.sub(r"[^a-z0-9-]", "-", custom.lower())
|
||||
custom_desc = re.sub(r"-+", "-", custom_desc).strip("-")
|
||||
if custom_desc:
|
||||
new = f"schmeeve-AI-{custom_desc}.png"
|
||||
new_path = old_path.with_name(f"schmeeve-AI-{custom_desc}.png")
|
||||
else:
|
||||
print(" Invalid name, skipping.")
|
||||
skipped += 1
|
||||
i += 1
|
||||
continue
|
||||
else:
|
||||
# empty = skip
|
||||
skipped += 1
|
||||
i += 1
|
||||
continue
|
||||
# Fall through to rename
|
||||
elif choice == "":
|
||||
pass # rename with proposed name
|
||||
pass
|
||||
elif choice == "a":
|
||||
# Rename current and all remaining without further prompts
|
||||
for j in range(i, len(items)):
|
||||
p, n = items[j]
|
||||
np = p.with_name(n)
|
||||
if np.exists():
|
||||
stem = np.stem
|
||||
counter = 1
|
||||
while np.exists():
|
||||
np = p.with_name(f"{stem}_{counter}{p.suffix}")
|
||||
counter += 1
|
||||
p.rename(np)
|
||||
renamed += len(items) - i
|
||||
p, np = items[j]
|
||||
if do_rename(p, np):
|
||||
renamed += 1
|
||||
break
|
||||
else:
|
||||
print(f" Unknown option '{choice}', skipping.")
|
||||
@@ -496,24 +493,16 @@ def main():
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Perform rename
|
||||
new_path = old_path.with_name(new)
|
||||
if new_path.exists():
|
||||
stem = new_path.stem
|
||||
counter = 1
|
||||
while new_path.exists():
|
||||
new_path = old_path.with_name(f"{stem}_{counter}{old_path.suffix}")
|
||||
counter += 1
|
||||
print(f" (file existed, saved as {new_path.name})")
|
||||
old_path.rename(new_path)
|
||||
if do_rename(old_path, new_path):
|
||||
renamed += 1
|
||||
i += 1
|
||||
|
||||
print(f"\n Renamed: {renamed} Skipped: {skipped}")
|
||||
|
||||
# One more pass: also look at .jpg? (maybe later)
|
||||
# Check for JPEGs with AI metadata
|
||||
jpg_pattern = "**/*.jpg" if args.recursive else "*.jpg"
|
||||
remaining_ai = 0
|
||||
for f in scan_dir.glob("*.jpg"):
|
||||
for f in scan_dir.glob(jpg_pattern):
|
||||
try:
|
||||
img = Image.open(f)
|
||||
if "prompt" in img.info:
|
||||
|
||||
@@ -3,6 +3,9 @@
|
||||
# sync-comfyui-snaps — Pull new images from nimo.loc ComfyUI output, sync to NAS,
|
||||
# then rename AI snaps and organize.
|
||||
#
|
||||
# NOTE: superseded by comfyui-sync (systemd user timer on nimo.loc). Kept for
|
||||
# reference/manual use; the cron entry below is NOT currently installed.
|
||||
#
|
||||
# Crontab (every 2 hours):
|
||||
# 0 */2 * * * /home/schmeeve/.local/bin/sync-comfyui-snaps >> /home/schmeeve/.local/log/sync-comfyui-snaps.log 2>&1
|
||||
#
|
||||
@@ -21,16 +24,21 @@ NIMO_SRC="/home/schmeeve/ComfyUI/output"
|
||||
|
||||
SHARE="//mini.nas/miniShare1"
|
||||
NAS_SUBPATH="Pictures"
|
||||
MOUNTPOINT="${HOME}/mnt/miniShare1"
|
||||
CREDENTIALS="${HOME}/.smb/mini.nas"
|
||||
|
||||
RENAME_SCRIPT="${HOME}/.local/bin/rename-ai-snaps"
|
||||
ORGANIZE_SCRIPT="${HOME}/.local/bin/organize-images"
|
||||
RENAME_SCRIPT="${HOME}/git/schmeeve-toolz/rename-ai-snaps"
|
||||
ORGANIZE_SCRIPT="${HOME}/git/schmeeve-toolz/organize-images"
|
||||
|
||||
START="$(date +%s)"
|
||||
echo "[sync-comfyui-snaps] Starting at $(date)"
|
||||
|
||||
# 1. Mount NAS if not already mounted
|
||||
# 1. Determine NAS mount — use pre-mounted /mini.nas/miniShare1 on nimo.loc
|
||||
# if available; otherwise mount via CIFS under ~/mnt/miniShare1
|
||||
if [ -d "/mini.nas/miniShare1" ]; then
|
||||
MOUNTPOINT="/mini.nas/miniShare1"
|
||||
echo "[sync-comfyui-snaps] Using existing mount at ${MOUNTPOINT}"
|
||||
else
|
||||
MOUNTPOINT="${HOME}/mnt/miniShare1"
|
||||
if mount | grep -q "${MOUNTPOINT}"; then
|
||||
echo "[sync-comfyui-snaps] Already mounted at ${MOUNTPOINT}"
|
||||
else
|
||||
@@ -47,6 +55,7 @@ else
|
||||
fi
|
||||
echo "[sync-comfyui-snaps] Mounted at ${MOUNTPOINT}"
|
||||
fi
|
||||
fi
|
||||
|
||||
DEST="${MOUNTPOINT}/${NAS_SUBPATH}"
|
||||
mkdir -p "${DEST}"
|
||||
@@ -66,18 +75,26 @@ rsync -vu --checksum \
|
||||
--progress --stats \
|
||||
"${SRC}" "${DEST}/"
|
||||
|
||||
# 3. Rename AI snaps (non-interactive for automation)
|
||||
# 3. Route fresh drops into Images/ (neatcli equivalent), then rename in place.
|
||||
# NOTE: rename-ai-snaps no longer takes --output; it only renames.
|
||||
# organize-images does the model/lora sorting.
|
||||
IMAGES_DIR="${DEST}/Images"
|
||||
mkdir -p "${IMAGES_DIR}"
|
||||
find "${DEST}" -maxdepth 1 -type f \
|
||||
\( -iname '*.png' -o -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.webp' \) \
|
||||
-exec mv -n {} "${IMAGES_DIR}/" \;
|
||||
|
||||
if [ -x "${RENAME_SCRIPT}" ]; then
|
||||
echo "[sync-comfyui-snaps] Running rename-ai-snaps on ${DEST}"
|
||||
"${RENAME_SCRIPT}" "${DEST}" --no-interactive
|
||||
echo "[sync-comfyui-snaps] Running rename-ai-snaps on ${IMAGES_DIR}"
|
||||
"${RENAME_SCRIPT}" "${IMAGES_DIR}" --no-interactive
|
||||
else
|
||||
echo "[sync-comfyui-snaps] WARNING: rename-ai-snaps not found at ${RENAME_SCRIPT}"
|
||||
fi
|
||||
|
||||
# 4. Organize images
|
||||
# 4. Organize images into AI/{model}/{lora}/
|
||||
if [ -x "${ORGANIZE_SCRIPT}" ]; then
|
||||
echo "[sync-comfyui-snaps] Running organize-images on ${DEST}"
|
||||
"${ORGANIZE_SCRIPT}" "${DEST}" -e
|
||||
echo "[sync-comfyui-snaps] Running organize-images on ${IMAGES_DIR}"
|
||||
"${ORGANIZE_SCRIPT}" "${IMAGES_DIR}" -e
|
||||
else
|
||||
echo "[sync-comfyui-snaps] WARNING: organize-images not found at ${ORGANIZE_SCRIPT}"
|
||||
fi
|
||||
|
||||
80
sync-newer-images
Executable file
80
sync-newer-images
Executable file
@@ -0,0 +1,80 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# sync-newer-images — For each top-level image or video in SRC, copy it to
|
||||
# DEST if it doesn't already exist anywhere under DEST.
|
||||
# Works on Linux and macOS (bash 3.2+ compatible).
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: sync-newer-images [options] <destination> <source>
|
||||
|
||||
For each top-level image or video in source, copy it into destination if no
|
||||
file with the same name exists anywhere under destination.
|
||||
|
||||
Options:
|
||||
-u, --update If a matching file already exists under destination,
|
||||
copy over it only if the content differs (uses cmp).
|
||||
-h, --help Show this help message and exit
|
||||
EOF
|
||||
exit 0
|
||||
}
|
||||
|
||||
update_mode=0
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
-h|--help) usage ;;
|
||||
-u|--update) update_mode=1; shift ;;
|
||||
*) break ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ $# -lt 2 ]; then
|
||||
echo "Error: destination and source are required" >&2
|
||||
echo "Usage: sync-newer-images [options] <destination> <source>" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
DEST="$1"
|
||||
SRC="$2"
|
||||
|
||||
if [ ! -d "$DEST" ]; then
|
||||
echo "Error: destination '$DEST' is not a directory" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -d "$SRC" ]; then
|
||||
echo "Error: source '$SRC' is not a directory" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Build deduplicated, newline-separated list of all basenames under DEST.
|
||||
basenames=$(find "$DEST" -type f 2>/dev/null | sed 's|.*/||' | sort -u)
|
||||
|
||||
copied=0
|
||||
|
||||
for srcfile in "$SRC"/*; do
|
||||
[ -f "$srcfile" ] || continue
|
||||
|
||||
ext="${srcfile##*.}"
|
||||
case "$(echo "$ext" | tr '[:upper:]' '[:lower:]')" in
|
||||
jpg|jpeg|png|gif|heic|webp|bmp|tiff|tif) ;;
|
||||
mov|mp4|m4v|avi|mkv|3gp) ;;
|
||||
*) continue ;;
|
||||
esac
|
||||
|
||||
filename="${srcfile##*/}"
|
||||
|
||||
if ! printf '%s\n' "$basenames" | grep -qxF "$filename"; then
|
||||
cp -v "$srcfile" "$DEST/"
|
||||
copied=$((copied + 1))
|
||||
elif [ "$update_mode" -eq 1 ]; then
|
||||
destfile=$(find "$DEST" -type f -name "$filename" 2>/dev/null | head -1)
|
||||
if [ -n "$destfile" ] && ! cmp -s "$srcfile" "$destfile"; then
|
||||
cp -v "$srcfile" "$destfile"
|
||||
copied=$((copied + 1))
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Copied ${copied} image(s)/video(s) from ${SRC} into ${DEST}"
|
||||
@@ -0,0 +1,5 @@
|
||||
{\rtf1\ansi\ansicpg1252\cocoartf2759
|
||||
\cocoatextscaling0\cocoaplatform0{\fonttbl}
|
||||
{\colortbl;\red255\green255\blue255;}
|
||||
{\*\expandedcolortbl;;}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{\rtf1\ansi\ansicpg1252\cocoartf2759
|
||||
\cocoatextscaling0\cocoaplatform0{\fonttbl}
|
||||
{\colortbl;\red255\green255\blue255;}
|
||||
{\*\expandedcolortbl;;}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{\rtf1\ansi\ansicpg1252\cocoartf2759
|
||||
\cocoatextscaling0\cocoaplatform0{\fonttbl}
|
||||
{\colortbl;\red255\green255\blue255;}
|
||||
{\*\expandedcolortbl;;}
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
{\rtf1\ansi\ansicpg1252\cocoartf2759
|
||||
\cocoatextscaling0\cocoaplatform0{\fonttbl}
|
||||
{\colortbl;\red255\green255\blue255;}
|
||||
{\*\expandedcolortbl;;}
|
||||
}
|
||||
Reference in New Issue
Block a user