re-separate rename/organize, canonical --resort for AI folders
rename-ai-snaps is rename-only again (+--include-renamed, +--only-generic); extract-ai-meta is the shared recursive metadata engine emitting canonical model/lora dest paths; organize-images does all sorting, gains --resort for idempotent tree re-sorts (used to fix ~10K old-scheme folders on the NAS).
This commit is contained in:
21
AGENTS.md
21
AGENTS.md
@@ -50,8 +50,29 @@ sleep ──→ idlecheck_caffeinatestuck
|
||||
|
||||
pull-all ──→ iterates repos in ~/git/ matching *schmeeve* remote
|
||||
checkin-all ──→ iterates repos in ~/git/ matching *schmeeve* remote
|
||||
|
||||
comfyui-sync (systemd timer on nimo.loc, ~/git checkout)
|
||||
├── neatcli organize --by-type (drops → Pictures/Images/)
|
||||
├── rename-ai-snaps . -n (rename in place, top-level only)
|
||||
└── organize-images -e (sort into AI/{model}/{lora}/)
|
||||
```
|
||||
|
||||
### Image pipeline (ComfyUI)
|
||||
|
||||
Strict separation of concerns — do NOT re-merge them:
|
||||
|
||||
| Script | Role |
|
||||
|---|---|
|
||||
| `rename-ai-snaps` | Rename ONLY, in place, to `schmeeve-AI-{keywords}.png`. Skips `schmeeve-AI-*` unless `--include-renamed`; `--only-generic` limits to `ComfyUI_*` names. PNG only. |
|
||||
| `organize-images` | ALL sorting/moving. Default: top-level → `AI/ Photos/ Screenshots/ Unsorted/`. `--resort <Images root>`: recursive canonical re-sort into `AI/{model}/{lora}/` (never renames files). |
|
||||
| `extract-ai-meta` | Shared metadata engine. JSON manifest `relpath → {model, loras, dest}`; `dest` is the canonical sanitized folder (`+`-joined LoRAs, capped 200 chars). Presence in manifest = is an AI image. |
|
||||
|
||||
Canonical folder sanitizer lives in `extract-ai-meta` (`sanitize_dir_name`):
|
||||
path prefixes stripped (`\`→`/`, basename), spaces/specials → `_`. Anything else
|
||||
creating `AI/` subfolders must match it exactly.
|
||||
|
||||
`sync-comfyui-snaps` is the dormant cron predecessor of `comfyui-sync`.
|
||||
|
||||
Sub-scripts are invoked via `/bin/bash` or `/bin/sh`, never sourced.
|
||||
|
||||
### Trigger chain
|
||||
|
||||
110
FIX-AI-FOLDERS.md
Normal file
110
FIX-AI-FOLDERS.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# FIX-AI-FOLDERS — Re-sort plan
|
||||
|
||||
## Problem
|
||||
|
||||
~17K files under `/Volumes/miniShare1/Pictures/Images/AI/` were sorted by an older
|
||||
scheme. Two scripts currently create `AI/{model}/{lora}/` folders with **different
|
||||
sanitizers**, fragmenting the tree:
|
||||
|
||||
| | `organize-images` (`clean()`) | `rename-ai-snaps` (`sanitize_dir_name`) |
|
||||
|---|---|---|
|
||||
| `Pony XL\trserkanWildpony_v20.safetensors` | `Pony XL\trserkanWildpony_v20/` or `Pony XL-trserkanWildpony_v20/` (its `\\`→`-` fix only catches *double* backslashes) | `trserkanWildpony_v20/` (strips path prefix, `_`-sanitizes) |
|
||||
| spaces | kept (`FLUX Handsome Male Model_1.0/`) | → underscores |
|
||||
|
||||
Legacy artifacts also exist: `None/` and `unknown/` folders (at Images root and inside
|
||||
`AI/`), plus loose files at the `AI/` root.
|
||||
|
||||
Additional blockers:
|
||||
|
||||
- `rename-ai-snaps` **skips all `schmeeve-AI-*` files** — 17,178 of 17,255 files in
|
||||
`AI/`. As-is it would touch almost nothing.
|
||||
- `rename-ai-snaps` is PNG-only (fine: `AI/` is 99.97% PNG; 5 jpgs reported, not moved).
|
||||
- All 30 sampled `AI/` files still carry full ComfyUI `prompt` metadata — a
|
||||
metadata-driven re-sort is viable.
|
||||
- `Unsorted/` sweep: 0 of 120 sampled files (jpg/png/webp) have ComfyUI metadata —
|
||||
they are web downloads, not ComfyUI outputs. Only files with real `prompt`
|
||||
metadata qualify for moving.
|
||||
- `nas_mount_base()` does not know `/Volumes/miniShare1`.
|
||||
- `~/.local/bin/organize-images` is stale (older than repo copy).
|
||||
|
||||
## Requirements (user decisions)
|
||||
|
||||
1. **Scope:** Everything — `AI/`, `None/`, `unknown/`, plus sweep `Unsorted/` and
|
||||
`Screenshots/` for PNGs with real ComfyUI metadata.
|
||||
2. **Filenames:** Keep existing names, **except** generic names (`ComfyUI_*` etc.)
|
||||
which get a proper keyword-based rename.
|
||||
3. **Architecture:** Re-separate concerns — `rename-ai-snaps` renames only (the
|
||||
folder-moving added in `e85fce1 "model first"` was a mistake), `organize-images`
|
||||
does all sorting. Fix its sanitizer; sync the stale `~/.local/bin` copy.
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. `rename-ai-snaps` → rename-only
|
||||
|
||||
- Strip the move/folder logic (`output_root`, `mkdir`, `shutil.move`, model/lora
|
||||
folder computation). Keep metadata extraction, keyword scoring, and
|
||||
interactive/batch renaming **in place** (filename changes only, never directory
|
||||
changes).
|
||||
- Default: skip `schmeeve-AI-*` as today. New flag `--include-renamed` for
|
||||
corpus-wide renames later.
|
||||
|
||||
### 2. `extract-ai-meta` → shared, recursive metadata engine
|
||||
|
||||
- Add `-r` recursive walk; relative-path keys instead of bare basenames.
|
||||
- Port the robust extraction from `rename-ai-snaps` (`_strip_model_name`,
|
||||
`UNETLoader`/extra checkpoint classes, generic `*lora_name*` fields). It currently
|
||||
only knows `CheckpointLoaderSimple` and keeps backslash path prefixes — the cause
|
||||
of `Pony XL\…` folders.
|
||||
- Keep its connected-LoRA check (excludes disconnected LoRA nodes).
|
||||
- Add a `prompt_present` flag per file so `organize-images` can classify AI files
|
||||
**without** `mdls`, dimension heuristics, or filename guesses.
|
||||
|
||||
### 3. `organize-images` → all sorting, canonical scheme
|
||||
|
||||
- **Canonical sanitizer** (single implementation; fixes the `\\` bug and space
|
||||
handling): strip path prefix → spaces → `_` → non `[a-zA-Z0-9_.-]` → `_` →
|
||||
collapse. `Pony XL\trserkanWildpony_v20.safetensors` → `trserkanWildpony_v20`
|
||||
everywhere.
|
||||
- **New `--resort` mode:** recursively walk `AI/` (including `schmeeve-AI-*` files),
|
||||
compute canonical `AI/{model}/{lora}/` per file, move only what's misplaced.
|
||||
**Filenames never touched.** Files already correct are skipped (idempotent).
|
||||
Missing model → `unknown/`; no loras → `no-lora/`.
|
||||
- **Sort `None/` and `unknown/`:** pass as targets; files with metadata route into
|
||||
`AI/{model}/{lora}/`, files without go to `AI/unknown/`.
|
||||
- **Sweep `Unsorted/` + `Screenshots/`:** only files with actual ComfyUI `prompt`
|
||||
metadata move into `AI/{model}/{lora}/`.
|
||||
- **Generic-name hook:** before moving files matching `ComfyUI_*`/hash-style names,
|
||||
invoke `rename-ai-snaps -n` on them (its skip-rule already protects
|
||||
`schmeeve-AI-*`).
|
||||
- **Collision policy:** destination exists → append `_1`, `_2`… (never overwrite,
|
||||
never delete).
|
||||
- **Performance:** one metadata scan → JSON manifest in a temp file; move logic
|
||||
reads the manifest (no per-file `mdls`/`sips` subprocess storms).
|
||||
- Keep dry-run default, `-e` to execute, `--limit` for batching.
|
||||
|
||||
### 4. Verification
|
||||
|
||||
- Full dry-run first: summary counts per destination folder + sample moves.
|
||||
- Spot-check that `Pony XL\…`, `Pony XL-…`, `None/`, loose-root files land
|
||||
correctly; confirm already-correct files are untouched; re-run dry-run
|
||||
post-execute to confirm ~0 remaining moves (idempotency proof).
|
||||
|
||||
### 5. Deploy + docs
|
||||
|
||||
- Copy updated scripts to `~/.local/bin/` (fixes stale `organize-images`) and
|
||||
`~/Dropbox/bin/`.
|
||||
- Add a short "Image pipeline" section to `AGENTS.md` documenting the restored
|
||||
separation: `rename-ai-snaps` = rename, `organize-images` = sort,
|
||||
`extract-ai-meta` = shared engine.
|
||||
|
||||
## Untouched
|
||||
|
||||
- `Unsorted/` web downloads (no metadata = no move), `Screenshots/` non-AI files,
|
||||
non-PNGs in `AI/` (5 jpgs — reported, not moved).
|
||||
- Nothing is ever deleted; worst case is a `_1` suffix.
|
||||
|
||||
## Note
|
||||
|
||||
With `schmeeve-AI-*` names kept as-is, a handful may not match what today's keyword
|
||||
scorer would produce — but placement will be correct, and any folder can be
|
||||
re-renamed later with `rename-ai-snaps --include-renamed`.
|
||||
146
extract-ai-meta
146
extract-ai-meta
@@ -1,13 +1,22 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract model and LoRA metadata from ComfyUI-generated images.
|
||||
|
||||
Outputs a JSON mapping of filename → {"model": "...", "loras": ["...", ...]}.
|
||||
Outputs a JSON mapping of path (relative to target) →
|
||||
{"model": "...", "loras": ["...", ...], "dest": "{model_dir}/{lora_dir}"}
|
||||
|
||||
Only files with parseable ComfyUI 'prompt' metadata are included, so
|
||||
presence in the output doubles as "is an AI-generated image" for
|
||||
organize-images. 'dest' is the canonical folder path (sans leading "AI/"):
|
||||
model/LoRA names are path-stripped and sanitized (spaces/specials → _),
|
||||
LoRAs joined with '+', missing model → "unknown", no LoRAs → "no-lora".
|
||||
|
||||
Usage: extract-ai-meta [path] [-r|--recursive]
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from PIL import Image
|
||||
@@ -17,6 +26,43 @@ except ImportError:
|
||||
|
||||
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
|
||||
CHECKPOINT_CLASS_TYPES = {
|
||||
"CheckpointLoaderSimple",
|
||||
"Checkpoint Loader with Name (Image Saver)",
|
||||
"CheckpointLoader",
|
||||
}
|
||||
|
||||
MODEL_CLASS_TYPES = {
|
||||
"UNETLoader",
|
||||
}
|
||||
|
||||
LORA_CLASS_TYPES = {
|
||||
"LoraLoader",
|
||||
"Power Lora Loader (rgthree)",
|
||||
}
|
||||
|
||||
# Filesystem-safe cap for the joined LoRA combo directory name
|
||||
MAX_LORA_DIR_LEN = 200
|
||||
|
||||
|
||||
def _strip_model_name(raw):
|
||||
"""Strip path prefix and .safetensors extension from a model/LoRA name."""
|
||||
name = raw.strip()
|
||||
# Handle backslash/forward-slash paths: "Pony\model.safetensors" or "Pony/model.safetensors"
|
||||
name = name.replace("\\", "/")
|
||||
name = os.path.basename(name)
|
||||
if name.endswith(".safetensors"):
|
||||
name = name[:-len(".safetensors")]
|
||||
return name
|
||||
|
||||
|
||||
def sanitize_dir_name(name):
|
||||
"""Canonical directory-name sanitizer (spaces/specials → _)."""
|
||||
name = name.strip().replace(" ", "_")
|
||||
name = re.sub(r"[^a-zA-Z0-9_.-]", "_", name)
|
||||
name = re.sub(r"_+", "_", name).strip("_")
|
||||
return name
|
||||
|
||||
|
||||
def is_lora_connected(node_id, data):
|
||||
"""Check if a LoRA node's output is consumed by any downstream node."""
|
||||
@@ -30,19 +76,23 @@ def is_lora_connected(node_id, data):
|
||||
|
||||
|
||||
def extract_ai_meta(filepath):
|
||||
"""Return (model, loras) or (None, []) if no ComfyUI metadata found."""
|
||||
"""Return (model, loras, has_prompt).
|
||||
|
||||
has_prompt is True whenever the file carries parseable ComfyUI 'prompt'
|
||||
metadata, even if no model/LoRA nodes were found in it.
|
||||
"""
|
||||
try:
|
||||
img = Image.open(filepath)
|
||||
except Exception:
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
if "prompt" not in img.info:
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
try:
|
||||
data = json.loads(img.info["prompt"])
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
model = None
|
||||
loras = []
|
||||
@@ -51,24 +101,59 @@ def extract_ai_meta(filepath):
|
||||
cls = node.get("class_type", "")
|
||||
inputs = node.get("inputs", {})
|
||||
|
||||
if cls == "CheckpointLoaderSimple":
|
||||
ckpt = inputs.get("ckpt_name", "")
|
||||
if ckpt:
|
||||
model = ckpt.rsplit(".", 1)[0]
|
||||
if model is None:
|
||||
if cls in CHECKPOINT_CLASS_TYPES and inputs.get("ckpt_name"):
|
||||
model = _strip_model_name(inputs["ckpt_name"])
|
||||
elif cls in MODEL_CLASS_TYPES and inputs.get("unet_name"):
|
||||
model = _strip_model_name(inputs["unet_name"])
|
||||
|
||||
if "lora" in cls.lower():
|
||||
lora = inputs.get("lora_name", "")
|
||||
if lora:
|
||||
if cls in LORA_CLASS_TYPES and inputs.get("lora_name"):
|
||||
# Only include LoRAs whose output is actually consumed
|
||||
if is_lora_connected(nid, data):
|
||||
loras.append(lora.rsplit(".", 1)[0])
|
||||
loras.append(_strip_model_name(inputs["lora_name"]))
|
||||
else:
|
||||
# Some custom nodes use fields ending in 'lora'/'lora_name'
|
||||
for field, val in inputs.items():
|
||||
if isinstance(val, str) and val.endswith(".safetensors") and "lora" in field.lower():
|
||||
name = _strip_model_name(val)
|
||||
if name:
|
||||
loras.append(name)
|
||||
|
||||
loras.sort()
|
||||
return model, loras
|
||||
loras = sorted(set(loras))
|
||||
return model, loras, True
|
||||
|
||||
|
||||
def dest_for(model, loras):
|
||||
"""Canonical relative destination dir (sans 'AI/' prefix)."""
|
||||
model_dir = sanitize_dir_name(model) if model else "unknown"
|
||||
if loras:
|
||||
lora_dir = "+".join(sanitize_dir_name(l) for l in loras if l)
|
||||
if len(lora_dir) > MAX_LORA_DIR_LEN:
|
||||
lora_dir = lora_dir[:MAX_LORA_DIR_LEN].rstrip("+_")
|
||||
if not lora_dir:
|
||||
lora_dir = "no-lora"
|
||||
else:
|
||||
lora_dir = "no-lora"
|
||||
return f"{model_dir}/{lora_dir}"
|
||||
|
||||
|
||||
def iter_images(target, recursive):
|
||||
if recursive:
|
||||
for dirpath, _dirnames, filenames in os.walk(target):
|
||||
for fn in sorted(filenames):
|
||||
if os.path.splitext(fn)[1].lower() in IMAGE_EXTS:
|
||||
yield os.path.join(dirpath, fn)
|
||||
else:
|
||||
for entry in sorted(os.listdir(target)):
|
||||
fpath = os.path.join(target, entry)
|
||||
if os.path.isfile(fpath) and os.path.splitext(entry)[1].lower() in IMAGE_EXTS:
|
||||
yield fpath
|
||||
|
||||
|
||||
def main():
|
||||
target = sys.argv[1] if len(sys.argv) > 1 else os.path.expanduser("~/Pictures")
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("-")]
|
||||
recursive = any(a in ("-r", "--recursive") for a in sys.argv[1:])
|
||||
target = args[0] if args else os.path.expanduser("~/Pictures")
|
||||
target = os.path.abspath(target)
|
||||
|
||||
if not os.path.isdir(target):
|
||||
@@ -76,18 +161,25 @@ def main():
|
||||
sys.exit(1)
|
||||
|
||||
meta = {}
|
||||
|
||||
for entry in sorted(os.listdir(target)):
|
||||
fpath = os.path.join(target, entry)
|
||||
if not os.path.isfile(fpath):
|
||||
continue
|
||||
ext = os.path.splitext(entry)[1].lower()
|
||||
if ext not in IMAGE_EXTS:
|
||||
scanned = 0
|
||||
for fpath in iter_images(target, recursive):
|
||||
scanned += 1
|
||||
if scanned % 200 == 0:
|
||||
print(f"\r …scanned {scanned} images ({len(meta)} with AI metadata)",
|
||||
end="", file=sys.stderr, flush=True)
|
||||
model, loras, has_prompt = extract_ai_meta(fpath)
|
||||
if not has_prompt:
|
||||
continue
|
||||
rel = os.path.relpath(fpath, target)
|
||||
meta[rel] = {
|
||||
"model": model,
|
||||
"loras": loras,
|
||||
"dest": dest_for(model, loras),
|
||||
}
|
||||
|
||||
model, loras = extract_ai_meta(fpath)
|
||||
if model or loras:
|
||||
meta[entry] = {"model": model, "loras": loras}
|
||||
if scanned:
|
||||
print(f"\r Scanned {scanned} images, {len(meta)} with AI metadata. ",
|
||||
file=sys.stderr, flush=True)
|
||||
|
||||
print(json.dumps(meta, indent=2))
|
||||
|
||||
|
||||
294
organize-images
294
organize-images
@@ -1,23 +1,34 @@
|
||||
#!/bin/bash
|
||||
# organize-images -- Sort images into AI/, Photos/, Screenshots/, Unsorted/
|
||||
# Default: sub-categorizes AI by model + LoRA combo (uses extract-ai-meta).
|
||||
# Use --flat for one flat AI/ folder.
|
||||
# AI files route into AI/{model}/{lora}/ via extract-ai-meta (canonical scheme).
|
||||
#
|
||||
# Modes:
|
||||
# (default) Sort top-level files of [path] into the 4 buckets.
|
||||
# --resort Recursively re-sort [path] (the Images root containing AI/),
|
||||
# moving any file with ComfyUI metadata to its canonical
|
||||
# AI/{model}/{lora}/ location. Filenames are never changed.
|
||||
#
|
||||
# Generic ComfyUI_* filenames are renamed first via rename-ai-snaps
|
||||
# (--only-generic) before sorting, when that script is available.
|
||||
|
||||
TARGET=""
|
||||
DRY_RUN=true
|
||||
VERBOSE=false
|
||||
INSTALL=false
|
||||
FLAT=false
|
||||
RESORT=false
|
||||
LIMIT=0
|
||||
|
||||
usage() {
|
||||
echo "Usage: $(basename "$0") [path] [-e|--execute] [-v|--verbose] [-i|--install] [--flat]"
|
||||
echo "Usage: $(basename "$0") [path] [-e|--execute] [-v|--verbose] [-i|--install] [--flat] [--resort] [--limit N]"
|
||||
echo " path Directory to organize (default: ~/Pictures)"
|
||||
echo " -e Execute moves (default: dry-run only)"
|
||||
echo " -v Verbose: show per-file classification"
|
||||
echo " -v Verbose: show per-file classification/moves"
|
||||
echo " -i Install this script to ~/.local/bin/"
|
||||
echo " --flat Don't sub-categorize by model/LoRA (flat AI/ folder)"
|
||||
echo " --limit N Only process N files (for batching)"
|
||||
echo " --resort Recursively re-sort [path] into canonical AI/{model}/{lora}/"
|
||||
echo " ([path] must be the Images root containing AI/)"
|
||||
echo " --limit N Only process N files (for batching; default mode only)"
|
||||
exit 0
|
||||
}
|
||||
|
||||
@@ -34,6 +45,7 @@ for arg in "$@"; do
|
||||
-v|--verbose) VERBOSE=true ;;
|
||||
-i|--install) INSTALL=true ;;
|
||||
--flat) FLAT=true ;;
|
||||
--resort) RESORT=true ;;
|
||||
--limit) LIMIT_NEXT=true ;;
|
||||
--limit=*) LIMIT="${arg#*=}" ;;
|
||||
-h|--help) usage ;;
|
||||
@@ -63,34 +75,223 @@ if [ ! -d "$TARGET" ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "organize-images — Classifying images in: $TARGET"
|
||||
[ "$DRY_RUN" = true ] && echo " DRY RUN (use -e to execute)" || echo " EXECUTING"
|
||||
[ "$LIMIT" -gt 0 ] && echo " Limit: first $LIMIT files"
|
||||
# --- Resolve helper scripts (same dir as this script, else PATH) ---
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
find_tool() {
|
||||
local n="$1"
|
||||
if [ -x "$SCRIPT_DIR/$n" ]; then
|
||||
echo "$SCRIPT_DIR/$n"
|
||||
elif command -v "$n" >/dev/null 2>&1; then
|
||||
command -v "$n"
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# --- Pre-scan: extract AI metadata for model/LoRA routing ---
|
||||
META_SCRIPT="$(find_tool extract-ai-meta)" || {
|
||||
echo "Error: extract-ai-meta not found (looked next to $0 and in PATH)" >&2
|
||||
exit 1
|
||||
}
|
||||
RENAME_SCRIPT="$(find_tool rename-ai-snaps || true)"
|
||||
|
||||
echo "organize-images — $([ "$RESORT" = true ] && echo "Re-sorting" || echo "Classifying") images in: $TARGET"
|
||||
[ "$DRY_RUN" = true ] && echo " DRY RUN (use -e to execute)" || echo " EXECUTING"
|
||||
[ "$LIMIT" -gt 0 ] && [ "$RESORT" = false ] && echo " Limit: first $LIMIT files"
|
||||
|
||||
# --- Resort mode: recursive canonical re-sort of an existing tree ---
|
||||
if [ "$RESORT" = true ]; then
|
||||
if [ ! -d "$TARGET/AI" ]; then
|
||||
echo "Error: --resort expects the Images root (a directory containing AI/)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Generic-name hook: rename ComfyUI_* files in place before sorting
|
||||
GENERIC_COUNT=$(find "$TARGET" -type f -name 'ComfyUI*.png' 2>/dev/null | wc -l | tr -d ' ')
|
||||
if [ "$GENERIC_COUNT" -gt 0 ]; then
|
||||
if [ -n "$RENAME_SCRIPT" ]; then
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
echo " $GENERIC_COUNT generic ComfyUI_*.png file(s) would be renamed first (rename-ai-snaps --only-generic)"
|
||||
else
|
||||
echo " Renaming $GENERIC_COUNT generic ComfyUI_*.png file(s) first..."
|
||||
"$RENAME_SCRIPT" "$TARGET" -r -n --only-generic
|
||||
fi
|
||||
else
|
||||
echo " Note: $GENERIC_COUNT generic ComfyUI_*.png file(s) found; rename-ai-snaps not available, keeping names"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Metadata manifest (recursive)
|
||||
echo " Scanning AI metadata (recursive)..."
|
||||
AI_META=$(mktemp)
|
||||
if ! "$META_SCRIPT" "$TARGET" -r > "$AI_META"; then
|
||||
echo "Error: AI metadata scan failed" >&2
|
||||
rm -f "$AI_META"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 - "$TARGET" "$AI_META" "$DRY_RUN" "$VERBOSE" <<'PYEOF'
|
||||
import json, os, shutil, sys
|
||||
|
||||
root = os.path.abspath(sys.argv[1])
|
||||
meta = json.load(open(sys.argv[2]))
|
||||
dry = sys.argv[3] == "true"
|
||||
verbose = sys.argv[4] == "true"
|
||||
|
||||
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
LEGACY_DIRS = {"None", "unknown"}
|
||||
|
||||
moved = ok = nometa = collisions = errors = 0
|
||||
samples = []
|
||||
legacy_no_meta = []
|
||||
planned = set()
|
||||
|
||||
for dirpath, dirnames, filenames in os.walk(root):
|
||||
dirnames.sort()
|
||||
for fn in sorted(filenames):
|
||||
if os.path.splitext(fn)[1].lower() not in IMAGE_EXTS:
|
||||
continue
|
||||
src = os.path.join(dirpath, fn)
|
||||
rel = os.path.relpath(src, root)
|
||||
entry = meta.get(rel)
|
||||
if not entry:
|
||||
nometa += 1
|
||||
top = rel.split(os.sep)[0]
|
||||
if top in LEGACY_DIRS:
|
||||
legacy_no_meta.append(rel)
|
||||
continue
|
||||
|
||||
dest_dir = os.path.join(root, "AI", entry["dest"])
|
||||
if os.path.abspath(dest_dir) == os.path.abspath(dirpath):
|
||||
ok += 1
|
||||
continue
|
||||
|
||||
dest = os.path.join(dest_dir, fn)
|
||||
taken = planned if dry else None
|
||||
if os.path.exists(dest) or (taken is not None and dest in taken):
|
||||
stem, ext = os.path.splitext(fn)
|
||||
c = 1
|
||||
while os.path.exists(os.path.join(dest_dir, f"{stem}_{c}{ext}")) or \
|
||||
(taken is not None and os.path.join(dest_dir, f"{stem}_{c}{ext}") in taken):
|
||||
c += 1
|
||||
dest = os.path.join(dest_dir, f"{stem}_{c}{ext}")
|
||||
collisions += 1
|
||||
if taken is not None:
|
||||
taken.add(dest)
|
||||
|
||||
dest_rel = os.path.relpath(dest, root)
|
||||
if dry:
|
||||
moved += 1
|
||||
if len(samples) < 20:
|
||||
samples.append((rel, dest_rel))
|
||||
if verbose:
|
||||
print(f" WOULD MOVE {rel} → {dest_rel}")
|
||||
else:
|
||||
try:
|
||||
os.makedirs(dest_dir, exist_ok=True)
|
||||
shutil.move(src, dest)
|
||||
moved += 1
|
||||
if verbose:
|
||||
print(f" MOVED {rel} → {dest_rel}")
|
||||
elif moved % 200 == 0:
|
||||
print(f"\r …moved {moved}", end="", flush=True)
|
||||
except OSError as e:
|
||||
errors += 1
|
||||
print(f" SKIPPED ({e}): {rel}")
|
||||
|
||||
if not verbose and not dry and moved:
|
||||
print(f"\r Moved {moved} file(s). ")
|
||||
|
||||
# Prune dirs left empty by the moves (keep top-level buckets)
|
||||
pruned = 0
|
||||
if not dry and moved:
|
||||
keep = {"AI", "Photos", "Screenshots", "Unsorted"}
|
||||
for dirpath, dirnames, filenames in os.walk(root, topdown=False):
|
||||
if dirpath == root:
|
||||
continue
|
||||
if os.path.relpath(dirpath, root).split(os.sep)[0] in keep and \
|
||||
os.path.dirname(dirpath) == root:
|
||||
continue
|
||||
try:
|
||||
os.rmdir(dirpath) # only succeeds when empty
|
||||
pruned += 1
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
print()
|
||||
print(f" {'DRY RUN — would move' if dry else 'Moved'}: {moved}")
|
||||
print(f" Already correctly placed: {ok}")
|
||||
print(f" Collisions (renamed with _N suffix): {collisions}")
|
||||
if errors:
|
||||
print(f" Errors: {errors}")
|
||||
print(f" No AI metadata (left in place): {nometa}")
|
||||
if pruned:
|
||||
print(f" Empty folders pruned: {pruned}")
|
||||
|
||||
if samples:
|
||||
print()
|
||||
print(" Sample moves:")
|
||||
for s, d in samples:
|
||||
print(f" {s}")
|
||||
print(f" → {d}")
|
||||
|
||||
if legacy_no_meta:
|
||||
print()
|
||||
print(f" Note: {len(legacy_no_meta)} file(s) in None//unknown/ have no AI metadata")
|
||||
print(" and were left in place for manual triage (not all are AI images).")
|
||||
for rel in legacy_no_meta[:10]:
|
||||
print(f" {rel}")
|
||||
if len(legacy_no_meta) > 10:
|
||||
print(f" … and {len(legacy_no_meta) - 10} more")
|
||||
print()
|
||||
PYEOF
|
||||
STATUS=$?
|
||||
rm -f "$AI_META"
|
||||
exit $STATUS
|
||||
fi
|
||||
|
||||
# --- Default mode: classify top-level files into AI/Photos/Screenshots/Unsorted ---
|
||||
|
||||
# Generic-name hook (non-recursive)
|
||||
GENERIC_COUNT=$(find "$TARGET" -maxdepth 1 -type f -name 'ComfyUI*.png' 2>/dev/null | wc -l | tr -d ' ')
|
||||
if [ "$GENERIC_COUNT" -gt 0 ] && [ -n "$RENAME_SCRIPT" ]; then
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
echo " $GENERIC_COUNT generic ComfyUI_*.png file(s) would be renamed first (rename-ai-snaps --only-generic)"
|
||||
else
|
||||
echo " Renaming $GENERIC_COUNT generic ComfyUI_*.png file(s) first..."
|
||||
"$RENAME_SCRIPT" "$TARGET" -n --only-generic
|
||||
fi
|
||||
fi
|
||||
|
||||
# Pre-scan: extract AI metadata for model/LoRA routing
|
||||
AI_META=""
|
||||
if [ "$FLAT" = false ]; then
|
||||
META_SCRIPT="$(cd "$(dirname "$0")" && pwd -P)/extract-ai-meta"
|
||||
if [ -x "$META_SCRIPT" ]; then
|
||||
echo " Scanning AI metadata for model/LoRA routing..."
|
||||
AI_META=$(mktemp)
|
||||
if ! "$META_SCRIPT" "$TARGET" > "$AI_META" 2>/dev/null; then
|
||||
echo " Warning: AI metadata scan failed, falling back to flat AI/"
|
||||
FLAT=true
|
||||
rm -f "$AI_META"
|
||||
AI_META=""
|
||||
elif [ ! -s "$AI_META" ]; then
|
||||
FLAT=true
|
||||
rm -f "$AI_META"
|
||||
fi
|
||||
else
|
||||
echo " Warning: extract-ai-meta not found, falling back to flat AI/"
|
||||
FLAT=true
|
||||
AI_META=""
|
||||
fi
|
||||
fi
|
||||
|
||||
[ "$FLAT" = false ] && echo " AI sub-categorization: by model/LoRA (use --flat to disable)"
|
||||
echo
|
||||
|
||||
# Basenames of files with AI metadata (manifest presence = is an AI image)
|
||||
AI_KEYS=""
|
||||
if [ -n "$AI_META" ]; then
|
||||
AI_KEYS=$(mktemp)
|
||||
python3 -c "
|
||||
import json, sys
|
||||
for k in json.load(open('$AI_META')):
|
||||
print(k)
|
||||
" > "$AI_KEYS" 2>/dev/null
|
||||
fi
|
||||
|
||||
classify() {
|
||||
local f="$1"
|
||||
|
||||
@@ -110,6 +311,11 @@ classify() {
|
||||
ComfyUI_*|*_DEFI_*|R.*.png|schmeeve-AI-*) echo "AI"; return ;;
|
||||
esac
|
||||
|
||||
# Files with ComfyUI metadata are AI regardless of name
|
||||
if [ -n "$AI_KEYS" ] && grep -Fxq "$name" "$AI_KEYS" 2>/dev/null; then
|
||||
echo "AI"; return
|
||||
fi
|
||||
|
||||
if mdls -name kMDItemAcquisitionMake "$f" 2>/dev/null | grep -q '= "'; then
|
||||
echo "Photos"; return
|
||||
fi
|
||||
@@ -121,24 +327,10 @@ classify() {
|
||||
echo "Photos"; return ;;
|
||||
esac
|
||||
|
||||
if [ "$ext" = "png" ]; then
|
||||
local dims w h area ratio
|
||||
dims=$(sips --getProperty pixelWidth --getProperty pixelHeight "$f" 2>/dev/null | grep -E 'pixel(Width|Height)' | awk '{print $NF}')
|
||||
w=$(echo "$dims" | head -1)
|
||||
h=$(echo "$dims" | tail -1)
|
||||
if [ -n "$w" ] && [ -n "$h" ] && [ "$w" -gt 0 ] && [ "$h" -gt 0 ]; then
|
||||
area=$((w * h))
|
||||
ratio=$(echo "scale=4; $w / $h" | bc 2>/dev/null || echo 1)
|
||||
if [ "$area" -gt 1048576 ] && [ "$(echo "$ratio > 0.75" | bc -l)" = 1 ] && [ "$(echo "$ratio < 1.33" | bc -l)" = 1 ]; then
|
||||
echo "AI"; return
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Unsorted"
|
||||
}
|
||||
|
||||
# --- Pre-scan: collect all image files ---
|
||||
# --- Collect top-level image files ---
|
||||
FILES=()
|
||||
for f in "$TARGET"/*; do
|
||||
[ -f "$f" ] || continue
|
||||
@@ -155,6 +347,8 @@ if [ "$LIMIT" -gt 0 ] && [ "$LIMIT" -lt "$TOTAL" ]; then
|
||||
fi
|
||||
if [ "$TOTAL" -eq 0 ]; then
|
||||
echo " No image files found."
|
||||
[ -n "$AI_META" ] && rm -f "$AI_META"
|
||||
[ -n "$AI_KEYS" ] && rm -f "$AI_KEYS"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -174,27 +368,16 @@ progress_bar() {
|
||||
printf "] %3d%% (%d/%d)" "$pct" "$current" "$total"
|
||||
}
|
||||
|
||||
# Destination for an AI file: canonical AI/{model}/{lora}/ from the manifest
|
||||
ai_dest() {
|
||||
python3 -c "
|
||||
import json, sys
|
||||
m = json.load(open('$AI_META'))
|
||||
AI_META_PATH="$AI_META" python3 -c "
|
||||
import json, os, sys
|
||||
m = json.load(open(os.environ['AI_META_PATH']))
|
||||
entry = m.get(sys.argv[1])
|
||||
if not entry or not isinstance(entry, dict):
|
||||
if not entry or not isinstance(entry, dict) or not entry.get('dest'):
|
||||
print('AI/unknown')
|
||||
sys.exit(0)
|
||||
def clean(s):
|
||||
if not s:
|
||||
return 'unknown'
|
||||
return s.replace('/', '-').replace('\\\\', '-')
|
||||
|
||||
model = clean(entry.get('model'))
|
||||
loras = entry.get('loras') or []
|
||||
if loras:
|
||||
combo = '+'.join(clean(l.strip()) for l in loras if l)
|
||||
combo = (combo[:200]).rstrip('+')
|
||||
print(f'AI/{model}/{combo}')
|
||||
else:
|
||||
print(f'AI/{model}/no-lora')
|
||||
print('AI/' + entry['dest'])
|
||||
" "$1"
|
||||
}
|
||||
|
||||
@@ -224,7 +407,15 @@ for f in "${FILES[@]}"; do
|
||||
|
||||
if [ "$DRY_RUN" = false ]; then
|
||||
mkdir -p "$dest"
|
||||
mv -f "$f" "$dest"
|
||||
base=$(basename "$f")
|
||||
final="$dest/$base"
|
||||
if [ -e "$final" ]; then
|
||||
stem="${base%.*}"; fext="${base##*.}"
|
||||
n=1
|
||||
while [ -e "$dest/${stem}_${n}.${fext}" ]; do n=$((n + 1)); done
|
||||
final="$dest/${stem}_${n}.${fext}"
|
||||
fi
|
||||
mv "$f" "$final"
|
||||
fi
|
||||
|
||||
case "$cat" in
|
||||
@@ -235,13 +426,16 @@ for f in "${FILES[@]}"; do
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "$AI_META" ] && rm -f "$AI_META"
|
||||
[ -n "$AI_KEYS" ] && rm -f "$AI_KEYS"
|
||||
|
||||
if [ "$VERBOSE" = false ]; then
|
||||
printf "\r %-60s\n" "Done."
|
||||
fi
|
||||
|
||||
echo
|
||||
echo " Results:"
|
||||
if [ "$FLAT" = false ] && [ -n "$AI_META" ]; then
|
||||
if [ "$FLAT" = false ] && [ "$cAI" -gt 0 ]; then
|
||||
echo " AI (by model/LoRA): $cAI"
|
||||
else
|
||||
echo " AI: $cAI"
|
||||
|
||||
295
rename-ai-snaps
295
rename-ai-snaps
@@ -1,19 +1,20 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
rename-ai-snaps — Scan PNGs for AI prompt metadata and reorg into model/lora folders.
|
||||
rename-ai-snaps — Scan PNGs for AI prompt metadata and rename descriptively.
|
||||
|
||||
Usage:
|
||||
./rename-ai-snaps [path] [options]
|
||||
|
||||
Scans for ComfyUI PNGs, reads embedded prompt metadata, extracts the
|
||||
checkpoint model and LoRA names, and moves files into:
|
||||
{output}/{model}/{lora}/schmeeve-AI-{keywords}.png
|
||||
Scans for ComfyUI PNGs, reads embedded prompt metadata, extracts keywords,
|
||||
and renames files IN PLACE to:
|
||||
schmeeve-AI-{keywords}.png
|
||||
|
||||
Renaming only — use organize-images to sort files into model/lora folders.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
@@ -85,6 +86,9 @@ NEGATIVE_INDICATORS = {
|
||||
"body out of frame", "bad hands", "bad face", "blurry",
|
||||
}
|
||||
|
||||
# Generic ComfyUI output filenames (ComfyUI_00042_.png, ComfyUI_temp_….png)
|
||||
GENERIC_RE = re.compile(r"^ComfyUI", re.IGNORECASE)
|
||||
|
||||
# ── Helpers ────────────────────────────────────────────────────────────────
|
||||
|
||||
def spinner():
|
||||
@@ -115,82 +119,24 @@ def is_quality_only(text):
|
||||
|
||||
# ── Metadata extraction from ComfyUI prompt JSON ───────────────────────────
|
||||
|
||||
CHECKPOINT_CLASS_TYPES = {
|
||||
"CheckpointLoaderSimple",
|
||||
"Checkpoint Loader with Name (Image Saver)",
|
||||
"CheckpointLoader",
|
||||
}
|
||||
|
||||
LORA_CLASS_TYPES = {
|
||||
"LoraLoader",
|
||||
"Power Lora Loader (rgthree)",
|
||||
}
|
||||
|
||||
MODEL_CLASS_TYPES = {
|
||||
"UNETLoader",
|
||||
}
|
||||
|
||||
|
||||
def _strip_model_name(raw):
|
||||
"""Strip path prefix and .safetensors extension from a model/LoRA name."""
|
||||
name = raw.strip()
|
||||
# Handle backslash/forward-slash paths: "Pony\\model.safetensors" or "Pony/model.safetensors"
|
||||
name = name.replace("\\", "/")
|
||||
name = os.path.basename(name)
|
||||
# Strip .safetensors extension
|
||||
if name.endswith(".safetensors"):
|
||||
name = name[:-len(".safetensors")]
|
||||
return name
|
||||
|
||||
|
||||
def extract_model_name(data):
|
||||
"""Return the checkpoint/UNET model name from the prompt JSON, or None."""
|
||||
model_name = None
|
||||
for node in data.values():
|
||||
inputs = node.get("inputs", {})
|
||||
class_type = node.get("class_type", "")
|
||||
if class_type in CHECKPOINT_CLASS_TYPES and "ckpt_name" in inputs:
|
||||
model_name = _strip_model_name(inputs["ckpt_name"])
|
||||
break
|
||||
if class_type in MODEL_CLASS_TYPES and "unet_name" in inputs:
|
||||
model_name = _strip_model_name(inputs["unet_name"])
|
||||
break
|
||||
return model_name if model_name else None
|
||||
|
||||
|
||||
def extract_lora_names(data):
|
||||
"""Return a sorted list of unique LoRA names from the prompt JSON."""
|
||||
names = []
|
||||
for node in data.values():
|
||||
inputs = node.get("inputs", {})
|
||||
class_type = node.get("class_type", "")
|
||||
if class_type in LORA_CLASS_TYPES and "lora_name" in inputs:
|
||||
names.append(_strip_model_name(inputs["lora_name"]))
|
||||
# Some custom nodes use 'lora' or similar — look for any field ending
|
||||
# in 'lora_name' or 'lora' that contains '.safetensors'
|
||||
for field, val in inputs.items():
|
||||
if isinstance(val, str) and val.endswith(".safetensors") and "lora" in field.lower():
|
||||
name = _strip_model_name(val)
|
||||
if name and name not in names:
|
||||
names.append(name)
|
||||
return sorted(set(names))
|
||||
|
||||
|
||||
def extract_prompts(filepath):
|
||||
"""Return (positive_prompt, negative_prompt) from a ComfyUI PNG."""
|
||||
def load_prompt_data(filepath):
|
||||
"""Return parsed ComfyUI prompt JSON from a PNG, or None."""
|
||||
try:
|
||||
img = Image.open(filepath)
|
||||
except Exception:
|
||||
return None, None
|
||||
return None
|
||||
|
||||
if "prompt" not in img.info:
|
||||
return None, None
|
||||
return None
|
||||
|
||||
try:
|
||||
data = json.loads(img.info["prompt"])
|
||||
return json.loads(img.info["prompt"])
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None, None
|
||||
return None
|
||||
|
||||
|
||||
def extract_prompts(data):
|
||||
"""Return (positive_prompt, negative_prompt) from parsed prompt JSON."""
|
||||
# Gather all text fields from all nodes
|
||||
candidates = []
|
||||
for node in data.values():
|
||||
@@ -325,69 +271,22 @@ def keywords_to_name(keywords):
|
||||
return f"schmeeve-AI-{desc}.png" if desc and len(desc) > 3 else None
|
||||
|
||||
|
||||
def extract_metadata(filepath):
|
||||
"""Return (model_name, lora_names, positive_prompt, negative_prompt)."""
|
||||
try:
|
||||
img = Image.open(filepath)
|
||||
except Exception:
|
||||
return None, None, None, None
|
||||
|
||||
if "prompt" not in img.info:
|
||||
return None, None, None, None
|
||||
|
||||
try:
|
||||
data = json.loads(img.info["prompt"])
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None, None, None, None
|
||||
|
||||
model = extract_model_name(data)
|
||||
loras = extract_lora_names(data)
|
||||
pos, neg = extract_prompts(filepath)
|
||||
return model, loras, pos, neg
|
||||
|
||||
|
||||
def sanitize_dir_name(name):
|
||||
"""Sanitize a string for use as a directory name."""
|
||||
name = name.strip().replace(" ", "_")
|
||||
name = re.sub(r"[^a-zA-Z0-9_.-]", "_", name)
|
||||
name = re.sub(r"_+", "_", name).strip("_")
|
||||
return name
|
||||
|
||||
|
||||
# ── main ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def nas_mount_base():
|
||||
"""Return the NAS mount base path for this machine."""
|
||||
candidates = [
|
||||
"/mini.nas/miniShare1",
|
||||
os.path.expanduser("~/mnt/mini.nas/miniShare1"),
|
||||
os.path.expanduser("~/mnt/miniShare1"),
|
||||
]
|
||||
for p in candidates:
|
||||
if os.path.isdir(p):
|
||||
return p
|
||||
return candidates[-1]
|
||||
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
|
||||
DEFAULT_OUTPUT = os.path.join(nas_mount_base(), "Pictures", "Images", "AI")
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Scan PNGs for AI prompt metadata and reorg into model/lora folders.",
|
||||
description="Scan PNGs for AI prompt metadata and rename in place. "
|
||||
"Does not move files — use organize-images for sorting.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"path", nargs="?", default=os.path.expanduser("~/Pictures"),
|
||||
help="Directory to scan for PNG files (default: ~/Pictures)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-o", "--output", default=DEFAULT_OUTPUT,
|
||||
help=f"Output root directory (default: {DEFAULT_OUTPUT})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-n", "--no-interactive", action="store_true",
|
||||
help="Auto-reorg without prompting",
|
||||
help="Auto-rename without prompting",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-r", "--recursive", action="store_true",
|
||||
@@ -396,7 +295,15 @@ def main():
|
||||
parser.add_argument(
|
||||
"-p", "--preview", "--dry-run", action="store_true",
|
||||
dest="dry_run",
|
||||
help="Show proposed changes without making any (combine with -n for full listing)",
|
||||
help="Show proposed changes without making any",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--include-renamed", action="store_true",
|
||||
help="Also process files already named schmeeve-AI-* (default: skip them)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--only-generic", action="store_true",
|
||||
help="Only rename generic ComfyUI_* filenames",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
@@ -405,8 +312,6 @@ def main():
|
||||
print(f"Error: {scan_dir} is not a directory")
|
||||
sys.exit(1)
|
||||
|
||||
output_root = Path(args.output).expanduser().resolve()
|
||||
|
||||
# Gather PNGs
|
||||
glob_pattern = "**/*.png" if args.recursive else "*.png"
|
||||
pngs = sorted(scan_dir.glob(glob_pattern))
|
||||
@@ -425,20 +330,20 @@ def main():
|
||||
for p in pngs:
|
||||
sys.stdout.write(f"\r {next(spin)} Analyzing PNGs")
|
||||
sys.stdout.flush()
|
||||
# Skip already-renamed files (schmeeve-AI-*)
|
||||
if p.name.startswith("schmeeve-AI-"):
|
||||
# Skip already-renamed files (schmeeve-AI-*) unless requested
|
||||
if not args.include_renamed and p.name.startswith("schmeeve-AI-"):
|
||||
continue
|
||||
model, loras, pos, neg = extract_metadata(str(p))
|
||||
if args.only_generic and not GENERIC_RE.match(p.name):
|
||||
continue
|
||||
data = load_prompt_data(str(p))
|
||||
if not data:
|
||||
continue
|
||||
pos, neg = extract_prompts(data)
|
||||
source = pos or neg
|
||||
if source:
|
||||
candidates = extract_all_candidates(source)
|
||||
if candidates:
|
||||
image_data[p] = {
|
||||
"candidates": candidates,
|
||||
"model": model,
|
||||
"loras": loras,
|
||||
"source": source,
|
||||
}
|
||||
image_data[p] = candidates
|
||||
for kw, _ in candidates:
|
||||
if kw not in word_images:
|
||||
word_images[kw] = set()
|
||||
@@ -447,48 +352,35 @@ def main():
|
||||
freq_map = {kw: len(images) for kw, images in word_images.items()}
|
||||
total_images = len(image_data)
|
||||
|
||||
# ── Phase 1b: Propose file paths ──
|
||||
# ── Phase 1b: Propose new names (in place — same directory) ──
|
||||
raw_proposals = {}
|
||||
for p, info in image_data.items():
|
||||
candidates = info["candidates"]
|
||||
model = info["model"]
|
||||
loras = info["loras"]
|
||||
|
||||
for p, candidates in image_data.items():
|
||||
if total_images > 1:
|
||||
keywords = select_best_keywords(candidates, freq_map, total_images)
|
||||
else:
|
||||
keywords = [kw for kw, _ in candidates[:5]]
|
||||
name = keywords_to_name(keywords)
|
||||
if not name:
|
||||
if not name or name == p.name:
|
||||
continue
|
||||
|
||||
# Determine folder structure
|
||||
model_dir = sanitize_dir_name(model) if model else "unknown"
|
||||
if loras:
|
||||
lora_dir = sanitize_dir_name("+".join(loras))
|
||||
else:
|
||||
lora_dir = "no-lora"
|
||||
|
||||
rel_path = Path(model_dir) / lora_dir / name
|
||||
raw_proposals[p] = rel_path
|
||||
raw_proposals[p] = name
|
||||
|
||||
# Clear the spinner line
|
||||
sys.stdout.write("\r" + " " * 60 + "\r")
|
||||
sys.stdout.flush()
|
||||
|
||||
# Deduplicate file names within their target directories
|
||||
# Deduplicate file names within their directories
|
||||
proposals = {}
|
||||
used_paths = {}
|
||||
for p, rel_path in raw_proposals.items():
|
||||
new_path = rel_path
|
||||
counter = 0
|
||||
for p, name in raw_proposals.items():
|
||||
new_path = p.with_name(name)
|
||||
key = str(new_path)
|
||||
counter = 0
|
||||
while key in used_paths:
|
||||
counter += 1
|
||||
ts = time.strftime("%m%d-%H%M%S", time.localtime(p.stat().st_mtime))
|
||||
stem = rel_path.stem
|
||||
ext = rel_path.suffix
|
||||
new_path = rel_path.with_name(f"{stem}-{ts}{'_' + str(counter) if counter > 1 else ''}{ext}")
|
||||
stem = new_path.stem
|
||||
ext = new_path.suffix
|
||||
new_path = new_path.with_name(f"{stem}-{ts}{'_' + str(counter) if counter > 1 else ''}{ext}")
|
||||
key = str(new_path)
|
||||
used_paths[key] = True
|
||||
proposals[p] = new_path
|
||||
@@ -499,9 +391,9 @@ def main():
|
||||
return
|
||||
|
||||
print(f"\n {'DRY RUN' if args.dry_run else 'PROPOSED'} — "
|
||||
f"{len(proposals)} file(s) to organize into {output_root}/\n")
|
||||
f"{len(proposals)} file(s) to rename in place\n")
|
||||
|
||||
for i, (old_path, rel_path) in enumerate(proposals.items(), 1):
|
||||
for i, (old_path, new_path) in enumerate(proposals.items(), 1):
|
||||
try:
|
||||
src_display = str(old_path.relative_to(scan_dir))
|
||||
except ValueError:
|
||||
@@ -509,7 +401,7 @@ def main():
|
||||
if len(src_display) > 55:
|
||||
src_display = src_display[:25] + "…" + src_display[-27:]
|
||||
print(f" {i:>3}. {src_display}")
|
||||
print(f" → {output_root / rel_path}")
|
||||
print(f" → {new_path.name}")
|
||||
|
||||
print()
|
||||
|
||||
@@ -517,40 +409,46 @@ def main():
|
||||
print(f" Dry-run complete. No files were changed.\n")
|
||||
return
|
||||
|
||||
# ── Phase 3: Rename/move (interactive or auto) ──
|
||||
if args.no_interactive:
|
||||
moved = 0
|
||||
for old_path, rel_path in proposals.items():
|
||||
new_path = output_root / rel_path
|
||||
new_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
def do_rename(old_path, new_path):
|
||||
"""Rename with collision suffix. Returns final path or None on failure."""
|
||||
if old_path == new_path:
|
||||
return new_path
|
||||
if new_path.exists():
|
||||
stem = new_path.stem
|
||||
ext = new_path.suffix
|
||||
counter = 1
|
||||
while new_path.exists():
|
||||
new_path = new_path.with_name(f"{stem}_{counter}{new_path.suffix}")
|
||||
new_path = new_path.with_name(f"{stem}_{counter}{ext}")
|
||||
counter += 1
|
||||
print(f" (name existed, using {new_path.name})")
|
||||
try:
|
||||
shutil.move(old_path, new_path)
|
||||
moved += 1
|
||||
old_path.rename(new_path)
|
||||
return new_path
|
||||
except (FileNotFoundError, OSError) as e:
|
||||
print(f" SKIPPED ({e}): {old_path.name}")
|
||||
print(f" Moved {moved} file(s)." if not args.dry_run else "")
|
||||
return None
|
||||
|
||||
# ── Phase 3: Rename (interactive or auto) ──
|
||||
if args.no_interactive:
|
||||
renamed = 0
|
||||
for old_path, new_path in proposals.items():
|
||||
if do_rename(old_path, new_path):
|
||||
renamed += 1
|
||||
print(f" Renamed {renamed} file(s).")
|
||||
else:
|
||||
moved = 0
|
||||
renamed = 0
|
||||
skipped = 0
|
||||
items = list(proposals.items())
|
||||
i = 0
|
||||
while i < len(items):
|
||||
old_path, rel_path = items[i]
|
||||
new_path = output_root / rel_path
|
||||
new_path_display = str(rel_path)
|
||||
old_path, new_path = items[i]
|
||||
|
||||
print(f"\n [{i+1}/{len(items)}]")
|
||||
print(f" From: {old_path.name}")
|
||||
print(f" To: {output_root / rel_path}")
|
||||
print(f" To: {new_path.name}")
|
||||
remaining = len(items) - i - 1
|
||||
rlabel = f"move all {remaining}" if remaining else ""
|
||||
sys.stdout.write(f" [Enter]=move [e]=edit [s]=skip{' [a]=' + rlabel if rlabel else ''} [q]=quit: ")
|
||||
rlabel = f"rename all {remaining}" if remaining else ""
|
||||
sys.stdout.write(f" [Enter]=rename [e]=edit [s]=skip{' [a]=' + rlabel if rlabel else ''} [q]=quit: ")
|
||||
sys.stdout.flush()
|
||||
choice = sys.stdin.readline().strip().lower()
|
||||
|
||||
@@ -564,15 +462,14 @@ def main():
|
||||
i += 1
|
||||
continue
|
||||
elif choice == "e":
|
||||
sys.stdout.write(f" Edit name (will be 'schmeeve-AI-{rel_path.parent}/...'): ")
|
||||
sys.stdout.write(f" Edit keywords (name becomes 'schmeeve-AI-...'): ")
|
||||
sys.stdout.flush()
|
||||
custom = sys.stdin.readline().strip()
|
||||
if custom:
|
||||
custom_desc = re.sub(r"[^a-z0-9-]", "-", custom.lower())
|
||||
custom_desc = re.sub(r"-+", "-", custom_desc).strip("-")
|
||||
if custom_desc:
|
||||
new_name = f"schmeeve-AI-{custom_desc}.png"
|
||||
rel_path = rel_path.with_name(new_name)
|
||||
new_path = old_path.with_name(f"schmeeve-AI-{custom_desc}.png")
|
||||
else:
|
||||
print(" Invalid name, skipping.")
|
||||
skipped += 1
|
||||
@@ -586,20 +483,9 @@ def main():
|
||||
pass
|
||||
elif choice == "a":
|
||||
for j in range(i, len(items)):
|
||||
p, rp = items[j]
|
||||
np = output_root / rp
|
||||
np.parent.mkdir(parents=True, exist_ok=True)
|
||||
if np.exists():
|
||||
stem = np.stem
|
||||
counter = 1
|
||||
while np.exists():
|
||||
np = np.with_name(f"{stem}_{counter}{np.suffix}")
|
||||
counter += 1
|
||||
try:
|
||||
shutil.move(p, np)
|
||||
except (FileNotFoundError, OSError) as e:
|
||||
print(f" SKIPPED ({e}): {p.name}")
|
||||
moved += len(items) - i
|
||||
p, np = items[j]
|
||||
if do_rename(p, np):
|
||||
renamed += 1
|
||||
break
|
||||
else:
|
||||
print(f" Unknown option '{choice}', skipping.")
|
||||
@@ -607,24 +493,11 @@ def main():
|
||||
i += 1
|
||||
continue
|
||||
|
||||
# Perform move
|
||||
new_path = output_root / rel_path
|
||||
new_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if new_path.exists():
|
||||
stem = new_path.stem
|
||||
counter = 1
|
||||
while new_path.exists():
|
||||
new_path = new_path.with_name(f"{stem}_{counter}{new_path.suffix}")
|
||||
counter += 1
|
||||
print(f" (file existed, saved as {new_path.name})")
|
||||
try:
|
||||
shutil.move(old_path, new_path)
|
||||
moved += 1
|
||||
except (FileNotFoundError, OSError) as e:
|
||||
print(f" SKIPPED ({e}): {old_path.name}")
|
||||
if do_rename(old_path, new_path):
|
||||
renamed += 1
|
||||
i += 1
|
||||
|
||||
print(f"\n Moved: {moved} Skipped: {skipped}")
|
||||
print(f"\n Renamed: {renamed} Skipped: {skipped}")
|
||||
|
||||
# Check for JPEGs with AI metadata
|
||||
jpg_pattern = "**/*.jpg" if args.recursive else "*.jpg"
|
||||
|
||||
@@ -3,6 +3,9 @@
|
||||
# sync-comfyui-snaps — Pull new images from nimo.loc ComfyUI output, sync to NAS,
|
||||
# then rename AI snaps and organize.
|
||||
#
|
||||
# NOTE: superseded by comfyui-sync (systemd user timer on nimo.loc). Kept for
|
||||
# reference/manual use; the cron entry below is NOT currently installed.
|
||||
#
|
||||
# Crontab (every 2 hours):
|
||||
# 0 */2 * * * /home/schmeeve/.local/bin/sync-comfyui-snaps >> /home/schmeeve/.local/log/sync-comfyui-snaps.log 2>&1
|
||||
#
|
||||
@@ -72,19 +75,26 @@ rsync -vu --checksum \
|
||||
--progress --stats \
|
||||
"${SRC}" "${DEST}/"
|
||||
|
||||
# 3. Rename AI snaps (non-interactive for automation)
|
||||
AI_OUTPUT="${DEST}/Images/AI"
|
||||
# 3. Route fresh drops into Images/ (neatcli equivalent), then rename in place.
|
||||
# NOTE: rename-ai-snaps no longer takes --output; it only renames.
|
||||
# organize-images does the model/lora sorting.
|
||||
IMAGES_DIR="${DEST}/Images"
|
||||
mkdir -p "${IMAGES_DIR}"
|
||||
find "${DEST}" -maxdepth 1 -type f \
|
||||
\( -iname '*.png' -o -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.webp' \) \
|
||||
-exec mv -n {} "${IMAGES_DIR}/" \;
|
||||
|
||||
if [ -x "${RENAME_SCRIPT}" ]; then
|
||||
echo "[sync-comfyui-snaps] Running rename-ai-snaps on ${DEST} → ${AI_OUTPUT}"
|
||||
"${RENAME_SCRIPT}" "${DEST}" --output "${AI_OUTPUT}" --no-interactive
|
||||
echo "[sync-comfyui-snaps] Running rename-ai-snaps on ${IMAGES_DIR}"
|
||||
"${RENAME_SCRIPT}" "${IMAGES_DIR}" --no-interactive
|
||||
else
|
||||
echo "[sync-comfyui-snaps] WARNING: rename-ai-snaps not found at ${RENAME_SCRIPT}"
|
||||
fi
|
||||
|
||||
# 4. Organize images
|
||||
# 4. Organize images into AI/{model}/{lora}/
|
||||
if [ -x "${ORGANIZE_SCRIPT}" ]; then
|
||||
echo "[sync-comfyui-snaps] Running organize-images on ${DEST}"
|
||||
"${ORGANIZE_SCRIPT}" "${DEST}" -e
|
||||
echo "[sync-comfyui-snaps] Running organize-images on ${IMAGES_DIR}"
|
||||
"${ORGANIZE_SCRIPT}" "${IMAGES_DIR}" -e
|
||||
else
|
||||
echo "[sync-comfyui-snaps] WARNING: organize-images not found at ${ORGANIZE_SCRIPT}"
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user