Compare commits
3 Commits
bf1361262a
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 32859a5df0 | |||
| 1c1dbf7f21 | |||
| fbd1cee0ba |
21
AGENTS.md
21
AGENTS.md
@@ -50,8 +50,29 @@ sleep ──→ idlecheck_caffeinatestuck
|
||||
|
||||
pull-all ──→ iterates repos in ~/git/ matching *schmeeve* remote
|
||||
checkin-all ──→ iterates repos in ~/git/ matching *schmeeve* remote
|
||||
|
||||
comfyui-sync (systemd timer on nimo.loc, ~/git checkout)
|
||||
├── neatcli organize --by-type (drops → Pictures/Images/)
|
||||
├── rename-ai-snaps . -n (rename in place, top-level only)
|
||||
└── organize-images -e (sort into AI/{model}/{lora}/)
|
||||
```
|
||||
|
||||
### Image pipeline (ComfyUI)
|
||||
|
||||
Strict separation of concerns — do NOT re-merge them:
|
||||
|
||||
| Script | Role |
|
||||
|---|---|
|
||||
| `rename-ai-snaps` | Rename ONLY, in place, to `schmeeve-AI-{keywords}.png`. Skips `schmeeve-AI-*` unless `--include-renamed`; `--only-generic` limits to `ComfyUI_*` names. PNG only. |
|
||||
| `organize-images` | ALL sorting/moving. Default: top-level → `AI/ Photos/ Screenshots/ Unsorted/`. `--resort <Images root>`: recursive canonical re-sort into `AI/{model}/{lora}/` (never renames files). |
|
||||
| `extract-ai-meta` | Shared metadata engine. JSON manifest `relpath → {model, loras, dest}`; `dest` is the canonical sanitized folder (`+`-joined LoRAs, capped 200 chars). Presence in manifest = is an AI image. |
|
||||
|
||||
Canonical folder sanitizer lives in `extract-ai-meta` (`sanitize_dir_name`):
|
||||
path prefixes stripped (`\`→`/`, basename), spaces/specials → `_`. Anything else
|
||||
creating `AI/` subfolders must match it exactly.
|
||||
|
||||
`sync-comfyui-snaps` is the dormant cron predecessor of `comfyui-sync`.
|
||||
|
||||
Sub-scripts are invoked via `/bin/bash` or `/bin/sh`, never sourced.
|
||||
|
||||
### Trigger chain
|
||||
|
||||
110
FIX-AI-FOLDERS.md
Normal file
110
FIX-AI-FOLDERS.md
Normal file
@@ -0,0 +1,110 @@
|
||||
# FIX-AI-FOLDERS — Re-sort plan
|
||||
|
||||
## Problem
|
||||
|
||||
~17K files under `/Volumes/miniShare1/Pictures/Images/AI/` were sorted by an older
|
||||
scheme. Two scripts currently create `AI/{model}/{lora}/` folders with **different
|
||||
sanitizers**, fragmenting the tree:
|
||||
|
||||
| | `organize-images` (`clean()`) | `rename-ai-snaps` (`sanitize_dir_name`) |
|
||||
|---|---|---|
|
||||
| `Pony XL\trserkanWildpony_v20.safetensors` | `Pony XL\trserkanWildpony_v20/` or `Pony XL-trserkanWildpony_v20/` (its `\\`→`-` fix only catches *double* backslashes) | `trserkanWildpony_v20/` (strips path prefix, `_`-sanitizes) |
|
||||
| spaces | kept (`FLUX Handsome Male Model_1.0/`) | → underscores |
|
||||
|
||||
Legacy artifacts also exist: `None/` and `unknown/` folders (at Images root and inside
|
||||
`AI/`), plus loose files at the `AI/` root.
|
||||
|
||||
Additional blockers:
|
||||
|
||||
- `rename-ai-snaps` **skips all `schmeeve-AI-*` files** — 17,178 of 17,255 files in
|
||||
`AI/`. As-is it would touch almost nothing.
|
||||
- `rename-ai-snaps` is PNG-only (fine: `AI/` is 99.97% PNG; 5 jpgs reported, not moved).
|
||||
- All 30 sampled `AI/` files still carry full ComfyUI `prompt` metadata — a
|
||||
metadata-driven re-sort is viable.
|
||||
- `Unsorted/` sweep: 0 of 120 sampled files (jpg/png/webp) have ComfyUI metadata —
|
||||
they are web downloads, not ComfyUI outputs. Only files with real `prompt`
|
||||
metadata qualify for moving.
|
||||
- `nas_mount_base()` does not know `/Volumes/miniShare1`.
|
||||
- `~/.local/bin/organize-images` is stale (older than repo copy).
|
||||
|
||||
## Requirements (user decisions)
|
||||
|
||||
1. **Scope:** Everything — `AI/`, `None/`, `unknown/`, plus sweep `Unsorted/` and
|
||||
`Screenshots/` for PNGs with real ComfyUI metadata.
|
||||
2. **Filenames:** Keep existing names, **except** generic names (`ComfyUI_*` etc.)
|
||||
which get a proper keyword-based rename.
|
||||
3. **Architecture:** Re-separate concerns — `rename-ai-snaps` renames only (the
|
||||
folder-moving added in `e85fce1 "model first"` was a mistake), `organize-images`
|
||||
does all sorting. Fix its sanitizer; sync the stale `~/.local/bin` copy.
|
||||
|
||||
## Plan
|
||||
|
||||
### 1. `rename-ai-snaps` → rename-only
|
||||
|
||||
- Strip the move/folder logic (`output_root`, `mkdir`, `shutil.move`, model/lora
|
||||
folder computation). Keep metadata extraction, keyword scoring, and
|
||||
interactive/batch renaming **in place** (filename changes only, never directory
|
||||
changes).
|
||||
- Default: skip `schmeeve-AI-*` as today. New flag `--include-renamed` for
|
||||
corpus-wide renames later.
|
||||
|
||||
### 2. `extract-ai-meta` → shared, recursive metadata engine
|
||||
|
||||
- Add `-r` recursive walk; relative-path keys instead of bare basenames.
|
||||
- Port the robust extraction from `rename-ai-snaps` (`_strip_model_name`,
|
||||
`UNETLoader`/extra checkpoint classes, generic `*lora_name*` fields). It currently
|
||||
only knows `CheckpointLoaderSimple` and keeps backslash path prefixes — the cause
|
||||
of `Pony XL\…` folders.
|
||||
- Keep its connected-LoRA check (excludes disconnected LoRA nodes).
|
||||
- Add a `prompt_present` flag per file so `organize-images` can classify AI files
|
||||
**without** `mdls`, dimension heuristics, or filename guesses.
|
||||
|
||||
### 3. `organize-images` → all sorting, canonical scheme
|
||||
|
||||
- **Canonical sanitizer** (single implementation; fixes the `\\` bug and space
|
||||
handling): strip path prefix → spaces → `_` → non `[a-zA-Z0-9_.-]` → `_` →
|
||||
collapse. `Pony XL\trserkanWildpony_v20.safetensors` → `trserkanWildpony_v20`
|
||||
everywhere.
|
||||
- **New `--resort` mode:** recursively walk `AI/` (including `schmeeve-AI-*` files),
|
||||
compute canonical `AI/{model}/{lora}/` per file, move only what's misplaced.
|
||||
**Filenames never touched.** Files already correct are skipped (idempotent).
|
||||
Missing model → `unknown/`; no loras → `no-lora/`.
|
||||
- **Sort `None/` and `unknown/`:** pass as targets; files with metadata route into
|
||||
`AI/{model}/{lora}/`, files without go to `AI/unknown/`.
|
||||
- **Sweep `Unsorted/` + `Screenshots/`:** only files with actual ComfyUI `prompt`
|
||||
metadata move into `AI/{model}/{lora}/`.
|
||||
- **Generic-name hook:** before moving files matching `ComfyUI_*`/hash-style names,
|
||||
invoke `rename-ai-snaps -n` on them (its skip-rule already protects
|
||||
`schmeeve-AI-*`).
|
||||
- **Collision policy:** destination exists → append `_1`, `_2`… (never overwrite,
|
||||
never delete).
|
||||
- **Performance:** one metadata scan → JSON manifest in a temp file; move logic
|
||||
reads the manifest (no per-file `mdls`/`sips` subprocess storms).
|
||||
- Keep dry-run default, `-e` to execute, `--limit` for batching.
|
||||
|
||||
### 4. Verification
|
||||
|
||||
- Full dry-run first: summary counts per destination folder + sample moves.
|
||||
- Spot-check that `Pony XL\…`, `Pony XL-…`, `None/`, loose-root files land
|
||||
correctly; confirm already-correct files are untouched; re-run dry-run
|
||||
post-execute to confirm ~0 remaining moves (idempotency proof).
|
||||
|
||||
### 5. Deploy + docs
|
||||
|
||||
- Copy updated scripts to `~/.local/bin/` (fixes stale `organize-images`) and
|
||||
`~/Dropbox/bin/`.
|
||||
- Add a short "Image pipeline" section to `AGENTS.md` documenting the restored
|
||||
separation: `rename-ai-snaps` = rename, `organize-images` = sort,
|
||||
`extract-ai-meta` = shared engine.
|
||||
|
||||
## Untouched
|
||||
|
||||
- `Unsorted/` web downloads (no metadata = no move), `Screenshots/` non-AI files,
|
||||
non-PNGs in `AI/` (5 jpgs — reported, not moved).
|
||||
- Nothing is ever deleted; worst case is a `_1` suffix.
|
||||
|
||||
## Note
|
||||
|
||||
With `schmeeve-AI-*` names kept as-is, a handful may not match what today's keyword
|
||||
scorer would produce — but placement will be correct, and any folder can be
|
||||
re-renamed later with `rename-ai-snaps --include-renamed`.
|
||||
92
comfyui-sync
Executable file
92
comfyui-sync
Executable file
@@ -0,0 +1,92 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
OUTPUT_SOURCE="$HOME/ComfyUI/output"
|
||||
VIDEO_SOURCE="$HOME/ComfyUI/videos"
|
||||
AUDIO_SOURCE="$HOME/ComfyUI/output/audio"
|
||||
NAS_MOUNT="/mini.nas/miniShare1"
|
||||
SHARE_DIR="$NAS_MOUNT/Pictures"
|
||||
STATE_FILE="$HOME/.local/state/comfyui-sync.state"
|
||||
NEATCLI="$(command -v /home/linuxbrew/.linuxbrew/bin/neatcli || true)"
|
||||
RENAME_SCRIPT="$HOME/git/schmeeve-toolz/rename-ai-snaps"
|
||||
ORGANIZE_SCRIPT="$HOME/git/schmeeve-toolz/organize-images"
|
||||
|
||||
mkdir -p "$(dirname "$STATE_FILE")"
|
||||
|
||||
if [ ! -d "$SHARE_DIR" ]; then
|
||||
echo "[comfyui-sync] ERROR: Share dir $SHARE_DIR does not exist"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! mountpoint -q "$NAS_MOUNT"; then
|
||||
echo "[comfyui-sync] $NAS_MOUNT is not mounted, attempting to mount..."
|
||||
if ! sudo -n mount "$NAS_MOUNT" 2>&1; then
|
||||
echo "[comfyui-sync] ERROR: mount attempt failed"
|
||||
exit 1
|
||||
fi
|
||||
if ! mountpoint -q "$NAS_MOUNT"; then
|
||||
echo "[comfyui-sync] ERROR: $NAS_MOUNT still not mounted after mount attempt, refusing to copy locally"
|
||||
exit 1
|
||||
fi
|
||||
echo "[comfyui-sync] Mount succeeded."
|
||||
fi
|
||||
|
||||
echo "[comfyui-sync] Starting at $(date)"
|
||||
|
||||
current_files=$( {
|
||||
find "$OUTPUT_SOURCE" -maxdepth 1 -type f -printf '%f\n' 2>/dev/null
|
||||
find "$VIDEO_SOURCE" -maxdepth 1 -type f -printf '%f\n' 2>/dev/null
|
||||
find "$AUDIO_SOURCE" -maxdepth 1 -type f -printf '%f\n' 2>/dev/null
|
||||
} | sort)
|
||||
|
||||
if [ -f "$STATE_FILE" ]; then
|
||||
new_files=$(comm -13 <(sort "$STATE_FILE") <(echo "$current_files"))
|
||||
else
|
||||
echo "[comfyui-sync] First run — all files considered new."
|
||||
new_files="$current_files"
|
||||
fi
|
||||
|
||||
new_count=$(echo "$new_files" | grep -c '[^[:space:]]' || true)
|
||||
|
||||
if [ "$new_count" -gt 0 ]; then
|
||||
echo "[comfyui-sync] Copying $new_count new file(s)..."
|
||||
while IFS= read -r f; do
|
||||
[ -z "$f" ] && continue
|
||||
if [ -f "$OUTPUT_SOURCE/$f" ]; then
|
||||
cp "$OUTPUT_SOURCE/$f" "$SHARE_DIR/"
|
||||
elif [ -f "$VIDEO_SOURCE/$f" ]; then
|
||||
cp "$VIDEO_SOURCE/$f" "$SHARE_DIR/"
|
||||
elif [ -f "$AUDIO_SOURCE/$f" ]; then
|
||||
cp "$AUDIO_SOURCE/$f" "$SHARE_DIR/"
|
||||
fi
|
||||
done <<<"$new_files"
|
||||
|
||||
echo "$current_files" >"$STATE_FILE"
|
||||
echo "[comfyui-sync] State file updated."
|
||||
else
|
||||
echo "[comfyui-sync] No new files to copy."
|
||||
fi
|
||||
|
||||
if [ -n "$NEATCLI" ]; then
|
||||
echo "[comfyui-sync] Running neatcli organize --by-type -e $SHARE_DIR"
|
||||
"$NEATCLI" organize --by-type -e "$SHARE_DIR"
|
||||
else
|
||||
echo "[comfyui-sync] WARNING: neatcli not found, skipping organize step"
|
||||
fi
|
||||
|
||||
IMAGES_DIR="$SHARE_DIR/Images"
|
||||
if [ -d "$IMAGES_DIR" ] && [ -x "$RENAME_SCRIPT" ]; then
|
||||
echo "[comfyui-sync] Running rename-ai-snaps on $IMAGES_DIR"
|
||||
cd "$IMAGES_DIR" && "$RENAME_SCRIPT" . --no-interactive
|
||||
elif [ ! -x "$RENAME_SCRIPT" ]; then
|
||||
echo "[comfyui-sync] WARNING: rename-ai-snaps not found, skipping rename step"
|
||||
fi
|
||||
|
||||
if [ -d "$IMAGES_DIR" ] && [ -x "$ORGANIZE_SCRIPT" ]; then
|
||||
echo "[comfyui-sync] Running organize-images on $IMAGES_DIR"
|
||||
"$ORGANIZE_SCRIPT" "$IMAGES_DIR" --execute
|
||||
elif [ ! -x "$ORGANIZE_SCRIPT" ]; then
|
||||
echo "[comfyui-sync] WARNING: organize-images not found, skipping organize step"
|
||||
fi
|
||||
|
||||
echo "[comfyui-sync] Done at $(date)"
|
||||
146
extract-ai-meta
146
extract-ai-meta
@@ -1,13 +1,22 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Extract model and LoRA metadata from ComfyUI-generated images.
|
||||
|
||||
Outputs a JSON mapping of filename → {"model": "...", "loras": ["...", ...]}.
|
||||
Outputs a JSON mapping of path (relative to target) →
|
||||
{"model": "...", "loras": ["...", ...], "dest": "{model_dir}/{lora_dir}"}
|
||||
|
||||
Only files with parseable ComfyUI 'prompt' metadata are included, so
|
||||
presence in the output doubles as "is an AI-generated image" for
|
||||
organize-images. 'dest' is the canonical folder path (sans leading "AI/"):
|
||||
model/LoRA names are path-stripped and sanitized (spaces/specials → _),
|
||||
LoRAs joined with '+', missing model → "unknown", no LoRAs → "no-lora".
|
||||
|
||||
Usage: extract-ai-meta [path] [-r|--recursive]
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from PIL import Image
|
||||
@@ -17,6 +26,43 @@ except ImportError:
|
||||
|
||||
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
|
||||
CHECKPOINT_CLASS_TYPES = {
|
||||
"CheckpointLoaderSimple",
|
||||
"Checkpoint Loader with Name (Image Saver)",
|
||||
"CheckpointLoader",
|
||||
}
|
||||
|
||||
MODEL_CLASS_TYPES = {
|
||||
"UNETLoader",
|
||||
}
|
||||
|
||||
LORA_CLASS_TYPES = {
|
||||
"LoraLoader",
|
||||
"Power Lora Loader (rgthree)",
|
||||
}
|
||||
|
||||
# Filesystem-safe cap for the joined LoRA combo directory name
|
||||
MAX_LORA_DIR_LEN = 200
|
||||
|
||||
|
||||
def _strip_model_name(raw):
|
||||
"""Strip path prefix and .safetensors extension from a model/LoRA name."""
|
||||
name = raw.strip()
|
||||
# Handle backslash/forward-slash paths: "Pony\model.safetensors" or "Pony/model.safetensors"
|
||||
name = name.replace("\\", "/")
|
||||
name = os.path.basename(name)
|
||||
if name.endswith(".safetensors"):
|
||||
name = name[:-len(".safetensors")]
|
||||
return name
|
||||
|
||||
|
||||
def sanitize_dir_name(name):
|
||||
"""Canonical directory-name sanitizer (spaces/specials → _)."""
|
||||
name = name.strip().replace(" ", "_")
|
||||
name = re.sub(r"[^a-zA-Z0-9_.-]", "_", name)
|
||||
name = re.sub(r"_+", "_", name).strip("_")
|
||||
return name
|
||||
|
||||
|
||||
def is_lora_connected(node_id, data):
|
||||
"""Check if a LoRA node's output is consumed by any downstream node."""
|
||||
@@ -30,19 +76,23 @@ def is_lora_connected(node_id, data):
|
||||
|
||||
|
||||
def extract_ai_meta(filepath):
|
||||
"""Return (model, loras) or (None, []) if no ComfyUI metadata found."""
|
||||
"""Return (model, loras, has_prompt).
|
||||
|
||||
has_prompt is True whenever the file carries parseable ComfyUI 'prompt'
|
||||
metadata, even if no model/LoRA nodes were found in it.
|
||||
"""
|
||||
try:
|
||||
img = Image.open(filepath)
|
||||
except Exception:
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
if "prompt" not in img.info:
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
try:
|
||||
data = json.loads(img.info["prompt"])
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None, []
|
||||
return None, [], False
|
||||
|
||||
model = None
|
||||
loras = []
|
||||
@@ -51,24 +101,59 @@ def extract_ai_meta(filepath):
|
||||
cls = node.get("class_type", "")
|
||||
inputs = node.get("inputs", {})
|
||||
|
||||
if cls == "CheckpointLoaderSimple":
|
||||
ckpt = inputs.get("ckpt_name", "")
|
||||
if ckpt:
|
||||
model = ckpt.rsplit(".", 1)[0]
|
||||
if model is None:
|
||||
if cls in CHECKPOINT_CLASS_TYPES and inputs.get("ckpt_name"):
|
||||
model = _strip_model_name(inputs["ckpt_name"])
|
||||
elif cls in MODEL_CLASS_TYPES and inputs.get("unet_name"):
|
||||
model = _strip_model_name(inputs["unet_name"])
|
||||
|
||||
if "lora" in cls.lower():
|
||||
lora = inputs.get("lora_name", "")
|
||||
if lora:
|
||||
if cls in LORA_CLASS_TYPES and inputs.get("lora_name"):
|
||||
# Only include LoRAs whose output is actually consumed
|
||||
if is_lora_connected(nid, data):
|
||||
loras.append(lora.rsplit(".", 1)[0])
|
||||
loras.append(_strip_model_name(inputs["lora_name"]))
|
||||
else:
|
||||
# Some custom nodes use fields ending in 'lora'/'lora_name'
|
||||
for field, val in inputs.items():
|
||||
if isinstance(val, str) and val.endswith(".safetensors") and "lora" in field.lower():
|
||||
name = _strip_model_name(val)
|
||||
if name:
|
||||
loras.append(name)
|
||||
|
||||
loras.sort()
|
||||
return model, loras
|
||||
loras = sorted(set(loras))
|
||||
return model, loras, True
|
||||
|
||||
|
||||
def dest_for(model, loras):
|
||||
"""Canonical relative destination dir (sans 'AI/' prefix)."""
|
||||
model_dir = sanitize_dir_name(model) if model else "unknown"
|
||||
if loras:
|
||||
lora_dir = "+".join(sanitize_dir_name(l) for l in loras if l)
|
||||
if len(lora_dir) > MAX_LORA_DIR_LEN:
|
||||
lora_dir = lora_dir[:MAX_LORA_DIR_LEN].rstrip("+_")
|
||||
if not lora_dir:
|
||||
lora_dir = "no-lora"
|
||||
else:
|
||||
lora_dir = "no-lora"
|
||||
return f"{model_dir}/{lora_dir}"
|
||||
|
||||
|
||||
def iter_images(target, recursive):
|
||||
if recursive:
|
||||
for dirpath, _dirnames, filenames in os.walk(target):
|
||||
for fn in sorted(filenames):
|
||||
if os.path.splitext(fn)[1].lower() in IMAGE_EXTS:
|
||||
yield os.path.join(dirpath, fn)
|
||||
else:
|
||||
for entry in sorted(os.listdir(target)):
|
||||
fpath = os.path.join(target, entry)
|
||||
if os.path.isfile(fpath) and os.path.splitext(entry)[1].lower() in IMAGE_EXTS:
|
||||
yield fpath
|
||||
|
||||
|
||||
def main():
|
||||
target = sys.argv[1] if len(sys.argv) > 1 else os.path.expanduser("~/Pictures")
|
||||
args = [a for a in sys.argv[1:] if not a.startswith("-")]
|
||||
recursive = any(a in ("-r", "--recursive") for a in sys.argv[1:])
|
||||
target = args[0] if args else os.path.expanduser("~/Pictures")
|
||||
target = os.path.abspath(target)
|
||||
|
||||
if not os.path.isdir(target):
|
||||
@@ -76,18 +161,25 @@ def main():
|
||||
sys.exit(1)
|
||||
|
||||
meta = {}
|
||||
|
||||
for entry in sorted(os.listdir(target)):
|
||||
fpath = os.path.join(target, entry)
|
||||
if not os.path.isfile(fpath):
|
||||
continue
|
||||
ext = os.path.splitext(entry)[1].lower()
|
||||
if ext not in IMAGE_EXTS:
|
||||
scanned = 0
|
||||
for fpath in iter_images(target, recursive):
|
||||
scanned += 1
|
||||
if scanned % 200 == 0:
|
||||
print(f"\r …scanned {scanned} images ({len(meta)} with AI metadata)",
|
||||
end="", file=sys.stderr, flush=True)
|
||||
model, loras, has_prompt = extract_ai_meta(fpath)
|
||||
if not has_prompt:
|
||||
continue
|
||||
rel = os.path.relpath(fpath, target)
|
||||
meta[rel] = {
|
||||
"model": model,
|
||||
"loras": loras,
|
||||
"dest": dest_for(model, loras),
|
||||
}
|
||||
|
||||
model, loras = extract_ai_meta(fpath)
|
||||
if model or loras:
|
||||
meta[entry] = {"model": model, "loras": loras}
|
||||
if scanned:
|
||||
print(f"\r Scanned {scanned} images, {len(meta)} with AI metadata. ",
|
||||
file=sys.stderr, flush=True)
|
||||
|
||||
print(json.dumps(meta, indent=2))
|
||||
|
||||
|
||||
294
organize-images
294
organize-images
@@ -1,23 +1,34 @@
|
||||
#!/bin/bash
|
||||
# organize-images -- Sort images into AI/, Photos/, Screenshots/, Unsorted/
|
||||
# Default: sub-categorizes AI by model + LoRA combo (uses extract-ai-meta).
|
||||
# Use --flat for one flat AI/ folder.
|
||||
# AI files route into AI/{model}/{lora}/ via extract-ai-meta (canonical scheme).
|
||||
#
|
||||
# Modes:
|
||||
# (default) Sort top-level files of [path] into the 4 buckets.
|
||||
# --resort Recursively re-sort [path] (the Images root containing AI/),
|
||||
# moving any file with ComfyUI metadata to its canonical
|
||||
# AI/{model}/{lora}/ location. Filenames are never changed.
|
||||
#
|
||||
# Generic ComfyUI_* filenames are renamed first via rename-ai-snaps
|
||||
# (--only-generic) before sorting, when that script is available.
|
||||
|
||||
TARGET=""
|
||||
DRY_RUN=true
|
||||
VERBOSE=false
|
||||
INSTALL=false
|
||||
FLAT=false
|
||||
RESORT=false
|
||||
LIMIT=0
|
||||
|
||||
usage() {
|
||||
echo "Usage: $(basename "$0") [path] [-e|--execute] [-v|--verbose] [-i|--install] [--flat]"
|
||||
echo "Usage: $(basename "$0") [path] [-e|--execute] [-v|--verbose] [-i|--install] [--flat] [--resort] [--limit N]"
|
||||
echo " path Directory to organize (default: ~/Pictures)"
|
||||
echo " -e Execute moves (default: dry-run only)"
|
||||
echo " -v Verbose: show per-file classification"
|
||||
echo " -v Verbose: show per-file classification/moves"
|
||||
echo " -i Install this script to ~/.local/bin/"
|
||||
echo " --flat Don't sub-categorize by model/LoRA (flat AI/ folder)"
|
||||
echo " --limit N Only process N files (for batching)"
|
||||
echo " --resort Recursively re-sort [path] into canonical AI/{model}/{lora}/"
|
||||
echo " ([path] must be the Images root containing AI/)"
|
||||
echo " --limit N Only process N files (for batching; default mode only)"
|
||||
exit 0
|
||||
}
|
||||
|
||||
@@ -34,6 +45,7 @@ for arg in "$@"; do
|
||||
-v|--verbose) VERBOSE=true ;;
|
||||
-i|--install) INSTALL=true ;;
|
||||
--flat) FLAT=true ;;
|
||||
--resort) RESORT=true ;;
|
||||
--limit) LIMIT_NEXT=true ;;
|
||||
--limit=*) LIMIT="${arg#*=}" ;;
|
||||
-h|--help) usage ;;
|
||||
@@ -63,34 +75,223 @@ if [ ! -d "$TARGET" ]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "organize-images — Classifying images in: $TARGET"
|
||||
[ "$DRY_RUN" = true ] && echo " DRY RUN (use -e to execute)" || echo " EXECUTING"
|
||||
[ "$LIMIT" -gt 0 ] && echo " Limit: first $LIMIT files"
|
||||
# --- Resolve helper scripts (same dir as this script, else PATH) ---
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
find_tool() {
|
||||
local n="$1"
|
||||
if [ -x "$SCRIPT_DIR/$n" ]; then
|
||||
echo "$SCRIPT_DIR/$n"
|
||||
elif command -v "$n" >/dev/null 2>&1; then
|
||||
command -v "$n"
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# --- Pre-scan: extract AI metadata for model/LoRA routing ---
|
||||
META_SCRIPT="$(find_tool extract-ai-meta)" || {
|
||||
echo "Error: extract-ai-meta not found (looked next to $0 and in PATH)" >&2
|
||||
exit 1
|
||||
}
|
||||
RENAME_SCRIPT="$(find_tool rename-ai-snaps || true)"
|
||||
|
||||
echo "organize-images — $([ "$RESORT" = true ] && echo "Re-sorting" || echo "Classifying") images in: $TARGET"
|
||||
[ "$DRY_RUN" = true ] && echo " DRY RUN (use -e to execute)" || echo " EXECUTING"
|
||||
[ "$LIMIT" -gt 0 ] && [ "$RESORT" = false ] && echo " Limit: first $LIMIT files"
|
||||
|
||||
# --- Resort mode: recursive canonical re-sort of an existing tree ---
|
||||
if [ "$RESORT" = true ]; then
|
||||
if [ ! -d "$TARGET/AI" ]; then
|
||||
echo "Error: --resort expects the Images root (a directory containing AI/)" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Generic-name hook: rename ComfyUI_* files in place before sorting
|
||||
GENERIC_COUNT=$(find "$TARGET" -type f -name 'ComfyUI*.png' 2>/dev/null | wc -l | tr -d ' ')
|
||||
if [ "$GENERIC_COUNT" -gt 0 ]; then
|
||||
if [ -n "$RENAME_SCRIPT" ]; then
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
echo " $GENERIC_COUNT generic ComfyUI_*.png file(s) would be renamed first (rename-ai-snaps --only-generic)"
|
||||
else
|
||||
echo " Renaming $GENERIC_COUNT generic ComfyUI_*.png file(s) first..."
|
||||
"$RENAME_SCRIPT" "$TARGET" -r -n --only-generic
|
||||
fi
|
||||
else
|
||||
echo " Note: $GENERIC_COUNT generic ComfyUI_*.png file(s) found; rename-ai-snaps not available, keeping names"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Metadata manifest (recursive)
|
||||
echo " Scanning AI metadata (recursive)..."
|
||||
AI_META=$(mktemp)
|
||||
if ! "$META_SCRIPT" "$TARGET" -r > "$AI_META"; then
|
||||
echo "Error: AI metadata scan failed" >&2
|
||||
rm -f "$AI_META"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
python3 - "$TARGET" "$AI_META" "$DRY_RUN" "$VERBOSE" <<'PYEOF'
|
||||
import json, os, shutil, sys
|
||||
|
||||
root = os.path.abspath(sys.argv[1])
|
||||
meta = json.load(open(sys.argv[2]))
|
||||
dry = sys.argv[3] == "true"
|
||||
verbose = sys.argv[4] == "true"
|
||||
|
||||
IMAGE_EXTS = {".png", ".jpg", ".jpeg", ".webp"}
|
||||
LEGACY_DIRS = {"None", "unknown"}
|
||||
|
||||
moved = ok = nometa = collisions = errors = 0
|
||||
samples = []
|
||||
legacy_no_meta = []
|
||||
planned = set()
|
||||
|
||||
for dirpath, dirnames, filenames in os.walk(root):
|
||||
dirnames.sort()
|
||||
for fn in sorted(filenames):
|
||||
if os.path.splitext(fn)[1].lower() not in IMAGE_EXTS:
|
||||
continue
|
||||
src = os.path.join(dirpath, fn)
|
||||
rel = os.path.relpath(src, root)
|
||||
entry = meta.get(rel)
|
||||
if not entry:
|
||||
nometa += 1
|
||||
top = rel.split(os.sep)[0]
|
||||
if top in LEGACY_DIRS:
|
||||
legacy_no_meta.append(rel)
|
||||
continue
|
||||
|
||||
dest_dir = os.path.join(root, "AI", entry["dest"])
|
||||
if os.path.abspath(dest_dir) == os.path.abspath(dirpath):
|
||||
ok += 1
|
||||
continue
|
||||
|
||||
dest = os.path.join(dest_dir, fn)
|
||||
taken = planned if dry else None
|
||||
if os.path.exists(dest) or (taken is not None and dest in taken):
|
||||
stem, ext = os.path.splitext(fn)
|
||||
c = 1
|
||||
while os.path.exists(os.path.join(dest_dir, f"{stem}_{c}{ext}")) or \
|
||||
(taken is not None and os.path.join(dest_dir, f"{stem}_{c}{ext}") in taken):
|
||||
c += 1
|
||||
dest = os.path.join(dest_dir, f"{stem}_{c}{ext}")
|
||||
collisions += 1
|
||||
if taken is not None:
|
||||
taken.add(dest)
|
||||
|
||||
dest_rel = os.path.relpath(dest, root)
|
||||
if dry:
|
||||
moved += 1
|
||||
if len(samples) < 20:
|
||||
samples.append((rel, dest_rel))
|
||||
if verbose:
|
||||
print(f" WOULD MOVE {rel} → {dest_rel}")
|
||||
else:
|
||||
try:
|
||||
os.makedirs(dest_dir, exist_ok=True)
|
||||
shutil.move(src, dest)
|
||||
moved += 1
|
||||
if verbose:
|
||||
print(f" MOVED {rel} → {dest_rel}")
|
||||
elif moved % 200 == 0:
|
||||
print(f"\r …moved {moved}", end="", flush=True)
|
||||
except OSError as e:
|
||||
errors += 1
|
||||
print(f" SKIPPED ({e}): {rel}")
|
||||
|
||||
if not verbose and not dry and moved:
|
||||
print(f"\r Moved {moved} file(s). ")
|
||||
|
||||
# Prune dirs left empty by the moves (keep top-level buckets)
|
||||
pruned = 0
|
||||
if not dry and moved:
|
||||
keep = {"AI", "Photos", "Screenshots", "Unsorted"}
|
||||
for dirpath, dirnames, filenames in os.walk(root, topdown=False):
|
||||
if dirpath == root:
|
||||
continue
|
||||
if os.path.relpath(dirpath, root).split(os.sep)[0] in keep and \
|
||||
os.path.dirname(dirpath) == root:
|
||||
continue
|
||||
try:
|
||||
os.rmdir(dirpath) # only succeeds when empty
|
||||
pruned += 1
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
print()
|
||||
print(f" {'DRY RUN — would move' if dry else 'Moved'}: {moved}")
|
||||
print(f" Already correctly placed: {ok}")
|
||||
print(f" Collisions (renamed with _N suffix): {collisions}")
|
||||
if errors:
|
||||
print(f" Errors: {errors}")
|
||||
print(f" No AI metadata (left in place): {nometa}")
|
||||
if pruned:
|
||||
print(f" Empty folders pruned: {pruned}")
|
||||
|
||||
if samples:
|
||||
print()
|
||||
print(" Sample moves:")
|
||||
for s, d in samples:
|
||||
print(f" {s}")
|
||||
print(f" → {d}")
|
||||
|
||||
if legacy_no_meta:
|
||||
print()
|
||||
print(f" Note: {len(legacy_no_meta)} file(s) in None//unknown/ have no AI metadata")
|
||||
print(" and were left in place for manual triage (not all are AI images).")
|
||||
for rel in legacy_no_meta[:10]:
|
||||
print(f" {rel}")
|
||||
if len(legacy_no_meta) > 10:
|
||||
print(f" … and {len(legacy_no_meta) - 10} more")
|
||||
print()
|
||||
PYEOF
|
||||
STATUS=$?
|
||||
rm -f "$AI_META"
|
||||
exit $STATUS
|
||||
fi
|
||||
|
||||
# --- Default mode: classify top-level files into AI/Photos/Screenshots/Unsorted ---
|
||||
|
||||
# Generic-name hook (non-recursive)
|
||||
GENERIC_COUNT=$(find "$TARGET" -maxdepth 1 -type f -name 'ComfyUI*.png' 2>/dev/null | wc -l | tr -d ' ')
|
||||
if [ "$GENERIC_COUNT" -gt 0 ] && [ -n "$RENAME_SCRIPT" ]; then
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
echo " $GENERIC_COUNT generic ComfyUI_*.png file(s) would be renamed first (rename-ai-snaps --only-generic)"
|
||||
else
|
||||
echo " Renaming $GENERIC_COUNT generic ComfyUI_*.png file(s) first..."
|
||||
"$RENAME_SCRIPT" "$TARGET" -n --only-generic
|
||||
fi
|
||||
fi
|
||||
|
||||
# Pre-scan: extract AI metadata for model/LoRA routing
|
||||
AI_META=""
|
||||
if [ "$FLAT" = false ]; then
|
||||
META_SCRIPT="$(cd "$(dirname "$0")" && pwd -P)/extract-ai-meta"
|
||||
if [ -x "$META_SCRIPT" ]; then
|
||||
echo " Scanning AI metadata for model/LoRA routing..."
|
||||
AI_META=$(mktemp)
|
||||
if ! "$META_SCRIPT" "$TARGET" > "$AI_META" 2>/dev/null; then
|
||||
echo " Warning: AI metadata scan failed, falling back to flat AI/"
|
||||
FLAT=true
|
||||
rm -f "$AI_META"
|
||||
AI_META=""
|
||||
elif [ ! -s "$AI_META" ]; then
|
||||
FLAT=true
|
||||
rm -f "$AI_META"
|
||||
fi
|
||||
else
|
||||
echo " Warning: extract-ai-meta not found, falling back to flat AI/"
|
||||
FLAT=true
|
||||
AI_META=""
|
||||
fi
|
||||
fi
|
||||
|
||||
[ "$FLAT" = false ] && echo " AI sub-categorization: by model/LoRA (use --flat to disable)"
|
||||
echo
|
||||
|
||||
# Basenames of files with AI metadata (manifest presence = is an AI image)
|
||||
AI_KEYS=""
|
||||
if [ -n "$AI_META" ]; then
|
||||
AI_KEYS=$(mktemp)
|
||||
python3 -c "
|
||||
import json, sys
|
||||
for k in json.load(open('$AI_META')):
|
||||
print(k)
|
||||
" > "$AI_KEYS" 2>/dev/null
|
||||
fi
|
||||
|
||||
classify() {
|
||||
local f="$1"
|
||||
|
||||
@@ -110,6 +311,11 @@ classify() {
|
||||
ComfyUI_*|*_DEFI_*|R.*.png|schmeeve-AI-*) echo "AI"; return ;;
|
||||
esac
|
||||
|
||||
# Files with ComfyUI metadata are AI regardless of name
|
||||
if [ -n "$AI_KEYS" ] && grep -Fxq "$name" "$AI_KEYS" 2>/dev/null; then
|
||||
echo "AI"; return
|
||||
fi
|
||||
|
||||
if mdls -name kMDItemAcquisitionMake "$f" 2>/dev/null | grep -q '= "'; then
|
||||
echo "Photos"; return
|
||||
fi
|
||||
@@ -121,24 +327,10 @@ classify() {
|
||||
echo "Photos"; return ;;
|
||||
esac
|
||||
|
||||
if [ "$ext" = "png" ]; then
|
||||
local dims w h area ratio
|
||||
dims=$(sips --getProperty pixelWidth --getProperty pixelHeight "$f" 2>/dev/null | grep -E 'pixel(Width|Height)' | awk '{print $NF}')
|
||||
w=$(echo "$dims" | head -1)
|
||||
h=$(echo "$dims" | tail -1)
|
||||
if [ -n "$w" ] && [ -n "$h" ] && [ "$w" -gt 0 ] && [ "$h" -gt 0 ]; then
|
||||
area=$((w * h))
|
||||
ratio=$(echo "scale=4; $w / $h" | bc 2>/dev/null || echo 1)
|
||||
if [ "$area" -gt 1048576 ] && [ "$(echo "$ratio > 0.75" | bc -l)" = 1 ] && [ "$(echo "$ratio < 1.33" | bc -l)" = 1 ]; then
|
||||
echo "AI"; return
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Unsorted"
|
||||
}
|
||||
|
||||
# --- Pre-scan: collect all image files ---
|
||||
# --- Collect top-level image files ---
|
||||
FILES=()
|
||||
for f in "$TARGET"/*; do
|
||||
[ -f "$f" ] || continue
|
||||
@@ -155,6 +347,8 @@ if [ "$LIMIT" -gt 0 ] && [ "$LIMIT" -lt "$TOTAL" ]; then
|
||||
fi
|
||||
if [ "$TOTAL" -eq 0 ]; then
|
||||
echo " No image files found."
|
||||
[ -n "$AI_META" ] && rm -f "$AI_META"
|
||||
[ -n "$AI_KEYS" ] && rm -f "$AI_KEYS"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
@@ -174,27 +368,16 @@ progress_bar() {
|
||||
printf "] %3d%% (%d/%d)" "$pct" "$current" "$total"
|
||||
}
|
||||
|
||||
# Destination for an AI file: canonical AI/{model}/{lora}/ from the manifest
|
||||
ai_dest() {
|
||||
python3 -c "
|
||||
import json, sys
|
||||
m = json.load(open('$AI_META'))
|
||||
AI_META_PATH="$AI_META" python3 -c "
|
||||
import json, os, sys
|
||||
m = json.load(open(os.environ['AI_META_PATH']))
|
||||
entry = m.get(sys.argv[1])
|
||||
if not entry or not isinstance(entry, dict):
|
||||
if not entry or not isinstance(entry, dict) or not entry.get('dest'):
|
||||
print('AI/unknown')
|
||||
sys.exit(0)
|
||||
def clean(s):
|
||||
if not s:
|
||||
return 'unknown'
|
||||
return s.replace('/', '-').replace('\\\\', '-')
|
||||
|
||||
model = clean(entry.get('model'))
|
||||
loras = entry.get('loras') or []
|
||||
if loras:
|
||||
combo = '+'.join(clean(l.strip()) for l in loras if l)
|
||||
combo = (combo[:200]).rstrip('+')
|
||||
print(f'AI/{model}/{combo}')
|
||||
else:
|
||||
print(f'AI/{model}/no-lora')
|
||||
print('AI/' + entry['dest'])
|
||||
" "$1"
|
||||
}
|
||||
|
||||
@@ -224,7 +407,15 @@ for f in "${FILES[@]}"; do
|
||||
|
||||
if [ "$DRY_RUN" = false ]; then
|
||||
mkdir -p "$dest"
|
||||
mv -f "$f" "$dest"
|
||||
base=$(basename "$f")
|
||||
final="$dest/$base"
|
||||
if [ -e "$final" ]; then
|
||||
stem="${base%.*}"; fext="${base##*.}"
|
||||
n=1
|
||||
while [ -e "$dest/${stem}_${n}.${fext}" ]; do n=$((n + 1)); done
|
||||
final="$dest/${stem}_${n}.${fext}"
|
||||
fi
|
||||
mv "$f" "$final"
|
||||
fi
|
||||
|
||||
case "$cat" in
|
||||
@@ -235,13 +426,16 @@ for f in "${FILES[@]}"; do
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "$AI_META" ] && rm -f "$AI_META"
|
||||
[ -n "$AI_KEYS" ] && rm -f "$AI_KEYS"
|
||||
|
||||
if [ "$VERBOSE" = false ]; then
|
||||
printf "\r %-60s\n" "Done."
|
||||
fi
|
||||
|
||||
echo
|
||||
echo " Results:"
|
||||
if [ "$FLAT" = false ] && [ -n "$AI_META" ]; then
|
||||
if [ "$FLAT" = false ] && [ "$cAI" -gt 0 ]; then
|
||||
echo " AI (by model/LoRA): $cAI"
|
||||
else
|
||||
echo " AI: $cAI"
|
||||
|
||||
517
rename-ai-snaps
Executable file
517
rename-ai-snaps
Executable file
@@ -0,0 +1,517 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
rename-ai-snaps — Scan PNGs for AI prompt metadata and rename descriptively.
|
||||
|
||||
Usage:
|
||||
./rename-ai-snaps [path] [options]
|
||||
|
||||
Scans for ComfyUI PNGs, reads embedded prompt metadata, extracts keywords,
|
||||
and renames files IN PLACE to:
|
||||
schmeeve-AI-{keywords}.png
|
||||
|
||||
Renaming only — use organize-images to sort files into model/lora folders.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from PIL import Image
|
||||
except ImportError:
|
||||
print("Error: Pillow (PIL) is required. Install with: pip install Pillow")
|
||||
sys.exit(1)
|
||||
|
||||
# ── stopwords and filter sets ──────────────────────────────────────────────
|
||||
|
||||
QUALITY_TAGS = {
|
||||
"score_6_up", "score_7_up", "score_8_up", "score_9",
|
||||
"score_6", "score_7", "score_8",
|
||||
"masterpiece", "best quality", "good quality", "normal quality",
|
||||
"high quality", "highly detailed", "very detailed", "extreme detail",
|
||||
"very_aesthetic", "absurdres", "8k", "4k",
|
||||
"photorealistic", "photograph",
|
||||
"depth of field", "solo focus", "cinematic",
|
||||
"newest", "amazing", "stunning",
|
||||
}
|
||||
|
||||
QUALITY_WORDS = {
|
||||
"best", "good", "high", "top", "ultra", "super",
|
||||
"mega", "hyper", "extreme", "extra", "ultimate",
|
||||
}
|
||||
|
||||
TECHNICAL_WORDS = {
|
||||
"detailed", "focus", "quality", "aesthetic", "realistic",
|
||||
"cinematic", "lighting", "rendering", "shading", "texture",
|
||||
"newest", "absurdres",
|
||||
}
|
||||
|
||||
STOP_WORDS = {
|
||||
"the", "a", "an", "of", "in", "on", "at", "to", "for", "with",
|
||||
"and", "or", "is", "are", "was", "were", "be", "been", "being",
|
||||
"have", "has", "had", "do", "does", "did", "will", "would",
|
||||
"could", "should", "may", "might", "can", "shall", "this",
|
||||
"that", "these", "those", "it", "its", "by", "from", "as",
|
||||
"into", "through", "during", "before", "after", "above", "below",
|
||||
"between", "out", "off", "over", "under", "again", "further",
|
||||
"then", "once", "here", "there", "when", "where", "why", "how",
|
||||
"all", "each", "every", "both", "few", "more", "most", "other",
|
||||
"some", "such", "no", "nor", "not", "only", "own", "same", "so",
|
||||
"than", "too", "very", "just", "about", "up", "down",
|
||||
"make", "get", "set", "put", "take", "give", "show", "use",
|
||||
"like", "look", "see", "want", "need", "let", "close", "full",
|
||||
"add", "new", "one", "two", "five",
|
||||
"also", "well", "back", "still", "even", "much",
|
||||
"you", "your", "my", "me", "we", "our", "they", "them", "their",
|
||||
}
|
||||
|
||||
GENERIC_WORDS = {
|
||||
"man", "men", "guy", "guys", "boy", "boys", "woman", "women",
|
||||
"girl", "girls", "people", "person", "human", "figure",
|
||||
"photo", "image", "picture", "shot", "view", "pose", "posing",
|
||||
"face", "head", "body", "skin", "hair", "eyes", "hand", "hands",
|
||||
"dark", "light", "bright", "color", "colour",
|
||||
}
|
||||
|
||||
NEGATIVE_INDICATORS = {
|
||||
"deformed", "distorted", "disfigured", "poorly drawn", "bad anatomy",
|
||||
"extra digits", "missing digits", "extra limbs", "missing limbs",
|
||||
"ugly", "tiling", "low quality", "worst quality", "normal quality",
|
||||
"lowres", "monochrome", "grayscale", "text", "watermark",
|
||||
"branding", "border", "cropped", "signature", "username",
|
||||
"error", "mutation", "mutated", "out of frame", "duplicate", "cloned",
|
||||
"body out of frame", "bad hands", "bad face", "blurry",
|
||||
}
|
||||
|
||||
# Generic ComfyUI output filenames (ComfyUI_00042_.png, ComfyUI_temp_….png)
|
||||
GENERIC_RE = re.compile(r"^ComfyUI", re.IGNORECASE)
|
||||
|
||||
# ── Helpers ────────────────────────────────────────────────────────────────
|
||||
|
||||
def spinner():
|
||||
chars = "⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏"
|
||||
i = 0
|
||||
while True:
|
||||
yield chars[i % len(chars)]
|
||||
i += 1
|
||||
|
||||
|
||||
def is_negative_text(text):
|
||||
lower = text.lower()
|
||||
score = 0
|
||||
for ind in NEGATIVE_INDICATORS:
|
||||
if ind in lower:
|
||||
score += 1
|
||||
return score >= 2
|
||||
|
||||
|
||||
def is_quality_only(text):
|
||||
lower = text.lower()
|
||||
words = re.findall(r"[a-z_]+", lower)
|
||||
if not words:
|
||||
return False
|
||||
meaningful = sum(1 for w in words if w not in QUALITY_TAGS and len(w) > 2)
|
||||
return meaningful == 0
|
||||
|
||||
|
||||
# ── Metadata extraction from ComfyUI prompt JSON ───────────────────────────
|
||||
|
||||
def load_prompt_data(filepath):
|
||||
"""Return parsed ComfyUI prompt JSON from a PNG, or None."""
|
||||
try:
|
||||
img = Image.open(filepath)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
if "prompt" not in img.info:
|
||||
return None
|
||||
|
||||
try:
|
||||
return json.loads(img.info["prompt"])
|
||||
except (json.JSONDecodeError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
def extract_prompts(data):
|
||||
"""Return (positive_prompt, negative_prompt) from parsed prompt JSON."""
|
||||
# Gather all text fields from all nodes
|
||||
candidates = []
|
||||
for node in data.values():
|
||||
inputs = node.get("inputs", {})
|
||||
for field in ("text", "prompt", "positive", "negative", "value", "string"):
|
||||
val = inputs.get(field, "")
|
||||
if isinstance(val, str) and len(val.strip()) > 3:
|
||||
candidates.append(val.strip())
|
||||
|
||||
# Second pass: resolve node references
|
||||
for node in data.values():
|
||||
inputs = node.get("inputs", {})
|
||||
for field in ("text", "prompt", "positive", "negative", "value", "string"):
|
||||
val = inputs.get(field)
|
||||
if isinstance(val, list) and len(val) == 2 and isinstance(val[0], str):
|
||||
ref_node = data.get(val[0], {})
|
||||
ref_inputs = ref_node.get("inputs", {})
|
||||
for rf in ("value", "text", "string"):
|
||||
rv = ref_inputs.get(rf, "")
|
||||
if isinstance(rv, str) and len(rv.strip()) > 3:
|
||||
candidates.append(rv.strip())
|
||||
break
|
||||
|
||||
if not candidates:
|
||||
return None, None
|
||||
|
||||
positives = [c for c in candidates if not is_negative_text(c)]
|
||||
negatives = [c for c in candidates if is_negative_text(c)]
|
||||
|
||||
positives = [c for c in positives if not is_quality_only(c)]
|
||||
|
||||
pos = max(positives, key=len) if positives else None
|
||||
neg = max(negatives, key=len) if negatives else None
|
||||
return pos, neg
|
||||
|
||||
|
||||
def extract_all_candidates(text):
|
||||
"""Extract all candidate keywords with position index for frequency scoring."""
|
||||
if not text:
|
||||
return []
|
||||
|
||||
text_lower = text.lower()
|
||||
cleaned = re.sub(r"[\[\(\{][^\]\)\}]*[\]\)\}]", "", text_lower)
|
||||
segments = re.split(r"[,.;:!?]+", cleaned)
|
||||
|
||||
seen = set()
|
||||
candidates = []
|
||||
position = 0
|
||||
|
||||
for seg in segments:
|
||||
seg = seg.strip()
|
||||
if not seg or len(seg) < 4:
|
||||
continue
|
||||
|
||||
words = re.findall(r"[a-zA-Z_]+", seg)
|
||||
good = []
|
||||
for w in words:
|
||||
wl = w.lower().strip("_")
|
||||
if len(wl) < 3:
|
||||
continue
|
||||
if wl in STOP_WORDS or wl in GENERIC_WORDS:
|
||||
continue
|
||||
if wl in QUALITY_TAGS or wl in QUALITY_WORDS:
|
||||
continue
|
||||
if wl in TECHNICAL_WORDS:
|
||||
continue
|
||||
if wl in NEGATIVE_INDICATORS:
|
||||
continue
|
||||
if wl.startswith("score") or wl.startswith("step"):
|
||||
continue
|
||||
if wl.isdigit():
|
||||
continue
|
||||
good.append(wl)
|
||||
|
||||
if good:
|
||||
taken = 0
|
||||
for g in good:
|
||||
if g not in seen and taken < 2:
|
||||
candidates.append((g, position))
|
||||
seen.add(g)
|
||||
taken += 1
|
||||
position += 1
|
||||
|
||||
return candidates
|
||||
|
||||
|
||||
def select_best_keywords(candidates, freq_map, total_images, max_keywords=5):
|
||||
if not candidates:
|
||||
return []
|
||||
|
||||
if total_images <= 1:
|
||||
return [kw for kw, _ in candidates[:max_keywords]]
|
||||
|
||||
scored = []
|
||||
for kw, pos in candidates:
|
||||
freq = freq_map.get(kw, 1)
|
||||
rarity = 1.0 - ((freq - 1) / max(total_images - 1, 1))
|
||||
max_pos = max(len(candidates), 1)
|
||||
pos_bonus = (pos / max_pos) * 0.3
|
||||
score = rarity + pos_bonus
|
||||
scored.append((score, kw, pos))
|
||||
|
||||
scored.sort(key=lambda x: (-x[0], x[2]))
|
||||
|
||||
selected = []
|
||||
seen = set()
|
||||
for _, kw, _ in scored:
|
||||
if kw not in seen:
|
||||
selected.append(kw)
|
||||
seen.add(kw)
|
||||
if len(selected) >= max_keywords:
|
||||
break
|
||||
|
||||
return selected
|
||||
|
||||
|
||||
def keywords_to_name(keywords):
|
||||
if not keywords:
|
||||
return None
|
||||
|
||||
desc = "-".join(keywords)
|
||||
desc = re.sub(r"[^a-z0-9-]", "-", desc)
|
||||
desc = re.sub(r"-+", "-", desc).strip("-")
|
||||
|
||||
if len(desc) > 40:
|
||||
desc = desc[:40].rstrip("-")
|
||||
if "-" in desc:
|
||||
truncated = "-".join(desc.split("-")[:-1])
|
||||
if truncated and len(truncated) > 10:
|
||||
desc = truncated
|
||||
|
||||
return f"schmeeve-AI-{desc}.png" if desc and len(desc) > 3 else None
|
||||
|
||||
|
||||
# ── main ───────────────────────────────────────────────────────────────────
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Scan PNGs for AI prompt metadata and rename in place. "
|
||||
"Does not move files — use organize-images for sorting.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"path", nargs="?", default=os.path.expanduser("~/Pictures"),
|
||||
help="Directory to scan for PNG files (default: ~/Pictures)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-n", "--no-interactive", action="store_true",
|
||||
help="Auto-rename without prompting",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-r", "--recursive", action="store_true",
|
||||
help="Search directories recursively (default: off)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-p", "--preview", "--dry-run", action="store_true",
|
||||
dest="dry_run",
|
||||
help="Show proposed changes without making any",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--include-renamed", action="store_true",
|
||||
help="Also process files already named schmeeve-AI-* (default: skip them)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--only-generic", action="store_true",
|
||||
help="Only rename generic ComfyUI_* filenames",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
scan_dir = Path(args.path).expanduser().resolve()
|
||||
if not scan_dir.is_dir():
|
||||
print(f"Error: {scan_dir} is not a directory")
|
||||
sys.exit(1)
|
||||
|
||||
# Gather PNGs
|
||||
glob_pattern = "**/*.png" if args.recursive else "*.png"
|
||||
pngs = sorted(scan_dir.glob(glob_pattern))
|
||||
|
||||
if not pngs:
|
||||
print(f" No PNG files found in {scan_dir}" +
|
||||
(" (recursive search)" if args.recursive else ""))
|
||||
return
|
||||
|
||||
# ── Phase 1: Collect candidates and build frequency map ──
|
||||
sys.stdout.write(" Analyzing PNGs")
|
||||
sys.stdout.flush()
|
||||
spin = spinner()
|
||||
image_data = {}
|
||||
word_images = {}
|
||||
for p in pngs:
|
||||
sys.stdout.write(f"\r {next(spin)} Analyzing PNGs")
|
||||
sys.stdout.flush()
|
||||
# Skip already-renamed files (schmeeve-AI-*) unless requested
|
||||
if not args.include_renamed and p.name.startswith("schmeeve-AI-"):
|
||||
continue
|
||||
if args.only_generic and not GENERIC_RE.match(p.name):
|
||||
continue
|
||||
data = load_prompt_data(str(p))
|
||||
if not data:
|
||||
continue
|
||||
pos, neg = extract_prompts(data)
|
||||
source = pos or neg
|
||||
if source:
|
||||
candidates = extract_all_candidates(source)
|
||||
if candidates:
|
||||
image_data[p] = candidates
|
||||
for kw, _ in candidates:
|
||||
if kw not in word_images:
|
||||
word_images[kw] = set()
|
||||
word_images[kw].add(p)
|
||||
|
||||
freq_map = {kw: len(images) for kw, images in word_images.items()}
|
||||
total_images = len(image_data)
|
||||
|
||||
# ── Phase 1b: Propose new names (in place — same directory) ──
|
||||
raw_proposals = {}
|
||||
for p, candidates in image_data.items():
|
||||
if total_images > 1:
|
||||
keywords = select_best_keywords(candidates, freq_map, total_images)
|
||||
else:
|
||||
keywords = [kw for kw, _ in candidates[:5]]
|
||||
name = keywords_to_name(keywords)
|
||||
if not name or name == p.name:
|
||||
continue
|
||||
raw_proposals[p] = name
|
||||
|
||||
# Clear the spinner line
|
||||
sys.stdout.write("\r" + " " * 60 + "\r")
|
||||
sys.stdout.flush()
|
||||
|
||||
# Deduplicate file names within their directories
|
||||
proposals = {}
|
||||
used_paths = {}
|
||||
for p, name in raw_proposals.items():
|
||||
new_path = p.with_name(name)
|
||||
key = str(new_path)
|
||||
counter = 0
|
||||
while key in used_paths:
|
||||
counter += 1
|
||||
ts = time.strftime("%m%d-%H%M%S", time.localtime(p.stat().st_mtime))
|
||||
stem = new_path.stem
|
||||
ext = new_path.suffix
|
||||
new_path = new_path.with_name(f"{stem}-{ts}{'_' + str(counter) if counter > 1 else ''}{ext}")
|
||||
key = str(new_path)
|
||||
used_paths[key] = True
|
||||
proposals[p] = new_path
|
||||
|
||||
# ── Phase 2: Display proposed changes ──
|
||||
if not proposals:
|
||||
print(" No renamable PNGs found.")
|
||||
return
|
||||
|
||||
print(f"\n {'DRY RUN' if args.dry_run else 'PROPOSED'} — "
|
||||
f"{len(proposals)} file(s) to rename in place\n")
|
||||
|
||||
for i, (old_path, new_path) in enumerate(proposals.items(), 1):
|
||||
try:
|
||||
src_display = str(old_path.relative_to(scan_dir))
|
||||
except ValueError:
|
||||
src_display = str(old_path)
|
||||
if len(src_display) > 55:
|
||||
src_display = src_display[:25] + "…" + src_display[-27:]
|
||||
print(f" {i:>3}. {src_display}")
|
||||
print(f" → {new_path.name}")
|
||||
|
||||
print()
|
||||
|
||||
if args.dry_run:
|
||||
print(f" Dry-run complete. No files were changed.\n")
|
||||
return
|
||||
|
||||
def do_rename(old_path, new_path):
|
||||
"""Rename with collision suffix. Returns final path or None on failure."""
|
||||
if old_path == new_path:
|
||||
return new_path
|
||||
if new_path.exists():
|
||||
stem = new_path.stem
|
||||
ext = new_path.suffix
|
||||
counter = 1
|
||||
while new_path.exists():
|
||||
new_path = new_path.with_name(f"{stem}_{counter}{ext}")
|
||||
counter += 1
|
||||
print(f" (name existed, using {new_path.name})")
|
||||
try:
|
||||
old_path.rename(new_path)
|
||||
return new_path
|
||||
except (FileNotFoundError, OSError) as e:
|
||||
print(f" SKIPPED ({e}): {old_path.name}")
|
||||
return None
|
||||
|
||||
# ── Phase 3: Rename (interactive or auto) ──
|
||||
if args.no_interactive:
|
||||
renamed = 0
|
||||
for old_path, new_path in proposals.items():
|
||||
if do_rename(old_path, new_path):
|
||||
renamed += 1
|
||||
print(f" Renamed {renamed} file(s).")
|
||||
else:
|
||||
renamed = 0
|
||||
skipped = 0
|
||||
items = list(proposals.items())
|
||||
i = 0
|
||||
while i < len(items):
|
||||
old_path, new_path = items[i]
|
||||
|
||||
print(f"\n [{i+1}/{len(items)}]")
|
||||
print(f" From: {old_path.name}")
|
||||
print(f" To: {new_path.name}")
|
||||
remaining = len(items) - i - 1
|
||||
rlabel = f"rename all {remaining}" if remaining else ""
|
||||
sys.stdout.write(f" [Enter]=rename [e]=edit [s]=skip{' [a]=' + rlabel if rlabel else ''} [q]=quit: ")
|
||||
sys.stdout.flush()
|
||||
choice = sys.stdin.readline().strip().lower()
|
||||
|
||||
if choice == "q":
|
||||
remaining = len(items) - i - 1
|
||||
if remaining:
|
||||
print(f" Skipping remaining {remaining} file(s).")
|
||||
break
|
||||
elif choice == "s":
|
||||
skipped += 1
|
||||
i += 1
|
||||
continue
|
||||
elif choice == "e":
|
||||
sys.stdout.write(f" Edit keywords (name becomes 'schmeeve-AI-...'): ")
|
||||
sys.stdout.flush()
|
||||
custom = sys.stdin.readline().strip()
|
||||
if custom:
|
||||
custom_desc = re.sub(r"[^a-z0-9-]", "-", custom.lower())
|
||||
custom_desc = re.sub(r"-+", "-", custom_desc).strip("-")
|
||||
if custom_desc:
|
||||
new_path = old_path.with_name(f"schmeeve-AI-{custom_desc}.png")
|
||||
else:
|
||||
print(" Invalid name, skipping.")
|
||||
skipped += 1
|
||||
i += 1
|
||||
continue
|
||||
else:
|
||||
skipped += 1
|
||||
i += 1
|
||||
continue
|
||||
elif choice == "":
|
||||
pass
|
||||
elif choice == "a":
|
||||
for j in range(i, len(items)):
|
||||
p, np = items[j]
|
||||
if do_rename(p, np):
|
||||
renamed += 1
|
||||
break
|
||||
else:
|
||||
print(f" Unknown option '{choice}', skipping.")
|
||||
skipped += 1
|
||||
i += 1
|
||||
continue
|
||||
|
||||
if do_rename(old_path, new_path):
|
||||
renamed += 1
|
||||
i += 1
|
||||
|
||||
print(f"\n Renamed: {renamed} Skipped: {skipped}")
|
||||
|
||||
# Check for JPEGs with AI metadata
|
||||
jpg_pattern = "**/*.jpg" if args.recursive else "*.jpg"
|
||||
remaining_ai = 0
|
||||
for f in scan_dir.glob(jpg_pattern):
|
||||
try:
|
||||
img = Image.open(f)
|
||||
if "prompt" in img.info:
|
||||
remaining_ai += 1
|
||||
except Exception:
|
||||
pass
|
||||
if remaining_ai:
|
||||
print(f" Note: {remaining_ai} JPEG(s) with AI metadata found (not yet supported).")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -3,6 +3,9 @@
|
||||
# sync-comfyui-snaps — Pull new images from nimo.loc ComfyUI output, sync to NAS,
|
||||
# then rename AI snaps and organize.
|
||||
#
|
||||
# NOTE: superseded by comfyui-sync (systemd user timer on nimo.loc). Kept for
|
||||
# reference/manual use; the cron entry below is NOT currently installed.
|
||||
#
|
||||
# Crontab (every 2 hours):
|
||||
# 0 */2 * * * /home/schmeeve/.local/bin/sync-comfyui-snaps >> /home/schmeeve/.local/log/sync-comfyui-snaps.log 2>&1
|
||||
#
|
||||
@@ -21,7 +24,6 @@ NIMO_SRC="/home/schmeeve/ComfyUI/output"
|
||||
|
||||
SHARE="//mini.nas/miniShare1"
|
||||
NAS_SUBPATH="Pictures"
|
||||
MOUNTPOINT="${HOME}/mnt/miniShare1"
|
||||
CREDENTIALS="${HOME}/.smb/mini.nas"
|
||||
|
||||
RENAME_SCRIPT="${HOME}/git/schmeeve-toolz/rename-ai-snaps"
|
||||
@@ -30,7 +32,13 @@ ORGANIZE_SCRIPT="${HOME}/git/schmeeve-toolz/organize-images"
|
||||
START="$(date +%s)"
|
||||
echo "[sync-comfyui-snaps] Starting at $(date)"
|
||||
|
||||
# 1. Mount NAS if not already mounted
|
||||
# 1. Determine NAS mount — use pre-mounted /mini.nas/miniShare1 on nimo.loc
|
||||
# if available; otherwise mount via CIFS under ~/mnt/miniShare1
|
||||
if [ -d "/mini.nas/miniShare1" ]; then
|
||||
MOUNTPOINT="/mini.nas/miniShare1"
|
||||
echo "[sync-comfyui-snaps] Using existing mount at ${MOUNTPOINT}"
|
||||
else
|
||||
MOUNTPOINT="${HOME}/mnt/miniShare1"
|
||||
if mount | grep -q "${MOUNTPOINT}"; then
|
||||
echo "[sync-comfyui-snaps] Already mounted at ${MOUNTPOINT}"
|
||||
else
|
||||
@@ -47,6 +55,7 @@ else
|
||||
fi
|
||||
echo "[sync-comfyui-snaps] Mounted at ${MOUNTPOINT}"
|
||||
fi
|
||||
fi
|
||||
|
||||
DEST="${MOUNTPOINT}/${NAS_SUBPATH}"
|
||||
mkdir -p "${DEST}"
|
||||
@@ -66,18 +75,26 @@ rsync -vu --checksum \
|
||||
--progress --stats \
|
||||
"${SRC}" "${DEST}/"
|
||||
|
||||
# 3. Rename AI snaps (non-interactive for automation)
|
||||
# 3. Route fresh drops into Images/ (neatcli equivalent), then rename in place.
|
||||
# NOTE: rename-ai-snaps no longer takes --output; it only renames.
|
||||
# organize-images does the model/lora sorting.
|
||||
IMAGES_DIR="${DEST}/Images"
|
||||
mkdir -p "${IMAGES_DIR}"
|
||||
find "${DEST}" -maxdepth 1 -type f \
|
||||
\( -iname '*.png' -o -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.webp' \) \
|
||||
-exec mv -n {} "${IMAGES_DIR}/" \;
|
||||
|
||||
if [ -x "${RENAME_SCRIPT}" ]; then
|
||||
echo "[sync-comfyui-snaps] Running rename-ai-snaps on ${DEST}"
|
||||
"${RENAME_SCRIPT}" "${DEST}" --no-interactive
|
||||
echo "[sync-comfyui-snaps] Running rename-ai-snaps on ${IMAGES_DIR}"
|
||||
"${RENAME_SCRIPT}" "${IMAGES_DIR}" --no-interactive
|
||||
else
|
||||
echo "[sync-comfyui-snaps] WARNING: rename-ai-snaps not found at ${RENAME_SCRIPT}"
|
||||
fi
|
||||
|
||||
# 4. Organize images
|
||||
# 4. Organize images into AI/{model}/{lora}/
|
||||
if [ -x "${ORGANIZE_SCRIPT}" ]; then
|
||||
echo "[sync-comfyui-snaps] Running organize-images on ${DEST}"
|
||||
"${ORGANIZE_SCRIPT}" "${DEST}" -e
|
||||
echo "[sync-comfyui-snaps] Running organize-images on ${IMAGES_DIR}"
|
||||
"${ORGANIZE_SCRIPT}" "${IMAGES_DIR}" -e
|
||||
else
|
||||
echo "[sync-comfyui-snaps] WARNING: organize-images not found at ${ORGANIZE_SCRIPT}"
|
||||
fi
|
||||
|
||||
Reference in New Issue
Block a user