feat: 媒体库字幕重生成统计脚本(数量/时长/旧字幕备份)
模型与管线切换后需要把媒体库里旧管线生成的字幕整批重跑。批量引擎按"视频旁 已有字幕即 SKIPPED"判定,重跑前必须先统计范围、再把旧字幕改名备份,否则建出来 的任务会把所有视频全部跳过。 - `scripts/plan_regenerate_subtitles.py`:扫描媒体库,按 CN 产物的最早 mtime 判定 生成时间,统计待重生成的数量/时长/体积并输出 JSON 计划;`--backup-old [--apply] [--select]` 把旧字幕原地改名为 `<原名>.old-<日期>` 供批量重跑。脚本不依赖仓库, 可拷到媒体库主机直接跑(CIFS 挂载下逐文件 ffprobe 太慢)。 - 测试覆盖真实 10 秒视频与真实字幕文件,含 ffprobe 时长探测、分类边界、 备份幂等与选择清单解析。
This commit is contained in:
@@ -160,6 +160,40 @@
|
|||||||
- **环境变量**:`WOV_BATCH_ENABLED`(默认 1)、`WOV_BATCH_INTERVAL_SECONDS`
|
- **环境变量**:`WOV_BATCH_ENABLED`(默认 1)、`WOV_BATCH_INTERVAL_SECONDS`
|
||||||
(默认 1.0),完整列表见 [configuration.md](./configuration.md)。
|
(默认 1.0),完整列表见 [configuration.md](./configuration.md)。
|
||||||
|
|
||||||
|
## 媒体库历史字幕重建(统计 → 备份 → 批量重跑)
|
||||||
|
|
||||||
|
模型或管线切换后,媒体库里由旧管线生成的字幕需要整批重跑。批量引擎按
|
||||||
|
“视频旁已有字幕即 SKIPPED”判定,所以重跑前必须先统计范围、再把旧字幕改名
|
||||||
|
备份,否则建出来的任务会把所有视频全部跳过。工具是 `scripts/plan_regenerate_subtitles.py`
|
||||||
|
(**不依赖仓库**,可拷到媒体库主机直接跑:CIFS 挂载下逐文件 ffprobe 与
|
||||||
|
目录扫描都慢得多,在 NAS 本地跑 524 个视频只要几秒)。
|
||||||
|
|
||||||
|
生成时间的判定口径:只看本流水线产出的 CN 产物(`<视频名>.CN.srt`、
|
||||||
|
`<视频名>.CN_dual_eye.ass`),取它们**最早**的 mtime——双目 `.ass` 可能被样式
|
||||||
|
统一脚本原地改写而“变新”,中文字幕的 mtime 才是真实生成时间。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1) 统计:数量 / 时长 / 体积 / 按月分布,并写出逐条明细 JSON
|
||||||
|
python3 scripts/plan_regenerate_subtitles.py /vol1/1000/123 \
|
||||||
|
--before 2026-09-01 --out data/experiments/regen_plan/plan_2026-09-01.json
|
||||||
|
|
||||||
|
# 2) 备份旧字幕(默认预览,--apply 才落盘):改名 <原名>.old-<当天日期>
|
||||||
|
python3 scripts/plan_regenerate_subtitles.py /vol1/1000/123 --backup-old
|
||||||
|
python3 scripts/plan_regenerate_subtitles.py /vol1/1000/123 --backup-old --apply \
|
||||||
|
--select sample10.txt # 只处理清单里的视频,支持 # 注释
|
||||||
|
|
||||||
|
# 3) 建批量任务(批量页或 POST /api/batch/jobs)
|
||||||
|
```
|
||||||
|
|
||||||
|
- 备份文件名带 `.old-<日期>` 后缀,扩展名不再是字幕后缀,批量引擎不会再把它当
|
||||||
|
旁挂字幕;确认新字幕无误后删除备份,需要回退则去掉后缀改回原名。
|
||||||
|
- 未备份的视频仍有旁挂字幕 → 创建任务时记 SKIPPED,因此“整库建一个任务”即可,
|
||||||
|
实际只会处理被备份的那批;已重跑的视频产出新 CN 字幕(mtime 变新),全量重建
|
||||||
|
时自动归入“已是最新”,不会重复处理。
|
||||||
|
- 实测数据(380 个待重生成 / 207.88 小时;10 个样本 6.54 小时素材跑 77.7 分钟、
|
||||||
|
GPU 平均 289.6 W、约 0.375 kWh,全量推算约 41 小时 / 整机 15 kWh)见
|
||||||
|
`data/experiments/regen_plan/REPORT.md`。
|
||||||
|
|
||||||
## 相关文档
|
## 相关文档
|
||||||
|
|
||||||
- 环境变量全表:[configuration.md](./configuration.md)
|
- 环境变量全表:[configuration.md](./configuration.md)
|
||||||
|
|||||||
@@ -51,6 +51,7 @@ tests/
|
|||||||
├── web/test_crop/ # 对应 web/assets/(框选几何换算)
|
├── web/test_crop/ # 对应 web/assets/(框选几何换算)
|
||||||
├── web/test_batch/ # 对应 web/assets/(批量页渲染)
|
├── web/test_batch/ # 对应 web/assets/(批量页渲染)
|
||||||
├── scripts/test_fix_zombie_batch_jobs/ # 对应 scripts/(僵尸批量任务修复)
|
├── scripts/test_fix_zombie_batch_jobs/ # 对应 scripts/(僵尸批量任务修复)
|
||||||
|
├── scripts/test_plan_regenerate_subtitles/ # 对应 scripts/(字幕重生成计划统计)
|
||||||
└── shared/ # 跨模块公共设施
|
└── shared/ # 跨模块公共设施
|
||||||
├── realdata_contract.py # 真实数据契约与对齐量化
|
├── realdata_contract.py # 真实数据契约与对齐量化
|
||||||
├── srt_entries.py # 按秒解析 SRT
|
├── srt_entries.py # 按秒解析 SRT
|
||||||
@@ -161,6 +162,7 @@ tests/
|
|||||||
| `web/assets/crop.js` | 框选几何换算 | `tests/web/test_crop/`(真实 node 执行) | 已覆盖 |
|
| `web/assets/crop.js` | 框选几何换算 | `tests/web/test_crop/`(真实 node 执行) | 已覆盖 |
|
||||||
| `web/assets/batch.js` | 批量页明细/进度渲染 | `tests/web/test_batch/`(真实 node 执行) | 已覆盖 |
|
| `web/assets/batch.js` | 批量页明细/进度渲染 | `tests/web/test_batch/`(真实 node 执行) | 已覆盖 |
|
||||||
| `scripts/fix_zombie_batch_jobs.py` | 僵尸批量任务诊断与修复 | `tests/scripts/test_fix_zombie_batch_jobs/` | 已覆盖 |
|
| `scripts/fix_zombie_batch_jobs.py` | 僵尸批量任务诊断与修复 | `tests/scripts/test_fix_zombie_batch_jobs/` | 已覆盖 |
|
||||||
|
| `scripts/plan_regenerate_subtitles.py` | 媒体库字幕重生成计划统计(数量/时长/分类) | `tests/scripts/test_plan_regenerate_subtitles/` | 已覆盖 |
|
||||||
| `tests/shared/srt_entries.py` | 按秒解析 SRT(测试公共设施) | `tests/shared/test_srt_entries/` | 已覆盖 |
|
| `tests/shared/srt_entries.py` | 按秒解析 SRT(测试公共设施) | `tests/shared/test_srt_entries/` | 已覆盖 |
|
||||||
| `tests/shared/realdata_contract.py` | 真实数据契约与对齐量化 | `tests/shared/test_alignment/` | 已覆盖 |
|
| `tests/shared/realdata_contract.py` | 真实数据契约与对齐量化 | `tests/shared/test_alignment/` | 已覆盖 |
|
||||||
| `tests/shared/env_isolation.py` | 环境/临时目录隔离 | 被 `tests/app/test_config` 等间接覆盖 | 已覆盖(间接) |
|
| `tests/shared/env_isolation.py` | 环境/临时目录隔离 | 被 `tests/app/test_config` 等间接覆盖 | 已覆盖(间接) |
|
||||||
|
|||||||
@@ -0,0 +1,320 @@
|
|||||||
|
"""统计媒体库中需要重新生成字幕的视频数量与时长,产出可复用的重生成计划。
|
||||||
|
|
||||||
|
判定口径(`--before` 可调):
|
||||||
|
|
||||||
|
- 只看本流水线产出的 CN 字幕:`<视频名>.CN.srt` 与 `<视频名>.CN_dual_eye.ass`;
|
||||||
|
- 生成时间取这些产物中**最早**的 mtime。双目 `.ass` 可能被样式统一脚本原地
|
||||||
|
改写而"变新",中文字幕的 mtime 才反映真实生成时间,取最早值两者兼容;
|
||||||
|
- 生成时间早于 `--before`(默认 2026-09-01,本地时区)即列为待重新生成;
|
||||||
|
- 完全没有 CN 字幕的视频单独统计,不算"重生成"(那属于首次生成)。
|
||||||
|
|
||||||
|
脚本不依赖仓库,可拷到媒体库所在主机直接跑(共享目录挂载方式下逐文件
|
||||||
|
ffprobe 太慢):目录内容只读,不改动任何文件;结果写 JSON 计划供批量重跑。
|
||||||
|
|
||||||
|
用法:
|
||||||
|
python3 scripts/plan_regenerate_subtitles.py /mnt/fnOS/123
|
||||||
|
python3 scripts/plan_regenerate_subtitles.py /vol1/1000/123 --before 2026-09-01 \
|
||||||
|
--out data/experiments/regen_plan.json
|
||||||
|
|
||||||
|
批量引擎看到视频旁已有字幕就会跳过该视频,重生成前需先改名备份旧字幕:
|
||||||
|
|
||||||
|
python3 scripts/plan_regenerate_subtitles.py /vol1/1000/123 --backup-old
|
||||||
|
python3 scripts/plan_regenerate_subtitles.py /vol1/1000/123 --backup-old --apply \
|
||||||
|
--select sample.txt
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# 与 src/wov_app/batch.py 的 VIDEO_EXTENSIONS / SUBTITLE_EXTENSIONS 保持一致,
|
||||||
|
# 本脚本要在媒体库主机上独立运行,故不复用仓库常量。
|
||||||
|
VIDEO_EXTENSIONS = {
|
||||||
|
".mp4", ".mkv", ".avi", ".mov", ".webm", ".flv", ".ts",
|
||||||
|
".m4v", ".wmv", ".mpg", ".mpeg", ".3gp",
|
||||||
|
}
|
||||||
|
SUBTITLE_EXTENSIONS = {".srt", ".ass", ".ssa", ".vtt"}
|
||||||
|
|
||||||
|
# CN 产物识别:这两个后缀由本流水线生成(见 batch.py 的产物命名约定);
|
||||||
|
# 媒体库里其它 .srt 多为片源自带字幕,不参与"字幕生成时间"判定。
|
||||||
|
CN_PRODUCT_MARKERS = (".cn.srt", ".cn_dual_eye.ass")
|
||||||
|
|
||||||
|
PENDING = "pending"
|
||||||
|
CURRENT = "current"
|
||||||
|
MISSING = "missing"
|
||||||
|
|
||||||
|
|
||||||
|
def iter_videos(root: Path) -> list[Path]:
|
||||||
|
"""递归列出媒体库下全部视频文件,按路径排序保证结果可复现。"""
|
||||||
|
paths = [
|
||||||
|
p for p in root.rglob("*")
|
||||||
|
if p.is_file() and p.suffix.lower() in VIDEO_EXTENSIONS
|
||||||
|
]
|
||||||
|
return sorted(paths)
|
||||||
|
|
||||||
|
|
||||||
|
def cn_products(video: Path) -> list[Path]:
|
||||||
|
"""列出视频旁由本流水线产出的 CN 字幕(同一目录、文件名含视频主名)。"""
|
||||||
|
stem = video.stem.lower()
|
||||||
|
found: list[Path] = []
|
||||||
|
for item in video.parent.iterdir():
|
||||||
|
if not item.is_file():
|
||||||
|
continue
|
||||||
|
name = item.name.lower()
|
||||||
|
if item.suffix.lower() not in SUBTITLE_EXTENSIONS:
|
||||||
|
continue
|
||||||
|
if stem in name and name.endswith(CN_PRODUCT_MARKERS):
|
||||||
|
found.append(item)
|
||||||
|
return sorted(found)
|
||||||
|
|
||||||
|
|
||||||
|
def sidecar_subtitles(video: Path) -> list[Path]:
|
||||||
|
"""列出批量引擎认定为“已有字幕”的旁挂字幕文件(与 batch.py 判定一致)。
|
||||||
|
|
||||||
|
同目录、字幕扩展名、文件名包含视频主名;主名只有单个字符时只接受
|
||||||
|
“主名.”前缀,避免 a.mp4 误配 apple.srt。
|
||||||
|
"""
|
||||||
|
stem = video.stem.lower()
|
||||||
|
try:
|
||||||
|
siblings = list(video.parent.iterdir())
|
||||||
|
except OSError:
|
||||||
|
return []
|
||||||
|
found: list[Path] = []
|
||||||
|
for item in siblings:
|
||||||
|
try:
|
||||||
|
if not item.is_file() or item.suffix.lower() not in SUBTITLE_EXTENSIONS:
|
||||||
|
continue
|
||||||
|
except OSError:
|
||||||
|
continue
|
||||||
|
name = item.name.lower()
|
||||||
|
if len(stem) <= 1:
|
||||||
|
matched = name.startswith(stem + ".")
|
||||||
|
else:
|
||||||
|
matched = stem in name
|
||||||
|
if matched:
|
||||||
|
found.append(item)
|
||||||
|
return sorted(found)
|
||||||
|
|
||||||
|
|
||||||
|
def read_select_list(path: Path) -> set[str]:
|
||||||
|
"""读取 `--select` 文件里的视频标识(每行一个,支持行首或行尾 `#` 注释)。"""
|
||||||
|
keys: set[str] = set()
|
||||||
|
for line in Path(path).read_text(encoding="utf-8").splitlines():
|
||||||
|
text = line.split(" #", 1)[0].strip()
|
||||||
|
if text and not text.startswith("#"):
|
||||||
|
keys.add(text)
|
||||||
|
return keys
|
||||||
|
|
||||||
|
|
||||||
|
def backup_old_subtitles(items: list[dict], suffix: str, apply: bool = False,
|
||||||
|
select: set[str] | None = None) -> list[tuple[Path, Path]]:
|
||||||
|
"""把待重生成视频的旁挂字幕原地改名,让批量引擎不再把该视频判为已有字幕。
|
||||||
|
|
||||||
|
改名目标为 `<原名><suffix>`;目标已存在(同一批重复执行)时跳过,不覆盖
|
||||||
|
旧备份。apply=False 只返回改名为计划,不动磁盘。select 按绝对路径、
|
||||||
|
相对路径或文件名筛选要处理的视频。
|
||||||
|
"""
|
||||||
|
changed: list[tuple[Path, Path]] = []
|
||||||
|
for item in items:
|
||||||
|
video = Path(item["video"])
|
||||||
|
if select is not None:
|
||||||
|
keys = (str(video), item.get("rel"), video.name)
|
||||||
|
if not any(key in select for key in keys if key):
|
||||||
|
continue
|
||||||
|
for sub in sidecar_subtitles(video):
|
||||||
|
target = sub.with_name(sub.name + suffix)
|
||||||
|
if target.exists():
|
||||||
|
continue
|
||||||
|
if apply:
|
||||||
|
sub.rename(target)
|
||||||
|
changed.append((sub, target))
|
||||||
|
return changed
|
||||||
|
|
||||||
|
|
||||||
|
def probe_duration(video: Path, timeout: float = 120.0) -> float | None:
|
||||||
|
"""用 ffprobe 读取容器时长(秒);失败返回 None,由调用方单独计数。"""
|
||||||
|
try:
|
||||||
|
proc = subprocess.run(
|
||||||
|
[
|
||||||
|
"ffprobe", "-v", "error",
|
||||||
|
"-show_entries", "format=duration",
|
||||||
|
"-of", "default=nw=1:nk=1", str(video),
|
||||||
|
],
|
||||||
|
capture_output=True, text=True, timeout=timeout,
|
||||||
|
)
|
||||||
|
except (OSError, subprocess.TimeoutExpired):
|
||||||
|
return None
|
||||||
|
if proc.returncode != 0:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return float(proc.stdout.strip())
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def build_items(root: Path, workers: int, probe: bool = True) -> list[dict]:
|
||||||
|
"""扫描全部视频,逐条产出路径、大小、时长与 CN 字幕生成时间。"""
|
||||||
|
videos = iter_videos(root)
|
||||||
|
durations: list[float | None] = [None] * len(videos)
|
||||||
|
if probe and videos:
|
||||||
|
with ThreadPoolExecutor(max_workers=workers) as pool:
|
||||||
|
durations = list(pool.map(probe_duration, videos))
|
||||||
|
|
||||||
|
items: list[dict] = []
|
||||||
|
for video, duration in zip(videos, durations):
|
||||||
|
products = cn_products(video)
|
||||||
|
mtimes = [p.stat().st_mtime for p in products]
|
||||||
|
items.append({
|
||||||
|
"video": str(video),
|
||||||
|
"rel": str(video.relative_to(root)),
|
||||||
|
"size": video.stat().st_size,
|
||||||
|
"duration": duration,
|
||||||
|
"generated_at": datetime.fromtimestamp(min(mtimes)).isoformat() if mtimes else None,
|
||||||
|
"cn_products": [
|
||||||
|
{"path": str(p), "mtime": datetime.fromtimestamp(p.stat().st_mtime).isoformat()}
|
||||||
|
for p in products
|
||||||
|
],
|
||||||
|
})
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
|
def classify(items: list[dict], before: datetime) -> dict[str, list[dict]]:
|
||||||
|
"""按 CN 字幕生成时间把视频分为待重生成/已最新/无 CN 字幕三组。"""
|
||||||
|
groups: dict[str, list[dict]] = {PENDING: [], CURRENT: [], MISSING: []}
|
||||||
|
for item in items:
|
||||||
|
if item["generated_at"] is None:
|
||||||
|
groups[MISSING].append(item)
|
||||||
|
elif datetime.fromisoformat(item["generated_at"]) < before:
|
||||||
|
groups[PENDING].append(item)
|
||||||
|
else:
|
||||||
|
groups[CURRENT].append(item)
|
||||||
|
return groups
|
||||||
|
|
||||||
|
|
||||||
|
def _agg(items: list[dict]) -> dict:
|
||||||
|
"""汇总一组的数量、总时长、总大小,时长缺失的视频单独计数。"""
|
||||||
|
return {
|
||||||
|
"count": len(items),
|
||||||
|
"seconds": sum(i["duration"] for i in items if i["duration"] is not None),
|
||||||
|
"bytes": sum(i["size"] for i in items),
|
||||||
|
"no_duration": sum(1 for i in items if i["duration"] is None),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _fmt_hours(seconds: float) -> str:
|
||||||
|
return f"{seconds / 3600:.2f} 小时"
|
||||||
|
|
||||||
|
|
||||||
|
def _fmt_duration(probed: bool, seconds: float) -> str:
|
||||||
|
"""未探测时长时(--backup-old 等只统计数量的场景)不显示 0 小时。"""
|
||||||
|
return _fmt_hours(seconds) if probed else "未探测时长"
|
||||||
|
|
||||||
|
|
||||||
|
def _fmt_size(num_bytes: int) -> str:
|
||||||
|
return f"{num_bytes / (1024 ** 3):.1f} GiB"
|
||||||
|
|
||||||
|
|
||||||
|
def format_summary(root: Path, items: list[dict], groups: dict[str, list[dict]],
|
||||||
|
before: datetime) -> str:
|
||||||
|
"""生成人类可读的统计报告(数量 / 时长 / 体积 / 按月分布)。"""
|
||||||
|
probed = any(i["duration"] is not None for i in items)
|
||||||
|
lines = [
|
||||||
|
f"媒体库: {root}",
|
||||||
|
f"视频总数: {len(items)} 个 / {_fmt_duration(probed, _agg(items)['seconds'])} / "
|
||||||
|
f"{_fmt_size(_agg(items)['bytes'])}",
|
||||||
|
"",
|
||||||
|
f"待重新生成(CN 字幕早于 {before:%Y-%m-%d}): ",
|
||||||
|
]
|
||||||
|
pending = _agg(groups[PENDING])
|
||||||
|
duration_text = _fmt_duration(probed, pending["seconds"])
|
||||||
|
lines.append(f" {pending['count']} 个 / {duration_text} / {_fmt_size(pending['bytes'])}")
|
||||||
|
|
||||||
|
by_month: dict[str, dict] = {}
|
||||||
|
for item in groups[PENDING]:
|
||||||
|
month = item["generated_at"][:7]
|
||||||
|
bucket = by_month.setdefault(month, {"count": 0, "seconds": 0.0})
|
||||||
|
bucket["count"] += 1
|
||||||
|
bucket["seconds"] += item["duration"] or 0.0
|
||||||
|
for month in sorted(by_month):
|
||||||
|
bucket = by_month[month]
|
||||||
|
lines.append(f" {month}: {bucket['count']} 个 / "
|
||||||
|
f"{_fmt_duration(probed, bucket['seconds'])}")
|
||||||
|
|
||||||
|
current = _agg(groups[CURRENT])
|
||||||
|
lines.append("")
|
||||||
|
lines.append(f"已是最新(生成时间 >= {before:%Y-%m-%d}): {current['count']} 个 / "
|
||||||
|
f"{_fmt_duration(probed, current['seconds'])} / {_fmt_size(current['bytes'])}")
|
||||||
|
missing = _agg(groups[MISSING])
|
||||||
|
lines.append(f"无 CN 字幕(首次生成): {missing['count']} 个 / "
|
||||||
|
f"{_fmt_duration(probed, missing['seconds'])} / {_fmt_size(missing['bytes'])}")
|
||||||
|
|
||||||
|
no_duration = sum(1 for i in items if i["duration"] is None)
|
||||||
|
if no_duration and probed:
|
||||||
|
lines.append(f"警告: {no_duration} 个视频未能读出时长,未计入时长合计")
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(description="统计需要重新生成字幕的视频数量与时长")
|
||||||
|
parser.add_argument("root", help="媒体库根目录")
|
||||||
|
parser.add_argument("--before", default="2026-09-01",
|
||||||
|
help="重生成分界线,早于该日期生成的 CN 字幕待重跑(默认 2026-09-01)")
|
||||||
|
parser.add_argument("--out", help="计划 JSON 输出路径(默认只打印统计)")
|
||||||
|
parser.add_argument("--workers", type=int, default=8, help="ffprobe 并发数(默认 8)")
|
||||||
|
parser.add_argument("--no-probe", action="store_true", help="跳过时长探测(只统计数量)")
|
||||||
|
parser.add_argument("--list-pending", action="store_true", help="额外打印待重生成的视频路径")
|
||||||
|
parser.add_argument("--backup-old", action="store_true",
|
||||||
|
help="把待重生成视频的旁挂字幕改名备份(默认预览,需 --apply 落盘)")
|
||||||
|
parser.add_argument("--apply", action="store_true", help="与 --backup-old 一起用时真正改名")
|
||||||
|
parser.add_argument("--select", help="只处理该文件列出的视频(每行一个路径或文件名)")
|
||||||
|
parser.add_argument("--suffix", help="备份后缀,默认 .old-<当天日期>")
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
root = Path(args.root).expanduser().resolve()
|
||||||
|
if not root.is_dir():
|
||||||
|
print(f"目录不存在: {root}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
before = datetime.fromisoformat(args.before)
|
||||||
|
|
||||||
|
need_duration = args.out is not None or args.list_pending or not args.backup_old
|
||||||
|
items = build_items(root, max(1, args.workers), probe=need_duration and not args.no_probe)
|
||||||
|
groups = classify(items, before)
|
||||||
|
print(format_summary(root, items, groups, before))
|
||||||
|
if args.list_pending:
|
||||||
|
print("\n待重新生成的视频:")
|
||||||
|
for item in groups[PENDING]:
|
||||||
|
print(f" {item['video']}")
|
||||||
|
|
||||||
|
if args.backup_old:
|
||||||
|
suffix = args.suffix or f".old-{datetime.now():%Y%m%d}"
|
||||||
|
select = read_select_list(Path(args.select)) if args.select else None
|
||||||
|
changed = backup_old_subtitles(groups[PENDING], suffix, apply=args.apply, select=select)
|
||||||
|
verb = "已改名" if args.apply else "待改名(预览,未落盘)"
|
||||||
|
print(f"\n旧字幕备份(后缀 {suffix}): {verb} {len(changed)} 个文件")
|
||||||
|
for source, target in changed:
|
||||||
|
print(f" {source} -> {target.name}")
|
||||||
|
|
||||||
|
if args.out:
|
||||||
|
out = Path(args.out).expanduser()
|
||||||
|
out.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
status_of = {id(i): name for name, group in groups.items() for i in group}
|
||||||
|
payload = {
|
||||||
|
"root": str(root),
|
||||||
|
"before": before.isoformat(),
|
||||||
|
"scanned_at": datetime.now().isoformat(),
|
||||||
|
"totals": {name: _agg(group) for name, group in groups.items()},
|
||||||
|
"items": [{**i, "status": status_of[id(i)]} for i in items],
|
||||||
|
}
|
||||||
|
out.write_text(json.dumps(payload, ensure_ascii=False, indent=1), encoding="utf-8")
|
||||||
|
print(f"\n计划 JSON 已写入: {out}")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
Binary file not shown.
@@ -0,0 +1,317 @@
|
|||||||
|
"""plan_regenerate_subtitles.py 的模块级测试(数据 → 测试过程 → 验证结果)。
|
||||||
|
|
||||||
|
被测模块:字幕重生成计划统计脚本。用例在临时目录里放**真实视频文件**与
|
||||||
|
真实字幕文件,调用脚本的真实函数;只有 ffprobe 子进程这一 I/O 边界在需要
|
||||||
|
确定性时长时注入固定值,其余用例跑真实 ffprobe(缺可执行文件时跳过)。
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from scripts.plan_regenerate_subtitles import (
|
||||||
|
CURRENT,
|
||||||
|
MISSING,
|
||||||
|
PENDING,
|
||||||
|
_agg,
|
||||||
|
backup_old_subtitles,
|
||||||
|
build_items,
|
||||||
|
classify,
|
||||||
|
cn_products,
|
||||||
|
format_summary,
|
||||||
|
main,
|
||||||
|
probe_duration,
|
||||||
|
read_select_list,
|
||||||
|
sidecar_subtitles,
|
||||||
|
)
|
||||||
|
|
||||||
|
DATA = Path(__file__).parent / "data"
|
||||||
|
CLIP = DATA / "clip_10s.mp4" # 真实 10 秒 mp4,供扫描与时长探测使用
|
||||||
|
BEFORE = datetime(2026, 9, 1)
|
||||||
|
|
||||||
|
|
||||||
|
def _set_mtime(path: Path, when: str) -> None:
|
||||||
|
"""把文件 mtime 设为指定日期(本地时区),模拟不同时期生成的字幕。"""
|
||||||
|
ts = datetime.fromisoformat(when).timestamp()
|
||||||
|
os.utime(path, (ts, ts))
|
||||||
|
|
||||||
|
|
||||||
|
def _make_video(tmp_path: Path, name: str) -> Path:
|
||||||
|
"""在临时目录放一份真实视频文件,返回其路径。"""
|
||||||
|
video = tmp_path / f"{name}.mp4"
|
||||||
|
shutil.copyfile(CLIP, video)
|
||||||
|
return video
|
||||||
|
|
||||||
|
|
||||||
|
def _make_subtitle(video: Path, suffix: str, when: str) -> Path:
|
||||||
|
"""在视频旁写一个真实字幕文件(内容为合法 SRT/ASS 片段)并设置 mtime。"""
|
||||||
|
sub = video.with_name(video.name[: -len(video.suffix)] + suffix)
|
||||||
|
if suffix.endswith(".srt"):
|
||||||
|
sub.write_text("1\n00:00:01,000 --> 00:00:02,000\n你好\n", encoding="utf-8")
|
||||||
|
else:
|
||||||
|
sub.write_text("[Script Info]\nTitle: t\n", encoding="utf-8")
|
||||||
|
_set_mtime(sub, when)
|
||||||
|
return sub
|
||||||
|
|
||||||
|
|
||||||
|
def test_products_pick_earliest_mtime_so_style_rewrite_stays_old(tmp_path: Path) -> None:
|
||||||
|
"""数据:中文字幕生成于 2025-12,双目 .ass 被样式脚本在 2026-09 原地改写。
|
||||||
|
|
||||||
|
过程:扫描该视频并读取 CN 产物生成时间。
|
||||||
|
|
||||||
|
验证:只认本流水线产物,生成时间取最早值(2025-12),视频判为待重生成。
|
||||||
|
"""
|
||||||
|
video = _make_video(tmp_path, "movie")
|
||||||
|
_make_subtitle(video, ".srt", "2025-11-01") # 片源自带字幕,不参与判定
|
||||||
|
_make_subtitle(video, ".CN.srt", "2025-12-01")
|
||||||
|
_make_subtitle(video, ".CN_dual_eye.ass", "2026-09-06")
|
||||||
|
|
||||||
|
items = build_items(tmp_path, workers=2, probe=False)
|
||||||
|
|
||||||
|
assert [p.name for p in cn_products(video)] == ["movie.CN.srt", "movie.CN_dual_eye.ass"]
|
||||||
|
assert items[0]["generated_at"].startswith("2025-12-01")
|
||||||
|
assert [i["video"] for i in classify(items, BEFORE)[PENDING]] == [str(video)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_classify_groups_by_generation_time(tmp_path: Path) -> None:
|
||||||
|
"""数据:三个视频——旧字幕、新字幕、完全没有 CN 字幕。
|
||||||
|
|
||||||
|
过程:扫描后按 2026-09-01 分界线分类。
|
||||||
|
|
||||||
|
验证:分别落入待重生成 / 已最新 / 无 CN 字幕三组,边界时刻本身算已最新。
|
||||||
|
"""
|
||||||
|
old = _make_video(tmp_path, "old")
|
||||||
|
_make_subtitle(old, ".CN.srt", "2026-04-05")
|
||||||
|
fresh = _make_video(tmp_path, "fresh")
|
||||||
|
_make_subtitle(fresh, ".CN.srt", "2026-09-01")
|
||||||
|
_make_subtitle(fresh, ".CN_dual_eye.ass", "2026-09-06")
|
||||||
|
never = _make_video(tmp_path, "never")
|
||||||
|
|
||||||
|
groups = classify(build_items(tmp_path, workers=2, probe=False), BEFORE)
|
||||||
|
|
||||||
|
assert [i["video"] for i in groups[PENDING]] == [str(old)]
|
||||||
|
assert [i["video"] for i in groups[CURRENT]] == [str(fresh)]
|
||||||
|
assert [i["video"] for i in groups[MISSING]] == [str(never)]
|
||||||
|
|
||||||
|
|
||||||
|
def test_name_with_extra_suffix_still_matches_product(tmp_path: Path) -> None:
|
||||||
|
"""数据:手工改过名的产物 `<视频名>666.CN.srt`(媒体库里真实存在这种命名)。
|
||||||
|
|
||||||
|
过程:扫描该视频的 CN 产物。
|
||||||
|
|
||||||
|
验证:按「文件名含视频主名 + CN 产物后缀」命中,与批量引擎的旁挂判定一致。
|
||||||
|
"""
|
||||||
|
video = _make_video(tmp_path, "clip")
|
||||||
|
_make_subtitle(video, "666.CN.srt", "2025-12-14")
|
||||||
|
|
||||||
|
items = build_items(tmp_path, workers=2, probe=False)
|
||||||
|
|
||||||
|
assert items[0]["generated_at"].startswith("2025-12-14")
|
||||||
|
assert classify(items, BEFORE)[PENDING]
|
||||||
|
|
||||||
|
|
||||||
|
def test_aggregate_sums_duration_and_size(tmp_path: Path) -> None:
|
||||||
|
"""数据:两个待重生成视频,时长各有值、其中一个探测失败。
|
||||||
|
|
||||||
|
过程:汇总该组的数量、时长、体积。
|
||||||
|
|
||||||
|
验证:时长合计跳过探测失败项并单独计数,体积按全部文件累计。
|
||||||
|
"""
|
||||||
|
video = _make_video(tmp_path, "a")
|
||||||
|
_make_subtitle(video, ".CN.srt", "2025-12-01")
|
||||||
|
items = [
|
||||||
|
{"size": 100, "duration": 12.5},
|
||||||
|
{"size": 50, "duration": None},
|
||||||
|
]
|
||||||
|
|
||||||
|
agg = _agg(items)
|
||||||
|
|
||||||
|
assert agg == {"count": 2, "seconds": 12.5, "bytes": 150, "no_duration": 1}
|
||||||
|
|
||||||
|
|
||||||
|
def test_main_writes_plan_json(tmp_path: Path) -> None:
|
||||||
|
"""数据:真实 10 秒视频 + 旧 CN 字幕 + 新 CN 字幕各一个。
|
||||||
|
|
||||||
|
过程:调用 main() 全流程并写出计划 JSON。
|
||||||
|
|
||||||
|
验证:统计报告含待重生成/已最新行数,JSON 里两条明细状态与时长正确。
|
||||||
|
"""
|
||||||
|
if shutil.which("ffprobe") is None:
|
||||||
|
pytest.skip("环境缺少 ffprobe,无法探测真实视频时长")
|
||||||
|
old = _make_video(tmp_path, "old")
|
||||||
|
_make_subtitle(old, ".CN.srt", "2025-12-01")
|
||||||
|
fresh = _make_video(tmp_path, "fresh")
|
||||||
|
_make_subtitle(fresh, ".CN.srt", "2026-09-10")
|
||||||
|
out = tmp_path / "plan.json"
|
||||||
|
|
||||||
|
code = main([str(tmp_path), "--before", "2026-09-01", "--out", str(out), "--workers", "2"])
|
||||||
|
|
||||||
|
assert code == 0
|
||||||
|
payload = json.loads(out.read_text(encoding="utf-8"))
|
||||||
|
by_name = {Path(i["video"]).name: i for i in payload["items"]}
|
||||||
|
assert by_name["old.mp4"]["status"] == PENDING
|
||||||
|
assert by_name["old.mp4"]["duration"] == pytest.approx(10.0, abs=0.5)
|
||||||
|
assert by_name["fresh.mp4"]["status"] == CURRENT
|
||||||
|
assert payload["totals"][PENDING]["count"] == 1
|
||||||
|
assert payload["totals"][PENDING]["seconds"] == pytest.approx(10.0, abs=0.5)
|
||||||
|
|
||||||
|
|
||||||
|
def test_probe_duration_reads_real_video() -> None:
|
||||||
|
"""数据:真实 10 秒 mp4(模块目录内素材)。
|
||||||
|
|
||||||
|
过程:调用真实 ffprobe 读取时长。
|
||||||
|
|
||||||
|
验证:返回约 10 秒,说明统计脚本的时长口径来自容器真实时长。
|
||||||
|
"""
|
||||||
|
if shutil.which("ffprobe") is None:
|
||||||
|
pytest.skip("环境缺少 ffprobe,无法探测真实视频时长")
|
||||||
|
|
||||||
|
assert probe_duration(CLIP) == pytest.approx(10.0, abs=0.1)
|
||||||
|
|
||||||
|
|
||||||
|
def test_format_summary_reports_all_groups() -> None:
|
||||||
|
"""数据:三组各一条明细(旧/新/无字幕),时长与体积已知。
|
||||||
|
|
||||||
|
过程:格式化统计报告。
|
||||||
|
|
||||||
|
验证:报告包含总量、待重生成、已最新、无 CN 字幕四段关键信息。
|
||||||
|
"""
|
||||||
|
root = Path("/media")
|
||||||
|
items = [
|
||||||
|
{"duration": 3600.0, "size": 0, "generated_at": "2025-12-01T00:00:00"},
|
||||||
|
{"duration": 1800.0, "size": 0, "generated_at": "2026-09-10T00:00:00"},
|
||||||
|
{"duration": None, "size": 0, "generated_at": None},
|
||||||
|
]
|
||||||
|
|
||||||
|
text = format_summary(root, items, classify(items, BEFORE), BEFORE)
|
||||||
|
|
||||||
|
assert "视频总数: 3 个" in text
|
||||||
|
assert "待重新生成(CN 字幕早于 2026-09-01)" in text
|
||||||
|
assert "已是最新" in text
|
||||||
|
assert "无 CN 字幕(首次生成)" in text
|
||||||
|
assert "警告: 1 个视频未能读出时长" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_sidecar_subtitles_matches_batch_engine_rule(tmp_path: Path) -> None:
|
||||||
|
"""数据:视频旁有片源 .srt、CN 产物与无关文件各一份。
|
||||||
|
|
||||||
|
过程:列出批量引擎会认定为"已有字幕"的旁挂文件。
|
||||||
|
|
||||||
|
验证:只收文件名含视频主名的字幕,不含无关图片与别人的字幕。
|
||||||
|
"""
|
||||||
|
video = _make_video(tmp_path, "movie")
|
||||||
|
source = _make_subtitle(video, ".srt", "2025-11-01")
|
||||||
|
cn_srt = _make_subtitle(video, ".CN.srt", "2025-12-01")
|
||||||
|
cn_ass = _make_subtitle(video, ".CN_dual_eye.ass", "2026-09-06")
|
||||||
|
(tmp_path / "movie-poster.jpg").write_bytes(b"poster")
|
||||||
|
(tmp_path / "other.srt").write_text("1\n", encoding="utf-8")
|
||||||
|
|
||||||
|
found = sidecar_subtitles(video)
|
||||||
|
|
||||||
|
assert found == [cn_srt, cn_ass, source]
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_renames_every_sidecar_and_unsets_skip(tmp_path: Path) -> None:
|
||||||
|
"""数据:待重生成视频带片源 .srt + CN 产物 + 既有备份目标冲突。
|
||||||
|
|
||||||
|
过程:执行改名备份。
|
||||||
|
|
||||||
|
验证:全部旁挂字幕改名成 .old 后缀、原文件消失;改名后批量引擎再也
|
||||||
|
不会把该视频判为"已有字幕"。
|
||||||
|
"""
|
||||||
|
video = _make_video(tmp_path, "movie")
|
||||||
|
source = _make_subtitle(video, ".srt", "2025-11-01")
|
||||||
|
cn_srt = _make_subtitle(video, ".CN.srt", "2025-12-01")
|
||||||
|
cn_ass = _make_subtitle(video, ".CN_dual_eye.ass", "2026-09-06")
|
||||||
|
|
||||||
|
changed = backup_old_subtitles([{"video": str(video)}], suffix=".old", apply=True)
|
||||||
|
|
||||||
|
assert len(changed) == 3
|
||||||
|
assert sidecar_subtitles(video) == []
|
||||||
|
for path in (source, cn_srt, cn_ass):
|
||||||
|
assert not path.exists()
|
||||||
|
assert (path.parent / (path.name + ".old")).read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_dry_run_only_reports(tmp_path: Path) -> None:
|
||||||
|
"""数据:一个带旧字幕的视频。
|
||||||
|
|
||||||
|
过程:不传 apply 走预览。
|
||||||
|
|
||||||
|
验证:返回改名计划,磁盘文件保持不变。
|
||||||
|
"""
|
||||||
|
video = _make_video(tmp_path, "movie")
|
||||||
|
sub = _make_subtitle(video, ".CN.srt", "2025-12-01")
|
||||||
|
|
||||||
|
changed = backup_old_subtitles([{"video": str(video)}], suffix=".old", apply=False)
|
||||||
|
|
||||||
|
assert changed == [(sub, sub.with_name(sub.name + ".old"))]
|
||||||
|
assert sub.exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_does_not_overwrite_existing_backup(tmp_path: Path) -> None:
|
||||||
|
"""数据:视频旁已有同名备份文件(同一批重复执行)。
|
||||||
|
|
||||||
|
过程:再次执行改名备份。
|
||||||
|
|
||||||
|
验证:已存在的备份不被覆盖,原文件也不被删除,返回空变更。
|
||||||
|
"""
|
||||||
|
video = _make_video(tmp_path, "movie")
|
||||||
|
sub = _make_subtitle(video, ".CN.srt", "2025-12-01")
|
||||||
|
backup = sub.with_name(sub.name + ".old")
|
||||||
|
backup.write_text("first\n", encoding="utf-8")
|
||||||
|
|
||||||
|
changed = backup_old_subtitles([{"video": str(video)}], suffix=".old", apply=True)
|
||||||
|
|
||||||
|
assert changed == []
|
||||||
|
assert backup.read_text(encoding="utf-8") == "first\n"
|
||||||
|
assert sub.exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_backup_respects_select_list(tmp_path: Path) -> None:
|
||||||
|
"""数据:两个待重生成视频,只选中其中一个(选择文件带行尾注释)。
|
||||||
|
|
||||||
|
过程:执行改名备份。
|
||||||
|
|
||||||
|
验证:只改选中视频的旁挂字幕,另一个保持原样。
|
||||||
|
"""
|
||||||
|
chosen = _make_video(tmp_path, "chosen")
|
||||||
|
chosen_sub = _make_subtitle(chosen, ".CN.srt", "2025-12-01")
|
||||||
|
kept = _make_video(tmp_path, "kept")
|
||||||
|
kept_sub = _make_subtitle(kept, ".CN.srt", "2025-12-02")
|
||||||
|
select = tmp_path / "select.txt"
|
||||||
|
select.write_text(f"# 只处理这一个\n{chosen} # 40.3min\n\n", encoding="utf-8")
|
||||||
|
|
||||||
|
changed = backup_old_subtitles(
|
||||||
|
[{"video": str(chosen)}, {"video": str(kept)}],
|
||||||
|
suffix=".old", apply=True, select=read_select_list(select),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert changed == [(chosen_sub, chosen_sub.with_name(chosen_sub.name + ".old"))]
|
||||||
|
assert kept_sub.exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_format_summary_marks_missing_durations(tmp_path: Path) -> None:
|
||||||
|
"""数据:只做分类统计、未探测时长的明细(--backup-old 场景)。
|
||||||
|
|
||||||
|
过程:格式化统计报告。
|
||||||
|
|
||||||
|
验证:报告不显示 0 小时,而是标注未探测时长,也不出现时长缺失警告。
|
||||||
|
"""
|
||||||
|
root = Path("/media")
|
||||||
|
items = [
|
||||||
|
{"duration": None, "size": 10, "generated_at": "2025-12-01T00:00:00"},
|
||||||
|
{"duration": None, "size": 20, "generated_at": "2026-09-10T00:00:00"},
|
||||||
|
]
|
||||||
|
|
||||||
|
text = format_summary(root, items, classify(items, BEFORE), BEFORE)
|
||||||
|
|
||||||
|
assert "未探测时长" in text
|
||||||
|
assert "0.00 小时" not in text
|
||||||
|
assert "警告" not in text
|
||||||
Reference in New Issue
Block a user