feat: 新增种子目录清理脚本与配套 skill
主程序只能把 magnet 交给 Transmission,无法挑选文件,种子内的广告 (*.url、宣传页、样片/预告)会一并下载,同一部作品还会被多个发布站各下一份。 新增离线清理脚本处理下载完成后的目录。 clean_seed_extras.py: - 保留:>=100MB 的视频(40+ 扩展名;无扩展名时读文件头嗅探)、字幕 (含 .srt.old-<日期> 备份)、与同目录视频同名的海报、folder.jpg 等通用封面 - 删除:其余文件;同目录内内容相同的大视频只保留一份,校验方式为 sha256(体积 + 头 4MB + 尾 4MB),体积相同但内容不同的不会误删 - 删除后自底向上级联清理空目录(可删到影片级目录,但不删 --root 本身) - 默认预演,需 --apply 才真正删除;--manifest 导出待删清单供逐段复核 - 其他开关:--no-subtitles/--no-posters/--no-covers/--no-dedupe/--min-size-mb/ --prefer-source/--keep-empty-dirs/--skip-recent-min .agents/skills/torrent-download-cleanup: - 记录完整流程(预演 -> 出清单 -> 用户确认 -> 执行 -> 复核)、保留/删除规则、 去重策略、代码维护注意(decide() 必须是 视频 -> 去重 -> 海报匹配 的顺序) - 附只读盘点脚本 find_missing_torrents.py:找出 Transmission 中数据已不在磁盘 的任务(自包含,只用 requests) tests/test_clean_seed_extras.py: 30 个离线用例(tmp_path 临时目录) tests/README.md: 补充脚本用法与测试说明
This commit is contained in:
+134
-13
@@ -9,8 +9,8 @@
|
||||
- 测试连接对象属性和方法
|
||||
- 测试关闭连接
|
||||
|
||||
### seed_is_exist 测试
|
||||
- `test_seed_is_exist.py` - seed_is_exist 功能真实数据库测试
|
||||
### seed_is_exist / all_seed_codes 测试
|
||||
- `test_seed_is_exist.py` - 真实数据库测试(含 all_seed_codes 作品级去重)
|
||||
- 首次运行自动创建 `LookingWeb` 库和 `seeds_info` 表(幂等)
|
||||
- 测试种子不存在 → False
|
||||
- 真实插入种子后 → True
|
||||
@@ -19,15 +19,58 @@
|
||||
- 数据库不可达 → False(不抛异常)
|
||||
- 测试数据测后自动清理
|
||||
|
||||
### qBittorrent 下载器测试
|
||||
### 作品级去重与 magnet 校验测试
|
||||
- `test_seed_dedupe.py` - 离线测试(mock 网络/数据库/下载器,22 个用例)
|
||||
- magnet 链接校验:40 位十六进制 / 32 位 base32 合法;拒绝站点“复制模板”行
|
||||
(多个 magnet 用 `__JOIN_ITEM_SEP__magnet_link__` 拼接)与非法 btih
|
||||
(如 `17i6okorc0zvocd3tzfompbktvveifcwt`)
|
||||
- 影片番号解析(标题中的“影片番号为XXX”)与 `SeedInfo.code`
|
||||
- `get_seed_from_element_detail` 只保留合法 magnet
|
||||
- `collect_new_seeds` 两级去重:同一作品(番号)只下载一次、同一链接不重复收集、
|
||||
解析不到番号时退回链接去重
|
||||
- `main()` 端到端:详情页 2 个合法 magnet(同一番号)+ 1 个模板行 → 只下载 1 个;
|
||||
数据库已有该番号 → 一条都不下载
|
||||
|
||||
### qBittorrent 下载器测试(备用下载器)
|
||||
- `test_qbittorrent_downloader.py` - qBittorrent 下载器单元测试
|
||||
- 测试初始化(带/不带认证)
|
||||
- 测试添加下载任务(成功/失败/异常)
|
||||
- 测试种子信息下载
|
||||
- `test_qbittorrent_downloader.py` - qBittorrent 下载器离线单元测试(19 个用例,全部 mock 网络)
|
||||
- 测试初始化(带/不带认证、登录成功/被拒/异常)
|
||||
- 测试添加下载任务(成功 / HTTP 200 + "Fails." 被拒 / 非 200 / 异常 / 空链接)
|
||||
- 测试种子信息下载(含 seed_info 为 None)
|
||||
- 测试带选项的下载(保存路径、分类、标签)
|
||||
- 测试连接测试
|
||||
- 测试获取下载列表
|
||||
- 注意:当前 11/12 用例因测试 mock 目标与实现不一致(`QbittorrentDownloader.session` 是实例属性而非模块属性)而失败,属既有问题,待修复。
|
||||
- 测试连接测试与获取下载列表
|
||||
- 说明:mock 目标为 `requests.Session.post/get`;`QbittorrentDownloader.session` 是实例属性,
|
||||
patch 模块属性会 AttributeError。qBittorrent Web API 的成功/失败不体现在 HTTP 状态码,
|
||||
添加 torrent 成功返回 200 + `Ok.`、被拒绝返回 200 + `Fails.`,判定必须读响应体。
|
||||
|
||||
### Transmission 下载器测试(主程序当前使用)
|
||||
- `test_transmission_downloader.py` - Transmission 下载器离线单元测试(26 个用例,全部 mock 网络)
|
||||
- 测试初始化(带/不带认证、409 握手后认证成功、401、非 200、异常)
|
||||
- 测试 409 会话握手:session id 过期自动更新重发、连续 409 不死循环
|
||||
- 测试添加下载任务(成功 / HTTP 200 + `result != success` 被拒 / 重复种子 / 401 / 非 JSON / 异常 / 空链接)
|
||||
- 测试保存路径映射为 `download-dir`、分类与标签映射为 `labels`
|
||||
- 测试种子信息下载(含 seedInfo 为 None、seedUrl 为空)
|
||||
- 测试连接测试与获取下载列表
|
||||
- 说明:下载器连接 `http://192.168.123.199:9095/transmission/rpc`(Transmission Web UI,
|
||||
Basic 认证 admin / GUOxy.5157)。mock 目标为 `requests.Session.post`;
|
||||
Transmission 首次请求返回 409 + `X-Transmission-Session-Id` 响应头,需要带上该头重发;
|
||||
业务失败不体现在 HTTP 状态码上,必须读响应体的 `result` 字段。
|
||||
|
||||
### 下载目录清理脚本测试
|
||||
- `test_clean_seed_extras.py` - 离线单元测试(30 个用例,全部使用 tmp_path 临时目录)
|
||||
- 文件头嗅探 `looks_like_video`:mp4/ftyp、mkv、avi、flv、wmv、mpg、rmvb 识别;
|
||||
mpeg-ts 需两次 0x47 同步字节校验;随机数据判为非视频
|
||||
- 保留规则:>=100MB 视频保留、小体积样片删除、无扩展名大视频按文件头保留
|
||||
- 字幕保留(含 `xxx.CN.srt.old-20260918` 备份);`--no-subtitles` / `--no-posters` 可关闭
|
||||
- 海报只保留与「同目录视频文件名主干」匹配的图片;
|
||||
通用封面(folder.jpg/poster.jpg/cover.jpg 等)默认保留,`--no-covers` 关闭;
|
||||
目录内没有视频时封面图视为无主垃圾一并删除
|
||||
- 重复视频去重:同目录同体积 → 头尾指纹校验 → 只留一份;
|
||||
体积相同但内容不同不去重、不同目录不去重、`--no-dedupe` 关闭、
|
||||
`--prefer-source` 指定优先站点、`dup_of` 记录保留的那一份
|
||||
- 安全行为:默认预演不删除;`--apply` 才删除;空目录级联删除但保留 root;
|
||||
`--keep-empty-dirs` 保留空目录;不跟随符号链接目录;目录不存在返回码 2
|
||||
|
||||
## 运行测试
|
||||
|
||||
```bash
|
||||
@@ -40,10 +83,36 @@ uv run python -m pytest tests/ -v
|
||||
# 运行 MySQLDatabase 测试
|
||||
uv run python -m pytest tests/test_database.py -v
|
||||
|
||||
# 运行 qBittorrent 测试
|
||||
# 运行 qBittorrent 测试(已不再被主程序使用,保留备用)
|
||||
uv run python -m pytest tests/test_qbittorrent_downloader.py -v
|
||||
|
||||
# 运行 Transmission 测试(主程序当前使用)
|
||||
uv run python -m pytest tests/test_transmission_downloader.py -v
|
||||
|
||||
# 运行作品级去重与 magnet 校验测试(离线)
|
||||
uv run python -m pytest tests/test_seed_dedupe.py -v
|
||||
|
||||
# 运行下载目录清理脚本测试(离线)
|
||||
uv run python -m pytest tests/test_clean_seed_extras.py -v
|
||||
```
|
||||
|
||||
## 去重规则(已修复的重复下载问题)
|
||||
|
||||
freejavbt 的同一个详情页会为同一部影片发布多个 magnet(如 VRKM-1872 的
|
||||
“5个文件”与“6个文件”,info-hash 不同、内容重叠)。主程序原来只用
|
||||
`seed_url`(完整 magnet 链接)去重,因此同一部影片的其他发布会被当成新种子
|
||||
再次下载。现在改为两级去重(`LookingWebMain.collect_new_seeds`):
|
||||
|
||||
1. **作品级(影片番号)**:标题中的“影片番号为XXX”为去重键,数据库里已有该番号
|
||||
(`MySqlService.all_seed_codes()`)或本次运行已收集过,就跳过;
|
||||
2. **链接级**:完整 magnet 链接(原有 `seed_is_exist` 逻辑),并额外记录本次运行
|
||||
已收集但尚未写库的链接(写库发生在下载之后)。
|
||||
|
||||
解析阶段还会丢弃两类脏数据(`is_valid_seed_url`):站点“复制模板”行拼接的伪 magnet,
|
||||
以及站点发布的非法 btih;这两类链接发给 Transmission 只会被拒收(`have_download=-1`)。
|
||||
|
||||
关闭作品级去重可设置 `LookingWebMain.DEDUPE_BY_SEED_CODE = False`(退回仅按链接去重)。
|
||||
|
||||
## 环境要求
|
||||
|
||||
- Python 3.7+
|
||||
@@ -51,11 +120,63 @@ uv run python -m pytest tests/test_qbittorrent_downloader.py -v
|
||||
- loguru
|
||||
- beautifulsoup4
|
||||
- pymysql
|
||||
- cryptography(MySQL 8 caching_sha2_password 认证必需)
|
||||
- pytest
|
||||
|
||||
## 测试覆盖
|
||||
|
||||
- MySQLDatabase: 4 个测试用例
|
||||
- seed_is_exist: 5 个测试用例(真实数据库)
|
||||
- qBittorrentDownloader: 12 个测试用例(11 个待修复)
|
||||
- 总计: 25 个测试用例
|
||||
- seed_is_exist / all_seed_codes: 8 个测试用例(真实数据库)
|
||||
- 作品级去重与 magnet 校验: 22 个测试用例(离线 mock)
|
||||
- qBittorrentDownloader: 19 个测试用例(离线 mock,备用下载器)
|
||||
- TransmissionDownloader: 26 个测试用例(离线 mock,主程序当前使用)
|
||||
- clean_seed_extras: 30 个测试用例(离线,tmp_path 临时目录)
|
||||
- 其他: cf_session / 主程序函数(真实网络)
|
||||
|
||||
## 下载完成后的清理脚本(clean_seed_extras.py)
|
||||
|
||||
主程序只能把 magnet 交给 Transmission,无法挑选文件,种子内的广告文件会被一并下载。
|
||||
下载完成后单独运行 `clean_seed_extras.py` 清理,保留「大体积视频 + 字幕 + 与视频同名的海报」:
|
||||
|
||||
```bash
|
||||
# 预演(默认,只统计不删除)
|
||||
uv run python clean_seed_extras.py
|
||||
|
||||
# 导出完整待删清单供逐段复核(不删除)
|
||||
uv run python clean_seed_extras.py --manifest clean_manifest.txt
|
||||
|
||||
# 确认无误后真正删除
|
||||
uv run python clean_seed_extras.py --apply
|
||||
|
||||
# 常用选项
|
||||
uv run python clean_seed_extras.py --apply --list-all # 列出每个被删文件
|
||||
uv run python clean_seed_extras.py --min-size-mb 50 # 大视频阈值改为 50MB
|
||||
uv run python clean_seed_extras.py --apply --no-subtitles # 连字幕一起删
|
||||
uv run python clean_seed_extras.py --apply --no-covers # 不保留 folder.jpg 等通用封面
|
||||
uv run python clean_seed_extras.py --apply --no-dedupe # 不做重复视频去重
|
||||
uv run python clean_seed_extras.py --prefer-source masex.tv,hkbisi.com
|
||||
uv run python clean_seed_extras.py --root /mnt/fnOS/123 # 指定目录(默认值)
|
||||
```
|
||||
|
||||
保留规则:
|
||||
|
||||
1. **视频**:扩展名在 `VIDEO_EXTS` 内且体积 >= `--min-size-mb`(默认 100MB);
|
||||
扩展名未知时按文件头(magic bytes)嗅探识别;
|
||||
2. **字幕**:文件名含字幕扩展名(`.srt/.ass/.ssa/.vtt/.sub/.idx/.sup/.smi` 等),
|
||||
含 `.srt.old-<日期>` 备份;
|
||||
3. **海报**:图片扩展名且文件名包含**同目录**下某个已保留视频的文件名主干;
|
||||
4. **通用封面**:`folder.jpg`/`poster.jpg`/`cover.jpg`/`backdrop.jpg`/`fanart.jpg` 等
|
||||
媒体库约定名(Jellyfin/Emby/Kodi 会当封面用),且同目录下确实还有保留的视频。
|
||||
|
||||
删除规则:其余文件删除;另外**同一目录内内容相同的大视频只保留一份**
|
||||
(同一部作品被多个发布站各下一份时,文件名只差站点前缀,如
|
||||
`masex.tv@vrkm01874_1_8k.mp4` 与 `hkbisi.com@vrkm01874_1_8k.mp4`)。
|
||||
|
||||
去重实现:先按「同目录 + 同体积」分组(几乎不耗 IO),再对每组算
|
||||
`sha256(体积 + 头部 4MB + 尾部 4MB)` 确认内容一致——**体积相同但内容不同的不会误删**。
|
||||
保留哪一份依次比较:无站点前缀 > `--prefer-source` 指定站点 >
|
||||
同目录内该前缀出现次数多 > 文件名短 > 字典序。
|
||||
|
||||
删除文件后自底向上级联清理空目录(子目录先删,父目录变空后继续向上删,
|
||||
可一路删到影片级目录,但不删 `--root` 本身;如 Jellyfin 的 `*.trickplay/` 缩略图目录、
|
||||
BT 的 BEP47 填充目录 `.pad/`)。脚本不跟随符号链接目录,删除符号链接只删链接本身。
|
||||
@@ -0,0 +1,320 @@
|
||||
"""clean_seed_extras 清理脚本离线单元测试
|
||||
|
||||
全部使用 tmp_path 临时目录,不触碰真实下载目录(/mnt/fnOS/123)。
|
||||
运行: uv run python -m pytest tests/test_clean_seed_extras.py -v
|
||||
"""
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
import clean_seed_extras as c
|
||||
|
||||
MB = 1024 * 1024
|
||||
|
||||
|
||||
def make_file(path, size, header=b""):
|
||||
"""创建指定体积的文件(稀疏文件,速度快),可选写入文件头"""
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "wb") as fp:
|
||||
if header:
|
||||
fp.write(header)
|
||||
os.truncate(path, size)
|
||||
return path
|
||||
|
||||
|
||||
def run(root, *extra):
|
||||
"""以预演/删除模式运行脚本主函数"""
|
||||
return c.main(["--root", str(root), "--quiet", *extra])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 文件头嗅探
|
||||
|
||||
def test_looks_like_video_常见容器(tmp_path):
|
||||
cases = {
|
||||
"a.mp4": b"\x00\x00\x00\x20ftypisom",
|
||||
"b.mkv": b"\x1a\x45\xdf\xa3\x01\x00\x00\x00",
|
||||
"c.avi": b"RIFF\x00\x00\x00\x00AVI LIST",
|
||||
"d.flv": b"FLV\x01\x05\x00\x00\x00",
|
||||
"e.wmv": b"\x30\x26\xb2\x75\x8e\x66\xcf\x11",
|
||||
"f.mpg": b"\x00\x00\x01\xba\x44\x00",
|
||||
"g.rmvb": b".RMF\x00\x00\x00\x12",
|
||||
}
|
||||
for name, header in cases.items():
|
||||
path = make_file(str(tmp_path / name), 200 * MB, header)
|
||||
assert c.looks_like_video(path) is True, name
|
||||
|
||||
|
||||
def test_looks_like_video_ts_需要两次同步字节(tmp_path):
|
||||
ts = make_file(str(tmp_path / "ok.ts"), 200 * MB, b"\x47" + b"\x00" * 187 + b"\x47")
|
||||
assert c.looks_like_video(ts) is True
|
||||
fake = make_file(str(tmp_path / "fake.ts"), 200 * MB, b"\x47" + b"\x00" * 511)
|
||||
assert c.looks_like_video(fake) is False
|
||||
|
||||
|
||||
def test_looks_like_video_非视频(tmp_path):
|
||||
junk = make_file(str(tmp_path / "junk.bin"), 200 * MB, os.urandom(64))
|
||||
assert c.looks_like_video(junk) is False
|
||||
assert c.looks_like_video(str(tmp_path / "not-exist")) is False
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 保留规则
|
||||
|
||||
def test_大视频保留_小视频删除(tmp_path):
|
||||
big = make_file(str(tmp_path / "x/big.mp4"), 120 * MB)
|
||||
small = make_file(str(tmp_path / "x/sample.mp4"), 20 * MB)
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(big) is True
|
||||
assert os.path.exists(small) is False
|
||||
|
||||
|
||||
def test_无扩展名大视频按文件头保留(tmp_path):
|
||||
video = make_file(str(tmp_path / "noext_video"), 150 * MB, b"\x00\x00\x00\x20ftypisom")
|
||||
junk = make_file(str(tmp_path / "noext_junk"), 150 * MB, os.urandom(64))
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(video) is True
|
||||
assert os.path.exists(junk) is False
|
||||
|
||||
|
||||
def test_字幕保留_含old备份(tmp_path):
|
||||
make_file(str(tmp_path / "x/movie.mp4"), 120 * MB)
|
||||
subs = [
|
||||
make_file(str(tmp_path / "x/movie.srt"), 1024),
|
||||
make_file(str(tmp_path / "x/movie.CN.srt"), 1024),
|
||||
make_file(str(tmp_path / "x/movie.CN_dual_eye.ass"), 1024),
|
||||
make_file(str(tmp_path / "x/movie.CN.srt.old-20260918"), 1024),
|
||||
]
|
||||
run(tmp_path, "--apply")
|
||||
assert all(os.path.exists(p) for p in subs)
|
||||
|
||||
|
||||
def test_海报仅保留与视频同名的(tmp_path):
|
||||
make_file(str(tmp_path / "x/ATID-643 淫魔病棟.mp4"), 120 * MB)
|
||||
keep = make_file(str(tmp_path / "x/ATID-643 淫魔病棟-poster.jpg"), 4096)
|
||||
drop_folder = make_file(str(tmp_path / "x/folder.jpg"), 4096)
|
||||
drop_other = make_file(str(tmp_path / "x/OTHER-999-poster.jpg"), 4096)
|
||||
run(tmp_path, "--apply", "--no-covers") # --no-covers = 严格模式,只留同名海报
|
||||
assert os.path.exists(keep) is True
|
||||
assert os.path.exists(drop_folder) is False
|
||||
assert os.path.exists(drop_other) is False
|
||||
|
||||
|
||||
def test_通用封面图默认保留(tmp_path):
|
||||
"""folder.jpg / poster.jpg 等媒体库封面默认保留(同目录有大视频时)"""
|
||||
make_file(str(tmp_path / "x/movie.mp4"), 120 * MB)
|
||||
covers = [
|
||||
make_file(str(tmp_path / "x/folder.jpg"), 4096),
|
||||
make_file(str(tmp_path / "x/poster.jpg"), 4096),
|
||||
make_file(str(tmp_path / "x/backdrop.jpg"), 4096),
|
||||
make_file(str(tmp_path / "x/fanart.png"), 4096),
|
||||
]
|
||||
run(tmp_path, "--apply")
|
||||
assert all(os.path.exists(p) for p in covers)
|
||||
|
||||
|
||||
def test_无视频目录的通用封面不保留(tmp_path):
|
||||
"""只有 folder.jpg 没有视频的目录属于无主垃圾,封面和目录都应删掉"""
|
||||
make_file(str(tmp_path / "shell/folder.jpg"), 4096)
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(str(tmp_path / "shell/folder.jpg")) is False
|
||||
assert os.path.exists(str(tmp_path / "shell")) is False
|
||||
|
||||
|
||||
def test_跨目录不误匹配海报(tmp_path):
|
||||
"""海报只与同目录的视频匹配,避免 A 目录海报被 B 目录视频保住"""
|
||||
make_file(str(tmp_path / "a/movie.mp4"), 120 * MB)
|
||||
other = make_file(str(tmp_path / "b/movie-poster.jpg"), 4096)
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(other) is False
|
||||
|
||||
|
||||
def test_关闭字幕与海报保留(tmp_path):
|
||||
make_file(str(tmp_path / "x/movie.mp4"), 120 * MB)
|
||||
sub = make_file(str(tmp_path / "x/movie.srt"), 1024)
|
||||
poster = make_file(str(tmp_path / "x/movie-poster.jpg"), 4096)
|
||||
cover = make_file(str(tmp_path / "x/folder.jpg"), 4096)
|
||||
run(tmp_path, "--apply", "--no-subtitles", "--no-posters")
|
||||
assert os.path.exists(sub) is False
|
||||
assert os.path.exists(poster) is False
|
||||
assert os.path.exists(cover) is False
|
||||
|
||||
|
||||
def test_阈值可调(tmp_path):
|
||||
small = make_file(str(tmp_path / "x/sample.mp4"), 20 * MB)
|
||||
run(tmp_path, "--apply", "--min-size-mb", "10")
|
||||
assert os.path.exists(small) is True
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 重复视频去重
|
||||
|
||||
HEADER = b"\x00\x00\x00\x20ftypisom"
|
||||
|
||||
|
||||
def test_重复大视频只保留一份(tmp_path):
|
||||
a = make_file(str(tmp_path / "x/hkbisi.com@abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
b = make_file(str(tmp_path / "x/masex.tv@abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
run(tmp_path, "--apply")
|
||||
assert sum(os.path.exists(p) for p in (a, b)) == 1
|
||||
|
||||
|
||||
def test_体积相同内容不同不去重(tmp_path):
|
||||
a = make_file(str(tmp_path / "x/a.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
b = make_file(str(tmp_path / "x/b.com@abc001_1.mp4"), 120 * MB, b"\x00\x00\x00\x20ftypmp42")
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(a) is True
|
||||
assert os.path.exists(b) is True
|
||||
|
||||
|
||||
def test_去重时保留无站点前缀的一份(tmp_path):
|
||||
plain = make_file(str(tmp_path / "x/abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
prefixed = make_file(str(tmp_path / "x/masex.tv@abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(plain) is True
|
||||
assert os.path.exists(prefixed) is False
|
||||
|
||||
|
||||
def test_prefer_source_优先保留指定站点(tmp_path):
|
||||
make_file(str(tmp_path / "x/masex.tv@abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
other = make_file(str(tmp_path / "x/hkbisi.com@abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
run(tmp_path, "--apply", "--prefer-source", "hkbisi.com")
|
||||
assert os.path.exists(other) is True
|
||||
assert os.path.exists(str(tmp_path / "x/masex.tv@abc001_1_8k.mp4")) is False
|
||||
|
||||
|
||||
def test_不同目录内容相同不去重(tmp_path):
|
||||
a = make_file(str(tmp_path / "x/abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
b = make_file(str(tmp_path / "y/abc001_1_8k.mp4"), 120 * MB, HEADER)
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(a) is True
|
||||
assert os.path.exists(b) is True
|
||||
|
||||
|
||||
def test_no_dedupe_关闭去重(tmp_path):
|
||||
a = make_file(str(tmp_path / "x/a.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
b = make_file(str(tmp_path / "x/b.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
run(tmp_path, "--apply", "--no-dedupe")
|
||||
assert os.path.exists(a) is True
|
||||
assert os.path.exists(b) is True
|
||||
|
||||
|
||||
def test_去重后副本的同名海报一并删除(tmp_path):
|
||||
"""去重必须在海报匹配之前:副本的海报不能留到下一轮才清(要求一次收敛)"""
|
||||
make_file(str(tmp_path / "x/a.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
make_file(str(tmp_path / "x/b.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
keep_poster = make_file(str(tmp_path / "x/a.com@abc001_1-poster.jpg"), 4096)
|
||||
dup_poster = make_file(str(tmp_path / "x/b.com@abc001_1-poster.jpg"), 4096)
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.exists(keep_poster) is True
|
||||
assert os.path.exists(dup_poster) is False
|
||||
# 再跑一次应当无事可做(一次收敛)
|
||||
entries, _ = c.collect(str(tmp_path))
|
||||
c.decide(entries, 100 * MB, True, True)
|
||||
assert [e.path for e in entries if not e.keep] == []
|
||||
|
||||
|
||||
def test_去重会记录保留的那一份(tmp_path):
|
||||
a = make_file(str(tmp_path / "x/a.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
make_file(str(tmp_path / "x/b.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
entries, _ = c.collect(str(tmp_path))
|
||||
c.decide(entries, 100 * MB, True, True)
|
||||
dups = [e for e in entries if e.dup_of]
|
||||
assert len(dups) == 1
|
||||
assert dups[0].reason == "重复视频"
|
||||
assert os.path.exists(dups[0].dup_of) is True
|
||||
assert dups[0].dup_of in (a, str(tmp_path / "x/b.com@abc001_1.mp4"))
|
||||
|
||||
|
||||
def test_重复视频单独成段写入清单(tmp_path):
|
||||
make_file(str(tmp_path / "x/a.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
make_file(str(tmp_path / "x/b.com@abc001_1.mp4"), 120 * MB, HEADER)
|
||||
manifest = tmp_path / "m.txt"
|
||||
c.main(["--root", str(tmp_path), "--quiet", "--manifest", str(manifest)])
|
||||
text = manifest.read_text(encoding="utf-8")
|
||||
assert "E. 重复大视频" in text
|
||||
assert "--- 保留:" in text
|
||||
assert "F. 待删除的空目录" in text
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 删除行为
|
||||
|
||||
def test_预演模式不删除任何文件(tmp_path):
|
||||
make_file(str(tmp_path / "x/movie.mp4"), 120 * MB)
|
||||
junk = make_file(str(tmp_path / "x/a.url"), 16)
|
||||
assert run(tmp_path) == 0
|
||||
assert os.path.exists(junk) is True
|
||||
assert os.path.exists(str(tmp_path / "x/movie.mp4")) is True
|
||||
|
||||
|
||||
def test_清理空目录_但保留root(tmp_path):
|
||||
make_file(str(tmp_path / "x/movie.mp4"), 120 * MB)
|
||||
make_file(str(tmp_path / "x/movie.trickplay/320 - 10x10/0.jpg"), 1024)
|
||||
empty = tmp_path / "empty-dir"
|
||||
empty.mkdir()
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.isdir(str(tmp_path)) is True
|
||||
assert os.path.exists(str(tmp_path / "x/movie.trickplay")) is False
|
||||
assert os.path.exists(str(empty)) is False
|
||||
|
||||
|
||||
def test_父目录级联删除到影片级目录(tmp_path):
|
||||
"""多层子目录被清空后,父目录应逐级向上删除,直到影片级目录"""
|
||||
make_file(str(tmp_path / "vrkm-1763/a-poster.jpg"), 4096)
|
||||
make_file(str(tmp_path / "vrkm-1763/sub/b.url"), 16)
|
||||
make_file(str(tmp_path / "vrkm-1763/sub/deep/deeper/c.jpg"), 1024)
|
||||
make_file(str(tmp_path / "keepme/x/movie.mp4"), 120 * MB)
|
||||
run(tmp_path, "--apply")
|
||||
# 整条空目录链连同影片级目录一起消失
|
||||
assert os.path.exists(str(tmp_path / "vrkm-1763")) is False
|
||||
assert os.path.exists(str(tmp_path / "vrkm-1763/sub/deep/deeper")) is False
|
||||
# 有视频的目录不受影响,root 本身保留
|
||||
assert os.path.exists(str(tmp_path / "keepme/x/movie.mp4")) is True
|
||||
assert os.path.isdir(str(tmp_path)) is True
|
||||
|
||||
|
||||
def test_全部删除后root仍保留(tmp_path):
|
||||
make_file(str(tmp_path / "only/a.url"), 16)
|
||||
make_file(str(tmp_path / "only/b.jpg"), 1024)
|
||||
run(tmp_path, "--apply")
|
||||
assert os.path.isdir(str(tmp_path)) is True
|
||||
assert os.listdir(str(tmp_path)) == []
|
||||
|
||||
|
||||
def test_keep_empty_dirs(tmp_path):
|
||||
make_file(str(tmp_path / "x/movie.mp4"), 120 * MB)
|
||||
make_file(str(tmp_path / "x/movie.trickplay/0.jpg"), 1024)
|
||||
run(tmp_path, "--apply", "--keep-empty-dirs")
|
||||
assert os.path.isdir(str(tmp_path / "x/movie.trickplay")) is True
|
||||
|
||||
|
||||
def test_不跟随符号链接目录(tmp_path):
|
||||
"""符号链接目录内的文件不应被扫描/删除"""
|
||||
outside = tmp_path / "outside"
|
||||
victim = make_file(str(outside / "keep-me.jpg"), 1024)
|
||||
target = tmp_path / "root" / "link"
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
os.symlink(str(outside), str(target))
|
||||
run(target.parent, "--apply")
|
||||
assert os.path.exists(victim) is True
|
||||
|
||||
|
||||
def test_目录不存在返回2(tmp_path):
|
||||
assert run(tmp_path / "not-exist") == 2
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 汇总统计
|
||||
|
||||
def test_汇总统计(tmp_path):
|
||||
make_file(str(tmp_path / "x/movie.mp4"), 120 * MB)
|
||||
make_file(str(tmp_path / "x/movie.srt"), 1024)
|
||||
make_file(str(tmp_path / "x/junk.url"), 16)
|
||||
entries, dirs = c.collect(str(tmp_path))
|
||||
c.decide(entries, 100 * MB, True, True, keep_covers=False)
|
||||
assert len(entries) == 3
|
||||
assert sum(1 for e in entries if e.keep) == 2
|
||||
assert sum(1 for e in entries if not e.keep) == 1
|
||||
assert [e.reason for e in entries if e.name == "movie.srt"] == ["字幕"]
|
||||
|
||||
|
||||
def test_human_size():
|
||||
assert c.human_size(0) == "0.0 B"
|
||||
assert c.human_size(1024) == "1.0 KB"
|
||||
assert c.human_size(100 * MB) == "100.0 MB"
|
||||
Reference in New Issue
Block a user