fix: 修复回忆列表并发死锁并收敛多表查询

This commit is contained in:
2026-11-21 16:54:07 +08:00
parent 4b65eb57b1
commit 9c3e22680d
17 changed files with 497 additions and 128 deletions
@@ -0,0 +1,74 @@
# Agent Note: 回忆列表首次请求 500 与逐行 N+1 查询
Status: implemented
## Problem
进入 `/review` 时前端同时发出两个回忆列表请求(「继续上次回忆」的 `status=ACTIVE&pageSize=1` 与「最近回忆」的 `pageSize=4`)。首次进入时其中一个稳定返回
`{"code":500,"message":"操作没有成功,请稍后再试"}`,另一个常常挂到前端 30s 超时才被放弃;刷新一次后又能正常返回。
根因有两层,互相放大:
1. **所有者锁的取法会死锁。** `ReviewRecallStore.lock()` 先发 `INSERT IGNORE INTO review_recall_owner_locks`,行已存在时这条语句只拿
共享锁(S),紧接着的 `SELECT ... FOR UPDATE` 要升级成排他锁(X)。两个并发请求各持 S 锁互等 X 锁,InnoDB 判定死锁并回滚其一,
Spring 抛 `DeadlockLoserDataAccessException`,被 `GlobalExceptionHandler` 兜底成 500。
`SHOW ENGINE INNODB STATUS` 的 `LATEST DETECTED DEADLOCK` 里两条事务都是
`SELECT owner_id FROM review_recall_owner_locks WHERE owner_id = '1' FOR UPDATE`,各自 `HOLDS S, WAITING X`;同一份并发用纯 JDBC 复现,
旧写法 6/6 轮死锁,改成新写法 0/12 次失败。
2. **读路径逐行放大成几十条查询。** `list()` 对每一行都 `store.load()`:按 `recall_num` 回表一次、`reconcile` 做两次全表校验(碎片 id、用户标签)、
再对每个来源 `selectById` 判可达性、最后 `store.detail()` 对每条 AI 请求各查 `review_recall_requests` 与 `ai_tasks` 两次。
样例回忆(13 个来源、13 条 AI 请求)单行约 **47 条 SQL**;远端 MySQL 单程 40–80ms,一次 4 行的列表就是十几秒,整个事务期间一直持着所有者锁,
把第 1 点里的冲突窗口拉到几十秒。而 `list` 返回的 `RecallSummary` 里根本没有 `requests` 字段,那 26 条查询是纯浪费。
## Decision
**锁的形态:** `ReviewRecallMapper.ensureOwnerLock` 改为
`INSERT INTO review_recall_owner_locks (owner_id) VALUES (?) ON DUPLICATE KEY UPDATE owner_id = owner_id`。
命中已有行时 MySQL 对这个「无变化更新」加的是排他锁,随后的 `SELECT ... FOR UPDATE` 在同一事务里已是 X 锁,不再有 S→X 升级,也就没有互等。
读路径继续保留这把所有者锁:`reconcile` 会就地抹除已不可访问的来源,必须和草稿写入串行,否则一次先读后写的草稿提交会把刚抹掉的内容写回来。
**读路径:**
- `list()` 直接用分页行里已经查出来的 `document_json` 解析文档,不再按 `recall_num` 回表;只发 `selectPage` 自带的一次 COUNT,不再额外 `selectCount`。
- 新增 `RecallReferenceLoader.SourceIndex`:按需把「可达碎片 id / 报告 id / 应用场景 id / 任务号 / 用户标签」各查一次(整表 `select(id)` 级别),
标签范围有效性与来源可达性都在内存里判断。`list` 一次建索引、逐行复用;单文档入口按需建一次。原来对每个来源一次 `selectById` 的循环消失。
- 新增 `RecallDetailAssembler`:`review_recall_requests` 与 `ai_tasks` 各一条 `IN` 取齐整批 taskNum,按 `taskNum` 建索引后逐条装饰。
列表走 `summary()`,连这两批都不查;详情、草稿、结束等写路径走 `detail(doc)`,文档内所有 AI 请求共用一次批量取数。
- `RecallExpressionValidator` 只在确实出现新出处时才装配一次详情,把「每条出处各查两次」变成「每次保存最多两次」。
样例回忆上的 SQL 条数:列表 **47 → 7**,详情 **44 → 8**;并发两请求的等待时间从 2.1–4.4s(且约 80% 概率 500)降到 0.4–0.9s,5 轮并发 10 次全部成功。
同一批把其它「随行数/次数放大」的读路径一并收敛(都是换成批量 IN 或一条 SET 更新,不改响应字段):
- `GET /study-sessions/tasks/{num}/reports`:逐个报告查学习预期 → 整页一次 `IN`。
- 标签导图「待更新」:逐张 `UPDATE` → 命中集合一条 `UPDATE ... WHERE id IN (...)`。
- 隐私清算:删除任务时对每个报告/应用场景各递归一次(每次重新加锁 + 重扫整张回忆表)→ 先收集 sourceId 再扫一遍;
`tagsChanged` 里逐篇回忆两次全表校验 → 整批共用一个来源索引。
- `GET /export/range`:逐任务调 `getReviewTaskStats`(每任务 4 条聚合)→ 一批任务一次取齐;`sumTaskDuration` 的整表 `GROUP BY`
加上 `task_num IN (...)` 过滤(T 个任务时约 `4 + 5T` 条降为固定 9 条)。
## Alternatives considered
- **读路径完全不加锁。** 列表并发度最高,也能彻底避开锁争用;但 `reconcile` 的抹除是隐私要求,无锁时一次并发草稿保存会用旧文档覆盖抹除结果,
等同于把已删除内容写回来。要安全就得改成带旧文档摘要的 CAS 写入,复杂度和风险都高于本次收益,因此保留锁、只把事务缩短到个位数查询。
- **在登录时就把所有者锁行种好,读路径只做 `SELECT ... FOR UPDATE`。** 稳态下同样没有 S→X 升级,语义更直白;但需要一条回填迁移,
而且「行缺失」的兜底插入仍在,等于两套路径并存。`ON DUPLICATE KEY UPDATE` 一行同时覆盖「确保存在」和「取得排他锁」,故采用后者。
- **捕获死锁后重试。** 能消掉 500,但两个请求仍在做无效功,且重试会掩盖下一次同类写法回归;根因是锁模式,应该在 SQL 形态上消灭。
- **把回忆列表改成单条 JOIN 查询。** 回忆的读取单位是整份 `document_json`(表达树 + 冻结的资料快照),跨表的只有 AI 请求状态;
真正需要跨表的是详情里的两批装饰,`IN` 已经够用,JOIN 只会把 JSON 文档拆成多行再在内存里重组,反而更慢。
- **把 `reconcile` 挪到定时任务、读路径不再校验。** 事务最短,但来源被删除后列表会继续显示 `VALID`/可用正文,直到下一轮扫描,不符合隐私口径。
## Consequences
列表仍是「每个用户串行」的读路径:并发两个列表请求时第二个会等第一个提交,只是等待从几十秒降到几百毫秒。
`list` 现在依赖分页行里的 `document_json`,若以后列表查询改成 `.select(...)` 裁列,必须同时把文档列选回来,否则所有行都会被当成不可读文档删除。
`SourceIndex` 用「整表 id 集合」判断可达性,前提是这些表按 `created_by` 租户过滤(拦截器已保证);绕过租户的查询不能复用这个索引。
`RecallsSummary` 不再经过 `detail()` 装饰,`reference` 也只保留文档里的汇总;如果以后摘要需要展示 AI 请求状态,必须显式调用 `detail()`,不能假设它已经在。
## Verification
- 真实 MySQL 并发复现:脚本对同一用户并发发两个列表请求,旧锁形态 6/6 轮死锁,新锁形态 0/12 次失败;`performance_schema.data_locks` 中不再出现同事务的 S+X 并存。
- SQL 条数:用 `logging.level...mapper=DEBUG` 抓单次调用语句数,列表 7 条、详情 8 条(改造前 47 / 44)。
- `./mvnw -o validate`(Checkstyle)与 `./mvnw -o test` 全量 608 个用例通过,含 `ReviewRecallServiceIntegrationTest` 的列表删除不可读文档、
资料清除、AI 结果装饰等断言。
@@ -6,8 +6,12 @@ import org.apache.ibatis.annotations.Mapper;
@Mapper @Mapper
public interface ReviewRecallMapper extends BaseMapper<ReviewRecallEntity> { public interface ReviewRecallMapper extends BaseMapper<ReviewRecallEntity> {
// 命中已有行时必须走「独占锁」而不是 INSERT IGNORE 的共享锁:否则本事务随后的 SELECT ... FOR UPDATE
// 要做 S→X 升级,两个并发请求各持共享锁互等,InnoDB 直接判定死锁并回滚其一(前端表现为 500)。
@com.baomidou.mybatisplus.annotation.InterceptorIgnore(tenantLine = "true") @com.baomidou.mybatisplus.annotation.InterceptorIgnore(tenantLine = "true")
@org.apache.ibatis.annotations.Insert("INSERT IGNORE INTO review_recall_owner_locks (owner_id) VALUES (#{userId})") @org.apache.ibatis.annotations.Insert(
"INSERT INTO review_recall_owner_locks (owner_id) VALUES (#{userId})"
+ " ON DUPLICATE KEY UPDATE owner_id = owner_id")
int ensureOwnerLock(@org.apache.ibatis.annotations.Param("userId") String userId); int ensureOwnerLock(@org.apache.ibatis.annotations.Param("userId") String userId);
// Dedicated lock row, not a login/session mutation. Explicit owner id prevents cross-user locking. // Dedicated lock row, not a login/session mutation. Explicit owner id prevents cross-user locking.
@@ -16,14 +20,18 @@ public interface ReviewRecallMapper extends BaseMapper<ReviewRecallEntity> {
"SELECT owner_id FROM review_recall_owner_locks WHERE owner_id = #{userId} FOR UPDATE") "SELECT owner_id FROM review_recall_owner_locks WHERE owner_id = #{userId} FOR UPDATE")
String lockUser(@org.apache.ibatis.annotations.Param("userId") String userId); String lockUser(@org.apache.ibatis.annotations.Param("userId") String userId);
@org.apache.ibatis.annotations.Select( @org.apache.ibatis.annotations.Select("<script>"
"SELECT task_num, SUM(active_duration_ms) AS active_duration_ms FROM review_recalls WHERE deleted = 0 AND" + "SELECT task_num, SUM(active_duration_ms) AS active_duration_ms FROM review_recalls WHERE deleted = 0 AND"
+ " scope_type = 'TASK' GROUP BY task_num") + " scope_type = 'TASK'"
+ " <if test='taskNums != null and taskNums.size() > 0'>AND task_num IN"
+ " <foreach item='num' collection='taskNums' open='(' separator=',' close=')'>#{num}</foreach></if>"
+ " GROUP BY task_num</script>")
@org.apache.ibatis.annotations.Results( @org.apache.ibatis.annotations.Results(
id = "taskRecallDuration", id = "taskRecallDuration",
value = { value = {
@org.apache.ibatis.annotations.Result(column = "task_num", property = "taskNum"), @org.apache.ibatis.annotations.Result(column = "task_num", property = "taskNum"),
@org.apache.ibatis.annotations.Result(column = "active_duration_ms", property = "activeDurationMs") @org.apache.ibatis.annotations.Result(column = "active_duration_ms", property = "activeDurationMs")
}) })
java.util.List<com.guo.learningprogresstracker.dto.recall.TaskRecallDuration> sumTaskDuration(); java.util.List<com.guo.learningprogresstracker.dto.recall.TaskRecallDuration> sumTaskDuration(
@org.apache.ibatis.annotations.Param("taskNums") java.util.Collection<String> taskNums);
} }
@@ -7,6 +7,7 @@ import com.guo.learningprogresstracker.dto.ReviewTaskStats;
import com.guo.learningprogresstracker.entity.StudyReportFragmentsEntity; import com.guo.learningprogresstracker.entity.StudyReportFragmentsEntity;
import com.guo.learningprogresstracker.entity.StudyReportsEntity; import com.guo.learningprogresstracker.entity.StudyReportsEntity;
import com.guo.learningprogresstracker.exception.NotFindEntitiesException; import com.guo.learningprogresstracker.exception.NotFindEntitiesException;
import java.util.Collection;
import java.util.List; import java.util.List;
/** /**
@@ -38,6 +39,14 @@ public interface ReviewService {
*/ */
ReviewTaskStats getReviewTaskStats(String taskNum); ReviewTaskStats getReviewTaskStats(String taskNum);
/**
* 批量获取一组任务的复习统计数据(导出范围用)。
*
* <p>逐任务调用会把「会话统计 / 报告数 / 碎片数 / 回忆时长」四条聚合乘以任务数,
* 这里一次取齐;任务不存在时仍返回该 taskNum 的条目,只是 taskName 为 null。</p>
*/
List<ReviewTaskStats> getReviewTaskStats(Collection<String> taskNums);
/** /**
* 获取指定任务下的所有报告和残片 * 获取指定任务下的所有报告和残片
*/ */
@@ -2,6 +2,8 @@ package com.guo.learningprogresstracker.service;
import com.guo.learningprogresstracker.entity.StudyExpectationsEntity; import com.guo.learningprogresstracker.entity.StudyExpectationsEntity;
import com.guo.learningprogresstracker.exception.ErrorParameterException; import com.guo.learningprogresstracker.exception.ErrorParameterException;
import java.util.Collection;
import java.util.Map;
/** /**
* 学习预期服务 * 学习预期服务
@@ -18,6 +20,13 @@ public interface StudyExpectationsService {
*/ */
StudyExpectationsEntity getBySessionNum(String sessionNum); StudyExpectationsEntity getBySessionNum(String sessionNum);
/**
* 批量取一批会话的学习预期:返回 sessionNum → 描述,没有预期的会话不在结果里。
*
* <p>列表页按会话逐条查是典型的 N+1,一次 IN 取齐即可。</p>
*/
Map<String, String> descriptionsBySessionNums(Collection<String> sessionNums);
/** /**
* 删除学习会话的学习预期(误操作结束时会话不留任何数据) * 删除学习会话的学习预期(误操作结束时会话不留任何数据)
*/ */
@@ -7,6 +7,7 @@ import com.guo.learningprogresstracker.dto.ExportDocument;
import com.guo.learningprogresstracker.dto.ExportEntry; import com.guo.learningprogresstracker.dto.ExportEntry;
import com.guo.learningprogresstracker.dto.ExportResult; import com.guo.learningprogresstracker.dto.ExportResult;
import com.guo.learningprogresstracker.dto.ExportTaskSection; import com.guo.learningprogresstracker.dto.ExportTaskSection;
import com.guo.learningprogresstracker.dto.ReviewTaskStats;
import com.guo.learningprogresstracker.entity.StudyReportFragmentsEntity; import com.guo.learningprogresstracker.entity.StudyReportFragmentsEntity;
import com.guo.learningprogresstracker.entity.StudyReportsEntity; import com.guo.learningprogresstracker.entity.StudyReportsEntity;
import com.guo.learningprogresstracker.entity.StudySessionsEntity; import com.guo.learningprogresstracker.entity.StudySessionsEntity;
@@ -152,8 +153,13 @@ public class ExportServiceImpl implements ExportService {
Map<String, ExportTaskSection> sections = new LinkedHashMap<>(); Map<String, ExportTaskSection> sections = new LinkedHashMap<>();
appendRangeItems(sections, reports, fragments); appendRangeItems(sections, reports, fragments);
// 统计一次取齐:逐任务调用会把 4 条聚合乘以任务数
Map<String, ReviewTaskStats> statsByTask = new LinkedHashMap<>();
for (ReviewTaskStats stats : reviewService.getReviewTaskStats(sections.keySet())) {
statsByTask.put(stats.getTaskNum(), stats);
}
sections.values().forEach(section -> { sections.values().forEach(section -> {
section.setStats(reviewService.getReviewTaskStats(section.getTaskNum())); section.setStats(statsByTask.get(section.getTaskNum()));
alignContentCounts(section); alignContentCounts(section);
}); });
@@ -30,6 +30,7 @@ import com.guo.learningprogresstracker.service.review.ReviewFeedService;
import com.guo.learningprogresstracker.service.review.ReviewStatsService; import com.guo.learningprogresstracker.service.review.ReviewStatsService;
import java.time.LocalDateTime; import java.time.LocalDateTime;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collection;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
@@ -80,6 +81,11 @@ public class ReviewServiceImpl implements ReviewService {
return reviewStatsService.getReviewTaskStats(pageNum, pageSize, markerName); return reviewStatsService.getReviewTaskStats(pageNum, pageSize, markerName);
} }
@Override
public List<ReviewTaskStats> getReviewTaskStats(Collection<String> taskNums) {
return reviewStatsService.getReviewTaskStats(taskNums);
}
@Override @Override
public ReviewTaskStats getReviewTaskStats(String taskNum) { public ReviewTaskStats getReviewTaskStats(String taskNum) {
return reviewStatsService.getReviewTaskStats(taskNum); return reviewStatsService.getReviewTaskStats(taskNum);
@@ -219,24 +219,26 @@ public class ReviewTagMindMapServiceImpl implements ReviewTagMindMapService {
} }
List<ReviewTagMindMapEntity> maps = tagMindMapMapper.selectList( List<ReviewTagMindMapEntity> maps = tagMindMapMapper.selectList(
Wrappers.<ReviewTagMindMapEntity>lambdaQuery().eq(ReviewTagMindMapEntity::getNeedsUpdate, false)); Wrappers.<ReviewTagMindMapEntity>lambdaQuery().eq(ReviewTagMindMapEntity::getNeedsUpdate, false));
int marked = 0; // 命中判断在内存里做完,最后一条 UPDATE 改一批:逐张 UPDATE 会随导图数量线性放大写往返
List<Integer> staleIds = new ArrayList<>();
for (ReviewTagMindMapEntity map : maps) { for (ReviewTagMindMapEntity map : maps) {
Set<String> mapTags = Set<String> mapTags =
Set.copyOf(TagScopeTool.normalizedTags(TagScopeTool.parseStoredTags(map.getTagNames()))); Set.copyOf(TagScopeTool.normalizedTags(TagScopeTool.parseStoredTags(map.getTagNames())));
if (Collections.disjoint(mapTags, changed)) { if (!Collections.disjoint(mapTags, changed)) {
continue; staleIds.add(map.getId());
}
}
if (staleIds.isEmpty()) {
return;
} }
tagMindMapMapper.update( tagMindMapMapper.update(
new ReviewTagMindMapEntity(), new ReviewTagMindMapEntity(),
Wrappers.<ReviewTagMindMapEntity>lambdaUpdate() Wrappers.<ReviewTagMindMapEntity>lambdaUpdate()
.eq(ReviewTagMindMapEntity::getId, map.getId()) .in(ReviewTagMindMapEntity::getId, staleIds)
.set(ReviewTagMindMapEntity::getNeedsUpdate, true) .set(ReviewTagMindMapEntity::getNeedsUpdate, true)
.set(ReviewTagMindMapEntity::getNeedsUpdateTime, LocalDateTime.now())); .set(ReviewTagMindMapEntity::getNeedsUpdateTime, LocalDateTime.now()));
marked++; log.info("标签[{}]的素材发生变化,{} 张标签导图已置为待更新(重建仍由用户手动触发)",
} String.join("、", changed), staleIds.size());
if (marked > 0) {
log.info("标签[{}]的素材发生变化,{} 张标签导图已置为待更新(重建仍由用户手动触发)", String.join("、", changed), marked);
}
} }
// ============ 内部方法 ============ // ============ 内部方法 ============
@@ -7,6 +7,9 @@ import com.guo.learningprogresstracker.exception.ErrorParameterException;
import com.guo.learningprogresstracker.mapper.StudyExpectationsMapper; import com.guo.learningprogresstracker.mapper.StudyExpectationsMapper;
import com.guo.learningprogresstracker.mapper.StudySessionsMapper; import com.guo.learningprogresstracker.mapper.StudySessionsMapper;
import com.guo.learningprogresstracker.service.StudyExpectationsService; import com.guo.learningprogresstracker.service.StudyExpectationsService;
import java.util.Collection;
import java.util.Map;
import java.util.stream.Collectors;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@@ -52,6 +55,22 @@ public class StudyExpectationsServiceImpl implements StudyExpectationsService {
.last("LIMIT 1")); .last("LIMIT 1"));
} }
@Override
public Map<String, String> descriptionsBySessionNums(Collection<String> sessionNums) {
if (sessionNums == null || sessionNums.isEmpty()) {
return Map.of();
}
return studyExpectationsMapper
.selectList(Wrappers.<StudyExpectationsEntity>lambdaQuery()
.in(StudyExpectationsEntity::getSessionNum, sessionNums))
.stream()
.filter(e -> e.getDescription() != null)
.collect(Collectors.toMap(
StudyExpectationsEntity::getSessionNum,
StudyExpectationsEntity::getDescription,
(first, second) -> first));
}
@Override @Override
public void deleteBySessionNum(String sessionNum) { public void deleteBySessionNum(String sessionNum) {
studyExpectationsMapper.delete( studyExpectationsMapper.delete(
@@ -27,6 +27,8 @@ import com.guo.learningprogresstracker.service.session.StudyReportDraftService;
import com.guo.learningprogresstracker.service.session.StudySessionConvergenceService; import com.guo.learningprogresstracker.service.session.StudySessionConvergenceService;
import java.time.LocalDateTime; import java.time.LocalDateTime;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Map;
import java.util.Objects;
import java.util.Optional; import java.util.Optional;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
@@ -293,11 +295,17 @@ public class StudySessionsServiceImpl extends ServiceImpl<StudySessionsMapper, S
.like(StudyReportsEntity::getExpectationVerdict, keyword)); .like(StudyReportsEntity::getExpectationVerdict, keyword));
} }
Page<StudyReportsEntity> result = studyReportsMapper.selectPage(pg, wrapper); Page<StudyReportsEntity> result = studyReportsMapper.selectPage(pg, wrapper);
// 填充每个报告对应会话的预期目标 // 填充每个报告对应会话的预期目标:整页一次 IN 取齐,不再逐条查会话预期
Map<String, String> expectations = studyExpectationsService.descriptionsBySessionNums(
result.getRecords().stream()
.map(StudyReportsEntity::getSessionNum)
.filter(Objects::nonNull)
.distinct()
.toList());
for (StudyReportsEntity report : result.getRecords()) { for (StudyReportsEntity report : result.getRecords()) {
if (report.getSessionNum() != null) { String description = expectations.get(report.getSessionNum());
Optional.ofNullable(studyExpectationsService.getBySessionNum(report.getSessionNum())) if (description != null) {
.ifPresent(e -> report.setSessionExpectation(e.getDescription())); report.setSessionExpectation(description);
} }
} }
return result; return result;
@@ -0,0 +1,136 @@
package com.guo.learningprogresstracker.service.recall;
import com.baomidou.mybatisplus.core.toolkit.Wrappers;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallAiResult;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDetail;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDocument;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallRequest;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSnapshot;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSummary;
import com.guo.learningprogresstracker.entity.AiTaskEntity;
import com.guo.learningprogresstracker.entity.ReviewRecallRequestEntity;
import com.guo.learningprogresstracker.mapper.AiTaskMapper;
import com.guo.learningprogresstracker.mapper.ReviewRecallRequestMapper;
import com.guo.learningprogresstracker.service.ai.RecallJson;
import java.time.ZoneId;
import java.util.Collection;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Component;
/**
* 回忆详情的出参装配:把文档里冻结的 requests 与库里的 AI 任务现状合并。
*
* <p>一次请求只查两批:{@code review_recall_requests} 与 {@code ai_tasks} 各一条 {@code IN},
* 不再对每条 request 各查两次。列表只要摘要,连这两批都不查。</p>
*/
@Component
@RequiredArgsConstructor
public class RecallDetailAssembler {
private final ReviewRecallRequestMapper requests;
private final AiTaskMapper aiTasks;
private final RecallJson json;
/** 一次请求内的 AI 任务快照,按 taskNum 索引。 */
public record Facts(Map<String, ReviewRecallRequestEntity> associations, Map<String, AiTaskEntity> tasks) {
public static Facts empty() {
return new Facts(Map.of(), Map.of());
}
public ReviewRecallRequestEntity association(String taskNum) {
return taskNum == null ? null : associations.get(taskNum);
}
public AiTaskEntity task(String taskNum) {
return taskNum == null ? null : tasks.get(taskNum);
}
}
public RecallDetail detail(RecallDocument doc) {
return detail(doc, facts(List.of(doc)));
}
/** 列表摘要:requests 留在文档里,toSummary 不会用到,不必再查库。 */
public RecallDetail summary(RecallDocument doc) {
decorateReference(doc);
return doc.getDetail();
}
public RecallDetail detail(RecallDocument doc, Facts facts) {
RecallDetail v = decorateReference(doc);
v.setRequests(v.getRequests().stream()
.map(r -> decorateRequest(r, facts.association(r.taskNum()), facts.task(r.taskNum())))
.toList());
return v;
}
public Facts facts(Collection<RecallDocument> docs) {
Set<String> taskNums = new LinkedHashSet<>();
for (RecallDocument doc : docs) {
for (RecallRequest request : doc.getDetail().getRequests()) {
if (request.taskNum() != null) {
taskNums.add(request.taskNum());
}
}
}
if (taskNums.isEmpty()) {
return Facts.empty();
}
Map<String, ReviewRecallRequestEntity> associations = requests
.selectList(Wrappers.<ReviewRecallRequestEntity>lambdaQuery()
.in(ReviewRecallRequestEntity::getTaskNum, taskNums))
.stream()
.collect(LinkedHashMap::new, (map, row) -> map.putIfAbsent(row.getTaskNum(), row), Map::putAll);
Map<String, AiTaskEntity> tasks = aiTasks
.selectList(Wrappers.<AiTaskEntity>lambdaQuery().in(AiTaskEntity::getTaskNum, taskNums))
.stream()
.collect(LinkedHashMap::new, (map, row) -> map.putIfAbsent(row.getTaskNum(), row), Map::putAll);
return new Facts(associations, tasks);
}
private static RecallDetail decorateReference(RecallDocument doc) {
RecallDetail v = doc.getDetail();
ReferenceSnapshot reference = doc.getReference();
v.setReference(new ReferenceSummary(
reference.status(),
RecallReferenceLoader.knowledgeSources(reference.sources()).size(),
reference.limitations()));
return v;
}
private RecallRequest decorateRequest(
RecallRequest r, ReviewRecallRequestEntity association, AiTaskEntity task) {
boolean redacted = association == null || Integer.valueOf(1).equals(association.getRedacted());
String status = redacted ? "FAILED" : task == null ? "FAILED" : task.getStatus();
String code = redacted ? "SOURCE_REDACTED" : task == null ? "AI_UNAVAILABLE" : task.getErrorCode();
String message = task == null ? "这次 AI 结果已不可用" : task.getErrorMessage();
RecallAiResult result = null;
if (task != null && "SUCCEEDED".equals(status)) {
try {
result = json.readStored(task.getResultPayload(), RecallAiResult.class);
} catch (RuntimeException ex) {
status = "FAILED";
code = "AI_UNAVAILABLE";
message = "这次 AI 结果是旧版本格式,暂时无法展示;请重新请求。";
}
}
return new RecallRequest(
r.requestId(),
r.taskNum(),
r.type(),
r.hintForm(),
r.promptVersion(),
status,
code,
message,
r.submittedAt(),
task == null || task.getFinishedTime() == null
? null
: task.getFinishedTime().atZone(ZoneId.systemDefault()).toInstant(),
result);
}
}
@@ -3,6 +3,7 @@ package com.guo.learningprogresstracker.service.recall;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.Expression; import com.guo.learningprogresstracker.dto.recall.RecallDtos.Expression;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ExpressionNode; import com.guo.learningprogresstracker.dto.recall.RecallDtos.ExpressionNode;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.Provenance; import com.guo.learningprogresstracker.dto.recall.RecallDtos.Provenance;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDetail;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDocument; import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDocument;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallRequest; import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallRequest;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.Timing; import com.guo.learningprogresstracker.dto.recall.RecallDtos.Timing;
@@ -32,7 +33,7 @@ public class RecallExpressionValidator {
/** 前端 JS 能精确表示的最大整数,超出即视为客户端计时失真。 */ /** 前端 JS 能精确表示的最大整数,超出即视为客户端计时失真。 */
private static final long MAX_SAFE_INTEGER_MS = 9007199254740991L; private static final long MAX_SAFE_INTEGER_MS = 9007199254740991L;
private final ReviewRecallStore store; private final RecallDetailAssembler details;
public void validate(RecallDocument doc, Expression expression) { public void validate(RecallDocument doc, Expression expression) {
if (expression == null || expression.schemaVersion() != 1 || expression.root() == null) { if (expression == null || expression.schemaVersion() != 1 || expression.root() == null) {
@@ -42,7 +43,28 @@ public class RecallExpressionValidator {
int[] counts = {0, 0}; int[] counts = {0, 0};
Map<String, Set<Provenance>> accepted = new HashMap<>(); Map<String, Set<Provenance>> accepted = new HashMap<>();
collectAcceptedProvenance(doc.getDetail().getExpression().root(), accepted); collectAcceptedProvenance(doc.getDetail().getExpression().root(), accepted);
validateNode(doc, expression.root(), 1, ids, counts, accepted); // 出处写过的直接放行;只有出现新出处时才查一次 AI 任务现状(两条 IN),不再按引用逐条查
RecallDetail read = hasPending(expression.root(), accepted) ? details.detail(doc) : null;
validateNode(read, expression.root(), 1, ids, counts, accepted);
}
/** 形状异常不在这里报错,交给后续校验拒绝;这里只判断「有没有需要核验的新出处」。 */
private static boolean hasPending(ExpressionNode node, Map<String, Set<Provenance>> accepted) {
if (node == null || node.provenance() == null || node.children() == null) {
return true;
}
Set<Provenance> known = accepted.getOrDefault(node.id(), Set.of());
for (Provenance p : node.provenance()) {
if (!known.contains(p)) {
return true;
}
}
for (ExpressionNode child : node.children()) {
if (hasPending(child, accepted)) {
return true;
}
}
return false;
} }
private static void collectAcceptedProvenance(ExpressionNode node, Map<String, Set<Provenance>> accepted) { private static void collectAcceptedProvenance(ExpressionNode node, Map<String, Set<Provenance>> accepted) {
@@ -59,7 +81,7 @@ public class RecallExpressionValidator {
/** 主干只做分发:形状 → 出处 → 子节点。 */ /** 主干只做分发:形状 → 出处 → 子节点。 */
private void validateNode( private void validateNode(
RecallDocument doc, RecallDetail read,
ExpressionNode n, ExpressionNode n,
int depth, int depth,
Set<String> ids, Set<String> ids,
@@ -67,10 +89,10 @@ public class RecallExpressionValidator {
Map<String, Set<Provenance>> accepted) { Map<String, Set<Provenance>> accepted) {
checkNodeShape(n, depth, ids, counts); checkNodeShape(n, depth, ids, counts);
for (Provenance p : n.provenance()) { for (Provenance p : n.provenance()) {
checkNodeProvenance(doc, n, p, accepted); checkNodeProvenance(read, n, p, accepted);
} }
for (ExpressionNode child : n.children()) { for (ExpressionNode child : n.children()) {
validateNode(doc, child, depth + 1, ids, counts, accepted); validateNode(read, child, depth + 1, ids, counts, accepted);
} }
} }
@@ -98,7 +120,7 @@ public class RecallExpressionValidator {
/** 单条出处:已写过的不再复核,其余必须指向本次回忆中已成功的请求结果。 */ /** 单条出处:已写过的不再复核,其余必须指向本次回忆中已成功的请求结果。 */
private void checkNodeProvenance( private void checkNodeProvenance(
RecallDocument doc, ExpressionNode n, Provenance p, Map<String, Set<Provenance>> accepted) { RecallDetail read, ExpressionNode n, Provenance p, Map<String, Set<Provenance>> accepted) {
if (p == null) { if (p == null) {
invalid(); invalid();
} }
@@ -106,10 +128,10 @@ public class RecallExpressionValidator {
return; return;
} }
if (!Set.of("HINT", "FEEDBACK").contains(p.kind() == null ? "" : p.kind()) if (!Set.of("HINT", "FEEDBACK").contains(p.kind() == null ? "" : p.kind())
|| !doc.getDetail().getRecallNum().equals(p.recallNum())) { || !read.getRecallNum().equals(p.recallNum())) {
invalid(); invalid();
} }
RecallRequest request = requestResult(doc, p.requestTaskNum()); RecallRequest request = requestResult(read, p.requestTaskNum());
boolean matched = "HINT".equals(p.kind()) boolean matched = "HINT".equals(p.kind())
? request.result().hints().stream().anyMatch(i -> i.id().equals(p.itemId())) ? request.result().hints().stream().anyMatch(i -> i.id().equals(p.itemId()))
: request.result().findings().stream().anyMatch(i -> i.id().equals(p.itemId())); : request.result().findings().stream().anyMatch(i -> i.id().equals(p.itemId()));
@@ -118,8 +140,8 @@ public class RecallExpressionValidator {
} }
} }
private RecallRequest requestResult(RecallDocument doc, String taskNum) { private static RecallRequest requestResult(RecallDetail read, String taskNum) {
return store.detail(doc).getRequests().stream() return read.getRequests().stream()
.filter(r -> r.taskNum().equals(taskNum) && "SUCCEEDED".equals(r.status()) && r.result() != null) .filter(r -> r.taskNum().equals(taskNum) && "SUCCEEDED".equals(r.status()) && r.result() != null)
.findFirst() .findFirst()
.orElseThrow(() -> new RecallBusinessException("REFERENCE_UNAVAILABLE", "这份 AI 回应尚不可展示,请等待完成或重新请求")); .orElseThrow(() -> new RecallBusinessException("REFERENCE_UNAVAILABLE", "这份 AI 回应尚不可展示,请等待完成或重新请求"));
@@ -12,9 +12,12 @@ import com.guo.learningprogresstracker.mapper.StudySessionsMapper;
import com.guo.learningprogresstracker.mapper.TaskApplicationMapper; import com.guo.learningprogresstracker.mapper.TaskApplicationMapper;
import com.guo.learningprogresstracker.service.ai.RecallJson; import com.guo.learningprogresstracker.service.ai.RecallJson;
import com.guo.learningprogresstracker.utils.TagScopeTool; import com.guo.learningprogresstracker.utils.TagScopeTool;
import java.util.ArrayList;
import java.util.HashSet;
import java.util.List; import java.util.List;
import java.util.Locale; import java.util.Locale;
import java.util.Set; import java.util.Set;
import java.util.stream.Collectors;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service; import org.springframework.stereotype.Service;
@@ -50,18 +53,24 @@ public class RecallPrivacyService {
@Transactional(rollbackFor = Exception.class) @Transactional(rollbackFor = Exception.class)
public void sourceDeleted(String type, String id) { public void sourceDeleted(String type, String id) {
store.lock(); store.lock();
String sourceId = type.toLowerCase(Locale.ROOT) + ":" + id; redactSources(Set.of(type.toLowerCase(Locale.ROOT) + ":" + id));
}
/** 一次扫描抹掉一批来源:删除任务时报告与应用可能有几十个,逐个递归会把整张回忆表扫几十遍。 */
private void redactSources(Set<String> sourceIds) {
for (ReviewRecallEntity row : store.all()) { for (ReviewRecallEntity row : store.all()) {
RecallDocument doc = readable(row); RecallDocument doc = readable(row);
if (doc == null) { if (doc == null) {
continue; continue;
} }
boolean found = doc.getReference().sources().stream() Set<String> hit = doc.getReference().sources().stream()
.anyMatch(s -> sourceId.equals(s.sourceId()) && "AVAILABLE".equals(s.status())); .filter(s -> "AVAILABLE".equals(s.status()) && sourceIds.contains(s.sourceId()))
if (!found) { .map(s -> s.sourceId())
.collect(Collectors.toSet());
if (hit.isEmpty()) {
continue; continue;
} }
doc.setReference(store.redactReference(doc.getReference(), Set.of(sourceId), "DELETED")); doc.setReference(store.redactReference(doc.getReference(), hit, "DELETED"));
store.redactRequests(doc); store.redactRequests(doc);
store.persist(doc); store.persist(doc);
} }
@@ -83,15 +92,19 @@ public class RecallPrivacyService {
.stream() .stream()
.map(StudySessionsEntity::getSessionNum) .map(StudySessionsEntity::getSessionNum)
.toList(); .toList();
Set<String> sourceIds = new HashSet<>();
if (!nums.isEmpty()) { if (!nums.isEmpty()) {
for (StudyReportsEntity report : reports.selectList( for (StudyReportsEntity report : reports.selectList(
Wrappers.<StudyReportsEntity>lambdaQuery().in(StudyReportsEntity::getSessionNum, nums))) { Wrappers.<StudyReportsEntity>lambdaQuery().in(StudyReportsEntity::getSessionNum, nums))) {
sourceDeleted("REPORT", report.getId().toString()); sourceIds.add("report:" + report.getId());
} }
} }
for (TaskApplicationEntity app : applications.selectList( for (TaskApplicationEntity app : applications.selectList(
Wrappers.<TaskApplicationEntity>lambdaQuery().eq(TaskApplicationEntity::getTaskNum, taskNum))) { Wrappers.<TaskApplicationEntity>lambdaQuery().eq(TaskApplicationEntity::getTaskNum, taskNum))) {
sourceDeleted("APPLICATION", app.getId().toString()); sourceIds.add("application:" + app.getId());
}
if (!sourceIds.isEmpty()) {
redactSources(sourceIds);
} }
} }
@@ -117,15 +130,20 @@ public class RecallPrivacyService {
@Transactional(rollbackFor = Exception.class) @Transactional(rollbackFor = Exception.class)
public void tagsChanged() { public void tagsChanged() {
store.lock(); store.lock();
List<RecallDocument> docs = new ArrayList<>();
for (ReviewRecallEntity row : store.all()) { for (ReviewRecallEntity row : store.all()) {
RecallDocument doc = readable(row); RecallDocument doc = readable(row);
if (doc == null) { if (doc != null) {
continue; docs.add(doc);
} }
}
// 标签是否还有碎片:整批共用一次来源索引,而不是每篇回忆再查两遍
RecallReferenceLoader.SourceIndex index = loader.index(docs);
for (RecallDocument doc : docs) {
RecallDetail detail = doc.getDetail(); RecallDetail detail = doc.getDetail();
if ("TAG".equals(detail.getScope().type()) if ("TAG".equals(detail.getScope().type())
&& "VALID".equals(detail.getScopeStatus()) && "VALID".equals(detail.getScopeStatus())
&& !loader.valid(detail.getScope())) { && !loader.valid(detail.getScope(), index)) {
detail.setScopeStatus("INVALID"); detail.setScopeStatus("INVALID");
store.persist(doc); store.persist(doc);
} }
@@ -1,6 +1,7 @@
package com.guo.learningprogresstracker.service.recall; package com.guo.learningprogresstracker.service.recall;
import com.baomidou.mybatisplus.core.toolkit.Wrappers; import com.baomidou.mybatisplus.core.toolkit.Wrappers;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDocument;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSnapshot; import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSnapshot;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSource; import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSource;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceUnit; import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceUnit;
@@ -10,6 +11,7 @@ import com.guo.learningprogresstracker.entity.BaseEntity;
import com.guo.learningprogresstracker.entity.MarkerEntity; import com.guo.learningprogresstracker.entity.MarkerEntity;
import com.guo.learningprogresstracker.entity.StudyReportFragmentsEntity; import com.guo.learningprogresstracker.entity.StudyReportFragmentsEntity;
import com.guo.learningprogresstracker.entity.StudyReportsEntity; import com.guo.learningprogresstracker.entity.StudyReportsEntity;
import com.guo.learningprogresstracker.entity.TaskApplicationEntity;
import com.guo.learningprogresstracker.entity.TaskEntity; import com.guo.learningprogresstracker.entity.TaskEntity;
import com.guo.learningprogresstracker.enums.TagCombineModeEnum; import com.guo.learningprogresstracker.enums.TagCombineModeEnum;
import com.guo.learningprogresstracker.exception.RecallBusinessException; import com.guo.learningprogresstracker.exception.RecallBusinessException;
@@ -21,6 +23,7 @@ import com.guo.learningprogresstracker.mapper.TasksMapper;
import com.guo.learningprogresstracker.utils.TagScopeTool; import com.guo.learningprogresstracker.utils.TagScopeTool;
import java.time.Instant; import java.time.Instant;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap; import java.util.HashMap;
import java.util.HashSet; import java.util.HashSet;
import java.util.List; import java.util.List;
@@ -28,6 +31,7 @@ import java.util.Locale;
import java.util.Map; import java.util.Map;
import java.util.Objects; import java.util.Objects;
import java.util.Set; import java.util.Set;
import java.util.stream.Collectors;
import lombok.RequiredArgsConstructor; import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Component; import org.springframework.stereotype.Component;
@@ -93,18 +97,82 @@ public class RecallReferenceLoader {
return TagScopeTool.scopeKey(tags, TagCombineModeEnum.valueOf(combineMode)); return TagScopeTool.scopeKey(tags, TagCombineModeEnum.valueOf(combineMode));
} }
/**
* 一次请求内共享的来源快照:可达性与标签范围有效性都从这几张「id 全集」里判断。
* 逐条 selectById 会把列表页放大成每行 N+1 条查询,而来源本来就是按租户整表可见的。
*/
public record SourceIndex(
Set<String> fragments,
Set<String> reports,
Set<String> applications,
Set<String> taskNums,
List<MarkerEntity> markers) {}
/** 文档集合的索引:列表页一次建好,逐行复用。 */
public SourceIndex index(Collection<RecallDocument> docs) {
List<ReferenceSource> sources = new ArrayList<>();
List<ScopeSnapshot> scopes = new ArrayList<>();
for (RecallDocument doc : docs) {
sources.addAll(doc.getReference().sources());
scopes.add(doc.getDetail().getScope());
}
return index(sources, scopes);
}
public SourceIndex index(Collection<ReferenceSource> sources, Collection<ScopeSnapshot> scopes) {
boolean tagScope = scopes.stream().anyMatch(s -> "TAG".equals(s.type()));
boolean taskScope = scopes.stream().anyMatch(s -> "TASK".equals(s.type()));
boolean fragmentSource = sources.stream().anyMatch(s -> "FRAGMENT".equals(s.type()));
boolean reportSource = sources.stream().anyMatch(s -> "REPORT".equals(s.type()));
boolean applicationSource = sources.stream().anyMatch(s -> "APPLICATION".equals(s.type()));
Set<String> fragmentIds = tagScope || fragmentSource ? fragmentIds() : Set.of();
Set<String> reportIds = reportSource ? reportIds() : Set.of();
Set<String> applicationIds = applicationSource ? applicationIds() : Set.of();
Set<String> taskNums = taskScope ? taskNums() : Set.of();
List<MarkerEntity> markers = tagScope ? actualMarkers() : List.of();
return new SourceIndex(fragmentIds, reportIds, applicationIds, taskNums, markers);
}
private Set<String> fragmentIds() {
return fragments.selectList(Wrappers.<StudyReportFragmentsEntity>lambdaQuery()
.select(StudyReportFragmentsEntity::getId))
.stream()
.map(f -> f.getId().toString())
.collect(Collectors.toSet());
}
private Set<String> reportIds() {
return reports.selectList(Wrappers.<StudyReportsEntity>lambdaQuery().select(StudyReportsEntity::getId))
.stream()
.map(r -> r.getId().toString())
.collect(Collectors.toSet());
}
private Set<String> applicationIds() {
return applications.selectList(Wrappers.<TaskApplicationEntity>lambdaQuery()
.select(TaskApplicationEntity::getId))
.stream()
.map(a -> a.getId().toString())
.collect(Collectors.toSet());
}
private Set<String> taskNums() {
return tasks.selectList(Wrappers.<TaskEntity>lambdaQuery().select(TaskEntity::getTaskNum)).stream()
.map(TaskEntity::getTaskNum)
.collect(Collectors.toSet());
}
public boolean valid(ScopeSnapshot scope) { public boolean valid(ScopeSnapshot scope) {
return valid(scope, index(List.of(), List.of(scope)));
}
public boolean valid(ScopeSnapshot scope, SourceIndex index) {
if ("TASK".equals(scope.type())) { if ("TASK".equals(scope.type())) {
return tasks.exists(Wrappers.<TaskEntity>lambdaQuery().eq(TaskEntity::getTaskNum, scope.taskNum())); return index.taskNums().contains(scope.taskNum());
} }
Set<String> actual = new HashSet<>(); Set<String> actual = new HashSet<>();
Set<String> accessible = new HashSet<>(); for (MarkerEntity marker : index.markers()) {
for (StudyReportFragmentsEntity f : fragments.selectList( if (index.fragments().contains(marker.getObjectNum())) {
Wrappers.<StudyReportFragmentsEntity>lambdaQuery().select(StudyReportFragmentsEntity::getId))) {
accessible.add(f.getId().toString());
}
for (MarkerEntity marker : actualMarkers()) {
if (accessible.contains(marker.getObjectNum())) {
actual.add(TagScopeTool.normalize(marker.getMarkerName())); actual.add(TagScopeTool.normalize(marker.getMarkerName()));
} }
} }
@@ -223,19 +291,11 @@ public class RecallReferenceLoader {
return units; return units;
} }
public boolean accessible(ReferenceSource source) { public boolean accessible(ReferenceSource source, SourceIndex index) {
return accessible(source.type(), source.entityId()); return switch (source.type()) {
} case "FRAGMENT" -> index.fragments().contains(source.entityId());
case "REPORT" -> index.reports().contains(source.entityId());
public boolean accessible(String type, String entityId) { case "APPLICATION" -> index.applications().contains(source.entityId());
Integer id = parseId(entityId);
if (id == null) {
return false;
}
return switch (type) {
case "FRAGMENT" -> fragments.selectById(id) != null;
case "REPORT" -> reports.selectById(id) != null;
case "APPLICATION" -> applications.selectById(id) != null;
default -> false; default -> false;
}; };
} }
@@ -186,15 +186,16 @@ public class ReviewRecallService {
.eq(scopeType != null, ReviewRecallEntity::getScopeType, scopeType) .eq(scopeType != null, ReviewRecallEntity::getScopeType, scopeType)
.eq(taskNum != null, ReviewRecallEntity::getTaskNum, taskNum) .eq(taskNum != null, ReviewRecallEntity::getTaskNum, taskNum)
.eq(scopeKey != null, ReviewRecallEntity::getScopeKey, scopeKey); .eq(scopeKey != null, ReviewRecallEntity::getScopeKey, scopeKey);
long count = recalls.selectCount(query); // selectPage 自己会发 COUNT;再单独 selectCount 等于同一条件查两遍
List<RecallSummary> rows = new ArrayList<>();
long removed = 0;
Page<ReviewRecallEntity> page = recalls.selectPage( Page<ReviewRecallEntity> page = recalls.selectPage(
new Page<>(pageNum, pageSize), new Page<>(pageNum, pageSize),
query.orderByDesc(ReviewRecallEntity::getLastModifiedTime).orderByDesc(ReviewRecallEntity::getId)); query.orderByDesc(ReviewRecallEntity::getLastModifiedTime).orderByDesc(ReviewRecallEntity::getId));
// 分页行里已经带了 document_json:不再按 recall_num 回表,也不再逐行查来源与 AI 任务
List<RecallDocument> readable = new ArrayList<>();
long removed = 0;
for (ReviewRecallEntity row : page.getRecords()) { for (ReviewRecallEntity row : page.getRecords()) {
try { try {
rows.add(convert.toSummary(store.detail(store.load(row.getRecallNum())))); readable.add(store.parse(row));
} catch (RecallBusinessException e) { } catch (RecallBusinessException e) {
if (!"STORED_DATA_INVALID".equals(e.getData().reason())) { if (!"STORED_DATA_INVALID".equals(e.getData().reason())) {
throw e; throw e;
@@ -203,7 +204,13 @@ public class ReviewRecallService {
log.warn("[回忆] 列表已删除旧版本或不兼容文档: recallNum={}", row.getRecallNum()); log.warn("[回忆] 列表已删除旧版本或不兼容文档: recallNum={}", row.getRecallNum());
} }
} }
return new RecallPage(pageNum, pageSize, Math.max(0, count - removed), rows); RecallReferenceLoader.SourceIndex index = loader.index(readable);
List<RecallSummary> rows = new ArrayList<>();
for (RecallDocument doc : readable) {
store.reconcile(doc, index);
rows.add(convert.toSummary(store.summary(doc)));
}
return new RecallPage(pageNum, pageSize, Math.max(0, page.getTotal() - removed), rows);
} }
private static void validateListQuery(int pageNum, int pageSize, String status, String scopeType, String taskNum) { private static void validateListQuery(int pageNum, int pageSize, String status, String scopeType, String taskNum) {
@@ -3,13 +3,11 @@ package com.guo.learningprogresstracker.service.recall;
import cn.dev33.satoken.stp.StpUtil; import cn.dev33.satoken.stp.StpUtil;
import com.baomidou.mybatisplus.core.toolkit.Wrappers; import com.baomidou.mybatisplus.core.toolkit.Wrappers;
import com.guo.learningprogresstracker.common.TenantContext; import com.guo.learningprogresstracker.common.TenantContext;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallAiResult;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDetail; import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDetail;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDocument; import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallDocument;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallRequest; import com.guo.learningprogresstracker.dto.recall.RecallDtos.RecallRequest;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSnapshot; import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSnapshot;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSource; import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSource;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceSummary;
import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceUnit; import com.guo.learningprogresstracker.dto.recall.RecallDtos.ReferenceUnit;
import com.guo.learningprogresstracker.entity.AiTaskEntity; import com.guo.learningprogresstracker.entity.AiTaskEntity;
import com.guo.learningprogresstracker.entity.ReviewRecallEntity; import com.guo.learningprogresstracker.entity.ReviewRecallEntity;
@@ -24,7 +22,6 @@ import com.guo.learningprogresstracker.service.ai.RecallAiClient;
import com.guo.learningprogresstracker.service.ai.RecallJson; import com.guo.learningprogresstracker.service.ai.RecallJson;
import java.time.Instant; import java.time.Instant;
import java.time.LocalDateTime; import java.time.LocalDateTime;
import java.time.ZoneId;
import java.time.ZoneOffset; import java.time.ZoneOffset;
import java.util.HashSet; import java.util.HashSet;
import java.util.List; import java.util.List;
@@ -48,6 +45,7 @@ public class ReviewRecallStore {
private final AiTaskMapper aiTasks; private final AiTaskMapper aiTasks;
private final RecallJson json; private final RecallJson json;
private final RecallReferenceLoader references; private final RecallReferenceLoader references;
private final RecallDetailAssembler details;
private final RecallAiClient client; private final RecallAiClient client;
public static String user() { public static String user() {
@@ -74,14 +72,17 @@ public class ReviewRecallStore {
} }
public RecallDocument load(String num) { public RecallDocument load(String num) {
ReviewRecallEntity row = entity(num); RecallDocument d = parse(entity(num));
try { reconcile(d, references.index(d.getReference().sources(), List.of(d.getDetail().getScope())));
RecallDocument d = json.readStored(row.getDocumentJson(), RecallDocument.class);
reconcile(d);
return d; return d;
}
/** 只把文档形状读不出来当作损坏并删行;数据库或事务异常照常上抛,不能连带删掉用户数据。 */
public RecallDocument parse(ReviewRecallEntity row) {
try {
return json.readStored(row.getDocumentJson(), RecallDocument.class);
} catch (IllegalStateException | NullPointerException | ClassCastException e) { } catch (IllegalStateException | NullPointerException | ClassCastException e) {
// 只把「文档形状读不出来」当作损坏并删行;数据库或事务异常照常上抛,不能连带删掉用户数据。 log.warn("[回忆] 旧版本或不兼容文档已删除: recallNum={}", row.getRecallNum());
log.warn("[回忆] 旧版本或不兼容文档已删除: recallNum={}", num);
deleteUnreadable(row); deleteUnreadable(row);
throw new RecallBusinessException("STORED_DATA_INVALID", "这次回忆的数据来自旧版本,已删除;请重新开始。"); throw new RecallBusinessException("STORED_DATA_INVALID", "这次回忆的数据来自旧版本,已删除;请重新开始。");
} }
@@ -132,52 +133,12 @@ public class ReviewRecallStore {
} }
public RecallDetail detail(RecallDocument doc) { public RecallDetail detail(RecallDocument doc) {
RecallDetail v = doc.getDetail(); return details.detail(doc);
ReferenceSnapshot reference = doc.getReference();
v.setReference(new ReferenceSummary(
reference.status(),
RecallReferenceLoader.knowledgeSources(reference.sources()).size(),
reference.limitations()));
v.setRequests(v.getRequests().stream()
.map(r -> {
ReviewRecallRequestEntity association = request(r.taskNum());
AiTaskEntity task = task(r.taskNum());
String status = association == null || Integer.valueOf(1).equals(association.getRedacted())
? "FAILED"
: task == null ? "FAILED" : task.getStatus();
String code = association == null || Integer.valueOf(1).equals(association.getRedacted())
? "SOURCE_REDACTED"
: task == null ? "AI_UNAVAILABLE" : task.getErrorCode();
String message = task == null ? "这次 AI 结果已不可用" : task.getErrorMessage();
RecallAiResult result = null;
if (task != null && "SUCCEEDED".equals(status)) {
try {
result = json.readStored(task.getResultPayload(), RecallAiResult.class);
} catch (RuntimeException ex) {
status = "FAILED";
code = "AI_UNAVAILABLE";
message = "这次 AI 结果是旧版本格式,暂时无法展示;请重新请求。";
} }
}
return new RecallRequest( /** 列表只取摘要:不查 AI 任务现状,requests 会由 toSummary 丢弃。 */
r.requestId(), public RecallDetail summary(RecallDocument doc) {
r.taskNum(), return details.summary(doc);
r.type(),
r.hintForm(),
r.promptVersion(),
status,
code,
message,
r.submittedAt(),
task == null || task.getFinishedTime() == null
? null
: task.getFinishedTime()
.atZone(ZoneId.systemDefault())
.toInstant(),
result);
})
.toList());
return v;
} }
public AiTaskEntity task(String num) { public AiTaskEntity task(String num) {
@@ -243,16 +204,16 @@ public class ReviewRecallStore {
} }
/** Redact the single current source set and every result that depended on it. */ /** Redact the single current source set and every result that depended on it. */
public void reconcile(RecallDocument doc) { public void reconcile(RecallDocument doc, RecallReferenceLoader.SourceIndex index) {
RecallDetail d = doc.getDetail(); RecallDetail d = doc.getDetail();
boolean changed = false; boolean changed = false;
if ("VALID".equals(d.getScopeStatus()) && !references.valid(d.getScope())) { if ("VALID".equals(d.getScopeStatus()) && !references.valid(d.getScope(), index)) {
d.setScopeStatus("INVALID"); d.setScopeStatus("INVALID");
changed = true; changed = true;
} }
Set<String> missing = new HashSet<>(); Set<String> missing = new HashSet<>();
for (ReferenceSource source : doc.getReference().sources()) { for (ReferenceSource source : doc.getReference().sources()) {
if ("AVAILABLE".equals(source.status()) && !references.accessible(source)) { if ("AVAILABLE".equals(source.status()) && !references.accessible(source, index)) {
missing.add(source.sourceId()); missing.add(source.sourceId());
} }
} }
@@ -17,8 +17,11 @@ import com.guo.learningprogresstracker.mapper.TasksMapper;
import java.time.DayOfWeek; import java.time.DayOfWeek;
import java.time.LocalDate; import java.time.LocalDate;
import java.time.LocalDateTime; import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections; import java.util.Collections;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.Set; import java.util.Set;
@@ -78,6 +81,24 @@ public class ReviewStatsService {
return result; return result;
} }
/** 一批任务一次取齐:逐任务调用会把 4 条聚合乘以任务数。 */
public List<ReviewTaskStats> getReviewTaskStats(Collection<String> taskNums) {
if (taskNums == null || taskNums.isEmpty()) {
return List.of();
}
Map<String, String> names = new LinkedHashMap<>();
for (TaskEntity task : tasksMapper.selectList(
Wrappers.<TaskEntity>lambdaQuery().in(TaskEntity::getTaskNum, taskNums))) {
names.put(task.getTaskNum(), task.getTaskName());
}
List<ReviewTaskStats> stats = new ArrayList<>();
for (String taskNum : new LinkedHashSet<>(taskNums)) {
stats.add(newTaskStats(taskNum, names.get(taskNum)));
}
fillReviewTaskStats(stats);
return stats;
}
public ReviewTaskStats getReviewTaskStats(String taskNum) { public ReviewTaskStats getReviewTaskStats(String taskNum) {
TaskEntity task = tasksMapper TaskEntity task = tasksMapper
.selectList(Wrappers.<TaskEntity>lambdaQuery() .selectList(Wrappers.<TaskEntity>lambdaQuery()
@@ -151,7 +172,10 @@ public class ReviewStatsService {
/** Current free recall duration is aggregated from the single-current-draft table. */ /** Current free recall duration is aggregated from the single-current-draft table. */
private void fillReviewDuration(Map<String, ReviewTaskStats> statsByTaskNum) { private void fillReviewDuration(Map<String, ReviewTaskStats> statsByTaskNum) {
for (TaskRecallDuration row : newRecallMapper.sumTaskDuration()) { if (statsByTaskNum.isEmpty()) {
return;
}
for (TaskRecallDuration row : newRecallMapper.sumTaskDuration(statsByTaskNum.keySet())) {
ReviewTaskStats stats = statsByTaskNum.get(row.getTaskNum()); ReviewTaskStats stats = statsByTaskNum.get(row.getTaskNum());
if (stats != null && row.getActiveDurationMs() != null) { if (stats != null && row.getActiveDurationMs() != null) {
stats.setReviewDurationSeconds(stats.getReviewDurationSeconds() + row.getActiveDurationMs() / 1000); stats.setReviewDurationSeconds(stats.getReviewDurationSeconds() + row.getActiveDurationMs() / 1000);
@@ -207,7 +207,7 @@ class ReviewServiceImplTest {
when(studyReportsMapper.countReportsByTask(any())).thenReturn(List.of(count("T1", 2L))); when(studyReportsMapper.countReportsByTask(any())).thenReturn(List.of(count("T1", 2L)));
when(studyReportFragmentsMapper.countFragmentsByTask(any())).thenReturn(List.of(count("T1", 3L))); when(studyReportFragmentsMapper.countFragmentsByTask(any())).thenReturn(List.of(count("T1", 3L)));
// 聚合结果里出现了不在任务列表中的 taskNum,应被忽略 // 聚合结果里出现了不在任务列表中的 taskNum,应被忽略
when(newRecallMapper.sumTaskDuration()) when(newRecallMapper.sumTaskDuration(any()))
.thenReturn(List.of(reviewDuration("T1", 420L), reviewDuration("T_UNKNOWN", 100L))); .thenReturn(List.of(reviewDuration("T1", 420L), reviewDuration("T_UNKNOWN", 100L)));
List<ReviewTaskStats> stats = List<ReviewTaskStats> stats =