feat(flywheel/rules-validation): Rules-Applied 规则 ID ingest 层真实性校验(U3)
All checks were successful
Push & PR Check / check (push) Successful in 1m17s
All checks were successful
Push & PR Check / check (push) Successful in 1m17s
校验单点放统计消费端(R12 边界单点):extract_rule_ids 从项目 CONVENTIONS.md 提取合法 ID 集合,filter_valid_rules 在入库前剔除 幻觉/typo ID(无 CONVENTIONS.md 时 fail-open 不校验)。接入全部 4 条写入链路(buff 增量/webhook/gitea 轮询/全量导入); sanitize_rules_applied 提供回溯清洗,重跑全量导入即可清历史污染。 Rules-Applied: R01,R06,R07
This commit is contained in:
parent
f04eb6e766
commit
aaf3343133
@ -372,8 +372,9 @@ modules:
|
|||||||
area: backend
|
area: backend
|
||||||
status: in_progress
|
status: in_progress
|
||||||
progress: 99
|
progress: 99
|
||||||
note: "体系分发主线:T1/T2/B1/B2/B3/V1 ✅(T2 2026-07-04:git-workflow 复杂度感知分支策略;B3 2026-07-04:manifest agent_button_carrier + splitter 三级载体继承);剩 B4 健康诊断遵守度(后置)"
|
note: "体系分发主线:T1/T2/B1/B2/B3/V1/U3 ✅(U3 2026-07-15:Rules-Applied ingest 层校验+回溯清洗);剩 📋 U1(全托管升级通道+批量重分发)/ B4(遵守度+回传覆盖率,吸收被否决的 U2)"
|
||||||
position: [3500, 0]
|
position: [3500, 0]
|
||||||
|
updated: 2026-07-15
|
||||||
|
|
||||||
- id: gitea-integration
|
- id: gitea-integration
|
||||||
name: Gitea 集成
|
name: Gitea 集成
|
||||||
|
|||||||
@ -30,6 +30,8 @@
|
|||||||
- 「完整分发」是陷阱,正确形态是分级菜单:新项目种全套,旧项目增量接入(复用 mcp_inject 的「命名空间隔离 + 合并不覆盖」语义,不动用户已有配置)
|
- 「完整分发」是陷阱,正确形态是分级菜单:新项目种全套,旧项目增量接入(复用 mcp_inject 的「命名空间隔离 + 合并不覆盖」语义,不动用户已有配置)
|
||||||
- 接入动作:现有 `apply_blueprint_buff` 升格为 `apply_onboarding_pack`,按档编排注入
|
- 接入动作:现有 `apply_blueprint_buff` 升格为 `apply_onboarding_pack`,按档编排注入
|
||||||
- 语义层进料口隐患:决策/验收/踩坑仍需主动写(与饿死复盘笔记同源),须配「从 PR 标题/commit body/review 评论半自动提取」,不再单纯指望人自觉
|
- 语义层进料口隐患:决策/验收/踩坑仍需主动写(与饿死复盘笔记同源),须配「从 PR 标题/commit body/review 评论半自动提取」,不再单纯指望人自觉
|
||||||
|
- **分发文件三档升级语义(2026-07-15 择升环评估)**:① 版本化块文件(AGENTS/git-workflow)→ MANAGED 块原位升级;② 炼境全托管文件(envoy.md/blueprint-gate.cjs,用户不定制)→ 内容不一致即覆盖;③ 用户可定制文件(lefthook.yml)→ 绝不覆盖,但 verify_onboarding 须能检测落后。此前 ②③ 混用「存在即跳过」导致回流修正推不到存量项目
|
||||||
|
- **回流触发不自动化(2026-07-15 多角度辩证)**:否决子项目侧 hook 自动触发 /envoy——自动化触发拆掉人肉过滤层,诱导低价值回传污染通道,伤「择」换「环」转速。回流频率问题用炼境侧可观测性解(「沉默项目」诊断信号,见 B4 卡),人留在过滤环里。校验类需求优先放统计消费端(炼境 ingest)而非分发 hook(R12 边界单点 + 零分发成本 + 可回溯)
|
||||||
|
|
||||||
## 复用的现有能力(~60% 地基已具备)
|
## 复用的现有能力(~60% 地基已具备)
|
||||||
|
|
||||||
@ -132,6 +134,29 @@
|
|||||||
- 版本化闭环要求(2026-07-03 用户澄清,体系分发的灵魂): 分发不是"发了就不管"——每类资产带版本分发 → 子项目按该版本实践产生经验(成功/失败)→ 经验自吸反哺炼境(trailer/failure_cases/复盘/返工率)→ 按版本分组对比效果 → 修订出新版 → 再分发。CONVENTIONS 是模范生(version+队列对比+梦核+同步,1.8→1.9 已跑通一整圈);B 系列每类资产按此标准建闭环。**前置依赖提升**:「接入包模板演进分发(MANAGED 块化)」concept 卡——模板无版本机制是当前最大缺口,没有它体系 v2 到不了存量项目
|
- 版本化闭环要求(2026-07-03 用户澄清,体系分发的灵魂): 分发不是"发了就不管"——每类资产带版本分发 → 子项目按该版本实践产生经验(成功/失败)→ 经验自吸反哺炼境(trailer/failure_cases/复盘/返工率)→ 按版本分组对比效果 → 修订出新版 → 再分发。CONVENTIONS 是模范生(version+队列对比+梦核+同步,1.8→1.9 已跑通一整圈);B 系列每类资产按此标准建闭环。**前置依赖提升**:「接入包模板演进分发(MANAGED 块化)」concept 卡——模板无版本机制是当前最大缺口,没有它体系 v2 到不了存量项目
|
||||||
- notes: 2026-07-03 概念探讨收束后的分发需求(体系详见 memory concept_dual_projection 与 mcp-server 模块 concept 卡)。用户全 agent 开发形态下,"给开发者立规范"="给 agent 注入规范",可完全机器化。四条现成管道分发四类资产:① B1(S) CONVENTIONS 1.10.0 加 R12 边界单点实现 + R13 验收须真实走通验证路径 + 载体选择原则;② B2(S) 接入包 AGENTS.md 模板加 agent 按钮章节(三要素/载体谱系/侦察驻军);③ B3(M) init/sync 提示词升级——建蓝图时声明项目按钮载体、acceptance 生成为可执行验证路径格式;④ B4(S,后置) 健康诊断加体系遵守度检查。移植体验=零新增动作,接入流程不变。
|
- notes: 2026-07-03 概念探讨收束后的分发需求(体系详见 memory concept_dual_projection 与 mcp-server 模块 concept 卡)。用户全 agent 开发形态下,"给开发者立规范"="给 agent 注入规范",可完全机器化。四条现成管道分发四类资产:① B1(S) CONVENTIONS 1.10.0 加 R12 边界单点实现 + R13 验收须真实走通验证路径 + 载体选择原则;② B2(S) 接入包 AGENTS.md 模板加 agent 按钮章节(三要素/载体谱系/侦察驻军);③ B3(M) init/sync 提示词升级——建蓝图时声明项目按钮载体、acceptance 生成为可执行验证路径格式;④ B4(S,后置) 健康诊断加体系遵守度检查。移植体验=零新增动作,接入流程不变。
|
||||||
|
|
||||||
|
### 📋 U1. 全托管文件升级通道 + 批量重分发(择升环断点修复 ①)
|
||||||
|
- status: planned
|
||||||
|
- complexity: M
|
||||||
|
- depends: onboarding-pack(卡 T1)
|
||||||
|
- files: src-tauri/src/commands/onboarding.rs, src-tauri/src/mcp_tools.rs, src-tauri/resources/onboarding/.claude/commands/envoy.md, src-tauri/resources/onboarding/.claude/hooks/blueprint-gate.cjs
|
||||||
|
- acceptance: [Tauri command] 对存量项目重跑 apply_onboarding_pack → envoy.md / blueprint-gate.cjs 与内嵌模板不一致时覆盖(action=updated),一致时 skipped,文件头含「由炼境托管,重跑接入包会覆盖」声明;[cargo test] dispatch_fully_managed 三态(不存在→written / 一致→skipped / 不一致→updated);[MCP tool] verify_onboarding 的 lefthook 检查项从内嵌 TMPL_LEFTHOOK 自省提取 commands 键集合,项目 lefthook.yml 缺键 → fail + 列出缺失键(禁止硬编码具体检查项名,模板演进校验器零维护);[MCP tool] apply_onboarding_pack 不传 project_id → 对所有已登记项目批量执行并返回逐项目摘要(复用 ingest_git_history 先例),验收通过后删除 batch_apply.py / scan_onboarded.py
|
||||||
|
- notes: dispatch_one「存在即跳过」使回流修正只能推到 AGENTS/git-workflow 两份版本化文档——envoy/gate 的 bug 修复永远到不了存量 12 项目,批量重分发靠临时 python 脚本硬扛(batch_apply.py 为实锤)。三档升级语义见决策记录。已知盲点(辩证承认):首次覆盖时子项目文件尚无托管声明,存在一次性盲覆盖窗口——envoy/gate 分发仅一周、被定制概率极低,且 updated 会在报告显式出现,风险接受
|
||||||
|
|
||||||
|
### ✅ U3. Rules-Applied 真实性校验(ingest 层单点实现,择升环断点修复 ②)
|
||||||
|
- status: done
|
||||||
|
- done_note: 2026-07-15 完成。纯函数 extract_rule_ids(R+数字 token 边界识别,防 PR13/R2D2 误提取)+ filter_valid_rules(valid=None 不校验 fail-open);校验接入全部 4 条写入链路(buff 增量 / webhook push / gitea 轮询 / 全量导入),规则集每项目每批次加载一次;sanitize_rules_applied 显式回溯清洗通道(UPSERT「不覆盖」约束原样保留,两者并行不悖);IngestResult 新增 sanitized 字段(Rust/TS/MCP 输出/FlywheelPanel 四层同步);record_commit 收窄为 #[cfg(test)],生产链路只剩 validated 版防绕过。cargo test 98 全过(新增 5 测:token 边界/过滤/入库剔除/message 链路/清洗幂等+仅清洗指定项目),pnpm typecheck 干净
|
||||||
|
- complexity: S
|
||||||
|
- files: src-tauri/src/commands/commit_metrics.rs
|
||||||
|
- acceptance: [cargo test] trailer 解析校验纯函数——规则 ID 不存在于该项目 .blueprint/CONVENTIONS.md(如 R99)→ 从 rules_applied 剔除并可观测(日志或 invalid 计数),真实 ID 保留;项目无 CONVENTIONS.md → 不校验全保留;[MCP tool] ingest_git_history 重跑 → 「规则命中 × 返工」交叉统计不再含未知 ID(存量污染可回溯清洗)
|
||||||
|
- notes: 辩证翻转载体:原方案(子项目 commit-msg hook)违反 R12 边界单点——统计消费端在炼境,校验就该在炼境 ingest 层。收益:零分发成本覆盖全部项目、重跑即回溯清洗、cargo test 可测(炼境自身 agent 按钮载体)。commit-msg hook 即时反馈版降为可选增强(若做依赖 U1 通道,不在本卡)。上限诚实:只挡 typo/幻觉 ID,「真 ID 假触发」留给宁缺毋滥纪律
|
||||||
|
|
||||||
|
### 📋 B4. 健康诊断遵守度 + 回传覆盖率(吸收被否决的 U2,择升环断点修复 ③)
|
||||||
|
- status: planned
|
||||||
|
- complexity: S
|
||||||
|
- files: src-tauri/src/mcp_tools.rs, src-tauri/src/commands/onboarding.rs
|
||||||
|
- acceptance: [MCP tool] get_diagnostics(或 verify_onboarding 检查链)新增「沉默项目」信号:项目近 30 天 commit_metrics 有数据但使者回传(list_envoy_returns 同源数据)为 0 → 列出项目名 + commit 数,提示去该项目跑 /envoy;近 30 天有回传或无 commit 活动 → 不报
|
||||||
|
- notes: U2(子项目侧 Stop hook 自动触发 /envoy)经多角度辩证否决(2026-07-15):自动化触发会拆掉人肉过滤层,诱导 agent 用 under_validation 噪声交差灌满回传通道——伤「择」换「环」转速,本末倒置。正解是炼境侧可观测性:让遗忘可被看见(信号驱动),人留在过滤环里决定是否回传。同时销掉原 B4 后置欠账(体系遵守度检查)
|
||||||
|
|
||||||
## 复盘笔记
|
## 复盘笔记
|
||||||
|
|
||||||
【复盘】V1 接入生效验收(2026-07-04)
|
【复盘】V1 接入生效验收(2026-07-04)
|
||||||
|
|||||||
@ -159,9 +159,16 @@ fn check_project(app: &AppHandle, project_path: &str) -> Result<(), String> {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
// 批量写入用事务包裹(首次可达数十条,避免逐条 fsync)
|
// 批量写入用事务包裹(首次可达数十条,避免逐条 fsync)
|
||||||
|
// U3:规则 ID 集合每批次加载一次,剔除 trailer 里的未知 ID
|
||||||
|
let valid_rules = crate::commands::commit_metrics::load_project_rule_ids(root);
|
||||||
let tx = conn.unchecked_transaction();
|
let tx = conn.unchecked_transaction();
|
||||||
for entry in &commits {
|
for entry in &commits {
|
||||||
let _ = crate::commands::commit_metrics::record_commit_entry(&conn, &project_id, entry);
|
let _ = crate::commands::commit_metrics::record_commit_entry(
|
||||||
|
&conn,
|
||||||
|
&project_id,
|
||||||
|
entry,
|
||||||
|
valid_rules.as_ref(),
|
||||||
|
);
|
||||||
}
|
}
|
||||||
if let Ok(tx) = tx {
|
if let Ok(tx) = tx {
|
||||||
let _ = tx.commit();
|
let _ = tx.commit();
|
||||||
|
|||||||
@ -5,6 +5,7 @@
|
|||||||
|
|
||||||
use rusqlite::Connection;
|
use rusqlite::Connection;
|
||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
|
use std::collections::HashSet;
|
||||||
|
|
||||||
/// 一条 commit 的解析结果。
|
/// 一条 commit 的解析结果。
|
||||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||||
@ -107,6 +108,104 @@ pub fn parse_rules_applied(message: &str) -> Option<String> {
|
|||||||
None
|
None
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── U3:Rules-Applied 真实性校验(ingest 层单点,防幻觉/typo ID 污染交叉统计)──
|
||||||
|
|
||||||
|
/// 从 CONVENTIONS.md 内容提取全部规则 ID(大写 R+数字,前后不粘字母数字)。纯函数。
|
||||||
|
pub fn extract_rule_ids(text: &str) -> HashSet<String> {
|
||||||
|
let b = text.as_bytes();
|
||||||
|
let mut ids = HashSet::new();
|
||||||
|
let mut i = 0;
|
||||||
|
while i < b.len() {
|
||||||
|
if b[i] == b'R' && (i == 0 || !b[i - 1].is_ascii_alphanumeric()) {
|
||||||
|
let mut j = i + 1;
|
||||||
|
while j < b.len() && b[j].is_ascii_digit() {
|
||||||
|
j += 1;
|
||||||
|
}
|
||||||
|
if j > i + 1 && (j == b.len() || !b[j].is_ascii_alphanumeric()) {
|
||||||
|
ids.insert(text[i..j].to_string());
|
||||||
|
}
|
||||||
|
i = j;
|
||||||
|
} else {
|
||||||
|
i += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ids
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 过滤 rules_applied 串:剔除不在 valid 集合中的 ID;全部无效返回 None。
|
||||||
|
/// valid 为 None(项目无 CONVENTIONS.md)时不校验、原样保留。纯函数。
|
||||||
|
pub fn filter_valid_rules(rules: &str, valid: Option<&HashSet<String>>) -> Option<String> {
|
||||||
|
let Some(set) = valid else {
|
||||||
|
return Some(rules.to_string());
|
||||||
|
};
|
||||||
|
let kept: Vec<&str> = rules
|
||||||
|
.split(',')
|
||||||
|
.map(str::trim)
|
||||||
|
.filter(|t| !t.is_empty() && set.contains(*t))
|
||||||
|
.collect();
|
||||||
|
if kept.is_empty() {
|
||||||
|
None
|
||||||
|
} else {
|
||||||
|
Some(kept.join(","))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 读取项目 .blueprint/CONVENTIONS.md 提取规则 ID 集合;文件不存在返回 None(不校验)。
|
||||||
|
/// 每项目每批次调用一次,勿放进逐 commit 循环。
|
||||||
|
pub fn load_project_rule_ids(root: &std::path::Path) -> Option<HashSet<String>> {
|
||||||
|
let content = std::fs::read_to_string(root.join(".blueprint").join("CONVENTIONS.md")).ok()?;
|
||||||
|
Some(extract_rule_ids(&content))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 远程采集链路(webhook/轮询)无本地 root,按 project_id 查 blueprint_buffs 路径再加载。
|
||||||
|
pub fn load_rule_ids_by_project_id(conn: &Connection, project_id: &str) -> Option<HashSet<String>> {
|
||||||
|
let path: String = conn
|
||||||
|
.query_row(
|
||||||
|
"SELECT project_path FROM blueprint_buffs WHERE project_id = ?1",
|
||||||
|
rusqlite::params![project_id],
|
||||||
|
|r| r.get(0),
|
||||||
|
)
|
||||||
|
.ok()?;
|
||||||
|
load_project_rule_ids(std::path::Path::new(&path))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 回溯清洗存量行:剔除 rules_applied 中不在 valid 集合的 ID(全无效置 NULL)。
|
||||||
|
/// 与 UPSERT 的「不覆盖已有值」约束并行不悖——本函数是显式清洗通道,仅在全量导入时调用。
|
||||||
|
/// 返回被清洗的行数,重跑幂等(已干净的行不再计数)。
|
||||||
|
pub fn sanitize_rules_applied(
|
||||||
|
conn: &Connection,
|
||||||
|
project_id: &str,
|
||||||
|
valid: &HashSet<String>,
|
||||||
|
) -> Result<u32, String> {
|
||||||
|
let rows: Vec<(String, String)> = {
|
||||||
|
let mut stmt = conn
|
||||||
|
.prepare(
|
||||||
|
"SELECT sha, rules_applied FROM commit_metrics
|
||||||
|
WHERE project_id = ?1 AND rules_applied IS NOT NULL",
|
||||||
|
)
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
let mapped = stmt
|
||||||
|
.query_map(rusqlite::params![project_id], |r| {
|
||||||
|
Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?))
|
||||||
|
})
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
mapped.filter_map(|r| r.ok()).collect()
|
||||||
|
};
|
||||||
|
let mut cleaned = 0u32;
|
||||||
|
for (sha, rules) in rows {
|
||||||
|
let filtered = filter_valid_rules(&rules, Some(valid));
|
||||||
|
if filtered.as_deref() != Some(rules.as_str()) {
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE commit_metrics SET rules_applied = ?1 WHERE sha = ?2",
|
||||||
|
rusqlite::params![filtered, sha],
|
||||||
|
)
|
||||||
|
.map_err(|e| e.to_string())?;
|
||||||
|
cleaned += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(cleaned)
|
||||||
|
}
|
||||||
|
|
||||||
/// UPSERT 写入一条 commit 记录:新 sha 插入;已存在且 rules_applied 为空时仅回填
|
/// UPSERT 写入一条 commit 记录:新 sha 插入;已存在且 rules_applied 为空时仅回填
|
||||||
/// rules_applied(绝不覆盖已有值——回填幂等的关键约束)。返回是否产生了写入。
|
/// rules_applied(绝不覆盖已有值——回填幂等的关键约束)。返回是否产生了写入。
|
||||||
fn upsert_metrics(
|
fn upsert_metrics(
|
||||||
@ -140,9 +239,27 @@ fn upsert_metrics(
|
|||||||
Ok(n > 0)
|
Ok(n > 0)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// 解析并写入一条 commit 记录(从完整 message 提取 conventional + Rules-Applied trailer)。
|
/// 解析并写入一条 commit 记录(从完整 message 提取 conventional + Rules-Applied trailer),
|
||||||
/// 生产链路的 git log 只给 subject,trailer 走 `record_commit_entry`;本函数保留给
|
/// 并按 valid_rules 剔除未知规则 ID(U3)。远程采集生产链路(webhook/轮询)用本函数,
|
||||||
/// 持有完整 message 的调用方与测试。
|
/// valid_rules 每项目加载一次(`load_rule_ids_by_project_id`)。
|
||||||
|
pub fn record_commit_validated(
|
||||||
|
conn: &Connection,
|
||||||
|
project_id: &str,
|
||||||
|
sha: &str,
|
||||||
|
message: &str,
|
||||||
|
committed_at: Option<&str>,
|
||||||
|
valid_rules: Option<&HashSet<String>>,
|
||||||
|
) -> Result<(), String> {
|
||||||
|
let p = parse_conventional(message);
|
||||||
|
let rules_applied =
|
||||||
|
parse_rules_applied(message).and_then(|r| filter_valid_rules(&r, valid_rules));
|
||||||
|
upsert_metrics(conn, project_id, sha, &p, rules_applied.as_deref(), committed_at)?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 不校验规则 ID 的写入(等价 valid_rules=None)。仅供测试铺数据;
|
||||||
|
/// 生产链路一律走 `record_commit_validated` / `record_commit_entry`(U3 校验强制)。
|
||||||
|
#[cfg(test)]
|
||||||
pub fn record_commit(
|
pub fn record_commit(
|
||||||
conn: &Connection,
|
conn: &Connection,
|
||||||
project_id: &str,
|
project_id: &str,
|
||||||
@ -150,21 +267,21 @@ pub fn record_commit(
|
|||||||
message: &str,
|
message: &str,
|
||||||
committed_at: Option<&str>,
|
committed_at: Option<&str>,
|
||||||
) -> Result<(), String> {
|
) -> Result<(), String> {
|
||||||
let p = parse_conventional(message);
|
record_commit_validated(conn, project_id, sha, message, committed_at, None)
|
||||||
let rules_applied = parse_rules_applied(message);
|
|
||||||
upsert_metrics(conn, project_id, sha, &p, rules_applied.as_deref(), committed_at)?;
|
|
||||||
Ok(())
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/// 写入一条来自 `read_git_log` 的记录(subject 与 trailer 分列,trailer 由 git 原生解析)。
|
/// 写入一条来自 `read_git_log` 的记录(subject 与 trailer 分列,trailer 由 git 原生解析)。
|
||||||
/// 同时把触碰文件写入 commit_files(INSERT OR IGNORE 幂等,供文件级风险画像)。
|
/// 同时把触碰文件写入 commit_files(INSERT OR IGNORE 幂等,供文件级风险画像)。
|
||||||
|
/// valid_rules 每项目加载一次传入(U3 校验);None 表示项目无 CONVENTIONS.md,不校验。
|
||||||
pub fn record_commit_entry(
|
pub fn record_commit_entry(
|
||||||
conn: &Connection,
|
conn: &Connection,
|
||||||
project_id: &str,
|
project_id: &str,
|
||||||
entry: &GitLogEntry,
|
entry: &GitLogEntry,
|
||||||
|
valid_rules: Option<&HashSet<String>>,
|
||||||
) -> Result<bool, String> {
|
) -> Result<bool, String> {
|
||||||
let p = parse_conventional(&entry.subject);
|
let p = parse_conventional(&entry.subject);
|
||||||
let rules = normalize_rule_tokens(&entry.rules_trailer);
|
let rules = normalize_rule_tokens(&entry.rules_trailer)
|
||||||
|
.and_then(|r| filter_valid_rules(&r, valid_rules));
|
||||||
let date = if entry.committed_at.is_empty() {
|
let date = if entry.committed_at.is_empty() {
|
||||||
None
|
None
|
||||||
} else {
|
} else {
|
||||||
@ -300,6 +417,8 @@ pub struct IngestResult {
|
|||||||
pub imported: u32,
|
pub imported: u32,
|
||||||
/// 已存在且无需回填(跳过)
|
/// 已存在且无需回填(跳过)
|
||||||
pub skipped: u32,
|
pub skipped: u32,
|
||||||
|
/// 存量行中被清洗掉未知规则 ID 的行数(U3 回溯清洗;项目无 CONVENTIONS.md 时恒为 0)
|
||||||
|
pub sanitized: u32,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// git log 单条记录。`rules_trailer` 是 git 原生解析出的 Rules-Applied trailer 值
|
/// git log 单条记录。`rules_trailer` 是 git 原生解析出的 Rules-Applied trailer 值
|
||||||
@ -406,16 +525,23 @@ pub fn ingest_full_git_history(
|
|||||||
)
|
)
|
||||||
.unwrap_or(project_id);
|
.unwrap_or(project_id);
|
||||||
|
|
||||||
|
// U3:每项目加载一次规则 ID 集合,新写入过滤 + 存量回溯清洗
|
||||||
|
let valid_rules = load_project_rule_ids(root);
|
||||||
|
|
||||||
let tx = conn.unchecked_transaction().map_err(|e| e.to_string())?;
|
let tx = conn.unchecked_transaction().map_err(|e| e.to_string())?;
|
||||||
let mut imported = 0u32;
|
let mut imported = 0u32;
|
||||||
for entry in &commits {
|
for entry in &commits {
|
||||||
// 新 sha 插入;已存在的行 UPSERT 回填 rules_applied(trailer 死链修复后的存量补数路径)
|
// 新 sha 插入;已存在的行 UPSERT 回填 rules_applied(trailer 死链修复后的存量补数路径)
|
||||||
if record_commit_entry(&conn, &effective_id, entry)? {
|
if record_commit_entry(&conn, &effective_id, entry, valid_rules.as_ref())? {
|
||||||
imported += 1;
|
imported += 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
let sanitized = match &valid_rules {
|
||||||
|
Some(set) => sanitize_rules_applied(&conn, &effective_id, set)?,
|
||||||
|
None => 0,
|
||||||
|
};
|
||||||
tx.commit().map_err(|e| e.to_string())?;
|
tx.commit().map_err(|e| e.to_string())?;
|
||||||
Ok(IngestResult { imported, skipped: total - imported })
|
Ok(IngestResult { imported, skipped: total - imported, sanitized })
|
||||||
}
|
}
|
||||||
|
|
||||||
// ── 跨项目 Git 健康度聚合 ─────────────────────────────────────────────────────
|
// ── 跨项目 Git 健康度聚合 ─────────────────────────────────────────────────────
|
||||||
@ -784,22 +910,22 @@ mod tests {
|
|||||||
assert_eq!(rules_of(&conn, "old"), None);
|
assert_eq!(rules_of(&conn, "old"), None);
|
||||||
|
|
||||||
// 回填:同 sha 带 trailer 重写 → 补上
|
// 回填:同 sha 带 trailer 重写 → 补上
|
||||||
let changed = record_commit_entry(&conn, "proj", &entry("old", "feat(a): x", "R01,R06")).unwrap();
|
let changed = record_commit_entry(&conn, "proj", &entry("old", "feat(a): x", "R01,R06"), None).unwrap();
|
||||||
assert!(changed, "回填应计为写入");
|
assert!(changed, "回填应计为写入");
|
||||||
assert_eq!(rules_of(&conn, "old").as_deref(), Some("R01,R06"));
|
assert_eq!(rules_of(&conn, "old").as_deref(), Some("R01,R06"));
|
||||||
|
|
||||||
// 不覆盖:已有值的行,不同 trailer 重写 → 保持原值
|
// 不覆盖:已有值的行,不同 trailer 重写 → 保持原值
|
||||||
let changed = record_commit_entry(&conn, "proj", &entry("old", "feat(a): x", "R99")).unwrap();
|
let changed = record_commit_entry(&conn, "proj", &entry("old", "feat(a): x", "R99"), None).unwrap();
|
||||||
assert!(!changed, "已有值不应被覆盖");
|
assert!(!changed, "已有值不应被覆盖");
|
||||||
assert_eq!(rules_of(&conn, "old").as_deref(), Some("R01,R06"));
|
assert_eq!(rules_of(&conn, "old").as_deref(), Some("R01,R06"));
|
||||||
|
|
||||||
// 无 trailer 重写已有 NULL 行 → 不产生无意义 UPDATE
|
// 无 trailer 重写已有 NULL 行 → 不产生无意义 UPDATE
|
||||||
record_commit(&conn, "proj", "old2", "chore: y", None).unwrap();
|
record_commit(&conn, "proj", "old2", "chore: y", None).unwrap();
|
||||||
let changed = record_commit_entry(&conn, "proj", &entry("old2", "chore: y", "")).unwrap();
|
let changed = record_commit_entry(&conn, "proj", &entry("old2", "chore: y", ""), None).unwrap();
|
||||||
assert!(!changed);
|
assert!(!changed);
|
||||||
|
|
||||||
// 新行带 trailer 直接写入
|
// 新行带 trailer 直接写入
|
||||||
assert!(record_commit_entry(&conn, "proj", &entry("new", "fix(b): z", "R05")).unwrap());
|
assert!(record_commit_entry(&conn, "proj", &entry("new", "fix(b): z", "R05"), None).unwrap());
|
||||||
assert_eq!(rules_of(&conn, "new").as_deref(), Some("R05"));
|
assert_eq!(rules_of(&conn, "new").as_deref(), Some("R05"));
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -844,8 +970,8 @@ mod tests {
|
|||||||
let conn = crate::db::conn_with_schema();
|
let conn = crate::db::conn_with_schema();
|
||||||
let mut e = entry("s1", "feat(a): x", "");
|
let mut e = entry("s1", "feat(a): x", "");
|
||||||
e.files = vec!["src/a.rs".into(), "src/b.rs".into()];
|
e.files = vec!["src/a.rs".into(), "src/b.rs".into()];
|
||||||
record_commit_entry(&conn, "proj", &e).unwrap();
|
record_commit_entry(&conn, "proj", &e, None).unwrap();
|
||||||
record_commit_entry(&conn, "proj", &e).unwrap(); // 重跑
|
record_commit_entry(&conn, "proj", &e, None).unwrap(); // 重跑
|
||||||
|
|
||||||
let count: i64 = conn
|
let count: i64 = conn
|
||||||
.query_row("SELECT COUNT(*) FROM commit_files WHERE sha = 's1'", [], |r| r.get(0))
|
.query_row("SELECT COUNT(*) FROM commit_files WHERE sha = 's1'", [], |r| r.get(0))
|
||||||
@ -990,5 +1116,80 @@ Rules-Applied: R06", None).unwrap();
|
|||||||
record_commit(&conn, "pz", "s9", "feat: plain", None).unwrap();
|
record_commit(&conn, "pz", "s9", "feat: plain", None).unwrap();
|
||||||
assert!(rule_hit_stats(&conn).unwrap().is_empty());
|
assert!(rule_hit_stats(&conn).unwrap().is_empty());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ── U3:Rules-Applied 真实性校验 ────────────────────────────────────────
|
||||||
|
|
||||||
|
fn valid_set(ids: &[&str]) -> HashSet<String> {
|
||||||
|
ids.iter().map(|s| s.to_string()).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extract_rule_ids_respects_token_boundaries() {
|
||||||
|
let text = "### R01 规则一\n- R12:xxx(见 |R06| 表格)\nPR13 前粘字母不算,R2D2 后粘字母不算,r03 小写不算";
|
||||||
|
let ids = extract_rule_ids(text);
|
||||||
|
assert!(ids.contains("R01"));
|
||||||
|
assert!(ids.contains("R12"));
|
||||||
|
assert!(ids.contains("R06"));
|
||||||
|
assert!(!ids.contains("R13"), "PR13 内的 R13 不是独立 ID");
|
||||||
|
assert!(!ids.contains("R2"), "R2D2 内的 R2 不是独立 ID");
|
||||||
|
assert!(!ids.contains("R03") && !ids.contains("r03"), "小写 r 不提取");
|
||||||
|
assert!(extract_rule_ids("没有任何规则").is_empty());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn filter_valid_rules_drops_unknown_ids() {
|
||||||
|
let set = valid_set(&["R01", "R06"]);
|
||||||
|
assert_eq!(filter_valid_rules("R01,R99", Some(&set)).as_deref(), Some("R01"));
|
||||||
|
assert_eq!(filter_valid_rules("R99", Some(&set)), None, "全无效返回 None");
|
||||||
|
assert_eq!(
|
||||||
|
filter_valid_rules("R01,R99", None).as_deref(),
|
||||||
|
Some("R01,R99"),
|
||||||
|
"无 CONVENTIONS.md 时不校验"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn record_commit_entry_filters_unknown_ids_at_ingest() {
|
||||||
|
let conn = crate::db::conn_with_schema();
|
||||||
|
let set = valid_set(&["R01"]);
|
||||||
|
record_commit_entry(&conn, "proj", &entry("v1", "feat(a): x", "R01,R99"), Some(&set)).unwrap();
|
||||||
|
assert_eq!(rules_of(&conn, "v1").as_deref(), Some("R01"), "未知 ID 应在入库前剔除");
|
||||||
|
record_commit_entry(&conn, "proj", &entry("v2", "feat(a): y", "R99"), Some(&set)).unwrap();
|
||||||
|
assert_eq!(rules_of(&conn, "v2"), None, "全无效则该列为 NULL");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn record_commit_validated_filters_trailer_in_message() {
|
||||||
|
let conn = crate::db::conn_with_schema();
|
||||||
|
let set = valid_set(&["R05"]);
|
||||||
|
record_commit_validated(&conn, "proj", "w1", "feat(x): a\n\nRules-Applied: R05,R99", None, Some(&set)).unwrap();
|
||||||
|
assert_eq!(rules_of(&conn, "w1").as_deref(), Some("R05"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn sanitize_cleans_stored_pollution_idempotently() {
|
||||||
|
let conn = crate::db::conn_with_schema();
|
||||||
|
// 存量污染:校验上线前写入的行
|
||||||
|
record_commit_entry(&conn, "proj", &entry("p1", "feat: a", "R01,R99"), None).unwrap();
|
||||||
|
record_commit_entry(&conn, "proj", &entry("p2", "fix: b", "R99"), None).unwrap();
|
||||||
|
record_commit_entry(&conn, "proj", &entry("p3", "feat: c", "R01"), None).unwrap();
|
||||||
|
record_commit_entry(&conn, "other", &entry("q1", "feat: d", "R99"), None).unwrap();
|
||||||
|
|
||||||
|
let set = valid_set(&["R01"]);
|
||||||
|
let cleaned = sanitize_rules_applied(&conn, "proj", &set).unwrap();
|
||||||
|
assert_eq!(cleaned, 2, "p1 剔除 R99、p2 置 NULL;p3 干净不动");
|
||||||
|
assert_eq!(rules_of(&conn, "p1").as_deref(), Some("R01"));
|
||||||
|
assert_eq!(rules_of(&conn, "p2"), None);
|
||||||
|
assert_eq!(rules_of(&conn, "p3").as_deref(), Some("R01"));
|
||||||
|
assert_eq!(rules_of(&conn, "q1").as_deref(), Some("R99"), "只清洗指定项目");
|
||||||
|
|
||||||
|
// 清洗后交叉统计不再含未知 ID(acceptance 断言)
|
||||||
|
let stats = rule_hit_stats(&conn).unwrap();
|
||||||
|
assert!(stats.iter().filter(|s| s.rule_id == "R99").all(|s| s.project_count == 1),
|
||||||
|
"proj 的 R99 已清,仅剩 other 项目的");
|
||||||
|
|
||||||
|
// 幂等:重跑零清洗
|
||||||
|
assert_eq!(sanitize_rules_applied(&conn, "proj", &set).unwrap(), 0);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -517,13 +517,16 @@ pub fn poll_all_repos() -> Result<(usize, usize), String> {
|
|||||||
Err(_) => continue,
|
Err(_) => continue,
|
||||||
};
|
};
|
||||||
polled += 1;
|
polled += 1;
|
||||||
|
// U3:按 project_id 加载规则 ID 集合(每仓库一次),剔除 trailer 里的未知 ID
|
||||||
|
let valid_rules =
|
||||||
|
crate::commands::commit_metrics::load_rule_ids_by_project_id(&conn, project_id);
|
||||||
for (sha, message, date) in parse_gitea_commits(&json) {
|
for (sha, message, date) in parse_gitea_commits(&json) {
|
||||||
let before: i64 = conn.query_row(
|
let before: i64 = conn.query_row(
|
||||||
"SELECT COUNT(*) FROM commit_metrics WHERE sha = ?1", params![sha], |r| r.get(0),
|
"SELECT COUNT(*) FROM commit_metrics WHERE sha = ?1", params![sha], |r| r.get(0),
|
||||||
).unwrap_or(0);
|
).unwrap_or(0);
|
||||||
if before == 0
|
if before == 0
|
||||||
&& crate::commands::commit_metrics::record_commit(
|
&& crate::commands::commit_metrics::record_commit_validated(
|
||||||
&conn, project_id, &sha, &message, date.as_deref(),
|
&conn, project_id, &sha, &message, date.as_deref(), valid_rules.as_ref(),
|
||||||
).is_ok()
|
).is_ok()
|
||||||
{
|
{
|
||||||
ingested += 1;
|
ingested += 1;
|
||||||
|
|||||||
@ -263,7 +263,7 @@ mod tests {
|
|||||||
rules_trailer: String::new(),
|
rules_trailer: String::new(),
|
||||||
files: files.iter().map(|s| s.to_string()).collect(),
|
files: files.iter().map(|s| s.to_string()).collect(),
|
||||||
};
|
};
|
||||||
record_commit_entry(conn, pid, &entry).unwrap();
|
record_commit_entry(conn, pid, &entry, None).unwrap();
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
|
|||||||
@ -566,7 +566,10 @@ pub async fn tools_call(params: Option<&Value>) -> Value {
|
|||||||
) {
|
) {
|
||||||
Ok(r) => {
|
Ok(r) => {
|
||||||
ok += 1;
|
ok += 1;
|
||||||
lines.push(format!("- {}: 写入 {} 条,跳过 {} 条", id, r.imported, r.skipped));
|
lines.push(format!(
|
||||||
|
"- {}: 写入 {} 条,跳过 {} 条,清洗 {} 条无效规则ID",
|
||||||
|
id, r.imported, r.skipped, r.sanitized
|
||||||
|
));
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Err(e) => {
|
||||||
failed += 1;
|
failed += 1;
|
||||||
|
|||||||
@ -133,14 +133,18 @@ async fn handle_push(project_id: &str, body: &Bytes) -> Response {
|
|||||||
.into_response()
|
.into_response()
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
// U3:按 project_id 找到本地路径加载规则 ID 集合,剔除 trailer 里的未知 ID
|
||||||
|
let valid_rules =
|
||||||
|
crate::commands::commit_metrics::load_rule_ids_by_project_id(&conn, project_id);
|
||||||
let mut received = 0;
|
let mut received = 0;
|
||||||
for c in &commits {
|
for c in &commits {
|
||||||
if crate::commands::commit_metrics::record_commit(
|
if crate::commands::commit_metrics::record_commit_validated(
|
||||||
&conn,
|
&conn,
|
||||||
project_id,
|
project_id,
|
||||||
&c.sha,
|
&c.sha,
|
||||||
&c.message,
|
&c.message,
|
||||||
c.timestamp.as_deref(),
|
c.timestamp.as_deref(),
|
||||||
|
valid_rules.as_ref(),
|
||||||
)
|
)
|
||||||
.is_ok()
|
.is_ok()
|
||||||
{
|
{
|
||||||
|
|||||||
@ -112,17 +112,21 @@ export function FlywheelPanel() {
|
|||||||
setIngestSummary(null);
|
setIngestSummary(null);
|
||||||
let totalImported = 0;
|
let totalImported = 0;
|
||||||
let totalSkipped = 0;
|
let totalSkipped = 0;
|
||||||
|
let totalSanitized = 0;
|
||||||
for (const { id, path } of projectsWithPath) {
|
for (const { id, path } of projectsWithPath) {
|
||||||
try {
|
try {
|
||||||
const r = await ingestFullGitHistory(id, path);
|
const r = await ingestFullGitHistory(id, path);
|
||||||
totalImported += r.imported;
|
totalImported += r.imported;
|
||||||
totalSkipped += r.skipped;
|
totalSkipped += r.skipped;
|
||||||
|
totalSanitized += r.sanitized;
|
||||||
} catch {
|
} catch {
|
||||||
// 单个项目失败不中断其他项目
|
// 单个项目失败不中断其他项目
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
await refetchHealth();
|
await refetchHealth();
|
||||||
setIngestSummary(`新导入 ${totalImported} 条,跳过 ${totalSkipped} 条`);
|
setIngestSummary(
|
||||||
|
`新导入 ${totalImported} 条,跳过 ${totalSkipped} 条${totalSanitized > 0 ? `,清洗 ${totalSanitized} 条无效规则ID` : ""}`,
|
||||||
|
);
|
||||||
setIngesting(false);
|
setIngesting(false);
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@ -1580,6 +1580,8 @@ export const getCommitStats = (projectId: string) =>
|
|||||||
export interface IngestResult {
|
export interface IngestResult {
|
||||||
imported: number;
|
imported: number;
|
||||||
skipped: number;
|
skipped: number;
|
||||||
|
/** 存量行中被清洗掉未知规则 ID 的行数(U3;项目无 CONVENTIONS.md 时为 0) */
|
||||||
|
sanitized: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
export const ingestFullGitHistory = (projectId: string, projectPath: string) =>
|
export const ingestFullGitHistory = (projectId: string, projectPath: string) =>
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user