feat(gitea): G6 远端轮询采集——公网 Gitea + 内网炼境拓扑的观测面方案

- poll_all_repos:后台线程每 10 分钟出站拉取所有绑定仓库最新 commit
  (GET /repos/:owner/:repo/commits),record_commit 幂等写入,与本地
  watcher / 全量导入三路径共存不重复计数
- gitea_poll_now 命令 + GiteaPanel「立即拉取」按钮与拓扑说明
- 决策记录:webhook 仅作同网补充,否决 frp 全端口映射(MCP 路由无鉴权,
  27190 保持零公网暴露);飞轮为统计系统,10 分钟延迟无影响

验证:cargo test 67 passed(新增轮询解析 2 测试)、typecheck 全绿

Rules-Applied: R01, R07
This commit is contained in:
lanrtop 2026-07-02 16:36:18 +09:00
parent de961d0e95
commit 8a64b29445
6 changed files with 173 additions and 2 deletions

View File

@ -376,8 +376,8 @@ modules:
name: Gitea 集成 name: Gitea 集成
area: backend area: backend
status: in_progress status: in_progress
progress: 70 progress: 78
note: "G1-G5 + Gitea Actions 生成完成2026-07-02PR/CI 事件联动、DeepSeek review、存量迁移concept" note: "G1-G6 + Gitea Actions 生成完成2026-07-02轮询采集适配公网拓扑PR/CI 事件联动、DeepSeek review、存量迁移concept"
position: [3500, 150] position: [3500, 150]
updated: 2026-07-02 updated: 2026-07-02

View File

@ -151,6 +151,15 @@ Gitea push → POST /webhook/gitea/:project_id
- files: src-tauri/src/commands/gitea.rs, src/lib/commands.ts - files: src-tauri/src/commands/gitea.rs, src/lib/commands.ts
- acceptance: gitea_setup_webhook 生成随机 secret调 Gitea API POST /repos/:owner/:repo/hooks 注册events=push写 webhook_secret + webhook_id 到 gitea_reposgitea_unlink_repo 清理绑定 - acceptance: gitea_setup_webhook 生成随机 secret调 Gitea API POST /repos/:owner/:repo/hooks 注册events=push写 webhook_secret + webhook_id 到 gitea_reposgitea_unlink_repo 清理绑定
### ✅ G6. 远端轮询采集webhook 不可达时的降级路径)
- status: done
- complexity: M
- depends: G2
- files: src-tauri/src/commands/gitea.rs, src-tauri/src/lib.rs, src/lib/commands.ts, src/components/dashboard/GiteaPanel.tsx
- acceptance: 后台常驻线程每 10 分钟对所有 gitea_repos 绑定仓库调 GET /api/v1/repos/:owner/:repo/commits 拉取最新 commitrecord_commit 幂等写入gitea_poll_now 命令供 UI 手动触发并返回摘要;解析函数有单元测试;轮询结果记 log_event适配「公网 Gitea + 内网炼境」拓扑(纯出站,无需 webhook 可达)
> 背景:用户实际拓扑为公网云 Gitea + 内网本机炼境webhook 入站不可达;且不宜 frp 全端口映射 27190MCP 路由无鉴权)。轮询是该拓扑下唯一安全的远端采集方式。
### ✅ G5. Gitea 治理面板 UI ### ✅ G5. Gitea 治理面板 UI
- status: done - status: done
- complexity: M - complexity: M
@ -158,8 +167,23 @@ Gitea push → POST /webhook/gitea/:project_id
- files: src/components/dashboard/GiteaPanel.tsx, src/components/dashboard/BlueprintModal.tsx - files: src/components/dashboard/GiteaPanel.tsx, src/components/dashboard/BlueprintModal.tsx
- acceptance: 治理面板新增"Gitea"区块;可添加/测试/删除实例;显示已绑定仓库列表(仓库名/webhook 状态);操作:新建仓库、迁移、绑定已有、设置 webhook显示可复制的 webhook 回调地址 - acceptance: 治理面板新增"Gitea"区块;可添加/测试/删除实例;显示已绑定仓库列表(仓库名/webhook 状态);操作:新建仓库、迁移、绑定已有、设置 webhook显示可复制的 webhook 回调地址
## 决策记录
- **公网 Gitea + 内网炼境为标准拓扑2026-07-02 裁决)**:观测面以 G6 出站轮询为主(每 10 分钟webhook 仅作同网/有穿透场景的补充。理由:① 炼境 27190 端口MCP 路由无鉴权)保持零公网暴露;② 飞轮是统计系统非监控系统10 分钟延迟在天/周分析粒度下为零影响;③ /ship 在 enterprise-system 已实证全出站 API 可支撑完整 PR 流水线;④ 主动拉取更贴合风筝线"持线人"定位。若未来需秒级实时:云服务器 nginx 仅反代 /webhook/gitea/ 路径 + 内网穿透(备案,不建议)
- 不用 frp 直接映射 27190MCP Server 工具路由无鉴权,全端口暴露 = append_project_note 等工具裸奔公网
## 复盘笔记 ## 复盘笔记
【复盘】G6 远端轮询采集2026-07-02
任务类型: 后端命令 + 前端组件
复杂度: M
实际轮数: 1轮预估 1 轮)
是否返工: 否
有效策略: 需求源自真实拓扑约束(用户确认公网 Gitea + 内网炼境),先做安全裁决(否决 frp 全端口映射)再写代码;解析函数纯函数化可测;复用 record_commit 幂等性使轮询与本地 watcher/全量导入三路径共存不重复计数
遗留风险: ① 轮询间隔 600s 硬编码仓库数大时50+)单轮串行请求可能拉长,届时可加并发或间隔配置;② limit=50 意味着 10 分钟内单仓库超 50 commit 会漏采(实际不可能,全量导入可兜底)
auto_applied_rules: R01, R07
rule_effectiveness: R01=有效gitea_poll_now 命令与 commands.ts 封装同步), R07=有效(解析纯函数 2 测试)
【复盘】Gitea Actions workflow 生成2026-07-02 【复盘】Gitea Actions workflow 生成2026-07-02
任务类型: 后端命令 + 前端组件 任务类型: 后端命令 + 前端组件
复杂度: M 复杂度: M

View File

@ -461,6 +461,90 @@ pub fn gitea_unlink_repo(project_id: String) -> Result<(), String> {
Ok(()) Ok(())
} }
// ── 远端轮询采集G6webhook 不可达时的降级路径)────────────────────────────
/// 从 Gitea commits API 响应解析出 (sha, message, date) 列表(纯函数,可测)
pub fn parse_gitea_commits(json: &serde_json::Value) -> Vec<(String, String, Option<String>)> {
json.as_array()
.map(|arr| {
arr.iter()
.filter_map(|c| {
let sha = c["sha"].as_str()?.to_string();
let message = c["commit"]["message"].as_str().unwrap_or("").to_string();
let date = c["commit"]["committer"]["date"]
.as_str()
.or_else(|| c["created"].as_str())
.map(|s| s.to_string());
Some((sha, message, date))
})
.collect()
})
.unwrap_or_default()
}
/// 轮询所有绑定仓库的最新 commit幂等写入 commit_metrics。
/// 返回 (轮询仓库数, 新写入 commit 数)。纯出站 HTTPS适配公网 Gitea + 内网炼境拓扑。
pub fn poll_all_repos() -> Result<(usize, usize), String> {
let conn = crate::db::pool().get().map_err(|e| e.to_string())?;
let repos: Vec<(String, String, String, String, String)> = {
let mut stmt = conn.prepare(
"SELECT gr.project_id, gr.gitea_owner, gr.gitea_repo_name, gi.url, gi.access_token
FROM gitea_repos gr JOIN gitea_instances gi ON gi.id = gr.instance_id",
).map_err(|e| e.to_string())?;
let rows = stmt.query_map([], |r| {
Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?, r.get(4)?))
}).map_err(|e| e.to_string())?;
rows.collect::<rusqlite::Result<Vec<_>>>().map_err(|e| e.to_string())?
};
let mut ingested = 0usize;
let mut polled = 0usize;
for (project_id, owner, repo, base_url, token) in &repos {
let url = format!("{base_url}/api/v1/repos/{owner}/{repo}/commits?limit=50&stat=false&verification=false");
let resp = match ureq::get(&url)
.set("Authorization", &format!("token {token}"))
.call()
{
Ok(r) => r,
Err(e) => {
crate::db::log_event("gitea_poll", "warn",
&format!("轮询 {owner}/{repo} 失败: {}", api_err("拉取 commits", e)), None);
continue;
}
};
let json: serde_json::Value = match resp.into_json() {
Ok(j) => j,
Err(_) => continue,
};
polled += 1;
for (sha, message, date) in parse_gitea_commits(&json) {
let before: i64 = conn.query_row(
"SELECT COUNT(*) FROM commit_metrics WHERE sha = ?1", params![sha], |r| r.get(0),
).unwrap_or(0);
if before == 0
&& crate::commands::commit_metrics::record_commit(
&conn, project_id, &sha, &message, date.as_deref(),
).is_ok()
{
ingested += 1;
}
}
}
if ingested > 0 {
crate::db::log_event("gitea_poll", "info",
&format!("轮询 {polled} 个仓库,新采集 {ingested} 条 commit"), None);
}
Ok((polled, ingested))
}
/// 手动触发一次轮询UI「立即拉取」按钮
#[tauri::command]
pub fn gitea_poll_now() -> Result<serde_json::Value, String> {
let (polled, ingested) = poll_all_repos()?;
Ok(serde_json::json!({ "polled": polled, "ingested": ingested }))
}
// ── 仓库绑定查询命令 ────────────────────────────────────────────────────────── // ── 仓库绑定查询命令 ──────────────────────────────────────────────────────────
/// 查询所有已绑定的仓库(含实例名) /// 查询所有已绑定的仓库(含实例名)
@ -604,4 +688,26 @@ mod tests {
); );
assert!(result.is_err(), "project_id 唯一索引未生效"); assert!(result.is_err(), "project_id 唯一索引未生效");
} }
// ── G6轮询解析 ──────────────────────────────────────────────────────
#[test]
fn parse_gitea_commits_extracts_fields() {
let json = serde_json::json!([
{"sha": "abc", "commit": {"message": "feat(x): a", "committer": {"date": "2026-07-02T10:00:00+08:00"}}},
{"sha": "def", "commit": {"message": "fix: b", "committer": {}}, "created": "2026-07-01T09:00:00+08:00"}
]);
let commits = parse_gitea_commits(&json);
assert_eq!(commits.len(), 2);
assert_eq!(commits[0].0, "abc");
assert_eq!(commits[0].2.as_deref(), Some("2026-07-02T10:00:00+08:00"));
assert_eq!(commits[1].2.as_deref(), Some("2026-07-01T09:00:00+08:00"), "committer.date 缺失时回退 created");
} }
#[test]
fn parse_gitea_commits_tolerates_garbage() {
assert!(parse_gitea_commits(&serde_json::json!({"not": "array"})).is_empty());
assert!(parse_gitea_commits(&serde_json::json!([{"no_sha": true}])).is_empty());
}
}

View File

@ -59,6 +59,16 @@ pub fn run() {
let mcp_port = mcp_server::read_port(); let mcp_port = mcp_server::read_port();
tauri::async_runtime::spawn(mcp_server::start(mcp_port)); tauri::async_runtime::spawn(mcp_server::start(mcp_port));
// ── Gitea 远端轮询G6每 10 分钟拉取绑定仓库 commit ───
// 出站 HTTPS 采集,适配公网 Gitea + 内网炼境拓扑webhook 不可达时的降级)
std::thread::spawn(|| {
std::thread::sleep(std::time::Duration::from_secs(30)); // 等 DB 与网络就绪
loop {
let _ = commands::gitea::poll_all_repos();
std::thread::sleep(std::time::Duration::from_secs(600));
}
});
// ── 启动健康扫描(异步,不阻塞窗口渲染)──────────────── // ── 启动健康扫描(异步,不阻塞窗口渲染)────────────────
tauri::async_runtime::spawn(async { tauri::async_runtime::spawn(async {
tokio::task::spawn_blocking(run_startup_health_scan).await.ok(); tokio::task::spawn_blocking(run_startup_health_scan).await.ok();
@ -338,6 +348,7 @@ pub fn run() {
commands::gitea::gitea_link_repo, commands::gitea::gitea_link_repo,
commands::gitea::gitea_setup_webhook, commands::gitea::gitea_setup_webhook,
commands::gitea::gitea_unlink_repo, commands::gitea::gitea_unlink_repo,
commands::gitea::gitea_poll_now,
// beast status (read-only) // beast status (read-only)
get_beast_global_status, get_beast_global_status,
// source library // source library

View File

@ -16,6 +16,7 @@ import {
giteaLinkRepo, giteaLinkRepo,
giteaSetupWebhook, giteaSetupWebhook,
giteaUnlinkRepo, giteaUnlinkRepo,
giteaPollNow,
type GiteaInstance, type GiteaInstance,
} from "../../lib/commands"; } from "../../lib/commands";
import { useUIStore } from "../../store/ui"; import { useUIStore } from "../../store/ui";
@ -239,6 +240,19 @@ export function GiteaPanel({ projectId, projectName }: Props) {
} }
}; };
const [polling, setPolling] = useState(false);
const handlePollNow = async () => {
setPolling(true);
try {
const r = await giteaPollNow();
showToast(`✓ 轮询 ${r.polled} 个仓库,新采集 ${r.ingested} 条 commit`);
} catch (e) {
showToast(`${e}`);
} finally {
setPolling(false);
}
};
const copy = (text: string) => { const copy = (text: string) => {
navigator.clipboard.writeText(text).catch(() => {}); navigator.clipboard.writeText(text).catch(() => {});
showToast("已复制 ✓"); showToast("已复制 ✓");
@ -301,6 +315,18 @@ export function GiteaPanel({ projectId, projectName }: Props) {
<p className="text-[10px] text-gray-400 leading-relaxed"> <p className="text-[10px] text-gray-400 leading-relaxed">
Gitea 访MCP 27190push commit Gitea 访MCP 27190push commit
</p> </p>
<div className="flex items-center gap-1.5 border-t border-gray-100 pt-1.5">
<p className="flex-1 text-[10px] text-gray-400 leading-relaxed">
Gitea + webhook 10
</p>
<button
onClick={handlePollNow}
disabled={polling}
className={btnCls}
>
{polling ? "拉取中…" : "立即拉取"}
</button>
</div>
</div> </div>
</> </>
) : instances.length === 0 ? ( ) : instances.length === 0 ? (

View File

@ -1708,6 +1708,10 @@ export const giteaSetupWebhook = (projectId: string, callbackBase: string) =>
export const giteaUnlinkRepo = (projectId: string) => export const giteaUnlinkRepo = (projectId: string) =>
invoke<void>("gitea_unlink_repo", { projectId }); invoke<void>("gitea_unlink_repo", { projectId });
/** 手动触发一次远端轮询采集(后台每 10 分钟自动跑一次) */
export const giteaPollNow = () =>
invoke<{ polled: number; ingested: number }>("gitea_poll_now");
// ── Beast Global Status (read-only) ─────────────────────────────────────────── // ── Beast Global Status (read-only) ───────────────────────────────────────────
export interface BeastGlobalStatus { export interface BeastGlobalStatus {