feat(gitea): G6 远端轮询采集——公网 Gitea + 内网炼境拓扑的观测面方案
- poll_all_repos：后台线程每 10 分钟出站拉取所有绑定仓库最新 commit （GET /repos/:owner/:repo/commits），record_commit 幂等写入，与本地 watcher / 全量导入三路径共存不重复计数 - gitea_poll_now 命令 + GiteaPanel「立即拉取」按钮与拓扑说明 - 决策记录：webhook 仅作同网补充，否决 frp 全端口映射（MCP 路由无鉴权， 27190 保持零公网暴露）；飞轮为统计系统，10 分钟延迟无影响 验证：cargo test 67 passed（新增轮询解析 2 测试）、typecheck 全绿 Rules-Applied: R01, R07
This commit is contained in:
lanrtop
parent
de961d0e95
commit
8a64b29445
@ -376,8 +376,8 @@ modules:
|
||||
name: Gitea 集成
|
||||
area: backend
|
||||
status: in_progress
|
||||
progress: 70
|
||||
note: "G1-G5 + Gitea Actions 生成完成（2026-07-02）；余：PR/CI 事件联动、DeepSeek review、存量迁移（concept）"
|
||||
progress: 78
|
||||
note: "G1-G6 + Gitea Actions 生成完成（2026-07-02，轮询采集适配公网拓扑）；余：PR/CI 事件联动、DeepSeek review、存量迁移（concept）"
|
||||
position: [3500, 150]
|
||||
updated: 2026-07-02
|
||||
|
||||
|
||||
@ -151,6 +151,15 @@ Gitea push → POST /webhook/gitea/:project_id
|
||||
- files: src-tauri/src/commands/gitea.rs, src/lib/commands.ts
|
||||
- acceptance: gitea_setup_webhook 生成随机 secret，调 Gitea API POST /repos/:owner/:repo/hooks 注册（events=push），写 webhook_secret + webhook_id 到 gitea_repos；gitea_unlink_repo 清理绑定
|
||||
|
||||
### ✅ G6. 远端轮询采集（webhook 不可达时的降级路径）
|
||||
- status: done
|
||||
- complexity: M
|
||||
- depends: G2
|
||||
- files: src-tauri/src/commands/gitea.rs, src-tauri/src/lib.rs, src/lib/commands.ts, src/components/dashboard/GiteaPanel.tsx
|
||||
- acceptance: 后台常驻线程每 10 分钟对所有 gitea_repos 绑定仓库调 GET /api/v1/repos/:owner/:repo/commits 拉取最新 commit，record_commit 幂等写入；gitea_poll_now 命令供 UI 手动触发并返回摘要；解析函数有单元测试；轮询结果记 log_event；适配「公网 Gitea + 内网炼境」拓扑（纯出站，无需 webhook 可达）
|
||||
|
||||
> 背景：用户实际拓扑为公网云 Gitea + 内网本机炼境，webhook 入站不可达；且不宜 frp 全端口映射 27190（MCP 路由无鉴权）。轮询是该拓扑下唯一安全的远端采集方式。
|
||||
|
||||
### ✅ G5. Gitea 治理面板 UI
|
||||
- status: done
|
||||
- complexity: M
|
||||
@ -158,8 +167,23 @@ Gitea push → POST /webhook/gitea/:project_id
|
||||
- files: src/components/dashboard/GiteaPanel.tsx, src/components/dashboard/BlueprintModal.tsx
|
||||
- acceptance: 治理面板新增"Gitea"区块；可添加/测试/删除实例；显示已绑定仓库列表（仓库名/webhook 状态）；操作：新建仓库、迁移、绑定已有、设置 webhook；显示可复制的 webhook 回调地址
|
||||
|
||||
## 决策记录
|
||||
|
||||
- **公网 Gitea + 内网炼境为标准拓扑（2026-07-02 裁决）**：观测面以 G6 出站轮询为主（每 10 分钟），webhook 仅作同网/有穿透场景的补充。理由：① 炼境 27190 端口（MCP 路由无鉴权）保持零公网暴露；② 飞轮是统计系统非监控系统，10 分钟延迟在天/周分析粒度下为零影响；③ /ship 在 enterprise-system 已实证全出站 API 可支撑完整 PR 流水线；④ 主动拉取更贴合风筝线"持线人"定位。若未来需秒级实时：云服务器 nginx 仅反代 /webhook/gitea/ 路径 + 内网穿透（备案，不建议）
|
||||
- 不用 frp 直接映射 27190：MCP Server 工具路由无鉴权，全端口暴露 = append_project_note 等工具裸奔公网
|
||||
|
||||
## 复盘笔记
|
||||
|
||||
【复盘】G6 远端轮询采集（2026-07-02）
|
||||
任务类型: 后端命令 + 前端组件
|
||||
复杂度: M
|
||||
实际轮数: 1轮（预估 1 轮）
|
||||
是否返工: 否
|
||||
有效策略: 需求源自真实拓扑约束（用户确认公网 Gitea + 内网炼境），先做安全裁决（否决 frp 全端口映射）再写代码；解析函数纯函数化可测；复用 record_commit 幂等性使轮询与本地 watcher/全量导入三路径共存不重复计数
|
||||
遗留风险: ① 轮询间隔 600s 硬编码，仓库数大时（50+）单轮串行请求可能拉长，届时可加并发或间隔配置；② limit=50 意味着 10 分钟内单仓库超 50 commit 会漏采（实际不可能，全量导入可兜底）
|
||||
auto_applied_rules: R01, R07
|
||||
rule_effectiveness: R01=有效（gitea_poll_now 命令与 commands.ts 封装同步）, R07=有效（解析纯函数 2 测试）
|
||||
|
||||
【复盘】Gitea Actions workflow 生成（2026-07-02）
|
||||
任务类型: 后端命令 + 前端组件
|
||||
复杂度: M
|
||||
|
||||
@ -461,6 +461,90 @@ pub fn gitea_unlink_repo(project_id: String) -> Result<(), String> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ── 远端轮询采集（G6：webhook 不可达时的降级路径）────────────────────────────
|
||||
|
||||
/// 从 Gitea commits API 响应解析出 (sha, message, date) 列表（纯函数，可测）
|
||||
pub fn parse_gitea_commits(json: &serde_json::Value) -> Vec<(String, String, Option<String>)> {
|
||||
json.as_array()
|
||||
.map(|arr| {
|
||||
arr.iter()
|
||||
.filter_map(|c| {
|
||||
let sha = c["sha"].as_str()?.to_string();
|
||||
let message = c["commit"]["message"].as_str().unwrap_or("").to_string();
|
||||
let date = c["commit"]["committer"]["date"]
|
||||
.as_str()
|
||||
.or_else(|| c["created"].as_str())
|
||||
.map(|s| s.to_string());
|
||||
Some((sha, message, date))
|
||||
})
|
||||
.collect()
|
||||
})
|
||||
.unwrap_or_default()
|
||||
}
|
||||
|
||||
/// 轮询所有绑定仓库的最新 commit，幂等写入 commit_metrics。
|
||||
/// 返回 (轮询仓库数, 新写入 commit 数)。纯出站 HTTPS，适配公网 Gitea + 内网炼境拓扑。
|
||||
pub fn poll_all_repos() -> Result<(usize, usize), String> {
|
||||
let conn = crate::db::pool().get().map_err(|e| e.to_string())?;
|
||||
let repos: Vec<(String, String, String, String, String)> = {
|
||||
let mut stmt = conn.prepare(
|
||||
"SELECT gr.project_id, gr.gitea_owner, gr.gitea_repo_name, gi.url, gi.access_token
|
||||
FROM gitea_repos gr JOIN gitea_instances gi ON gi.id = gr.instance_id",
|
||||
).map_err(|e| e.to_string())?;
|
||||
let rows = stmt.query_map([], |r| {
|
||||
Ok((r.get(0)?, r.get(1)?, r.get(2)?, r.get(3)?, r.get(4)?))
|
||||
}).map_err(|e| e.to_string())?;
|
||||
rows.collect::<rusqlite::Result<Vec<_>>>().map_err(|e| e.to_string())?
|
||||
};
|
||||
|
||||
let mut ingested = 0usize;
|
||||
let mut polled = 0usize;
|
||||
for (project_id, owner, repo, base_url, token) in &repos {
|
||||
let url = format!("{base_url}/api/v1/repos/{owner}/{repo}/commits?limit=50&stat=false&verification=false");
|
||||
let resp = match ureq::get(&url)
|
||||
.set("Authorization", &format!("token {token}"))
|
||||
.call()
|
||||
{
|
||||
Ok(r) => r,
|
||||
Err(e) => {
|
||||
crate::db::log_event("gitea_poll", "warn",
|
||||
&format!("轮询 {owner}/{repo} 失败: {}", api_err("拉取 commits", e)), None);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let json: serde_json::Value = match resp.into_json() {
|
||||
Ok(j) => j,
|
||||
Err(_) => continue,
|
||||
};
|
||||
polled += 1;
|
||||
for (sha, message, date) in parse_gitea_commits(&json) {
|
||||
let before: i64 = conn.query_row(
|
||||
"SELECT COUNT(*) FROM commit_metrics WHERE sha = ?1", params![sha], |r| r.get(0),
|
||||
).unwrap_or(0);
|
||||
if before == 0
|
||||
&& crate::commands::commit_metrics::record_commit(
|
||||
&conn, project_id, &sha, &message, date.as_deref(),
|
||||
).is_ok()
|
||||
{
|
||||
ingested += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ingested > 0 {
|
||||
crate::db::log_event("gitea_poll", "info",
|
||||
&format!("轮询 {polled} 个仓库，新采集 {ingested} 条 commit"), None);
|
||||
}
|
||||
Ok((polled, ingested))
|
||||
}
|
||||
|
||||
/// 手动触发一次轮询（UI「立即拉取」按钮）
|
||||
#[tauri::command]
|
||||
pub fn gitea_poll_now() -> Result<serde_json::Value, String> {
|
||||
let (polled, ingested) = poll_all_repos()?;
|
||||
Ok(serde_json::json!({ "polled": polled, "ingested": ingested }))
|
||||
}
|
||||
|
||||
// ── 仓库绑定查询命令 ──────────────────────────────────────────────────────────
|
||||
|
||||
/// 查询所有已绑定的仓库（含实例名）
|
||||
@ -604,4 +688,26 @@ mod tests {
|
||||
);
|
||||
assert!(result.is_err(), "project_id 唯一索引未生效");
|
||||
}
|
||||
|
||||
// ── G6：轮询解析 ──────────────────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn parse_gitea_commits_extracts_fields() {
|
||||
let json = serde_json::json!([
|
||||
{"sha": "abc", "commit": {"message": "feat(x): a", "committer": {"date": "2026-07-02T10:00:00+08:00"}}},
|
||||
{"sha": "def", "commit": {"message": "fix: b", "committer": {}}, "created": "2026-07-01T09:00:00+08:00"}
|
||||
]);
|
||||
let commits = parse_gitea_commits(&json);
|
||||
assert_eq!(commits.len(), 2);
|
||||
assert_eq!(commits[0].0, "abc");
|
||||
assert_eq!(commits[0].2.as_deref(), Some("2026-07-02T10:00:00+08:00"));
|
||||
assert_eq!(commits[1].2.as_deref(), Some("2026-07-01T09:00:00+08:00"), "committer.date 缺失时回退 created");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parse_gitea_commits_tolerates_garbage() {
|
||||
assert!(parse_gitea_commits(&serde_json::json!({"not": "array"})).is_empty());
|
||||
assert!(parse_gitea_commits(&serde_json::json!([{"no_sha": true}])).is_empty());
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@ -59,6 +59,16 @@ pub fn run() {
|
||||
let mcp_port = mcp_server::read_port();
|
||||
tauri::async_runtime::spawn(mcp_server::start(mcp_port));
|
||||
|
||||
// ── Gitea 远端轮询（G6）：每 10 分钟拉取绑定仓库 commit ───
|
||||
// 出站 HTTPS 采集，适配公网 Gitea + 内网炼境拓扑（webhook 不可达时的降级）
|
||||
std::thread::spawn(|| {
|
||||
std::thread::sleep(std::time::Duration::from_secs(30)); // 等 DB 与网络就绪
|
||||
loop {
|
||||
let _ = commands::gitea::poll_all_repos();
|
||||
std::thread::sleep(std::time::Duration::from_secs(600));
|
||||
}
|
||||
});
|
||||
|
||||
// ── 启动健康扫描（异步，不阻塞窗口渲染）────────────────
|
||||
tauri::async_runtime::spawn(async {
|
||||
tokio::task::spawn_blocking(run_startup_health_scan).await.ok();
|
||||
@ -338,6 +348,7 @@ pub fn run() {
|
||||
commands::gitea::gitea_link_repo,
|
||||
commands::gitea::gitea_setup_webhook,
|
||||
commands::gitea::gitea_unlink_repo,
|
||||
commands::gitea::gitea_poll_now,
|
||||
// beast status (read-only)
|
||||
get_beast_global_status,
|
||||
// source library
|
||||
|
||||
@ -16,6 +16,7 @@ import {
|
||||
giteaLinkRepo,
|
||||
giteaSetupWebhook,
|
||||
giteaUnlinkRepo,
|
||||
giteaPollNow,
|
||||
type GiteaInstance,
|
||||
} from "../../lib/commands";
|
||||
import { useUIStore } from "../../store/ui";
|
||||
@ -239,6 +240,19 @@ export function GiteaPanel({ projectId, projectName }: Props) {
|
||||
}
|
||||
};
|
||||
|
||||
const [polling, setPolling] = useState(false);
|
||||
const handlePollNow = async () => {
|
||||
setPolling(true);
|
||||
try {
|
||||
const r = await giteaPollNow();
|
||||
showToast(`✓ 轮询 ${r.polled} 个仓库，新采集 ${r.ingested} 条 commit`);
|
||||
} catch (e) {
|
||||
showToast(`❌ ${e}`);
|
||||
} finally {
|
||||
setPolling(false);
|
||||
}
|
||||
};
|
||||
|
||||
const copy = (text: string) => {
|
||||
navigator.clipboard.writeText(text).catch(() => {});
|
||||
showToast("已复制 ✓");
|
||||
@ -301,6 +315,18 @@ export function GiteaPanel({ projectId, projectName }: Props) {
|
||||
<p className="text-[10px] text-gray-400 leading-relaxed">
|
||||
需 Gitea 服务器能访问上述地址（MCP 端口默认 27190）。push 事件将实时写入飞轮 commit 数据。
|
||||
</p>
|
||||
<div className="flex items-center gap-1.5 border-t border-gray-100 pt-1.5">
|
||||
<p className="flex-1 text-[10px] text-gray-400 leading-relaxed">
|
||||
公网 Gitea + 内网炼境时 webhook 不可达——已由后台轮询兜底（每 10 分钟出站拉取）。
|
||||
</p>
|
||||
<button
|
||||
onClick={handlePollNow}
|
||||
disabled={polling}
|
||||
className={btnCls}
|
||||
>
|
||||
{polling ? "拉取中…" : "立即拉取"}
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
) : instances.length === 0 ? (
|
||||
|
||||
@ -1708,6 +1708,10 @@ export const giteaSetupWebhook = (projectId: string, callbackBase: string) =>
|
||||
export const giteaUnlinkRepo = (projectId: string) =>
|
||||
invoke<void>("gitea_unlink_repo", { projectId });
|
||||
|
||||
/** 手动触发一次远端轮询采集（后台每 10 分钟自动跑一次） */
|
||||
export const giteaPollNow = () =>
|
||||
invoke<{ polled: number; ingested: number }>("gitea_poll_now");
|
||||
|
||||
// ── Beast Global Status (read-only) ───────────────────────────────────────────
|
||||
|
||||
export interface BeastGlobalStatus {
|
||||
|
||||
Loading…
Reference in New Issue
Block a user