P0 — 新建 common/js/media-upload.js (12KB)
- uploadImage(filePath) / uploadMedia(filePath, kind) 落地
- 此前 ai-chat-tools-panel.vue 引用但文件从未入库, 选图立即抛 "Cannot find module"
- 12 cloud provider 分流: 4-field STS (alioss/qiniu/txcos/baidubos/qingstor/jdoss/ucloud/wyoss) vs single-URL signed PUT (huaweiobs/volcengine/awsoss/minio)
- 端点必须 addon namespace: /<addon>/api/upload/{image|file|ossSts}, 普通 /api/kefu/upload/* 不命中 InitRoute
- 补 docs 设计稿漏写的 `import store from '@/store/index.js'` (H5 上传需带 token)
P1 — ai-chat-message.vue 用户附件全支持
- P1.1 filter 不再丢 file/audio/video 消息
- P1.2 用户气泡加 type==='file' 分支: 文件图标 + 名称 + 大小 + 上传状态
- P1.3 sendMessage type 不再强制把 audio/video 折成 file
P2 — AI 侧 audio 播放引擎接通
- P2.1 previewAttachment audio 分支 → 跳 file-preview
- P2.2 音频气泡接 uni.createInnerAudioContext: data 加 innerAudioCtx/_innerAudioSrc, 改写 playAudio/pauseAudio, beforeDestroy 必 destroy, slider 区分 change (显示) / changeend (实际 seek)
- P2.3 file-preview.vue 加 image-viewer + audio-viewer 两个分支;inferFileType 识别 jpg/png/mp3/wav 等扩展
P3 — sendMessageWithContent 透传 attachments 参数
- 默认 attachments = [], 传给 sendStreamMessage/sendNormalMessage
- tools panel 选完文件 push 到 attachments → 链到 send
lang — zh-cn + en-us file-preview 加 4 项
- audioUrlNotExist / audioDefaultTitle / audioPlaying / audioPaused
docs — docs/ai-agent-integration.md (跨 PR 累积同步, 2149 行)
- 版本基线提到 PR-WB-MP-INLINE-MEDIA (2026-09-16) 及之后
- 加 §3.1.5 attachments.data 字段协议 (PR-WB-MP-INLINE-MEDIA)
- 12 cloud provider 替换原"阿里云"单点 (PR-OSS-REGISTRY)
- media-upload.js 章节补 fileToBase64 (PR-FRONTEND-MEDIA-UPLOAD)
- 上传策略表 backend/oss_sts/auto 全替换为"云存储"措辞
对齐后端:
- PR-WB-MP-INLINE-MEDIA (a707b90): media data 字段
- PR-OSS-REGISTRY (12 provider): OssProviderRegistry
- PR-FRONTEND-MEDIA-UPLOAD (本 commit): 前端落地
兼容性:
- 后端未升级到 PR-WB-MP-INLINE-MEDIA 之前, attachments 不带 data 字段, 走 URL 路径 (退化到老行为, 不破)
- 后端 oss_sts 单点 alioss 时, securityToken 不带 -SIGNED 后缀, 走 4-field STS 分支 (与 PR-OSS-REGISTRY 之前兼容)
回归:
- node --check media-upload.js OK
- lang zh ESM import OK, 输出 8 个 key 全对
- ai-chat-message.vue 11 处 PR-FRONTEND-MEDIA-UPLOAD 标记, file-preview.vue 9 处
- 旧 chat 路径不带 attachments 走默认 [], 不破坏现有响应结构
安全约束: 不涉及 nginx 层防盗链 / 限速
2149 lines
88 KiB
Markdown
2149 lines
88 KiB
Markdown
# AI Agent 接入使用说明 (Uniapp 前端)
|
||
|
||
> 适用版本: backend PR-WB-MP-INLINE-MEDIA (2026-09-16) 及之后; PR-OSS-REGISTRY (2026-09-15) 之后 12 个云存储 provider
|
||
> 适用前端: `D:/projects/shop-projects/frontend` (uniapp 多端: 微信小程序 MP / H5 / App / Harmony)
|
||
> 配套阅读:
|
||
> - `common/js/ai-service.js`、`pages_tool/ai-chat/`
|
||
> - 后端 `docs/mp-media-upload-integration.md` (媒体上传后端单点真源)
|
||
> - 后端 `docs/ai-agent-protocol-spec.md` (本文档对应的完整后端协议规范, 仓内后端目录)
|
||
> - `D:/projects/refs/codebuddy-agent-sdk/docs/wb-cloud-sdk-coverage.md` (WB Cloud SDK 协议约束矩阵 — J7/J9/J10/J1/Q6)
|
||
|
||
本文档面向前端工程师, 解决把 WorkBuddy Agent / AI 智能客服集成进 uniapp 时
|
||
最常踩的 4 个坑:
|
||
|
||
1. **AI 思考流数据** (`thought` event + `message_end.thoughts` 总结)
|
||
2. **Question 选项及反馈** (`ask_user_question` + `elicit_response` 回传)
|
||
3. **图片/音视频/文件上传下载** (走我们服务器 vs 走云存储直传, 何时切)
|
||
4. **多轮会话** (`conversation_id` 持久化 + per-agent 隔离)
|
||
|
||
每个坑都按 "现状 → 目标 → 代码示例 → 关键注意点" 4 段写, 可直接复制粘贴。
|
||
|
||
---
|
||
|
||
## 0. 30 秒读懂 (TL;DR)
|
||
|
||
| 主题 | 现状 (PR 前) | 目标 (本文档落地后) |
|
||
|---|---|---|
|
||
| 思考流 | 丢, 不渲染 | 折叠面板渲染, 与正文区分 |
|
||
| Question 选项 | 丢, 用户看不到 | 渲染为可点击选项卡片, 选择后回传 |
|
||
| 媒体上传 | 完全没接 `wx.uploadFile` | 接 `wx.uploadFile` + 12 个云存储直传 + 类型/大小分流 |
|
||
| attachments.data 字段 | 没概念 (后端 attachments 全 url-only, 部署到云上后 WB 服务端 fetch 不一定能通) | ≤5MB 图片/音频塞 base64 inline, 完全不依赖 J9;>5MB 才走 resource_link |
|
||
| 多轮会话 | `conversation_id` 已持久化但 per-agent 隔离缺失 | per-agent 隔离 + 历史消息渲染 |
|
||
|
||
涉及到的核心代码改动点 (按文件):
|
||
|
||
- `common/js/ai-service.js` — SSE/WS 中间层改成 `onEvent(event, data)`, 透出所有 SSE event 类型
|
||
- `pages_tool/ai-chat/ai-chat-message.vue` — 思考流折叠 + 选项卡片渲染 + 媒体上传按钮接 `wx.uploadFile`
|
||
- `pages_tool/ai-chat/components/ai-chat-tools-panel.vue` — `selectImage`/`selectFile` 改成真上传
|
||
- `common/js/media-upload.js` — **新增**, 封装走我们服务器 + 12 个云存储直传决策 + fileToBase64
|
||
- `common/js/ai-chat-message-store.js` — **新增**, 集中管理消息状态 (思考流/选项/附件)
|
||
|
||
---
|
||
|
||
## 1. AI 思考流数据
|
||
|
||
### 1.1 后端协议 (单点真源)
|
||
|
||
**关键事实: 思考流有 3 条下发通道 (2 类 × 2/1 路), 行为各不同:**
|
||
|
||
| 通道 | event | 频次 | 用途 |
|
||
|---|---|---|---|
|
||
| WS (`/ws/aiagent` 或 `/ws/aikefu`) | `thought` | 每个 WB Agent `agent_thought_chunk` 增量 | 实时渲染 |
|
||
| HTTP SSE aikefu (`/api/aikefu/kefu/chat`) | `thought` | 每个增量 | 实时渲染 |
|
||
| HTTP SSE aiagent (`/api/aiagent/chat`) | **(无独立 event)** | — | **完全不带 thought 流** (后端不抛) |
|
||
|
||
**重点:**
|
||
|
||
- **WS 路径 (微信小程序主用)**: 实时推 `thought` 增量 event, 直接拼到 UI 即可
|
||
- **HTTP SSE aikefu 路径**: 实时推 `thought` 增量 event + 流结束 `message_end.thoughts` (累计完整文本)
|
||
- **HTTP SSE aiagent 路径**: **不推任何 thought 数据** — 既无中途 event, `message_end` 也不带 `thoughts` 字段. 前端如果用 aiagent HTTP, 必须告诉用户"看不到 AI 思考过程"
|
||
|
||
**SSE 帧列表 (完整, 按通道分别说明):**
|
||
|
||
| event | aiagent HTTP | aikefu HTTP | WS (两者) |
|
||
|---|---|---|---|
|
||
| `message` | ✓ `{answer}` | ✓ `{answer}` | ✓ `{answer}` |
|
||
| `thought` | ✗ (无) | ✓ `{thought}` (实时) | ✓ `{thought}` (实时) |
|
||
| `tool_call` | ✗ (无单独 event, 见下) | ✓ `{tool_call: {...}}` | ✓ `{tool_call: {...}}` |
|
||
| `question` | ✗ (无, message_end 也没有 pending_question — **完全丢失**) | ✓ `{pending_question: {...}}` | ✓ `{pending_question: {...}}` |
|
||
| `message_end` | **极简** (`conversation_id, agent_id, stop_reason` 三字段) | **完整快照** (见下) | **完整快照** |
|
||
| `error` | JSON envelope (非 SSE 流) | ✓ SSE `{code, message}` | ✓ `{message, code, line, file}` |
|
||
|
||
**message_end 字段 (按通道差异):**
|
||
|
||
```jsonc
|
||
// aiagent HTTP (Chat.php:530-545) — 极简, 只有 3 字段
|
||
{
|
||
"event": "message_end",
|
||
"conversation_id": "openai_1700000000_xxxx",
|
||
"agent_id": "agent_aikefu",
|
||
"stop_reason": "end_turn" | "pending_stale" | ""
|
||
// 没有 message_id / content / thoughts / tool_calls / tokens / phase / effective_model
|
||
}
|
||
|
||
// aikefu HTTP B-path (Kefu.php:1138-1157) + A-path (Kefu.php:1449-1465) + WS — 完整快照
|
||
{
|
||
"event": "message_end",
|
||
"message_id": "wb_xxxx", // 一定有
|
||
"conversation_id": "...",
|
||
"thoughts": "完整思考 (累计)", // aikefu 才有, aiagent 没
|
||
"pending_question": null | {...}, // aikefu 才有, aiagent HTTP 完全没这字段
|
||
"tool_calls": [{ // aikefu 才有
|
||
"tool_call_id": "tc_xxx",
|
||
"tool_name": "RAGSearch / AskUserQuestion",
|
||
"status": "completed",
|
||
"started_at": ..., "completed_at": ..., "duration_ms": 10000,
|
||
"raw_input": "...", "raw_output": "...", "error": null
|
||
}],
|
||
"tokens": { // aikefu 才有
|
||
"input": 100, "output": 200,
|
||
"cached_read": 50, "cached_write": 0,
|
||
"cost_amount": 0.002, "cost_currency": "CNY", "model": "gpt-4o"
|
||
},
|
||
"phase": "planning|working|completed|idle", // aikefu 才有
|
||
"effective_model": "gpt-4o", // aikefu 才有
|
||
"stop_reason": "end_turn"
|
||
// 注意: 没有 "content" 字段! 前端拿最终正文必须从 message 事件流累积, 不能等 message_end
|
||
}
|
||
```
|
||
|
||
**关键不对称 (前端必须知道):**
|
||
|
||
- **aiagent HTTP `message_end` 只有 3 字段**. 想拿完整快照必须换 aikefu HTTP 或 WS
|
||
- **aiagent HTTP 没有独立 `thought` event 也没有 `message_end.thoughts`**. 思考过程**完全不可见**
|
||
- **aiagent HTTP 没有独立 `tool_call` event**. 要看工具调用得等流结束后查 DB 历史
|
||
- **aiagent HTTP 没有独立 `question` event**. 用户没答导致 `pending_stale` 时, 也没法做 fallback 渲染
|
||
- **`effective_model` / `tokens` / `phase` 仅 aikefu 有**. aiagent HTTP 流调试时这些字段永远是 `undefined`
|
||
|
||
> **修复建议**: 集成时如果产品需要"显示 AI 思考过程", 必须走 WS 或 aikefu HTTP. 走 aiagent HTTP 的话, 思考面板要么不渲染要么显示 "思考过程不可用".
|
||
|
||
### 1.2 现状 (PR 前)
|
||
|
||
`common/js/ai-service.js` 的 WS 消息处理只识别 `event === 'message'`, 其他事件被丢:
|
||
|
||
```js
|
||
// ai-service.js:201-209
|
||
if (data.event === 'message') {
|
||
var text = data.answer || data.content || data.text || '';
|
||
// ... append ...
|
||
} else if (data.event === 'message_end' || data.event === 'done') {
|
||
// ... save conversation_id ...
|
||
} else {
|
||
console.log('[AI WS] 未识别的消息:', JSON.stringify(data).substring(0, 200));
|
||
}
|
||
```
|
||
|
||
H5 的 `sendHttpStream` 同样 (L452-466)。
|
||
|
||
**结论: 思考流后端已经在发, 前端 100% 丢。**
|
||
|
||
### 1.3 目标行为
|
||
|
||
思考流以**折叠面板**形式渲染在 AI 消息**上方**:
|
||
|
||
```
|
||
┌─────────────────────────────────────┐
|
||
│ 🧠 思考中... (点击展开) │ ← 折叠态 (流未结束)
|
||
│ 用户问的是订单问题, 我需要... │ ← 展开后
|
||
│ 查订单 API... │
|
||
├─────────────────────────────────────┤
|
||
│ 您的订单已发货, 预计明天到达 │ ← 用户可见正文
|
||
└─────────────────────────────────────┘
|
||
```
|
||
|
||
要点:
|
||
|
||
- 流未结束 (收到 `thought` 增量) → 面板显示 "🧠 思考中..." + 累计字符数
|
||
- 流结束 (`message_end`) → 面板变成可点击的 "💡 思考过程 (N 步)" 折叠
|
||
- aiagent HTTP 路径: 流过程中不渲染, 流结束从 `message_end.thoughts` 一次渲染
|
||
- B 路 Dify 兼容 (`> ` 前缀) : 把 `answer` 里 `> ` 前缀行也塞同一面板
|
||
|
||
### 1.4 代码示例
|
||
|
||
**Step 1: 改造 `common/js/ai-service.js`** — 把 onChunk 改成 onEvent, 透出所有事件类型
|
||
|
||
```js
|
||
// common/js/ai-service.js
|
||
// 在文件顶部, 加 helper 把 SSE event 标准化
|
||
|
||
/**
|
||
* 统一 SSE/WS 事件分发器
|
||
* 事件类型: 'message' | 'thought' | 'tool_call' | 'question'
|
||
* | 'message_end' | 'error' | 'attachments' | 'ping'
|
||
*/
|
||
function dispatchEvent(onEvent, event, data) {
|
||
if (typeof onEvent === 'function') {
|
||
try { onEvent(event, data); }
|
||
catch (e) { console.error('[AI Service] onEvent handler threw:', e); }
|
||
}
|
||
}
|
||
|
||
// === sendStreamMessage (MP, WS) ===
|
||
async sendStreamMessage(message, onEvent, onComplete, opts = {}) {
|
||
// #ifdef MP
|
||
try { await ensureWebSocketConnected(); }
|
||
catch (e) {
|
||
if (onComplete) onComplete({ error: e.message });
|
||
throw e;
|
||
}
|
||
|
||
wsMessageActive = true;
|
||
eventHandlers.onEvent = onEvent || null;
|
||
wsMessageOnComplete = onComplete || null;
|
||
|
||
return new Promise((resolve, reject) => {
|
||
wsMessageResolve = resolve;
|
||
wsMessageReject = reject;
|
||
wsSocketTask.send({
|
||
data: JSON.stringify({
|
||
action: 'chat',
|
||
agent_id: getAgentId(),
|
||
uniacid: this.getUniacid(),
|
||
query: message,
|
||
user_id: getUserId(),
|
||
conversation_id: this.getConversationId() || '',
|
||
attachments: opts.attachments || [],
|
||
...getDomainParams()
|
||
})
|
||
});
|
||
});
|
||
// #endif
|
||
},
|
||
|
||
// === sendHttpStream (H5, SSE) ===
|
||
async sendHttpStream(message, onEvent, onComplete, opts = {}) {
|
||
// #ifdef H5
|
||
var chatUrl = resolveApiUrl('chat');
|
||
var url = Config.baseUrl + chatUrl;
|
||
const response = await fetch(url, {
|
||
method: 'POST',
|
||
headers: {
|
||
'Content-Type': 'application/json',
|
||
'Accept': 'text/event-stream',
|
||
},
|
||
body: JSON.stringify({
|
||
uniacid: this.getUniacid(),
|
||
token: store.state.token || 'test_token',
|
||
inputs: {},
|
||
query: message,
|
||
response_mode: 'streaming',
|
||
agent_id: getAgentId(),
|
||
user_id: getUserId(),
|
||
conversation_id: this.getConversationId() || '',
|
||
attachments: opts.attachments || [],
|
||
...getDomainParams()
|
||
})
|
||
});
|
||
if (!response.ok || !response.body) {
|
||
throw new Error(lang.lang('aiService.invalidResponse'));
|
||
}
|
||
const reader = response.body.getReader();
|
||
const decoder = new TextDecoder('utf-8');
|
||
let buffer = '';
|
||
while (true) {
|
||
const { done, value } = await reader.read();
|
||
if (done) break;
|
||
buffer += decoder.decode(value, { stream: true });
|
||
const lines = buffer.split('\n');
|
||
buffer = lines.pop() || '';
|
||
for (const line of lines) {
|
||
const trimmed = line.trim();
|
||
if (!trimmed.startsWith('data:')) continue;
|
||
const jsonStr = trimmed.slice(5).trim();
|
||
if (!jsonStr || jsonStr === '[DONE]') continue;
|
||
try {
|
||
const data = JSON.parse(jsonStr);
|
||
// ★ ★ ★ 核心改动: 按 event 分类分发 ★ ★ ★
|
||
switch (data.event) {
|
||
case 'message':
|
||
var text = data.answer || data.text || data.content || '';
|
||
if (!text && data.data) text = data.data.content || '';
|
||
dispatchEvent(onEvent, 'message', { content: text, message_id: data.message_id });
|
||
break;
|
||
case 'thought':
|
||
// WS 路径和 aikefu HTTP 才有这个; aiagent HTTP 不发独立 thought
|
||
dispatchEvent(onEvent, 'thought', { content: data.thought || '', message_id: data.message_id });
|
||
break;
|
||
case 'tool_call':
|
||
// tool_call 是 upsert 语义, 不是 append
|
||
dispatchEvent(onEvent, 'tool_call', data.tool_call || {});
|
||
break;
|
||
case 'question':
|
||
// ★ 见 §2: AskUserQuestion 单独处理
|
||
dispatchEvent(onEvent, 'question', {
|
||
pending_question: data.pending_question || data.data?.pending_question,
|
||
message_id: data.message_id,
|
||
conversation_id: data.conversation_id,
|
||
});
|
||
break;
|
||
case 'message_end':
|
||
dispatchEvent(onEvent, 'message_end', {
|
||
conversation_id: data.conversation_id || data.data?.conversation_id,
|
||
message_id: data.message_id,
|
||
content: data.content,
|
||
thoughts: data.thoughts || '',
|
||
tool_calls: data.tool_calls || [],
|
||
tokens: data.tokens || null,
|
||
phase: data.phase || 'completed',
|
||
stop_reason: data.stop_reason || '',
|
||
effective_model: data.effective_model || '',
|
||
});
|
||
break;
|
||
case 'error':
|
||
dispatchEvent(onEvent, 'error', { code: data.code || -1, message: data.message || 'unknown' });
|
||
break;
|
||
default:
|
||
// 兜底: B 路 Dify 把思考塞 answer 里 (> 前缀)
|
||
if (data.event === 'message' && data.answer && data.answer.startsWith('> ')) {
|
||
dispatchEvent(onEvent, 'thought', { content: data.answer });
|
||
} else if (data.event === 'ping') {
|
||
// keep-alive, 忽略
|
||
} else {
|
||
console.log('[AI SSE] 未识别事件:', data.event, JSON.stringify(data).substring(0, 200));
|
||
}
|
||
}
|
||
} catch (e) {
|
||
console.warn('解析失败:', e, line);
|
||
}
|
||
}
|
||
}
|
||
// 流结束: 触发 onComplete
|
||
if (onComplete) onComplete({});
|
||
// #endif
|
||
},
|
||
```
|
||
|
||
WS 路径同样改造 `onMessage` (L181-240):
|
||
|
||
```js
|
||
// ai-service.js WS onMessage 改造
|
||
wsSocketTask.onMessage(function(res) {
|
||
try {
|
||
var data = JSON.parse(res.data);
|
||
if (data.type === 'auth_success') { wsAuthenticated = true; resolve(); return; }
|
||
if (data.type === 'auth_failed') { /* ... */ return; }
|
||
if (!wsMessageActive) return;
|
||
|
||
switch (data.event) {
|
||
case 'message':
|
||
var text = data.answer || data.text || data.content || '';
|
||
if (!text && data.data) text = data.data.content || '';
|
||
wsMessageContent += text;
|
||
dispatchEvent(eventHandlers.onEvent, 'message', { content: text, message_id: data.message_id });
|
||
break;
|
||
case 'thought':
|
||
dispatchEvent(eventHandlers.onEvent, 'thought', { content: data.thought || '' });
|
||
break;
|
||
case 'tool_call':
|
||
dispatchEvent(eventHandlers.onEvent, 'tool_call', data.tool_call || {});
|
||
break;
|
||
case 'question':
|
||
dispatchEvent(eventHandlers.onEvent, 'question', {
|
||
pending_question: data.pending_question,
|
||
message_id: data.message_id,
|
||
conversation_id: data.conversation_id,
|
||
});
|
||
break;
|
||
case 'message_end':
|
||
case 'done':
|
||
var conversationId = data.conversation_id || (data.data && data.data.conversation_id) || '';
|
||
if (conversationId) this.setConversationId(conversationId);
|
||
wsMessageActive = false;
|
||
wsMessageContent = '';
|
||
var endPayload = {
|
||
conversation_id: conversationId,
|
||
message_id: data.message_id,
|
||
content: data.content || wsMessageContent,
|
||
thoughts: data.thoughts || '',
|
||
tool_calls: data.tool_calls || [],
|
||
tokens: data.tokens || null,
|
||
phase: data.phase || 'completed',
|
||
stop_reason: data.stop_reason || '',
|
||
};
|
||
dispatchEvent(eventHandlers.onEvent, 'message_end', endPayload);
|
||
if (wsMessageOnComplete) wsMessageOnComplete(endPayload);
|
||
if (wsMessageResolve) wsMessageResolve(endPayload);
|
||
break;
|
||
case 'error':
|
||
dispatchEvent(eventHandlers.onEvent, 'error', { code: data.code || -1, message: data.message });
|
||
break;
|
||
case 'ping':
|
||
// WB B-path keep-alive, 忽略
|
||
break;
|
||
default:
|
||
// 兼容老格式: data.data.content
|
||
if (data.data && data.data.content) {
|
||
wsMessageContent += data.data.content;
|
||
dispatchEvent(eventHandlers.onEvent, 'message', { content: data.data.content });
|
||
}
|
||
}
|
||
} catch (e) { console.error('[AI WS] 解析失败:', e, res.data); }
|
||
});
|
||
```
|
||
|
||
顶部加 `eventHandlers`:
|
||
|
||
```js
|
||
let eventHandlers = { onEvent: null };
|
||
```
|
||
|
||
**Step 2: 渲染层 — `ai-chat-message.vue`** — 思考流面板
|
||
|
||
```vue
|
||
<template>
|
||
<!-- AI 消息块 -->
|
||
<view v-for="msg in messages" :key="msg.id" class="message" :class="{ 'ai-message': msg.role === 'ai' }">
|
||
<!-- ★ 思考流折叠面板 -->
|
||
<view v-if="msg.role === 'ai' && msg.thoughtsText" class="thought-panel"
|
||
:class="{ streaming: msg.isStreaming && msg.thoughtsText }">
|
||
<view class="thought-header" @click="toggleThought(msg)">
|
||
<text class="thought-icon">{{ msg.isStreaming ? '🧠' : '💡' }}</text>
|
||
<text class="thought-label">
|
||
{{ msg.isStreaming ? '思考中...' : `思考过程 (${msg.thoughtSteps || 1} 步)` }}
|
||
</text>
|
||
<text v-if="!msg.isStreaming" class="thought-toggle">{{ msg.thoughtExpanded ? '▼' : '▶' }}</text>
|
||
</view>
|
||
<view v-if="msg.isStreaming || msg.thoughtExpanded" class="thought-content">
|
||
<text>{{ msg.thoughtsText }}</text>
|
||
</view>
|
||
</view>
|
||
|
||
<!-- 正文 -->
|
||
<view class="message-content">{{ msg.content }}</view>
|
||
|
||
<!-- 工具调用 (可选, 用于调试可见性) -->
|
||
<view v-if="msg.toolCalls && msg.toolCalls.length" class="tool-calls">
|
||
<view v-for="tc in msg.toolCalls" :key="tc.tool_call_id" class="tool-call-item"
|
||
:class="['status-' + tc.status]">
|
||
<text>🔧 {{ tc.tool_name }}: {{ tc.status }}{{ tc.duration_ms ? ` (${tc.duration_ms}ms)` : '' }}</text>
|
||
</view>
|
||
</view>
|
||
|
||
<!-- 附件 (见 §3) -->
|
||
<view v-if="msg.attachments && msg.attachments.length" class="attachments">
|
||
<view v-for="att in msg.attachments" :key="att.url" class="attachment" @click="previewAttachment(att)">
|
||
<image v-if="att.type === 'image'" :src="att.url" mode="aspectFill" class="att-image" />
|
||
<video v-else-if="att.type === 'video'" :src="att.url" controls class="att-video" />
|
||
<view v-else class="att-file">
|
||
<text class="file-icon">📎</text>
|
||
<text class="file-name">{{ att.name || '文件' }}</text>
|
||
<text class="file-size">{{ formatSize(att.size) }}</text>
|
||
</view>
|
||
</view>
|
||
</view>
|
||
|
||
<!-- 选项卡 (见 §2) -->
|
||
<view v-if="msg.pendingQuestion && !msg.questionAnswered" class="question-card">
|
||
<!-- 见 §2 详细实现 -->
|
||
</view>
|
||
|
||
<!-- Token 用量 (可选, 调试面板) -->
|
||
<view v-if="msg.tokens && msg.tokens.input + msg.tokens.output > 0" class="token-usage">
|
||
<text class="token-text">tokens: {{ msg.tokens.input }}+{{ msg.tokens.output }}</text>
|
||
</view>
|
||
</view>
|
||
</template>
|
||
|
||
<script>
|
||
// 把 onEvent 改造: 不再只接 (chunk), 而是接 (event, data)
|
||
async sendStreamMessage(userMessage, attachments = []) {
|
||
const loadingMsg = this._ensureLoadingMessage();
|
||
|
||
// 当前流式消息的状态对象
|
||
const streamingMsg = this._transformLoadingToStreaming(loadingMsg);
|
||
|
||
const onEvent = (event, data) => {
|
||
switch (event) {
|
||
case 'message':
|
||
streamingMsg.content = (streamingMsg.content || '') + data.content;
|
||
this.scrollToBottom();
|
||
break;
|
||
case 'thought':
|
||
// WS 路径: 增量; aiagent HTTP 路径: 不会触发 (流结束才一次)
|
||
streamingMsg.thoughtsText = (streamingMsg.thoughtsText || '') + data.content;
|
||
this.scrollToBottom();
|
||
break;
|
||
case 'tool_call':
|
||
// ★ 必须按 tool_call_id upsert, WB 会重复推 pending
|
||
if (!streamingMsg.toolCalls) streamingMsg.toolCalls = [];
|
||
const idx = streamingMsg.toolCalls.findIndex(tc => tc.tool_call_id === data.tool_call_id);
|
||
if (idx >= 0) {
|
||
streamingMsg.toolCalls[idx] = { ...streamingMsg.toolCalls[idx], ...data };
|
||
} else {
|
||
streamingMsg.toolCalls.push(data);
|
||
}
|
||
break;
|
||
case 'question':
|
||
// ★ 见 §2
|
||
streamingMsg.pendingQuestion = data.pending_question;
|
||
streamingMsg.questionAnswered = false;
|
||
streamingMsg.rpc_id = data.pending_question?.rpc_id;
|
||
streamingMsg.conversation_id = data.conversation_id;
|
||
streamingMsg.message_id = data.message_id;
|
||
this.startQuestionTimer(streamingMsg);
|
||
break;
|
||
case 'message_end':
|
||
// ★ 重要: aiagent HTTP 路径下, 这是唯一拿 thoughts 的地方
|
||
streamingMsg.isStreaming = false;
|
||
streamingMsg.content = data.content || streamingMsg.content;
|
||
if (data.thoughts) streamingMsg.thoughtsText = data.thoughts;
|
||
if (data.tool_calls) streamingMsg.toolCalls = data.tool_calls;
|
||
if (data.tokens) streamingMsg.tokens = data.tokens;
|
||
if (data.phase) streamingMsg.phase = data.phase;
|
||
if (data.stop_reason === 'pending_stale') {
|
||
// 用户长时间没回答, 后端默认 reject
|
||
streamingMsg.questionAnswered = true;
|
||
streamingMsg.questionTimedOut = true;
|
||
}
|
||
if (data.conversation_id) {
|
||
this.currentConversationId = data.conversation_id;
|
||
aiService.setConversationId(data.conversation_id);
|
||
}
|
||
this._finishStream(streamingMsg, data);
|
||
break;
|
||
case 'error':
|
||
streamingMsg.isStreaming = false;
|
||
streamingMsg.content += '\n[错误] ' + (data.message || '未知错误');
|
||
break;
|
||
case 'attachments':
|
||
if (!streamingMsg.attachments) streamingMsg.attachments = [];
|
||
streamingMsg.attachments.push(...(data || []));
|
||
break;
|
||
}
|
||
};
|
||
|
||
try {
|
||
// #ifdef H5
|
||
await aiService.sendHttpStream(userMessage, onEvent, null, { attachments });
|
||
// #endif
|
||
// #ifdef MP-WEIXIN
|
||
await aiService.sendStreamMessage(userMessage, onEvent, null, { attachments });
|
||
// #endif
|
||
} catch (e) {
|
||
this._handleStreamError(streamingMsg, e);
|
||
}
|
||
},
|
||
|
||
toggleThought(msg) {
|
||
if (!msg.isStreaming) msg.thoughtExpanded = !msg.thoughtExpanded;
|
||
}
|
||
</script>
|
||
|
||
<style lang="scss" scoped>
|
||
.thought-panel {
|
||
background: #f6f8fa;
|
||
border-left: 3rpx solid #8b8b8b;
|
||
padding: 12rpx 16rpx;
|
||
margin-bottom: 12rpx;
|
||
border-radius: 8rpx;
|
||
font-size: 24rpx;
|
||
color: #666;
|
||
&.streaming {
|
||
background: #fff8e1;
|
||
border-left-color: #ff9800;
|
||
.thought-label { color: #ff9800; }
|
||
}
|
||
.thought-header { display: flex; align-items: center; }
|
||
.thought-icon { margin-right: 8rpx; }
|
||
.thought-label { flex: 1; }
|
||
.thought-toggle { font-size: 20rpx; color: #999; }
|
||
.thought-content {
|
||
margin-top: 8rpx;
|
||
white-space: pre-wrap;
|
||
line-height: 1.6;
|
||
max-height: 400rpx;
|
||
overflow-y: auto;
|
||
}
|
||
}
|
||
.tool-calls {
|
||
margin-top: 8rpx;
|
||
padding: 8rpx 12rpx;
|
||
background: #f0f4f8;
|
||
border-radius: 4rpx;
|
||
font-size: 22rpx;
|
||
.tool-call-item { padding: 4rpx 0; color: #666; }
|
||
.status-completed { color: #2e7d32; }
|
||
.status-failed { color: #c62828; }
|
||
.status-pending, .status-in_progress { color: #f57c00; }
|
||
}
|
||
.token-usage {
|
||
margin-top: 4rpx;
|
||
font-size: 20rpx;
|
||
color: #999;
|
||
text-align: right;
|
||
}
|
||
</style>
|
||
```
|
||
|
||
### 1.5 关键注意点
|
||
|
||
1. **aiagent HTTP 路径下, 中途没有 `thought` event**: 流式渲染期间, 思考面板一直空; 流结束后从 `message_end.thoughts` 一次渲染完整文本。这是协议事实, 不是 bug
|
||
2. **WS 路径下 `thought` 是增量**: 必须用 `+=` 拼接, 不要 set 覆盖; 同一段思考可能被切成多个 chunk
|
||
3. **`tool_call` 必须 upsert**: WB 协议对同一个 `tool_call_id` 会推 10+ 次 `pending`, 必须按 id 合并, 不能 append
|
||
4. **`tokens` 数据 (input/output/cost/model)**: 调试面板用, 普通用户不需要看到
|
||
5. **`stop_reason` 区分**: `end_turn` = 正常结束, `pending_stale` = 用户长时间没回答 + 后端 reject, `error` = 异常
|
||
6. **B 路 Dify 兼容 `> `> 前缀**: 不是协议标准, 是 Dify 把思考内容混进 `answer` 的习惯, 后端透传, 前端要兜底拆
|
||
7. **思考步数限制**: 折叠面板最多展示 20 步 / 累计 4000 字符, 超出折叠 ("还有 5 步未显示")
|
||
|
||
### 1.6 后端 SSE event 来源 (供调试)
|
||
|
||
如果前端收不到 `thought`, 用 curl 验 WS 路径 (aikefu HTTP 路径也行):
|
||
|
||
```bash
|
||
# aikefu HTTP (B-path 或 A-path 都行, 这里是 B-path)
|
||
curl -N 'https://your-domain.com/api/aikefu/kefu/chat' \
|
||
-H 'Content-Type: application/json' \
|
||
-H 'Accept: text/event-stream' \
|
||
-d '{
|
||
"config_uuid":"your_uuid",
|
||
"agent_id":"agent_aikefu",
|
||
"query":"hi",
|
||
"user_id":"guest_test",
|
||
"response_mode":"streaming"
|
||
}'
|
||
```
|
||
|
||
应该看到 (aikefu HTTP 走 `event:` + `data:` 两行):
|
||
|
||
```
|
||
: open\n\n ← aikefu HTTP 立即推, 让响应头 flush
|
||
|
||
event: message
|
||
data: {"event":"message","answer":"您好...","message_id":"wb_xxx",...}
|
||
|
||
event: thought
|
||
data: {"event":"thought","thought":"用户打招呼...",...}
|
||
|
||
event: message_end
|
||
data: {"event":"message_end","conversation_id":"...","message_id":"wb_xxx","thoughts":"...","tokens":{...},"stop_reason":"end_turn"}
|
||
|
||
```
|
||
|
||
> **注意格式差异**: aikefu HTTP 写 `event: <name>\ndata: <json>\n\n` 两行式; aiagent HTTP 只写 `data: <json>\n\n` 一行式. 上面的示例只适用于 aikefu. 验 aiagent HTTP 时, `event` 字段只能从 data 行的 JSON 体内取.
|
||
|
||
如果只有 `message` 和 `message_end`, 看:
|
||
- `runtime/log/app_run_info/2026-09-14-wb-rpc.log` 里 `processSseEvent` 日志
|
||
- `runtime/log/app_run_info/2026-09-14-wb-rest.log` 看是 A-path 还是 B-path
|
||
|
||
---
|
||
|
||
## 2. Question 选项及反馈
|
||
|
||
### 2.1 后端协议 (真实格式, 不要再用旧的 `options[]` 假设)
|
||
|
||
WB Agent 用 `AskUserQuestion` 工具触发提问, 后端 `WbAgentOsClient::processSseEvent` 解析两种来源:
|
||
|
||
1. **JSON-RPC 协议** `method === '_codebuddy.ai/question'`, 携带 `params.schema.questions[]`
|
||
2. **Session update 协议** `sessionUpdate === 'tool_call_update'`, `_meta.codebuddy.ai.toolName === 'AskUserQuestion'`
|
||
|
||
两种协议解析后统一抛 `pending_question`:
|
||
|
||
```json
|
||
{
|
||
"tool_call_id": "tc_xxx",
|
||
"questions": [
|
||
{
|
||
"question": "您想了解哪种商品的售后政策?",
|
||
"options": [
|
||
{"label": "退款", "description": "7 天无理由退款"},
|
||
{"label": "换货", "description": "30 天内可换同款"},
|
||
{"label": "维修", "description": "1 年保修"},
|
||
{"label": "仅咨询"}
|
||
],
|
||
"header": "售后类型",
|
||
"multiSelect": false
|
||
}
|
||
],
|
||
"source": "elicitation",
|
||
"asked_at": 1700000000,
|
||
"rpc_id": 0
|
||
}
|
||
```
|
||
|
||
**关键字段 (跟旧版假设差别很大):**
|
||
|
||
| 字段 | 含义 |
|
||
|---|---|
|
||
| `tool_call_id` | 工具调用 ID, 回传时必须带 |
|
||
| `questions[]` | 数组! 不是单 question 字符串 |
|
||
| `questions[].question` | 问题文本 |
|
||
| `questions[].options[]` | 选项数组 |
|
||
| `options[].label` | 显示给用户的文本, **也是回传的值** (没有 `value` 字段) |
|
||
| `options[].description` | 鼠标 hover 提示, 可选 |
|
||
| `questions[].header` | 短标签 (顶部 chip), 可选 |
|
||
| `questions[].multiSelect` | 是否多选 |
|
||
| `rpc_id` | **JSON-RPC 的请求 ID, 回传答案时必带** |
|
||
|
||
### 2.2 Answer 回传机制 (3 种, 不要瞎写)
|
||
|
||
| 场景 | 端点 / 动作 | 字段 |
|
||
|---|---|---|
|
||
| **管理员后台测试对话框** (aiagent shop controller) | `POST /aiagent/shop/agent/testChatRespond` (form, 无 `/addon/` 前缀) | `conversation_id, message_id, tool_call_id, rpc_id, answers[]` |
|
||
| **WS 通道** (MP / H5 任意, 流还没结束) | WS `{action: 'elicit_response', ...}` | `conversation_id, message_id, tool_call_id, rpc_id, answers: {"q0":"label",...}` |
|
||
| **HTTP SSE aiagent 通道** | **没有原生 answer 回传路径** | 必须改用 WS, 或等 `pending_stale` (≈10s) 后端 emit message_end |
|
||
| **HTTP SSE aikefu 通道** | 同样没有原生路径 | 同上 |
|
||
|
||
**`answers` 格式:**
|
||
|
||
```json
|
||
// 单选 (multiSelect=false)
|
||
{
|
||
"q0": "退款"
|
||
}
|
||
|
||
// 多选 (multiSelect=true), 多个 label 用数组
|
||
{
|
||
"q0": ["退款", "换货"]
|
||
}
|
||
```
|
||
|
||
`q0` 是 question 在数组里的 index (字符串化), value 是 label 文本 (不是 id, 不是 value)。
|
||
|
||
### 2.3 现状 (PR 前)
|
||
|
||
`ai-service.js` SSE 解析只识别 `event === 'message'`, `pending_question` / `question` event 被丢 (`ai-service.js:233-236` 的 `else` 分支只 log)。
|
||
|
||
用户看到的是 AI 一句问话后**直接卡住**, 实际是后端在等前端回传选项。
|
||
|
||
### 2.4 目标行为
|
||
|
||
AI 消息末尾渲染**选项卡片**:
|
||
|
||
```
|
||
┌─────────────────────────────────────┐
|
||
│ 售后类型 │ ← header (可选)
|
||
│ 您想了解哪种商品的售后政策? │ ← question text
|
||
│ │
|
||
│ [退款] [换货] [维修] [仅咨询] │ ← 选项 (label)
|
||
│ 7天无理由 30天可换 1年保修 ← description (可选)
|
||
└─────────────────────────────────────┘
|
||
```
|
||
|
||
要点:
|
||
- 单选: 点击选项 → 卡片折叠 ("已选择: 退款 ✓") + 自动发送
|
||
- 多选: 多个选项可点 → 底部出现 "确认 (N)" 按钮
|
||
- 30 秒未选 → 倒计时 + 自动选第一个, 后端 emit `stop_reason='pending_stale'`
|
||
|
||
### 2.5 代码示例
|
||
|
||
**Step 1: ai-service 加 `elicitResponse` 方法**
|
||
|
||
```js
|
||
// common/js/ai-service.js 末尾追加
|
||
|
||
/**
|
||
* 向 WorkBuddy Agent 回传 AskUserQuestion 的用户选项
|
||
*
|
||
* @param {Object} params
|
||
* @param {string} params.conversation_id
|
||
* @param {string} params.message_id
|
||
* @param {string} params.tool_call_id
|
||
* @param {number} params.rpc_id
|
||
* @param {Object} params.answers - {q0: 'label', q1: ['a','b']}
|
||
* @returns {Promise<boolean>}
|
||
*/
|
||
async elicitResponse({ conversation_id, message_id, tool_call_id, rpc_id, answers }) {
|
||
const addon = resolveAddonName(currentAgentId);
|
||
|
||
// ★ aiagent + aikefu 都支持 WS elicit_response (WebSocket.php:146/147)
|
||
// 走同一段 WS 发送逻辑
|
||
// #ifdef MP-WEIXIN || APP-PLUS || MP-HARMONY
|
||
await ensureWebSocketConnected();
|
||
wsSocketTask.send({
|
||
data: JSON.stringify({
|
||
action: 'elicit_response',
|
||
conversation_id,
|
||
message_id,
|
||
tool_call_id,
|
||
rpc_id,
|
||
answers,
|
||
uniacid: this.getUniacid(),
|
||
token: store.state.token || 'test_token',
|
||
})
|
||
});
|
||
return true;
|
||
// #endif
|
||
|
||
// #ifdef H5
|
||
// H5 端没有原生 WS 长连接, HTTP SSE 也没有 elicit_response 端点
|
||
// 兜底: 复用 chat 端点 + action=elicit_response body
|
||
// (此端点需后端补 — 当前后端 5 个 chat 入口 (aiagent Chat + aikefu Kefu B+A + 2 个 WS) 都没注册 elicit-response HTTP 路由)
|
||
// 临时方案: 30s 超时后端 emit pending_stale, 用户的选项卡片就当 UI 提示展示, 不真回传给 Agent
|
||
console.warn('[ai-service] H5 elicit_response 无可用通道, 仅展示用户选项 + 30s 后由 pending_stale 兜底');
|
||
return false;
|
||
// #endif
|
||
},
|
||
```
|
||
|
||
**Step 2: 渲染层 — 选项卡片**
|
||
|
||
```vue
|
||
<!-- ai-chat-message.vue template, 插在 message block 里 -->
|
||
<view v-if="msg.pendingQuestion && !msg.questionAnswered" class="question-card">
|
||
<view v-for="(q, qi) in msg.pendingQuestion.questions" :key="qi" class="question-block">
|
||
<view v-if="q.header" class="question-header-chip">{{ q.header }}</view>
|
||
<view class="question-text">{{ q.question }}</view>
|
||
<view v-if="msg.questionTimeout > 0" class="question-timer">⏱ {{ msg.questionTimeout }}s</text></view>
|
||
<view class="question-options">
|
||
<view v-for="opt in q.options" :key="opt.label"
|
||
class="option-btn"
|
||
:class="{
|
||
selected: q.multiSelect
|
||
? (msg.selectedAnswers[`q${qi}`] || []).includes(opt.label)
|
||
: msg.selectedAnswers[`q${qi}`] === opt.label
|
||
}"
|
||
@click="toggleOption(msg, qi, q, opt)">
|
||
<text class="option-label">{{ opt.label }}</text>
|
||
<text v-if="opt.description" class="option-desc">{{ opt.description }}</text>
|
||
</view>
|
||
</view>
|
||
<view v-if="q.multiSelect && (msg.selectedAnswers[`q${qi}`] || []).length" class="confirm-row">
|
||
<button class="confirm-btn" @click="confirmMultiSelect(msg, qi)">
|
||
确认 ({{ (msg.selectedAnswers[`q${qi}`] || []).length }})
|
||
</button>
|
||
</view>
|
||
</view>
|
||
</view>
|
||
|
||
<script>
|
||
// methods
|
||
toggleOption(msg, qi, q, opt) {
|
||
if (!msg.selectedAnswers) msg.selectedAnswers = {};
|
||
const key = `q${qi}`;
|
||
if (q.multiSelect) {
|
||
const arr = msg.selectedAnswers[key] || [];
|
||
const idx = arr.indexOf(opt.label);
|
||
if (idx >= 0) arr.splice(idx, 1);
|
||
else arr.push(opt.label);
|
||
msg.selectedAnswers[key] = arr;
|
||
} else {
|
||
// 单选: 立即选中 + 立即回传
|
||
msg.selectedAnswers[key] = opt.label;
|
||
this.submitQuestionAnswer(msg);
|
||
}
|
||
},
|
||
|
||
confirmMultiSelect(msg) {
|
||
this.submitQuestionAnswer(msg);
|
||
},
|
||
|
||
async submitQuestionAnswer(msg) {
|
||
if (!msg.rpc_id) {
|
||
uni.showToast({ title: 'rpc_id 缺失, 无法回传', icon: 'none' });
|
||
return;
|
||
}
|
||
msg.questionAnswered = true;
|
||
msg.questionTimedOut = false;
|
||
|
||
try {
|
||
await aiService.elicitResponse({
|
||
conversation_id: msg.conversation_id || aiService.getConversationId(),
|
||
message_id: msg.message_id,
|
||
tool_call_id: msg.pendingQuestion.tool_call_id,
|
||
rpc_id: msg.rpc_id,
|
||
answers: msg.selectedAnswers,
|
||
});
|
||
// 后端会接着上次的 tool_call 上下文继续流
|
||
// 不需要前端额外做什么, 继续等 onEvent 回调
|
||
} catch (e) {
|
||
uni.showToast({ title: '选项提交失败: ' + e.message, icon: 'none' });
|
||
msg.questionAnswered = false;
|
||
}
|
||
},
|
||
|
||
startQuestionTimer(msg) {
|
||
msg.questionTimeout = 30;
|
||
if (!this.questionTimers) this.questionTimers = {};
|
||
if (this.questionTimers[msg.id]) clearInterval(this.questionTimers[msg.id]);
|
||
this.questionTimers[msg.id] = setInterval(() => {
|
||
if (!msg.pendingQuestion || msg.questionAnswered) {
|
||
clearInterval(this.questionTimers[msg.id]);
|
||
return;
|
||
}
|
||
msg.questionTimeout -= 1;
|
||
if (msg.questionTimeout <= 0) {
|
||
clearInterval(this.questionTimers[msg.id]);
|
||
// 超时: 自动选第一个选项, 触发后端 pending_stale
|
||
const firstQ = msg.pendingQuestion.questions[0];
|
||
if (firstQ && firstQ.options[0]) {
|
||
msg.selectedAnswers = { q0: firstQ.options[0].label };
|
||
msg.questionTimedOut = true;
|
||
this.submitQuestionAnswer(msg);
|
||
}
|
||
}
|
||
}, 1000);
|
||
},
|
||
|
||
// ★ Vue 2 项目 (uniapp MP 默认 Vue 2). 用 beforeDestroy, 不用 beforeUnmount (那是 Vue 3)
|
||
// 证据: frontend/pages_tool/ai-chat/ai-chat-message.vue:480 也用的 beforeDestroy
|
||
beforeDestroy() {
|
||
Object.values(this.questionTimers || {}).forEach(t => clearInterval(t));
|
||
}
|
||
</script>
|
||
|
||
<style lang="scss" scoped>
|
||
.question-card {
|
||
background: #fff;
|
||
border: 1rpx solid #e0e0e0;
|
||
border-radius: 12rpx;
|
||
padding: 16rpx;
|
||
margin-top: 12rpx;
|
||
.question-header-chip {
|
||
display: inline-block;
|
||
background: #f0f4f8;
|
||
color: #666;
|
||
padding: 4rpx 12rpx;
|
||
border-radius: 16rpx;
|
||
font-size: 22rpx;
|
||
margin-bottom: 8rpx;
|
||
}
|
||
.question-text { font-size: 28rpx; color: #333; margin-bottom: 12rpx; }
|
||
.question-timer { font-size: 22rpx; color: #f57c00; float: right; }
|
||
.question-options {
|
||
display: flex;
|
||
flex-wrap: wrap;
|
||
gap: 12rpx;
|
||
.option-btn {
|
||
background: #f6f8fa;
|
||
border: 1rpx solid #e0e0e0;
|
||
border-radius: 8rpx;
|
||
padding: 12rpx 20rpx;
|
||
font-size: 26rpx;
|
||
&.selected {
|
||
background: #e3f2fd;
|
||
border-color: #1976d2;
|
||
color: #1976d2;
|
||
}
|
||
.option-label { display: block; }
|
||
.option-desc { display: block; font-size: 22rpx; color: #999; margin-top: 4rpx; }
|
||
}
|
||
}
|
||
.confirm-row {
|
||
margin-top: 12rpx;
|
||
text-align: right;
|
||
.confirm-btn {
|
||
background: #1976d2;
|
||
color: #fff;
|
||
border-radius: 8rpx;
|
||
padding: 8rpx 24rpx;
|
||
font-size: 26rpx;
|
||
}
|
||
}
|
||
}
|
||
</style>
|
||
```
|
||
|
||
### 2.6 关键注意点
|
||
|
||
1. **`label` 是回传的值, 不是 `value`**: 协议没有 `value` 字段, 显示啥就回传啥 (改文案会导致原选项答非所问)
|
||
2. **answers 是对象 `{q0: label, q1: [label,...]}`**, 不是数组
|
||
3. **多 question**: WB 协议允许一次推多个 question, UI 要按数组渲染, 全部答完才提交
|
||
4. **WS 是 HTTP SSE 的兜底**: HTTP SSE aiagent 通道没有原生 answer 回传, 实际项目里建议前端始终连 WS, elicit_response 走 WS
|
||
5. **后端等答案 ≈ 10s × 3 协议尝试** (`PENDING_STALE_AFTER`), 超时后 emit `message_end` with `stop_reason='pending_stale'`, 前端要展示 "对方没回应, 已跳过提问"
|
||
6. **不同步 `pending_question` 到 history**: 历史消息不带 question, 用户重看历史不会触发 "重新答"
|
||
7. **`rpc_id` 是 int**: JSON-RPC 的请求 ID, 回传时原样带回去, 后端用它绑回原始 RPC 调用
|
||
8. **`description` 是可选 hover 提示**: WB Agent 可能不填, 前端判断 `opt.description` 再渲染
|
||
|
||
### 2.7 后端兜底 (前端不用管)
|
||
|
||
- 后端 `tool_call_id` 找不到 → 回 `{code: -1, msg: '找不到对应的提问'}`
|
||
- 后端超时 (10s × 3 协议) → 走 `pending_stale` 分支, emit message_end, `stop_reason='pending_stale'`
|
||
|
||
---
|
||
|
||
## 3. 图片/音视频/文件上传下载
|
||
|
||
### 3.1 后端协议 (单点真源, 详见 `docs/mp-media-upload-integration.md`)
|
||
|
||
**端点汇总 (按 agent addon 分组):**
|
||
|
||
| 用途 | aiagent addon | aikefu addon |
|
||
|---|---|---|
|
||
| 图片上传 (走我们服务器) | `POST /aiagent/api/upload/image` | `POST /aikefu/api/upload/image` |
|
||
| 文件上传 (audio/video/file, 走我们服务器) | `POST /aiagent/api/upload/file` | `POST /aikefu/api/upload/file` |
|
||
| 拿云存储直传凭证 | `POST /aiagent/api/upload/ossSts` | `POST /aikefu/api/upload/ossSts` |
|
||
| 拿 agent 配置 (含 upload_strategy) | `POST /api/aiagent/info` | `POST /api/kefu/info` |
|
||
|
||
**URL 模式说明 (2026-09-14 PR-WB-MP-MEDIA-API-PREFIX 修正):**
|
||
|
||
上传端点必须用 **addon 模式 URL** (`/<addon>/api/upload/<action>`,addon 名在前),原因:
|
||
|
||
- 之前曾用 `/api/<addon>/upload/<action>` (addon 名在后) — 后端 InitRoute 默认 system_array 命中分支不切 namespace,精确路由注册也实测未生效,fallback 默认解析 → `controller not exists: app\api\controller\Aikefu`
|
||
- addon 模式 URL (`/aikefu/api/upload/image`) 经 InitRoute 的 addon 分支 (L131-162) 自动切 namespace 到 `addon\aikefu\api`,L187 简写 `Route::rule($pathinfo, 'api/upload/image')` 即可 dispatch 到正确 controller
|
||
- 历史教训: 不要在前端用 `'/api/aikefu/upload/...'` 这种不规范路径,即使后端有显式路由注册也会被 thinkphp 路由匹配漏掉
|
||
- nginx 已加 `merge_slashes on;` 兜底前端拼 baseURL+path 时的双斜杠 (`//aikefu/api/upload/image` → `/aikefu/api/upload/image`)
|
||
|
||
**注意 info 端点路径差异 (与上传端点不一致):**
|
||
|
||
- aiagent: `/api/aiagent/info` (Chat.php 的 info() 方法)
|
||
- aikefu: **`/api/kefu/info`** (app\api\controller\Kefu.php:182, 触发 `KefuGetInfo` event), **不是** `/api/aikefu/kefu/getInfo` 也不是 `/api/aikefu/info`
|
||
|
||
> **关键**: aikefu 端的 info 端点挂在**主项目** `/api/kefu/info` (不进 addon 路由), 是历史遗留. 调上传时 `Config.baseUrl + '/api/kefu/info'` 即可, 别拼 addon 名前缀. addon's `getInfo` 事件不是 REST 端点, 是 ThinkPHP 事件总线调用, 仅后端内部用.
|
||
|
||
前端 `aiService.getApiPrefix()` 当前返回 `'/api/kefu'` 或 `'/api/aiagent'` (用于 chat/info/history 端点),**不**用于上传端点 — 上传端点必须直接拼 addon 模式 URL `'/aikefu/api/upload/<action>'` 或 `'/aiagent/api/upload/<action>'`。
|
||
|
||
**关键决策点 (按 agent 配置 `upload_strategy`):**
|
||
|
||
> 本表 `upload_strategy` 内部代号 vs 后台 UI 商家看到的文字:
|
||
>
|
||
> | 后端代号 | 后台 UI 商家看到的文字 |
|
||
> |---|---|
|
||
> | `backend` | 走我们服务器传 (推荐, 默认) |
|
||
> | `oss_sts` | 走云存储传 (高级, 大视频用) |
|
||
> | `auto` | 智能选 (小图走我们, 大文件走云存储) |
|
||
>
|
||
> 前端代码按后端代号 (`backend` / `oss_sts` / `auto`) 读, 不要把 UI 文字写死.
|
||
>
|
||
> **oss_sts 走哪家云** = `media_policy.oss_provider` 字段,可选 12 个 provider (alioss/qiniu/txcos/huaweiobs/baidubos/qingstor/jdoss/volcengine/ucloud/wyoss/awsoss/minio);空串 = 后端按 OssProviderRegistry 自动选. PR-OSS-REGISTRY (2026-09-15) 后已是 12 provider,不再仅阿里云.
|
||
|
||
| 后端代号 (UI 别名) | image ≤2MB | image >2MB | audio/video/file |
|
||
|---|---|---|---|
|
||
| `backend` (走我们服务器) | 走 PHP 中转 (默认, 小白选) | 走 PHP 中转 (后端再转云存储) | 走 PHP 中转 |
|
||
| `oss_sts` (走云存储) | **强制云存储直传** | **强制云存储直传** | **强制云存储直传** |
|
||
| `auto` (智能选) | 走 PHP 中转 (省事) | 云存储直传 (性能) | 云存储直传 |
|
||
|
||
**核心结论: 前端先 `POST /info` 拿 `upload_strategy`, 再决定走哪条路径。**
|
||
|
||
### 3.1.5 attachments.data 字段协议 (PR-WB-MP-INLINE-MEDIA 2026-09-16 新增)
|
||
|
||
> 这是**最容易被忽略但最重要的协议层细节**。背景: backend 模式上传的文件 URL 落在 `/upload/{site_id}/...`,**`/upload/` 路径 nginx 无鉴权**,把这种 URL 透传给 WB Agent 让它 fetch — 等于把"任意访客"的边界扩到 WB 服务端。base64 inline 是唯一不依赖 WB 服务端 fetch 公网 URL 的稳路径。
|
||
|
||
**attachments[] 单条 dict 协议 (PR-WB-MP-INLINE-MEDIA 之后):**
|
||
|
||
```jsonc
|
||
{
|
||
"type": "image", // image | audio | video | file
|
||
"url": "https://...", // 可选 (data 存在时不强求; 否则必填)
|
||
"data": "<base64 字节>", // ★ PR-WB-MP-INLINE-MEDIA 新增, JSON 上下文必为 base64
|
||
"encoding": "base64", // 可选, 显式标记 (默认 base64)
|
||
"mime": "image/png", // data 内联时必填 (ACP ContentBlock mimeType)
|
||
"size": 1024, // 原始字节 (WB 阈值 5MB 判断用, 后端计算 sizeof(base64_decode(data)) 兜底)
|
||
"name": "x.png"
|
||
}
|
||
```
|
||
|
||
**核心规则:**
|
||
|
||
| 场景 | 后端处理 (A 路 `WbAgentOsClient::buildMediaBlocks` / B 路 `WbAcpClient::buildMediaBlocks`) |
|
||
|---|---|
|
||
| `image` 或 `audio` + `data` + `size ≤ 5MB` (5242880 字节) | ACP ContentBlock `image`/`audio` base64 inline, **WB 完全不 fetch URL** |
|
||
| `data` 超限 (5MB) 或 `type` 不支持 inline (video/file) | fallback 到 `resource_link` (透传 URL 给 WB, 依赖 J9 服务端 fetch) |
|
||
| 仅 `url`,无 `data` | `WbAgentOsClient` 服务端自 curl (`fetchInline`) 转 base64 后 inline; `WbAcpClient` 直接 resource_link (J9 依赖) |
|
||
| 无 `url` 且无 `data` | drop + error_log 留 trace (避免静默丢) |
|
||
| URL 是 `javascript:` / 非 http(s) 协议 | drop (MediaGuard SSRF 闸门) |
|
||
|
||
**storage/transport 分离 (后端保证,前端不用关心):**
|
||
|
||
- `data` 字段**只在 process memory 流动**,**不会写 DB** (attachments 列存的是 url+meta, 不存 base64)
|
||
- 后端 `MessageMedia::buildFields()` 落库前**显式 `unset(data, encoding)`**
|
||
- 前端回显只需要 `url`,数据库无 base64 也不会被泄露
|
||
|
||
**协议层 A/B 对称 (后端 2026-09-16 修复):**
|
||
|
||
| 路径 | 旧行为 | 新行为 |
|
||
|---|---|---|
|
||
| A `WbAgentOsClient::buildMediaBlocks` (L2763) | `url` 空即**静默丢** | `data` 优先, `url` 兜底 |
|
||
| B `WbAcpClient::buildMediaBlocks` (L349-419) | 已认 `data` 字段 | 0 改动 |
|
||
| `MediaGuard::check` (L116-122) | `data` 字段被 SSRF 闸门拦掉 | `data` 短路放行 (无出站请求) |
|
||
|
||
**前端必须做的 (本期新增):**
|
||
|
||
1. `uni.uploadFile()` 上传成功后, 再调 `uni.getFileSystemManager().readFile({filePath, encoding:'base64'})` 读 base64
|
||
2. 把 `data` (base64) + `url` + `mime` + `size` + `name` 一起塞进 chat 入参 `attachments[]`
|
||
3. **大文件 (>5MB) 不要塞 `data`** — base64 会让 chat JSON 膨胀 ~4/3 倍, 走 URL 透传 (依赖 J9, 见下面 caveat)
|
||
4. H5 端用 `FileReader.readAsDataURL()` 拿 base64
|
||
|
||
**WB SDK 协议层约束 (详见 `wb-cloud-sdk-coverage.md`):**
|
||
|
||
| 编号 | 含义 | SDK 支持 | 我们怎么走 |
|
||
|---|---|---|---|
|
||
| **J7** | ACP ContentBlock union (image/audio/resource) 支持 base64 inline | ✅ session.py:59-65 | ≤5MB 文件走这条 |
|
||
| **J9** | WB 服务端是否 fetch 外部 URL | ❌ 不在 SDK (服务端策略) | >5MB / 非 image-audio 走这条 (兜底) |
|
||
| **J10** | WB `/uploads/*` 分片上传 API | ❌ grep 无 | **目前无解**, 等 WB 出官方 API 后再优化 |
|
||
|
||
**仍未覆盖的边界 (本期没办法):**
|
||
|
||
- >5MB 文件仍依赖 J9, 部署到云服务器后 WB 服务端 fetch `/upload/{site_id}/...` 不一定通 — 此时只能让 Agent 在前端提示用户 "请发小图"
|
||
- 想 100% 消除 J9 依赖, 等 WB 出 `/uploads/*` 分片上传 API (PR-J10, P2 阻塞)
|
||
|
||
|
||
|
||
### 3.2 现状 (PR 前)
|
||
|
||
`pages_tool/ai-chat/components/ai-chat-tools-panel.vue:76-94` 的 `selectImage()`:
|
||
|
||
```js
|
||
selectImage() {
|
||
uni.chooseImage({
|
||
count: 1,
|
||
sizeType: ['compressed'],
|
||
sourceType: ['album', 'camera'],
|
||
success: (res) => {
|
||
const imageMessage = {
|
||
id: ++this.messageId,
|
||
role: 'user',
|
||
type: 'image',
|
||
content: '',
|
||
imageUrl: res.tempFilePaths[0], // ★ 只存本地路径, 没真上传
|
||
timestamp: Date.now()
|
||
}
|
||
this.$emit('image-selected', imageMessage)
|
||
}
|
||
})
|
||
}
|
||
```
|
||
|
||
`selectFile()` 同样: 只拿到 `file.name / file.size`, 没 `wx.uploadFile`。
|
||
|
||
`ai-chat-message.vue` 渲染 `message.imageUrl` 时**直接显示本地路径**, AI 端永远看不到这张图。
|
||
|
||
**结论: 上传功能完全没接通, 用户发的图/文件 AI 端收到的 attachments 是空的。**
|
||
|
||
> 仓库里有 `common/js/util.js:1005-1031` 的 `upload_file_server` 是走 PHP 中转到 `/api/upload/...` (产品图/头像用), **不能直接借给 AI chat**, 因为 AI chat 需要带 `agent_id` / `config_uuid` 路由到对应 addon, 还要走 upload_strategy 决策。
|
||
|
||
### 3.3 目标行为
|
||
|
||
```
|
||
用户点 [图片] 按钮
|
||
↓
|
||
uni.chooseImage → 选图完成
|
||
↓
|
||
aiService.getServiceConfig() → 拿 upload_strategy (后端代号, 参见 §3.1 表)
|
||
↓
|
||
判断:
|
||
- upload_strategy=backend + size ≤2MB → POST /upload/image (走我们服务器)
|
||
- upload_strategy=oss_sts 或 auto+size>2MB → POST /upload/ossSts → uni.uploadFile 直传云存储
|
||
- audio/video/file → 一律 POST /upload/ossSts
|
||
↓
|
||
拿到 { url, mime, size, name } 后, **fileToBase64 拿 data 字段** (≤5MB 时), 发聊天时塞 attachments[] (见 §3.1.5)
|
||
```
|
||
|
||
### 3.4 代码示例
|
||
|
||
> **前置依赖 (PR 前必须先做)**: `common/js/ai-service.js` 当前**没有暴露**下面两个方法:
|
||
> - `getCurrentConfigUuid()` — 全仓 grep 0 命中, 不存在
|
||
> - `getUserId()` — `ai-service.js:83` 是 private module function, 不在 `export default {}` 里 (line 292)
|
||
>
|
||
> 本节示例假定已经加了 (PR 提交时一起加). 改动小, diff 如下:
|
||
>
|
||
> ```diff
|
||
> // common/js/ai-service.js
|
||
> + import Config from './config.js'
|
||
>
|
||
> // 已有 (line 83) — 私有的, 改成 public
|
||
> - function getUserId() { ... }
|
||
> + export function getUserId() { ... }
|
||
>
|
||
> // 新增 (插在 export default 之前)
|
||
> + export function getCurrentConfigUuid() {
|
||
> + return uni.getStorageSync('current_config_uuid') || '';
|
||
> + }
|
||
> ```
|
||
>
|
||
> 否则下面所有 `aiService.getCurrentConfigUuid?.()` 会拿到 `undefined` 静默走空, `aiService.getUserId?.()` 同理, 导致上传请求少字段被后端拒.
|
||
|
||
> **代码块术语速查 (后续 §3.4 出现的内部代号 vs 后台 UI 别名)**:
|
||
> - `uploadViaPhp()` / `PHP 中转` / `PHP passthrough` / `backend` = "走我们服务器" (商家看到的)
|
||
> - `uploadViaSts()` / `STS 直传` / `STS 凭证` / `oss_sts` = "走云存储直传" (商家看到的)
|
||
> - 端点路径 `/upload/ossSts` 后端真名是 `Upload::ossSts()`, 别名沿用 (历史叫法, 实际后端走 12 个云存储 provider 任选)
|
||
> - HTTP header `Authorization: STS <ak>:<sk>` 是**仅 8 个 4-field STS provider** (alioss/qiniu/txcos/baidubos/qingstor/jdoss/ucloud/wyoss) 的协议规范, 另外 4 个 single-URL provider (huaweiobs/volcengine/awsoss/minio) 走预签 URL, 不带 Authorization header — 详见 §3.4 uploadViaSts 内的 provider 分流注释
|
||
|
||
**Step 1: 新建 `common/js/media-upload.js`**
|
||
|
||
```js
|
||
// common/js/media-upload.js (新增)
|
||
import aiService from './ai-service.js'
|
||
import Config from './config.js'
|
||
|
||
const SIZE_LIMIT_PHP_PASSTHROUGH = 2 * 1024 * 1024 // 2MB
|
||
|
||
/**
|
||
* 上传图片 (type=image)
|
||
* @param {string} filePath - uni.chooseImage 返回的 tempFilePaths[0]
|
||
* @returns {Promise<{url, mime, size, name, kind, used_strategy}>}
|
||
*/
|
||
export async function uploadImage(filePath) {
|
||
const addon = aiService.getAddonName(); // 'aikefu' | 'aiagent'
|
||
const config = await aiService.getServiceConfig();
|
||
const strategy = config?.upload_strategy || 'backend';
|
||
|
||
// H5 端: >10MB 先压
|
||
let compressedPath = filePath;
|
||
// #ifdef H5
|
||
const fileSize = await getFileSize(filePath);
|
||
if (fileSize > 10 * 1024 * 1024) {
|
||
try {
|
||
const compressed = await uni.compressImage({
|
||
src: filePath,
|
||
quality: 80,
|
||
compressedWidth: 2048,
|
||
compressedHeight: 2048,
|
||
});
|
||
compressedPath = compressed.tempFilePath || filePath;
|
||
} catch (e) {
|
||
console.warn('[media-upload] compressImage failed, use original:', e);
|
||
}
|
||
}
|
||
// #endif
|
||
|
||
const useSts = strategy === 'oss_sts'
|
||
|| (strategy === 'auto' && (await getFileSize(compressedPath)) > SIZE_LIMIT_PHP_PASSTHROUGH);
|
||
|
||
if (useSts) {
|
||
return await uploadViaSts(compressedPath, 'image', addon, aiService.getAgentId(), config);
|
||
} else {
|
||
return await uploadViaPhp(compressedPath, 'image', addon, aiService.getAgentId());
|
||
}
|
||
}
|
||
|
||
/**
|
||
* 上传 audio / video / file (一律 STS)
|
||
*/
|
||
export async function uploadMedia(filePath, kind) {
|
||
if (!['audio', 'video', 'file'].includes(kind)) {
|
||
throw new Error(`invalid kind: ${kind}`);
|
||
}
|
||
const addon = aiService.getAddonName();
|
||
const config = await aiService.getServiceConfig();
|
||
return await uploadViaSts(filePath, kind, addon, aiService.getAgentId(), config);
|
||
}
|
||
|
||
/**
|
||
* PHP 中转 (multipart/form-data)
|
||
* 端点必须用 addon 模式 URL: /<addon>/api/upload/<image|file>
|
||
* (不能用 /api/<addon>/upload/<...>, InitRoute 在 system_array 命中分支不切 namespace)
|
||
*/
|
||
async function uploadViaPhp(filePath, kind, addon, agentId) {
|
||
const url = (addon === 'aikefu' ? '/aikefu/api/upload/' : '/aiagent/api/upload/')
|
||
+ (kind === 'image' ? 'image' : 'file');
|
||
|
||
// #ifdef MP-WEIXIN
|
||
return new Promise((resolve, reject) => {
|
||
uni.uploadFile({
|
||
url: Config.baseUrl + url,
|
||
filePath,
|
||
name: 'file',
|
||
formData: {
|
||
kind,
|
||
agent_id: addon === 'aiagent' ? agentId : undefined,
|
||
config_uuid: addon === 'aikefu' ? (aiService.getCurrentConfigUuid?.() || '') : undefined,
|
||
uniacid: aiService.getUniacid(),
|
||
user_id: aiService.getUserId?.() || '',
|
||
},
|
||
success: (res) => {
|
||
try {
|
||
const data = typeof res.data === 'string' ? JSON.parse(res.data) : res.data;
|
||
if (data.code === 0 || data.success) {
|
||
resolve({
|
||
url: data.data.url,
|
||
mime: data.data.mime,
|
||
size: data.data.size,
|
||
name: data.data.name,
|
||
kind: data.data.kind || kind,
|
||
used_strategy: data.data.used_strategy || 'backend',
|
||
oss_sts_required: data.data.oss_sts_required || false,
|
||
});
|
||
} else {
|
||
reject(new Error(data.msg || '上传失败'));
|
||
}
|
||
} catch (e) {
|
||
reject(new Error('解析响应失败: ' + e.message));
|
||
}
|
||
},
|
||
fail: (err) => reject(new Error('网络错误: ' + (err.errMsg || err.message))),
|
||
});
|
||
});
|
||
// #endif
|
||
|
||
// #ifdef H5
|
||
const form = new FormData();
|
||
const file = await pathToFile(filePath, kind);
|
||
form.append('file', file);
|
||
form.append('kind', kind);
|
||
if (addon === 'aiagent') form.append('agent_id', agentId);
|
||
else form.append('config_uuid', aiService.getCurrentConfigUuid?.() || '');
|
||
form.append('uniacid', aiService.getUniacid());
|
||
const res = await fetch(Config.baseUrl + url, {
|
||
method: 'POST',
|
||
headers: { 'token': store.state.token || '' },
|
||
body: form,
|
||
});
|
||
const data = await res.json();
|
||
if (data.code === 0 || data.success) {
|
||
return {
|
||
url: data.data.url,
|
||
mime: data.data.mime,
|
||
size: data.data.size,
|
||
name: data.data.name,
|
||
kind: data.data.kind || kind,
|
||
used_strategy: data.data.used_strategy || 'backend',
|
||
oss_sts_required: data.data.oss_sts_required || false,
|
||
};
|
||
}
|
||
throw new Error(data.msg || '上传失败');
|
||
// #endif
|
||
}
|
||
|
||
/**
|
||
* STS 直传 (拿临时凭证 → 直传云存储)
|
||
*
|
||
* PR-OSS-REGISTRY (2026-09-15) 后端支持 12 个 provider, 协议层分两类:
|
||
* A) 4-field STS (8 个: alioss/qiniu/txcos/baidubos/qingstor/jdoss/ucloud/wyoss)
|
||
* → 前端拼 PUT URL, Authorization header = STS <ak>:<sk>
|
||
* B) single-URL (4 个: huaweiobs/volcengine/awsoss/minio)
|
||
* → sts.accessKeyId 已是完整预签 PUT URL, sts.securityToken='<PROVIDER>-SIGNED' 标记, 不带 Authorization
|
||
*
|
||
* 后端 stsResp.data.data 可能字段:
|
||
* { provider, bucket, region, accessKeyId, accessKeySecret, securityToken,
|
||
* key_prefix_tpl, site_id, host? (single-URL 模式), endpoint? (qiniu 7 参兼容) }
|
||
*/
|
||
async function uploadViaSts(filePath, kind, addon, agentId, config) {
|
||
// 1. 拿 STS 凭证 (addon 模式 URL)
|
||
const stsUrl = (addon === 'aikefu' ? '/aikefu/api/upload/' : '/aiagent/api/upload/') + 'ossSts';
|
||
const stsResp = await uni.request({
|
||
url: Config.baseUrl + stsUrl,
|
||
method: 'POST',
|
||
header: { 'Content-Type': 'application/json', 'token': store.state.token || '' },
|
||
data: {
|
||
agent_id: addon === 'aiagent' ? agentId : undefined,
|
||
config_uuid: addon === 'aikefu' ? (aiService.getCurrentConfigUuid?.() || '') : undefined,
|
||
uniacid: aiService.getUniacid(),
|
||
user_id: aiService.getUserId?.() || '',
|
||
},
|
||
});
|
||
if (stsResp.statusCode !== 200 || (stsResp.data.code !== 0 && !stsResp.data.success)) {
|
||
throw new Error(stsResp.data?.msg || 'STS 凭证获取失败');
|
||
}
|
||
const sts = stsResp.data.data;
|
||
const provider = sts.provider || 'alioss'; // 老后端默认 alioss
|
||
|
||
// 2. 拼 PUT URL (用后端返回的 key_prefix_tpl 替换占位符)
|
||
const ext = (filePath.split('.').pop() || 'bin').toLowerCase();
|
||
const uuid = 'xxxxxxxxxxxx4xxxyxxxxxxxxxxxxxxx'.replace(/[xy]/g, c => {
|
||
const r = Math.random() * 16 | 0;
|
||
return (c === 'x' ? r : (r & 0x3 | 0x8)).toString(16);
|
||
});
|
||
const now = new Date();
|
||
const key = sts.key_prefix_tpl
|
||
.replace('{site_id}', sts.site_id || '0')
|
||
.replace('{agent_id}', agentId || '0')
|
||
.replace('{yyyy}', now.getFullYear().toString())
|
||
.replace('{mm}', String(now.getMonth() + 1).padStart(2, '0'))
|
||
.replace('{dd}', String(now.getDate()).padStart(2, '0'))
|
||
.replace('{rand}', uuid)
|
||
+ '.' + ext;
|
||
|
||
// ★ PR-OSS-REGISTRY (2026-09-15): provider 分流
|
||
// single-URL 模式: securityToken 末尾是 -SIGNED (OBS-SIGNED / TOS-SIGNED / AWS-SIGNED / MINIO-SIGNED)
|
||
const isSingleUrlMode = typeof sts.securityToken === 'string'
|
||
&& /-(SIGNED)$/.test(sts.securityToken);
|
||
|
||
let ossUrl, headers;
|
||
if (isSingleUrlMode) {
|
||
// B) single-URL: sts.accessKeyId 是完整预签 PUT URL, 不需要 Authorization header
|
||
ossUrl = sts.accessKeyId;
|
||
headers = { 'Content-Type': guessMime(ext) };
|
||
} else if (provider === 'qiniu') {
|
||
// A) qiniu 兼容模式: sts.accessKeyId = uptoken, endpoint 来自 sts.endpoint 或 sts.host
|
||
ossUrl = (sts.endpoint || sts.host || `https://upload-${sts.region}.qiniup.com`) + '/' + key;
|
||
headers = {
|
||
'Authorization': 'UpToken ' + sts.accessKeyId,
|
||
'Content-Type': guessMime(ext),
|
||
};
|
||
} else {
|
||
// A) 4-field STS 默认 (alioss/txcos/baidubos/qingstor/jdoss/ucloud/wyoss)
|
||
// endpoint 模板按 provider 区分, 后端未返 endpoint 时按 OSS 模板拼
|
||
const tpl = (provider === 'alioss') ? `https://${sts.bucket}.${sts.region}.aliyuncs.com`
|
||
: (provider === 'txcos') ? `https://${sts.bucket}-${sts.region}.cos.${sts.region}.myqcloud.com`
|
||
: (provider === 'baidubos') ? `https://${sts.bucket}.${sts.region}.bcebos.com`
|
||
: (provider === 'jdoss') ? `https://${sts.bucket}.${sts.region}.jdcloud-oss.com`
|
||
: (provider === 'wyoss') ? `https://${sts.bucket}.${sts.host || sts.region}`
|
||
: (sts.endpoint ? sts.endpoint.replace('{bucket}', sts.bucket) : `https://${sts.bucket}.${sts.region}`);
|
||
ossUrl = `${tpl}/${key}`;
|
||
headers = {
|
||
'Authorization': `STS ${sts.accessKeyId}:${sts.accessKeySecret}`,
|
||
'x-oss-security-token': sts.securityToken,
|
||
'Content-Type': guessMime(ext),
|
||
};
|
||
}
|
||
|
||
// 3. 直传云存储
|
||
// #ifdef MP-WEIXIN
|
||
await new Promise((resolve, reject) => {
|
||
uni.uploadFile({
|
||
url: ossUrl,
|
||
filePath,
|
||
name: 'file',
|
||
formData: {},
|
||
header: headers,
|
||
success: (res) => {
|
||
if (res.statusCode >= 200 && res.statusCode < 300) resolve();
|
||
else reject(new Error(`${provider} PUT ${res.statusCode}`));
|
||
},
|
||
fail: reject,
|
||
});
|
||
});
|
||
// #endif
|
||
// #ifdef H5
|
||
const file = await pathToFile(filePath, kind);
|
||
const ossRes = await fetch(ossUrl, {
|
||
method: 'PUT',
|
||
headers,
|
||
body: file,
|
||
});
|
||
if (!ossRes.ok) throw new Error(`${provider} PUT ${ossRes.status}`);
|
||
// #endif
|
||
|
||
// 4. 返回公网 URL
|
||
return {
|
||
url: ossUrl.split('?')[0], // single-URL 模式 URL 带 ?<签名参数>, 拼 chat attachment 用 base URL
|
||
mime: guessMime(ext),
|
||
size: file.size || 0,
|
||
name: key.split('/').pop(),
|
||
kind,
|
||
used_strategy: 'oss_sts',
|
||
provider,
|
||
};
|
||
}
|
||
|
||
async function getFileSize(filePath) {
|
||
// #ifdef MP-WEIXIN
|
||
return new Promise((resolve) => {
|
||
uni.getFileInfo({
|
||
filePath,
|
||
success: (res) => resolve(res.size),
|
||
fail: () => resolve(0),
|
||
});
|
||
});
|
||
// #endif
|
||
// #ifdef H5
|
||
try {
|
||
const res = await fetch(filePath);
|
||
const blob = await res.blob();
|
||
return blob.size;
|
||
} catch (e) { return 0; }
|
||
// #endif
|
||
}
|
||
|
||
async function pathToFile(filePath, kind) {
|
||
// #ifdef H5
|
||
const res = await fetch(filePath);
|
||
const blob = await res.blob();
|
||
return new File([blob], filePath.split('/').pop() || `${kind}.bin`, { type: blob.type });
|
||
// #endif
|
||
}
|
||
|
||
function guessMime(ext) {
|
||
const map = {
|
||
jpg: 'image/jpeg', jpeg: 'image/jpeg', png: 'image/png', gif: 'image/gif', webp: 'image/webp',
|
||
mp3: 'audio/mpeg', wav: 'audio/wav', aac: 'audio/aac',
|
||
mp4: 'video/mp4', mov: 'video/quicktime',
|
||
pdf: 'application/pdf', txt: 'text/plain',
|
||
};
|
||
return map[ext] || 'application/octet-stream';
|
||
}
|
||
|
||
export default { uploadImage, uploadMedia };
|
||
```
|
||
|
||
**Step 2: 改造 `selectImage()`**
|
||
|
||
```vue
|
||
<!-- pages_tool/ai-chat/components/ai-chat-tools-panel.vue -->
|
||
<script>
|
||
import { uploadImage, uploadMedia } from '@/common/js/media-upload.js'
|
||
|
||
methods: {
|
||
async selectImage() {
|
||
uni.chooseImage({
|
||
count: 9, // 改: 允许多张, 上限由后端 max_attachments_per_msg 控制
|
||
sizeType: ['compressed'],
|
||
sourceType: ['album', 'camera'],
|
||
success: async (res) => {
|
||
this.hideToolsPanel()
|
||
// 1. 先把所有图显示为 "上传中" 的占位消息
|
||
const pendingMessages = res.tempFilePaths.map((p, i) => ({
|
||
id: ++this.messageId,
|
||
role: 'user',
|
||
type: 'image',
|
||
content: '',
|
||
imageUrl: p,
|
||
uploading: true,
|
||
uploadError: null,
|
||
uploaded: null,
|
||
timestamp: Date.now() + i,
|
||
}));
|
||
this.$emit('images-selected', pendingMessages);
|
||
|
||
// 2. 并发上传 (但同一时间最多 3 个, 防止 wx.uploadFile 并发上限)
|
||
const queue = [...pendingMessages];
|
||
const CONCURRENCY = 3;
|
||
await Promise.all(Array(CONCURRENCY).fill().map(async () => {
|
||
while (queue.length) {
|
||
const msg = queue.shift();
|
||
try {
|
||
const uploaded = await uploadImage(msg.imageUrl);
|
||
msg.uploading = false;
|
||
msg.uploaded = uploaded;
|
||
msg.imageUrl = uploaded.url;
|
||
} catch (e) {
|
||
msg.uploading = false;
|
||
msg.uploadError = e.message;
|
||
}
|
||
}
|
||
}));
|
||
|
||
this.$emit('images-uploaded', pendingMessages);
|
||
},
|
||
fail: (err) => {
|
||
if (err.errMsg && !err.errMsg.includes('cancel')) {
|
||
uni.showToast({ title: '选图失败', icon: 'none' });
|
||
}
|
||
}
|
||
});
|
||
},
|
||
|
||
async selectFile() {
|
||
// #ifdef H5
|
||
const input = document.createElement('input');
|
||
input.type = 'file';
|
||
input.accept = '*/*';
|
||
input.onchange = async (e) => {
|
||
const file = e.target.files[0];
|
||
if (!file) return;
|
||
this.hideToolsPanel();
|
||
await this._uploadOneFile(file, file.type.startsWith('image/') ? 'image' : 'file');
|
||
};
|
||
input.click();
|
||
// #endif
|
||
// #ifdef APP-PLUS
|
||
uni.chooseFile({
|
||
success: async (res) => {
|
||
this.hideToolsPanel();
|
||
await this._uploadOneFile(res.tempFiles[0], 'file');
|
||
}
|
||
});
|
||
// #endif
|
||
},
|
||
|
||
async _uploadOneFile(file, kind) {
|
||
const pending = {
|
||
id: ++this.messageId,
|
||
role: 'user',
|
||
type: kind,
|
||
content: '',
|
||
fileName: file.name || (kind + '.bin'),
|
||
fileSize: file.size,
|
||
fileType: file.type,
|
||
imageUrl: file.path || (file.name ? URL.createObjectURL(file) : ''),
|
||
uploading: true,
|
||
uploadError: null,
|
||
uploaded: null,
|
||
timestamp: Date.now(),
|
||
};
|
||
this.$emit('file-selected', pending);
|
||
|
||
try {
|
||
const uploaded = kind === 'image'
|
||
? await uploadImage(file.path || file)
|
||
: await uploadMedia(file.path || file, kind);
|
||
pending.uploading = false;
|
||
pending.uploaded = uploaded;
|
||
if (uploaded.url) pending.imageUrl = uploaded.url;
|
||
this.$emit('file-uploaded', pending);
|
||
} catch (e) {
|
||
pending.uploading = false;
|
||
pending.uploadError = e.message;
|
||
uni.showToast({ title: '上传失败: ' + e.message, icon: 'none' });
|
||
}
|
||
}
|
||
}
|
||
</script>
|
||
```
|
||
|
||
**Step 3: 发送消息时携带 attachments (含 `data` 字段, PR-WB-MP-INLINE-MEDIA)**
|
||
|
||
```js
|
||
// ai-chat-message.vue sendMessage()
|
||
async sendMessage() {
|
||
const hasContent = this.inputText?.trim() !== '';
|
||
const readyAttachments = this.messages
|
||
.filter(m => m.role === 'user' && m.uploaded && !m.sendWithBatch)
|
||
.map(m => {
|
||
m.sendWithBatch = true;
|
||
const a = {
|
||
type: m.uploaded.kind,
|
||
url: m.uploaded.url,
|
||
mime: m.uploaded.mime,
|
||
name: m.uploaded.name,
|
||
size: m.uploaded.size,
|
||
};
|
||
// ★ PR-WB-MP-INLINE-MEDIA: ≤5MB 图片/音频塞 base64, 让 WB ContentBlock 内联
|
||
// >5MB 不要塞 data (避免 chat JSON 膨胀 ~4/3 倍, 走 URL 透传依赖 J9)
|
||
if ((a.type === 'image' || a.type === 'audio')
|
||
&& a.size > 0 && a.size <= 5 * 1024 * 1024
|
||
&& m.uploaded.base64) {
|
||
a.data = m.uploaded.base64;
|
||
a.encoding = 'base64';
|
||
}
|
||
return a;
|
||
});
|
||
|
||
if (!hasContent && !readyAttachments.length) return;
|
||
|
||
const queryText = this.inputText?.trim() || (readyAttachments.length ? '[附件]' : '');
|
||
|
||
await this._doSendStream(queryText, readyAttachments);
|
||
}
|
||
|
||
async _doSendStream(queryText, attachments) {
|
||
// 改造后的 sendStreamMessage 接 attachments 参数
|
||
await this.sendStreamMessage(queryText, attachments);
|
||
}
|
||
```
|
||
|
||
**Step 4: 上传成功后 fileToBase64 拿 data 字段**
|
||
|
||
`m.uploaded.base64` 字段需要在 §3.4 Step 2 `_uploadOneFile()` 拿到上传结果后补:
|
||
|
||
```js
|
||
// ai-chat-message.vue / ai-chat-tools-panel.vue _uploadOneFile 改造
|
||
async _uploadOneFile(file, kind) {
|
||
// ... 原有 pending 占位代码 ...
|
||
|
||
try {
|
||
const uploaded = kind === 'image'
|
||
? await uploadImage(file.path || file)
|
||
: await uploadMedia(file.path || file, kind);
|
||
pending.uploading = false;
|
||
pending.uploaded = uploaded;
|
||
if (uploaded.url) pending.imageUrl = uploaded.url;
|
||
|
||
// ★ PR-WB-MP-INLINE-MEDIA: 上传成功后, ≤5MB 文件再读 base64 塞进 uploaded
|
||
if ((kind === 'image' || kind === 'audio')
|
||
&& uploaded.size > 0 && uploaded.size <= 5 * 1024 * 1024) {
|
||
try {
|
||
pending.uploaded.base64 = await readFileAsBase64(file.path || file);
|
||
} catch (e) {
|
||
console.warn('[ai-chat] fileToBase64 失败, 后端会走 URL 透传 (依赖 J9):', e);
|
||
// 不阻断, 大不了降级到 J9
|
||
}
|
||
}
|
||
|
||
this.$emit('file-uploaded', pending);
|
||
} catch (e) {
|
||
// ... 原有错误处理 ...
|
||
}
|
||
},
|
||
|
||
async function readFileAsBase64(filePath) {
|
||
// #ifdef MP-WEIXIN
|
||
return new Promise((resolve, reject) => {
|
||
uni.getFileSystemManager().readFile({
|
||
filePath,
|
||
encoding: 'base64',
|
||
success: (res) => resolve(res.data),
|
||
fail: reject,
|
||
});
|
||
});
|
||
// #endif
|
||
// #ifdef H5
|
||
const res = await fetch(filePath);
|
||
const blob = await res.blob();
|
||
return await new Promise((resolve, reject) => {
|
||
const reader = new FileReader();
|
||
reader.onload = () => resolve(reader.result.split(',')[1]); // 去掉 "data:<mime>;base64," 前缀
|
||
reader.onerror = reject;
|
||
reader.readAsDataURL(blob);
|
||
});
|
||
// #endif
|
||
// #ifdef APP-PLUS / MP-HARMONY
|
||
return new Promise((resolve, reject) => {
|
||
plus.io.resolveLocalFileSystemURL(filePath, (entry) => {
|
||
entry.file((f) => {
|
||
const reader = new plus.io.FileReader();
|
||
reader.onload = (e) => resolve(e.target.result.split(',')[1]);
|
||
reader.onerror = reject;
|
||
reader.readAsDataURL(f);
|
||
}, reject);
|
||
}, reject);
|
||
});
|
||
// #endif
|
||
}
|
||
```
|
||
|
||
**为什么必须 fileToBase64**: 后端 upload 接口只返 `{url, mime, size, name}`,**不含 base64 字节**。base64 必须在上传后**前端读本地文件**重新拿, 然后跟 url 一起塞 chat attachment。
|
||
|
||
`sendStreamMessage` 在 §1.4 已改造接 `attachments`, 无需再改。
|
||
|
||
### 3.5 下载 / 预览
|
||
|
||
OSS 直传后返回的 `url` 是公网 URL, 可直接 `<image>` / `<video>` / 浏览器下载。
|
||
|
||
```js
|
||
previewAttachment(att) {
|
||
if (att.type === 'image') {
|
||
uni.previewImage({ urls: [att.url], current: att.url });
|
||
} else if (att.type === 'video') {
|
||
uni.navigateTo({ url: '/pages_tool/file-preview/file-preview?url=' + encodeURIComponent(att.url) });
|
||
} else {
|
||
// file: 用浏览器下载 (H5) 或 wx.openDocument (MP)
|
||
// #ifdef H5
|
||
window.open(att.url, '_blank');
|
||
// #endif
|
||
// #ifdef MP-WEIXIN
|
||
uni.downloadFile({
|
||
url: att.url,
|
||
success: (res) => {
|
||
wx.openDocument({
|
||
filePath: res.tempFilePath,
|
||
success: () => console.log('打开文档成功'),
|
||
});
|
||
}
|
||
});
|
||
// #endif
|
||
}
|
||
}
|
||
```
|
||
|
||
### 3.6 关键注意点
|
||
|
||
1. **`upload_strategy` 必须每次拿**: 后端配置, 不要硬编码或缓存; 调 `getServiceConfig()` 拿最新值
|
||
2. **STS 凭证**有时效 (默认 3600s), 不要缓存 — 每次上传前重新拿
|
||
3. **wx.uploadFile 并发上限 3**: 微信小程序同时只能 3 个 uploadFile, 代码里 `CONCURRENCY=3` 限制
|
||
4. **H5 端跨域**: 云存储 bucket 必须配 CORS 允许前端域名 (`https://your-h5.com`), 否则直传失败
|
||
5. **大图先压**: H5 端 >10MB 图片先 `uni.compressImage({quality:80})`
|
||
6. **失败重试**: 上传失败要给用户 "重试" 按钮, 不要直接丢消息
|
||
7. **`oss_sts_required: true`**: 后端 image/file 响应里的字段, 提示 "下次大文件应走 STS", 前端据此在下一次主动选 STS
|
||
8. **后端 attachments 持久化 vs 流式**: aiagent HTTP 路径下, attachments 只存 DB 不进 SSE 流, 历史消息渲染时从 DB 读 (`getConversationHistory` 接口返的 attachments[]); aikefu HTTP 路径 attachments 进 SSE 流
|
||
9. **视频抽帧**: 后端 `video_mode=frame_analysis` 会预抽帧, 但前端 attachments 仍只发原视频 URL, 后端在 `WbAgentOsClient::buildMediaBlocks` 里处理
|
||
10. **★ attachments.data 字段是 ≤5MB 文件 100% 可靠的路径** (PR-WB-MP-INLINE-MEDIA 2026-09-16): backend 模式 URL 落在 `/upload/{site_id}/...`, **`/upload/` 路径 nginx 无鉴权**, 部署到云服务器后 WB 服务端 (J9) fetch 这个 URL **不一定通** (取决于 WB 后台策略, 不在 SDK 范围). **前端必须 fileToBase64 后塞 `data` 字段**, 让 ACP ContentBlock 直接 base64 内联, 跳过 J9. >5MB 才走 URL 透传 (兜底依赖 J9). 详见 §3.1.5
|
||
11. **12 个云存储 provider 协议层差异** (PR-OSS-REGISTRY 2026-09-15): 8 个走 4-field STS (alioss/qiniu/txcos/baidubos/qingstor/jdoss/ucloud/wyoss), 4 个走 single-URL 预签 (huaweiobs/volcengine/awsoss/minio). `sts.securityToken` 末尾是 `-SIGNED` 标记 single-URL 模式; 该模式下 `sts.accessKeyId` 是完整预签 PUT URL, **不要拼 Authorization header**. 详见 `media-upload.js` `uploadViaSts()` 实现
|
||
12. **backend 模式下 backend 路径前缀**: 后端 `buildPublicUrl` 用 `request()->domain()`, nginx 反代后**必须配 `Host` 头透传** (默认 `proxy_set_header Host $host;`), 否则预览 URL host 错. 跟 nginx 鉴权无关 — 仅 URL host 拼对
|
||
|
||
### 3.7 aikefu vs aiagent 端点区别
|
||
|
||
aikefu 端所有 upload 端点额外需要 `config_uuid` (不是 `agent_id`):
|
||
|
||
```js
|
||
// aikefu 端
|
||
{
|
||
config_uuid: 'xxxxx', // 必填
|
||
file: ...
|
||
}
|
||
|
||
// aiagent 端
|
||
{
|
||
agent_id: 'agent_xxx', // 必填
|
||
file: ...
|
||
}
|
||
```
|
||
|
||
前端代码通过 `aiService.getAddonName()` 区分, 在 `uploadViaPhp()` / `uploadViaSts()` 里自动加对应字段 (见 Step 1)。
|
||
|
||
---
|
||
|
||
## 4. 多轮会话
|
||
|
||
### 4.1 后端协议
|
||
|
||
`conversation_id` 是多轮会话的唯一标识:
|
||
|
||
- 第一条消息**不带** `conversation_id` → 后端新建, 流结束 (`message_end`) 时返回
|
||
- 后续消息带 `conversation_id` → 后端查历史, 在该会话上下文中继续
|
||
- 后端 `WB Agent` 端在 service-side session 维护历史, 本端**默认** `history_mode=off` (PR-WB-RUNTIME-CONTEXT 默认值, 见 `src/addon/aiagent/lib/WbAgentOsClient.php`)
|
||
|
||
**关键设计决策 (来自后端 memory `feedback_wb_default_values_rationale.md`):**
|
||
|
||
| 字段 | 默认 | 原因 |
|
||
|---|---|---|
|
||
| `history_mode` | `off` | 服务端 session 已含历史, 本端重发会重复 |
|
||
| `source_app` | `newshop` | 项目标识, 不是平台身份 |
|
||
| `buyer_ref` | guest 标记 | 不存 PII |
|
||
|
||
**前端不需要理解这个设计**, 只需保证:
|
||
|
||
1. 把后端返回的 `conversation_id` 存到本地 (持久化)
|
||
2. 下次发消息时带回去
|
||
3. 用户主动 "新对话" 时清掉 (调 `clear-conversation` 端点)
|
||
4. **per-agent 隔离** (本文档新增): 不同 agent 的 conversation_id 单独存
|
||
|
||
### 4.2 现状 (PR 前)
|
||
|
||
`ai-service.js` 已经做对了 70%:
|
||
|
||
- L6 `let currentConversationId = null` 内存变量
|
||
- L69-76 启动时从 storage 读 (`ai_conversation_id` key)
|
||
- L218-224 WS 收到 `conversation_id` 时持久化
|
||
- L529-557 `getConversationId / setConversationId / clearConversationId` 方法齐
|
||
- L646-662 `clearConversation()` 调后端 `clear-conversation` 端点
|
||
- L668-708 `getConversationHistory()` 拉历史消息
|
||
|
||
**漏的点:**
|
||
|
||
1. ❌ 单 key `ai_conversation_id` 不区分 agent: 用户在智能客服 + 知识库两个 agent 之间切换, conversation_id 串
|
||
2. ❌ 没有"显示历史消息"入口 — 用户切回页面看不到上次聊到哪
|
||
3. ❌ 多模态 attachments 不参与 conversation 上下文 (见 §3)
|
||
4. ❌ 用户主动 "清空对话" 时, 没调 `clearConversation()`, 只调 `clearConversationId()` (前端状态), 服务端历史还在
|
||
5. ❌ 切 agent 时没 reset conversation_id, 旧 agent 的 id 误塞给新 agent
|
||
|
||
### 4.3 目标行为
|
||
|
||
| 场景 | 行为 |
|
||
|---|---|
|
||
| 首次进入 | 不带 conversation_id, 后端新建, 存到 storage |
|
||
| 继续聊 | 带 storage 里的 conversation_id, 上下文接续 |
|
||
| 退出页面 | conversation_id 保持, 下次进入自动恢复 |
|
||
| 主动 "新对话" | 调 `clearConversation()` + `clearConversationId()`, 不带 conversation_id 发下一条 |
|
||
| 切换 agent | 旧 agent 的 conversation_id 单独存 (`ai_conversation_id_<agent_id>`), 切回时恢复 |
|
||
| 历史消息 | 进入页面时调 `getConversationHistory()` 渲染滚动区 |
|
||
|
||
### 4.4 代码示例
|
||
|
||
**Step 1: ai-service 加 per-agent conversation 隔离**
|
||
|
||
```js
|
||
// common/js/ai-service.js
|
||
// 替换原 CONVERSATION_KEY 单 key 模式
|
||
const CONVERSATION_KEY_PREFIX = 'ai_conversation_id_';
|
||
|
||
function getConversationKey(agentId) {
|
||
return CONVERSATION_KEY_PREFIX + (agentId || DEFAULT_AGENT_ID);
|
||
}
|
||
|
||
// 替换 L68-76 的初始化逻辑
|
||
try {
|
||
const saved = uni.getStorageSync(getConversationKey(currentAgentId));
|
||
if (saved) currentConversationId = saved;
|
||
} catch (e) {
|
||
console.warn('读取会话ID失败:', e);
|
||
}
|
||
|
||
// 改 setConversationId / clearConversationId
|
||
setConversationId(id) {
|
||
if (id && id !== currentConversationId) {
|
||
currentConversationId = id;
|
||
try {
|
||
uni.setStorageSync(getConversationKey(currentAgentId), id);
|
||
} catch (e) { console.error('保存会话ID失败:', e); }
|
||
}
|
||
},
|
||
|
||
clearConversationId() {
|
||
currentConversationId = null;
|
||
try {
|
||
uni.removeStorageSync(getConversationKey(currentAgentId));
|
||
} catch (e) { console.error('清除会话ID失败:', e); }
|
||
},
|
||
|
||
// 加 getAllConversations (用于"会话列表"功能, 可选)
|
||
getAllConversations() {
|
||
const out = {};
|
||
try {
|
||
const info = uni.getStorageInfoSync();
|
||
info.keys.filter(k => k.startsWith(CONVERSATION_KEY_PREFIX))
|
||
.forEach(k => {
|
||
const agentId = k.substring(CONVERSATION_KEY_PREFIX.length);
|
||
out[agentId] = uni.getStorageSync(k);
|
||
});
|
||
} catch (e) { /* ignore */ }
|
||
return out;
|
||
},
|
||
|
||
// 改 setAgentId, 切完自动恢复
|
||
setAgentId(agentId) {
|
||
var oldAddon = resolveAddonName(currentAgentId);
|
||
setAgentId(agentId);
|
||
var newAddon = resolveAddonName(currentAgentId);
|
||
if (oldAddon !== newAddon) {
|
||
try { disconnectWebSocket(); } catch (e) {}
|
||
}
|
||
// ★ 切完 agent, 重新读该 agent 的 conversation_id
|
||
currentConversationId = null;
|
||
try {
|
||
const saved = uni.getStorageSync(getConversationKey(agentId));
|
||
if (saved) currentConversationId = saved;
|
||
} catch (e) {}
|
||
}
|
||
```
|
||
|
||
**Step 2: ai-chat-message.vue 进入页面拉历史**
|
||
|
||
```js
|
||
// pages_tool/ai-chat/ai-chat-message.vue
|
||
async loadHistory() {
|
||
const cid = aiService.getConversationId();
|
||
if (!cid) return;
|
||
try {
|
||
const hist = await aiService.getConversationHistory({
|
||
conversation_id: cid,
|
||
limit: 20,
|
||
});
|
||
if (hist.success && hist.messages.length) {
|
||
this.messages = hist.messages.reverse().map(m => this._normalizeHistoryMessage(m));
|
||
this.scrollToBottom();
|
||
}
|
||
} catch (e) {
|
||
console.warn('[AI Chat] 加载历史失败 (不影响新对话):', e);
|
||
}
|
||
},
|
||
|
||
_normalizeHistoryMessage(m) {
|
||
return {
|
||
id: m.message_id || ++this.messageId,
|
||
role: m.role === 'user' ? 'user' : 'ai',
|
||
type: m.attachments?.length ? (m.attachments[0].type || 'file') : 'text',
|
||
content: m.content || m.answer || '',
|
||
imageUrl: m.attachments?.[0]?.url || '',
|
||
attachments: m.attachments || [],
|
||
timestamp: (m.created_at || m.create_time || m.time) * 1000 || Date.now(),
|
||
// 历史消息没有思考流 / 选项, 不渲染
|
||
thoughtsText: '',
|
||
pendingQuestion: null,
|
||
toolCalls: m.tool_calls || [],
|
||
tokens: m.tokens || null,
|
||
};
|
||
}
|
||
|
||
onLoad(options) {
|
||
// 现有代码 ...
|
||
this.loadHistory();
|
||
}
|
||
```
|
||
|
||
**Step 3: 用户主动 "新对话" 按钮**
|
||
|
||
```vue
|
||
<!-- ai-chat-message.vue template, header 区 -->
|
||
<view class="new-chat-btn" @click="onNewChat">
|
||
<text>🔄 新对话</text>
|
||
</view>
|
||
|
||
<script>
|
||
async onNewChat() {
|
||
const cid = aiService.getConversationId();
|
||
if (cid) {
|
||
try {
|
||
await aiService.clearConversation(cid);
|
||
} catch (e) {
|
||
console.warn('[AI Chat] clearConversation 后端失败, 继续清前端:', e);
|
||
}
|
||
}
|
||
aiService.clearConversationId();
|
||
this.messages = [];
|
||
this.inputText = '';
|
||
uni.showToast({ title: '已开始新对话', icon: 'success' });
|
||
}
|
||
</script>
|
||
```
|
||
|
||
**Step 4: 跨设备同步 (可选, 高级)**
|
||
|
||
后端 `getConversationHistory` 接口支持跨设备读历史, 但前端默认只读 storage 里的 conversation_id。如果用户换手机或清缓存, 历史会丢。高级方案: 用 store 缓存历史消息。
|
||
|
||
```js
|
||
// store/index.js 加历史消息缓存
|
||
state: {
|
||
conversationHistory: {}, // { [conversation_id]: Message[] }
|
||
}
|
||
```
|
||
|
||
**本文档不展开, 留作 PR-AI-CHAT-HISTORY 后续优化。**
|
||
|
||
### 4.5 关键注意点
|
||
|
||
1. **不要在 query 里塞 history**: 本端 `history_mode=off`, 把整段历史塞 query 是冗余的, WB 服务端 session 已含历史
|
||
2. **conversation_id 持久化 key 必须 per-agent**: 一个前端用户可能跟多个 agent 聊 (智能客服 + 知识库), 不能共用一个 conversation_id
|
||
3. **H5 storage 是 localStorage, MP 是 wx.setStorageSync**: `uni.getStorageSync` 跨端兼容, 别直接用 `wx.*`
|
||
4. **历史消息无思考流/选项**: 后端存的是 `content` + `attachments`, 不存思考步骤、不存 pending_question; 历史消息里 `thoughtsText=''`, `pendingQuestion=null`
|
||
5. **`clearConversation` 后端真的删数据**: 不只是前端隐藏, 商家调试场景下, 调多次会清空正式用户历史, 加 dev mode 提示
|
||
6. **attachment 不参与历史上下文**: 当前 `getConversationHistory` 接口只返文本和 attachment 列表, 不参与 conversation session 续接 (后端用 server-side session)
|
||
|
||
### 4.6 调试清单
|
||
|
||
| 症状 | 排查 |
|
||
|---|---|
|
||
| 多轮上下文丢失 | `console.log` 看 `aiService.getConversationId()` 是否非空, storage 里 `ai_conversation_id_<agent>` 是否在 |
|
||
| 切 agent 后旧消息还在 | 检查 per-agent key 是否生效, `setAgentId` 里是否 reset `currentConversationId` |
|
||
| 后端 `message_end` 不返 conversation_id | 后端 bug, 看 `runtime/log/app_run_info/2026-09-14-wb-rpc.log` 里 `processSseEvent` 是否漏发 |
|
||
| "新对话" 后下一条还是上下文 | 检查 `clearConversation` 后端是否真删, 调 `getConversationHistory` 看还有没有 |
|
||
| 跨设备同步历史 | 当前不支持, 等 PR-AI-CHAT-HISTORY |
|
||
| H5 端 storage 不持久 | 浏览器隐私模式 / `localStorage` 被禁用都会清, 不算 bug |
|
||
|
||
---
|
||
|
||
## 5. 联调测试
|
||
|
||
### 5.1 后端 e2e (curl)
|
||
|
||
```bash
|
||
# 1. 健康检查
|
||
curl -X POST 'https://your-domain.com/api/aiagent/health'
|
||
|
||
# 2. 拿 agent 配置 (含 upload_strategy)
|
||
curl -X POST 'https://your-domain.com/api/aiagent/info' \
|
||
-H 'Content-Type: application/json' \
|
||
-d '{"agent_id":"agent_aikefu"}'
|
||
|
||
# 3. 流式聊天
|
||
curl -N 'https://your-domain.com/api/aiagent/chat' \
|
||
-H 'Content-Type: application/json' \
|
||
-H 'Accept: text/event-stream' \
|
||
-d '{
|
||
"agent_id":"agent_aikefu",
|
||
"query":"你好",
|
||
"response_mode":"streaming",
|
||
"user_id":"guest_test"
|
||
}'
|
||
|
||
# 4. 上传图片 (PHP 中转, addon 模式 URL)
|
||
curl -X POST 'https://your-domain.com/aiagent/api/upload/image' \
|
||
-F 'file=@/path/to/test.jpg' \
|
||
-F 'agent_id=agent_aikefu'
|
||
|
||
# 5. 拿 STS 凭证 (addon 模式 URL)
|
||
curl -X POST 'https://your-domain.com/aiagent/api/upload/ossSts' \
|
||
-H 'Content-Type: application/json' \
|
||
-d '{"agent_id":"agent_aikefu"}'
|
||
|
||
# 6. 拉历史
|
||
curl -X POST 'https://your-domain.com/api/aiagent/getHistory' \
|
||
-H 'Content-Type: application/json' \
|
||
-d '{"agent_id":"agent_aikefu","conversation_id":"openai_1700_xxx","user_id":"guest_test","limit":20}'
|
||
|
||
# 7. 清对话 (URL 用 kebab-case, ThinkPHP 自动转: clearConversation → /clear-conversation)
|
||
curl -X POST 'https://your-domain.com/api/aiagent/clear-conversation' \
|
||
-H 'Content-Type: application/json' \
|
||
-d '{"agent_id":"agent_aikefu","conversation_id":"openai_1700_xxx","user_id":"guest_test"}'
|
||
```
|
||
|
||
### 5.2 真实 WorkBuddy 流验证 (端到端)
|
||
|
||
1. 商家后台创建一个 Agent, 选 `upload_strategy=auto`
|
||
2. 微信小程序打开 AI 客服页
|
||
3. 发文字 → 应该看到流式回复
|
||
4. 发图片 (≤2MB) → 应该走 PHP 中转; **前端塞了 `data` 字段 → AI 端一定能看到图** (PR-WB-MP-INLINE-MEDIA 内联); 没塞则依赖 J9, 部署到云服务器后可能看不到
|
||
5. 发图片 (>2MB) → 应该走云存储直传 (策略 oss_sts/auto); **不传 `data` 字段** (>5MB 阈值), AI 端看到图依赖 J9 (WB 服务端 fetch 公网 URL), 部署到云服务器后可能看不到
|
||
6. 继续发文字 → 应该接上下文
|
||
7. 退出 + 重进 → 应该看到历史消息
|
||
8. 点 "新对话" → 下一条应该不接上下文
|
||
9. 切到另一个 agent → 应该看到该 agent 的历史 (或新建)
|
||
|
||
如果任何步骤卡住, 看:
|
||
- 前端: `console.log('[AI WS]', ...)` 或 `[AI SSE]` 日志
|
||
- 后端: `runtime/log/app_run_info/2026-09-14-wb-rpc.log` 和 `wb-rest.log`
|
||
- 验证 `data` 字段是否真塞进 attachments: DevTools Network 看 `chat` 请求 body 里 `attachments[0].data` 是非空 base64 字符串
|
||
- 验证后端是否走 inline (而非 resource_link): `runtime/log/app_run_info/2026-09-14-wb-rpc.log` 里 `buildMediaBlocks` 输出 type=image/audio base64 inline (非 resource_link)
|
||
- 验证 storage/transport 分离: 查 DB attachments 列 JSON, 应不含 base64 字节 (只含 url+meta)
|
||
|
||
---
|
||
|
||
## 6. 已知坑与 FAQ
|
||
|
||
### 6.1 思考流收不到 (前端)
|
||
|
||
**根因 1**: aiagent HTTP 路径下, 中途根本没有独立 `thought` event, 只能在 `message_end.thoughts` 拿完整文本。这是协议事实, 不是 bug。
|
||
|
||
**根因 2**: 后端没切到 A 路。检查 `runtime/log/app_run_info/2026-09-14-wb-rpc.log` 里 `platform_type` 字段。
|
||
|
||
**根因 3**: WB Agent 平台本身没下发 `agent_thought_chunk` (有些模型不支持思考步骤)。
|
||
|
||
**根因 4**: B 路 Dify 把思考内容塞 `answer` 里, 用 `> ` 前缀。检查后端是否做了 prefix 拆。
|
||
|
||
### 6.2 Question 选项后端没反应
|
||
|
||
**根因 1**: 后端 `_codebuddy.ask_user_question` / `AskUserQuestion` 工具未挂载, Agent 不会主动调用。
|
||
|
||
**根因 2**: 后端 H5 HTTP SSE 路径没有原生 answer 回传端点 (5 个 chat 入口都只支持 WS `elicit_response`)。前端 H5 端目前 `elicitResponse()` 走 console.warn + return false 兜底, 用户选项仅 UI 展示, 不真回传给 Agent; 30s 超时后端 emit `pending_stale`。**H5 真要回传, 需要后端补 elicit-response HTTP 端点 (本文档范围外)。**
|
||
|
||
**根因 3**: WB Agent 平台对 `tool_call` 有 PENDING_STALE_AFTER (≈10s) 守卫, 用户长时间不选, 后端 emit `message_end` with `stop_reason='pending_stale'`, 默认 reject。
|
||
|
||
**根因 4**: 前端回传 `answers` 时把 `value` 字段当成回传值 — 协议没有 `value`, 必须用 `label` 文本。
|
||
|
||
### 6.3 上传后图片 403
|
||
|
||
**如果图片能上传成功 (`url` 返了), 但浏览器打开 URL 显示 403:**
|
||
|
||
- **CORS 问题**: 云存储 bucket 没配前端域名白名单, 浏览器拿不到响应。后端不需要管, 运维去对应云存储控制台配 CORS:
|
||
- alioss: OSS 控制台 → Bucket → 跨域设置 → 创建规则: 来源 `https://your-h5.com`, 允许 Methods `GET`, 允许 Headers `*`
|
||
- qiniu: 七牛控制台 → 对象存储 → 空间设置 → 跨域设置
|
||
- txcos: 腾讯云 COS 控制台 → Bucket → 权限管理 → 跨域访问 CORS 设置
|
||
- huaweiobs / volcengine / awsoss / minio / 其他: 各自控制台 CORS 配置项
|
||
- **不区分 provider 的关键点**: 来源 (Origin) 必须填前端实际域名 (不是 `*`); Methods 至少含 GET; 允许 Headers 含 `*`
|
||
- **Bucket ACL 是 private**: 改成 public-read, 或走后端签名 URL (本 PR 范围外). 12 个 provider 都需对应控制台改 ACL, 没有统一入口
|
||
|
||
**如果上传时 403**: 看后端 `OssStsClient` 抛的错误 (5 类 STS 关键词), 按 `docs/mp-media-upload-integration.md` §9 对照。
|
||
|
||
### 6.4 多轮上下文串了
|
||
|
||
最常见的根因是 conversation_id 串了:
|
||
|
||
- 同一前端用户切了 agent, 但代码没 per-agent 隔离 → 旧 agent 的 conversation_id 塞给新 agent (本文档 §4 已修)
|
||
- 用户复制了 conversation_id 直接调后端 API (绕过前端) → 上下文乱了
|
||
- `clearConversation` 后端成功但前端没 clear → 下一条还带旧 id
|
||
|
||
### 6.5 attachments 上传后 AI 端收不到
|
||
|
||
排查顺序:
|
||
1. 前端看 `wx.uploadFile` 成功回调里 `res.statusCode === 200`
|
||
2. 后端看 `runtime/log/app_run_info/2026-09-14-wb-rest.log` 是否有上传端点命中
|
||
3. 后端看 `WbAgentOsClient::buildMediaBlocks()` 是否有 attachments 入参
|
||
4. WB 端看 mock 服务端 (`tests/mock_wb_acp_server.js`) 收的 `session/prompt.prompt[]` 数组
|
||
5. **★ backend 模式部署到云服务器后 AI 端看不到图** (PR-WB-MP-INLINE-MEDIA 修复): `/upload/{site_id}/...` URL 透传给 WB 服务端 (J9) fetch, WB 后台策略不一定允许 fetch 内网/CDN 防盗链 URL — **必须前端塞 `data` (base64) 字段**让 ACP ContentBlock 内联, 完全跳过 J9 (见 §3.1.5). 验证方法:
|
||
- DevTools Network 看 chat 请求 `attachments[0].data` 字段非空
|
||
- 后端 `runtime/log/app_run_info/2026-09-14-wb-rpc.log` 看 `buildMediaBlocks` 输出 `type=image/audio base64 inline` (而非 `type=resource_link`)
|
||
6. **★ attachments.data 字段被静默丢弃** (老前端兼容): 旧前端只发 url 不发 data, 大文件会走 `resource_link` 依赖 J9;**别误判 "data 字段没用"** — 实际是没塞, 实际后端走的是 fallback 路径. 检查方法同上
|
||
7. **A 路 vs B 路协议不对称** (PR-WB-MP-INLINE-MEDIA 之前): 旧版 `WbAgentOsClient::buildMediaBlocks` 在 url 空时**静默丢**, 没有任何 error_log (要看 2026-09-16 之前的 log, 才会发现附件没进 prompt). 修复后无 url + 有 data 不丢, 但需前端塞 data
|
||
|
||
### 6.6 attachments 在流式响应里看不到
|
||
|
||
aiagent HTTP 路径下, attachments 只在历史消息 (DB) 里有, **不在 SSE 流里**。前端要看 AI 端的 attachments, 需要从 `getConversationHistory` 接口拉历史再渲染, 或者用 aikefu HTTP 路径 (流里有 attachments)。
|
||
|
||
### 6.7 移动端 H5 rpx 失效 (样式坑)
|
||
|
||
uniapp 内联 `:style` 绑定的 rpx 在 H5 端被浏览器丢弃, CSS 文件里的 rpx 走 postcss 转换能识别 (详见 memory `feedback_article_image_layout.md`)。本集成 doc 不涉及样式, 但 UI 同学做选项卡片 / 思考流面板时记得:
|
||
|
||
```vue
|
||
<!-- ✓ CSS 文件里写 rpx -->
|
||
<style lang="scss" scoped>
|
||
.thought-panel { padding: 12rpx 16rpx; }
|
||
</style>
|
||
|
||
<!-- ✗ 内联 style 写 rpx 在 H5 端失效 -->
|
||
<view :style="{ padding: '12rpx 16rpx' }"> <!-- H5 端不会转 rpx -->
|
||
```
|
||
|
||
---
|
||
|
||
## 7. 相关 PR 与文档
|
||
|
||
| 文档 | 路径 | 说明 |
|
||
|---|---|---|
|
||
| 媒体上传后端单点真源 | `backend/docs/mp-media-upload-integration.md` | 后端 media_policy / HINTS / STS 错误翻译 |
|
||
| WB Agent 平台协议 | `backend/src/addon/aiagent/docs/workbuddy_api.md` | ACP / AgentOS REST+ACP 双轨 |
|
||
| **后端协议完整规范** | `backend/docs/ai-agent-protocol-spec.md` | SSE/WS 帧结构 + REST 端点 + A/B 路径差异 |
|
||
| **WB Cloud SDK 协议约束矩阵** | `D:/projects/refs/codebuddy-agent-sdk/docs/wb-cloud-sdk-coverage.md` | J7 (inline base64 ✅) / J9 (URL fetch, 服务端策略) / J10 (无 `/uploads/*` API) / J1 (KB 挂载) / Q6 (source_app 白名单) |
|
||
| **后端 PR-WB-MP-INLINE-MEDIA ANALYSIS** | `backend/tests/wb_media_backend/ANALYSIS.md` §8 | attachments.data 字段 PR 详情 + 4 处文件改动面 + 协议层 A/B 对称修复 + 26 项 sanity 测试结果 |
|
||
| ai-chat 前端入口 | `frontend/pages_tool/ai-chat/index.vue` | 路由 + agent_id 接收 |
|
||
| ai-chat 消息渲染 | `frontend/pages_tool/ai-chat/ai-chat-message.vue` | 流式消息 / 历史 / 预览 |
|
||
| ai-service 客户端 SDK | `frontend/common/js/ai-service.js` | agent 路由 + WS/SSE + 会话管理 |
|
||
| **本文档对应新增文件** | | |
|
||
| media-upload 客户端 SDK | `frontend/common/js/media-upload.js` | PHP 中转 + 12 个云存储 STS 直传决策 + provider 分流 (4-field vs single-URL) |
|
||
|
||
---
|
||
|
||
## 8. 版本历史
|
||
|
||
| 日期 | PR | 改动 |
|
||
|---|---|---|
|
||
| 2026-09-16 | (PR-WB-MP-INLINE-MEDIA) | **前端文档**: §3.1.5 新增 attachments.data 字段协议 + 12 个云存储 provider 协议层差异 + J7/J9/J10 引用; §3.4 Step 3/4 sendMessage 塞 data + fileToBase64; §3.6 加 J9 提示; §5.2 步骤 4/5 区分 inline vs resource_link; §6.5 加 backend 模式 J9 根因; §7 引 WB SDK coverage + backend ANALYSIS §8; "阿里云" → "云存储" (PR-OSS-REGISTRY 后已是 12 provider); §0 TL;DR 加 attachments.data 行 |
|
||
| 2026-09-15 | (PR-OSS-REGISTRY) | **前端文档**: 全文 "阿里云" → "云存储" (12 provider: alioss/qiniu/txcos/huaweiobs/baidubos/qingstor/jdoss/volcengine/ucloud/wyoss/awsoss/minio); `uploadViaSts()` 加 provider 分流 (4-field STS vs single-URL 预签); §3.4 术语块补 single-URL 模式说明; §3.6 #11 加 provider 协议差异 |
|
||
| 2026-09-14 | (PR-WB-MP-MEDIA-API-PREFIX) | **前端文档**: 上传端点 URL 模式修正 `/api/<addon>/upload/...` → `/<addon>/api/upload/...` (addon 名在前); nginx 加 `merge_slashes on;` 兜底双斜杠; InitRoute 清理实测无效的精确路由注册 |
|
||
| 2026-09-14 | c6ff516 (PR-WB-MP-MEDIA-ONBOARDING-r1) | 后端 media_policy HINTS + STS 错误翻译 + 11 字段 |
|
||
| 2026-09-14 | 611780b (PR-WB-MP-MEDIA-SHARED) | aiagent + aikefu 共享 MediaUploadPolicy |
|
||
| 2026-09-12 | 422e547 (PR-WB-MP-MEDIA-2026-09-12) | aiagent 专属 upload 端点 + OssStsClient |
|
||
| 2026-09-12 | f525215 (PR-WB-SYSTEM-PROMPT) | WB 系统提示词按 platform_type 差异化 |
|
||
| 2026-09-12 | 6945598 (PR-WB-RUNTIME-CONTEXT) | WB Agent 客户端运行时上下文双通道 |
|
||
| 2026-09-12 | 160e724 (PR-CONTEXT-CLEANUP) | WB Agent 上下文完整自描述 |
|
||
|
||
前端文档版本: v1.2 (2026-09-16) — attachments.data 字段协议 + 12 云存储 provider 协议层 (PR-WB-MP-INLINE-MEDIA + PR-OSS-REGISTRY) |