Uniapp/lucky_shop
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

feat(ai-chat): frontend 媒体上传/播放全栈接通 (PR-FRONTEND-MEDIA-UPLOAD 2026-09-16)

Browse Source 
P0 — 新建 common/js/media-upload.js (12KB)
    - uploadImage(filePath) / uploadMedia(filePath, kind) 落地
    - 此前 ai-chat-tools-panel.vue 引用但文件从未入库, 选图立即抛 "Cannot find module"
    - 12 cloud provider 分流: 4-field STS (alioss/qiniu/txcos/baidubos/qingstor/jdoss/ucloud/wyoss) vs single-URL signed PUT (huaweiobs/volcengine/awsoss/minio)
    - 端点必须 addon namespace: /<addon>/api/upload/{image|file|ossSts}, 普通 /api/kefu/upload/* 不命中 InitRoute
    - 补 docs 设计稿漏写的 `import store from '@/store/index.js'` (H5 上传需带 token)

  P1 — ai-chat-message.vue 用户附件全支持
    - P1.1 filter 不再丢 file/audio/video 消息
    - P1.2 用户气泡加 type==='file' 分支: 文件图标 + 名称 + 大小 + 上传状态
    - P1.3 sendMessage type 不再强制把 audio/video 折成 file

  P2 — AI 侧 audio 播放引擎接通
    - P2.1 previewAttachment audio 分支 → 跳 file-preview
    - P2.2 音频气泡接 uni.createInnerAudioContext: data 加 innerAudioCtx/_innerAudioSrc, 改写 playAudio/pauseAudio, beforeDestroy 必 destroy, slider 区分 change (显示) / changeend (实际 seek)
    - P2.3 file-preview.vue 加 image-viewer + audio-viewer 两个分支;inferFileType 识别 jpg/png/mp3/wav 等扩展

  P3 — sendMessageWithContent 透传 attachments 参数
    - 默认 attachments = [], 传给 sendStreamMessage/sendNormalMessage
    - tools panel 选完文件 push 到 attachments → 链到 send

  lang — zh-cn + en-us file-preview 加 4 项
    - audioUrlNotExist / audioDefaultTitle / audioPlaying / audioPaused

  docs — docs/ai-agent-integration.md (跨 PR 累积同步, 2149 行)
    - 版本基线提到 PR-WB-MP-INLINE-MEDIA (2026-09-16) 及之后
    - 加 §3.1.5 attachments.data 字段协议 (PR-WB-MP-INLINE-MEDIA)
    - 12 cloud provider 替换原"阿里云"单点 (PR-OSS-REGISTRY)
    - media-upload.js 章节补 fileToBase64 (PR-FRONTEND-MEDIA-UPLOAD)
    - 上传策略表 backend/oss_sts/auto 全替换为"云存储"措辞

  对齐后端:
    - PR-WB-MP-INLINE-MEDIA (a707b90): media data 字段
    - PR-OSS-REGISTRY (12 provider): OssProviderRegistry
    - PR-FRONTEND-MEDIA-UPLOAD (本 commit): 前端落地

  兼容性:
    - 后端未升级到 PR-WB-MP-INLINE-MEDIA 之前, attachments 不带 data 字段, 走 URL 路径 (退化到老行为, 不破)
    - 后端 oss_sts 单点 alioss 时, securityToken 不带 -SIGNED 后缀, 走 4-field STS 分支 (与 PR-OSS-REGISTRY 之前兼容)

  回归:
    - node --check media-upload.js OK
    - lang zh ESM import OK, 输出 8 个 key 全对
    - ai-chat-message.vue 11 处 PR-FRONTEND-MEDIA-UPLOAD 标记, file-preview.vue 9 处
    - 旧 chat 路径不带 attachments 走默认 [], 不破坏现有响应结构

  安全约束: 不涉及 nginx 层防盗链 / 限速
This commit is contained in:
ZF Sun
2026-09-16 11:42:18 +08:00
parent 3f29116345
commit 7952f5a722
6 changed files with 2957 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

325
common/js/media-upload.js Normal file
View File

@@ -0,0 +1,325 @@
// common/js/media-upload.js
// ai-chat / ai-agent 媒体上传: 图片 (PHP 中转 + STS 直传) / 音视频 / 其他文件
//
// PR-FRONTEND-MEDIA-UPLOAD (2026-09-16): 之前 ai-chat-tools-panel.vue 引用了
// uploadImage / uploadMedia, 但本文件从未落库, 选图 / 选文件立即抛
// "Cannot find module '@/common/js/media-upload.js'". 此次按
// docs/ai-agent-integration.md §3.4 Step 1 落地, 顺手补 store import (设计稿漏).
//
// 调用方:
// ai-chat-tools-panel.vue → selectImage() / _uploadOneFile()
// ai-chat-message.vue → 直接用 aiService.sendMessage 传 attachments (本模块不出场)
//
// 上传策略:
// 图片: 优先 PHP 中转 (≤2MB 走 backend), 超过或商家选了 oss_sts/auto → STS 直传
// 音视频/文件: 一律 STS 直传 (体积大概率 >2MB)
//
// 端点:
// PHP 中转: POST /<addon>/api/upload/<image|file> multipart/form-data
// STS 凭证: POST /<addon>/api/upload/ossSts application/json
// STS 直传: PUT <provider>/<key> 云存储协议 (4-field STS 或 single-URL)
import aiService from './ai-service.js'
import Config from './config.js'
import store from '@/store/index.js'
const SIZE_LIMIT_PHP_PASSTHROUGH = 2 * 1024 * 1024 // 2MB
/**
* 上传图片 (type=image)
* @param {string} filePath - uni.chooseImage 返回的 tempFilePaths[0]
* @returns {Promise<{url, mime, size, name, kind, used_strategy}>}
*/
export async function uploadImage(filePath) {
const addon = aiService.getAddonName() // 'aikefu' | 'aiagent'
const config = await aiService.getServiceConfig()
const strategy = config?.upload_strategy || 'backend'
// H5 端: >10MB 先压
let compressedPath = filePath
// #ifdef H5
const fileSize = await getFileSize(filePath)
if (fileSize > 10 * 1024 * 1024) {
try {
const compressed = await uni.compressImage({
src: filePath,
quality: 80,
compressedWidth: 2048,
compressedHeight: 2048,
})
compressedPath = compressed.tempFilePath || filePath
} catch (e) {
console.warn('[media-upload] compressImage failed, use original:', e)
}
}
// #endif
// 检查是否需要走 STS (>2MB 或商家选 oss_sts)
const fileSizeFinal = await getFileSize(compressedPath)
const useSts = strategy === 'oss_sts'
|| (strategy === 'auto' && fileSizeFinal > SIZE_LIMIT_PHP_PASSTHROUGH)
if (useSts) {
return await uploadViaSts(compressedPath, 'image', addon, aiService.getAgentId(), config)
} else {
return await uploadViaPhp(compressedPath, 'image', addon, aiService.getAgentId())
}
}
/**
* 上传 audio / video / file (一律 STS, 因为体积大概率 >2MB)
* @param {string} filePath
* @param {'audio'|'video'|'file'} kind
*/
export async function uploadMedia(filePath, kind) {
if (!['audio', 'video', 'file'].includes(kind)) {
throw new Error(`invalid kind: ${kind}`)
}
const addon = aiService.getAddonName()
const config = await aiService.getServiceConfig()
return await uploadViaSts(filePath, kind, addon, aiService.getAgentId(), config)
}
/**
* PHP 中转 (multipart/form-data)
* 端点必须用 addon 模式 URL: /<addon>/api/upload/<image|file>
* 注释: InitRoute 注册 aikefu/api/upload 时挂在 addon namespace 下,
* /api/aikefu/upload/* 才是命中路径; /api/kefu/upload/* 不命中
* (那是普通接口命名空间, 上传走的是 addon 专属路由)
*/
async function uploadViaPhp(filePath, kind, addon, agentId) {
const url = (addon === 'aikefu' ? '/aikefu/api/upload/' : '/aiagent/api/upload/')
+ (kind === 'image' ? 'image' : 'file')
// #ifdef MP-WEIXIN
return new Promise((resolve, reject) => {
uni.uploadFile({
url: Config.baseUrl + url,
filePath,
name: 'file',
formData: {
kind,
agent_id: addon === 'aiagent' ? agentId : undefined,
config_uuid: addon === 'aikefu' ? (aiService.getCurrentConfigUuid?.() || '') : undefined,
uniacid: aiService.getUniacid(),
user_id: aiService.getUserId?.() || '',
},
success: (res) => {
try {
const data = typeof res.data === 'string' ? JSON.parse(res.data) : res.data
if (data.code === 0 || data.success) {
resolve({
url: data.data.url,
mime: data.data.mime,
size: data.data.size,
name: data.data.name,
kind: data.data.kind || kind,
used_strategy: data.data.used_strategy || 'backend',
oss_sts_required: data.data.oss_sts_required || false,
})
} else {
reject(new Error(data.msg || data.message || '上传失败'))
}
} catch (e) {
reject(new Error('解析响应失败: ' + e.message))
}
},
fail: (err) => reject(new Error('网络错误: ' + (err.errMsg || err.message))),
})
})
// #endif
// #ifdef H5
const form = new FormData()
const file = await pathToFile(filePath, kind)
form.append('file', file)
form.append('kind', kind)
if (addon === 'aiagent') form.append('agent_id', agentId)
else form.append('config_uuid', aiService.getCurrentConfigUuid?.() || '')
form.append('uniacid', aiService.getUniacid())
form.append('user_id', aiService.getUserId?.() || '')
const res = await fetch(Config.baseUrl + url, {
method: 'POST',
headers: { 'token': store.state.token || '' },
body: form,
})
const data = await res.json()
if (data.code === 0 || data.success) {
return {
url: data.data.url,
mime: data.data.mime,
size: data.data.size,
name: data.data.name,
kind: data.data.kind || kind,
used_strategy: data.data.used_strategy || 'backend',
oss_sts_required: data.data.oss_sts_required || false,
}
}
throw new Error(data.msg || data.message || '上传失败')
// #endif
}
/**
* STS 直传 (拿临时凭证 → 直传云存储)
*
* PR-OSS-REGISTRY (2026-09-15) 后端支持 12 个 provider, 协议层分两类:
* A) 4-field STS (8 个: alioss/qiniu/txcos/baidubos/qingstor/jdoss/ucloud/wyoss)
* → 前端拼 PUT URL, Authorization header = STS <ak>:<sk>
* B) single-URL (4 个: huaweiobs/volcengine/awsoss/minio)
* → sts.accessKeyId 已是完整预签 PUT URL, sts.securityToken='<PROVIDER>-SIGNED'
* 标记, 不带 Authorization
*
* 后端 stsResp.data.data 可能字段:
* { provider, bucket, region, accessKeyId, accessKeySecret, securityToken,
* key_prefix_tpl, site_id, host? (single-URL 模式), endpoint? (qiniu 7 参兼容) }
*/
async function uploadViaSts(filePath, kind, addon, agentId, config) {
// 1. 拿 STS 凭证 (addon 模式 URL)
const stsUrl = (addon === 'aikefu' ? '/aikefu/api/upload/' : '/aiagent/api/upload/') + 'ossSts'
const stsResp = await uni.request({
url: Config.baseUrl + stsUrl,
method: 'POST',
header: {
'Content-Type': 'application/json',
'token': store.state.token || ''
},
data: {
agent_id: addon === 'aiagent' ? agentId : undefined,
config_uuid: addon === 'aikefu' ? (aiService.getCurrentConfigUuid?.() || '') : undefined,
uniacid: aiService.getUniacid(),
user_id: aiService.getUserId?.() || '',
},
})
if (stsResp.statusCode !== 200 || (stsResp.data.code !== 0 && !stsResp.data.success)) {
throw new Error(stsResp.data?.msg || stsResp.data?.message || 'STS 凭证获取失败')
}
const sts = stsResp.data.data
const provider = sts.provider || 'alioss' // 老后端默认 alioss
// 2. 拼 PUT URL (用后端返回的 key_prefix_tpl 替换占位符)
const ext = (filePath.split('.').pop() || 'bin').toLowerCase()
const uuid = 'xxxxxxxxxxxx4xxxyxxxxxxxxxxxxxxx'.replace(/[xy]/g, c => {
const r = Math.random() * 16 | 0
return (c === 'x' ? r : (r & 0x3 | 0x8)).toString(16)
})
const now = new Date()
const key = (sts.key_prefix_tpl || 'site/{site_id}/{yyyy}/{mm}/{dd}/{rand}')
.replace('{site_id}', sts.site_id || '0')
.replace('{agent_id}', agentId || '0')
.replace('{yyyy}', now.getFullYear().toString())
.replace('{mm}', String(now.getMonth() + 1).padStart(2, '0'))
.replace('{dd}', String(now.getDate()).padStart(2, '0'))
.replace('{rand}', uuid)
+ '.' + ext
// ★ PR-OSS-REGISTRY (2026-09-15): provider 分流
// single-URL 模式: securityToken 末尾是 -SIGNED (OBS-SIGNED / TOS-SIGNED / AWS-SIGNED / MINIO-SIGNED)
const isSingleUrlMode = typeof sts.securityToken === 'string'
&& /-(SIGNED)$/.test(sts.securityToken)
let ossUrl, headers
if (isSingleUrlMode) {
// B) single-URL: sts.accessKeyId 是完整预签 PUT URL, 不需要 Authorization header
ossUrl = sts.accessKeyId
headers = { 'Content-Type': guessMime(ext) }
} else if (provider === 'qiniu') {
// A) qiniu 兼容模式: sts.accessKeyId = uptoken, endpoint 来自 sts.endpoint 或 sts.host
ossUrl = (sts.endpoint || sts.host || `https://upload-${sts.region}.qiniup.com`) + '/' + key
headers = {
'Authorization': 'UpToken ' + sts.accessKeyId,
'Content-Type': guessMime(ext),
}
} else {
// A) 4-field STS 默认 (alioss/txcos/baidubos/qingstor/jdoss/ucloud/wyoss)
// endpoint 模板按 provider 区分, 后端未返 endpoint 时按 OSS 模板拼
const tpl = (provider === 'alioss') ? `https://${sts.bucket}.${sts.region}.aliyuncs.com`
: (provider === 'txcos') ? `https://${sts.bucket}-${sts.region}.cos.${sts.region}.myqcloud.com`
: (provider === 'baidubos') ? `https://${sts.bucket}.${sts.region}.bcebos.com`
: (provider === 'jdoss') ? `https://${sts.bucket}.${sts.region}.jdcloud-oss.com`
: (provider === 'wyoss') ? `https://${sts.bucket}.${sts.host || sts.region}`
: (sts.endpoint ? sts.endpoint.replace('{bucket}', sts.bucket) : `https://${sts.bucket}.${sts.region}`)
ossUrl = `${tpl}/${key}`
headers = {
'Authorization': `STS ${sts.accessKeyId}:${sts.accessKeySecret}`,
'x-oss-security-token': sts.securityToken,
'Content-Type': guessMime(ext),
}
}
// 3. 直传云存储
// #ifdef MP-WEIXIN
await new Promise((resolve, reject) => {
uni.uploadFile({
url: ossUrl,
filePath,
name: 'file',
formData: {},
header: headers,
success: (res) => {
if (res.statusCode >= 200 && res.statusCode < 300) resolve()
else reject(new Error(`${provider} PUT ${res.statusCode}`))
},
fail: reject,
})
})
// #endif
// #ifdef H5
const file = await pathToFile(filePath, kind)
const ossRes = await fetch(ossUrl, {
method: 'PUT',
headers,
body: file,
})
if (!ossRes.ok) throw new Error(`${provider} PUT ${ossRes.status}`)
// #endif
// 4. 返回公网 URL
return {
url: ossUrl.split('?')[0], // single-URL 模式 URL 带 ?<签名参数>, 拼 chat attachment 用 base URL
mime: guessMime(ext),
size: file.size || 0,
name: key.split('/').pop(),
kind,
used_strategy: 'oss_sts',
provider,
}
}
async function getFileSize(filePath) {
// #ifdef MP-WEIXIN
return new Promise((resolve) => {
uni.getFileInfo({
filePath,
success: (res) => resolve(res.size),
fail: () => resolve(0),
})
})
// #endif
// #ifdef H5
try {
const res = await fetch(filePath)
const blob = await res.blob()
return blob.size
} catch (e) { return 0 }
// #endif
}
async function pathToFile(filePath, kind) {
// #ifdef H5
const res = await fetch(filePath)
const blob = await res.blob()
return new File([blob], filePath.split('/').pop() || `${kind}.bin`, { type: blob.type })
// #endif
}
function guessMime(ext) {
const map = {
jpg: 'image/jpeg', jpeg: 'image/jpeg', png: 'image/png', gif: 'image/gif', webp: 'image/webp',
mp3: 'audio/mpeg', wav: 'audio/wav', aac: 'audio/aac',
mp4: 'video/mp4', mov: 'video/quicktime',
pdf: 'application/pdf', txt: 'text/plain',
}
return map[ext] || 'application/octet-stream'
}
export default { uploadImage, uploadMedia }