Compare commits

...

17 Commits

Author SHA1 Message Date
digua 8564db7241 release: v0.21.1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 16:37:59 +08:00
digua fa123bab49 fix: keep forced index modal open on generation failure
Move auto-close logic from finally to success path so the modal
remains open for retry when generate() fails in force mode.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 16:28:50 +08:00
digua 53e0d21ec7 fix(sync): prevent data loss on small pages and validate retry import results
P1: Small terminal pages with hasMore=false were incorrectly treated as
empty based on file size alone, causing valid messages to be silently
dropped. Now checks actual message content via fileContainsMessages().

P1: Retry import results were not validated, so import failures (including
needFullResync) were masked as success. Now properly handles all failure
cases and tracks retryHasMore to exit the retry loop correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 16:28:50 +08:00
digua 92c8a795f9 fix(ui): spin sync icon in-place instead of showing separate loader
Replace UButton :loading with animate-spin on the arrow-path icon itself.
The sync button's icon now rotates when syncing, and the button is disabled.
The "sync all" button is also disabled when any session is syncing.
2026-05-23 16:28:50 +08:00
digua 811bb3c09a fix: prevent SessionIndexModal from blocking empty sessions 2026-05-23 16:28:50 +08:00
digua cb516daea0 fix(sync): raise page limit from 50 to 5000 for large chat histories
50 pages (50k messages) was too low for groups with 100k+ messages.
Raise to 5000 pages (5M messages) as a safety valve while allowing
full-sync to complete in a single pull cycle for most conversations.
2026-05-23 16:28:50 +08:00
digua a5bbff34e4 feat(sync): use incremental session index after pull sync
After pull-sync imports new messages, generate session index incrementally
instead of full regeneration. This preserves existing sessions and their
summaries, only processing newly imported (unindexed) messages.

Adds generateIncremental method across the full stack:
- core: already had generateIncrementalSessionIndex
- node-runtime: expose generateIncrementalIndex
- CLI: add POST /_web/sessions/:id/generate-incremental-index route
- Electron: add session:generateIncremental IPC handler
- Frontend: add generateIncremental to SessionIndexAdapter interface
2026-05-23 16:28:50 +08:00
digua fcfa5d0167 fix(sync): auto-generate session index after pull sync completes
After importing messages via pull sync, automatically generate session
index for the affected sessions. This ensures the timeline and session
overview are immediately available without manual index generation.
2026-05-23 16:28:50 +08:00
digua 9cf6ac7725 fix(sync): refresh sidebar session list after pull completes or data deleted
After a sync finishes (triggerPull/triggerPullAll/onPullResult), reload the
session list so newly imported chats appear in the sidebar immediately.
Also refresh after removing a subscription with deleteData=true so the
deleted session disappears from the sidebar.
2026-05-23 16:28:50 +08:00
digua b952ebc8bf fix(sync): fetch all sessions when remote server lacks pagination support
When the remote server doesn't return a `page` field and the result count
equals the requested limit, automatically re-fetch with limit=5000 to get
the complete session list instead of showing truncated results.
2026-05-23 16:28:50 +08:00
digua 4cd60aa0c1 docs: update Pull protocol to since+nextSince pagination model
- Change Accept header to application/json (NDJSON reserved for future)
- Remove offset/end request params, recommend since+limit only
- Mark hasMore+nextSince as required sync fields
- Move nextOffset/watermark to "protocol reserved" section
- Add cold-start retry explanation and pagination flow example
2026-05-23 16:28:50 +08:00
digua 7d5c541933 feat(sync): add option to delete imported chats when removing subscription
- Show confirmation dialog with two choices: keep data or delete imported chats
- Backend supports deleteData param across CLI routes and Electron IPC
- DataSourceManager.removeSession now returns the removed session info
- Add i18n strings for the delete confirmation dialog (zh-CN/en-US/ja-JP/zh-TW)
2026-05-23 16:28:50 +08:00
digua 31fc9be1ed fix(sync): improve pull reliability with retry logic and simplified pagination
- Add multi-retry (3 attempts, 2s/3s/5s delays) for cold-start empty responses
- Remove unused offset/end parameters from fetch URL construction
- Fix Accept header to application/json only (avoids content negotiation issues)
- Add PullProgress type and progress tracking in PullEngine
- Export progress via getProgress() for frontend polling
2026-05-23 16:28:50 +08:00
digua f55fb9cf00 fix(desktop): fall back to package version when app.getVersion returns 0.0.0
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 16:28:50 +08:00
digua 5aaf1ce331 fix(sync): validate DB schema in sessionExists to prevent "no such table: message" error
sessionExists() previously only checked file existence, causing pull sync
to enter incremental import path on invalid/empty DB files. Now validates
that the message table exists, and maps schema errors to needFullResync
as a safety fallback.
2026-05-23 16:28:50 +08:00
digua 795d10d8c9 feat(changelog): add icon and i18n for ci change type 2026-05-23 16:28:50 +08:00
digua 2d342865ff feat(capture): add per-version screenshot to ChangelogModal and make markdown list fix opt-in
Add a hover-triggered CaptureButton to each expanded changelog entry so
users can screenshot individual version notes. Make the @zumer/snapdom
markdown list compatibility fix opt-in via a new `markdownFix` option
(default false), so custom-styled lists like changelog bullets are no
longer disrupted during capture.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 16:28:50 +08:00
51 changed files with 1090 additions and 167 deletions
+1
View File
@@ -37,3 +37,4 @@ yarn.lock
!.vscode/snippets.code-snippets
.docs
.playwright-mcp/
-6
View File
@@ -29,12 +29,6 @@
"chatlab",
"chat",
"analysis",
"wechat",
"qq",
"telegram",
"whatsapp",
"discord",
"mcp",
"cli"
],
"scripts": {
@@ -11,6 +11,15 @@ export function registerSessionIndexRoutes(server: FastifyInstance, adapter: Ses
return { sessionCount }
})
server.post<{
Params: { id: string }
Body: { gapThreshold?: number }
}>('/_web/sessions/:id/generate-incremental-index', async (request) => {
const gapThreshold = (request.body as any)?.gapThreshold ?? 1800
const sessionCount = sessionIndexService.generateIncrementalIndex(adapter, request.params.id, gapThreshold)
return { sessionCount }
})
server.post<{ Params: { id: string } }>('/_web/sessions/:id/clear-index', async (request) => {
sessionIndexService.clearIndex(adapter, request.params.id)
return { success: true }
+33 -3
View File
@@ -14,6 +14,7 @@ import type { HttpFetcher, DataImporter, SyncNotifier, ImportResult, FetchParams
import { NOOP_LOGGER } from '@openchatlab/sync'
import { buildPullUrl } from '@openchatlab/sync'
import type { DatabaseManager } from '@openchatlab/node-runtime'
import { openBetterSqliteDatabase } from '@openchatlab/node-runtime'
import { parseFile } from '../import/chatlab-reader'
import { importData } from '../import/importer'
@@ -28,7 +29,7 @@ export class NodeFetcher implements HttpFetcher {
async fetchToTempFile(baseUrl: string, remoteSessionId: string, token: string, params: FetchParams): Promise<string> {
const url = buildPullUrl(baseUrl, remoteSessionId, params)
const headers: Record<string, string> = {
Accept: 'application/json, application/x-ndjson',
Accept: 'application/json',
}
if (token) headers['Authorization'] = `Bearer ${token}`
@@ -69,7 +70,32 @@ export class DirectImporter implements DataImporter {
sessionExists(sessionId: string): boolean {
const dbPath = this.dbManager.getDbPath(sessionId)
return fs.existsSync(dbPath)
if (!fs.existsSync(dbPath)) return false
try {
const db = openBetterSqliteDatabase(dbPath, { readonly: true, nativeBinding: this.nativeBinding })
const row = db
.prepare("SELECT COUNT(*) as cnt FROM sqlite_master WHERE type='table' AND name='message'")
.get() as { cnt: number }
db.close()
if (row.cnt === 0) {
this.logger.warn(`[DirectImporter] DB file exists but has no message table: ${sessionId}, removing`)
try {
fs.unlinkSync(dbPath)
} catch {
/* ignore */
}
return false
}
return true
} catch {
this.logger.warn(`[DirectImporter] Cannot validate DB file: ${sessionId}, removing`)
try {
fs.unlinkSync(dbPath)
} catch {
/* ignore */
}
return false
}
}
async importFile(tempFile: string, targetSessionId: string | undefined, externalId: string): Promise<ImportResult> {
@@ -106,7 +132,11 @@ export class DirectImporter implements DataImporter {
}
}
if (result.error?.includes('not found') || result.error?.includes('session_not_found')) {
if (
result.error?.includes('not found') ||
result.error?.includes('session_not_found') ||
result.error?.includes('no such table')
) {
return { success: false, newMessageCount: 0, sessionId, needFullResync: true }
}
+2 -1
View File
@@ -15,6 +15,7 @@ import { registerAutomationRoutes } from './routes'
export interface SyncRouteContext {
dsManager: DataSourceManager
pullEngine: PullEngine
dbManager: DatabaseManager
serverInfo: { port: number; host: string; token: string }
}
@@ -49,7 +50,7 @@ export function initSync(
logger: syncLogger,
})
registerAutomationRoutes(server, { dsManager, pullEngine, serverInfo })
registerAutomationRoutes(server, { dsManager, pullEngine, dbManager, serverInfo })
initScheduler({
dsManager,
+16 -3
View File
@@ -15,7 +15,7 @@ import {
import type { SyncRouteContext } from './index'
export function registerAutomationRoutes(server: FastifyInstance, ctx: SyncRouteContext): void {
const { dsManager, pullEngine, serverInfo } = ctx
const { dsManager, pullEngine, dbManager, serverInfo } = ctx
// ==================== Config (read-only in CLI mode) ====================
@@ -85,10 +85,17 @@ export function registerAutomationRoutes(server: FastifyInstance, ctx: SyncRoute
server.delete<{
Params: { id: string; sessId: string }
Querystring: { deleteData?: string }
}>('/_web/automation/data-sources/:id/sessions/:sessId', async (request, reply) => {
const ok = dsManager.removeSession(request.params.id, request.params.sessId)
if (!ok) return reply.code(404).send({ error: 'Session not found' })
const removed = dsManager.removeSession(request.params.id, request.params.sessId)
if (!removed) return reply.code(404).send({ error: 'Session not found' })
reloadTimer(request.params.id)
if (request.query.deleteData === 'true' && removed.targetSessionId) {
dbManager.close(removed.targetSessionId)
const dbPath = dbManager.getDbPath(removed.targetSessionId)
const fs = await import('node:fs')
if (fs.existsSync(dbPath)) fs.unlinkSync(dbPath)
}
return { success: true }
})
@@ -105,6 +112,12 @@ export function registerAutomationRoutes(server: FastifyInstance, ctx: SyncRoute
return pullEngine.triggerPullAll(request.params.id)
})
// ==================== Sync Progress ====================
server.get('/_web/automation/sync-progress', async () => {
return pullEngine.getProgress()
})
// ==================== Remote Session Discovery ====================
server.get<{
+4
View File
@@ -1,9 +1,12 @@
import { resolve } from 'path'
import { readFileSync } from 'fs'
import { defineConfig } from 'electron-vite'
import vue from '@vitejs/plugin-vue'
import ui from '@nuxt/ui/vite'
const rootDir = resolve(__dirname, '../..')
const rootPkg = JSON.parse(readFileSync(resolve(rootDir, 'package.json'), 'utf-8'))
const appVersion: string = rootPkg.version
export default defineConfig({
main: {
@@ -13,6 +16,7 @@ export default defineConfig({
},
},
define: {
__APP_VERSION__: JSON.stringify(appVersion),
'process.env.APTABASE_APP_KEY': JSON.stringify(process.env.APTABASE_APP_KEY || ''),
// ws 的原生加速依赖是可选项;主进程打包时禁用它们,避免 Vite 将缺失的可选依赖改写为启动即抛错。
'process.env.WS_NO_BUFFER_UTIL': JSON.stringify('true'),
+30 -4
View File
@@ -9,6 +9,7 @@
import * as fs from 'fs'
import * as path from 'path'
import * as crypto from 'crypto'
import Database from 'better-sqlite3'
import { net, BrowserWindow } from 'electron'
import { getTempDir } from '../paths'
import * as worker from '../worker/workerManager'
@@ -29,7 +30,7 @@ export class ElectronFetcher implements HttpFetcher {
const request = net.request(url)
if (token) request.setHeader('Authorization', `Bearer ${token}`)
request.setHeader('Accept', 'application/json, application/x-ndjson')
request.setHeader('Accept', 'application/json')
let tempFile = ''
let writeStream: fs.WriteStream | null = null
@@ -84,7 +85,32 @@ export class WorkerImporter implements DataImporter {
sessionExists(sessionId: string): boolean {
const dbPath = path.join(worker.getDbDirectory(), `${sessionId}.db`)
return fs.existsSync(dbPath)
if (!fs.existsSync(dbPath)) return false
try {
const db = new Database(dbPath, { readonly: true })
const row = db
.prepare("SELECT COUNT(*) as cnt FROM sqlite_master WHERE type='table' AND name='message'")
.get() as { cnt: number }
db.close()
if (row.cnt === 0) {
this.logger.warn(`[Pull] DB file exists but has no message table: ${sessionId}, removing`)
try {
fs.unlinkSync(dbPath)
} catch {
/* ignore */
}
return false
}
return true
} catch {
this.logger.warn(`[Pull] Cannot validate DB file: ${sessionId}, removing`)
try {
fs.unlinkSync(dbPath)
} catch {
/* ignore */
}
return false
}
}
async importFile(tempFile: string, targetSessionId: string | undefined, externalId: string): Promise<ImportResult> {
@@ -106,8 +132,8 @@ export class WorkerImporter implements DataImporter {
}
return { success: true, newMessageCount: result.newMessageCount, sessionId }
}
if (result.error === 'error.session_not_found') {
this.logger.warn(`[Pull] Session ${sessionId} not found locally, need full resync`)
if (result.error === 'error.session_not_found' || result.error?.includes('no such table')) {
this.logger.warn(`[Pull] Session ${sessionId} not found or schema invalid, need full resync`)
return { success: false, newMessageCount: 0, sessionId, needFullResync: true }
}
this.logger.error(`[Pull] Incremental import failed: ${result.error}`)
+2
View File
@@ -1,3 +1,5 @@
declare const __APP_VERSION__: string
declare module '*.md?raw' {
const content: string
export default content
+8 -3
View File
@@ -11,6 +11,7 @@ import { setConfigManager } from '../api'
import { getSettingsDir } from '../paths'
import { apiLogger } from '../api/logger'
import { getImportingStatus } from '../api/routes/import'
import { deleteSession as deleteSessionFile } from '../database/core'
import { ElectronFetcher, WorkerImporter, BrowserWindowNotifier } from '../api/adapters'
import {
ConfigManager,
@@ -185,10 +186,14 @@ export function registerApiHandlers(_ctx: IpcContext): void {
}
)
ipcMain.handle('api:removeImportSession', (_event, sourceId: string, sessionId: string) => {
const result = dsManager.removeSession(sourceId, sessionId)
ipcMain.handle('api:removeImportSession', (_event, sourceId: string, sessionId: string, deleteData?: boolean) => {
const removed = dsManager.removeSession(sourceId, sessionId)
if (!removed) return false
reloadTimer(sourceId)
return result
if (deleteData && removed.targetSessionId) {
deleteSessionFile(removed.targetSessionId)
}
return true
})
// ==================== Sync ====================
+9
View File
@@ -803,6 +803,15 @@ export function registerChatHandlers(ctx: IpcContext): void {
}
})
ipcMain.handle('session:generateIncremental', async (_, sessionId: string, gapThreshold?: number) => {
try {
return await worker.generateIncrementalSessions(sessionId, gapThreshold)
} catch (error) {
console.error('Failed to generate incremental session index:', error)
throw error
}
})
/**
* 检查是否已生成会话索引
*/
+2 -1
View File
@@ -98,7 +98,8 @@ export function registerWindowHandlers(ctx: IpcContext): void {
// ==================== 应用信息 ====================
ipcMain.handle('app:getVersion', () => {
return app.getVersion()
const v = app.getVersion()
return v && v !== '0.0.0' ? v : typeof __APP_VERSION__ !== 'undefined' ? __APP_VERSION__ : v
})
// 重启应用
+2 -2
View File
@@ -126,8 +126,8 @@ export const apiServerApi = {
return ipcRenderer.invoke('api:addImportSessions', sourceId, sessions)
},
removeImportSession: (sourceId: string, sessionId: string): Promise<boolean> => {
return ipcRenderer.invoke('api:removeImportSession', sourceId, sessionId)
removeImportSession: (sourceId: string, sessionId: string, deleteData?: boolean): Promise<boolean> => {
return ipcRenderer.invoke('api:removeImportSession', sourceId, sessionId, deleteData)
},
// ==================== 同步 ====================
+4 -3
View File
@@ -240,9 +240,10 @@ export const sessionApi = {
return ipcRenderer.invoke('session:generate', sessionId, gapThreshold)
},
/**
* 检查是否已生成会话索引
*/
generateIncremental: (sessionId: string, gapThreshold?: number): Promise<number> => {
return ipcRenderer.invoke('session:generateIncremental', sessionId, gapThreshold)
},
hasIndex: (sessionId: string): Promise<boolean> => {
return ipcRenderer.invoke('session:hasIndex', sessionId)
},
+2 -1
View File
@@ -1074,7 +1074,7 @@ interface ApiServerApi {
sourceId: string,
sessions: Array<{ name: string; remoteSessionId: string }>
) => Promise<ImportSession[]>
removeImportSession: (sourceId: string, sessionId: string) => Promise<boolean>
removeImportSession: (sourceId: string, sessionId: string, deleteData?: boolean) => Promise<boolean>
triggerPull: (sourceId: string, sessionId?: string) => Promise<{ success: boolean; error?: string }>
triggerPullAll: (sourceId: string) => Promise<{ success: boolean; error?: string }>
fetchRemoteSessions: (
@@ -1107,6 +1107,7 @@ interface ChatSessionItem {
interface SessionApi {
generate: (sessionId: string, gapThreshold?: number) => Promise<number>
generateIncremental: (sessionId: string, gapThreshold?: number) => Promise<number>
hasIndex: (sessionId: string) => Promise<boolean>
getStats: (sessionId: string) => Promise<SessionStats>
clear: (sessionId: string) => Promise<boolean>
+47 -29
View File
@@ -138,7 +138,7 @@ ChatLab 在 UI 中展示该列表,用户选择需要导入的对话。
```
GET {baseUrl}/sessions/{sessionId}/messages?format=chatlab&since={timestamp}
Authorization: Bearer {token}
Accept: application/json, application/x-ndjson
Accept: application/json
```
| 参数 | 必填 | 说明 |
@@ -146,9 +146,11 @@ Accept: application/json, application/x-ndjson
| `sessionId` | 是 | 来自阶段一返回的对话 `id` |
| `format` | 是 | 固定为 `chatlab`,要求数据源返回 ChatLab 标准格式 |
| `since` | 否 | Unix 时间戳(秒级)。省略或为 `0` 时为全量拉取,大于 0 时为增量拉取 |
| `end` | 否 | Unix 时间戳(秒级)。指定时间范围上界,用于历史回填等分段拉取 |
| `limit` | 否 | 单次返回的最大消息数,用于分页 |
| `offset` | 否 | 分页偏移量,配合 `limit` 使用 |
::: tip 未来演进
后续版本可能支持 `Accept: application/x-ndjson` 以启用 NDJSON 流式响应,当前版本仅使用 JSON。
:::
### 数据携带规则
@@ -156,6 +158,10 @@ Accept: application/json, application/x-ndjson
- **增量同步**`since > 0`):**必须**包含 `messages``meta` / `members` **仅在发生实际变更时携带**,未变更时不得携带,以避免历史快照覆盖当前状态
- 无新数据时返回空 `messages` 数组
::: tip 数据准备
数据源在首次收到某个会话的 `since=0` 请求时,如需时间准备数据(如从磁盘加载、索引构建等),可先返回空 `messages` + `hasMore: false`。ChatLab 会自动重试(最多 3 次,间隔递增),等待数据源就绪后正常返回数据。
:::
### 响应格式
响应为标准 [ChatLab Format](./chatlab-format.md)JSON 或 JSONL),并附带 `sync` 同步元信息。
@@ -168,46 +174,60 @@ Accept: application/json, application/x-ndjson
"messages": [ ... ],
"sync": {
"hasMore": true,
"nextSince": 1711468800,
"nextOffset": 5000,
"watermark": 1711470000
"nextSince": 1711468800
}
}
```
### sync 同步元信息
| 字段 | 类型 | 说明 |
| ------------ | ------- | ------------------------------------------------------------------------------------------ |
| `hasMore` | boolean | 是否还有更多数据。为 `true` 时 ChatLab 自动续拉 |
| `nextSince` | number | 下一次请求建议使用的 `since` |
| `nextOffset` | number | 下一次请求建议使用的 `offset` 值,用于分页续拉 |
| `watermark` | number | 本次拉取的快照上界时间戳,防止分页期间新数据写入导致的窗口漂移 |
| 字段 | 类型 | 必填 | 说明 |
| ------------ | ------- | ------ | ------------------------------------------------------------------------------------------ |
| `hasMore` | boolean | **是** | 是否还有更多数据。为 `true` 时 ChatLab 自动续拉 |
| `nextSince` | number | **是** | 下一次请求建议使用的 `since`(通常为本批最后一条消息的时间戳) |
ChatLab 的分页续拉完全基于 `hasMore` + `nextSince` 时间戳链。数据源返回一批消息后,将 `nextSince` 设为本批最后一条消息的时间戳,ChatLab 下次请求时传入该值即可获取后续数据。ChatLab 内置的去重机制会正确处理时间戳边界的消息重叠。
::: details 协议预留字段(当前版本不使用)
以下字段在协议中保留,ChatLab 当前版本不主动使用,未来版本可能启用:
| 字段 | 类型 | 说明 |
| ------------ | ------- | -------------------------------------------------------------- |
| `nextOffset` | number | 分页偏移量,配合 `offset` 参数使用 |
| `watermark` | number | 快照上界时间戳,用于保证分页期间数据一致性 |
数据源可以不实现这些字段。ChatLab 的去重机制(基于 `platformMessageId` 或内容哈希)已能保证数据完整性。
:::
**sync 块的必要性规则:**
| 数据源返回方式 | sync 块要求 | 说明 |
| -------------------------- | ----------- | ---------------------------------------------------------------- |
| 单次返回全部数据(不分页) | 可选 | ChatLab 视 `messages` 为完整结果 |
| 支持 `limit`/`offset` 分页 | **必须** | 至少包含 `hasMore`;若无法保证快照一致性,还必须包含 `watermark` |
| 支持 `limit` 分页 | **必须** | 至少包含 `hasMore` + `nextSince` |
::: warning 注意
若数据源支持分页但未返回 `sync` 块,ChatLab 不保证自动续拉和分页一致性——仅处理首次返回的数据。
若数据源支持分页但未返回 `sync` 块,ChatLab 不保证自动续拉——仅处理首次返回的数据。
:::
### 快照一致性要求
- 同一次分页拉取(通过 `limit`/`offset` 遍历)**应当**基于同一数据快照视图
- 若数据源无法保证快照一致性,**必须**返回 `sync.watermark`,供 ChatLab 固定拉取窗口上界
### 分批拉取策略
对于大量历史数据(如数万条消息),推荐两种分批方式:
对于大量历史数据(如数万条消息),推荐分批方式:
1. **时间窗口分批**:通过 `since` + `end` 指定时间段,逐段拉取
2. **分页分批**:通过 `limit` + `offset` 做数据库级分页,结合 `sync` 块自动续拉
**时间戳链分批**(推荐):通过 `since` + `limit` 分批拉取,数据源通过 `sync.nextSince` 返回下次请求的起始时间戳,ChatLab 自动续拉直到 `hasMore=false`
ChatLab 会自动处理分批场景,通过去重机制保证不重复写入。
```
第 1 页:GET /sessions/:id/messages?format=chatlab&since=0&limit=1000
→ 返回 1000 条,sync: { hasMore: true, nextSince: 1711400000 }
第 2 页:GET /sessions/:id/messages?format=chatlab&since=1711400000&limit=1000
→ 返回 1000 条,sync: { hasMore: true, nextSince: 1711440000 }
第 N 页:...
→ 返回 500 条,sync: { hasMore: false, nextSince: 1711468800 }
```
ChatLab 内置去重机制保证不重复写入,即使 `nextSince` 边界上有消息重叠也不会产生重复数据。
---
@@ -282,12 +302,10 @@ ChatLab 接收到 SSE 事件后,**触发一次该 session 的增量拉取**(
### 增强实现
| 能力 | 说明 |
| -------------------------- | ------------------------------------------------------------ |
| `GET /push/messages` | SSE 实时通知(仅唤醒拉取,不传输完整数据) |
| 支持 `limit`/`offset` 分页 | 大量历史数据的分拉取 |
| 支持 `end` 参数 | 时间窗口分段拉取 |
| 响应包含 `sync` 块 | 提供 `hasMore`/`nextSince`/`nextOffset`/`watermark` 续拉信息 |
| 能力 | 说明 |
| ------------------------------- | ------------------------------------------------------------ |
| `GET /push/messages` | SSE 实时通知(仅唤醒拉取,不传输完整数据) |
| 支持 `limit` + `sync` 分页 | 大量历史数据的分拉取,通过 `hasMore` + `nextSince` 续拉 |
### 数据格式
+35
View File
@@ -1,4 +1,39 @@
[
{
"version": "0.21.1",
"date": "2026-05-23",
"summary": "优化同步拉取可靠性与数据安全性,新增移除订阅时清理聊天记录选项,并修复 UI 动画和弹窗交互问题。",
"changes": [
{
"type": "feat",
"items": [
"Pull 同步后自动生成会话索引",
"新增移除订阅时清理已导入聊天记录的选项",
"【CLI Web】版本日志弹窗支持按版本截图,Markdown 列表修正改为可选",
"【MCP】新增 ci 变更类型的图标与国际化支持"
]
},
{
"type": "fix",
"items": [
"修复 Pull 同步在小页面场景下可能丢失数据的问题,并校验重试导入结果",
"修复拉取同步后会话索引未自动生成的问题",
"修复 Pull 完成或数据删除后侧边栏会话列表未刷新的问题",
"修复远端服务不支持分页时无法获取全部会话的问题",
"优化拉取同步的重试机制与分页策略,提升稳定性",
"修复会话存在性校验缺少 schema 检查导致表缺失报错的问题",
"修复强制生成会话索引弹窗在失败时意外关闭的问题",
"修复空会话状态下会话索引弹窗阻塞页面的问题",
"【桌面端】修复应用版本号显示为 0.0.0 时自动回退读取 package.json 版本",
"【CLI Web】同步图标改为原地旋转动画,避免出现独立加载器"
]
},
{
"type": "docs",
"items": ["更新 Pull 协议文档为 since+nextSince 分页模式"]
}
]
},
{
"version": "0.21.0",
"date": "2026-05-22",
+35
View File
@@ -1,4 +1,39 @@
[
{
"version": "0.21.1",
"date": "2026-05-23",
"summary": "Improve pull sync reliability and data safety, add an option to clean up imported chats when removing subscriptions, and fix UI animation and modal interaction issues.",
"changes": [
{
"type": "feat",
"items": [
"Auto-generate session index after pull sync",
"Add option to delete imported chats when removing a subscription",
"【CLI Web】Support per-version screenshots in the changelog modal and make Markdown list fixes opt-in",
"【MCP】Add icon and i18n support for the ci change type"
]
},
{
"type": "fix",
"items": [
"Fix potential data loss on small pages during pull sync and validate retry import results",
"Fix session index not auto-generating after pull sync completes",
"Fix sidebar session list not refreshing after pull completes or data is deleted",
"Fix inability to fetch all sessions when the remote server lacks pagination support",
"Improve pull sync retry logic and pagination strategy for better stability",
"Fix missing schema check in session existence validation that caused 'no such table' errors",
"Fix forced session index modal unexpectedly closing on generation failure",
"Fix session index modal blocking the page when sessions are empty",
"【Desktop】Fall back to the package.json version when app.getVersion returns 0.0.0",
"【CLI Web】Sync icon now spins in-place instead of showing a separate loader"
]
},
{
"type": "docs",
"items": ["Update Pull protocol docs to the since+nextSince pagination model"]
}
]
},
{
"version": "0.21.0",
"date": "2026-05-22",
+35
View File
@@ -1,4 +1,39 @@
[
{
"version": "0.21.1",
"date": "2026-05-23",
"summary": "Pull 同期の信頼性とデータの安全性を改善し、サブスクリプション削除時にインポート済みチャットをクリーンアップするオプションを追加。UI アニメーションとモーダルの操作に関する問題を修正しました。",
"changes": [
{
"type": "feat",
"items": [
"Pull 同期後にセッションインデックスを自動生成",
"サブスクリプション削除時にインポート済みチャットを削除するオプションを追加",
"【CLI Web】更新履歴モーダルでバージョンごとのスクリーンショットに対応、Markdown リスト修正をオプションに変更",
"【MCP】ci 変更タイプのアイコンと多言語対応を追加"
]
},
{
"type": "fix",
"items": [
"Pull 同期時の小ページでデータが失われる可能性がある問題を修正し、リトライインポート結果の検証を追加",
"Pull 同期完了後にセッションインデックスが自動生成されない問題を修正",
"Pull 完了またはデータ削除後にサイドバーのセッションリストが更新されない問題を修正",
"リモートサーバーがページネーション未対応の場合に全セッションを取得できない問題を修正",
"Pull 同期のリトライロジックとページネーション戦略を最適化し、安定性を向上",
"セッション存在チェック時のスキーマ検証が不足し、テーブル未検出エラーが発生する問題を修正",
"強制セッションインデックス生成モーダルが生成失敗時に予期せず閉じる問題を修正",
"セッションが空の状態でセッションインデックスモーダルがページをブロックする問題を修正",
"【デスクトップ】app.getVersion が 0.0.0 を返す場合に package.json のバージョンにフォールバック",
"【CLI Web】独立したローダーを表示せず、同期アイコンをその場で回転アニメーションに変更"
]
},
{
"type": "docs",
"items": ["Pull プロトコルのドキュメントを since+nextSince ページネーションモデルに更新"]
}
]
},
{
"version": "0.21.0",
"date": "2026-05-22",
+35
View File
@@ -1,4 +1,39 @@
[
{
"version": "0.21.1",
"date": "2026-05-23",
"summary": "優化 Pull 同步可靠性與資料安全性,新增移除訂閱時清除已匯入聊天記錄的選項,並修正 UI 動畫與彈窗互動問題。",
"changes": [
{
"type": "feat",
"items": [
"Pull 同步完成後自動產生會話索引",
"新增移除訂閱時清除已匯入聊天記錄的選項",
"【CLI Web】版本紀錄彈窗支援依版本截圖,Markdown 列表修正改為可選",
"【MCP】新增 ci 變更類型的圖示與多語系支援"
]
},
{
"type": "fix",
"items": [
"修正 Pull 同步在小頁面情境下可能遺失資料的問題,並驗證重試匯入結果",
"修正拉取同步後會話索引未自動產生的問題",
"修正 Pull 完成或資料刪除後側邊欄會話列表未重新整理的問題",
"修正遠端服務不支援分頁時無法取得全部會話的問題",
"優化拉取同步的重試機制與分頁策略,提升穩定性",
"修正會話存在性檢查缺少 schema 驗證導致資料表不存在報錯的問題",
"修正強制產生會話索引彈窗在失敗時意外關閉的問題",
"修正空會話狀態下會話索引彈窗阻塞頁面的問題",
"【桌面端】修正應用程式版本號顯示為 0.0.0 時自動回退讀取 package.json 版本",
"【CLI Web】同步圖示改為原地旋轉動畫,不再出現獨立的載入指示器"
]
},
{
"type": "docs",
"items": ["更新 Pull 協議文件為 since+nextSince 分頁模式"]
}
]
},
{
"version": "0.21.0",
"date": "2026-05-22",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "ChatLab",
"version": "0.21.0",
"version": "0.21.1",
"private": true,
"description": "本地化的聊天记录分析工具,通过 SQL 和 AI Agent 回顾你的社交记忆",
"repository": {
+8 -1
View File
@@ -24,7 +24,14 @@ export {
export type { MembersPaginatedDTO } from './member-service'
// Session index service
export { generateIndex, clearIndex, getFtsStatus, searchFts, rebuildFts } from './session-index-service'
export {
generateIndex,
generateIncrementalIndex,
clearIndex,
getFtsStatus,
searchFts,
rebuildFts,
} from './session-index-service'
// Summary service
export { generateSummary, generateAllSummaries } from './summary-service'
@@ -6,6 +6,7 @@
import {
generateSessionIndex as coreGenerateSessionIndex,
generateIncrementalSessionIndex as coreGenerateIncrementalSessionIndex,
clearSessionIndex as coreClearSessionIndex,
} from '@openchatlab/core'
import { hasFtsTable, searchByFts, rebuildFtsIndex } from '../fts'
@@ -16,6 +17,15 @@ export function generateIndex(adapter: SessionRuntimeAdapter, sessionId: string,
return coreGenerateSessionIndex(db, gapThreshold)
}
export function generateIncrementalIndex(
adapter: SessionRuntimeAdapter,
sessionId: string,
gapThreshold: number = 1800
): number {
const db = adapter.ensureWritable(sessionId)
return coreGenerateIncrementalSessionIndex(db, gapThreshold)
}
export function clearIndex(adapter: SessionRuntimeAdapter, sessionId: string): void {
const db = adapter.ensureWritable(sessionId)
coreClearSessionIndex(db)
+5 -5
View File
@@ -166,15 +166,15 @@ export class DataSourceManager {
return added
}
removeSession(sourceId: string, sessionId: string): boolean {
removeSession(sourceId: string, sessionId: string): ImportSession | null {
const sources = this.loadAll()
const ds = sources.find((s) => s.id === sourceId)
if (!ds) return false
const before = ds.sessions.length
if (!ds) return null
const removed = ds.sessions.find((s) => s.id === sessionId)
if (!removed) return null
ds.sessions = ds.sessions.filter((s) => s.id !== sessionId)
if (ds.sessions.length === before) return false
this.saveAll(sources)
return true
return removed
}
updateSession(sourceId: string, sessionId: string, updates: Partial<ImportSession>): ImportSession | null {
+1
View File
@@ -21,6 +21,7 @@ export type {
SyncMeta,
ImportResult,
PullSessionResult,
PullProgress,
HttpFetcher,
DataImporter,
SyncNotifier,
+221
View File
@@ -0,0 +1,221 @@
import { afterEach, describe, it } from 'node:test'
import assert from 'node:assert/strict'
import * as fs from 'fs'
import * as os from 'os'
import * as path from 'path'
import { PullEngine } from './pull-engine'
import type { DataSource, DataImporter, FetchParams, HttpFetcher, ImportSession, SyncNotifier } from './types'
const tempFiles: string[] = []
afterEach(() => {
for (const file of tempFiles.splice(0)) {
try {
if (fs.existsSync(file)) fs.unlinkSync(file)
} catch {
/* ignore */
}
}
})
function writeTempJson(data: unknown): string {
const file = path.join(os.tmpdir(), `chatlab-pull-engine-test-${process.pid}-${tempFiles.length}.json`)
fs.writeFileSync(file, JSON.stringify(data), 'utf-8')
tempFiles.push(file)
return file
}
async function withImmediateTimers<T>(fn: () => Promise<T>): Promise<T> {
const originalSetTimeout = globalThis.setTimeout
;(globalThis as any).setTimeout = (callback: (...args: any[]) => void) => {
callback()
return 0
}
try {
return await fn()
} finally {
globalThis.setTimeout = originalSetTimeout
}
}
function createDataSource(): DataSource {
return {
id: 'ds_test',
name: 'Test Source',
baseUrl: 'http://example.test/api/v1',
token: '',
intervalMinutes: 10,
pullLimit: 1000,
enabled: true,
createdAt: 1,
sessions: [],
}
}
function createSession(): ImportSession {
return {
id: 'is_test',
name: 'Test Session',
remoteSessionId: 'remote_session',
targetSessionId: 'local_session',
lastPullAt: 100,
lastStatus: 'idle',
lastError: '',
lastNewMessages: 0,
}
}
function createEngine(options: {
files: string[]
importResult: Awaited<ReturnType<DataImporter['importFile']>>
dataSource: DataSource
sessionUpdates?: Array<{ sessionId: string; updates: Partial<ImportSession> }>
pullResults?: Array<{ status: 'success' | 'error'; detail: string }>
}): PullEngine {
const files = [...options.files]
const fetcher: HttpFetcher = {
async fetchToTempFile(
_baseUrl: string,
_remoteSessionId: string,
_token: string,
_params: FetchParams
): Promise<string> {
const file = files.shift()
if (!file) throw new Error('Unexpected retry fetch')
return file
},
}
const importer: DataImporter = {
sessionExists: () => true,
importFile: async () => options.importResult,
}
const notifier: SyncNotifier = {
onSessionListChanged: () => {},
onPullResult: (_sourceId, _sessionId, status, detail) => {
options.pullResults?.push({ status, detail })
},
}
const dsManager = {
get: () => options.dataSource,
updateSession: (_sourceId: string, sessionId: string, updates: Partial<ImportSession>) => {
options.sessionUpdates?.push({ sessionId, updates })
},
}
return new PullEngine({
fetcher,
importer,
notifier,
dsManager: dsManager as any,
})
}
describe('PullEngine', () => {
it('imports a small final page instead of treating it as empty', async () => {
const session = createSession()
const dataSource = createDataSource()
dataSource.sessions = [session]
const smallFinalPage = writeTempJson({
chatlab: { version: '0.0.2', exportedAt: 100 },
meta: { name: 'Test Session', platform: 'test', type: 'group' },
members: [{ platformId: 'u1', accountName: 'Alice' }],
messages: [{ sender: 'u1', timestamp: 101, type: 0, content: 'hi' }],
sync: { hasMore: false, nextSince: 101 },
})
let importCount = 0
const engine = createEngine({
files: [smallFinalPage],
dataSource,
importResult: {
success: true,
newMessageCount: 1,
sessionId: session.targetSessionId,
},
})
;(engine as any).importer.importFile = async () => {
importCount++
return { success: true, newMessageCount: 1, sessionId: session.targetSessionId }
}
const result = await withImmediateTimers(() => engine.executePullSession(dataSource.id, dataSource, session))
assert.equal(result.success, true)
assert.equal(result.newMessageCount, 1)
assert.equal(importCount, 1)
})
it('reports retry import failure instead of marking the pull successful', async () => {
const session = createSession()
const dataSource = createDataSource()
dataSource.sessions = [session]
const emptyInitialPage = writeTempJson({
chatlab: { version: '0.0.2', exportedAt: 100 },
meta: { name: 'Test Session', platform: 'test', type: 'group' },
members: [],
messages: [],
sync: { hasMore: false, nextSince: 100 },
})
const retryPage = writeTempJson({
chatlab: { version: '0.0.2', exportedAt: 100 },
meta: { name: 'Test Session', platform: 'test', type: 'group' },
members: [{ platformId: 'u1', accountName: 'Alice' }],
messages: [
{
sender: 'u1',
timestamp: 101,
type: 0,
content: 'x'.repeat(1200),
},
],
sync: { hasMore: false, nextSince: 101 },
})
const emptyTerminalPage = writeTempJson({
chatlab: { version: '0.0.2', exportedAt: 100 },
meta: { name: 'Test Session', platform: 'test', type: 'group' },
members: [],
messages: [],
sync: { hasMore: false, nextSince: 101 },
})
const emptyRetryPage1 = writeTempJson({
chatlab: { version: '0.0.2', exportedAt: 100 },
meta: { name: 'Test Session', platform: 'test', type: 'group' },
members: [],
messages: [],
sync: { hasMore: false, nextSince: 101 },
})
const emptyRetryPage2 = writeTempJson({
chatlab: { version: '0.0.2', exportedAt: 100 },
meta: { name: 'Test Session', platform: 'test', type: 'group' },
members: [],
messages: [],
sync: { hasMore: false, nextSince: 101 },
})
const emptyRetryPage3 = writeTempJson({
chatlab: { version: '0.0.2', exportedAt: 100 },
meta: { name: 'Test Session', platform: 'test', type: 'group' },
members: [],
messages: [],
sync: { hasMore: false, nextSince: 101 },
})
const sessionUpdates: Array<{ sessionId: string; updates: Partial<ImportSession> }> = []
const pullResults: Array<{ status: 'success' | 'error'; detail: string }> = []
const engine = createEngine({
files: [emptyInitialPage, retryPage, emptyTerminalPage, emptyRetryPage1, emptyRetryPage2, emptyRetryPage3],
dataSource,
sessionUpdates,
pullResults,
importResult: {
success: false,
newMessageCount: 0,
sessionId: session.targetSessionId,
error: 'retry import failed',
},
})
const result = await withImmediateTimers(() => engine.executePullSession(dataSource.id, dataSource, session))
assert.equal(result.success, false)
assert.equal(result.error, 'retry import failed')
assert.equal(pullResults.at(-1)?.status, 'error')
assert.equal(sessionUpdates.at(-1)?.updates.lastStatus, 'error')
})
})
+150 -11
View File
@@ -18,10 +18,11 @@ import type {
FetchParams,
SyncMeta,
PullSessionResult,
PullProgress,
} from './types'
import type { DataSourceManager } from './data-source-manager'
const MAX_PAGES_PER_PULL = 50
const MAX_PAGES_PER_PULL = 5000
const PULL_OVERLAP_SECONDS = 60
// ==================== Helpers ====================
@@ -31,7 +32,6 @@ export function buildPullUrl(baseUrl: string, remoteSessionId: string, params: F
const qs: string[] = ['format=chatlab']
if (params.since !== undefined && params.since > 0) qs.push(`since=${params.since}`)
if (params.offset !== undefined && params.offset > 0) qs.push(`offset=${params.offset}`)
if (params.end !== undefined && params.end > 0) qs.push(`end=${params.end}`)
if (params.limit !== undefined && params.limit > 0) qs.push(`limit=${params.limit}`)
return base + '?' + qs.join('&')
}
@@ -77,6 +77,31 @@ export function parseSyncFromFile(filePath: string): SyncMeta | null {
}
}
function fileContainsMessages(filePath: string): boolean {
try {
if (filePath.endsWith('.jsonl')) {
const content = fs.readFileSync(filePath, 'utf-8')
for (const line of content.split('\n')) {
const trimmed = line.trim()
if (!trimmed) continue
try {
const obj = JSON.parse(trimmed)
if (obj._type === 'message') return true
} catch {
continue
}
}
return false
}
const raw = fs.readFileSync(filePath, 'utf-8')
const parsed = JSON.parse(raw)
return Array.isArray(parsed.messages) && parsed.messages.length > 0
} catch {
return false
}
}
function cleanupTempFile(filePath: string): void {
try {
if (fs.existsSync(filePath)) fs.unlinkSync(filePath)
@@ -105,6 +130,7 @@ export class PullEngine {
private logger: SyncLogger
private isImporting: () => boolean
private pullingSourceIds = new Set<string>()
private progressMap = new Map<string, PullProgress>()
constructor(options: PullEngineOptions) {
this.fetcher = options.fetcher
@@ -115,6 +141,10 @@ export class PullEngine {
this.isImporting = options.isImporting ?? (() => false)
}
getProgress(): PullProgress[] {
return Array.from(this.progressMap.values())
}
private async importTempFile(
baseUrl: string,
sess: ImportSession,
@@ -140,6 +170,12 @@ export class PullEngine {
}
async executePullSession(sourceId: string, ds: DataSource, sess: ImportSession): Promise<PullSessionResult> {
const currentDs = this.dsManager.get(sourceId)
if (!currentDs || !currentDs.sessions.some((s) => s.id === sess.id)) {
this.logger.info(`[Pull] Skipping "${sess.name}": session no longer exists`)
return { success: true, newMessageCount: 0 }
}
if (this.isImporting()) {
this.logger.info(`[Pull] Skipping "${sess.name}": import in progress`)
return { success: false, newMessageCount: 0, error: 'Import in progress' }
@@ -149,30 +185,118 @@ export class PullEngine {
let totalNewMessages = 0
let since = sess.lastPullAt
let offset = 0
let end: number | undefined
let pageCount = 0
let resyncAttempted = false
this.progressMap.set(sess.id, { sessionId: sess.id, sessionName: sess.name, current: 0, pages: 0, done: false })
try {
while (pageCount < MAX_PAGES_PER_PULL) {
pageCount++
const tempFile = await this.fetcher.fetchToTempFile(ds.baseUrl, sess.remoteSessionId, ds.token, {
since,
offset,
end,
limit: ds.pullLimit,
})
try {
const stat = fs.statSync(tempFile)
this.logger.info(`[Pull] "${sess.name}" page ${pageCount}: fetched ${stat.size} bytes`)
const sync0 = parseSyncFromFile(tempFile)
if (stat.size < 1024) {
if (!fileContainsMessages(tempFile) && sync0?.hasMore === false) {
cleanupTempFile(tempFile)
const retryDelays = [2000, 3000, 5000]
let retrySuccess = false
let retryHasMore = false
for (let ri = 0; ri < retryDelays.length; ri++) {
this.logger.info(
`[Pull] "${sess.name}" page ${pageCount} got empty response, retry ${ri + 1}/${retryDelays.length} after ${retryDelays[ri]}ms`
)
await new Promise((r) => setTimeout(r, retryDelays[ri]))
const retryFile = await this.fetcher.fetchToTempFile(ds.baseUrl, sess.remoteSessionId, ds.token, {
since,
limit: ds.pullLimit,
})
const retryStat = fs.statSync(retryFile)
this.logger.info(`[Pull] "${sess.name}" retry ${ri + 1}: fetched ${retryStat.size} bytes`)
const retrySync = parseSyncFromFile(retryFile)
if (retryStat.size < 1024 && !fileContainsMessages(retryFile) && retrySync?.hasMore === false) {
cleanupTempFile(retryFile)
continue
}
const retryResult = await this.importTempFile(ds.baseUrl, sess, retryFile)
cleanupTempFile(retryFile)
if (retryResult.needFullResync && !resyncAttempted) {
resyncAttempted = true
this.logger.info(`[Pull] Resetting since=0 for "${sess.name}" full resync`)
since = 0
pageCount = 0
sess.targetSessionId = ''
sess.lastPullAt = 0
this.dsManager.updateSession(sourceId, sess.id, { targetSessionId: '', lastPullAt: 0 })
retrySuccess = true
retryHasMore = true
break
}
if (retryResult.needFullResync) {
const errMsg = 'Full resync failed'
this.logger.error(`[Pull] Full resync already attempted for "${sess.name}", aborting`)
this.dsManager.updateSession(sourceId, sess.id, {
lastPullAt: Math.floor(Date.now() / 1000),
lastStatus: 'error',
lastError: errMsg,
})
this.notifier.onPullResult(sourceId, sess.id, 'error', errMsg)
this.markProgressDone(sess.id)
return { success: false, newMessageCount: 0, error: errMsg }
}
if (!retryResult.success) {
const errMsg = retryResult.error || 'Import failed'
this.dsManager.updateSession(sourceId, sess.id, {
lastPullAt: Math.floor(Date.now() / 1000),
lastStatus: 'error',
lastError: errMsg,
})
this.notifier.onPullResult(sourceId, sess.id, 'error', errMsg)
this.markProgressDone(sess.id)
return { success: false, newMessageCount: 0, error: errMsg }
}
if (retryResult.success && retryResult.sessionId && !sess.targetSessionId) {
sess.targetSessionId = retryResult.sessionId
this.dsManager.updateSession(sourceId, sess.id, { targetSessionId: retryResult.sessionId })
}
totalNewMessages += retryResult.newMessageCount
this.progressMap.set(sess.id, {
sessionId: sess.id,
sessionName: sess.name,
current: totalNewMessages,
pages: pageCount,
done: false,
})
if (retrySync?.hasMore && retrySync.nextSince !== undefined) {
since = retrySync.nextSince
}
retryHasMore = !!retrySync?.hasMore
retrySuccess = true
break
}
if (!retrySuccess) break
if (!retryHasMore) break
continue
}
}
if (stat.size === 0) {
cleanupTempFile(tempFile)
break
}
const sync = parseSyncFromFile(tempFile)
const sync = sync0
const result = await this.importTempFile(ds.baseUrl, sess, tempFile)
cleanupTempFile(tempFile)
@@ -180,7 +304,6 @@ export class PullEngine {
resyncAttempted = true
this.logger.info(`[Pull] Resetting since=0 for "${sess.name}" full resync`)
since = 0
offset = 0
pageCount = 0
sess.targetSessionId = ''
sess.lastPullAt = 0
@@ -197,6 +320,7 @@ export class PullEngine {
lastError: errMsg,
})
this.notifier.onPullResult(sourceId, sess.id, 'error', errMsg)
this.markProgressDone(sess.id)
return { success: false, newMessageCount: 0, error: errMsg }
}
@@ -208,6 +332,7 @@ export class PullEngine {
lastError: errMsg,
})
this.notifier.onPullResult(sourceId, sess.id, 'error', errMsg)
this.markProgressDone(sess.id)
return { success: false, newMessageCount: 0, error: errMsg }
}
@@ -217,13 +342,17 @@ export class PullEngine {
}
totalNewMessages += result.newMessageCount
this.progressMap.set(sess.id, {
sessionId: sess.id,
sessionName: sess.name,
current: totalNewMessages,
pages: pageCount,
done: false,
})
if (!sync || !sync.hasMore) break
if (sync.nextSince !== undefined) since = sync.nextSince
if (sync.nextOffset !== undefined) offset = sync.nextOffset
else offset = 0
if (sync.watermark !== undefined && !end) end = sync.watermark
} catch (importErr) {
cleanupTempFile(tempFile)
throw importErr
@@ -242,6 +371,7 @@ export class PullEngine {
})
if (totalNewMessages > 0) this.notifier.onSessionListChanged()
this.notifier.onPullResult(sourceId, sess.id, 'success', `+${totalNewMessages} messages`)
this.markProgressDone(sess.id)
return { success: true, newMessageCount: totalNewMessages }
} catch (error: any) {
const errMsg = error.message || 'Pull failed'
@@ -252,10 +382,19 @@ export class PullEngine {
lastError: errMsg,
})
this.notifier.onPullResult(sourceId, sess.id, 'error', errMsg)
this.markProgressDone(sess.id)
return { success: false, newMessageCount: 0, error: errMsg }
}
}
private markProgressDone(sessionId: string): void {
const p = this.progressMap.get(sessionId)
if (p) {
p.done = true
setTimeout(() => this.progressMap.delete(sessionId), 5000)
}
}
async pullAllSessions(ds: DataSource): Promise<void> {
if (this.pullingSourceIds.has(ds.id)) {
this.logger.info(`[Pull] Skipping pullAllSessions for "${ds.baseUrl}": pull already in progress`)
+8
View File
@@ -101,6 +101,14 @@ export interface PullSessionResult {
error?: string
}
export interface PullProgress {
sessionId: string
sessionName: string
current: number
pages: number
done: boolean
}
/**
* Downloads remote data to a temporary file.
* Platform implementations: Electron uses `net.request`, Node.js uses `fetch`.
+1
View File
@@ -394,6 +394,7 @@ watch(
size="xs"
type="element"
:target-element="conversationContentRef"
markdown-fix
/>
</div>
@@ -486,6 +486,7 @@ async function handleCopyMarkdown() {
size="xs"
type="element"
target-selector=".qa-pair"
markdown-fix
/>
</div>
</div>
@@ -13,6 +13,8 @@ const props = defineProps<{
sessionId: string
/** 弹窗打开状态(v-model */
modelValue?: boolean
/** 会话消息总数,为 0 时不自动弹出 */
messageCount?: number
}>()
const emit = defineEmits<{
@@ -47,6 +49,8 @@ const canClose = computed(() => {
// 检查会话索引状态并自动弹出
async function checkAndAutoOpen() {
if (!props.sessionId) return
// 没有消息的会话无需索引,跳过自动弹出
if (props.messageCount === 0) return
isLoading.value = true
try {
@@ -93,8 +97,6 @@ async function generateSessionIndex() {
hasIndex.value = true
sessionCount.value = count
emit('generated', count)
// 生成完成后自动关闭
forceMode.value = false
isOpen.value = false
} catch (error) {
+3
View File
@@ -24,6 +24,8 @@ const props = withDefaults(
targetElement?: HTMLElement | null
/** 当 type='element' 时,从按钮向上查找目标元素的选择器 */
targetSelector?: string
/** 是否应用 Markdown 列表渲染兼容修复(仅截取 Markdown 内容时传 true */
markdownFix?: boolean
}>(),
{
size: 'sm',
@@ -48,6 +50,7 @@ async function handleCapture(event: Event) {
const defaultOptions = {
hideSelectors: [`#${buttonId.value}`],
mobileWidth: screenshotMobileAdapt.value ? true : undefined,
markdownFix: props.markdownFix || undefined,
}
if (props.type === 'page') {
@@ -43,7 +43,8 @@ const hasDiscoveryRun = ref(false)
type SessionTypeFilter = 'all' | SessionTypeSelection
const activeSessionTypeFilter = ref<SessionTypeFilter>('all')
const discoveryPageSize = 100
const DISCOVERY_PAGE_SIZE = 200
const DISCOVERY_MAX_NO_PAGE = 5000
watch(
() => props.open,
@@ -182,13 +183,27 @@ async function fetchDiscoveryPage(options: { append: boolean; resetSelection: bo
try {
const result = await store.fetchRemoteSessions(formData.value.baseUrl, formData.value.token, {
keyword: discoveryKeyword.value.trim() || undefined,
limit: discoveryPageSize,
limit: DISCOVERY_PAGE_SIZE,
cursor: options.append ? discoveryNextCursor.value : undefined,
})
remoteSessions.value = options.append ? mergeRemoteSessions(remoteSessions.value, result.sessions) : result.sessions
discoveryHasMore.value = Boolean(result.page?.hasMore)
discoveryNextCursor.value = result.page?.nextCursor
// Server doesn't support pagination and returned exactly `limit` items — likely truncated.
// Re-fetch with a much larger limit to get all sessions at once.
if (!options.append && !result.page && result.sessions.length >= DISCOVERY_PAGE_SIZE) {
const fullResult = await store.fetchRemoteSessions(formData.value.baseUrl, formData.value.token, {
keyword: discoveryKeyword.value.trim() || undefined,
limit: DISCOVERY_MAX_NO_PAGE,
})
remoteSessions.value = fullResult.sessions
discoveryHasMore.value = false
discoveryNextCursor.value = undefined
} else {
remoteSessions.value = options.append
? mergeRemoteSessions(remoteSessions.value, result.sessions)
: result.sessions
discoveryHasMore.value = Boolean(result.page?.hasMore)
discoveryNextCursor.value = result.page?.nextCursor
}
} catch (err: any) {
discoveryError.value = err.message || t('settings.api.dataSources.discovery.error')
} finally {
+103 -20
View File
@@ -10,7 +10,7 @@ import McpSettingsTab from './McpSettingsTab.vue'
const { t, locale } = useI18n()
const store = useApiServerStore()
const { config, status, loading, isRunning, hasError, isPortInUse, dataSources, pullingId, isWebMode } =
const { config, status, loading, isRunning, hasError, isPortInUse, dataSources, pullingIds, syncProgress, isWebMode } =
storeToRefs(store)
const activeSubTab = ref('sync')
@@ -45,6 +45,8 @@ const showManageModal = ref(false)
const managingDataSource = ref<DataSource | null>(null)
const showDeleteModal = ref(false)
const deletingDataSource = ref<DataSource | null>(null)
const showDeleteSessionModal = ref(false)
const deletingSession = ref<{ ds: DataSource; sess: ImportSession } | null>(null)
let unlistenStartupError: (() => void) | null = null
let unlistenPullResult: (() => void) | null = null
@@ -175,8 +177,25 @@ async function removeSource() {
deletingDataSource.value = null
}
async function removeSession(ds: DataSource, sess: ImportSession) {
await store.removeImportSession(ds.id, sess.id)
function confirmRemoveSession(ds: DataSource, sess: ImportSession) {
deletingSession.value = { ds, sess }
showDeleteSessionModal.value = true
}
async function removeSessionKeepData() {
if (!deletingSession.value) return
const { ds, sess } = deletingSession.value
await store.removeImportSession(ds.id, sess.id, false)
showDeleteSessionModal.value = false
deletingSession.value = null
}
async function removeSessionDeleteData() {
if (!deletingSession.value) return
const { ds, sess } = deletingSession.value
await store.removeImportSession(ds.id, sess.id, true)
showDeleteSessionModal.value = false
deletingSession.value = null
}
async function syncAllInSource(ds: DataSource) {
@@ -248,10 +267,27 @@ function subscribedRemoteIds(ds: DataSource): Set<string> {
<UButton size="xs" variant="ghost" @click="toggleSourceEnabled(ds)">
<UIcon :name="ds.enabled ? 'i-heroicons-pause' : 'i-heroicons-play'" class="h-3.5 w-3.5" />
</UButton>
<UButton size="xs" variant="ghost" @click="syncAllInSource(ds)">
<UIcon name="i-heroicons-arrow-path" class="h-3.5 w-3.5" />
<UButton
size="xs"
variant="ghost"
:disabled="pullingIds.has(ds.id) || ds.sessions.some((s) => pullingIds.has(s.id))"
@click="syncAllInSource(ds)"
>
<UIcon
name="i-heroicons-arrow-path"
class="h-3.5 w-3.5"
:class="{
'animate-spin': pullingIds.has(ds.id) || ds.sessions.some((s) => pullingIds.has(s.id)),
}"
/>
</UButton>
<UButton size="xs" variant="ghost" color="error" @click="confirmDeleteSource(ds)">
<UButton
size="xs"
variant="ghost"
color="error"
:disabled="ds.sessions.some((s) => pullingIds.has(s.id))"
@click="confirmDeleteSource(ds)"
>
<UIcon name="i-heroicons-trash" class="h-3.5 w-3.5" />
</UButton>
</div>
@@ -269,11 +305,13 @@ function subscribedRemoteIds(ds: DataSource): Set<string> {
<span
class="inline-block h-2 w-2 rounded-full"
:class="
sess.lastStatus === 'success'
? 'bg-green-500'
: sess.lastStatus === 'error'
? 'bg-red-500'
: 'bg-gray-400'
pullingIds.has(sess.id)
? 'animate-pulse bg-blue-500'
: sess.lastStatus === 'success'
? 'bg-green-500'
: sess.lastStatus === 'error'
? 'bg-red-500'
: 'bg-gray-400'
"
></span>
<span class="text-sm font-medium text-gray-900 dark:text-white">{{ sess.name }}</span>
@@ -282,22 +320,44 @@ function subscribedRemoteIds(ds: DataSource): Set<string> {
<UButton
size="xs"
variant="ghost"
:loading="pullingId === sess.id"
:disabled="pullingIds.has(sess.id)"
@click="syncSession(ds, sess)"
>
<UIcon name="i-heroicons-arrow-path" class="h-3.5 w-3.5" />
<UIcon
name="i-heroicons-arrow-path"
class="h-3.5 w-3.5"
:class="{ 'animate-spin': pullingIds.has(sess.id) }"
/>
</UButton>
<UButton size="xs" variant="ghost" color="error" @click="removeSession(ds, sess)">
<UButton
size="xs"
variant="ghost"
color="error"
:disabled="pullingIds.has(sess.id)"
@click="confirmRemoveSession(ds, sess)"
>
<UIcon name="i-heroicons-trash" class="h-3.5 w-3.5" />
</UButton>
</div>
</div>
<div class="mt-1 flex items-center gap-2 text-xs text-gray-500 dark:text-gray-400">
<span>
{{ t('settings.api.dataSources.every') }} {{ ds.intervalMinutes }}
{{ t('settings.api.dataSources.minutes') }}
</span>
<template v-if="sess.lastPullAt">
<template v-if="syncProgress.get(sess.id)">
<span class="text-blue-500">
{{ t('settings.api.dataSources.syncing') }}...
{{ syncProgress.get(sess.id)!.current }}
{{ t('settings.api.dataSources.messages') }}
</span>
</template>
<template v-else-if="pullingIds.has(sess.id) && !syncProgress.get(sess.id)">
<span class="text-blue-500">{{ t('settings.api.dataSources.syncing') }}...</span>
</template>
<template v-else>
<span>
{{ t('settings.api.dataSources.every') }} {{ ds.intervalMinutes }}
{{ t('settings.api.dataSources.minutes') }}
</span>
</template>
<template v-if="!pullingIds.has(sess.id) && sess.lastPullAt">
<span class="text-gray-300 dark:text-gray-600">·</span>
<span class="text-gray-400">
{{ t('settings.api.dataSources.lastSync') }}: {{ formatTime(sess.lastPullAt) }}
@@ -530,7 +590,7 @@ function subscribedRemoteIds(ds: DataSource): Set<string> {
@sessions-added="handleSessionsAdded"
/>
<!-- Delete confirmation modal -->
<!-- Delete data source confirmation modal -->
<UModal v-model:open="showDeleteModal" :ui="{ content: 'z-[101]', overlay: 'z-[100]' }">
<template #content>
<div class="p-4">
@@ -547,5 +607,28 @@ function subscribedRemoteIds(ds: DataSource): Set<string> {
</div>
</template>
</UModal>
<!-- Delete session confirmation modal -->
<UModal v-model:open="showDeleteSessionModal" :ui="{ content: 'z-[101]', overlay: 'z-[100]' }">
<template #content>
<div class="p-4">
<h3 class="mb-3 font-semibold text-gray-900 dark:text-white">
{{ t('settings.api.dataSources.deleteSessionConfirm.title') }}
</h3>
<p class="mb-4 text-sm text-gray-600 dark:text-gray-400">
{{ t('settings.api.dataSources.deleteSessionConfirm.message', { name: deletingSession?.sess.name }) }}
</p>
<div class="flex justify-end gap-2">
<UButton variant="soft" @click="showDeleteSessionModal = false">{{ t('common.cancel') }}</UButton>
<UButton variant="soft" @click="removeSessionKeepData">
{{ t('settings.api.dataSources.deleteSessionConfirm.keepData') }}
</UButton>
<UButton color="error" @click="removeSessionDeleteData">
{{ t('settings.api.dataSources.deleteSessionConfirm.deleteData') }}
</UButton>
</div>
</div>
</template>
</UModal>
</div>
</template>
+42 -39
View File
@@ -27,6 +27,8 @@ export interface ScreenCaptureOptions {
* - 传入 false 或不传:不进行移动端适配
*/
mobileWidth?: number | boolean
/** 是否应用 Markdown 列表渲染兼容修复(仅截取 Markdown 内容时需要,默认 false */
markdownFix?: boolean
}
/**
@@ -324,8 +326,7 @@ export function useScreenCapture() {
})
// 修复 Markdown 列表元素在 @zumer/snapdom 中的渲染问题
// BUG: @zumer/snapdom 在处理 <ol>/<ul> 的 list-style 时会产生额外的黑色边框和位置偏移
// 解决方案:临时移除 list-style,手动添加数字/圆点作为文本前缀
// 仅在显式启用 markdownFix 时应用,避免影响自定义列表样式(如 Changelog 圆点)
const listElements: {
el: HTMLElement
originalStyles: {
@@ -338,45 +339,47 @@ export function useScreenCapture() {
}
addedPrefixes: HTMLSpanElement[]
}[] = []
const lists = element.querySelectorAll('ol, ul')
lists.forEach((list) => {
const htmlEl = list as HTMLElement
const isOrdered = htmlEl.tagName.toLowerCase() === 'ol'
const addedPrefixes: HTMLSpanElement[] = []
if (options?.markdownFix) {
const lists = element.querySelectorAll('ol, ul')
lists.forEach((list) => {
const htmlEl = list as HTMLElement
const isOrdered = htmlEl.tagName.toLowerCase() === 'ol'
const addedPrefixes: HTMLSpanElement[] = []
listElements.push({
el: htmlEl,
originalStyles: {
listStyleType: htmlEl.style.listStyleType,
paddingLeft: htmlEl.style.paddingLeft,
marginLeft: htmlEl.style.marginLeft,
border: htmlEl.style.border,
outline: htmlEl.style.outline,
boxShadow: htmlEl.style.boxShadow,
},
addedPrefixes,
listElements.push({
el: htmlEl,
originalStyles: {
listStyleType: htmlEl.style.listStyleType,
paddingLeft: htmlEl.style.paddingLeft,
marginLeft: htmlEl.style.marginLeft,
border: htmlEl.style.border,
outline: htmlEl.style.outline,
boxShadow: htmlEl.style.boxShadow,
},
addedPrefixes,
})
// 移除列表样式以避免 @zumer/snapdom 渲染问题
htmlEl.style.listStyleType = 'none'
htmlEl.style.paddingLeft = '0'
htmlEl.style.marginLeft = '0'
// 移除边框以修复 @zumer/snapdom 的黑色边框 bug
htmlEl.style.border = 'none'
htmlEl.style.outline = 'none'
htmlEl.style.boxShadow = 'none'
// 为每个 li 添加手动前缀
const lis = htmlEl.querySelectorAll(':scope > li')
lis.forEach((li, index) => {
const prefix = document.createElement('span')
prefix.className = '__screen-capture-list-prefix__'
prefix.style.cssText = 'display: inline-block; min-width: 1.5em; margin-right: 0.25em; text-align: right;'
prefix.textContent = isOrdered ? `${index + 1}.` : '•'
li.insertBefore(prefix, li.firstChild)
addedPrefixes.push(prefix)
})
})
// 移除列表样式以避免 @zumer/snapdom 渲染问题
htmlEl.style.listStyleType = 'none'
htmlEl.style.paddingLeft = '0'
htmlEl.style.marginLeft = '0'
// 移除边框以修复 @zumer/snapdom 的黑色边框 bug
htmlEl.style.border = 'none'
htmlEl.style.outline = 'none'
htmlEl.style.boxShadow = 'none'
// 为每个 li 添加手动前缀
const lis = htmlEl.querySelectorAll(':scope > li')
lis.forEach((li, index) => {
const prefix = document.createElement('span')
prefix.className = '__screen-capture-list-prefix__'
prefix.style.cssText = 'display: inline-block; min-width: 1.5em; margin-right: 0.25em; text-align: right;'
prefix.textContent = isOrdered ? `${index + 1}.` : '•'
li.insertBefore(prefix, li.firstChild)
addedPrefixes.push(prefix)
})
})
}
// 清理可能导致 URI malformed 错误的特殊字符(孤立的 Unicode 代理对)
const textNodesBackup: { node: Text; originalText: string }[] = []
+2 -1
View File
@@ -135,7 +135,8 @@
"refactor": "Refactors",
"docs": "Docs",
"chore": "Other",
"style": "Style"
"style": "Style",
"ci": "CI"
}
}
}
+8
View File
@@ -494,6 +494,8 @@
"every": "Every",
"minutes": "min",
"lastSync": "Last sync",
"syncing": "Syncing",
"messages": "messages",
"addBtn": "Add Data Source",
"form": {
"modalTitle": "Add Data Source",
@@ -544,6 +546,12 @@
"deleteConfirm": {
"title": "Delete Data Source",
"message": "Are you sure you want to delete \"{name}\"? All import configurations under this data source will be removed. Already imported local conversations will not be deleted."
},
"deleteSessionConfirm": {
"title": "Delete Subscription",
"message": "Are you sure you want to delete \"{name}\"?",
"keepData": "Remove subscription only",
"deleteData": "Also delete imported chats"
}
},
"usage": {
+2 -1
View File
@@ -135,7 +135,8 @@
"refactor": "リファクタリング",
"docs": "ドキュメント更新",
"chore": "その他の改善",
"style": "スタイル改善"
"style": "スタイル改善",
"ci": "CI/ビルド"
}
}
}
+8
View File
@@ -493,6 +493,8 @@
"every": "",
"minutes": "分ごと",
"lastSync": "最終同期",
"syncing": "同期中",
"messages": "件",
"addBtn": "データソースを追加",
"form": {
"modalTitle": "データソースを追加",
@@ -544,6 +546,12 @@
"deleteConfirm": {
"title": "データソースを削除",
"message": "データソース「{name}」を削除してもよろしいですか?このデータソース下のすべてのインポート設定が削除されます。既にインポート済みのローカル会話は削除されません。"
},
"deleteSessionConfirm": {
"title": "サブスクリプションを削除",
"message": "「{name}」を削除してもよろしいですか?",
"keepData": "サブスクリプションのみ削除",
"deleteData": "インポートしたチャットも削除"
}
},
"usage": {
+2 -1
View File
@@ -135,7 +135,8 @@
"refactor": "重构优化",
"docs": "文档更新",
"chore": "其他优化",
"style": "样式优化"
"style": "样式优化",
"ci": "CI/构建"
}
}
}
+8
View File
@@ -494,6 +494,8 @@
"every": "每",
"minutes": "分钟",
"lastSync": "上次同步",
"syncing": "同步中",
"messages": "条消息",
"addBtn": "添加数据源",
"form": {
"modalTitle": "添加数据源",
@@ -544,6 +546,12 @@
"deleteConfirm": {
"title": "删除数据源",
"message": "确认删除数据源「{name}」?该数据源下的所有导入配置将被移除。已导入的本地对话不会被删除。"
},
"deleteSessionConfirm": {
"title": "删除订阅",
"message": "确认删除订阅「{name}」?",
"keepData": "仅删除订阅",
"deleteData": "同时删除导入的聊天"
}
},
"usage": {
+2 -1
View File
@@ -135,7 +135,8 @@
"refactor": "重構優化",
"docs": "文件更新",
"chore": "其他優化",
"style": "樣式優化"
"style": "樣式優化",
"ci": "CI/建置"
}
}
}
+8
View File
@@ -493,6 +493,8 @@
"every": "每",
"minutes": "分鐘",
"lastSync": "上次同步",
"syncing": "同步中",
"messages": "則訊息",
"addBtn": "新增資料來源",
"form": {
"modalTitle": "新增資料來源",
@@ -544,6 +546,12 @@
"deleteConfirm": {
"title": "刪除資料來源",
"message": "確認刪除資料來源「{name}」?該資料來源下的所有匯入配置將被移除。已匯入的本機對話不會被刪除。"
},
"deleteSessionConfirm": {
"title": "刪除訂閱",
"message": "確認刪除訂閱「{name}」?",
"keepData": "僅刪除訂閱",
"deleteData": "同時刪除匯入的聊天"
}
},
"usage": {
+6 -1
View File
@@ -255,7 +255,12 @@ const { headerDescription } = useSessionHeaderDescription({
</div>
<!-- 会话索引弹窗内部自动检测并弹出 -->
<SessionIndexModal v-if="currentSessionId" v-model="showSessionIndexModal" :session-id="currentSessionId" />
<SessionIndexModal
v-if="currentSessionId && session && session.messageCount > 0"
v-model="showSessionIndexModal"
:session-id="currentSessionId"
:message-count="session.messageCount"
/>
<!-- 增量导入弹窗 -->
<IncrementalImportModal
+19 -1
View File
@@ -8,6 +8,7 @@ import { sanitizeSummary } from '@/utils/sanitizeSummary'
import { getChangeTypeConfig } from './changelogTypeConfig'
import { IS_ELECTRON } from '@/utils/platform'
import { usePlatformService } from '@/services'
import CaptureButton from '@/components/common/CaptureButton.vue'
const { t } = useI18n()
const settingsStore = useSettingsStore()
@@ -23,6 +24,7 @@ const loadError = ref<string | null>(null)
// 展开的版本
// 使用 Map 来跟踪每个版本的展开状态,undefined 表示使用默认状态
const expandedState = ref<Map<string, boolean>>(new Map())
const hoveredVersion = ref<string | null>(null)
// 当前软件版本(用于高亮显示和默认展开)
const currentAppVersion = ref<string | null>(null)
@@ -134,6 +136,7 @@ function getChangeTypeLabel(type: string) {
docs: t('home.changelog.types.docs'),
chore: t('home.changelog.types.chore'),
style: t('home.changelog.types.style'),
ci: t('home.changelog.types.ci'),
}
return labels[type] || type
}
@@ -306,7 +309,14 @@ defineExpose({ open, openWithData, close, fetchChangelogs, getLatestVersion })
<!-- Changelog List -->
<div v-else class="space-y-6">
<div v-for="(log, index) in changelogs" :key="log.version" class="relative">
<div
v-for="(log, index) in changelogs"
:key="log.version"
class="relative group"
data-changelog-entry
@mouseenter="hoveredVersion = log.version"
@mouseleave="hoveredVersion = null"
>
<!-- Timeline line -->
<div
v-if="index < changelogs.length - 1"
@@ -400,6 +410,14 @@ defineExpose({ open, openWithData, close, fetchChangelogs, getLatestVersion })
</div>
</div>
</div>
<!-- 截屏按钮展开 + hover 时显示 -->
<CaptureButton
v-if="isExpanded(log.version, index) && hoveredVersion === log.version"
class="absolute right-0 top-0 z-20"
size="xs"
type="element"
target-selector="[data-changelog-entry]"
/>
</div>
</div>
</div>
@@ -38,6 +38,11 @@ export const CHANGELOG_TYPE_CONFIG: Record<string, ChangeTypeConfigItem> = {
color: 'text-blue-500',
bgColor: 'bg-blue-100 dark:bg-blue-900/30',
},
ci: {
icon: 'i-heroicons-command-line',
color: 'text-teal-500',
bgColor: 'bg-teal-100 dark:bg-teal-900/30',
},
}
export function getChangeTypeConfig(type: string): ChangeTypeConfigItem | undefined {
+6 -1
View File
@@ -266,7 +266,12 @@ const otherMemberAvatar = computed(() => {
</div>
<!-- 会话索引弹窗内部自动检测并弹出 -->
<SessionIndexModal v-if="currentSessionId" v-model="showSessionIndexModal" :session-id="currentSessionId" />
<SessionIndexModal
v-if="currentSessionId && session && session.messageCount > 0"
v-model="showSessionIndexModal"
:session-id="currentSessionId"
:message-count="session.messageCount"
/>
<!-- 增量导入弹窗 -->
<IncrementalImportModal
+4
View File
@@ -16,6 +16,10 @@ export class ElectronSessionIndexAdapter implements SessionIndexAdapter {
return window.sessionApi.generate(sessionId, gapThreshold)
}
generateIncremental(sessionId: string, gapThreshold?: number): Promise<number> {
return window.sessionApi.generateIncremental(sessionId, gapThreshold)
}
hasIndex(sessionId: string): Promise<boolean> {
return window.sessionApi.hasIndex(sessionId)
}
+11
View File
@@ -33,6 +33,17 @@ export class FetchSessionIndexAdapter implements SessionIndexAdapter {
return result.sessionCount
}
async generateIncremental(sessionId: string, gapThreshold: number = 1800): Promise<number> {
const resp = await fetch(`/_web/sessions/${sessionId}/generate-incremental-index`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ gapThreshold }),
})
if (!resp.ok) throw new Error(`Failed to generate incremental session index: ${resp.status}`)
const result = (await resp.json()) as { sessionCount: number }
return result.sessionCount
}
async hasIndex(sessionId: string): Promise<boolean> {
const stats = await this.getStats(sessionId)
return stats.hasIndex
+1
View File
@@ -39,6 +39,7 @@ export interface CanGenerateInfo {
export interface SessionIndexAdapter {
generate(sessionId: string, gapThreshold?: number): Promise<number>
generateIncremental(sessionId: string, gapThreshold?: number): Promise<number>
hasIndex(sessionId: string): Promise<boolean>
getStats(sessionId: string): Promise<SessionStats>
clear(sessionId: string): Promise<boolean>
+109 -20
View File
@@ -9,6 +9,9 @@
import { ref, computed } from 'vue'
import { defineStore } from 'pinia'
import { IS_ELECTRON } from '@/utils/platform'
import { useSessionStore } from './session'
import { useSessionIndexService } from '@/services/session-index/service'
import { getSessionGapThreshold } from '@/composables/useUiConfig'
export interface ApiServerConfig {
enabled: boolean
@@ -93,7 +96,7 @@ interface ApiTransport {
sourceId: string,
sessions: Array<{ name: string; remoteSessionId: string }>
): Promise<ImportSession[]>
removeImportSession(sourceId: string, sessionId: string): Promise<boolean>
removeImportSession(sourceId: string, sessionId: string, deleteData?: boolean): Promise<boolean>
triggerPull(sourceId: string, sessionId?: string): Promise<{ success: boolean; error?: string }>
triggerPullAll(sourceId: string): Promise<{ success: boolean; error?: string }>
onPullResult(cb: () => void): () => void
@@ -118,7 +121,7 @@ function createElectronTransport(): ApiTransport {
updateDataSource: (id, updates) => api.updateDataSource(id, updates),
deleteDataSource: (id) => api.deleteDataSource(id),
addImportSessions: (sourceId, sessions) => api.addImportSessions(sourceId, sessions),
removeImportSession: (sourceId, sessionId) => api.removeImportSession(sourceId, sessionId),
removeImportSession: (sourceId, sessionId, deleteData?) => api.removeImportSession(sourceId, sessionId, deleteData),
triggerPull: (sourceId, sessionId?) => api.triggerPull(sourceId, sessionId),
triggerPullAll: (sourceId) => api.triggerPullAll(sourceId),
onPullResult: (cb) => api.onPullResult(cb),
@@ -196,9 +199,10 @@ function createWebTransport(): ApiTransport {
body: JSON.stringify({ sessions }),
}),
removeImportSession: async (sourceId, sessionId) => {
removeImportSession: async (sourceId, sessionId, deleteData?) => {
const qs = deleteData ? '?deleteData=true' : ''
const result = await fetchJson<{ success: boolean }>(
`/_web/automation/data-sources/${sourceId}/sessions/${sessionId}`,
`/_web/automation/data-sources/${sourceId}/sessions/${sessionId}${qs}`,
{ method: 'DELETE' }
)
return result.success
@@ -254,7 +258,8 @@ export const useApiServerStore = defineStore('apiServer', () => {
const loading = ref(false)
const dataSources = ref<DataSource[]>([])
const pullingId = ref<string | null>(null)
const pullingIds = ref(new Set<string>())
const syncProgress = ref<Map<string, { current: number; pages: number }>>(new Map())
const available = computed(() => true)
const isRunning = computed(() => status.value.running)
@@ -385,7 +390,12 @@ export const useApiServerStore = defineStore('apiServer', () => {
const added = await transport.addImportSessions(sourceId, sessions)
await fetchDataSources()
if (added.length > 0) {
pollDataSourceUpdates(sourceId)
for (const s of added) pullingIds.value.add(s.id)
pullingIds.value = new Set(pullingIds.value)
pollDataSourceUpdates(
sourceId,
added.map((s) => s.id)
)
}
return added
} catch (err) {
@@ -394,23 +404,49 @@ export const useApiServerStore = defineStore('apiServer', () => {
}
}
function pollDataSourceUpdates(sourceId: string, maxAttempts = 10, intervalMs = 3000) {
function pollDataSourceUpdates(sourceId: string, sessionIds: string[], maxAttempts = 24, intervalMs = 5000) {
let attempt = 0
const timer = setInterval(async () => {
attempt++
await fetchDataSources()
await Promise.all([fetchDataSources(), fetchSyncProgress()])
const ds = dataSources.value.find((s) => s.id === sourceId)
const allDone = ds?.sessions.every((s) => s.lastStatus !== 'idle')
if (allDone || attempt >= maxAttempts) {
const pending = sessionIds.filter((id) => {
const sess = ds?.sessions.find((s) => s.id === id)
return sess && sess.lastStatus === 'idle'
})
if (pending.length === 0 || attempt >= maxAttempts) {
for (const id of sessionIds) {
pullingIds.value.delete(id)
syncProgress.value.delete(id)
}
pullingIds.value = new Set(pullingIds.value)
syncProgress.value = new Map(syncProgress.value)
clearInterval(timer)
}
}, intervalMs)
}
async function removeImportSession(sourceId: string, sessionId: string) {
async function fetchSyncProgress() {
if (!isWebMode.value) return
try {
const ok = await transport.removeImportSession(sourceId, sessionId)
if (ok) await fetchDataSources()
const list = await fetch('/_web/automation/sync-progress').then((r) => r.json())
const map = new Map<string, { current: number; pages: number }>()
for (const item of list as Array<{ sessionId: string; current: number; pages: number; done: boolean }>) {
if (!item.done) map.set(item.sessionId, { current: item.current, pages: item.pages })
}
syncProgress.value = map
} catch {
/* ignore */
}
}
async function removeImportSession(sourceId: string, sessionId: string, deleteData?: boolean) {
try {
const ok = await transport.removeImportSession(sourceId, sessionId, deleteData)
if (ok) {
await fetchDataSources()
if (deleteData) useSessionStore().loadSessions()
}
return ok
} catch (err) {
console.error('[ApiServerStore] Failed to remove import session:', err)
@@ -421,39 +457,91 @@ export const useApiServerStore = defineStore('apiServer', () => {
// ==================== 同步 ====================
async function triggerPull(sourceId: string, sessionId?: string) {
pullingId.value = sessionId || sourceId
const trackId = sessionId || sourceId
pullingIds.value.add(trackId)
pullingIds.value = new Set(pullingIds.value)
const progressTimer = startProgressPolling()
try {
const result = await transport.triggerPull(sourceId, sessionId)
await fetchDataSources()
await useSessionStore().loadSessions()
generateIndexForSource(sourceId, sessionId)
return result
} catch (err) {
console.error('[ApiServerStore] Failed to trigger pull:', err)
return { success: false, error: String(err) }
} finally {
pullingId.value = null
clearInterval(progressTimer)
pullingIds.value.delete(trackId)
pullingIds.value = new Set(pullingIds.value)
syncProgress.value.delete(trackId)
syncProgress.value = new Map(syncProgress.value)
}
}
async function triggerPullAll(sourceId: string) {
pullingId.value = sourceId
const ds = dataSources.value.find((s) => s.id === sourceId)
const ids = [sourceId, ...(ds?.sessions.map((s) => s.id) ?? [])]
for (const id of ids) pullingIds.value.add(id)
pullingIds.value = new Set(pullingIds.value)
const progressTimer = startProgressPolling()
try {
const result = await transport.triggerPullAll(sourceId)
await fetchDataSources()
await useSessionStore().loadSessions()
generateIndexForSource(sourceId)
return result
} catch (err) {
console.error('[ApiServerStore] Failed to trigger pull all:', err)
return { success: false, error: String(err) }
} finally {
pullingId.value = null
clearInterval(progressTimer)
for (const id of ids) {
pullingIds.value.delete(id)
syncProgress.value.delete(id)
}
pullingIds.value = new Set(pullingIds.value)
syncProgress.value = new Map(syncProgress.value)
}
}
function startProgressPolling(): ReturnType<typeof setInterval> {
return setInterval(fetchSyncProgress, 3000)
}
function listenPullResult() {
return transport.onPullResult(() => {
fetchDataSources()
return transport.onPullResult(async () => {
await fetchDataSources()
await useSessionStore().loadSessions()
generateIndexForAllSources()
})
}
function generateIndexForSource(sourceId: string, sessionId?: string) {
const ds = dataSources.value.find((s) => s.id === sourceId)
if (!ds) return
const targets = sessionId
? ds.sessions.filter((s) => s.id === sessionId && s.targetSessionId)
: ds.sessions.filter((s) => s.targetSessionId)
const indexService = useSessionIndexService()
const threshold = getSessionGapThreshold()
for (const sess of targets) {
indexService.generateIncremental(sess.targetSessionId, threshold).catch(() => {})
}
}
function generateIndexForAllSources() {
const indexService = useSessionIndexService()
const threshold = getSessionGapThreshold()
for (const ds of dataSources.value) {
for (const sess of ds.sessions) {
if (sess.targetSessionId) {
indexService.generateIncremental(sess.targetSessionId, threshold).catch(() => {})
}
}
}
}
async function fetchRemoteSessions(
baseUrl: string,
token?: string,
@@ -467,7 +555,8 @@ export const useApiServerStore = defineStore('apiServer', () => {
status,
loading,
dataSources,
pullingId,
pullingIds,
syncProgress,
available,
isRunning,
hasError,