Commit Graph
100 Commits
Author SHA1 Message Date
digua d058e8a60f fix(electron): fix CI build by pinning electron version and cleanup
- Pin electron to 35.7.5 (electron-builder requires fixed version)
- Remove redundant @electron/rebuild (built into electron-builder)
- Use electron-builder install-app-deps for postinstall
2026-05-19 19:14:11 +08:00
digua 9099b15ce6 chore(deps): upgrade vite toolchain 2026-05-19 18:39:14 +08:00
digua e80d8e471e docs: update 2026-05-19 00:40:00 +08:00
digua 2bcbb84711 feat: add interactive dev mode selector (pnpm dev)
Adds a zero-dependency Node.js script with arrow-key selection
for choosing between Desktop (Electron), Web (CLI serve + frontend),
Server Only (API backend), and Frontend-only dev modes.

Remembers last selection in node_modules/.cache/dev-mode.
2026-05-19 00:35:00 +08:00
digua b12c07dc32 refactor: extract Electron shell to apps/desktop/
Move Electron-specific code (main, preload, shared) and configs
(electron.vite, electron-builder) into apps/desktop/ to prepare
for multi-platform support (Web Browser standalone).

- Create apps/desktop/ with dedicated package.json and tsconfig
- Split Electron deps (electron, electron-updater, better-sqlite3,
  etc.) from shared frontend deps in root package.json
- Declare better-sqlite3 and @node-rs/jieba in packages/node-runtime
- Add stream-json to packages/parser where it belongs
- Update all path aliases (@electron/* → apps/desktop/*) in tsconfig,
  vite configs, and relative imports in preload/main
- Keep slim root tsconfig.node.json as shared base for packages
- Update CI/CD release workflow for new directory layout
- Fix pre-existing type errors (pi-ai Model generic, SQLLabTab
  LLMChatStreamChunk, filter.test.ts empty methods)
2026-05-19 00:31:00 +08:00
digua ba29346bfe refactor: move chart-* modules from packages/ to src/features/
These are frontend-only Vue/ECharts components that depend on app internals (@/),
not cross-platform shared packages. Moving them to src/features/charts-* keeps
packages/ purely for reusable Node.js libraries.
2026-05-18 23:57:00 +08:00
digua d3ce4023e5 fix(core): add missing re-exports in query/index.ts
The query barrel file was missing re-exports for getSessionsByTimeRange,
getRecentChatSessions, loadSessionMessages, getPrivateChatMemberAvatar,
getExportSessionData, and searchMessagesWithFtsAsync.
2026-05-18 23:43:00 +08:00
digua 31359712b3 refactor(core): extract remaining inline SQL from electron adapters
- Extract getPrivateChatMemberAvatar to @openchatlab/core
- Extract getExportSessionData (meta/members/messages SQL) to @openchatlab/core
- Extract loadSessionMessages for summary generation to @openchatlab/core
- Replace inline SQL in summary buildDeps with shared core query functions
- Extract createCompressionLlmAdapter factory to @openchatlab/node-runtime
- Simplify server and electron compression adapter creation via shared factory
2026-05-18 23:36:42 +08:00
digua 53fa23f488 refactor(ai): unify formatAIError and extract FTS search SQL to core
- Extract formatAIError to @openchatlab/node-runtime, eliminate duplicate in Electron and Server
- Extract stripAvatarFields to @openchatlab/core
- Extract FTS message search SQL to core (searchMessagesWithFtsAsync)
- Remove redundant sync searchMessagesWithLike from Electron worker
2026-05-18 23:28:39 +08:00
digua 0d454d8a52 refactor(core): extract session time queries and remove dead code
- Remove dead code: electron/main/ai/agent/content-parser.ts (duplicate of @openchatlab/core)
- Extract getSessionsByTimeRange and getRecentChatSessions to @openchatlab/core
- Route IPC session:getByTimeRange and session:getRecent through Worker instead of inline SQL
- Eliminate duplicate getSessionSummaries in workerManager.ts, delegate to core via Worker
2026-05-18 23:22:25 +08:00
digua 1536e241fc docs: add versioning and commit scope conventions to AGENTS.md
- All distribution targets share root package.json version (lock-step)
- Commit scope: free-form for shared changes, platform scope (electron/
  cli/web) only when the change is platform-specific
2026-05-18 22:58:00 +08:00
digua 620ae40d45 fix(web): show app version in sidebar instead of empty string
Inject __APP_VERSION__ from package.json via Vite define, read it in
WebPlatformAdapter.getVersion(), and call getVersion() on all platforms
(not just Electron) in Sidebar onMounted.
2026-05-18 22:43:00 +08:00
digua e5dc1fcb9b fix(dev): tsx watch not detecting workspace package changes
tsx watch's symlink-based file watching doesn't trigger on changes to
pnpm workspace packages (tsx#221). Add --include flags for packages/core
and packages/node-runtime source directories to both dev:serve script
and vite.web.config.mts auto-serve plugin. Extract long command to
scripts/dev-serve.sh for cleaner package.json.
2026-05-18 22:43:00 +08:00
digua cdb2bfea7c feat: unify LLM streaming with thinking display and Web/cache API shims
- Extract runSimpleLlmStream to @openchatlab/node-runtime for shared
  Electron/Server streaming with thinking_start/delta/end event handling
- Simplify Electron IPC ai.ts by delegating to shared stream function
- Add thinking/thinkingDone fields to LlmStreamChunk and preload types
- Add collapsible thinking process display in SQL LAB with auto-scroll
- Implement Web chatStream via SSE with proper error propagation
- Add /_web/cache/save-to-downloads and show-in-folder server routes
- Add cacheApi shim in Web registry for server-side file operations
- Increase SQL LAB maxTokens to 4096 to accommodate reasoning models
- Detect finishReason:length and show token limit error message
- Add i18n keys for thinking/thinkingProcess/errorTokenLimit (4 locales)
2026-05-18 22:32:00 +08:00
digua 58a558b7aa fix(server): align web AI agent with Electron implementation
- Replace mapCoreEventToStream with shared AgentEventHandler for rich
  status events (contextTokens, totalUsage, updatedAt)
- Add formatAIError for user-friendly error messages (429/503/quota)
- Compute maxToolResultTokens dynamically from model contextWindow
- Add steerMessage (answerWithoutTools) and onConvertToLlm callback
- Forward ownerInfo/mentionedMembers from web frontend to server
2026-05-18 22:23:44 +08:00
digua 7190606ca7 refactor: extract AI translations to shared i18n module in node-runtime
Move AI-related translations (tools, agent prompts, LLM, summary) from
Electron-specific locale files into @openchatlab/node-runtime, enabling
single-source maintenance across Electron and Server.

- Add packages/node-runtime/src/ai/i18n/ with createAiTranslate() and
  bundled locale data (zh-CN, en-US, ja-JP, zh-TW)
- Electron locale files now compose: Electron-specific keys + shared AI
  translations from node-runtime
- Server agent.ts switches to shared buildSystemPrompt with evidence
  policy, data snapshot injection, ownerInfo and mentionedMembers support
- Server route fetches dataSnapshot from getChatOverview before agent run
2026-05-18 22:04:01 +08:00
digua 1620a5b4a3 refactor: remove pure re-export files from electron/main
Delete 9 files that were pure passthrough re-exports with zero logic:
- database/sessionCache.ts → consumers import @openchatlab/node-runtime directly
- nlp/stopwords.ts, ftsTokenizer.ts, types.ts, segmenter.ts, index.ts → consumers import @openchatlab/core or @openchatlab/node-runtime directly
- ai/agent/event-handler.ts → consumer imports @openchatlab/node-runtime directly
- ai/llm/provider-registry.ts, model-catalog.ts → consumer imports @openchatlab/core directly

This eliminates unnecessary indirection layers while keeping thin adapters
that contain real Electron-specific logic (DI, singletons, path wiring).
2026-05-18 21:19:15 +08:00
digua 09cdf165b5 feat(server): add merge HTTP routes and simplify Electron merger layer
- Server: add MergeSessionCache + 5 merge endpoints (parse, conflicts, execute, clear, export-for-merge)
- Electron tempCache.ts: delegate to shared TempDbReader/TempDbWriter via BetterSqliteAdapter
- Electron merger/index.ts: remove dead wrapTempDbReader, use reader.toDataSource() directly
2026-05-18 21:07:04 +08:00
digua a3a1d289ba feat(node-runtime): extract Markdown export engine and add server export route
- Extract exportFilterResultToMarkdown to node-runtime/export with writer abstraction
- Electron export.ts becomes thin wrapper (fs.createWriteStream + worker IPC progress)
- Server adds POST /_web/sessions/:id/export/markdown returning downloadable Markdown
2026-05-18 21:01:27 +08:00
digua 8c130c7221 feat(core): unified SQL execution with executeSql() and getSchemaDetailed()
- Add executeSql() with options: columnar, timing, maxRows (stmt.readonly + regex fallback)
- Add getSchemaDetailed() returning table + column info via PRAGMA table_info
- Electron sql.ts delegates to shared executeSql (columnar=true, timing=true)
- Server web.ts uses executeSql (with timing) and getSchemaDetailed (simplified schema route)
2026-05-18 20:45:24 +08:00
digua 9ee3060377 feat(node-runtime): extract session cache to shared package
- Extract JSON cache CRUD + overview/members compute to node-runtime/cache
- Functions now accept DatabaseAdapter instead of better-sqlite3 Database
- Electron sessionCache.ts becomes pure re-export
- Updated all Electron callers to wrap DB with BetterSqliteAdapter
2026-05-18 20:41:48 +08:00
digua 064f3ce144 feat(node-runtime): extract chat DB migrations and add auto-migration on writable open
- Extract migration definitions (v1-v4) to shared getChatDbMigrations() with DI for tokenizer
- DatabaseManager.openWritable() auto-runs pending migrations
- Electron migrations.ts becomes thin wrapper delegating to shared definitions
2026-05-18 20:38:38 +08:00
digua f78077655f fix(server): auto-generate session index after import
After streaming or incremental import, automatically generate the
session index (chat_session table) so the frontend does not show
the "generate index" prompt when entering a conversation.
2026-05-18 20:21:30 +08:00
digua 7b29b34e9a feat(server): rewire import to shared StreamingImporter + add incremental routes
- Replace buffered import with high-performance streaming pipeline
- Add POST /_web/sessions/:id/import/incremental (SSE)
- Add POST /_web/sessions/:id/import/incremental/analyze
- Add POST /_web/import/analyze (dry-run)
- CLI import now reports diagnostics from StreamingImporter
2026-05-18 20:06:57 +08:00
digua 58f87133c7 refactor(electron): rewire worker import modules to shared implementations
streamImport.ts: 1080 -> ~110 lines, delegates to node-runtime StreamingImporter
incrementalImport.ts: 451 -> ~65 lines, delegates to node-runtime IncrementalImporter
Both inject Electron-specific deps: IPC progress, better-sqlite3 adapter, cache hook
2026-05-18 20:03:11 +08:00
digua 8d430249c9 feat(node-runtime): extract TempDbWriter/Reader and session export
Platform-agnostic temp DB management for merge operations using
DatabaseAdapter. Includes TempDbWriter (streaming write), TempDbReader
(streaming read + MergerDataSource adapter), exportSessionToJson,
and cleanup helpers.
2026-05-18 20:00:59 +08:00
digua 2e564e8063 feat(node-runtime): extract IncrementalImporter from Electron worker
Platform-agnostic incremental import with dual-path dedup
(platformMessageId + content hash), member upsert, meta patching,
FTS sync, and injectable dependencies (IncrementalImportDeps).
2026-05-18 19:58:25 +08:00
digua bf91c4b7df feat(node-runtime): extract StreamingImporter from Electron worker
Platform-agnostic streaming import with batched transactions, deferred
indexes, nickname history tracking, FTS indexing, and format fallback.
All Electron-specific dependencies replaced with injectable interfaces
(StreamImportDeps, ImportLogger, ImportProgressCallback).
2026-05-18 19:56:27 +08:00
digua 3830ccdd1b feat(server): wire shared managers, FTS, and summary routes to web API
Add Assistant/Skill CRUD routes using shared AssistantManager/SkillManagerCore,
FTS search/rebuild routes, and session summary generation routes. Clean up
unused imports in ai.ts.
2026-05-18 19:43:16 +08:00
digua 1294448959 refactor(electron): introduce ElectronPathProvider and convert modules to PathProvider
Create ElectronPathProvider implementing @openchatlab/core PathProvider interface,
and convert 12 modules from direct paths.ts imports to PathProvider-based access.

Key changes:
- New: ElectronPathProvider + path-context.ts singleton
- assistant/manager.ts: delegate to @openchatlab/node-runtime AssistantManager (394→115 lines)
- skills/manager.ts: delegate to @openchatlab/node-runtime SkillManagerCore (369→100 lines)
- 10 other modules: replace getXxxDir() imports with getPathProvider().getXxxDir()

Desktop-only modules (IPC, workerManager, i18n, index) intentionally kept as-is.
2026-05-18 19:25:12 +08:00
digua f597c9b1b8 chore(electron): remove dead code crypto.ts and device-key.ts
encryptApiKey/decryptApiKey and device key management have zero
importers — legacy functionality already handled by
@openchatlab/config/migrations/crypto-legacy. Removes ~278 lines.
2026-05-18 19:07:22 +08:00
digua 249df9c8e3 feat(node-runtime): extract RAG subsystem to shared package
Move the entire RAG module (~2458 lines) from electron/main/ai/rag/ to
@openchatlab/node-runtime. Includes chunking, embedding, vector store,
and semantic pipeline.

Key changes:
- Config operations use injected aiDataDir instead of getAiDataDir()
- Logger uses injectable RagLogger interface
- LLM config provided via getLLMConfig/getAssistantConfig callbacks
- Deduplicated cosineSimilarity into shared utils.ts
- Electron rag/index.ts is now a thin wrapper (~128 lines)
2026-05-18 19:06:01 +08:00
digua 172d227caa refactor(server): delegate import data writing to shared writeParseResultToDb
Replace ~120 lines of inline SQL in server stream-import.ts and ~60 lines
in importer.ts fullImport with calls to shared writeParseResultToDb from
@openchatlab/node-runtime. Server imports now gain name history tracking
that was previously Electron-only.
2026-05-18 18:55:01 +08:00
digua 1796731535 feat(node-runtime): extract import perf logger to shared package
Move performance logging from electron/main/worker/core/perfLogger.ts to
@openchatlab/node-runtime. Log directory is now caller-provided instead
of derived from Electron paths. Electron wrapper provides the path.

Reduces Electron perfLogger from ~181 lines to ~28 lines.
2026-05-18 18:50:30 +08:00
digua 0017dcbf9e feat(node-runtime): extract FTS5 index operations to shared package
Move core FTS operations (create, build, rebuild, insert, search) to
@openchatlab/node-runtime working with DatabaseAdapter. Electron fts.ts
becomes a thin adapter that adds session-level DB connection management.

Reduces Electron fts.ts from ~207 lines to ~90 lines of pure wrappers.
2026-05-18 18:49:14 +08:00
digua c0e1647ab6 feat(core): extract updateSessionOwnerId and renameSession to @openchatlab/core
Move pure SQL session CRUD operations (owner_id update, rename) to the
shared core package as DatabaseAdapter functions. Electron wrappers now
delegate to core, keeping only connection management and error handling.
2026-05-18 18:47:35 +08:00
digua 1f11f94212 feat(node-runtime): extract writeParseResultToDb to shared import module
Move the core data writing logic (meta/member/message insertion with
member_name_history tracking) from electron/main/database/core.ts to
@openchatlab/node-runtime. Electron importData() now delegates to the
shared function via BetterSqliteAdapter, reducing ~130 lines to ~10.

This enables both Electron and Server to share the same import logic
including name history tracking.
2026-05-18 18:45:40 +08:00
digua dc5978fa78 refactor(electron): unify migration system with @openchatlab/core runMigrations
Replace Electron's independent migration runner with core's runMigrations
engine. Migration scripts stay in Electron layer (they depend on NLP
tokenizer and i18n), but execution logic (version check, transaction,
version bump) is now delegated to the shared framework.

- Re-export CURRENT_SCHEMA_VERSION from @openchatlab/core
- Wrap better-sqlite3 Database in BetterSqliteAdapter for core compat
- Convert migration up() signatures to accept DatabaseAdapter
- Remove ~100 lines of duplicated migration infrastructure
2026-05-18 18:42:17 +08:00
digua 1fb608a4ed refactor(electron): replace inline DDL with @openchatlab/core schema constants
Remove ~76 lines of duplicated CREATE TABLE/INDEX/FTS SQL from
database/core.ts createDatabase(), delegating to CHAT_DB_SCHEMA and
FTS_TABLE_SCHEMA from @openchatlab/core — the single source of truth.
2026-05-18 18:39:12 +08:00
diguaandCursor 03c07a4fe9 fix(core): replace Node.js crypto with browser-safe FNV-1a hash in dedup
The generateMessageKey function used crypto.createHash (Node.js only),
causing Vite browser externalization error. Replaced with pure-JS FNV-1a
64-bit hash that works in both Node.js and browser environments.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 00:17:56 +08:00
diguaandCursor 8412180470 feat(node-runtime): extract SkillManagerCore class with abstract FS
Platform-agnostic skill CRUD manager with dependency injection for file
system ops, builtin skill resources, and content hashing. Supports init,
CRUD, builtin import/reimport, cloud markdown import, and AI skill menu
generation.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 00:14:51 +08:00
diguaandCursor cb9849c164 feat(node-runtime): extract AssistantManager class with abstract FS
Platform-agnostic AssistantManager with dependency injection for file
system ops, builtin configs, and ID generation. Supports init, CRUD,
builtin import/reimport/reset, and cloud markdown import.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 00:10:52 +08:00
diguaandCursor 7feab45447 feat(node-runtime): extract LLM config CRUD to shared LLMConfigStore class
Introduce ConfigStorage abstraction and LLMConfigStore class with
dependency injection for i18n, UUID generation, and auth profile hooks.
All CRUD operations (add/update/delete/slots) are platform-agnostic.
Electron can implement ConfigStorage with fs + getAiDataDir.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 00:07:28 +08:00
diguaandCursor 640d16c6ec feat(node-runtime): extract merger orchestration to @openchatlab/node-runtime
Move merge orchestration (data source reading, conflict checking,
ChatLab output assembly, JSONL serialization) to shared package with
abstract MergerDataSource interface. Electron merger becomes thin adapter
wrapping TempDbReader and handling file I/O.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-18 00:02:20 +08:00
diguaandCursor 03cb8aa196 feat(core): extract merger pure algorithms to @openchatlab/core
Move platform ID collision detection, normalization, conflict detection,
member merge, and message dedup-sort from electron/main/merger to a
shared package with zero Node.js dependencies.

Also fix pre-existing unused-variable TS errors in summary test.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:54:07 +08:00
diguaandCursor d24ad22980 refactor(node-runtime): extract summary pipeline to @openchatlab/node-runtime
Move session summary generation (573 lines) to shared package with
SummaryDeps injection for DB/LLM/i18n. Pure algorithms (isValidMessage,
filterValidMessages, splitIntoSegments, prompt builders) are now shared.
Electron summary/index.ts implements SummaryDeps and delegates.
Added 14 unit tests covering message filtering, segmentation,
summary generation, and eligibility checks.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:36:21 +08:00
diguaandCursor 38b599034f refactor(node-runtime): extract event-handler to @openchatlab/node-runtime
Move AgentEventHandler, TokenUsage, AgentRuntimeStatus, AgentStreamChunk
to packages/node-runtime/src/ai/agent/event-handler.ts.
Token estimation and event-to-chunk mapping are now shared.
Electron agent/types.ts re-exports from node-runtime.
Added 8 unit tests covering token estimation, tool tracking, usage cloning.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:32:54 +08:00
diguaandCursor 0ac90030cb refactor(node-runtime): extract prompt-builder to @openchatlab/node-runtime
Move buildSystemPrompt from electron/main/ai/agent/ to shared package.
i18n t() is now injected via BuildSystemPromptOptions parameter.
Types OwnerInfo, MentionedMember, SkillContext are shared.
Electron prompt-builder.ts injects its own t() and delegates.
Added 8 unit tests with mock translation function.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:28:28 +08:00
diguaandCursor d1942fb18f refactor(core): extract member write operations to @openchatlab/core
Move mergeMembers, deleteMember, updateMemberAliases, ensureAliasesColumn,
ensureAvatarColumn to packages/core/src/query/member-ops.ts.
All functions accept DatabaseAdapter. Electron basic.ts now wraps raw
Database with BetterSqliteAdapter and delegates to core.
Added 8 unit tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:25:56 +08:00
diguaandCursor abcb1fc28f refactor(core): extract filter algorithms to @openchatlab/core
Move filterMessagesWithContext and getMultipleSessionsMessages to
packages/core/src/query/filter.ts. Functions now accept DatabaseAdapter
instead of raw better-sqlite3 Database. Types FilterMessage, ContextBlock,
PaginationInfo, FilterResultWithPagination moved to core.
Electron filter.ts is now a thin wrapper that opens DB and delegates.
Added 6 unit tests with mock DatabaseAdapter.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:22:21 +08:00
diguaandCursor f42c8d71ec refactor(node-runtime): extract ftsTokenizer to @openchatlab/node-runtime
Move tokenizeForFts and tokenizeQueryForFts from electron/main/nlp/ to
packages/node-runtime/src/nlp/fts-tokenizer.ts. Uses the already-shared
getJieba() internally. Electron ftsTokenizer.ts now re-exports.
Added 9 unit tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:13:17 +08:00
diguaandCursor 669c42477c refactor(core): extract content-parser to @openchatlab/core
Move extractThinkingContent and stripToolCallTags (pure string algorithms
with zero deps) from electron/main/ai/agent/ to packages/core/src/ai/.
Electron agent now imports from @openchatlab/core.
Added 12 unit tests covering all parsing paths.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 23:11:17 +08:00
diguaandCursor 44f170b73e chore: remove dead database/analysis.ts (1160 lines)
All analysis queries have been migrated to @openchatlab/core and are
consumed via Worker query modules and Server routes. This file had
zero imports and was fully superseded.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 22:43:09 +08:00
diguaandCursor 226c053006 refactor: extract fetchRemoteModels + validateApiKey to @openchatlab/node-runtime
Move remote LLM API operations into shared module with optional
headers and logger injection. Electron wraps with UA headers and
aiLogger; server re-exports directly.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 14:15:00 +08:00
diguaandCursor 73569c69bf refactor: extract shared buildPiModel to @openchatlab/node-runtime
Move URL normalization helpers and PiModel construction into a shared
llm-builder module. Electron and server both delegate to the core
builder, with Electron adding optional headers and provider validation.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 14:11:49 +08:00
diguaandCursor 9c2b578c87 refactor: extract shared buildMemberIdMap to @openchatlab/core
Eliminate duplicate buildMemberIdMap in server importer and
stream-import. Core function uses DatabaseAdapter interface.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 14:00:44 +08:00
diguaandCursor 044d472c4b refactor: unify message dedup key generator in @openchatlab/core
Extract canonical generateMessageKey (SHA-256 + base64url) into core.
Electron tempDb, merger, and server importer now share the same
algorithm, eliminating the server's divergent hex-truncated variant.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 13:56:45 +08:00
diguaandCursor 0d0367c296 refactor: unify import DDL with core CHAT_DB_SCHEMA
Split CHAT_DB_SCHEMA into CHAT_DB_TABLES and CHAT_DB_INDEXES constants.
Electron import utils now uses core definitions, eliminating schema drift.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 13:52:57 +08:00
diguaandCursor 6a44547c67 refactor: unify AI tool session queries via core
Align searchSessions and getSessionMessages in @openchatlab/core with
Electron's richer return shape and FTS support:

- Add senderId, senderPlatformId to SessionPreviewMessage type and SQL
- Add optional ftsMatchExpression param to searchSessions for FTS path
- Electron aiTools.ts: delegates to core with FTS tokenization at edge
- Electron types.ts: re-export SessionSearchResultItem/SessionMessagesResult from core
- workerManager.ts: remove duplicate type definitions, re-export from session module

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 13:44:06 +08:00
diguaandCursor 5d31184400 refactor: extract session index queries to @openchatlab/core
Move all session index (chat_session) logic from Electron worker to
shared core functions behind DatabaseAdapter:

- Add DEFAULT_SESSION_GAP_THRESHOLD, ChatSessionItem, SessionIndexStats types
- Add hasSessionIndex, getSessionIndexStats, getChatSessionList read queries
- Add getChatSessionSummary, saveChatSessionSummary summary read/write
- Add updateSessionGapThreshold meta update
- Add clearSessionIndex, generateSessionIndex, generateIncrementalSessionIndex
- Electron sessionIndex.ts: thin wrappers handling DB open/close lifecycle
- Server web.ts: generate-index and clear-index routes delegate to core
- Electron types.ts: re-export core types, keep Electron-only filter types
- Add 20 unit tests covering generation, incremental, clear, stats, summaries

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 13:34:46 +08:00
diguaandCursor 7c11aefacd refactor: unify member query helpers in core
Extract duplicated member list/pagination logic from electron/main/worker/query/basic.ts
and packages/server/src/http/routes/web.ts into shared core functions.

- Add getMembersPaginated with search, sort, pagination to core
- Extend MemberWithAliases type with avatar field
- Extend getMembersWithAliases with avatar column detection
- Electron basic.ts: getMemberNameHistory, getMembers, getMembersPaginated delegate to core
- Server web.ts: /_web/members and /_web/members/paginated both delegate to core
- Retain Electron-only schema migrations and member DML operations
- Add 14 unit tests covering aliases/avatar presence, pagination, search, sort, system filter

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 13:24:54 +08:00
diguaandCursor f4d5010e77 refactor: unify session list/detail queries into @openchatlab/core
Extract duplicated session info assembly from electron/main/worker/query/sessions.ts,
packages/server/src/http/routes/web.ts, and sessions.ts into shared core functions.

- Add CoreSessionInfo type covering all single-DB queryable AnalysisSession fields
- Add buildSessionInfo pure mapper (meta + overview → flat info)
- Add getSessionInfo convenience (one-call DB query)
- Add getSummaryCount and getLastPlatformMessageId helpers
- Electron: thin adapter with cache-first overview, platform-specific dbPath/avatar/aiCount
- Server web.ts & sessions.ts: delegate to getSessionInfo, eliminate inline assembly
- Add 9 unit tests for all new core session functions

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 13:08:48 +08:00
diguaandCursor 9ad0c49acf refactor: unify message query functions into @openchatlab/core
Extract duplicated query logic from electron/main/worker/query/messages.ts
and src/services/message/fetch.ts into shared async functions in core.

- Add AsyncSqlExecutor abstraction (packages/core/src/query/executor.ts)
- Add 8 shared async query functions (message-query-functions.ts)
- Extend buildMsgConditions with memberId/systemFilter/textOnlyFilter
- Electron: thin adapter via sync-to-async executor, FTS remains platform-specific
- Web/CLI: thin adapter via pluginQuery executor
- Fix pre-existing SearchMessageResult type mismatch (add missing fields)
- Add 10 unit tests verifying executor consistency

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-17 12:56:25 +08:00
digua ebca4a4f09 refactor: unify message query layer with shared SQL template and mapper
Extract duplicated 12-column message SELECT, types, and row mapper into
@openchatlab/core/query/message-sql.ts as single source of truth.
Migrate Electron worker, CLI Web FetchMessageAdapter, filter.ts, and
core hydrateMessagesByIds to use shared FULL_MSG_SELECT + mapMessageRow.
Remove DbMessageRow, sanitizeMessageRow, and all inline SQL duplicates.
2026-05-16 00:40:49 +08:00
digua 40bdc9c1b0 fix: resolve sync display bugs and unify import logic
- Fix subscription sync not showing +xx: trigger foreground pull after
  adding sessions instead of relying on background scheduler
- Fix negative message count (+-N): use messageCount directly as it
  already represents net new writes after dedup
- Eliminate duplicated fullImportFile: delegate to importData() to avoid
  missing fields (avatar, roles, reply_to_message_id, group_avatar)
- Fix member avatar not displaying in CLI Web: SQL alias mismatch
  (avatar vs senderAvatar)
- Add skipInitialPull param to scheduler to avoid race with foreground pull
- Add @openchatlab/sync to server package.json for proper tsx watch support
2026-05-16 00:11:30 +08:00
digua 6f8106db80 refactor: migrate Electron sync to @openchatlab/sync shared package
Replace Electron-specific config, data source, pull scheduler, and
discovery modules with @openchatlab/sync shared implementations.
Create ElectronFetcher, WorkerImporter, and BrowserWindowNotifier
adapters. Delete 5 redundant files (~27KB of duplicated code).
2026-05-15 23:31:00 +08:00
digua 158e792956 feat: add @openchatlab/sync package and CLI automation support
Extract platform-agnostic sync logic (config, data source, discovery,
pull engine, scheduler) into @openchatlab/sync. Implement server-side
adapters (NodeFetcher, DirectImporter, NoopNotifier) and HTTP routes
(/_web/automation/*) for the CLI web UI. Adapt Pinia store with
dual-transport (IPC for Electron, HTTP for Web).
2026-05-15 23:21:10 +08:00
digua 56a7939356 feat: implement all CoreDataProvider methods for Server Agent
Extract 10 query functions from Electron worker to @openchatlab/core:

- message-queries: getMessageContext, getSearchMessageContext,
  getConversationBetween, getMemberNameHistory, getMembersWithAliases,
  executeParameterizedSql
- session-queries: getChatOverview, searchSessions, getSessionMessages,
  getSessionSummaries
- filters: hasTable, hasColumn utilities

CoreDataProvider now delegates to core functions instead of throwing
NotImplementedError. All 27 Server Agent tools are functional.

Also inject segmentText (batchSegmentWithFrequency) into server
ToolExecutionContext so keyword_frequency tool works properly.
2026-05-15 22:23:27 +08:00
digua 74ea5563cb refactor: split tool registry into MCP and Agent, align agent tools with Electron
- Rename TOOL_REGISTRY to MCP_TOOL_REGISTRY (MCP Server only)
- Create AGENT_TOOL_REGISTRY with full Electron-aligned tool set (27 tools)
- MCP-only tools (list_sessions, get_session_info, search_keyword) excluded
  from agent registry
- Update ai.ts to use AGENT_TOOL_REGISTRY, mcp.ts to use MCP_TOOL_REGISTRY
- Add truncation strategies for newly registered agent tools

Note: Most newly registered agent tools require CoreDataProvider method
implementations (currently NotImplementedError). See follow-up task doc.
2026-05-15 22:03:40 +08:00
digua c789df5495 refactor: unify AI tool naming across Electron, CLI Web, and MCP
Remove chatlab_ prefix from all tool definitions in @openchatlab/tools,
aligning with Electron-style names (get_*, search_*, etc.). The chatlab_
prefix is now only added at the MCP registration layer via MCP_TOOL_PREFIX
constant, keeping MCP namespace isolation without polluting internal names.

This ensures the AI agent uses identical tool names in both Electron and
CLI Web environments, fixing skill system compatibility and tool catalog
consistency.
2026-05-15 21:39:30 +08:00
digua 17f7ebc094 docs: update 2026-05-15 22:30:18 +08:00
digua 6090c6689f refactor: extract AiLogger to @openchatlab/node-runtime for shared use
Move the file-based AI logging implementation from Electron-specific
code to a shared AiLogger class in @openchatlab/node-runtime. Both
Electron and Server/CLI now use the same logger writing to
~/.chatlab/logs/ai/. Server-side agent, tool adapter, and compression
now produce the same structured log files as Electron.
2026-05-15 22:23:49 +08:00
digua 932b2553b0 refactor: remove 3 pure re-export wrappers in electron/main/ai
Delete tokenizer/index.ts, skills/parser.ts, and assistant/parser.ts
which were single-line re-exports from @openchatlab/node-runtime.
All consumers now import directly from the shared package.
2026-05-15 22:12:50 +08:00
digua 596eaa3051 refactor: remove redundant electron/main/ai/compression wrapper
Inline createCompressionLlmAdapter and logger into the sole caller
(ipc/ai.ts) and import checkAndCompress/manualCompress directly from
@openchatlab/node-runtime. The "old API compatibility" layer was
unnecessary since there is only one internal consumer.
2026-05-15 22:08:31 +08:00
digua 787e645a60 refactor: remove redundant electron/main/ai/preprocessor directory
The 4-file preprocessor wrapper directory was entirely re-exports from
@openchatlab/node-runtime after the pipeline unification. Replace all
consumers with direct imports and add logger injection support to
PreprocessingPipelineOptions so Electron retains aiLogger integration.
2026-05-15 19:58:03 +08:00
digua 984b778033 refactor: extract shared preprocessing pipeline to @openchatlab/node-runtime
Unify the duplicated preprocessing logic between Electron (wrapWithPreprocessing)
and Server (applyPreprocessing) into a single applyPreprocessingPipeline() function
in @openchatlab/node-runtime. Both platforms now delegate to the same implementation,
eliminating ~50 lines of redundant code and fixing a redundant token counting bug
in the Server version.
2026-05-15 19:52:01 +08:00
digua ad1c6eef1e chore: remove unused electron/main/ai/tools/utils/format.ts 2026-05-15 19:40:51 +08:00
digua b5e296b631 chore: remove dead code sqlToolRunner.ts (replaced by @openchatlab/tools/sql) 2026-05-15 19:39:27 +08:00
digua 9d55676892 refactor(tools): complete migration of all Electron tools to shared definitions
Move all 24 Electron-native tool definitions to @openchatlab/tools:
- 13 data query tools (chat-overview, search-messages, deep-search,
  get-recent-messages, get-message-context, search-sessions,
  get-session-messages, get-members, get-member-name-history,
  get-conversation-between, get-session-summaries,
  response-time-analysis, keyword-frequency)
- 11 SQL template tools (message_type_breakdown, etc.)

Electron definitions/ now contains only index.ts with adaptSharedTool()
calls. All tool logic lives in the shared package.

Key changes:
- Expand ToolDataProvider with 12 new methods for full data access
- Add ToolExecutionContext fields: searchContextBefore/After,
  maxMessagesLimit, segmentText callback, translateTemplate callback
- Create shared utils: format.ts (i18n), time-params.ts, schemas.ts
- Create SQL tool system: definitions, executor, types
- WorkerDataProvider implements all new methods via workerManager
- CoreDataProvider stubs new methods (Server doesn't use these yet)
- Delete 16 Electron tool files (-1280 lines, +482 lines)
2026-05-15 19:34:35 +08:00
digua f35676cb23 refactor(electron): replace member_stats and time_stats with shared tool definitions
Create shared-tool-adapter to convert @openchatlab/tools ToolDefinition → AgentTool.
Delete Electron-specific get-member-stats.ts and get-time-stats.ts,
replaced by shared definitions via adaptSharedTool().
Make ToolExecutionContext.db optional to support Electron (no DatabaseAdapter).
2026-05-15 19:11:59 +08:00
digua 619b59270b refactor(tools): migrate 6 tool handlers to use ToolDataProvider
Replace direct @openchatlab/core function calls with dataProvider abstraction:
- member-stats, time-stats, search, recent-messages, sql, schema
- Handlers are now async, using context.dataProvider for data access
- Server consumers inject CoreDataProvider alongside db for backward compat
2026-05-15 19:07:21 +08:00
digua b1df81b064 feat(tools): implement CoreDataProvider and WorkerDataProvider
Add ToolDataProvider implementations for both platforms:
- CoreDataProvider: wraps @openchatlab/core sync queries for Server/MCP
- WorkerDataProvider: wraps workerManager async IPC for Electron
2026-05-15 19:05:00 +08:00
digua 21a0ce36f4 refactor(tools): add ToolDataProvider interface and async handler support
Prepare @openchatlab/tools for unified tool layer:
- ToolDefinition.handler now supports async (ToolResult | Promise<ToolResult>)
- Add ToolDataProvider abstraction interface for platform-agnostic data access
- Add TimeFilter, ToolCategory, TruncationStrategy types
- Update Server consumers (tool-adapter, mcp) to await handler results
2026-05-15 19:02:49 +08:00
digua 653bd966fa fix: dev native startup issues 2026-05-15 18:54:50 +08:00
digua 347c7d69dd refactor(electron): migrate all pi-ai/pi-agent-core imports to node-runtime
- Convert 20 tool definition files to import Type/AgentTool from
  @openchatlab/node-runtime instead of direct pi-* packages
- Migrate compression, summary, RAG, LLM config, and IPC modules
  to use node-runtime re-exports (completeSimple, streamSimple, etc.)
- Remove @mariozechner/pi-agent-core and @mariozechner/pi-ai from
  root package.json — now only node-runtime depends on them directly
- Electron is now fully decoupled from pi-* direct imports
2026-05-14 23:46:13 +08:00
digua 8dd3e2396c refactor(ai): extract shared Agent core to @openchatlab/node-runtime
- Create runAgentCore() in node-runtime/src/ai/agent/ encapsulating
  PiAgentCore construction, history conversion, event subscription,
  abort forwarding, tool round management, and usage collection
- Re-export pi-agent-core/pi-ai types (AgentTool, Type, Model, etc.)
  and functions (completeSimple, streamSimple) from node-runtime
- Refactor Server agent.ts to use runAgentCore, removing all direct
  pi-agent-core/pi-ai imports (~335→~210 lines)
- Remove pi-agent-core and pi-ai from Server package.json dependencies
- Refactor Electron agent/index.ts and event-handler.ts to use
  runAgentCore with DI hooks (streamFn, onConvertToLlm, onDebugContext)
- Document remaining Electron pi-ai usage points for future migration
2026-05-14 23:36:11 +08:00
digua 6348c3f4e8 refactor(electron): switch tools module to shared @openchatlab/node-runtime imports
- Replace local formatToolResultAsText, truncateFormattedMessages,
  anonymizeMessageNames with imports from @openchatlab/node-runtime
- Replace local createActivateSkillTool with shared version via DI
  (inject Electron's getSkillConfig and CORE_TOOL_NAMES)
- Retain Electron-specific translateTool (i18n) and getAllTools (tool registry)
- Net reduction of ~170 lines of duplicated code
2026-05-14 22:11:04 +08:00
digua 9dfb2dfbf4 feat(server,web): implement validateApiKey and fetchRemoteModels for web mode
- Create remote-api.ts with platform-agnostic validateApiKey (completeSimple
  test call) and fetchRemoteModels (OpenAI/Gemini model listing endpoints)
- Add POST /_web/ai/llm/validate-key and POST /_web/ai/llm/remote-models
  HTTP endpoints
- Update web shim to call new endpoints instead of returning stubs
2026-05-14 22:09:12 +08:00
digua d87bdd5712 feat(server,web): add custom provider/model CRUD HTTP endpoints
- Create custom-store.ts with platform-agnostic CRUD for custom-providers.json
  and custom-models.json (parameterized by aiDataDir)
- Add 6 HTTP endpoints: POST/PUT/DELETE for custom providers and models
- Update web shim to call new endpoints instead of returning WEB_STUB
2026-05-14 22:07:54 +08:00
digua 4b1a1e411b feat(server,web): connect compression config through SSE stream
- Add compressionConfig to SSE agent/stream request body with sensible defaults
- Pass compressionConfig from route handler to runServerAgent
- Update frontend runStream to forward compressionConfig to the SSE endpoint
2026-05-14 22:06:17 +08:00
digua 245a2e39a0 feat(tools,server): integrate preprocessing pipeline for web tool results
- Extend ToolResult with optional rawMessages field for structured message data
- Update search and recent-messages tools to return rawMessages from query results
- Add preprocessing wrapper in server tool-adapter: auto-apply data cleaning,
  denoising, desensitization, compact formatting, and token-aware truncation
  when tools return rawMessages
2026-05-14 22:04:54 +08:00
digua 07056a2258 feat: extract context compression to @openchatlab/node-runtime
Move compression logic from Electron to shared package with pluggable
LLM adapter interface. Integrate into server agent with pre-run
compression support. Electron side becomes a thin compatibility wrapper.
2026-05-14 21:43:10 +08:00
digua 68b1c7a1b2 feat: add LLM config write endpoints for Web mode
Add CRUD HTTP endpoints for LLM configurations (add/update/delete config,
set default assistant slot, set fast model slot).
Update web shim to call these endpoints instead of returning stubs.
2026-05-14 21:38:54 +08:00
digua 58d7da6367 feat: add SkillManager and activate_skill tool to shared package
Extract SkillManager from Electron to @openchatlab/node-runtime.
Create shared createActivateSkillTool factory.
Integrate skill menu into server agent system prompt and SSE route.
2026-05-14 21:34:58 +08:00
digua 76b4be1264 refactor: extract preprocessor pipeline to @openchatlab/node-runtime
Move preprocessing (pipeline, types, builtin-rules, format utilities)
from Electron to shared package. Electron side becomes thin wrappers.
2026-05-14 21:32:27 +08:00
digua b195750f34 refactor: extract tokenizer to @openchatlab/node-runtime
Move countTokens/countMessagesTokens from Electron to shared package.
Electron side becomes a thin re-export wrapper.
2026-05-14 21:27:39 +08:00
digua 6352a3f191 refactor: implement Migration Runner for config data migrations
Extract migration logic from loadConfigStore into a shared Migration Runner
system (packages/config/src/migrations/). Runs at app startup in both
Electron and CLI, enabling CLI to handle encrypted key migration independently.
2026-05-14 21:14:15 +08:00
digua b479fd3b2d feat(web): enable AI chat in web mode with shared conversation module
- Extract AIConversationManager to @openchatlab/node-runtime for Electron + Server reuse
- Add /_web/ai/conversations/* HTTP endpoints for conversation/message CRUD
- Create FetchAIAdapter to replace WebAIAdapter stub
- Implement server-side Agent with SSE streaming (/_web/ai/agent/stream)
- Add agentApi SSE client shim in frontend registry
- Migrate API key storage from encrypted llm-config.json to auth-profiles.json
- Add @openchatlab/config auth-profiles module for credential management
2026-05-14 20:55:54 +08:00
digua 12f9ae90eb feat(web): enable AI page data loading in web mode via HTTP endpoints
- Move assistant/skill MD parsers to @openchatlab/node-runtime (shared)
- Move BUILTIN_TOOL_CATALOG, provider-registry, model-catalog to @openchatlab/core
- Add /_web/ai/* HTTP routes for assistants, skills, LLM config, providers, models, tools
- Frontend shims: read ops call HTTP, write/agent ops return stubs
- Remove IS_ELECTRON guard in llm.ts to enable web-mode config loading
- Improve web-mode AI chat error message to be user-friendly
2026-05-14 20:29:06 +08:00
digua 4b15d65b13 refactor(nlp): extract shared NLP logic to core + node-runtime
Move platform-agnostic NLP types, stopwords, POS tag definitions, and
text utilities into @openchatlab/core. Move segmenter engine, word
frequency computation, and dict manager into @openchatlab/node-runtime.

Electron nlp/ becomes a thin re-export layer; server nlp.ts shrinks
from ~450 to ~85 lines retaining only HTTP route definitions. Both
platforms now share the same NLP implementation via node-runtime.

Also fixes createJiebaNlpProvider to use shared getJieba() with disk
dict loading (was using empty Jieba instance causing degraded POS tags
in language preference analysis), adds ensureDefaultDict at server
startup, and adds /_web/nlp/* HTTP routes with window.nlpApi shim.
2026-05-14 19:50:03 +08:00