Move tokenizeForFts and tokenizeQueryForFts from electron/main/nlp/ to
packages/node-runtime/src/nlp/fts-tokenizer.ts. Uses the already-shared
getJieba() internally. Electron ftsTokenizer.ts now re-exports.
Added 9 unit tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
Move extractThinkingContent and stripToolCallTags (pure string algorithms
with zero deps) from electron/main/ai/agent/ to packages/core/src/ai/.
Electron agent now imports from @openchatlab/core.
Added 12 unit tests covering all parsing paths.
Co-authored-by: Cursor <cursoragent@cursor.com>
All analysis queries have been migrated to @openchatlab/core and are
consumed via Worker query modules and Server routes. This file had
zero imports and was fully superseded.
Co-authored-by: Cursor <cursoragent@cursor.com>
Move remote LLM API operations into shared module with optional
headers and logger injection. Electron wraps with UA headers and
aiLogger; server re-exports directly.
Co-authored-by: Cursor <cursoragent@cursor.com>
Move URL normalization helpers and PiModel construction into a shared
llm-builder module. Electron and server both delegate to the core
builder, with Electron adding optional headers and provider validation.
Co-authored-by: Cursor <cursoragent@cursor.com>
Eliminate duplicate buildMemberIdMap in server importer and
stream-import. Core function uses DatabaseAdapter interface.
Co-authored-by: Cursor <cursoragent@cursor.com>
Extract canonical generateMessageKey (SHA-256 + base64url) into core.
Electron tempDb, merger, and server importer now share the same
algorithm, eliminating the server's divergent hex-truncated variant.
Co-authored-by: Cursor <cursoragent@cursor.com>
Split CHAT_DB_SCHEMA into CHAT_DB_TABLES and CHAT_DB_INDEXES constants.
Electron import utils now uses core definitions, eliminating schema drift.
Co-authored-by: Cursor <cursoragent@cursor.com>
Align searchSessions and getSessionMessages in @openchatlab/core with
Electron's richer return shape and FTS support:
- Add senderId, senderPlatformId to SessionPreviewMessage type and SQL
- Add optional ftsMatchExpression param to searchSessions for FTS path
- Electron aiTools.ts: delegates to core with FTS tokenization at edge
- Electron types.ts: re-export SessionSearchResultItem/SessionMessagesResult from core
- workerManager.ts: remove duplicate type definitions, re-export from session module
Co-authored-by: Cursor <cursoragent@cursor.com>
Extract duplicated member list/pagination logic from electron/main/worker/query/basic.ts
and packages/server/src/http/routes/web.ts into shared core functions.
- Add getMembersPaginated with search, sort, pagination to core
- Extend MemberWithAliases type with avatar field
- Extend getMembersWithAliases with avatar column detection
- Electron basic.ts: getMemberNameHistory, getMembers, getMembersPaginated delegate to core
- Server web.ts: /_web/members and /_web/members/paginated both delegate to core
- Retain Electron-only schema migrations and member DML operations
- Add 14 unit tests covering aliases/avatar presence, pagination, search, sort, system filter
Co-authored-by: Cursor <cursoragent@cursor.com>
Extract duplicated 12-column message SELECT, types, and row mapper into
@openchatlab/core/query/message-sql.ts as single source of truth.
Migrate Electron worker, CLI Web FetchMessageAdapter, filter.ts, and
core hydrateMessagesByIds to use shared FULL_MSG_SELECT + mapMessageRow.
Remove DbMessageRow, sanitizeMessageRow, and all inline SQL duplicates.
- Fix subscription sync not showing +xx: trigger foreground pull after
adding sessions instead of relying on background scheduler
- Fix negative message count (+-N): use messageCount directly as it
already represents net new writes after dedup
- Eliminate duplicated fullImportFile: delegate to importData() to avoid
missing fields (avatar, roles, reply_to_message_id, group_avatar)
- Fix member avatar not displaying in CLI Web: SQL alias mismatch
(avatar vs senderAvatar)
- Add skipInitialPull param to scheduler to avoid race with foreground pull
- Add @openchatlab/sync to server package.json for proper tsx watch support
Extract platform-agnostic sync logic (config, data source, discovery,
pull engine, scheduler) into @openchatlab/sync. Implement server-side
adapters (NodeFetcher, DirectImporter, NoopNotifier) and HTTP routes
(/_web/automation/*) for the CLI web UI. Adapt Pinia store with
dual-transport (IPC for Electron, HTTP for Web).
Extract 10 query functions from Electron worker to @openchatlab/core:
- message-queries: getMessageContext, getSearchMessageContext,
getConversationBetween, getMemberNameHistory, getMembersWithAliases,
executeParameterizedSql
- session-queries: getChatOverview, searchSessions, getSessionMessages,
getSessionSummaries
- filters: hasTable, hasColumn utilities
CoreDataProvider now delegates to core functions instead of throwing
NotImplementedError. All 27 Server Agent tools are functional.
Also inject segmentText (batchSegmentWithFrequency) into server
ToolExecutionContext so keyword_frequency tool works properly.
- Rename TOOL_REGISTRY to MCP_TOOL_REGISTRY (MCP Server only)
- Create AGENT_TOOL_REGISTRY with full Electron-aligned tool set (27 tools)
- MCP-only tools (list_sessions, get_session_info, search_keyword) excluded
from agent registry
- Update ai.ts to use AGENT_TOOL_REGISTRY, mcp.ts to use MCP_TOOL_REGISTRY
- Add truncation strategies for newly registered agent tools
Note: Most newly registered agent tools require CoreDataProvider method
implementations (currently NotImplementedError). See follow-up task doc.
Remove chatlab_ prefix from all tool definitions in @openchatlab/tools,
aligning with Electron-style names (get_*, search_*, etc.). The chatlab_
prefix is now only added at the MCP registration layer via MCP_TOOL_PREFIX
constant, keeping MCP namespace isolation without polluting internal names.
This ensures the AI agent uses identical tool names in both Electron and
CLI Web environments, fixing skill system compatibility and tool catalog
consistency.
Move the file-based AI logging implementation from Electron-specific
code to a shared AiLogger class in @openchatlab/node-runtime. Both
Electron and Server/CLI now use the same logger writing to
~/.chatlab/logs/ai/. Server-side agent, tool adapter, and compression
now produce the same structured log files as Electron.
Delete tokenizer/index.ts, skills/parser.ts, and assistant/parser.ts
which were single-line re-exports from @openchatlab/node-runtime.
All consumers now import directly from the shared package.
Inline createCompressionLlmAdapter and logger into the sole caller
(ipc/ai.ts) and import checkAndCompress/manualCompress directly from
@openchatlab/node-runtime. The "old API compatibility" layer was
unnecessary since there is only one internal consumer.
The 4-file preprocessor wrapper directory was entirely re-exports from
@openchatlab/node-runtime after the pipeline unification. Replace all
consumers with direct imports and add logger injection support to
PreprocessingPipelineOptions so Electron retains aiLogger integration.
Unify the duplicated preprocessing logic between Electron (wrapWithPreprocessing)
and Server (applyPreprocessing) into a single applyPreprocessingPipeline() function
in @openchatlab/node-runtime. Both platforms now delegate to the same implementation,
eliminating ~50 lines of redundant code and fixing a redundant token counting bug
in the Server version.
Create shared-tool-adapter to convert @openchatlab/tools ToolDefinition → AgentTool.
Delete Electron-specific get-member-stats.ts and get-time-stats.ts,
replaced by shared definitions via adaptSharedTool().
Make ToolExecutionContext.db optional to support Electron (no DatabaseAdapter).
Replace direct @openchatlab/core function calls with dataProvider abstraction:
- member-stats, time-stats, search, recent-messages, sql, schema
- Handlers are now async, using context.dataProvider for data access
- Server consumers inject CoreDataProvider alongside db for backward compat
Add ToolDataProvider implementations for both platforms:
- CoreDataProvider: wraps @openchatlab/core sync queries for Server/MCP
- WorkerDataProvider: wraps workerManager async IPC for Electron
- Convert 20 tool definition files to import Type/AgentTool from
@openchatlab/node-runtime instead of direct pi-* packages
- Migrate compression, summary, RAG, LLM config, and IPC modules
to use node-runtime re-exports (completeSimple, streamSimple, etc.)
- Remove @mariozechner/pi-agent-core and @mariozechner/pi-ai from
root package.json — now only node-runtime depends on them directly
- Electron is now fully decoupled from pi-* direct imports
- Create runAgentCore() in node-runtime/src/ai/agent/ encapsulating
PiAgentCore construction, history conversion, event subscription,
abort forwarding, tool round management, and usage collection
- Re-export pi-agent-core/pi-ai types (AgentTool, Type, Model, etc.)
and functions (completeSimple, streamSimple) from node-runtime
- Refactor Server agent.ts to use runAgentCore, removing all direct
pi-agent-core/pi-ai imports (~335→~210 lines)
- Remove pi-agent-core and pi-ai from Server package.json dependencies
- Refactor Electron agent/index.ts and event-handler.ts to use
runAgentCore with DI hooks (streamFn, onConvertToLlm, onDebugContext)
- Document remaining Electron pi-ai usage points for future migration
- Replace local formatToolResultAsText, truncateFormattedMessages,
anonymizeMessageNames with imports from @openchatlab/node-runtime
- Replace local createActivateSkillTool with shared version via DI
(inject Electron's getSkillConfig and CORE_TOOL_NAMES)
- Retain Electron-specific translateTool (i18n) and getAllTools (tool registry)
- Net reduction of ~170 lines of duplicated code
- Create remote-api.ts with platform-agnostic validateApiKey (completeSimple
test call) and fetchRemoteModels (OpenAI/Gemini model listing endpoints)
- Add POST /_web/ai/llm/validate-key and POST /_web/ai/llm/remote-models
HTTP endpoints
- Update web shim to call new endpoints instead of returning stubs
- Create custom-store.ts with platform-agnostic CRUD for custom-providers.json
and custom-models.json (parameterized by aiDataDir)
- Add 6 HTTP endpoints: POST/PUT/DELETE for custom providers and models
- Update web shim to call new endpoints instead of returning WEB_STUB
- Add compressionConfig to SSE agent/stream request body with sensible defaults
- Pass compressionConfig from route handler to runServerAgent
- Update frontend runStream to forward compressionConfig to the SSE endpoint
- Extend ToolResult with optional rawMessages field for structured message data
- Update search and recent-messages tools to return rawMessages from query results
- Add preprocessing wrapper in server tool-adapter: auto-apply data cleaning,
denoising, desensitization, compact formatting, and token-aware truncation
when tools return rawMessages
Move compression logic from Electron to shared package with pluggable
LLM adapter interface. Integrate into server agent with pre-run
compression support. Electron side becomes a thin compatibility wrapper.
Add CRUD HTTP endpoints for LLM configurations (add/update/delete config,
set default assistant slot, set fast model slot).
Update web shim to call these endpoints instead of returning stubs.
Extract SkillManager from Electron to @openchatlab/node-runtime.
Create shared createActivateSkillTool factory.
Integrate skill menu into server agent system prompt and SSE route.
Extract migration logic from loadConfigStore into a shared Migration Runner
system (packages/config/src/migrations/). Runs at app startup in both
Electron and CLI, enabling CLI to handle encrypted key migration independently.
- Extract AIConversationManager to @openchatlab/node-runtime for Electron + Server reuse
- Add /_web/ai/conversations/* HTTP endpoints for conversation/message CRUD
- Create FetchAIAdapter to replace WebAIAdapter stub
- Implement server-side Agent with SSE streaming (/_web/ai/agent/stream)
- Add agentApi SSE client shim in frontend registry
- Migrate API key storage from encrypted llm-config.json to auth-profiles.json
- Add @openchatlab/config auth-profiles module for credential management
Move platform-agnostic NLP types, stopwords, POS tag definitions, and
text utilities into @openchatlab/core. Move segmenter engine, word
frequency computation, and dict manager into @openchatlab/node-runtime.
Electron nlp/ becomes a thin re-export layer; server nlp.ts shrinks
from ~450 to ~85 lines retaining only HTTP route definitions. Both
platforms now share the same NLP implementation via node-runtime.
Also fixes createJiebaNlpProvider to use shared getJieba() with disk
dict loading (was using empty Jieba instance causing degraded POS tags
in language preference analysis), adds ensureDefaultDict at server
startup, and adds /_web/nlp/* HTTP routes with window.nlpApi shim.
diguaandCursor