Commit Graph
743 Commits
Author SHA1 Message Date
diguaandCursor dd3cb10414 refactor(services): unify CLI Web and Electron service layer
Extract shared business logic from monolithic web.ts into reusable
services in @openchatlab/node-runtime (session, member, index, summary,
export), using SessionRuntimeAdapter to abstract platform differences.
Split web.ts into 12 domain-specific route modules under routes/web/.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
diguaandCursor 808b9026f2 fix(ai): parse MiniMax streaming <think> tags as thinking events
MiniMax models embed thinking content as <think> tags in the content
field rather than using a separate reasoning field. Add a streaming
think-tag parser that detects these tags in real-time during streaming
and routes them to the thinking UI instead of displaying raw tags.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
diguaandCursor 9421662a52 feat(import): support unified directory import for Electron and Web
Add directory import support for multi-file formats (e.g. shuakami chunked-jsonl).
Users can select or drag-and-drop a directory via the "Import Folder" checkbox.

- Add `findEntryFileInDirectory()` to parser layer for format-agnostic entry file discovery
- Add `importDirectory` method to ImportAdapter interface
- Implement FetchImportAdapter.importDirectory with multi-file FormData upload
- Add `POST /_web/import-directory` CLI route with directory reconstruction
- Add `chat:importDirectory` IPC handler in Electron main process
- Enhance FileDropZone with `directory` prop and drag-and-drop directory support
- Refactor FetchImportAdapter to extract reusable `consumeSseStream` helper

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
diguaandCursor 8bb904fb22 fix(import): support incremental import in CLI web mode
The IncrementalImportModal was Electron-only: showOpenDialog returned
canceled in web mode and FetchImportAdapter threw notSupported for both
analyzeIncrementalImport and incrementalImport despite CLI routes
already existing. Fix by using FileDropZone's built-in file input for
file selection and implementing the two fetch adapter methods via
FormData upload + SSE streaming.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
digua 04427a98d8 chore(cli): prepare npm release docs 2026-05-22 10:20:39 +08:00
digua 6fa92bc2e4 chore(mcp): prepare npm publish 2026-05-22 10:20:39 +08:00
digua d1737c7e8d chore: update npm config 2026-05-22 10:20:39 +08:00
digua 6c7e9f746f refactor(mcp): simplify MCP settings integration 2026-05-22 10:20:39 +08:00
diguaandCursor 6e7a3aa235 feat(mcp): add standalone bin entry and settings page MCP integration
- Add packages/mcp-server/src/bin.ts for `npx -y chatlab-mcp` standalone usage
- Add MCP sub-tab under Automation settings (Settings → Automation → MCP)
- MCP settings page: mode selector (npx/CLI/desktop), config generator with
  one-click copy for Cursor/Claude Desktop, troubleshooting tips
- Add i18n translations for settings.mcp.* (zh-CN, en-US, zh-TW, ja-JP)
- Update tsup config: add bin entry, enable splitting, externalize native modules

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
diguaandCursor 584f581433 fix(mcp): pass nativeBinding to DatabaseManager for ABI compatibility
The MCP entry was loading better-sqlite3 from node_modules which may
have been compiled for a different Node.js ABI. Now passes the
pre-built native binding from apps/cli/native/ (maintained by
ensure-native.mjs), matching how other CLI commands handle this.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
diguaandCursor 68652248fe feat(mcp): enrich toolset to 33 tools with token compression and dual format
- Expand MCP_TOOL_REGISTRY from 8 to 33 tools by including all
  AGENT_TOOL_REGISTRY tools (analytics, deep search, conversation, etc.)
- Add format=text|json parameter to all MCP tools (default: text for
  most, json for SQL tools) for token-efficient LLM consumption
- Create format.ts: message filtering, consecutive-sender merging,
  content truncation, and compact text output
- Inspired by community chatlab-mcp-server project

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
diguaandCursor d022bed48f refactor(mcp): extract MCP core into shared chatlab-mcp package
Move MCP Server logic (tool registration, resource registration, schema
conversion) from apps/cli/src/mcp.ts to packages/mcp-server so both CLI
and Desktop can share the same MCP implementation.

- New package: chatlab-mcp (packages/mcp-server) with McpDatabaseManager
  interface abstraction
- CLI mcp.ts slimmed to thin entry (~37 lines) calling startMcpServer()
- Desktop helper prototype at apps/desktop/helper/mcp.ts for dev-time
  MCP without GUI
- All 8 MCP tools and 3 resources unchanged

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-22 10:20:39 +08:00
digua e3370b52c2 fix(web): align dev server node runtime 2026-05-22 10:20:39 +08:00
digua eebb428dd3 release: v0.20.0 v0.20.0 2026-05-19 23:59:11 +08:00
digua 9c93aa9701 fix(home): sequence onboarding modals 2026-05-19 23:55:00 +08:00
digua 415dcefc7d fix(ai): use provider name instead of config name for auth profile keys 2026-05-19 23:50:50 +08:00
digua 0bd5bc9f38 fix(electron): auto-rebuild better-sqlite3 for Electron headers
Add @electron/rebuild devDependency and postinstall script to
apps/desktop so that better-sqlite3 is automatically recompiled
against Electron's Node.js headers after every pnpm install,
preventing NODE_MODULE_VERSION mismatch at runtime.
2026-05-19 22:47:00 +08:00
digua 08260499f8 feat: migrate localStorage to backend storage with declarative persist plugin
- Add backendPersist Pinia plugin for declarative preferences.json persistence
- Move Preferences types to @openchatlab/shared-types (single source of truth)
- Migrate sessionGapThreshold from localStorage to config.toml [ui]
- Migrate chatlab_filter_history from localStorage to preferences.json
- Fix stale localStorage locale read in i18n/index.ts
- Simplify usePreferencesSync (172 → 75 lines) by delegating to plugin

Adding a new backend-synced preference now only requires 3 changes:
1. shared-types interface  2. node-runtime DEFAULTS  3. store backendPersist.pick
2026-05-19 22:42:00 +08:00
digua 05394c2155 feat(cli): auto-detect and migrate Electron desktop data on first CLI run
When a desktop app user installs the CLI, automatically detect existing
Electron data directory (including custom paths via storage.json),
repoint config.toml, and migrate system data (ai, settings, cache, etc.)
to ~/.chatlab/. Block CLI execution with actionable instructions when
Electron was used but databases cannot be located.
2026-05-19 22:19:00 +08:00
digua 0aa30359ab fix(cli): resolve ESM module resolution for tsx dev mode
- Remove "type": "module" from apps/cli/package.json to avoid
  ESM/CJS interop issues with workspace packages in tsx dev mode
- Configure tsup to output .mjs files (always treated as ESM by Node.js)
- Rename bin entry from chatlab.js to chatlab.mjs
- Change chatlab web default port to 3100 (serve keeps 3110)
- Fix vite plugin: use pnpm instead of npx for tsx invocation
2026-05-19 22:13:00 +08:00
digua e363e8bd75 refactor(cli): unify all CLI text to English
All command descriptions, console output, option labels, comments
and error messages are now in English, consistent with AGENTS.md
rules and npm package positioning.
2026-05-19 21:59:00 +08:00
digua 09d4b1ddac feat(cli): add chatlab web command for one-click Web UI launch
- New `chatlab web` subcommand: starts HTTP API + serves Web SPA + auto-opens browser
- Options: --port, --host, --no-open
- Cross-platform `openBrowser()` using open/start/xdg-open
- Add `setWebMode()` in auth: static resources bypass token auth in web mode
- Add `build:full` and `prepare:web` scripts for dist-web preparation
2026-05-19 21:53:00 +08:00
digua b1427e2422 feat(cli): add tsup build pipeline for npm publishing
- Rename package to `chatlab-cli`, version 0.1.0
- Add tsup config with ESM output, bundle @openchatlab/* + stream-json
- Update bin/chatlab.js from stub to actual entry point
- Unify version reading via src/version.ts (CLI/MCP/HTTP API)
- Add publishing fields: type, files, engines, publishConfig, keywords
- Move @openchatlab/* to devDependencies (bundled into dist/)
- Add better-sqlite3 to dependencies, @node-rs/jieba to optionalDependencies
- Fix --web default path for bundled dist/ directory structure
2026-05-19 21:46:00 +08:00
digua 78e41a65b5 refactor: migrate CLI from packages/server to apps/cli
Move the CLI application from packages/server/ to apps/cli/ to align
with the monorepo convention (apps/ for deployable applications,
packages/ for shared libraries). Rename package to @openchatlab/chatlab.

Also fix a bug in ensureToken() that used raw string manipulation to
write config.toml, causing duplicate token fields that corrupted the
TOML file. This led to config parse failures and user_data_dir being
overwritten with defaults. Now uses writeConfigField() for safe writes.
2026-05-19 21:33:00 +08:00
digua 9db30879d3 fix(electron): decouple worker thread from electron module dependency
Worker thread crashed in packaged builds with "Cannot find module 'electron'"
because streamImport.ts transitively imported paths.ts (which requires electron)
through merger/tempCache.ts. Replaced with worker-local implementations that
receive tempDir via workerData, matching the existing dbDir/cacheDir pattern.
2026-05-19 21:13:59 +08:00
digua ba851f4ae9 fix(electron): fix data migration path mismatch after package extraction
The extraction of Electron code to apps/desktop/ changed the package name
from "ChatLab" to "@openchatlab/desktop", which caused app.getPath('userData')
to return a wrong path in dev mode. This made the migration system unable to
find existing databases, resulting in a fresh-app experience for upgrading users.

- Add productName "ChatLab" to desktop package.json to ensure correct userData path
- Add verifyDataPath() safety net: auto-correct config.toml when configured dir
  has no databases but old Electron path does
2026-05-19 20:39:25 +08:00
digua 1c6f4b3776 fix(electron): avoid install-app-deps during workspace install 2026-05-19 19:40:54 +08:00
digua c08a465116 chore(electron): remove Linux desktop build targets
Linux users will be served by the upcoming CLI package instead.
2026-05-19 19:17:59 +08:00
digua d058e8a60f fix(electron): fix CI build by pinning electron version and cleanup
- Pin electron to 35.7.5 (electron-builder requires fixed version)
- Remove redundant @electron/rebuild (built into electron-builder)
- Use electron-builder install-app-deps for postinstall
2026-05-19 19:14:11 +08:00
digua 9099b15ce6 chore(deps): upgrade vite toolchain 2026-05-19 18:39:14 +08:00
digua e80d8e471e docs: update 2026-05-19 00:40:00 +08:00
digua 2bcbb84711 feat: add interactive dev mode selector (pnpm dev)
Adds a zero-dependency Node.js script with arrow-key selection
for choosing between Desktop (Electron), Web (CLI serve + frontend),
Server Only (API backend), and Frontend-only dev modes.

Remembers last selection in node_modules/.cache/dev-mode.
2026-05-19 00:35:00 +08:00
digua b12c07dc32 refactor: extract Electron shell to apps/desktop/
Move Electron-specific code (main, preload, shared) and configs
(electron.vite, electron-builder) into apps/desktop/ to prepare
for multi-platform support (Web Browser standalone).

- Create apps/desktop/ with dedicated package.json and tsconfig
- Split Electron deps (electron, electron-updater, better-sqlite3,
  etc.) from shared frontend deps in root package.json
- Declare better-sqlite3 and @node-rs/jieba in packages/node-runtime
- Add stream-json to packages/parser where it belongs
- Update all path aliases (@electron/* → apps/desktop/*) in tsconfig,
  vite configs, and relative imports in preload/main
- Keep slim root tsconfig.node.json as shared base for packages
- Update CI/CD release workflow for new directory layout
- Fix pre-existing type errors (pi-ai Model generic, SQLLabTab
  LLMChatStreamChunk, filter.test.ts empty methods)
2026-05-19 00:31:00 +08:00
digua ba29346bfe refactor: move chart-* modules from packages/ to src/features/
These are frontend-only Vue/ECharts components that depend on app internals (@/),
not cross-platform shared packages. Moving them to src/features/charts-* keeps
packages/ purely for reusable Node.js libraries.
2026-05-18 23:57:00 +08:00
digua d3ce4023e5 fix(core): add missing re-exports in query/index.ts
The query barrel file was missing re-exports for getSessionsByTimeRange,
getRecentChatSessions, loadSessionMessages, getPrivateChatMemberAvatar,
getExportSessionData, and searchMessagesWithFtsAsync.
2026-05-18 23:43:00 +08:00
digua 31359712b3 refactor(core): extract remaining inline SQL from electron adapters
- Extract getPrivateChatMemberAvatar to @openchatlab/core
- Extract getExportSessionData (meta/members/messages SQL) to @openchatlab/core
- Extract loadSessionMessages for summary generation to @openchatlab/core
- Replace inline SQL in summary buildDeps with shared core query functions
- Extract createCompressionLlmAdapter factory to @openchatlab/node-runtime
- Simplify server and electron compression adapter creation via shared factory
2026-05-18 23:36:42 +08:00
digua 53fa23f488 refactor(ai): unify formatAIError and extract FTS search SQL to core
- Extract formatAIError to @openchatlab/node-runtime, eliminate duplicate in Electron and Server
- Extract stripAvatarFields to @openchatlab/core
- Extract FTS message search SQL to core (searchMessagesWithFtsAsync)
- Remove redundant sync searchMessagesWithLike from Electron worker
2026-05-18 23:28:39 +08:00
digua 0d454d8a52 refactor(core): extract session time queries and remove dead code
- Remove dead code: electron/main/ai/agent/content-parser.ts (duplicate of @openchatlab/core)
- Extract getSessionsByTimeRange and getRecentChatSessions to @openchatlab/core
- Route IPC session:getByTimeRange and session:getRecent through Worker instead of inline SQL
- Eliminate duplicate getSessionSummaries in workerManager.ts, delegate to core via Worker
2026-05-18 23:22:25 +08:00
digua 1536e241fc docs: add versioning and commit scope conventions to AGENTS.md
- All distribution targets share root package.json version (lock-step)
- Commit scope: free-form for shared changes, platform scope (electron/
  cli/web) only when the change is platform-specific
2026-05-18 22:58:00 +08:00
digua 620ae40d45 fix(web): show app version in sidebar instead of empty string
Inject __APP_VERSION__ from package.json via Vite define, read it in
WebPlatformAdapter.getVersion(), and call getVersion() on all platforms
(not just Electron) in Sidebar onMounted.
2026-05-18 22:43:00 +08:00
digua e5dc1fcb9b fix(dev): tsx watch not detecting workspace package changes
tsx watch's symlink-based file watching doesn't trigger on changes to
pnpm workspace packages (tsx#221). Add --include flags for packages/core
and packages/node-runtime source directories to both dev:serve script
and vite.web.config.mts auto-serve plugin. Extract long command to
scripts/dev-serve.sh for cleaner package.json.
2026-05-18 22:43:00 +08:00
digua cdb2bfea7c feat: unify LLM streaming with thinking display and Web/cache API shims
- Extract runSimpleLlmStream to @openchatlab/node-runtime for shared
  Electron/Server streaming with thinking_start/delta/end event handling
- Simplify Electron IPC ai.ts by delegating to shared stream function
- Add thinking/thinkingDone fields to LlmStreamChunk and preload types
- Add collapsible thinking process display in SQL LAB with auto-scroll
- Implement Web chatStream via SSE with proper error propagation
- Add /_web/cache/save-to-downloads and show-in-folder server routes
- Add cacheApi shim in Web registry for server-side file operations
- Increase SQL LAB maxTokens to 4096 to accommodate reasoning models
- Detect finishReason:length and show token limit error message
- Add i18n keys for thinking/thinkingProcess/errorTokenLimit (4 locales)
2026-05-18 22:32:00 +08:00
digua 58a558b7aa fix(server): align web AI agent with Electron implementation
- Replace mapCoreEventToStream with shared AgentEventHandler for rich
  status events (contextTokens, totalUsage, updatedAt)
- Add formatAIError for user-friendly error messages (429/503/quota)
- Compute maxToolResultTokens dynamically from model contextWindow
- Add steerMessage (answerWithoutTools) and onConvertToLlm callback
- Forward ownerInfo/mentionedMembers from web frontend to server
2026-05-18 22:23:44 +08:00
digua 7190606ca7 refactor: extract AI translations to shared i18n module in node-runtime
Move AI-related translations (tools, agent prompts, LLM, summary) from
Electron-specific locale files into @openchatlab/node-runtime, enabling
single-source maintenance across Electron and Server.

- Add packages/node-runtime/src/ai/i18n/ with createAiTranslate() and
  bundled locale data (zh-CN, en-US, ja-JP, zh-TW)
- Electron locale files now compose: Electron-specific keys + shared AI
  translations from node-runtime
- Server agent.ts switches to shared buildSystemPrompt with evidence
  policy, data snapshot injection, ownerInfo and mentionedMembers support
- Server route fetches dataSnapshot from getChatOverview before agent run
2026-05-18 22:04:01 +08:00
digua 1620a5b4a3 refactor: remove pure re-export files from electron/main
Delete 9 files that were pure passthrough re-exports with zero logic:
- database/sessionCache.ts → consumers import @openchatlab/node-runtime directly
- nlp/stopwords.ts, ftsTokenizer.ts, types.ts, segmenter.ts, index.ts → consumers import @openchatlab/core or @openchatlab/node-runtime directly
- ai/agent/event-handler.ts → consumer imports @openchatlab/node-runtime directly
- ai/llm/provider-registry.ts, model-catalog.ts → consumer imports @openchatlab/core directly

This eliminates unnecessary indirection layers while keeping thin adapters
that contain real Electron-specific logic (DI, singletons, path wiring).
2026-05-18 21:19:15 +08:00
digua 09cdf165b5 feat(server): add merge HTTP routes and simplify Electron merger layer
- Server: add MergeSessionCache + 5 merge endpoints (parse, conflicts, execute, clear, export-for-merge)
- Electron tempCache.ts: delegate to shared TempDbReader/TempDbWriter via BetterSqliteAdapter
- Electron merger/index.ts: remove dead wrapTempDbReader, use reader.toDataSource() directly
2026-05-18 21:07:04 +08:00
digua a3a1d289ba feat(node-runtime): extract Markdown export engine and add server export route
- Extract exportFilterResultToMarkdown to node-runtime/export with writer abstraction
- Electron export.ts becomes thin wrapper (fs.createWriteStream + worker IPC progress)
- Server adds POST /_web/sessions/:id/export/markdown returning downloadable Markdown
2026-05-18 21:01:27 +08:00
digua 8c130c7221 feat(core): unified SQL execution with executeSql() and getSchemaDetailed()
- Add executeSql() with options: columnar, timing, maxRows (stmt.readonly + regex fallback)
- Add getSchemaDetailed() returning table + column info via PRAGMA table_info
- Electron sql.ts delegates to shared executeSql (columnar=true, timing=true)
- Server web.ts uses executeSql (with timing) and getSchemaDetailed (simplified schema route)
2026-05-18 20:45:24 +08:00
digua 9ee3060377 feat(node-runtime): extract session cache to shared package
- Extract JSON cache CRUD + overview/members compute to node-runtime/cache
- Functions now accept DatabaseAdapter instead of better-sqlite3 Database
- Electron sessionCache.ts becomes pure re-export
- Updated all Electron callers to wrap DB with BetterSqliteAdapter
2026-05-18 20:41:48 +08:00
digua 064f3ce144 feat(node-runtime): extract chat DB migrations and add auto-migration on writable open
- Extract migration definitions (v1-v4) to shared getChatDbMigrations() with DI for tokenizer
- DatabaseManager.openWritable() auto-runs pending migrations
- Electron migrations.ts becomes thin wrapper delegating to shared definitions
2026-05-18 20:38:38 +08:00