Commit Graph
725 Commits
Author SHA1 Message Date
digua 05394c2155 feat(cli): auto-detect and migrate Electron desktop data on first CLI run
When a desktop app user installs the CLI, automatically detect existing
Electron data directory (including custom paths via storage.json),
repoint config.toml, and migrate system data (ai, settings, cache, etc.)
to ~/.chatlab/. Block CLI execution with actionable instructions when
Electron was used but databases cannot be located.
2026-05-19 22:19:00 +08:00
digua 0aa30359ab fix(cli): resolve ESM module resolution for tsx dev mode
- Remove "type": "module" from apps/cli/package.json to avoid
  ESM/CJS interop issues with workspace packages in tsx dev mode
- Configure tsup to output .mjs files (always treated as ESM by Node.js)
- Rename bin entry from chatlab.js to chatlab.mjs
- Change chatlab web default port to 3100 (serve keeps 3110)
- Fix vite plugin: use pnpm instead of npx for tsx invocation
2026-05-19 22:13:00 +08:00
digua e363e8bd75 refactor(cli): unify all CLI text to English
All command descriptions, console output, option labels, comments
and error messages are now in English, consistent with AGENTS.md
rules and npm package positioning.
2026-05-19 21:59:00 +08:00
digua 09d4b1ddac feat(cli): add chatlab web command for one-click Web UI launch
- New `chatlab web` subcommand: starts HTTP API + serves Web SPA + auto-opens browser
- Options: --port, --host, --no-open
- Cross-platform `openBrowser()` using open/start/xdg-open
- Add `setWebMode()` in auth: static resources bypass token auth in web mode
- Add `build:full` and `prepare:web` scripts for dist-web preparation
2026-05-19 21:53:00 +08:00
digua b1427e2422 feat(cli): add tsup build pipeline for npm publishing
- Rename package to `chatlab-cli`, version 0.1.0
- Add tsup config with ESM output, bundle @openchatlab/* + stream-json
- Update bin/chatlab.js from stub to actual entry point
- Unify version reading via src/version.ts (CLI/MCP/HTTP API)
- Add publishing fields: type, files, engines, publishConfig, keywords
- Move @openchatlab/* to devDependencies (bundled into dist/)
- Add better-sqlite3 to dependencies, @node-rs/jieba to optionalDependencies
- Fix --web default path for bundled dist/ directory structure
2026-05-19 21:46:00 +08:00
digua 78e41a65b5 refactor: migrate CLI from packages/server to apps/cli
Move the CLI application from packages/server/ to apps/cli/ to align
with the monorepo convention (apps/ for deployable applications,
packages/ for shared libraries). Rename package to @openchatlab/chatlab.

Also fix a bug in ensureToken() that used raw string manipulation to
write config.toml, causing duplicate token fields that corrupted the
TOML file. This led to config parse failures and user_data_dir being
overwritten with defaults. Now uses writeConfigField() for safe writes.
2026-05-19 21:33:00 +08:00
digua 9db30879d3 fix(electron): decouple worker thread from electron module dependency
Worker thread crashed in packaged builds with "Cannot find module 'electron'"
because streamImport.ts transitively imported paths.ts (which requires electron)
through merger/tempCache.ts. Replaced with worker-local implementations that
receive tempDir via workerData, matching the existing dbDir/cacheDir pattern.
2026-05-19 21:13:59 +08:00
digua ba851f4ae9 fix(electron): fix data migration path mismatch after package extraction
The extraction of Electron code to apps/desktop/ changed the package name
from "ChatLab" to "@openchatlab/desktop", which caused app.getPath('userData')
to return a wrong path in dev mode. This made the migration system unable to
find existing databases, resulting in a fresh-app experience for upgrading users.

- Add productName "ChatLab" to desktop package.json to ensure correct userData path
- Add verifyDataPath() safety net: auto-correct config.toml when configured dir
  has no databases but old Electron path does
2026-05-19 20:39:25 +08:00
digua 1c6f4b3776 fix(electron): avoid install-app-deps during workspace install 2026-05-19 19:40:54 +08:00
digua c08a465116 chore(electron): remove Linux desktop build targets
Linux users will be served by the upcoming CLI package instead.
2026-05-19 19:17:59 +08:00
digua d058e8a60f fix(electron): fix CI build by pinning electron version and cleanup
- Pin electron to 35.7.5 (electron-builder requires fixed version)
- Remove redundant @electron/rebuild (built into electron-builder)
- Use electron-builder install-app-deps for postinstall
2026-05-19 19:14:11 +08:00
digua 9099b15ce6 chore(deps): upgrade vite toolchain 2026-05-19 18:39:14 +08:00
digua e80d8e471e docs: update 2026-05-19 00:40:00 +08:00
digua 2bcbb84711 feat: add interactive dev mode selector (pnpm dev)
Adds a zero-dependency Node.js script with arrow-key selection
for choosing between Desktop (Electron), Web (CLI serve + frontend),
Server Only (API backend), and Frontend-only dev modes.

Remembers last selection in node_modules/.cache/dev-mode.
2026-05-19 00:35:00 +08:00
digua b12c07dc32 refactor: extract Electron shell to apps/desktop/
Move Electron-specific code (main, preload, shared) and configs
(electron.vite, electron-builder) into apps/desktop/ to prepare
for multi-platform support (Web Browser standalone).

- Create apps/desktop/ with dedicated package.json and tsconfig
- Split Electron deps (electron, electron-updater, better-sqlite3,
  etc.) from shared frontend deps in root package.json
- Declare better-sqlite3 and @node-rs/jieba in packages/node-runtime
- Add stream-json to packages/parser where it belongs
- Update all path aliases (@electron/* → apps/desktop/*) in tsconfig,
  vite configs, and relative imports in preload/main
- Keep slim root tsconfig.node.json as shared base for packages
- Update CI/CD release workflow for new directory layout
- Fix pre-existing type errors (pi-ai Model generic, SQLLabTab
  LLMChatStreamChunk, filter.test.ts empty methods)
2026-05-19 00:31:00 +08:00
digua ba29346bfe refactor: move chart-* modules from packages/ to src/features/
These are frontend-only Vue/ECharts components that depend on app internals (@/),
not cross-platform shared packages. Moving them to src/features/charts-* keeps
packages/ purely for reusable Node.js libraries.
2026-05-18 23:57:00 +08:00
digua d3ce4023e5 fix(core): add missing re-exports in query/index.ts
The query barrel file was missing re-exports for getSessionsByTimeRange,
getRecentChatSessions, loadSessionMessages, getPrivateChatMemberAvatar,
getExportSessionData, and searchMessagesWithFtsAsync.
2026-05-18 23:43:00 +08:00
digua 31359712b3 refactor(core): extract remaining inline SQL from electron adapters
- Extract getPrivateChatMemberAvatar to @openchatlab/core
- Extract getExportSessionData (meta/members/messages SQL) to @openchatlab/core
- Extract loadSessionMessages for summary generation to @openchatlab/core
- Replace inline SQL in summary buildDeps with shared core query functions
- Extract createCompressionLlmAdapter factory to @openchatlab/node-runtime
- Simplify server and electron compression adapter creation via shared factory
2026-05-18 23:36:42 +08:00
digua 53fa23f488 refactor(ai): unify formatAIError and extract FTS search SQL to core
- Extract formatAIError to @openchatlab/node-runtime, eliminate duplicate in Electron and Server
- Extract stripAvatarFields to @openchatlab/core
- Extract FTS message search SQL to core (searchMessagesWithFtsAsync)
- Remove redundant sync searchMessagesWithLike from Electron worker
2026-05-18 23:28:39 +08:00
digua 0d454d8a52 refactor(core): extract session time queries and remove dead code
- Remove dead code: electron/main/ai/agent/content-parser.ts (duplicate of @openchatlab/core)
- Extract getSessionsByTimeRange and getRecentChatSessions to @openchatlab/core
- Route IPC session:getByTimeRange and session:getRecent through Worker instead of inline SQL
- Eliminate duplicate getSessionSummaries in workerManager.ts, delegate to core via Worker
2026-05-18 23:22:25 +08:00
digua 1536e241fc docs: add versioning and commit scope conventions to AGENTS.md
- All distribution targets share root package.json version (lock-step)
- Commit scope: free-form for shared changes, platform scope (electron/
  cli/web) only when the change is platform-specific
2026-05-18 22:58:00 +08:00
digua 620ae40d45 fix(web): show app version in sidebar instead of empty string
Inject __APP_VERSION__ from package.json via Vite define, read it in
WebPlatformAdapter.getVersion(), and call getVersion() on all platforms
(not just Electron) in Sidebar onMounted.
2026-05-18 22:43:00 +08:00
digua e5dc1fcb9b fix(dev): tsx watch not detecting workspace package changes
tsx watch's symlink-based file watching doesn't trigger on changes to
pnpm workspace packages (tsx#221). Add --include flags for packages/core
and packages/node-runtime source directories to both dev:serve script
and vite.web.config.mts auto-serve plugin. Extract long command to
scripts/dev-serve.sh for cleaner package.json.
2026-05-18 22:43:00 +08:00
digua cdb2bfea7c feat: unify LLM streaming with thinking display and Web/cache API shims
- Extract runSimpleLlmStream to @openchatlab/node-runtime for shared
  Electron/Server streaming with thinking_start/delta/end event handling
- Simplify Electron IPC ai.ts by delegating to shared stream function
- Add thinking/thinkingDone fields to LlmStreamChunk and preload types
- Add collapsible thinking process display in SQL LAB with auto-scroll
- Implement Web chatStream via SSE with proper error propagation
- Add /_web/cache/save-to-downloads and show-in-folder server routes
- Add cacheApi shim in Web registry for server-side file operations
- Increase SQL LAB maxTokens to 4096 to accommodate reasoning models
- Detect finishReason:length and show token limit error message
- Add i18n keys for thinking/thinkingProcess/errorTokenLimit (4 locales)
2026-05-18 22:32:00 +08:00
digua 58a558b7aa fix(server): align web AI agent with Electron implementation
- Replace mapCoreEventToStream with shared AgentEventHandler for rich
  status events (contextTokens, totalUsage, updatedAt)
- Add formatAIError for user-friendly error messages (429/503/quota)
- Compute maxToolResultTokens dynamically from model contextWindow
- Add steerMessage (answerWithoutTools) and onConvertToLlm callback
- Forward ownerInfo/mentionedMembers from web frontend to server
2026-05-18 22:23:44 +08:00
digua 7190606ca7 refactor: extract AI translations to shared i18n module in node-runtime
Move AI-related translations (tools, agent prompts, LLM, summary) from
Electron-specific locale files into @openchatlab/node-runtime, enabling
single-source maintenance across Electron and Server.

- Add packages/node-runtime/src/ai/i18n/ with createAiTranslate() and
  bundled locale data (zh-CN, en-US, ja-JP, zh-TW)
- Electron locale files now compose: Electron-specific keys + shared AI
  translations from node-runtime
- Server agent.ts switches to shared buildSystemPrompt with evidence
  policy, data snapshot injection, ownerInfo and mentionedMembers support
- Server route fetches dataSnapshot from getChatOverview before agent run
2026-05-18 22:04:01 +08:00
digua 1620a5b4a3 refactor: remove pure re-export files from electron/main
Delete 9 files that were pure passthrough re-exports with zero logic:
- database/sessionCache.ts → consumers import @openchatlab/node-runtime directly
- nlp/stopwords.ts, ftsTokenizer.ts, types.ts, segmenter.ts, index.ts → consumers import @openchatlab/core or @openchatlab/node-runtime directly
- ai/agent/event-handler.ts → consumer imports @openchatlab/node-runtime directly
- ai/llm/provider-registry.ts, model-catalog.ts → consumer imports @openchatlab/core directly

This eliminates unnecessary indirection layers while keeping thin adapters
that contain real Electron-specific logic (DI, singletons, path wiring).
2026-05-18 21:19:15 +08:00
digua 09cdf165b5 feat(server): add merge HTTP routes and simplify Electron merger layer
- Server: add MergeSessionCache + 5 merge endpoints (parse, conflicts, execute, clear, export-for-merge)
- Electron tempCache.ts: delegate to shared TempDbReader/TempDbWriter via BetterSqliteAdapter
- Electron merger/index.ts: remove dead wrapTempDbReader, use reader.toDataSource() directly
2026-05-18 21:07:04 +08:00
digua a3a1d289ba feat(node-runtime): extract Markdown export engine and add server export route
- Extract exportFilterResultToMarkdown to node-runtime/export with writer abstraction
- Electron export.ts becomes thin wrapper (fs.createWriteStream + worker IPC progress)
- Server adds POST /_web/sessions/:id/export/markdown returning downloadable Markdown
2026-05-18 21:01:27 +08:00
digua 8c130c7221 feat(core): unified SQL execution with executeSql() and getSchemaDetailed()
- Add executeSql() with options: columnar, timing, maxRows (stmt.readonly + regex fallback)
- Add getSchemaDetailed() returning table + column info via PRAGMA table_info
- Electron sql.ts delegates to shared executeSql (columnar=true, timing=true)
- Server web.ts uses executeSql (with timing) and getSchemaDetailed (simplified schema route)
2026-05-18 20:45:24 +08:00
digua 9ee3060377 feat(node-runtime): extract session cache to shared package
- Extract JSON cache CRUD + overview/members compute to node-runtime/cache
- Functions now accept DatabaseAdapter instead of better-sqlite3 Database
- Electron sessionCache.ts becomes pure re-export
- Updated all Electron callers to wrap DB with BetterSqliteAdapter
2026-05-18 20:41:48 +08:00
digua 064f3ce144 feat(node-runtime): extract chat DB migrations and add auto-migration on writable open
- Extract migration definitions (v1-v4) to shared getChatDbMigrations() with DI for tokenizer
- DatabaseManager.openWritable() auto-runs pending migrations
- Electron migrations.ts becomes thin wrapper delegating to shared definitions
2026-05-18 20:38:38 +08:00
digua f78077655f fix(server): auto-generate session index after import
After streaming or incremental import, automatically generate the
session index (chat_session table) so the frontend does not show
the "generate index" prompt when entering a conversation.
2026-05-18 20:21:30 +08:00
digua 7b29b34e9a feat(server): rewire import to shared StreamingImporter + add incremental routes
- Replace buffered import with high-performance streaming pipeline
- Add POST /_web/sessions/:id/import/incremental (SSE)
- Add POST /_web/sessions/:id/import/incremental/analyze
- Add POST /_web/import/analyze (dry-run)
- CLI import now reports diagnostics from StreamingImporter
2026-05-18 20:06:57 +08:00
digua 58f87133c7 refactor(electron): rewire worker import modules to shared implementations
streamImport.ts: 1080 -> ~110 lines, delegates to node-runtime StreamingImporter
incrementalImport.ts: 451 -> ~65 lines, delegates to node-runtime IncrementalImporter
Both inject Electron-specific deps: IPC progress, better-sqlite3 adapter, cache hook
2026-05-18 20:03:11 +08:00
digua 8d430249c9 feat(node-runtime): extract TempDbWriter/Reader and session export
Platform-agnostic temp DB management for merge operations using
DatabaseAdapter. Includes TempDbWriter (streaming write), TempDbReader
(streaming read + MergerDataSource adapter), exportSessionToJson,
and cleanup helpers.
2026-05-18 20:00:59 +08:00
digua 2e564e8063 feat(node-runtime): extract IncrementalImporter from Electron worker
Platform-agnostic incremental import with dual-path dedup
(platformMessageId + content hash), member upsert, meta patching,
FTS sync, and injectable dependencies (IncrementalImportDeps).
2026-05-18 19:58:25 +08:00
digua bf91c4b7df feat(node-runtime): extract StreamingImporter from Electron worker
Platform-agnostic streaming import with batched transactions, deferred
indexes, nickname history tracking, FTS indexing, and format fallback.
All Electron-specific dependencies replaced with injectable interfaces
(StreamImportDeps, ImportLogger, ImportProgressCallback).
2026-05-18 19:56:27 +08:00
digua 3830ccdd1b feat(server): wire shared managers, FTS, and summary routes to web API
Add Assistant/Skill CRUD routes using shared AssistantManager/SkillManagerCore,
FTS search/rebuild routes, and session summary generation routes. Clean up
unused imports in ai.ts.
2026-05-18 19:43:16 +08:00
digua 1294448959 refactor(electron): introduce ElectronPathProvider and convert modules to PathProvider
Create ElectronPathProvider implementing @openchatlab/core PathProvider interface,
and convert 12 modules from direct paths.ts imports to PathProvider-based access.

Key changes:
- New: ElectronPathProvider + path-context.ts singleton
- assistant/manager.ts: delegate to @openchatlab/node-runtime AssistantManager (394→115 lines)
- skills/manager.ts: delegate to @openchatlab/node-runtime SkillManagerCore (369→100 lines)
- 10 other modules: replace getXxxDir() imports with getPathProvider().getXxxDir()

Desktop-only modules (IPC, workerManager, i18n, index) intentionally kept as-is.
2026-05-18 19:25:12 +08:00
digua f597c9b1b8 chore(electron): remove dead code crypto.ts and device-key.ts
encryptApiKey/decryptApiKey and device key management have zero
importers — legacy functionality already handled by
@openchatlab/config/migrations/crypto-legacy. Removes ~278 lines.
2026-05-18 19:07:22 +08:00
digua 249df9c8e3 feat(node-runtime): extract RAG subsystem to shared package
Move the entire RAG module (~2458 lines) from electron/main/ai/rag/ to
@openchatlab/node-runtime. Includes chunking, embedding, vector store,
and semantic pipeline.

Key changes:
- Config operations use injected aiDataDir instead of getAiDataDir()
- Logger uses injectable RagLogger interface
- LLM config provided via getLLMConfig/getAssistantConfig callbacks
- Deduplicated cosineSimilarity into shared utils.ts
- Electron rag/index.ts is now a thin wrapper (~128 lines)
2026-05-18 19:06:01 +08:00
digua 172d227caa refactor(server): delegate import data writing to shared writeParseResultToDb
Replace ~120 lines of inline SQL in server stream-import.ts and ~60 lines
in importer.ts fullImport with calls to shared writeParseResultToDb from
@openchatlab/node-runtime. Server imports now gain name history tracking
that was previously Electron-only.
2026-05-18 18:55:01 +08:00
digua 1796731535 feat(node-runtime): extract import perf logger to shared package
Move performance logging from electron/main/worker/core/perfLogger.ts to
@openchatlab/node-runtime. Log directory is now caller-provided instead
of derived from Electron paths. Electron wrapper provides the path.

Reduces Electron perfLogger from ~181 lines to ~28 lines.
2026-05-18 18:50:30 +08:00
digua 0017dcbf9e feat(node-runtime): extract FTS5 index operations to shared package
Move core FTS operations (create, build, rebuild, insert, search) to
@openchatlab/node-runtime working with DatabaseAdapter. Electron fts.ts
becomes a thin adapter that adds session-level DB connection management.

Reduces Electron fts.ts from ~207 lines to ~90 lines of pure wrappers.
2026-05-18 18:49:14 +08:00
digua c0e1647ab6 feat(core): extract updateSessionOwnerId and renameSession to @openchatlab/core
Move pure SQL session CRUD operations (owner_id update, rename) to the
shared core package as DatabaseAdapter functions. Electron wrappers now
delegate to core, keeping only connection management and error handling.
2026-05-18 18:47:35 +08:00
digua 1f11f94212 feat(node-runtime): extract writeParseResultToDb to shared import module
Move the core data writing logic (meta/member/message insertion with
member_name_history tracking) from electron/main/database/core.ts to
@openchatlab/node-runtime. Electron importData() now delegates to the
shared function via BetterSqliteAdapter, reducing ~130 lines to ~10.

This enables both Electron and Server to share the same import logic
including name history tracking.
2026-05-18 18:45:40 +08:00
digua dc5978fa78 refactor(electron): unify migration system with @openchatlab/core runMigrations
Replace Electron's independent migration runner with core's runMigrations
engine. Migration scripts stay in Electron layer (they depend on NLP
tokenizer and i18n), but execution logic (version check, transaction,
version bump) is now delegated to the shared framework.

- Re-export CURRENT_SCHEMA_VERSION from @openchatlab/core
- Wrap better-sqlite3 Database in BetterSqliteAdapter for core compat
- Convert migration up() signatures to accept DatabaseAdapter
- Remove ~100 lines of duplicated migration infrastructure
2026-05-18 18:42:17 +08:00
digua 1fb608a4ed refactor(electron): replace inline DDL with @openchatlab/core schema constants
Remove ~76 lines of duplicated CREATE TABLE/INDEX/FTS SQL from
database/core.ts createDatabase(), delegating to CHAT_DB_SCHEMA and
FTS_TABLE_SCHEMA from @openchatlab/core — the single source of truth.
2026-05-18 18:39:12 +08:00
CEQ151 3583548f1f fix: strengthen agent evidence retrieval 2026-05-18 17:31:54 +08:00