Pass a model download proxy through the semantic index worker and local embedding provider so local Hugging Face downloads use the effective proxy.
Restart idle local workers when the proxy changes before preload, build, or search calls, and initialize app logging inside the worker process for preload diagnostics.
Combine the PR review fixes for push import validation, deduplication, system sender handling, and pull import lock scoping into one reviewable change.
- Change default port from 5200 to 3110 in sync config-manager, Vue store,
and settings UI to match CLI DEFAULT_API_PORT
- Desktop API server now tries up to 10 consecutive ports when preferred
port is occupied (e.g. CLI already running); reports actual bound port
- Scope Desktop import route lock to session (Set<string>) matching the
push-importer fix
- Docs: remove unimplemented JSONL/X-Dry-Run claims, fix port references,
replace platform-specific examples with generic ones
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Reject path-traversal sessionIds (SAFE_SESSION_ID_RE)
- Reject X-Dry-Run header with 400 instead of silently writing
- Propagate DataDirCompatibilityError as 409 instead of swallowing to 500
- Scope concurrency lock to session instead of process-wide boolean
- Fix hash dedup hole: load hashes for ALL existing messages, not just
those without platformMessageId
- Fix backfill segment index: detect when written timestamps predate the
existing tail and use full generateSessionIndex rebuild instead of
incremental; apply same fix to shared incremental-importer used by
Desktop and pull sync
- Validate member platformId, message content/platformMessageId types, and
option enum values in validatePayload before any writes
- Use writeParseResultToDb in fullImport to populate member_name_history;
pre-deduplicate messages before delegating and auto-create minimal members
for senders absent from the members list
- Add content-hash to existingKeys when accepting a pmid-keyed row so a
same-content no-pmid copy later in the same batch is caught
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Session index (segment/message_context tables) was previously generated
inconsistently: CLI used postImportHook, Desktop relied on a separate
explicit RPC call, and the CLI sync/pull legacy importer never built it.
Move generateSessionIndex / generateIncrementalSessionIndex into the
shared importers (streaming-importer.ts, incremental-importer.ts) so
every caller gets it automatically. Remove the now-redundant calls from
CLI's postImportHook. Also add the calls to the legacy importer.ts used
by CLI sync/pull, which had no session index at all.
Incremental imports use generateIncrementalSessionIndex, which preserves
existing AI summaries. Full imports call generateSessionIndex only on
newly created sessions where no summaries exist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Both fields are documented in chatlab-api.md but were missing from the
response. lastPlatformMessageId is used as the incremental import boundary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the Push Import protocol documented in chatlab-import.md.
Creates a new session on first call, appends and deduplicates on
subsequent calls (platformMessageId preferred, content hash fallback).
Adds IMPORT_IN_PROGRESS (409) and IMPORT_FAILED (500) error codes.
Includes a global concurrency lock and incremental session index
generation after each write.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- stream-json and stream-chain were used directly in GoogleChatTakeoutResolver
but not declared in node-runtime's package.json; fixes module resolution
failure in non-hoisted installs
- when detectFormat returns multiChat but scanMultiChatFile returns empty,
the code fell through to single-file import and could silently import the
first chat; now returns early with an error message in both cases
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Verify end-to-end parsing of localized archive structures and update
semantic-index route fake to match current service interface.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add archive reader, Google Chat resolver and source manager in node-runtime.
Expose prepared archive sources via CLI HTTP route and Electron IPC.
Add frontend service adapters and an import selection flow with i18n support
for all four locales. Fix rescanning on repeated archive imports.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add streaming zip reader dependency (fflate) and a new parser for Google
Chat takeout conversation files. Handles localized message types, reactions,
topic threads, and deduplication of multi-file exports.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Front-end error reporter was posting to /_web/_web/logs/report (double
base URL prefix); fixed to /logs/report
- uncaughtException/unhandledRejection handlers in CLI and Electron did not
exit after logging, overriding Node's default crash behavior; added
process.exit(1)
- NodePathProvider constructor in run() and chatlab status triggered
migrateFromElectronIfNeeded() as a side effect; added getSystemLogsDir()
to get the system logs path without resolving user data
- formatData() was called outside write()'s try/catch, so errors with
circular cause objects could escape appLogger.error(); moved inside
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Build a shared appLogger (node-runtime) that all three runtimes write to a
single rolling logs/app.log (10 MB → rename to app.old.log). Replaces the
four independent hand-written loggers (app.log/api.log Electron wrappers)
with a unified entry point while leaving AiLogger/perf-logger intact.
Key additions:
- packages/node-runtime/src/logging/app-logger.ts: initAppLogger/appLogger
singleton, INFO threshold by default (CHATLAB_LOG_LEVEL to override),
atomic rename rotation, Error data auto-extracts message+stack
- Three-runtime init: Electron main, CLI Web (http/index.ts), CLI (run())
- Global crash capture: uncaughtException + unhandledRejection on all three
Node runtimes now land in app.log scope 'crash'
- Fastify setErrorHandler fallback logs scope 'http' on CLI Web and Electron
internal server for unexpected 5xx errors
- Frontend: window.onerror + unhandledrejection + Vue errorHandler feed into
POST /_web/logs/report (scope 'web') with dedup+throttle
- AGENTS.md: logging rules section requiring key-path logs on new/changed features
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- SemanticIndexSessionModal called act('disable') and referenced the
removed i18n key action.disable after the rename to remove; update
both the action call and i18n key.
- setConfig triggered preload even when saved.enabled is false; guard
preload behind saved.enabled so disabling the feature does not
download the local embedding model.
- worker-client setConfig returned early when transport was null, so
preload never ran in a fresh Electron session after model selection;
start the worker for local+enabled configs to ensure preload executes.
- rebuild() called computeDbPathHash(sessionId) directly instead of
resolveHash(), creating a duplicate state row for sessions stored
under a legacy full-dbPath hash; use resolveHash() and pass the
resolved hash to enableParams() to update the existing record.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace "停用" (disable, deferred cleanup) with "移除" (remove, immediate):
- Cancels any active build job
- Deletes vector chunks immediately (store.deleteByDbPathHash)
- Removes the state record entirely (stateStore.remove)
Applies to both the per-session button and the picker's deselect flow.
Fix legacy hash mismatch: old records were stored with a hash computed from
the full dbPath; current code hashes only the sessionId basename. Add
resolveHash() which falls back to a full-table scan by dbPath basename so
all operations (build/pause/cancel/remove/status/search) can find legacy
records regardless of which hash scheme was used.
Cap the enabled sessions list at max-h-72 with overflow-y-auto so the
settings page does not grow unboundedly with many sessions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Move local embedding model storage from getCacheDir() to getAiDataDir()
(~/.chatlab/ai/models/semantic-index) so the "clear cache" button in
settings no longer wipes downloaded models.
Also trigger model download immediately in the background when a local
model is configured, instead of waiting for the first index build.
Download status (idle/downloading/ready/error) is tracked in
SemanticIndexService, exposed via the SemanticIndexRuntime interface and
worker RPC, and surfaced in the settings UI as a pulsing indicator.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The "单次检索片段数" input was surfaced in settings but adds noise without
meaningful benefit for most users. Remove the UI control entirely and fix
normalizeConfig to always return SEARCH_MAX_RESULTS_DEFAULT (10), ignoring
any previously saved value so existing custom settings are silently reset to
default on next load.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Extract shared AnalyticsService to packages/node-runtime using direct
Aptabase HTTP API (removes @aptabase/electron dependency)
- Add POST /_web/telemetry/track route for generic event reporting
- Desktop: migrate analytics.json from userData to systemDir (~/.chatlab/),
trigger daily active from renderer via new IPC so user-selected locale is used
- CLI: inject APTABASE_APP_KEY at build time, create AnalyticsService on
server start, report app_active with platform: cli-web
- Both platforms report user-selected locale via platform service abstraction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When a model service is renamed and its API key changed simultaneously,
onApiKeyCreated derives a new profile name from the new service name and
writes a new auth profile, but the old profile (keyed by the old name)
was never deleted, leaving an orphaned credential on disk.
Capture the old authProfile before applying updates. If onApiKeyCreated
returns a different profile name, call onApiKeyDeleted on the old config
to trigger cleanup before persisting the new profile.
No cleanup is needed for key-only updates (profile name unchanged) or
name-only updates (onApiKeyCreated is not called).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When deleteConfig was called, the auth profile written to
auth-profiles.json during addConfig was never cleaned up, leaving
orphaned credentials on disk indefinitely.
Add onApiKeyDeleted callback to LLMConfigStoreDeps and call it in
deleteConfig with the removed config so callers can clean up the
corresponding auth profile. Wire up deleteAuthProfile in both the
CLI HTTP server and the Desktop internal API.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Cast FakeSemanticIndexRuntime via unknown to satisfy the full
SemanticIndexRuntime interface in the factory return type.
Enable allowImportingTsExtensions in tsconfig.web.json so test files
importing .mts helpers pass vue-tsc type checking.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the first small response is empty with hasMore=false and carries
sync.nextSince, the file was deleted before that cursor value was saved.
If retry pages also omit nextSince, nextPullSince stays at the old
lastPullAt and the next poll re-fetches the same completed tail window.
Fold sync0.nextSince into nextPullSince immediately after cleanupTempFile,
before the retry loop, matching the existing pattern for retry pages.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
With API batched embeddings, insertChunks() and stateStore.updateProgress()
write to two separate SQLite databases, so they cannot be atomic. If the
worker is terminated between the two writes, the next resume attempts to
re-insert the same chunk_ids and hits the UNIQUE constraint, leaving the
session stuck in 'failed' state.
Use INSERT OR IGNORE on chunk_vector_index and skip the vec-table insert
when changes === 0. Duplicate inserts during resume are now silently
discarded instead of throwing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Centralise the better-sqlite3 native-binding selection for tests in
tests/helpers/sqlite.mts so test files consistently use the CLI-side
binding (apps/cli/native/better_sqlite3.node) rather than the default
one, which may be compiled for a different Node.js ABI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Delete NotificationBus interface (no implementations, no callers)
- Delete useAsyncData/useMultipleAsyncData composables (zero import sites)
- Inline CHART_CAPABILITY_SKILL_ID into tool-catalog.ts (was a single-constant file)
- Inline AsyncSqlExecutor into message-query-functions.ts (was a single-interface file)
- Rename tools TimeFilter → ToolTimeRange to eliminate name collision with shared-types
- Shrink useToast wrapper: two interfaces → one type, 40 lines → 17 lines
- Simplify deepMerge: remove recursion, config is at most 1 level deep
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bring the Electron desktop client to parity with CLI web for semantic
index, and package the local embedding runtime correctly.
- Inject semanticIndexService into the desktop AI tool debug executor so
manual tool runs reach semantic search tools, matching the Agent stream.
Extract a platform-agnostic tool-executor-core with unit tests.
- Persist the API key by reference in the degraded semantic-index config
route (vector service unavailable) so API mode keys are not dropped;
make auth-profile read/write injectable and add route + unit regression
tests that never touch the real ~/.chatlab.
- Package desktop local embedding deps: declare @huggingface/transformers,
asarUnpack onnxruntime-node / @img native libs, and trim onnxruntime-node
binaries to the build target platform/arch to avoid shipping ~175M of
unrelated Linux/Windows binaries.
Co-authored-by: Cursor <cursoragent@cursor.com>
When sqlite-vec fails to load, use SemanticIndexConfigStore directly to
back the GET/PUT /_web/ai/semantic-index/config routes so users can still
configure the vector model from settings. Session management routes (enable,
build, search, etc.) remain unavailable since they require the vector store.
Export SemanticIndexConfigStore, SEMANTIC_INDEX_CONFIG_FILE, and
isSemanticIndexConfigured from node-runtime and semantic-index packages.
When sqlite-vec fails to load (e.g. native binding not found), the service
is null and routes were silently skipped, causing 404 errors in the
settings UI. Register read-only stubs for /config and /enabled so the
settings tab can open without errors when the service is unavailable.
Also export defaultSemanticIndexConfig from the node-runtime package root.
When buildAllPending finds a completed session whose live message count
has shrunk (e.g. after member deletion), enqueue rebuild instead of
skipping it, to remove stale vectors for deleted messages.
Expose hasNewMessages in listEnabledStatuses so the Settings UI pendingCount
correctly reflects sessions with appended messages, unblocking the
"Build Pending" button that was otherwise disabled for stale-completed sessions.
Allow API-mode semantic index configs without an API key when the base
URL does not require one (e.g. local Ollama); update settings UI to
reflect the keyless-allowed state and enable the session modal when a
keyless config is fully configured.
Prevent stale semantic index writes by checking write guard before
persisting chunks; make rebuild detection explicit on app restart via
reactivate() path; preserve full evidence snippet text without truncation;
add regression test for same-second chunk tie-breaking in mapMessageToChunk.
retrieve-chat-evidence:
- repair semantic cleanup and keyword time filter; pass context timeFilter
(seconds→ms) when no explicit date params are provided
- classifyGroup: skip Chinese keyword matching for non-Chinese locales and
update the English LLM instruction to allow judgment on uncertain groups,
so English-language chats are not permanently stuck at "cannot confirm"
- mergeCandidates: replace ID-range overlap check with timestamp overlap
so keyword hits are not incorrectly dropped after incremental imports
backfill older messages with non-monotonic IDs
semantic-search-current-chat:
- forward context.timeFilter (seconds→ms) to searchForTool so the tool
respects session time range the same way other retrieval tools do
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Session management:
- buildAllPending: recheck completed sessions against live message count
to enqueue rebuild when new messages arrived since last index run
- runner: detect non-append-only backfills and force full re-index instead
of resuming from a stale cursor that would skip the backfilled messages
- service/session-state-store: add reactivate() and use it when re-enabling
a stale session so the old model identity is preserved in the state row;
on restart buildAllPending() can still detect isStale() and enqueue rebuild
- chunker-config: use sessionId (not absolute path) as the stable key for
db_path_hash so migrating userDataDir does not orphan existing indexes
- cleanupUnused: cross-check state rows against listSessionIds() to detect
sessions whose DB was deleted and mark them pending cleanup
Retrieval accuracy:
- hasApiKey: guard against missing authProfile to avoid false positives
- FtsSearcher interface: return {id, ts} pairs; fts-searcher fetches ts from
message table so chunk mapping has the correct millisecond timestamp
- store.mapMessageToChunk: use (start_ts, start_message_id) composite ordering
and end-boundary check to break same-second ties deterministically
- message-range-reader: use (ts, id) composite bounds on both sides so
same-second messages at chunk boundaries are not attributed to wrong parents
- evidence.ts: add startTs/endTs to EvidenceHit; locate core evidence
boundaries by hit message IDs to avoid same-second ambiguity
- hybrid-search: expand ANN candidate pool to 10× when time filter is active
to reduce chance of missing in-range chunks in long conversations
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
digua
