Thread ownerId and member aliases through the TS and native parsers. Persist aliases across full and incremental imports so exported ChatLab JSON retains user identity metadata.
Remove persisted annual summary snapshots whenever a session is deleted through shared or desktop cleanup paths. Drop cached in-memory snapshots when their backing files disappear so stale responses cannot expose deleted session data.
The UI stayed on "parsing messages" during the whole post-parse phase:
the importer emitted an 'importing' stage the frontend had no text for,
and on CLI Web the synchronous DB writes / index / FTS build starved the
event loop so buffered SSE progress events only arrived after the import
finished.
- Replace 'importing' with 'saving' (message writes) and a new 'indexing'
stage (indexes / FTS / session index) in ParseProgress, with UI texts
for all four locales.
- Yield a macrotask between native parser batches and before heavy
synchronous steps so SSE progress events flush while importing.
Co-authored-by: Cursor <cursoragent@cursor.com>
getJieba() invalidated its cache whenever no dict file existed on disk,
so users without a custom dict rebuilt a Jieba instance per tokenize call.
FTS indexing tokenizes every message, making imports seconds slower
(43MB sample: 9.2s -> 3.5s after the fix). Cache now only invalidates
when the on-disk dict state actually changes (added or removed).
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(semantic-index): stabilize embedding warmup
Improve warmup job reliability and prevent raw API response body from
reaching app.log. Status code alone is sufficient for diagnosing failures;
stripping the response detail eliminates the risk of chat content being
written to the log via OpenAI-compatible services that echo submitted input.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(semantic-index): clear preload error after warmup retry
When local model preload fails, a later warmup can successfully recreate the extractor through embedDocuments but leave modelPreloadStatus stuck on error.
Mark the local model ready after a completed warmup writes chunks, and cover the retry path with a regression test.
* fix(semantic-index): guard warmup model status updates
Capture the model configuration used by a warmup job before it starts, and only clear local preload errors when the current local model still matches that job.
Add a regression test for switching local models while an older warmup is still completing.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Wait for initial contact rows before scrolling to groupmates so tab navigation does not land on temporary rows. Avoid writing contacts session facts or latest facts when a session DB fingerprint changes during computation.
Pass a model download proxy through the semantic index worker and local embedding provider so local Hugging Face downloads use the effective proxy.
Restart idle local workers when the proxy changes before preload, build, or search calls, and initialize app logging inside the worker process for preload diagnostics.
Combine the PR review fixes for push import validation, deduplication, system sender handling, and pull import lock scoping into one reviewable change.
- Reject path-traversal sessionIds (SAFE_SESSION_ID_RE)
- Reject X-Dry-Run header with 400 instead of silently writing
- Propagate DataDirCompatibilityError as 409 instead of swallowing to 500
- Scope concurrency lock to session instead of process-wide boolean
- Fix hash dedup hole: load hashes for ALL existing messages, not just
those without platformMessageId
- Fix backfill segment index: detect when written timestamps predate the
existing tail and use full generateSessionIndex rebuild instead of
incremental; apply same fix to shared incremental-importer used by
Desktop and pull sync
- Validate member platformId, message content/platformMessageId types, and
option enum values in validatePayload before any writes
- Use writeParseResultToDb in fullImport to populate member_name_history;
pre-deduplicate messages before delegating and auto-create minimal members
for senders absent from the members list
- Add content-hash to existingKeys when accepting a pmid-keyed row so a
same-content no-pmid copy later in the same batch is caught
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Session index (segment/message_context tables) was previously generated
inconsistently: CLI used postImportHook, Desktop relied on a separate
explicit RPC call, and the CLI sync/pull legacy importer never built it.
Move generateSessionIndex / generateIncrementalSessionIndex into the
shared importers (streaming-importer.ts, incremental-importer.ts) so
every caller gets it automatically. Remove the now-redundant calls from
CLI's postImportHook. Also add the calls to the legacy importer.ts used
by CLI sync/pull, which had no session index at all.
Incremental imports use generateIncrementalSessionIndex, which preserves
existing AI summaries. Full imports call generateSessionIndex only on
newly created sessions where no summaries exist.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the Push Import protocol documented in chatlab-import.md.
Creates a new session on first call, appends and deduplicates on
subsequent calls (platformMessageId preferred, content hash fallback).
Adds IMPORT_IN_PROGRESS (409) and IMPORT_FAILED (500) error codes.
Includes a global concurrency lock and incremental session index
generation after each write.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
digua
