Cast FakeSemanticIndexRuntime via unknown to satisfy the full
SemanticIndexRuntime interface in the factory return type.
Enable allowImportingTsExtensions in tsconfig.web.json so test files
importing .mts helpers pass vue-tsc type checking.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the first small response is empty with hasMore=false and carries
sync.nextSince, the file was deleted before that cursor value was saved.
If retry pages also omit nextSince, nextPullSince stays at the old
lastPullAt and the next poll re-fetches the same completed tail window.
Fold sync0.nextSince into nextPullSince immediately after cleanupTempFile,
before the retry loop, matching the existing pattern for retry pages.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
With API batched embeddings, insertChunks() and stateStore.updateProgress()
write to two separate SQLite databases, so they cannot be atomic. If the
worker is terminated between the two writes, the next resume attempts to
re-insert the same chunk_ids and hits the UNIQUE constraint, leaving the
session stuck in 'failed' state.
Use INSERT OR IGNORE on chunk_vector_index and skip the vec-table insert
when changes === 0. Duplicate inserts during resume are now silently
discarded instead of throwing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Centralise the better-sqlite3 native-binding selection for tests in
tests/helpers/sqlite.mts so test files consistently use the CLI-side
binding (apps/cli/native/better_sqlite3.node) rather than the default
one, which may be compiled for a different Node.js ABI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Delete NotificationBus interface (no implementations, no callers)
- Delete useAsyncData/useMultipleAsyncData composables (zero import sites)
- Inline CHART_CAPABILITY_SKILL_ID into tool-catalog.ts (was a single-constant file)
- Inline AsyncSqlExecutor into message-query-functions.ts (was a single-interface file)
- Rename tools TimeFilter → ToolTimeRange to eliminate name collision with shared-types
- Shrink useToast wrapper: two interfaces → one type, 40 lines → 17 lines
- Simplify deepMerge: remove recursion, config is at most 1 level deep
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bring the Electron desktop client to parity with CLI web for semantic
index, and package the local embedding runtime correctly.
- Inject semanticIndexService into the desktop AI tool debug executor so
manual tool runs reach semantic search tools, matching the Agent stream.
Extract a platform-agnostic tool-executor-core with unit tests.
- Persist the API key by reference in the degraded semantic-index config
route (vector service unavailable) so API mode keys are not dropped;
make auth-profile read/write injectable and add route + unit regression
tests that never touch the real ~/.chatlab.
- Package desktop local embedding deps: declare @huggingface/transformers,
asarUnpack onnxruntime-node / @img native libs, and trim onnxruntime-node
binaries to the build target platform/arch to avoid shipping ~175M of
unrelated Linux/Windows binaries.
Co-authored-by: Cursor <cursoragent@cursor.com>
When sqlite-vec fails to load, use SemanticIndexConfigStore directly to
back the GET/PUT /_web/ai/semantic-index/config routes so users can still
configure the vector model from settings. Session management routes (enable,
build, search, etc.) remain unavailable since they require the vector store.
Export SemanticIndexConfigStore, SEMANTIC_INDEX_CONFIG_FILE, and
isSemanticIndexConfigured from node-runtime and semantic-index packages.
When sqlite-vec fails to load (e.g. native binding not found), the service
is null and routes were silently skipped, causing 404 errors in the
settings UI. Register read-only stubs for /config and /enabled so the
settings tab can open without errors when the service is unavailable.
Also export defaultSemanticIndexConfig from the node-runtime package root.
sqlite-vec loads its native .dylib/.dll/.so via require.resolve(), which
in a packaged Electron app resolves to a path inside app.asar where native
extensions cannot be loaded. Adds sqlite-vec* to asarUnpack so the extension
lands in app.asar.unpacked and require.resolve() returns the correct path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When buildAllPending finds a completed session whose live message count
has shrunk (e.g. after member deletion), enqueue rebuild instead of
skipping it, to remove stale vectors for deleted messages.
Expose hasNewMessages in listEnabledStatuses so the Settings UI pendingCount
correctly reflects sessions with appended messages, unblocking the
"Build Pending" button that was otherwise disabled for stale-completed sessions.
Store only the tool result payload needed for re-rendering, not the full
pre-processed display content, to prevent unbounded growth of the chat
state serialized to disk.
Allow API-mode semantic index configs without an API key when the base
URL does not require one (e.g. local Ollama); update settings UI to
reflect the keyless-allowed state and enable the session modal when a
keyless config is fully configured.
Prevent stale semantic index writes by checking write guard before
persisting chunks; make rebuild detection explicit on app restart via
reactivate() path; preserve full evidence snippet text without truncation;
add regression test for same-second chunk tie-breaking in mapMessageToChunk.
agent-stream-runner: pass timeFilter and maxMessagesLimit from request
params into the tool context so evidence and retrieval tools respect the
active session time range in CLI Web sessions, matching the desktop runner.
tool-adapter: pass preprocessConfig, anonymizeNames and ownerPlatformId to
applyPreprocessingPipeline in the rawMessages branch so desensitization and
name anonymization apply to search_messages / recent_messages results,
matching the desktop adapter behavior.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
retrieve-chat-evidence:
- repair semantic cleanup and keyword time filter; pass context timeFilter
(seconds→ms) when no explicit date params are provided
- classifyGroup: skip Chinese keyword matching for non-Chinese locales and
update the English LLM instruction to allow judgment on uncertain groups,
so English-language chats are not permanently stuck at "cannot confirm"
- mergeCandidates: replace ID-range overlap check with timestamp overlap
so keyword hits are not incorrectly dropped after incremental imports
backfill older messages with non-monotonic IDs
semantic-search-current-chat:
- forward context.timeFilter (seconds→ms) to searchForTool so the tool
respects session time range the same way other retrieval tools do
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Session management:
- buildAllPending: recheck completed sessions against live message count
to enqueue rebuild when new messages arrived since last index run
- runner: detect non-append-only backfills and force full re-index instead
of resuming from a stale cursor that would skip the backfilled messages
- service/session-state-store: add reactivate() and use it when re-enabling
a stale session so the old model identity is preserved in the state row;
on restart buildAllPending() can still detect isStale() and enqueue rebuild
- chunker-config: use sessionId (not absolute path) as the stable key for
db_path_hash so migrating userDataDir does not orphan existing indexes
- cleanupUnused: cross-check state rows against listSessionIds() to detect
sessions whose DB was deleted and mark them pending cleanup
Retrieval accuracy:
- hasApiKey: guard against missing authProfile to avoid false positives
- FtsSearcher interface: return {id, ts} pairs; fts-searcher fetches ts from
message table so chunk mapping has the correct millisecond timestamp
- store.mapMessageToChunk: use (start_ts, start_message_id) composite ordering
and end-boundary check to break same-second ties deterministically
- message-range-reader: use (ts, id) composite bounds on both sides so
same-second messages at chunk boundaries are not attributed to wrong parents
- evidence.ts: add startTs/endTs to EvidenceHit; locate core evidence
boundaries by hit message IDs to avoid same-second ambiguity
- hybrid-search: expand ANN candidate pool to 10× when time filter is active
to reduce chance of missing in-range chunks in long conversations
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For "what X did we discuss / which ones / what did we talk about" style
enumeration and topic-discovery questions, the model was brute-forcing
search_messages across many keyword-guess rounds instead of using vector
search. Strengthen the semantic-search guidance to send these questions to
semantic_search_current_chat and explicitly discourage multi-round keyword
guessing; restrict search_messages to known exact word/quote/sender/time.
Co-authored-by: Cursor <cursoragent@cursor.com>
Evidence messages carry millisecond timestamps but the preprocessing pipeline
renders times assuming seconds (multiplies by 1000), producing absurd years
(e.g. 56150) inside semantic snippets. Convert ms back to seconds before
rendering. Also strip the per-line time prefix from semantic snippets so the
evidence block shows a single group time instead of a time on every message,
and drop the redundant time span in the group header. Adds regression tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the per-source timestamp line and show a single overall time per group
(falling back to the source min/max when no group range is set). Each source
now renders as a compact sender + snippet line, reducing visual clutter.
Co-authored-by: Cursor <cursoragent@cursor.com>
The evidence block is now collapsed by default, showing only a compact header
with per-status group counts and the retrieval mode. Expanding reveals query,
criteria, warnings, grouped sources and a total-sources footnote. Tightens
spacing throughout.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add a structured "evidence" content block produced by retrieve_chat_evidence:
extract the payload from tool results, persist it in AI chat history, and
render grouped, desensitized sources in EvidenceBlock.vue. Sources open the
chat record drawer at the matched message for traceability. Evidence payload
types move to @openchatlab/core so tools, node-runtime and the frontend share
one definition. Includes zh-CN/en-US/ja-JP/zh-TW i18n.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add an "evidence" planner intent and tiered retrieval guidance so
fact/occurrence/count questions are routed to retrieve_chat_evidence
instead of a single low-level search path.
- planner: new evidence intent + prompt strategy suggesting only
retrieve_chat_evidence for that step
- semantic-search guidance: three-tier tool selection (evidence > semantic > keyword)
- expose retrieve_chat_evidence as an always-on Agent tool (desktop special
append; CLI via AGENT_TOOL_REGISTRY core), semantic tool stays dynamic
- export evidence tool + payload types from @openchatlab/tools
- TDD: planner evidence-intent test
Co-authored-by: Cursor <cursoragent@cursor.com>
Thread an optional timeFilter through SemanticSearchToolOptions →
searchForTool → search → hybridSearch so evidence retrieval no longer
mixes time ranges between semantic and keyword paths.
- hybridSearch filters dense + FTS-mapped chunks by startTs/endTs overlap
(millisecond, single-sided supported) and amplifies the candidate pool
when filtering to avoid under-recall
- retrieve_chat_evidence now passes its parsed ms range to the semantic path
- TDD: hybrid-search overlap unit tests + service wiring test
Co-authored-by: Cursor <cursoragent@cursor.com>
Introduce a high-level evidence retrieval tool for fact/occurrence/count
questions. It resolves auto mode, runs semantic + keyword paths, de-identifies
keyword snippets via an injected desensitizeMessages callback, applies a
tool-layer millisecond time-range fallback (keyword path can't rely on the CLI
provider's timeFilter), and merges/dedupes candidates into conservative groups.
- Add ChatEvidencePayload / group / source types and EvidenceTimeRangeMs to tools
- Register tool in AGENT_TOOL_REGISTRY only (Agent-only, excluded from MCP)
- Inject desensitizeMessages (preprocessMessages) in desktop + CLI adapters
- TDD: tool + registry unit tests
Co-authored-by: Cursor <cursoragent@cursor.com>
Add the missing translations for the v8 database migration across the
desktop locales and cover them with a regression assertion.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add the frontend semantic-index service layer and the index management UI:
a settings section for vector model selection and index lifecycle with live
build-progress polling, a conversation picker plus current-conversation
entry, a session-scoped modal, and the tool-block summary rendering in the
chat message flow. Include the en/ja/zh-CN/zh-TW translations.
Co-authored-by: Cursor <cursoragent@cursor.com>
Add shared /_web/ai/semantic-index HTTP routes and wire the service into
CLI web and Electron internal API. Introduce the semantic_search_current_chat
agent tool (AGENT-only, not MCP) with desensitized output and a narrowed
service interface, forward the semantic-search context through the CLI and
Desktop tool adapters, and integrate on-demand semantic search into the
agent pipeline with guidance and a max-result-token evidence budget.
Co-authored-by: Cursor <cursoragent@cursor.com>
Remove the deprecated always-on RAG system and introduce the new
semantic-index core: parent/child chunker with version/hash derivation,
embedding providers (local + OpenAI-compatible) and profiles, sqlite-vec
vector store, dense+FTS+RRF hybrid retrieval, evidence assembly, serial
warmup job queue and single-session runner, session business-state store,
and the SemanticIndexService that ties them together. Add an independent
config store and a dedicated vector directory via the path providers.
Co-authored-by: Cursor <cursoragent@cursor.com>
- capture worker reference in sendToWorkerWithProgress to match
sendToWorker and avoid stale-closure risk on worker restart
- replace `as any` casts with typed SupportedLocale / BatchSegmentOptions
assertions in debug-executor
- destructure params.params as toolParams to remove unused testId binding
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Switching to a different session left allWords/topicMiniWords/stats
populated with the previous session's data. With the earlier fix that
skips isLoading when allWords is non-empty, users could see and interact
with stale words until the new request resolved.
Extend the sessionId watcher to also clear all word cloud state so the
loading indicator triggers correctly for cross-session navigation while
still keeping old data visible for same-session filter refinements.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- quick-start: simplify download links, add clb alias tip, improve copy
- intro: expand with features, supported platforms table, and next steps
- index: optimize title and descriptions for GEO/SEO
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add CN/EN acknowledgments pages with Leo Oliveira story
- Add English translations of Push and Pull protocol docs
- Register acknowledgments in sidebar as top-level entry
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Mark iMessage as supported, remove WeChat from EN README
- Update dev setup to reflect interactive target selector
- Add docs site links to docs/README.md
- Remove single-PR rule, highlight feature discussion requirement
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
digua
LeoandLeo