mirror of
https://github.com/hellodigua/ChatLab.git
synced 2026-10-26 02:04:37 +08:00
fix(semantic-index): make insertOne idempotent to survive mid-batch crash
With API batched embeddings, insertChunks() and stateStore.updateProgress() write to two separate SQLite databases, so they cannot be atomic. If the worker is terminated between the two writes, the next resume attempts to re-insert the same chunk_ids and hits the UNIQUE constraint, leaving the session stuck in 'failed' state. Use INSERT OR IGNORE on chunk_vector_index and skip the vec-table insert when changes === 0. Duplicate inserts during resume are now silently discarded instead of throwing. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
1 parent
dc1c236cc0
commit
f69f419192
1 file changed
+3
-1
@@ -126,7 +126,8 @@ export class EmbeddingIndexStore {
|
||||
|
||||
const meta = this.db
|
||||
.prepare(
|
||||
`INSERT INTO chunk_vector_index (${CHUNK_COLUMNS})
|
||||
// ponytail: OR IGNORE makes resume after mid-batch crash idempotent; vec insert skipped below if chunk exists
|
||||
`INSERT OR IGNORE INTO chunk_vector_index (${CHUNK_COLUMNS})
|
||||
VALUES (@chunkId, @dbPathHash, @strategyId, @modelId, @dim, @parentId, @startMessageId, @endMessageId,
|
||||
@startTs, @endTs, @messageCount, @rawContentHash, @embeddingInputHash, @chunkerVersion,
|
||||
@chunkerConfigHash, @indexedAt, @status)`
|
||||
@@ -151,6 +152,7 @@ export class EmbeddingIndexStore {
|
||||
status: record.status,
|
||||
})
|
||||
|
||||
if (meta.changes === 0) return
|
||||
this.db
|
||||
.prepare(
|
||||
`INSERT INTO ${vecTableName(record.dim)} (vector_id, db_path_hash, model_id, embedding)
|
||||
|
||||
Reference in new issue
Block a user