perf(backup): batch dump INSERTs and restore preserved tables in one transaction (#6122)

* perf(backup): batch dump INSERTs and restore preserved tables in one transaction

Backup import was pathologically slow for databases with large log
tables. Measured on a database with 26k log rows (Windows, debug build):

- local file import (import_sql_string): 28.0s -> 0.39s (~71x)
- WebDAV/S3 sync import (import_sql_string_for_sync): 118.9s -> 0.33s
  (~360x)
- export: 367ms/15.4MB -> 193ms/3.9MB (column list no longer repeated
  per row)

Root causes, isolated with a phase-level benchmark (kept in the test
suite as ignored perf harnesses):

1. dump_sql emitted one INSERT statement per row. The restore side
   parses/prepares/finalizes every statement individually, so import
   cost scaled with statement count, not data volume. A control run on
   an in-memory database was equally slow, proving this is CPU-bound
   parsing, not disk I/O; journal/synchronous pragmas made no
   difference. dump_sql now emits multi-row VALUES batches of 200 rows
   (~200x fewer statements; SQLite has supported multi-row VALUES since
   3.7.11, and both old and new formats import through the same generic
   execute_batch, so dumps remain cross-version compatible).

2. restore_tables re-inserted preserved local tables (proxy logs,
   stream check logs, usage rollups) with one implicit autocommit
   transaction per row against the disk-backed staging database - one
   fsync per row, ~4.6ms each on Windows. All restores now run in a
   single transaction with the INSERT prepared once per table, which
   also makes a mid-restore failure roll back atomically instead of
   leaving a half-restored table.

Regression tests:
- dump_sql_batches_rows_into_multi_row_inserts pins the batch shape
  (450 rows -> exactly 3 INSERT statements; fails if reverted to
  per-row)
- multi_row_dump_round_trips_special_values round-trips quotes,
  newlines, commas, CJK, emoji, BLOB storage class, and NULL through
  export+import
- existing authorizer / genuine-export / sync-preserve tests still pass

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(backup): preserve SQL dump compatibility

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: YUZHEthefool <2804776511@qq.com>
This commit is contained in:
authored and GitHub committed 2026-08-04 23:04:13 +08:00
1 parent 59a2bd1040
commit 668bbda916
2 files changed
+1035 -102

No files matched your search

File diff suppressed because it is too large. Load diff
+1 -1
View File
@@ -54,7 +54,7 @@ const LEGACY_SCHEMA_SQL: &str = r#"
// v3.8.x(schema v1)的真实表结构快照:用于验证从 v3.8.* 升级到当前版本的迁移链路
// 参考:tag v3.8.3 的 src-tauri/src/database/schema.rs
const V3_8_SCHEMA_V1_SQL: &str = r#"
pub(super) const V3_8_SCHEMA_V1_SQL: &str = r#"
CREATE TABLE providers (
id TEXT NOT NULL,
app_type TEXT NOT NULL,