perf(backup): batch dump INSERTs and restore preserved tables in one transaction (#6122)

Browse Source

* perf(backup): batch dump INSERTs and restore preserved tables in one transaction

Backup import was pathologically slow for databases with large log
tables. Measured on a database with 26k log rows (Windows, debug build):

- local file import (import_sql_string): 28.0s -> 0.39s (~71x)
- WebDAV/S3 sync import (import_sql_string_for_sync): 118.9s -> 0.33s
  (~360x)
- export: 367ms/15.4MB -> 193ms/3.9MB (column list no longer repeated
  per row)

Root causes, isolated with a phase-level benchmark (kept in the test
suite as ignored perf harnesses):

1. dump_sql emitted one INSERT statement per row. The restore side
   parses/prepares/finalizes every statement individually, so import
   cost scaled with statement count, not data volume. A control run on
   an in-memory database was equally slow, proving this is CPU-bound
   parsing, not disk I/O; journal/synchronous pragmas made no
   difference. dump_sql now emits multi-row VALUES batches of 200 rows
   (~200x fewer statements; SQLite has supported multi-row VALUES since
   3.7.11, and both old and new formats import through the same generic
   execute_batch, so dumps remain cross-version compatible).

2. restore_tables re-inserted preserved local tables (proxy logs,
   stream check logs, usage rollups) with one implicit autocommit
   transaction per row against the disk-backed staging database - one
   fsync per row, ~4.6ms each on Windows. All restores now run in a
   single transaction with the INSERT prepared once per table, which
   also makes a mid-restore failure roll back atomically instead of
   leaving a half-restored table.

Regression tests:
- dump_sql_batches_rows_into_multi_row_inserts pins the batch shape
  (450 rows -> exactly 3 INSERT statements; fails if reverted to
  per-row)
- multi_row_dump_round_trips_special_values round-trips quotes,
  newlines, commas, CJK, emoji, BLOB storage class, and NULL through
  export+import
- existing authorizer / genuine-export / sync-preserve tests still pass

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(backup): preserve SQL dump compatibility

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: YUZHEthefool <2804776511@qq.com>

...

This commit is contained in:

3 people authored and GitHub committed 2026-08-04 23:04:13 +08:00

1 parent 59a2bd1040

commit 668bbda916

2 files changed

+1035 -102

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src-tauri/src/database/backup.rs

+1034 -101

File diff suppressed because it is too large. Load diff