mirror of
https://github.com/farion1231/cc-switch.git
synced 2026-10-25 17:22:27 +08:00
perf(backup): batch dump INSERTs and restore preserved tables in one transaction (#6122)
* perf(backup): batch dump INSERTs and restore preserved tables in one transaction Backup import was pathologically slow for databases with large log tables. Measured on a database with 26k log rows (Windows, debug build): - local file import (import_sql_string): 28.0s -> 0.39s (~71x) - WebDAV/S3 sync import (import_sql_string_for_sync): 118.9s -> 0.33s (~360x) - export: 367ms/15.4MB -> 193ms/3.9MB (column list no longer repeated per row) Root causes, isolated with a phase-level benchmark (kept in the test suite as ignored perf harnesses): 1. dump_sql emitted one INSERT statement per row. The restore side parses/prepares/finalizes every statement individually, so import cost scaled with statement count, not data volume. A control run on an in-memory database was equally slow, proving this is CPU-bound parsing, not disk I/O; journal/synchronous pragmas made no difference. dump_sql now emits multi-row VALUES batches of 200 rows (~200x fewer statements; SQLite has supported multi-row VALUES since 3.7.11, and both old and new formats import through the same generic execute_batch, so dumps remain cross-version compatible). 2. restore_tables re-inserted preserved local tables (proxy logs, stream check logs, usage rollups) with one implicit autocommit transaction per row against the disk-backed staging database - one fsync per row, ~4.6ms each on Windows. All restores now run in a single transaction with the INSERT prepared once per table, which also makes a mid-restore failure roll back atomically instead of leaving a half-restored table. Regression tests: - dump_sql_batches_rows_into_multi_row_inserts pins the batch shape (450 rows -> exactly 3 INSERT statements; fails if reverted to per-row) - multi_row_dump_round_trips_special_values round-trips quotes, newlines, commas, CJK, emoji, BLOB storage class, and NULL through export+import - existing authorizer / genuine-export / sync-preserve tests still pass Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(backup): preserve SQL dump compatibility --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: YUZHEthefool <2804776511@qq.com>
This commit is contained in:
2 files changed
+1035
-102
No files matched your search
+1034
-101
File diff suppressed because it is too large.
Load diff
@@ -54,7 +54,7 @@ const LEGACY_SCHEMA_SQL: &str = r#"
|
||||
|
||||
// v3.8.x(schema v1)的真实表结构快照:用于验证从 v3.8.* 升级到当前版本的迁移链路
|
||||
// 参考:tag v3.8.3 的 src-tauri/src/database/schema.rs
|
||||
const V3_8_SCHEMA_V1_SQL: &str = r#"
|
||||
pub(super) const V3_8_SCHEMA_V1_SQL: &str = r#"
|
||||
CREATE TABLE providers (
|
||||
id TEXT NOT NULL,
|
||||
app_type TEXT NOT NULL,
|
||||
|
||||
Reference in new issue
Block a user