* perf(backup): batch dump INSERTs and restore preserved tables in one transaction
Backup import was pathologically slow for databases with large log
tables. Measured on a database with 26k log rows (Windows, debug build):
- local file import (import_sql_string): 28.0s -> 0.39s (~71x)
- WebDAV/S3 sync import (import_sql_string_for_sync): 118.9s -> 0.33s
(~360x)
- export: 367ms/15.4MB -> 193ms/3.9MB (column list no longer repeated
per row)
Root causes, isolated with a phase-level benchmark (kept in the test
suite as ignored perf harnesses):
1. dump_sql emitted one INSERT statement per row. The restore side
parses/prepares/finalizes every statement individually, so import
cost scaled with statement count, not data volume. A control run on
an in-memory database was equally slow, proving this is CPU-bound
parsing, not disk I/O; journal/synchronous pragmas made no
difference. dump_sql now emits multi-row VALUES batches of 200 rows
(~200x fewer statements; SQLite has supported multi-row VALUES since
3.7.11, and both old and new formats import through the same generic
execute_batch, so dumps remain cross-version compatible).
2. restore_tables re-inserted preserved local tables (proxy logs,
stream check logs, usage rollups) with one implicit autocommit
transaction per row against the disk-backed staging database - one
fsync per row, ~4.6ms each on Windows. All restores now run in a
single transaction with the INSERT prepared once per table, which
also makes a mid-restore failure roll back atomically instead of
leaving a half-restored table.
Regression tests:
- dump_sql_batches_rows_into_multi_row_inserts pins the batch shape
(450 rows -> exactly 3 INSERT statements; fails if reverted to
per-row)
- multi_row_dump_round_trips_special_values round-trips quotes,
newlines, commas, CJK, emoji, BLOB storage class, and NULL through
export+import
- existing authorizer / genuine-export / sync-preserve tests still pass
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(backup): preserve SQL dump compatibility
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: YUZHEthefool <2804776511@qq.com>
`import_sql_string_inner` validated only that the file starts with the
`-- CC Switch SQLite 导出` comment, then handed the whole text to
`execute_batch`. Anything after that prefix ran unchecked, so a crafted
backup could `ATTACH DATABASE '/path/x.db'` and create a SQLite file
anywhere the user can write. The side effect lands before
`validate_basic_state`, so the file survives even when the import as a
whole fails. `settings` is in neither SYNC_SKIP_TABLES nor
SYNC_PRESERVE_TABLES, so the WebDAV/S3 sync path reaches the same code.
Install a SQLite authorizer for the duration of the external batch only,
then clear it so our own schema maintenance is unaffected.
Deny what can leave the temp database rather than allow-listing what
`dump_sql` emits. The batch runs on a throwaway NamedTempFile whose
entire contents are already decided by that same SQL, so DELETE/DROP/
UPDATE hand an attacker nothing new -- the only meaningful boundary is
the temp file itself. A strict allow-list only adds the risk of refusing
a legitimate backup whose schema has a shape we did not anticipate.
The denied set was measured, not guessed: `ATTACH DATABASE 'x'`,
`VACUUM INTO 'x'` and bare `VACUUM` all surface as `AuthAction::Attach`,
so one rule covers all three -- which keyword scanning would not, since
`VACUUM INTO` contains no "ATTACH". Also deny vtable creation
(file-backed modules such as csvfile can read and write arbitrary paths)
and `Unknown`, so future SQLite statements fail closed.
Tests cover both denied statements (asserting no file is left on disk,
not merely that the call errors) and a real export round-trip, which
guards against the allow-list regressing into false refusals.
- Apply rustfmt diffs in claude_desktop_config.rs
- Allow needless_return on current_platform_paths (cfg-mirrored arms)
- Allow too_many_arguments on RequestForwarder::forward
- Replace `let mut + reassign` with struct literals in tests
(settings, backup, provider, response_processor)
- Use Path::new instead of PathBuf::from to fix cmp_owned in misc tests
- Replace 3.14 with 3.5 in config test to avoid approx_constant lint
Split LOCAL_ONLY_TABLES into SYNC_SKIP_TABLES (export) and
SYNC_PRESERVE_TABLES (import). provider_health is skipped on export
but no longer restored on import, since it has a foreign key on
providers(id, app_type) that may not match the remote dataset.
Health data is ephemeral and rebuilds automatically at runtime.
- Add usage_daily_rollups table (schema v6) to aggregate proxy request
logs into daily summaries, reducing query overhead for statistics
- Add rollup_and_prune DAO that aggregates old detail logs (>N days)
into rollup rows and deletes the originals
- Update all usage stats queries to UNION detail logs with rollup data
- Introduce incremental auto-vacuum for SQLite, with startup and
periodic cleanup of old stream_check_logs and request log rollups
- Split backup export/import into full vs sync variants: WebDAV sync
now skips local-only table data (proxy_request_logs,
stream_check_logs, provider_health, proxy_live_backup,
usage_daily_rollups) while preserving them on import
- Add enable_logging guard to skip request log writes when disabled
- Apply cargo fmt formatting fixes across multiple modules
Extract backup & restore into a standalone AccordionItem in Advanced settings.
Add configurable auto-backup interval (disabled/6h/12h/24h/48h/7d) and retention
count (3-50) via settings. Add per-backup rename with inline editing UI.
Four improvements to the database backup mechanism:
1. Auto backup before schema migration - creates a snapshot when
upgrading from an older database version, providing a safety net
beyond the existing SAVEPOINT rollback mechanism.
2. Periodic startup backup - checks on app launch whether the latest
backup is older than 24 hours and creates a new one if needed,
ensuring all users have recent backups regardless of usage patterns.
3. Backfill failure notification - switch now returns SwitchResult with
warnings instead of silently ignoring backfill errors, so users are
informed when their manual config changes may not have been saved.
4. Backup management UI - new BackupListSection in Settings > Data
Management showing all backup snapshots with restore capability,
including a confirmation dialog and automatic safety backup before
restore.
Jason
