Commit Graph
4 Commits
Author SHA1 Message Date
digua d5f34af445 refactor(parser): standardize format identifiers 2026-08-04 17:29:09 +08:00
digua c0548f63c8 feat(import): prepare native parser rollout 2026-07-05 22:49:38 +08:00
diguaandCursor c265925e3e fix(import): report accurate post-parse progress stages in real time
The UI stayed on "parsing messages" during the whole post-parse phase:
the importer emitted an 'importing' stage the frontend had no text for,
and on CLI Web the synchronous DB writes / index / FTS build starved the
event loop so buffered SSE progress events only arrived after the import
finished.

- Replace 'importing' with 'saving' (message writes) and a new 'indexing'
  stage (indexes / FTS / session index) in ParseProgress, with UI texts
  for all four locales.
- Yield a macrotask between native parser batches and before heavy
  synchronous steps so SSE progress events flush while importing.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-05 22:49:38 +08:00
diguaandCursor 1e7aefe08b feat(import): add Rust native parsing kernels for WeFlow and ChatLab JSON
Add @openchatlab/parser-native (napi-rs v3) with a shared byte-level JSON
scanner and two format kernels. Parsing large exports is 4-10x faster in
real-data benchmarks. The native module is an optional local build; the
pure-TS parsers remain as automatic fallback when it is unavailable,
disabled via CHATLAB_DISABLE_NATIVE_PERF=1, or rejects off-spec files.

The ChatLab kernel also parses the members array structurally, fixing the
TS head-regex heuristic that silently dropped avatars/roles for files with
nested arrays or long member lists.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-05 22:49:38 +08:00