feat(import): add Rust native parsing kernels for WeFlow and ChatLab JSON

Add @openchatlab/parser-native (napi-rs v3) with a shared byte-level JSON
scanner and two format kernels. Parsing large exports is 4-10x faster in
real-data benchmarks. The native module is an optional local build; the
pure-TS parsers remain as automatic fallback when it is unavailable,
disabled via CHATLAB_DISABLE_NATIVE_PERF=1, or rejects off-spec files.

The ChatLab kernel also parses the members array structurally, fixing the
TS head-regex heuristic that silently dropped avatars/roles for files with
nested arrays or long member lists.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
diguaandCursor committed 2026-07-05 22:49:38 +08:00
1 parent bbf89332b2
commit 1e7aefe08b
29 files changed
+4953 -6

No files matched your search

+4 -2
View File
@@ -19,8 +19,10 @@ export default defineConfig({
define: {
'process.env.APTABASE_APP_KEY': JSON.stringify(process.env.APTABASE_APP_KEY || ''),
},
noExternal: [/^@openchatlab\//, 'chatlab-mcp', 'stream-json'],
external: ['better-sqlite3', '@node-rs/jieba'],
noExternal: [/^@openchatlab\/(?!parser-native)/, 'chatlab-mcp', 'stream-json'],
// parser-native 是本地构建的可选 Rust 内核:不打包也不声明依赖,
// 运行时 require 失败会自动回退 TS 解析器。
external: ['better-sqlite3', '@node-rs/jieba', '@openchatlab/parser-native'],
banner: {
js: [
"import { createRequire as __createRequire } from 'module';",