Nagi 0.1 performance and verification report

This version is a prototype language with a two-layer backend, a Rust compiler, and a working runtime. High generates editable Low text, which is parsed and checked again before compilation to native code through Rust. Working paths include HTTP body → typed class → SQLite → class → JSON CRUD, ordinary Low calls and replacements, async/scopes, actor communication, and worker restarts.

A general-purpose language, a production custom runtime, and BEAM-level fault isolation remain incomplete. Actors, supervisors, and queues are standard test functions calling the real runtime; general High declarations are not implemented. The numbers below come from logs in the stated environment. Public example names were anonymized while preserving their original ASCII byte lengths. Measurement values were not changed.

This is the English version of the September 30, 2026 report. Its test counts and implementation status describe that measurement snapshot. The Japanese source is available on GitHub.

Environment and reproduction conditions

CPU/microbenchmarks ran individually on CPU 0, without overlapping server load or builds. Rust used five warmups, a pilot to set loop counts, and the median of seven repetitions. Python used five warmups and 7×25 runs; Node used 50 JIT warmups and 7×100 runs. All used the same prepared 100,000 elements, excluding array creation from timing. Integer input was (i*17+13)%997-498; float input divided that value by seven. Node Number represents these inputs/results exactly but does not share the full i64 range semantics.

Frequency, shared-host contention, and cache state were not fully controlled. These are exploratory measurements in one environment. A few percent difference between runs does not establish language superiority. Raw timings are retained rather than selecting only favorable results.

Allocation counts cover the calling thread's Rust GlobalAlloc requests, excluding SQLite's C allocator and jobs/tasks allocated on other threads. Reallocations also count as allocations. Requested bytes are cumulative, not live/peak memory. Returned values are dropped after counts are read, so their deallocations are absent. The counter is disabled during timing, although its TLS branch remains. Static cost reports identify locations rather than dynamic allocation counts.

CPU: matching inputs and results

Whole-kernel times are µs/op. ns/item divides by 100,000 elements and is an amortized value, not the isolated latency of one addition.

Checksums matched across all four implementations, with float tolerance 1e-7: {"integer_sum": -3184, "map_reduce": 90448, "filter_reduce": 12461527, "branch": 2106, "float_sum": -454.85714285709753, "loop": 299995}. Nagi/Rust kernels all had zero Rust allocations. Python/Node allocations were not measured and cannot be compared with that zero.

CPython's built-in sum was a separate reference at 686.172 µs/op. The table uses explicit Python loops, not native NumPy kernels. Repeated input warms caches, so the results also do not measure DRAM bandwidth or huge datasets.

Nagi uses native integers and contiguous arrays without dynamic numeric boxing or type checks inside loops. Generated and handwritten Rust integer-sum disassemblies both contain four packed i64 paddq additions (integer-sum.asm/rust-integer-sum.asm). LLVM vectorization and unrolling reduce ns/item. Differences from handwritten Rust include generated code, layout, and run variance; they do not show an independent Nagi backend outperforming Rust.

JSON, views, and strings

The JSON tests use the same small input: id=42, name=alice, age=18. ns/op covers the whole parsing/encoding operation; allocations are measured dynamically for one operation.

Direct decoding into User was 2.29× faster than going through an intermediate Value tree in this run. Allocations fell from 5 to 1 and requested bytes from 646 to 5. Deserialization goes straight to a Rust struct without a dictionary/map/general JSON tree. The remaining allocation is the owned String field.

Borrowed classes are handwritten Rust experiments using &str into the input. Despite zero allocations, they were not faster than owned classes in this run. Escaped JSON strings or retention beyond the input lifetime need owned data. High borrowed class fields are not implemented.

Encoding creates one Vec with capacity 128 bytes. Buffer reuse measured zero allocations but is not generally applied to HTTP. Responses follow native class → Vec<u8> → Body, without fully streaming serialization to sockets. Parse-modify-encode allocates a String and an output Vec.

The view test only creates a range reference into a 1 MiB buffer. It confirms pointer equality with input+1 and zero allocations. Copy actually duplicates 1,048,575 bytes. These do different work, so their time ratio is not a speedup for an equivalent operation. Borrowing HTTP bodies avoids an additional copy; it does not make the entire network/kernel/input-buffer path zero-copy.

SQLite to classes: conversion and ownership

An in-memory database is preloaded with 10,000 rows. Cached SELECT retrieves 1/100/1,000/10,000 rows with id i64, name TEXT "alice", and age i32. Timing includes SQL execution, stepping, class construction, and Vec storage, excluding connection creation and seed inserts.

Resolving column names once at query start was 1.55× faster at 10,000 rows than resolving them per row. FromRow reads native fields directly without tuple/dictionary/ORM intermediates. TEXT outlives SQLite stepping and therefore needs an owned String per row; whole-row conversion is not allocation-free.

The difference from id-only scanning combines TEXT reads, String allocation, three-field conversion, and Vec storage. It does not isolate pure conversion time. Reserve reduces requested bytes and reallocations, but increased time in this 10,000-row run. No extra COUNT or guessed reserve was added to every query.

Inserting 1,000 rows in a transaction and rolling back took 488.667 µs/op; one insert/rollback took 3.463 µs/op. Zero Rust allocations do not imply zero SQLite C allocation, page processing, or disk I/O. This used memory storage and did not measure persistence/fsync.

HTTP database jobs go through a dedicated thread, bounded channel 64, and oneshot replies. Their roundtrip, job boxing, and scheduling are excluded from the synchronous microbenchmark but included in HTTP db_single_row below. Typed column checks happen at runtime; compile-time schema/SQL validation is absent.

Changes after measurement

Column indices previously used a Vec for every query. Up to 16 columns now fit inline, with overflow using a Vec. Existing reordered/missing-column tests passed before rerunning the same benchmark.

The established effect is one fewer allocation and 32 fewer bytes per query. The 10,000-row timing difference is small and is not treated as statistically established acceleration. Initial logs are micro-before.jsonl; later logs are micro-after.jsonl.

Actors gained pipeline sending with a maximum of 32 messages. Individual messages still pass through the same mailbox, and all oneshot replies are received to verify 20,000. This does not batch everything into one +20,000 operation. JSON buffer reuse and DB Vec reserve remain experiments, outside the standard HTTP/DB paths.

HTTP: matching successful payloads

All servers use CPU 0 and one executor/event-loop worker. Wrk uses CPUs 7/8, two threads, and 64 keep-alive connections. Each case has a one-second warmup followed by three seconds of measurement, with three trials per framework/case. Framework order is reshuffled per trial with a fixed seed. Nagi/Axum share timeout/body-limit middleware. Only Nagi has an additional persistent SQLite thread. The middleware was not fully ported to Node/aiohttp, so the comparison does not guarantee identical internal work across frameworks.

Nagi/Axum use the same native struct. Node uses JSON.parse → field checks → JSON.stringify; aiohttp uses json.loads → slots dataclass → type checks → json.dumps. Successful statuses and bodies match. Nagi does not skip schema checks. small_json does not resend fixed JSON bytes.

req/s and p50/p95/p99 are medians of the three individual trials; max is the largest maximum. Percentiles do not come from merged histograms. Latencies are µs; errors total socket/status/timeout failures.

These are closed-loop wrk results, not fixed-arrival-rate open-loop or overload tail-latency tests. Handwritten Axum using the same runtime is the main native reference. Generated wrappers, handlers, bodies, allocations, and SQLite workers should be assessed separately from CPU kernels.

For small POSTs, Nagi achieved about 39.9k req/s versus Axum's 60.8k, lower in this run. Large POSTs were both around 36k. No profile isolates a particular function behind the small-POST gap. Further measurements must separate wrapper and router/state costs. Native compilation alone does not optimize every HTTP path.

The following table shows server CPU usage during measurement, maximum final RSS across three trials, context switches/s summed over server threads, and maximum thread/FD counts. One logical CPU is 100%; CPU usage divides /proc CPU tick differences by wall time. Load-generator CPU/RSS are excluded.

All 54 trials recorded zero socket/status/timeout errors. These successful-response load tests do not assess production SLOs with authentication, TLS, HTTP/2, slow clients/DoS, or large streams.

Continuous load for 120 seconds

Nagi /echo ran for 120 seconds with a 4,096-character name, 128 connections, two wrk threads, and one server worker. It achieved 35,238 req/s, p50 3222 µs, p95 7209 µs, p99 16531 µs, max 176174 µs, and zero socket/status/timeout errors.

One-second RSS samples ranged from 6.05–8.06 MiB, ending at 8.06 MiB under load and 7.16–7.16 MiB in five samples after stopping. FD counts were 10–138 under load and 10–10 afterward.

This checks for observed abnormalities and unreleased FDs within a finite period. It does not prove absence of memory leaks, multi-day stability, allocator fragmentation, or GC pauses. Retained high-water RSS needs further observation separating drops, allocator retention, and fragmentation. Whole-process allocation/free tracking and heap profiling were not performed.

Concurrent tasks and TCP connections

The task test uses four Tokio workers. It creates 1,000/10,000/50,000/100,000 tasks and confirms each reaches its first poll using an atomic counter. No task can complete until the semaphore gate opens. RSS is measured with all tasks waiting, followed by release and full joining. Times are medians of three trials; RSS values are maxima. The harness includes gate/Arc/counter/spawn/join costs, not isolated application-task cost.

The older concurrency-before.jsonl counts total spawned tasks without establishing simultaneous waiting or peak usage. It is not directly comparable to the gated test. Final RSS includes allocator retention, not unfinished task counts. CPU work uses a separate spawn_blocking path with semaphore limit four; forcibly stopping started work is not implemented.

The TCP test holds 1,000/10,000 real simultaneous connections and receives a health response from each. Opening uses concurrency 128; the server has one worker. 50,000/100,000 TCP connections were not tested with the FD limit 16,384 and IPv4 ephemeral ports 32768–60999. A result for 100,000 tasks is not a result for 100,000 TCP connections.

FDs returned to baseline in both trials. RSS remained at its high-water mark immediately after disconnect. Around 250 MiB held 10,000 connections; the approximate 25 KiB/connection difference covers the whole process, including HTTP buffers, runtime, and allocator. Kernel socket memory is outside RSS.

Actors, supervisors, and queues

Five trials use the same CounterActor, mailbox capacity 64, four Tokio workers, and 20,000 messages. Serial RPC waits for each reply before sending another; the pipeline allows up to 32 in flight. µs/message is amortized elapsed/messages, not individual p50/p99 latency.

Pipeline throughput was 24.95× the serial version in this test. It overlaps waits and scheduler roundtrips; individual RPC latency did not necessarily fall by that factor. Mailbox/reply allocations and ownership conversions remain. High shared uses explicit Arc, while internal task/channel/DB-state Arcs also remain.

The supervisor detected three worker panics and restarted three times. Restart latency was 1.951–2.137 ms from detection to first poll, including 1 ms backoff. An independent worker continued 12 times. Crash-loop testing stopped restarts after reaching a limit of two in a one-second window. SIGSEGV/abort/process kill isolation, request replay/loss, and arbitrary supervisor trees were outside the test.

The queue test used 1,000 jobs, eight workers, channel capacity 64, at most three attempts, and 1/2 ms retry backoff. It recorded 909 completions, 91 dead letters, 312 retries, and peak in-flight eight. Failures were injected from IDs. This counts dead letters without persistent DLQ, replay after process restart, or an exactly-once API.

After creating/stopping 100 database workers, live workers stayed 0→0. Unit tests confirm scope error/panic cancels and joins siblings and releases guards. External parent-future drops and body panics only request abort through JoinSet Drop; they cannot await asynchronous cleanup at that point.

Safety and correctness checks

A successful High check alone does not guarantee soundness. Safe Rust is generated and must pass Rust borrow/Send checks before becoming a native binary. Runtime unsafe is limited to observation implementations such as System GlobalAlloc forwarding; arbitrary Low unsafe/FFI is not available. Generated code and runtime dependencies have not had an independent security audit.

Implementation status and assessment

Effective directions included native primitives with contiguous arrays, direct typed JSON, once-per-query column resolution, removal of small unnecessary Vecs, and actor pipelines. Borrowed JSON and reserve results show why allocation reductions must be assessed separately from speed. HTTP includes handlers/runtime/serialization/scheduling and cannot be predicted from CPU-kernel speed ratios.

Next priorities are arithmetic/borrow rules, modules/generics, arbitrary-state actor lowering, general typed database parameters, and measured request arenas/buffer reuse. Prototype development speed does not establish the effort needed to finish the language. Low self-hosting comes after String/Map/module/allocator APIs and bootstrap-equivalence tests.

Rerunning and raw logs

Use the repository README for builds and functional tests. During performance measurements, avoid concurrent builds or other CPU benchmarks and adjust CPU affinity to your environment.

taskset -c 0 ./native-target/release/nagi-cpu > benchmarks/results/cpu-nagi.jsonl
taskset -c 0 ./target/release/examples/microbench > benchmarks/results/micro-after.jsonl
taskset -c 0 python3 benchmarks/python_cpu.py > benchmarks/results/cpu-python.jsonl
taskset -c 0 node benchmarks/node_cpu.js > benchmarks/results/cpu-node.jsonl
./target/release/examples/concurrency_bench > benchmarks/results/concurrency-after.jsonl 2> benchmarks/results/fault-after.log
python3 tests/connections.py > benchmarks/results/connections.json
python3 scripts/http_bench.py --wrk /absolute/path/to/wrk --soak 120
python3 scripts/summarize_results.py

results/ contains CPU/JSON/SQLite/allocation/concurrency JSONL, individual wrk text logs, aggregated HTTP JSON, one-second resource samples, fault stderr, environment data, and checksum verification. The full wrk source is excluded; its commit and Lua scripts are recorded. Linux-specific taskset/proc measurements need replacements on other OSes. CI definitions were included but had not been run remotely when this report was written.

The original snapshot refers to 22 design/specification pages in docs/. Reproduction sources are in compiler/runtime/examples/tests/benchmarks/scripts. Later Docs and compiler additions are described in the current guides.