Kriskras99 opened a new pull request, #672:
URL: https://github.com/apache/avro-rs/pull/672
This PR is **not** ready for review, see it more as a progress update.
The new parser uses a different JSON library
[`sonic`](github.com/cloudwego/sonic-rs) which allows the parser to iterate
over borrowed strings from the file. This saves a ton of heap allocations.
`sonic` also has SIMD optimisations when `RUSTFLAGS="-C target-cpu=native"` is
used, but on the small files we generally have, the impact of that is small.
TODO:
- [ ] Proper error handling instead of panicking everywhere
- [ ] Including schema path as is done in #670
- [ ] Dedicated parse error that doesn't require the whole
`Error::new(Details::...))` until we're at the edge
- [ ] Recursion and allocation limits
- [ ] Test against all the parser tests
| Parser | Small record | Large record |
|--------------|--------------|--------------|
| Old | 23.5 us | 25.7 ms |
| New | 15.8 us | 16.0 ms |
| Old (native) | 24.0 us | 26.1 ms |
| New (native) | 13.8 us | 15.6 ms |
1. The large record is
[`large_schema.asvc`](https://github.com/apache/avro/blob/main/lang/c%2B%2B/jsonschemas/large_schema.avsc)
2. The small record is
[`RecordWithRequiredFields.avsc`](https://github.com/apache/avro/blob/main/share/test/schemas/RecordWithRequiredFields.avsc)
3. Take the benchmarks with a grain of salt, laptops are quite bad for
benchmarking.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
