Jens-G opened a new pull request, #3867: URL: https://github.com/apache/thrift/pull/3867
JIRA: [THRIFT-6269](https://issues.apache.org/jira/browse/THRIFT-6269) Client: erl This PR covers the first of the two points in THRIFT-6269, `loop_recv` in the socket transport. The second point, `read_all_1` in the JSON protocol reading one byte per transport call, is left to a separate change. The ticket suggests taking the two one at a time. `thrift_socket_transport:read/2` waits in `loop_recv/3` until the requested length has arrived. Each pass ran `iolist_to_binary([Buf, Data])`, copying everything buffered so far together with the new piece, and then took `iolist_size/1` of the result. A read that arrives in n pieces therefore copied O(n²) bytes. ### Change - `loop_recv/3` carries the pieces as an iolist, `[Buf, Data]`, and counts the bytes it has in an integer. - It flattens once, when enough bytes are there, and splits off the requested length as before. - `read/2` passes along the count of what is already buffered. The buffer's type is already `iodata()`, and `read_exact/2` flattens it on entry, so an iolist left there after an error is handled. `thrift_sslsocket_transport` asks `ssl:recv/3` for the exact length and has no such loop. The framed transport only flattens when its buffer is empty. ### Test New `read_in_pieces_test_` in `test_thrift_socket_transport.erl`: - Setup: with `gen_tcp:recv` mocked to hand out 4 KiB per call, it reads 1 MiB and checks the data and the empty remainder. - Measure: the reductions the read took. Copying costs about one reduction per 256 bytes, the same on OTP 25 and 28, so the count follows the bytes copied. - Limit: 65,536 reductions, `Len div 16`. - Unmodified library: 540,931 reductions, so the test fails. - With the change: 17,372 reductions. `rebar3 eunit` passes all 353 tests on OTP 25.3 with rebar3 3.18.0, and on OTP 26, 27 and 28. `rebar3 fmt -c` reports nothing for the changed files. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
