As I understand, the performance problem of the read built-in utility originates from its need to read one byte at a time in order not to swallow input it doesn't process.
Would it make sense to add an "exclusive" option (call it "-x" for now) to read, where "read -x" essentially means "I promise to do all processing on this open file with read -x or not to complain if input gets lost"? This may be infeasable due to not being able to guarantee that any "read -x" really uses the built-in version. The point, of course, is to allow the shell to buffer the input.