hi, Andrew, Welcome any improvement proposal for that. Could you file an issue in jira first to show us your idea and an example query to reproduce the issue you described?
Bests, Takeshi On Wed, May 19, 2021 at 11:38 AM Andrew Melo <andrew.m...@gmail.com> wrote: > Hello, > > When reading a very wide (> 1000 cols) input, WholeStageCodeGen blows > past the 64kB source limit and fails. Looking at the generated code, a > big part of the code is simply the DSv2 convention that the codegen'd > variable names are the same as the columns instead of something more > compact like 'c1', 'c2', etc.. > > Would there be any interest in accepting a patch that shortens these > variable names to try and stay under the limit? > > Thanks > Andrew > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > > -- --- Takeshi Yamamuro
