jayzhan211 commented on issue #13814: URL: https://github.com/apache/datafusion/issues/13814#issuecomment-2774185206
> I did some profiling. > > RUSTC_BOOTSTRAP=1 cargo rustc -p datafusion-catalog -- -Z self-profile -Z self-profile-events=default,args >  > >  > > culprits are some asynchronous functions(Memtable::{scan, insert_into}, StreamTable::{scan, insert_into} StreamTableFactory::create, StreamWrite::write_all, ViewTable::scan ):- Each of them taking 1-2 sec(5-10% of total time) in analysis phase only. > > there are a couple hundred evaluate_obligation call under each type_check of pattern `{...{some_datafusion_object} as std::marker::Sync/Send> ... }` which means that compiler is trying to check if `some_datafusion_object` fulfils the trait bound of being Send/Sync. > > This does not happen with other asynchronous method eg. `ListingSchemaProvider::refresh`, so maybe we can try to check the difference between them and see what is going wrong with above mentioned functions. ListingSchemaProvider::refresh is async method in the struct while `scan` and `insert_into` are async trait method. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
