XiangpengHao commented on issue #13818: URL: https://github.com/apache/datafusion/issues/13818#issuecomment-2725268714
> Hi [@XiangpengHao](https://github.com/XiangpengHao) and [@waynexia](https://github.com/waynexia) is there a way to make the a website like in [parquet-viewer](https://parquet-viewer.xiangpeng.systems/) without reading the whole file from s3 like you implemented into the memory or are you saving it into filesystem? because as far as I know its not possible to read and write to filesystem in wasm as of yet? Parquet viewer does not read the whole file; it only reads the metadata and data relevant to your query. Try this example: https://parquet-viewer.xiangpeng.systems/?url=https%3A%2F%2Fhuggingface.co%2Fdatasets%2FHuggingFaceFW%2Ffineweb%2Fresolve%2Fmain%2Fdata%2FCC-MAIN-2014-42%2F000_00005.parquet You almost instantly get the query results even if the parquet file is a few GBs large. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
