gaogaotiantian commented on PR #54015: URL: https://github.com/apache/spark/pull/54015#issuecomment-3813313801
It depends. If the user is using pandas 2, there will be no behavior change even if they upgrade their pyspark. pyspark follows pandas 3 behavior when the local pandas version is 3. We should probably have a general announcement somewhere about this. But documenting every single detail about pandas 3 vs pandas 2 should not be our job. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
