Re: Trying to figure out how to use multiprocessing with HdfsClient

2018-01-13 Thread Eli
Sure, I'm kinda new to multiprocessing so I thought it was perhaps something I've missed on my own part. Thanks, Eli Sent with [ProtonMail](https://protonmail.com) Secure Email. > Original Message ---- > Subject: Re: Trying to figure out how to use multiprocessing

Re: Trying to figure out how to use multiprocessing with HdfsClient

2018-01-10 Thread Wes McKinney
The HadoopFileSystem appears to not be safe for multiprocessing. I'm opening a bug report here: https://issues.apache.org/jira/browse/ARROW-1986 In general, when things don't work as you expect and seems like it may be a bug, please open a JIRA. Thanks, Wes On Wed, Jan 10, 2018 at 3:42 AM, Eli