Re: ./dev/run-tests failing at master

2020-05-14 Thread suddhu
I did this and it seems to work now. sudo update-alternatives --install /usr/bin/python python /usr/bin/python3 10 even the cpython version got upgraded. Sorry if you feel I was asking too many rudimentary questions, setting up spark has been rough without a mentor. Thanks :-D FYI, I'm working

Re: ./dev/run-tests failing at master

2020-05-14 Thread suddhu
Thanks for the response!! Here is my configuration: flake8 --version 3.8.1 (mccabe: 0.6.1, pycodestyle: 2.6.0, pyflakes: 2.2.0) CPython 2.7.16 on Linux It seems I need to upgrade my CPython... -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

Re: [DatasourceV2] Allowing Partial Writes to DSV2 Tables

2020-05-14 Thread Ryan Blue
ACCEPT_ANY_SCHEMA isn't a good way to solve the problem because you often want at least some checking in Spark to validate the rows match. It's a good way to be unblocked, but not a long-term solution. On Thu, May 14, 2020 at 4:57 AM Russell Spitzer wrote: > Yeah! That is working for me. Thanks!

Re: ./dev/run-tests failing at master

2020-05-14 Thread shane knapp ☠
this is the flake8 versioning from a jenkins worker: $ flake8 --version 3.6.0 (mccabe: 0.6.1, pycodestyle: 2.4.0, pyflakes: 2.0.0) CPython 3.6.8 on Linux be sure you've got all the right versions of packages in there. On Thu, May 14, 2020 at 12:19 PM suddhu wrote: > Thanks for the response Jeff

Re: ./dev/run-tests failing at master

2020-05-14 Thread suddhu
Thanks for the response Jeff and Sean. It has been quite frustrating setting up the dev environment without any help. Its comforting to have some help finally. I've added "alias python=python3" in my bashrc. So the default python accessed is 3.7.2 I have flake8 installed in both python2 and pyt

Re: ./dev/run-tests failing at master

2020-05-14 Thread suddhu
Thanks for the response Jeff and Sean. It has been quite frustrating setting up the dev environment without any help. Its comforting to have some help finally. I've added "alias python=python3" in my bashrc. So the default python accessed is 3.7.2 I have flake8 installed in both python2 and pyt

Re: ./dev/run-tests failing at master

2020-05-14 Thread Jeff Evans
Are you positive you set up your Python environment correctly? To me, those error messages look like you are running Python 2, but it should be Python 3. On Thu, May 14, 2020 at 1:34 PM Sudharshann D wrote: > Hello! ;) > > I'm new to spark development and have been trying to set up my dev > env

Re: ./dev/run-tests failing at master

2020-05-14 Thread Sean Owen
Hm, flake8 works OK for me locally in master, and on Jenkins it seems. Could be a version issue? On Thu, May 14, 2020 at 1:34 PM Sudharshann D wrote: > Hello! ;) > > I'm new to spark development and have been trying to set up my dev > environment for hours without much success. :-( > > Firstly,

./dev/run-tests failing at master

2020-05-14 Thread Sudharshann D
Hello! ;) I'm new to spark development and have been trying to set up my dev environment for hours without much success. :-( Firstly, I'm wondering why my ./dev/run-tests fails even though in on the master branch. This is the error: flake8 checks failed: ./examples/src/main/python/sql/arrow.py

Re: [build system] jenkins rebooting now

2020-05-14 Thread Hyukjin Kwon
Thanks Shane. On Fri, 15 May 2020, 02:29 Dongjoon Hyun, wrote: > Thank you so much, Shane! > > > On Thu, May 14, 2020 at 9:51 AM Xiao Li wrote: > >> Thank you, Shane! >> >> On Thu, May 14, 2020 at 9:50 AM shane knapp ☠ >> wrote: >> >>> we're back. doesn't seem to have fixed the issue of the w

Re: [build system] jenkins rebooting now

2020-05-14 Thread Dongjoon Hyun
Thank you so much, Shane! On Thu, May 14, 2020 at 9:51 AM Xiao Li wrote: > Thank you, Shane! > > On Thu, May 14, 2020 at 9:50 AM shane knapp ☠ wrote: > >> we're back. doesn't seem to have fixed the issue of the workers >> connecting to repository.apache.org but i'm still investigating. >> >>

Re: [build system] jenkins rebooting now

2020-05-14 Thread Xiao Li
Thank you, Shane! On Thu, May 14, 2020 at 9:50 AM shane knapp ☠ wrote: > we're back. doesn't seem to have fixed the issue of the workers > connecting to repository.apache.org but i'm still investigating. > > On Thu, May 14, 2020 at 9:11 AM shane knapp ☠ wrote: > >> that is all. >> >> -- >> Sha

Re: [build system] jenkins rebooting now

2020-05-14 Thread shane knapp ☠
we're back. doesn't seem to have fixed the issue of the workers connecting to repository.apache.org but i'm still investigating. On Thu, May 14, 2020 at 9:11 AM shane knapp ☠ wrote: > that is all. > > -- > Shane Knapp > Computer Guy / Voice of Reason > UC Berkeley EECS Research / RISELab Staff

[build system] jenkins rebooting now

2020-05-14 Thread shane knapp ☠
that is all. -- Shane Knapp Computer Guy / Voice of Reason UC Berkeley EECS Research / RISELab Staff Technical Lead https://rise.cs.berkeley.edu

Re: [DatasourceV2] Allowing Partial Writes to DSV2 Tables

2020-05-14 Thread Russell Spitzer
Yeah! That is working for me. Thanks! On Thu, May 14, 2020 at 12:10 AM Wenchen Fan wrote: > I think we already have this table capacity: ACCEPT_ANY_SCHEMA. Can you > try that? > > On Thu, May 14, 2020 at 6:17 AM Russell Spitzer > wrote: > >> I would really appreciate that, I'm probably going to