Re: [PSA] Please read: PR builder now runs test and build in your forked repository

Kent Yao Wed, 14 Apr 2021 21:15:04 -0700

Thanks Hyukjin and Yikun,

> 2. New Forks have to turn on GitHub action by the fork owner manually

And we may still need a suitable place to make this note clearer to new contributors or someone delete and re-fork their forked repo.

Thanks

Kent Yao

@ Data Science Center, Hangzhou Research Institute, NetEase Corp.

a spark enthusiast

kyuubiis a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark.

spark-authorizerA Spark SQL extension which provides SQL Standard Authorization for Apache Spark.
spark-postgres A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
itatchiA library that brings useful functions from various modern database management systems to Apache Spark.

On 04/15/2021 12:09，Hyukjin Kwon<gurwls...@gmail.com> wrote：

The issue is fixed now. Please keep monitoring this. Thank you all! The spark community is super active and cooperative!

2021년 4월 15일 (목) 오전 11:01, Hyukjin Kwon <gurwls...@gmail.com>님이 작성:
The fix will be straightforward. We can either, in Github Actions workflow,:
- remove fast forward option and see if ti works
- or git rebase before merge the branch

2021년 4월 15일 (목) 오전 11:00, Hyukjin Kwon <gurwls...@gmail.com>님이 작성:
I think it works mostly correctly as Dongjoon investigated and shared (Thanks a lot!).
One problem seems to be syncing to the master seems too strict (https://github.com/apache/spark/pull/32168#issuecomment-819736508). Thanks Yikun.
I think we should make it less strict. I can create a PR right away but would like to encourage Yikun or Kent to do it in order to keep the credits of their investigation.

2021년 4월 15일 (목) 오전 7:21, Dongjoon Hyun <dongjoon.h...@gmail.com>님이 작성:
Hi, Kent.

I checked (1) in your PR, but those test result comments look correct to me.
Please note that both Jenkins and GitHub Action leave the same number of comments on the same GitHash.
Given that, there are not fake comments. It looks like a real result of your commits on that PR.

GitHash: 23248c3
https://github.com/apache/spark/pull/32144#issuecomment-819679970 (GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819647368 (Jenkins)

GitHash: 8dbed7b
https://github.com/apache/spark/pull/32144#issuecomment-819684782 (GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819578976 (Jenkins)

GitHash: a3a6c5e
https://github.com/apache/spark/pull/32144#issuecomment-819690465 (GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819793557 (Jenkins)

GitHash: b6d26b7
https://github.com/apache/spark/pull/32144#issuecomment-819691416 (GitHub Action)
https://github.com/apache/spark/pull/32144#issuecomment-819791485 (Jenkins)

Could you recheck it?

1. Github-actions notification could be wrong when another PR opened
with some same commits, and you will get a lot of fake comments then.
Meanwhile, the new PR get no comments, even if it is actually the
chosen one.
1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970

On Wed, Apr 14, 2021 at 10:41 AM Kent Yao <yaooq...@gmail.com> wrote:
Hi ALL, here is something I notice after this change:

1. Github-actions notification could be wrong when another PR opened
with some same commits, and you will get a lot of fake comments then.
Meanwhile, the new PR get no comments, even if it is actually the
chosen one.
1.1 https://github.com/apache/spark/pull/32144#issuecomment-819679970
2. New Forks have to turn on GitHub action by the fork owner manually
3. `Notify test workflow` keeps waiting when the build flow canceled
or the whole fork gone
4. After refreshed master or even re-forked :(, I still got failures
and seems not alone
4.1. https://github.com/apache/spark/pull/32168 (PR after sync)
4.2. https://github.com/apache/spark/pull/32172 (PR after re-forked)
4.3. https://github.com/attilapiros/spark/runs/2344911058?check_suite_focus=true
(some other failures noticed)

Bests,

Kent

Dongjoon Hyun <dongjoon.h...@gmail.com> 于2021年4月14日周三下午11:34写道：
>
> Thank you again, Hyukjin.
>
> Bests,
> Dongjoon.
>
> On Wed, Apr 14, 2021 at 5:25 AM Kent Yao <yaooq...@gmail.com> wrote:
>>
>> Cool, thanks!
>>
>> Hyukjin Kwon <gurwls...@gmail.com> 于2021年4月14日周三下午8:19写道：
>>>
>>> Good point! I had to clarify.
>>> Once is enough. The sync is needed for your branch to include the changes of https://github.com/apache/spark/pull/32092.
>>>
>>>
>>> 2021년 4월 14일 (수) 오후 9:11, Kent Yao <yaooq...@gmail.com>님이 작성:
>>>>
>>>> Hi Hyukjin,
>>>>
>>>> > Please sync your branch to the latest master branch in Apache Spark in order for the main repository to run the workflow and detect it.
>>>>
>>>> Do we need to sync master for every PR or just one-time cost to keep up with the current master branch？
>>>>
>>>> Kent Yao
>>>> @ Data Science Center, Hangzhou Research Institute, NetEase Corp.
>>>> a spark enthusiast
>>>> kyuubiis a unified multi-tenant JDBC interface for large-scale data processing and analytics, built on top of Apache Spark.
>>>>
>>>> spark-authorizerA Spark SQL extension which provides SQL Standard Authorization for Apache Spark.
>>>> spark-postgres A library for reading data from and transferring data to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
>>>> spark-func-extrasA library that brings excellent and useful functions from various modern database management systems to Apache Spark.
>>>>
>>>>
>>>>
>>>>
>>>> On 04/14/2021 15:41，Kent Yao<y...@apache.org> wrote：
>>>>
>>>> Cool~Thanks, Hyukjin
>>>>
>>>> Yuanjian Li <xyliyuanj...@gmail.com> 于2021年4月14日周三下午3:39写道：
>>>>>
>>>>> Awesome! Thanks for making this happen, Hyukjin!
>>>>>
>>>>> Yi Wu <yi...@databricks.com> 于2021年4月14日周三下午2:51写道：
>>>>>>
>>>>>> Thanks for the great work, Hyukjin!
>>>>>>
>>>>>> On Wed, Apr 14, 2021 at 1:00 PM Gengliang Wang <ltn...@gmail.com> wrote:
>>>>>>>
>>>>>>> Thanks for the amazing work, Hyukjin!
>>>>>>> I created a PR for trial and it looks well so far: https://github.com/apache/spark/pull/32158
>>>>>>>
>>>>>>> On Wed, Apr 14, 2021 at 12:47 PM Hyukjin Kwon <gurwls...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run the GitHub Actions
>>>>>>>> workflows in your forked repository.
>>>>>>>>
>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>
>>>>>>>> You create a PR and your repository triggers the workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>> Apache Spark repository finds your workflow, and links it in a comment in your PR
>>>>>>>>
>>>>>>>> Please let me know if you guys find any weird behaviour related to this.
>>>>>>>>
>>>>>>>>
>>>>>>>> What does that mean to contributors?
>>>>>>>>
>>>>>>>> Please sync your branch to the latest master branch in Apache Spark in order for your forked repository to run the workflow, and
>>>>>>>> for the main repository to detect the workflow.
>>>>>>>>
>>>>>>>>
>>>>>>>> What does that mean to committers?
>>>>>>>>
>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds are running (in contributor's forked repository).
>>>>>>>> Please check the build notified by github-actions bot before merging it.
>>>>>>>> There would be a followup work to reflect the status of the forked repository's build to the status of PR.
>>>>>>>>
>>>>>>>> 2021년 4월 14일 (수) 오후 1:42, Hyukjin Kwon <gurwls...@gmail.com>님이 작성:
>>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> After https://github.com/apache/spark/pull/32092 merged, now we run the GitHub Actions
>>>>>>>>> workflows in your forked repository.
>>>>>>>>>
>>>>>>>>> In short, please see this example HyukjinKwon#34
>>>>>>>>>
>>>>>>>>> You create a PR and your repository triggers the workflow. Your PR uses the resources allocated to you for testing.
>>>>>>>>> Apache Spark repository finds your workflow, and links it in a comment in your PR
>>>>>>>>>
>>>>>>>>> Please let me know if you guys find any weird behaviour related to this.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What does that mean to contributors?
>>>>>>>>>
>>>>>>>>> Please sync your branch to the latest master branch in Apache Spark in order for the main repository to run the workflow and detect it.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What does that mean to committers?
>>>>>>>>>
>>>>>>>>> Now, GitHub Actions will show a green even when GitHub Actions builds are running (in contributor's forked repository). Please check the build notified by github-actions bot before merging it.
>>>>>>>>> There would be a followup work to reflect the status of the forked repository's build to
>>>>>>>>> the status of PR.
>>>>>>>>>
>>>>>>>>>
>>
>>
>> --
>>
>>

--

Kent Yao
@ Data Science Center, Hangzhou Research Institute, NetEase Corp.
a spark enthusiast
kyuubiis a unified multi-tenant JDBC interface for large-scale data
processing and analytics, built on top of Apache Spark.

spark-authorizerA Spark SQL extension which provides SQL Standard
Authorization for Apache Spark.
spark-postgres A library for reading data from and transferring data
to Postgres / Greenplum with Spark SQL and DataFrames, 10~100x faster.
itatchiA library that brings useful functions from various modern
database management systems to Apache Spark.

Re: [PSA] Please read: PR builder now runs test and build in your forked repository

Reply via email to