This issue is still persistent. https://issues.apache.org/jira/browse/SPARK-26132 https://issues.apache.org/jira/browse/SPARK-26129 https://issues.apache.org/jira/browse/SPARK-26127 https://issues.apache.org/jira/browse/SPARK-26109 https://issues.apache.org/jira/browse/SPARK-26106 https://issues.apache.org/jira/browse/SPARK-26104
the links are still duplicated. Looks the scripts are being ran at multiple places. 2018년 10월 30일 (화) 오후 4:58, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: > Duplicated link problem looks still persistent: > > https://issues.apache.org/jira/browse/SPARK-25881 > https://issues.apache.org/jira/browse/SPARK-25880 > > I suspect if there are two places that runs this script. Not a big deal > but people that can fix this are specific. > I am leaving another reminder here. > > Thanks. > > > 2018년 10월 1일 (월) 오후 7:15, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: > >> Seems fixed but looks it starts to leave duplicated PR links for some >> recent JIRAs. Not a big deal but are they being ran in multiple places >> maybe? >> >> For instance, >> >> https://issues.apache.org/jira/browse/SPARK-25579 >> https://issues.apache.org/jira/browse/SPARK-25574 >> https://issues.apache.org/jira/browse/SPARK-25564 >> >> >> 2018년 9월 17일 (월) 오후 10:09, Ilan Filonenko <ilan....@gmail.com>님이 작성: >> >>> Same over here: >>> >>> https://issues.apache.org/jira/browse/SPARK-25291 / >>> https://github.com/apache/spark/pull/22415 >>> >>> On Sun, Sep 16, 2018 at 10:09 PM Hyukjin Kwon <gurwls...@gmail.com> >>> wrote: >>> >>>> Seems same thing is happening again. >>>> >>>> For instance, >>>> - https://issues.apache.org/jira/browse/SPARK-25440 / >>>> https://github.com/apache/spark/pull/22429 >>>> - https://issues.apache.org/jira/browse/SPARK-25429 / >>>> https://github.com/apache/spark/pull/22420 >>>> >>>> >>>> 2017년 8월 3일 (목) 오전 9:06, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: >>>> >>>>> I think it works for anyone who can leave a web link and comment. >>>>> >>>>> >>>>> For "in progress" Resolution, it looks I am unable to manually set. >>>>> Please let me know if anyone knows. >>>>> >>>>> >>>>> For single JIRA, I manually modified the script to process single item >>>>> before. I guess you know what the script does but want to rather test, >>>>> right? >>>>> >>>>> This looks requiring Python JIRA client, which I installed by pip >>>>> install jira==1.0.7 and I personally tested (and just double checked this) >>>>> after commenting out JIRA related codes: >>>>> >>>>> >>>>> diff --git a/dev/github_jira_sync.py b/dev/github_jira_sync.py >>>>> index acc9aeabbb9..c8adf0c4ade 100755 >>>>> --- a/dev/github_jira_sync.py >>>>> +++ b/dev/github_jira_sync.py >>>>> @@ -101,8 +101,8 @@ def get_max_pr(): >>>>> return 0 >>>>> >>>>> >>>>> -jira_client = jira.client.JIRA({'server': JIRA_API_BASE}, >>>>> - basic_auth=(JIRA_USERNAME, >>>>> JIRA_PASSWORD)) >>>>> +# jira_client = jira.client.JIRA({'server': JIRA_API_BASE}, >>>>> +# basic_auth=(JIRA_USERNAME, >>>>> JIRA_PASSWORD)) >>>>> >>>>> jira_prs = get_jira_prs() >>>>> >>>>> @@ -124,14 +124,15 @@ for issue, pr in sorted(jira_prs, key=lambda kv: >>>>> int(kv[1]['number'])): >>>>> url = pr['html_url'] >>>>> title = "[Github] Pull Request #%s (%s)" % (pr['number'], >>>>> pr['user']['login']) >>>>> try: >>>>> - existing_links = map(lambda l: l.raw['object']['url'], >>>>> jira_client.remote_links(issue)) >>>>> + # existing_links = map(lambda l: l.raw['object']['url'], >>>>> jira_client.remote_links(issue)) >>>>> + pass >>>>> except: >>>>> print("Failure reading JIRA %s (does it exist?)" % issue) >>>>> print(sys.exc_info()[0]) >>>>> continue >>>>> >>>>> - if url in existing_links: >>>>> - continue >>>>> + # if url in existing_links: >>>>> + # continue >>>>> >>>>> icon = {"title": "Pull request #%s" % pr['number'], >>>>> "url16x16": "https://assets-cdn.github.com/favicon.ico"} >>>>> @@ -139,12 +140,12 @@ for issue, pr in sorted(jira_prs, key=lambda kv: >>>>> int(kv[1]['number'])): >>>>> # For all possible fields see: >>>>> # >>>>> https://developer.atlassian.com/display/JIRADEV/Fields+in+Remote+Issue+Links >>>>> # application = {"name": "Github pull requests", "type": >>>>> "org.apache.spark.jira.github"} >>>>> - jira_client.add_remote_link(issue, destination) >>>>> + # jira_client.add_remote_link(issue, destination) >>>>> >>>>> comment = "User '%s' has created a pull request for this issue:" >>>>> % pr['user']['login'] >>>>> comment += "\n%s" % pr['html_url'] >>>>> - if pr_num >= MIN_COMMENT_PR: >>>>> - jira_client.add_comment(issue, comment) >>>>> + # if pr_num >= MIN_COMMENT_PR: >>>>> + # jira_client.add_comment(issue, comment) >>>>> >>>>> print("Added link %s <-> PR #%s" % (issue, pr['number'])) >>>>> num_updates += 1 >>>>> >>>>> >>>>> and >>>>> >>>>> >>>>> rm .github-jira-max >>>>> ./dev/github_jira_sync.py >>>>> >>>>> >>>>> >>>>> 2017-08-03 5:20 GMT+09:00 Josh Rosen <joshro...@databricks.com>: >>>>> >>>>>> Usually the backend of https://spark-prs.appspot.com does the >>>>>> linking while processing PR update tasks. It appears that the site's >>>>>> connections to JIRA have started failing: >>>>>> >>>>>> ConnectionError: ('Connection aborted.', HTTPException('Deadline >>>>>> exceeded while waiting for HTTP response from URL: >>>>>> https://issues.apache.org/jira/rest/api/2/serverInfo',)) >>>>>> >>>>>> From Stackdriver's log-based metrics, I can spot that this problem >>>>>> started around July 24th. We're already using a much-higher-than-default >>>>>> URL fetch timeout, so it's possible that the problem is related to access >>>>>> credentials, IP blocks, outdated client libraries, or something else. >>>>>> >>>>>> On Wed, Aug 2, 2017 at 1:10 PM Bryan Cutler <cutl...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Thanks Hyukjin! I didn't see your previous message.. It looks like >>>>>>> your manual run worked pretty well for the JIRAs I'm following, the only >>>>>>> thing is that it didn't mark them as "in progress", but that's not a big >>>>>>> deal. Otherwise that helps until we can find out why it's not doing >>>>>>> this >>>>>>> automatically. I'm not familiar with that script, can anyone run it to >>>>>>> apply to a single JIRA they are working on? >>>>>>> >>>>>>> On Wed, Aug 2, 2017 at 12:09 PM, Hyukjin Kwon <gurwls...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I was wondering about this too.. >>>>>>>> >>>>>>>> >>>>>>>> Yes, actually, I have been manually adding some links by resembling >>>>>>>> the same steps in the script before. >>>>>>>> >>>>>>>> I was thinking it'd rather be nicer to run this manually once and >>>>>>>> then I ran this against single JIRA >>>>>>>> >>>>>>>> first - https://issues.apache.org/jira/browse/SPARK-21526 to show >>>>>>>> how it looks like and check if there >>>>>>>> >>>>>>>> is any issue or objection just in case. >>>>>>>> >>>>>>>> >>>>>>>> Will run this manually now once. I will revert all my action >>>>>>>> manually if there is any issue by doing this. >>>>>>>> >>>>>>>> >>>>>>>> 2017-08-03 3:50 GMT+09:00 Sean Owen <so...@cloudera.com>: >>>>>>>> >>>>>>>>> Hyukjin mentioned this here earlier today and had run it manually, >>>>>>>>> but yeah I'm not sure where it normally runs or why it hasn't. Shane >>>>>>>>> not >>>>>>>>> sure if you're the person to ask? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <cutl...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Devs, >>>>>>>>>> >>>>>>>>>> I've noticed a couple PRs recently have not been automatically >>>>>>>>>> linked to the related JIRAs. This was one of mine (I linked it >>>>>>>>>> manually) >>>>>>>>>> https://issues.apache.org/jira/browse/SPARK-21583, but I've seen >>>>>>>>>> it happen elsewhere. I think this is the script that does it, but it >>>>>>>>>> hasn't been changed recently >>>>>>>>>> https://github.com/apache/spark/blob/master/dev/github_jira_sync.py. >>>>>>>>>> Anyone else seen this or know what's going on? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Bryan >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>
