Duplicated link problem looks still persistent: https://issues.apache.org/jira/browse/SPARK-25881 https://issues.apache.org/jira/browse/SPARK-25880
I suspect if there are two places that runs this script. Not a big deal but people that can fix this are specific. I am leaving another reminder here. Thanks. 2018년 10월 1일 (월) 오후 7:15, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: > Seems fixed but looks it starts to leave duplicated PR links for some > recent JIRAs. Not a big deal but are they being ran in multiple places > maybe? > > For instance, > > https://issues.apache.org/jira/browse/SPARK-25579 > https://issues.apache.org/jira/browse/SPARK-25574 > https://issues.apache.org/jira/browse/SPARK-25564 > > > 2018년 9월 17일 (월) 오후 10:09, Ilan Filonenko <ilan....@gmail.com>님이 작성: > >> Same over here: >> >> https://issues.apache.org/jira/browse/SPARK-25291 / >> https://github.com/apache/spark/pull/22415 >> >> On Sun, Sep 16, 2018 at 10:09 PM Hyukjin Kwon <gurwls...@gmail.com> >> wrote: >> >>> Seems same thing is happening again. >>> >>> For instance, >>> - https://issues.apache.org/jira/browse/SPARK-25440 / >>> https://github.com/apache/spark/pull/22429 >>> - https://issues.apache.org/jira/browse/SPARK-25429 / >>> https://github.com/apache/spark/pull/22420 >>> >>> >>> 2017년 8월 3일 (목) 오전 9:06, Hyukjin Kwon <gurwls...@gmail.com>님이 작성: >>> >>>> I think it works for anyone who can leave a web link and comment. >>>> >>>> >>>> For "in progress" Resolution, it looks I am unable to manually set. >>>> Please let me know if anyone knows. >>>> >>>> >>>> For single JIRA, I manually modified the script to process single item >>>> before. I guess you know what the script does but want to rather test, >>>> right? >>>> >>>> This looks requiring Python JIRA client, which I installed by pip >>>> install jira==1.0.7 and I personally tested (and just double checked this) >>>> after commenting out JIRA related codes: >>>> >>>> >>>> diff --git a/dev/github_jira_sync.py b/dev/github_jira_sync.py >>>> index acc9aeabbb9..c8adf0c4ade 100755 >>>> --- a/dev/github_jira_sync.py >>>> +++ b/dev/github_jira_sync.py >>>> @@ -101,8 +101,8 @@ def get_max_pr(): >>>> return 0 >>>> >>>> >>>> -jira_client = jira.client.JIRA({'server': JIRA_API_BASE}, >>>> - basic_auth=(JIRA_USERNAME, >>>> JIRA_PASSWORD)) >>>> +# jira_client = jira.client.JIRA({'server': JIRA_API_BASE}, >>>> +# basic_auth=(JIRA_USERNAME, >>>> JIRA_PASSWORD)) >>>> >>>> jira_prs = get_jira_prs() >>>> >>>> @@ -124,14 +124,15 @@ for issue, pr in sorted(jira_prs, key=lambda kv: >>>> int(kv[1]['number'])): >>>> url = pr['html_url'] >>>> title = "[Github] Pull Request #%s (%s)" % (pr['number'], >>>> pr['user']['login']) >>>> try: >>>> - existing_links = map(lambda l: l.raw['object']['url'], >>>> jira_client.remote_links(issue)) >>>> + # existing_links = map(lambda l: l.raw['object']['url'], >>>> jira_client.remote_links(issue)) >>>> + pass >>>> except: >>>> print("Failure reading JIRA %s (does it exist?)" % issue) >>>> print(sys.exc_info()[0]) >>>> continue >>>> >>>> - if url in existing_links: >>>> - continue >>>> + # if url in existing_links: >>>> + # continue >>>> >>>> icon = {"title": "Pull request #%s" % pr['number'], >>>> "url16x16": "https://assets-cdn.github.com/favicon.ico"} >>>> @@ -139,12 +140,12 @@ for issue, pr in sorted(jira_prs, key=lambda kv: >>>> int(kv[1]['number'])): >>>> # For all possible fields see: >>>> # >>>> https://developer.atlassian.com/display/JIRADEV/Fields+in+Remote+Issue+Links >>>> # application = {"name": "Github pull requests", "type": >>>> "org.apache.spark.jira.github"} >>>> - jira_client.add_remote_link(issue, destination) >>>> + # jira_client.add_remote_link(issue, destination) >>>> >>>> comment = "User '%s' has created a pull request for this issue:" % >>>> pr['user']['login'] >>>> comment += "\n%s" % pr['html_url'] >>>> - if pr_num >= MIN_COMMENT_PR: >>>> - jira_client.add_comment(issue, comment) >>>> + # if pr_num >= MIN_COMMENT_PR: >>>> + # jira_client.add_comment(issue, comment) >>>> >>>> print("Added link %s <-> PR #%s" % (issue, pr['number'])) >>>> num_updates += 1 >>>> >>>> >>>> and >>>> >>>> >>>> rm .github-jira-max >>>> ./dev/github_jira_sync.py >>>> >>>> >>>> >>>> 2017-08-03 5:20 GMT+09:00 Josh Rosen <joshro...@databricks.com>: >>>> >>>>> Usually the backend of https://spark-prs.appspot.com does the linking >>>>> while processing PR update tasks. It appears that the site's connections >>>>> to >>>>> JIRA have started failing: >>>>> >>>>> ConnectionError: ('Connection aborted.', HTTPException('Deadline >>>>> exceeded while waiting for HTTP response from URL: >>>>> https://issues.apache.org/jira/rest/api/2/serverInfo',)) >>>>> >>>>> From Stackdriver's log-based metrics, I can spot that this problem >>>>> started around July 24th. We're already using a much-higher-than-default >>>>> URL fetch timeout, so it's possible that the problem is related to access >>>>> credentials, IP blocks, outdated client libraries, or something else. >>>>> >>>>> On Wed, Aug 2, 2017 at 1:10 PM Bryan Cutler <cutl...@gmail.com> wrote: >>>>> >>>>>> Thanks Hyukjin! I didn't see your previous message.. It looks like >>>>>> your manual run worked pretty well for the JIRAs I'm following, the only >>>>>> thing is that it didn't mark them as "in progress", but that's not a big >>>>>> deal. Otherwise that helps until we can find out why it's not doing this >>>>>> automatically. I'm not familiar with that script, can anyone run it to >>>>>> apply to a single JIRA they are working on? >>>>>> >>>>>> On Wed, Aug 2, 2017 at 12:09 PM, Hyukjin Kwon <gurwls...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> I was wondering about this too.. >>>>>>> >>>>>>> >>>>>>> Yes, actually, I have been manually adding some links by resembling >>>>>>> the same steps in the script before. >>>>>>> >>>>>>> I was thinking it'd rather be nicer to run this manually once and >>>>>>> then I ran this against single JIRA >>>>>>> >>>>>>> first - https://issues.apache.org/jira/browse/SPARK-21526 to show >>>>>>> how it looks like and check if there >>>>>>> >>>>>>> is any issue or objection just in case. >>>>>>> >>>>>>> >>>>>>> Will run this manually now once. I will revert all my action >>>>>>> manually if there is any issue by doing this. >>>>>>> >>>>>>> >>>>>>> 2017-08-03 3:50 GMT+09:00 Sean Owen <so...@cloudera.com>: >>>>>>> >>>>>>>> Hyukjin mentioned this here earlier today and had run it manually, >>>>>>>> but yeah I'm not sure where it normally runs or why it hasn't. Shane >>>>>>>> not >>>>>>>> sure if you're the person to ask? >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Aug 2, 2017 at 7:47 PM Bryan Cutler <cutl...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Devs, >>>>>>>>> >>>>>>>>> I've noticed a couple PRs recently have not been automatically >>>>>>>>> linked to the related JIRAs. This was one of mine (I linked it >>>>>>>>> manually) >>>>>>>>> https://issues.apache.org/jira/browse/SPARK-21583, but I've seen >>>>>>>>> it happen elsewhere. I think this is the script that does it, but it >>>>>>>>> hasn't been changed recently >>>>>>>>> https://github.com/apache/spark/blob/master/dev/github_jira_sync.py. >>>>>>>>> Anyone else seen this or know what's going on? >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Bryan >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>
