I've added some code that should help here: https://gem5-review.googlesource.com/c/public/gem5/+/57275
-- Dr. Bobby R. Bruce Room 3050, Kemper Hall, UC Davis Davis, CA, 95616 web: https://www.bobbybruce.net On Tue, Mar 1, 2022 at 10:41 AM Bobby Bruce <bbr...@ucdavis.edu> wrote: > I'll change my answer slightly: It seems like this error is happening from > trying to download the resources.json file (hosted in the resources repo) > too much. I suspect there are similar restrictions here. I think I can fix > it too. I'll probably add a wait-and-retry for the resource downloader > anyway as it'll make the downloader more stable. > > -- > Dr. Bobby R. Bruce > Room 3050, > Kemper Hall, UC Davis > Davis, > CA, 95616 > > web: https://www.bobbybruce.net > > > On Tue, Mar 1, 2022 at 10:28 AM Bobby Bruce <bbr...@ucdavis.edu> wrote: > >> Nice find Gabe. That would certainly explain a lot. Turns out there's a >> 50Gbps quota that will return a 429 error if hit: >> https://cloud.google.com/storage/quotas. I'll see if I can get this >> increased and, either way, I'll implement a wait-and-retry strategy in the >> downloader for when this error is received. I suspect this is hit when lots >> of Kokoro instances are spun up at one time. >> >> -- >> Dr. Bobby R. Bruce >> Room 3050, >> Kemper Hall, UC Davis >> Davis, >> CA, 95616 >> >> web: https://www.bobbybruce.net >> >> >> On Tue, Mar 1, 2022 at 2:08 AM Giacomo Travaglini via gem5-dev < >> gem5-dev@gem5.org> wrote: >> >>> Hi Gabe, >>> >>> >>> >>> A possible workaround could be to use the --bin-path option in testlib >>> >>> >>> >>> Kind Regards >>> >>> >>> >>> Giacomo >>> >>> >>> >>> *From: *Gabe Black via gem5-dev <gem5-dev@gem5.org> >>> *Date: *Tuesday, 1 March 2022 at 04:10 >>> *To: *gem5 Developer List <gem5-dev@gem5.org> >>> *Cc: *Gabe Black <gabe.bl...@gmail.com> >>> *Subject: *[gem5-dev] resource downloader failure >>> >>> Hi folks. I've been trying to run tests locally, and I've been running >>> into occasional flakiness due to a problem with the resource downloader. >>> This reminds me somewhat of flakiness I was seeing in kokoro as well, so >>> they are probably related. The error output is this: >>> >>> >>> >>> $ cat >>> ./testing-results/SuiteUID:atomic-cpu_1-cores_classic_kvm-fork-run-test-GCN3_X86-x86_64-opt/TestUID:atomic-cpu_1-cores_classic_kvm-fork-run-test-GCN3_X86-x86_64-opt/simerr >>> warn: <orphan X86Board>.kvm_vm already has parent not resetting parent. >>> Note: kvm_vm is not a parameter of X86Board >>> warn: (Previously declared as <orphan X86Board>.processor.kvm_vm >>> HTTPError: HTTP Error 429: Too Many Requests >>> >>> At: >>> /usr/lib/python3.10/urllib/request.py(643): http_error_default >>> /usr/lib/python3.10/urllib/request.py(496): _call_chain >>> /usr/lib/python3.10/urllib/request.py(563): error >>> /usr/lib/python3.10/urllib/request.py(634): http_response >>> /usr/lib/python3.10/urllib/request.py(525): open >>> /usr/lib/python3.10/urllib/request.py(216): urlopen >>> build/GCN3_X86/python/gem5/resources/downloader.py(77): >>> _get_resources_json >>> build/GCN3_X86/python/gem5/resources/downloader.py(207): >>> get_resources_json_obj >>> build/GCN3_X86/python/gem5/resources/resource.py(162): __init__ >>> /home/gblack/gem5/work/tests/gem5/configs/boot_kvm_fork_run.py(205): >>> <module> >>> build/GCN3_X86/python/m5/main.py(434): main >>> >>> >>> >>> >>> >>> Is there a limit that needs to be bumped up? Some way to consolidate >>> requests so we're not hitting the existing limit? >>> >>> >>> >>> Gabe >>> IMPORTANT NOTICE: The contents of this email and any attachments are >>> confidential and may also be privileged. If you are not the intended >>> recipient, please notify the sender immediately and do not disclose the >>> contents to any other person, use it for any purpose, or store or copy the >>> information in any medium. Thank you. >>> _______________________________________________ >>> gem5-dev mailing list -- gem5-dev@gem5.org >>> To unsubscribe send an email to gem5-dev-le...@gem5.org >>> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s >> >>
_______________________________________________ gem5-dev mailing list -- gem5-dev@gem5.org To unsubscribe send an email to gem5-dev-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
