Hi Giacomo, As requested, I have created an issue: https://gem5.atlassian.net/browse/GEM5-711 In the meantime, I have run a few more configurations. It appears this bug is also present when using the VExpress_GEM5_V1 platform or using the DerivO3CPU model. I have summarized in the issue the configurations I have tested and the ones that seem to work.
Thanks, Nathanael From: Giacomo Travaglini [mailto:giacomo.travagl...@arm.com] Sent: Thursday, 30 July 2020 11:32 To: Nathanael Premillieu <nathanael.premill...@huawei.com>; gem5 users mailing list <gem5-users@gem5.org> Cc: Ciro Santilli <ciro.santi...@arm.com> Subject: RE: [gem5-users] Re: Current status of gem5 capabilities regarding multicores full system simulation Thanks Nathanael, Could you open a JIRA ticket for that? https://gem5.atlassian.net/projects/GEM5/issues Kind Regards Giacomo From: Nathanael Premillieu <nathanael.premill...@huawei.com<mailto:nathanael.premill...@huawei.com>> Sent: 29 July 2020 16:46 To: Giacomo Travaglini <giacomo.travagl...@arm.com<mailto:giacomo.travagl...@arm.com>>; gem5 users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Cc: Ciro Santilli <ciro.santi...@arm.com<mailto:ciro.santi...@arm.com>> Subject: RE: [gem5-users] Re: Current status of gem5 capabilities regarding multicores full system simulation Hi Giacomo, Thanks for the patch. I've tried with it and it seems to work. But not for all number of cores. With gem5 v20.0.0.2, I've tried with n=1, 2, 3, 4, 5 and linux can boot, but with n>=6 (I've tried 6, 8, 16, 32, 64), I have the following error, soon after start: panic: panic condition (pkt->needsWritable() != pkt->isInvalidate()) && !pkt->req->isCacheMaintenance() occurred: global got snoop WriteReq [80a70800:80a70803] UC where needsWritable, does not match isInvalidate With gem5 develop branch (v20.0.0.3-294-g0f4ecba2a), I see them same thing. For reference, the command line I have used: $ build/ARM/gem5.opt configs/example/fs.py --kernel=binaries/vmlinux.arm64 --machine-type=VExpress_GEM5_V2 --disk-image=aarch64-ubuntu-trusty-headless.img --cpu-type=O3_ARM_v7a_3 --caches --l2cache -n 6 And a more detailed failing output: Global frequency set at 1000000000000 ticks per second warn: No dot file generated. Please install pydot to generate the dot file and pdf. warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes) info: kernel located at: binaries/vmlinux.arm64 warn: Highest ARM exception-level set to AArch32 but the workload is for AArch64. Assuming you wanted these to match. system.vncserver: Listening for connections on port 5900 system.terminal: Listening for connections on port 3456 system.realview.uart1.device: Listening for connections on port 3457 system.realview.uart2.device: Listening for connections on port 3458 system.realview.uart3.device: Listening for connections on port 3459 0: system.remote_gdb: listening for remote gdb on port 7000 0: system.remote_gdb: listening for remote gdb on port 7001 0: system.remote_gdb: listening for remote gdb on port 7002 0: system.remote_gdb: listening for remote gdb on port 7004 0: system.remote_gdb: listening for remote gdb on port 7005 0: system.remote_gdb: listening for remote gdb on port 7006 info: Using bootloader at address 0x10 info: Using kernel entry physical address at 0x80080000 info: Loading DTB file: m5out/system.dtb at address 0x88000000 **** REAL SIMULATION **** warn: Existing EnergyCtrl, but no enabled DVFSHandler found. info: Entering event queue @ 0. Starting simulation... warn: SCReg: Access to unknown device dcc0:site0:pos0:fn7:dev0 warn: instruction 'csdb' unimplemented warn: Gicv3Distributor::write(): setting ARE to 0 is not supported! panic: panic condition (pkt->needsWritable() != pkt->isInvalidate()) && !pkt->req->isCacheMaintenance() occurred: global got snoop WriteReq [80a70800:80a70803] UC where needsWritable, does not match isInvalidate Memory Usage: 1142332 KBytes Program aborted at tick 13365162500 --- BEGIN LIBC BACKTRACE --- ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xbe72f0)[0x55936b9ea2f0] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xbfae1e)[0x55936b9fde1e] /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7faf21ea03c0] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7faf2144418b] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7faf21423859] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xfc80d0)[0x55936bdcb0d0] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xfbbf93)[0x55936bdbef93] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0x1e65cbc)[0x55936cc68cbc] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0x1e69939)[0x55936cc6c939] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xfb4a0e)[0x55936bdb7a0e] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xfa7eb9)[0x55936bdaaeb9] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xbef2e9)[0x55936b9f22e9] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xc104e8)[0x55936ba134e8] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0xc112dd)[0x55936ba142dd] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0x21544b0)[0x55936cf574b0] ../../../tools/gem5/gem5.git/build/ARM/gem5.opt(+0x8e3aef)[0x55936b6e6aef] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(+0x2a12d8)[0x7faf2214f2d8] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x8b70)[0x7faf21f29f70] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0x8fb)[0x7faf2207388b] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x90)[0x7faf2214ecd0] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(+0x72ffd)[0x7faf21f20ffd] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x8067)[0x7faf21f29467] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(+0x7e0db)[0x7faf21f2c0db] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(+0x72ffd)[0x7faf21f20ffd] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x3c6b)[0x7faf21f2506b] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0x8fb)[0x7faf2207388b] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x90)[0x7faf2214ecd0] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(+0x72ffd)[0x7faf21f20ffd] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x8067)[0x7faf21f29467] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0x8fb)[0x7faf2207388b] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(PyEval_EvalCodeEx+0x3e)[0x7faf22073c0e] /lib/x86_64-linux-gnu/libpython3.8.so.1.0(PyEval_EvalCode+0x1b)[0x7faf22073fbb] --- END LIBC BACKTRACE --- Aborted (core dumped) Thanks, Nathanael From: Giacomo Travaglini [mailto:giacomo.travagl...@arm.com] Sent: Tuesday, 28 July 2020 15:08 To: Nathanael Premillieu <nathanael.premill...@huawei.com<mailto:nathanael.premill...@huawei.com>>; gem5 users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Cc: Ciro Santilli <ciro.santi...@arm.com<mailto:ciro.santi...@arm.com>> Subject: RE: [gem5-users] Re: Current status of gem5 capabilities regarding multicores full system simulation Nathanael, The only difference is that V1 is using GICv2 and V2 is using GICv3. About what I am suggesting: while using V2 with DTB autogen (as Ciro suggested), apply the following diff diff --git a/src/dev/arm/RealView.py b/src/dev/arm/RealView.py index f78b41e..d667659 100644 --- a/src/dev/arm/RealView.py +++ b/src/dev/arm/RealView.py @@ -1184,7 +1184,7 @@ class VExpress_GEM5_V1(VExpress_GEM5_V1_Base): class VExpress_GEM5_V2_Base(VExpress_GEM5_Base): gic = Gicv3(dist_addr=0x2c000000, redist_addr=0x2c010000, maint_int=ArmPPI(num=25), - its=Gicv3Its(pio_addr=0x2e010000)) + its=NULL) Let me know if it works, Giacomo From: Nathanael Premillieu <nathanael.premill...@huawei.com<mailto:nathanael.premill...@huawei.com>> Sent: 28 July 2020 13:07 To: Giacomo Travaglini <giacomo.travagl...@arm.com<mailto:giacomo.travagl...@arm.com>>; gem5 users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Cc: Ciro Santilli <ciro.santi...@arm.com<mailto:ciro.santi...@arm.com>> Subject: RE: [gem5-users] Re: Current status of gem5 capabilities regarding multicores full system simulation Hi, Thank you all for your answers. Jason: The page http://www.arm.com/ResearchEnablement/SystemModeling doesn't seem to exist anymore. But I guess you are referring to the Arm Research Enablement Kit located here: https://github.com/arm-university/arm-gem5-rsk. I have looked at it but it is fairly old (3 years, some links are broken) and mainly target MinorCPU while I'm more interested in O3. For the starter_fs script, I think it is linked to the Arm Research Enablement Kit and it does not seems to support O3 model. Ciro: Thanks for opening a bug report! >From what I have seen, the nightly test is using O3_ARM_v7a_3 model so I guess >it is ok if I use it too. It is also using VExpress_GEM5_V1 as a platform, so >yeah like Giacomo has suggested, I guess the problem is coming from the >VExpress_GEM5_V2 platform. Thanks for the hint on the DTB files, makes things easier :) Giacomo: I'm not sure to understand what you are suggesting, could you be more precise? Also, is there limitations in using VExpress_GEM5_V1 (which seems to work) instead of VExpress_GEM5_V2? Thanks, Nathanael From: Giacomo Travaglini [mailto:giacomo.travagl...@arm.com] Sent: Tuesday, 28 July 2020 09:56 To: gem5 users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Cc: Nathanael Premillieu <nathanael.premill...@huawei.com<mailto:nathanael.premill...@huawei.com>>; Ciro Santilli <ciro.santi...@arm.com<mailto:ciro.santi...@arm.com>> Subject: Re: [gem5-users] Re: Current status of gem5 capabilities regarding multicores full system simulation I have the suspicion the problem is in the Gicv3 ITS doing DMAs to the coherent XBar without a cache controller. Could you disable it from V2? Otherwise you would need to put a cache between the ITS and the XBar Let me know if it works, Giacomo Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: Ciro Santilli via gem5-users <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Sent: Tuesday, July 28, 2020 8:30:47 AM To: gem5-users@gem5.org<mailto:gem5-users@gem5.org> <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Cc: Nathanael Premillieu <nathanael.premill...@huawei.com<mailto:nathanael.premill...@huawei.com>>; Ciro Santilli <ciro.santi...@arm.com<mailto:ciro.santi...@arm.com>> Subject: [gem5-users] Re: Current status of gem5 capabilities regarding multicores full system simulation Welcome back Nathanael! We put great importance on ARM Linux kernel boot, and we try to make it so that it either works or at least we have a bug report for it. I reproduce the problem with VExpress_GEM5_V2 but not VExpress_GEM5_V1, opening a but for V2 at: https://gem5.atlassian.net/browse/GEM5-706 Besides that, some notes: O3_ARM_v7a_3: does not sound like a good idea for aarch64 since v7 in name, not sure it has all functional units required. --dtb-file is not needed anymore unless you are hacking the DTB manually. Just remove that option to use DTB auto-generation which is more likely to work. The newly setup nightlies (thanks Bobby!) contain a two core O3 test that passed yesterday: https://jenkins.gem5.org/job/Nightly/17/artifact/tests/.testing-results/results.xml <testcase name="realview-o3-dual-ARM-x86_64-opt" classname="TestUID:tests/gem5/fs/linux/arm/test.py:realview-o3-dual-ARM-x86_64-opt:realview-o3-dual-ARM-x86_64-opt" status="Passed"> ________________________________ From: Nathanael Premillieu via gem5-users <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Sent: Monday, July 27, 2020 6:52 PM To: gem5-users@gem5.org<mailto:gem5-users@gem5.org> <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Cc: Nathanael Premillieu <nathanael.premill...@huawei.com<mailto:nathanael.premill...@huawei.com>> Subject: [gem5-users] Current status of gem5 capabilities regarding multicores full system simulation Hi all, After a few years away, I'm back to gem5. I have to say a lot of things have changed, for the better, thank you all! I'm trying to assess what is the current state of gem5 regarding simulating configurations with multiples cores in a full system environment with the different cpu models. Especially with the Arm architecture and the classic memory model in a full system simulation using the O3CPU model. And what happens if you add SMT on top of that. I've looked at this page: https://www.gem5.org/documentation/benchmark_status/, but from what I can infer, those results are only valid for X86. I've tried for example to boot linux (using the kernel provided here: http://dist.gem5.org/dist/current/arm/aarch-system-201901106.tar.bz2 and the disk image here: http://dist.gem5.org/dist/current/arm/disks/aarch64-ubuntu-trusty-headless.img.bz2) but without success. But I don't know if I'm doing something wrong or if I'm trying to do something that is unsupported for the moment. I'm using gem5 version v20.0.0.2 and here is my command line: $ build/ARM/gem5.opt configs/example/fs.py --kernel=binaries/vmlinux.arm64 --machine-type=VExpress_GEM5_V2 --dtb-file=system/arm/dt/armv8_gem5_v2_4cpu.dtb --disk-image=aarch64-ubuntu-trusty-headless.img --cpu-type=O3_ARM_v7a_3 --caches --l2cache -n 4 I get the following error: Assertion `reqLookupResult.it->first == line_addr' failed. Global frequency set at 1000000000000 ticks per second warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes) info: kernel located at: binaries/vmlinux.arm64 warn: Highest ARM exception-level set to AArch32 but the workload is for AArch64. Assuming you wanted these to match. system.vncserver: Listening for connections on port 5900 system.terminal: Listening for connections on port 3456 system.realview.uart1.device: Listening for connections on port 3457 system.realview.uart2.device: Listening for connections on port 3458 system.realview.uart3.device: Listening for connections on port 3459 0: system.remote_gdb: listening for remote gdb on port 7000 0: system.remote_gdb: listening for remote gdb on port 7001 0: system.remote_gdb: listening for remote gdb on port 7002 0: system.remote_gdb: listening for remote gdb on port 7003 info: Using bootloader at address 0x10 info: Using kernel entry physical address at 0x80080000 info: Loading DTB file: system/arm/dt/armv8_gem5_v2_4cpu.dtb at address 0x88000000 **** REAL SIMULATION **** warn: Existing EnergyCtrl, but no enabled DVFSHandler found. info: Entering event queue @ 0. Starting simulation... warn: SCReg: Access to unknown device dcc0:site0:pos0:fn7:dev0 warn: instruction 'csdb' unimplemented warn: Gicv3Distributor::write(): setting ARE to 0 is not supported! gem5.opt: build/ARM/mem/snoop_filter.cc:165: void SnoopFilter::finishRequest(bool, Addr, bool): Assertion `reqLookupResult.it->first == line_addr' failed. Program aborted at tick 12946004000 I get the same error using only one core (-n 1). I've also tried with the current state of the develop branch (commit v20.0.0.3-290-gcef72adab) and I get the same error. Thanks, Nathanael IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
_______________________________________________ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
