Hi Chen,
The only issue I can spot at this point is the “is_top_level” being set on the
NUMACache. As there are caches above these NUMACaches the value should be
false. For the rest I don’t dare say. Is there any chance you could turn this
into a patch that applies on top of the latest trunk?
Thanks,
Andreas
From: Chen Houwu via gem5-users
<gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Reply-To: Chen Houwu <chenho...@gmail.com<mailto:chenho...@gmail.com>>, gem5
users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Date: Saturday, August 9, 2014 at 3:42 AM
To: "gem5-users@gem5.org<mailto:gem5-users@gem5.org>"
<gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Subject: Re: [gem5-users] help : cc-NUMA in gem5
Hi Andreas,
I just follow your instructions to build the NUMA config,
but it does not work,
gem5 would start to run and quit immediately,
here is the debug ouput
=======debug output begin=======
command line: /vagrant/gem5/build/X86/gem5.opt gem5/configs/example/fs.py
--num-cpus=4 --disk-image=x86root-parsec.img --kernel=vmlinux
Global frequency set at 1000000000000 ticks per second
info: kernel located at: /vagrant/binaries/vmlinux
0: rtc: Real-time clock set to Sun Jan 1 00:00:00 2012
Listening for com_1 connection on port 3456
warn: Reading current count from inactive timer.
0: system.remote_gdb.listener: listening for remote gdb #0 on port 7000
0: system.remote_gdb.listener: listening for remote gdb #1 on port 7001
0: system.remote_gdb.listener: listening for remote gdb #2 on port 7002
0: system.remote_gdb.listener: listening for remote gdb #3 on port 7003
Program received signal SIGSEGV, Segmentation fault.
0x0000000000a845f2 in PacketQueue::checkFunctional (this=0x28aac98,
pkt=pkt@entry=0x7fffffffd250) at build/X86/mem/packet_queue.cc:71
71 {
(gdb) list
66 sendDeferredPacket();
67 }
=====debug output end======
I am not proficient enough to debug this problem,
so I want your help to point out what's wrong with my configs.
I just created:
A. a 4 cpu numa system,
B. each cpu connect to a membus and then a dram controller,
C. each dram is 128MB,
D. every two membus is connected to each other by a Cache(in two directions),
E. with address range set to the slave mebus's address range.
F. IO devices are connected to the first node.
Here is the modification I did for the NUMA model.
========diff start===============
diff --git a/configs/common/CacheConfig.py b/configs/common/CacheConfig.py
index b467b16..85f68b5 100644
--- a/configs/common/CacheConfig.py
+++ b/configs/common/CacheConfig.py
@@ -67,14 +67,21 @@ def config_cache(options, system):
# are not connected using addTwoLevelCacheHierarchy. Use the
# same clock as the CPUs, and set the L1-to-L2 bus width to 32
# bytes (256 bits).
- system.l2 = l2_cache_class(clk_domain=system.cpu_clk_domain,
- size=options.l2_size,
- assoc=options.l2_assoc)
+ l2s = [l2_cache_class(clk_domain=system.cpu_clk_domain,
+ size=options.l2_size,
+ assoc=options.l2_assoc)
+ for x in option.num_cpus]
+
+ tol2buses = [CoherentBus(clk_domain=system.cpu_clk_domain,
+ width=32)
+ for x in option.num_cpus]
+
+ for i, l2 in enumerate(l2s):
+ l2.cpu_side = tol2buses[i].master
+ l2.mem_side = system.membuses[i].slave
- system.tol2bus = CoherentBus(clk_domain = system.cpu_clk_domain,
- width = 32)
- system.l2.cpu_side = system.tol2bus.master
- system.l2.mem_side = system.membus.slave
+ system.l2s = l2s
+ system.tol2buses = tol2buses
for i in xrange(options.num_cpus):
if options.caches:
@@ -93,8 +100,9 @@ def config_cache(options, system):
system.cpu[i].addPrivateSplitL1Caches(icache, dcache)
system.cpu[i].createInterruptController()
if options.l2cache:
- system.cpu[i].connectAllPorts(system.tol2bus, system.membus)
+ system.cpu[i].connectAllPorts(system.tol2buses[i],
+ system.membuses[i])
else:
- system.cpu[i].connectAllPorts(system.membus)
+ system.cpu[i].connectAllPorts(system.membuses[i])
return system
diff --git a/configs/common/Caches.py b/configs/common/Caches.py
index 9f7ac7a..0c1f7f1 100644
--- a/configs/common/Caches.py
+++ b/configs/common/Caches.py
@@ -72,6 +72,14 @@ class IOCache(BaseCache):
forward_snoops = False
is_top_level = True
+class NUMACache(BaseCache):
+ assoc = 8
+ hit_latency = 50
+ response_latency = 50
+ mshrs = 20
+ tgts_per_mshr = 12
+ is_top_level = True
+
class PageTableWalkerCache(BaseCache):
assoc = 2
hit_latency = 2
diff --git a/configs/common/FSConfig.py b/configs/common/FSConfig.py
index dc24adf..79518da 100644
--- a/configs/common/FSConfig.py
+++ b/configs/common/FSConfig.py
@@ -331,13 +331,13 @@ def connectX86ClassicSystem(x86_sys, numCPUs):
interrupts_address_space_base = 0xa000000000000000
APIC_range_size = 1 << 12;
- x86_sys.membus = MemBus()
+ x86_sys.membuses = [MemBus() for x in range(numCPUs)]
# North Bridge
x86_sys.iobus = NoncoherentBus()
x86_sys.bridge = Bridge(delay='50ns')
x86_sys.bridge.master = x86_sys.iobus.slave
- x86_sys.bridge.slave = x86_sys.membus.master
+ x86_sys.bridge.slave = x86_sys.membuses[0].master
# Allow the bridge to pass through the IO APIC (two pages),
# everything in the IO address range up to the local APIC, and
# then the entire PCI address space and beyond
@@ -356,7 +356,7 @@ def connectX86ClassicSystem(x86_sys, numCPUs):
# the local APIC (two pages)
x86_sys.apicbridge = Bridge(delay='50ns')
x86_sys.apicbridge.slave = x86_sys.iobus.master
- x86_sys.apicbridge.master = x86_sys.membus.slave
+ x86_sys.apicbridge.master = x86_sys.membuses[0].slave
x86_sys.apicbridge.ranges = [AddrRange(interrupts_address_space_base,
interrupts_address_space_base +
numCPUs * APIC_range_size
@@ -365,7 +365,7 @@ def connectX86ClassicSystem(x86_sys, numCPUs):
# connect the io bus
x86_sys.pc.attachIO(x86_sys.iobus)
- x86_sys.system_port = x86_sys.membus.slave
+ x86_sys.system_port = x86_sys.membuses[0].slave
def connectX86RubySystem(x86_sys):
# North Bridge
diff --git a/configs/common/MemConfig.py b/configs/common/MemConfig.py
index e954407..e8c623e 100644
--- a/configs/common/MemConfig.py
+++ b/configs/common/MemConfig.py
@@ -153,7 +153,13 @@ def config_mem(options, system):
# For every range (most systems will only have one), create an
# array of controllers and set their parameters to match their
# address mapping in the case of a DRAM
- for r in system.mem_ranges:
+
+ AddrRange = m5.objects.AddrRange
+ numa_mem_ranges = [AddrRange('0MB', size='128MB'),
+ AddrRange('128MB', size='128MB'),
+ AddrRange('256MB', size='128MB'),
+ AddrRange('384MB', size='128MB')]
+ for r in numa_mem_ranges:
for i in xrange(nbr_mem_ctrls):
# Create an instance so we can figure out the address
# mapping and row-buffer size
@@ -191,4 +197,4 @@ def config_mem(options, system):
# Connect the controllers to the membus
for i in xrange(len(system.mem_ctrls)):
- system.mem_ctrls[i].port = system.membus.master
+ system.mem_ctrls[i].port = system.membuses[i].master
diff --git a/configs/common/NUMAConfig.py b/configs/common/NUMAConfig.py
new file mode 100644
index 0000000..71ea0a5
--- /dev/null
+++ b/configs/common/NUMAConfig.py
@@ -0,0 +1,86 @@
+import itertools
+
+from m5.objects import *
+from Benchmarks import *
+from m5.util import *
+from Caches import *
+
+def config_numa(options, system):
+ link_nodes(options, system)
+
+def link_nodes(options, system):
+ numa_caches = []
+ for src, dst in itertools.permutations(range(options.num_cpus), r=2):
+ cache = NUMACache(size="256kB",
+ addr_ranges=[system.mem_ctrls[dst].range])
+ numa_caches.append(cache)
+ cache.cpu_side = system.membuses[src].master
+ cache.mem_side = system.membuses[dst].slave
+ system.numa_caches = numa_caches
diff --git a/configs/example/fs.py b/configs/example/fs.py
index 5847482..de2b0eb 100644
--- a/configs/example/fs.py
+++ b/configs/example/fs.py
@@ -60,6 +60,7 @@ from Benchmarks import *
import Simulation
import CacheConfig
import MemConfig
+import NUMAConfig
from Caches import *
import Options
@@ -177,7 +178,7 @@ def build_test_system(np):
else:
test_sys.iobridge = Bridge(delay='50ns', ranges =
test_sys.mem_ranges)
test_sys.iobridge.slave = test_sys.iobus.master
- test_sys.iobridge.master = test_sys.membus.slave
+ test_sys.iobridge.master = test_sys.membuses[0].slave
# Sanity check
if options.fastmem:
@@ -195,6 +196,7 @@ def build_test_system(np):
CacheConfig.config_cache(options, test_sys)
MemConfig.config_mem(options, test_sys)
+ NUMAConfig.config_numa(options, test_sys)
return test_sys
========diff end===============
Regards,
Chen Houwu
On Mon, 7 Apr 2014 08:32:55 +0100, Andreas Hansson
<andreas.hans...@arm.com><mailto:andreas.hans...@arm.com> wrote:
Hi Faris,
I do not thing there are any existing examples out there. In essence, this is
what I would suggest (and believe should work):
1 change an existing script to create two completely separate CC-UMA “chiplets”
with CPU clusters, along with their L2/L3, membus and (multi-channel) DRAM
controller
2 connect these two blobs with a “numa cache” in each direction, from membus to
membus. Configured right, this will allow the CPUs in one of the chiplets to
talk to the DRAMs in the other in a coherent fashion.
I hope that provides enough information to get you started.
Andreas
From: faris priadi <farispri...@gmail.com<mailto:farispri...@gmail.com>>
Reply-To: gem5 users mailing list
<gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Date: Sunday, 6 April 2014 09:47
To: gem5 users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Subject: Re: [gem5-users] help : cc-NUMA in gem5
Hi,
I tried to configure NUMA in gem5 , but i'm still stuck. Actually about create
own memory controller each CPU, because that i found in internet and forums
were about multi-channels in gem5. Isn't similar configuring multi-channels and
adding own memory controller for each CPU in gem5? thanks
On Tue, Apr 1, 2014 at 11:57 PM, faris priadi
<farispri...@gmail.com<mailto:farispri...@gmail.com>> wrote:
Hi Andreas,
Ok , I will try it first dan try to understand implement my model to gem5 .
anyway i have tried to increase memory size to 4GB with --mem-size in ALPHA
Fullsystem but this following error happened :
1146167500: system.cpu00.break_event: break event panic triggered
fatal: Unable to find destination for addr 0xfffff00188 on bus system.piobus
@ cycle 14723200500
[findPort:build/ALPHA_FS/mem/bus.cc, line 353]
Memory Usage: 5619064 KBytes
Do you know what error it is ? Can you help me solve this? . Thanks
On Tue, Apr 1, 2014 at 2:48 PM, Andreas Hansson
<andreas.hans...@arm.com<mailto:andreas.hans...@arm.com>> wrote:
Hi Faris,
Not at all. You simply assemble the existing building blocks in a slightly
different manner. You should be able to do this entirely through the Python
scripts.
Andreas
From: faris priadi <farispri...@gmail.com<mailto:farispri...@gmail.com>>
Reply-To: gem5 users mailing list
<gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Date: Tuesday, 1 April 2014 03:13
To: gem5 users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Subject: Re: [gem5-users] help : cc-NUMA in gem5
Thank you Andreas for quick response.actually i still little bit confuse . Do
i also have to modify .cc/hh file manually to create NUMA model ?
On Tue, Apr 1, 2014 at 1:46 AM, Andreas Hansson
<andreas.hans...@arm.com<mailto:andreas.hans...@arm.com>> wrote:
Hi Faris,
The short answer is yes. You will have to manually adapt se.py or fs.py based
on your needs. You could, for example, create two clusters of CPUs, each with
their own LLC and memory controllers (and memory ranges), and then connect
their membus instances with a “glue” cache.
Andreas
From: faris priadi <farispri...@gmail.com<mailto:farispri...@gmail.com>>
Reply-To: gem5 users mailing list
<gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Date: Monday, 31 March 2014 18:58
To: "gem5-users@gem5.org<mailto:gem5-users@gem5.org>"
<gem5-users@gem5.org<mailto:gem5-users@gem5.org>>
Subject: [gem5-users] help : cc-NUMA in gem5
Hello everyone ,
I have final project about cc-NUMA and now i try to use gem5 , but i need help
to configure NUMA in gem5. Can gem5 be used to simulate cc-NUMA system ?
--
Best Regards,
Faris Priadi ,Amd.
082121325532<tel:082121325532>
farispri...@gmail.com<mailto:farispri...@gmail.com>
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered
in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2548782
_______________________________________________
gem5-users mailing list
gem5-users@gem5.org<mailto:gem5-users@gem5.org>
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
--
Best Regards,
Faris Priadi ,Amd.
082121325532<tel:082121325532>
farispri...@gmail.com<mailto:farispri...@gmail.com>
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered
in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2548782
_______________________________________________
gem5-users mailing list
gem5-users@gem5.org<mailto:gem5-users@gem5.org>
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
--
Best Regards,
Faris Priadi
farispri...@gmail.com<mailto:farispri...@gmail.com>
--
Best Regards,
Faris Priadi ,Amd.
082121325532
farispri...@gmail.com<mailto:farispri...@gmail.com>
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered
in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2548782
_______________________________________________
gem5-users mailing list
gem5-users@gem5.org<mailto:gem5-users@gem5.org>http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
-- IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered
in England & Wales, Company No: 2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ,
Registered in England & Wales, Company No: 2548782_______________________________________________
gem5-users mailing list
gem5-users@gem5.org
http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
