Hello An update on my previous email...
Have been simulating the multicore system with Parsec/Splash2 benchmarks for different permutations of clusivity for L2$ and L3$. The results being in the following table. Note, by L3$, I mean the L3$ within the HNF L2$ clusivity L3$ clusivity Comments Strict Inclusive (sincl) (default) Mostly inclusive (mincl) (default) All tests complete okay mincl mincl All tests complete okay mincl Mostly exclusive (mexcl) 10 tests abort with the assertion failure sincl mexcl 10 tests abort with the assertion failure >From the above, the deduction being that setting L3$ clusivity to mostly >exclusive would be the cause of the problem. The definition of mostly_inclusive (defined by default for the HNFCache controller in CHI_config.py) and mostly_exclusive (based on the write up @ https://www.gem5.org/documentation/general_docs/ruby/CHI/ and my understanding being that the L3$ now becomes a victim cache of the L2$) that I have used being: mostly inclusive mostly exclusive comments alloc_on_seq_acc False False alloc_on_seq_line_write False False alloc_on_readshared True False alloc_on_readunique False False alloc_on_readonce True False alloc_on_writeback True True For the L3$, writebacks and evictions being the mechanism of allocating a cache line dealloc_on_unique True True Upstream $line becomes unique, then deallocate from L3$ dealloc_on_shared False True Upstream $line becomes shared, then deallocate from L3$ dealloc_backinv_unique False False If the L3$ line is deallocated due to replacement, then don't back invalidate the upstream cache line. dealloc_backinv_shared False False If the L3$ line is deallocated due to replacement, then don't back invalidate the upstream cache line. Any insight as to why the above encoding for mostly exclusive might be wrong and thus causing the assertions to fire, would be greatly appreciated. Thanks in advance JO From: Javed Osmany Sent: 21 April 2022 16:03 To: gem5 users mailing list <gem5-users@gem5.org> Cc: Javed Osmany <javed.osm...@huawei.com> Subject: CHi - assertion error when modelling "mostly inclusive" for private L2$ Hello I am simulating a multicore Ruby system using CHI, using the Parsec/Splash2 benchmarks & gem5-21.2.1.0. It consists of three clusters : 1) Little cluster of 4 CPUs, each CPU has private L1$ and L2$ 2) Middle cluster of 3 CPUs, each CPU has private L1$ and L2$ 3) Big cluster of 1 CPU with private L1$ and L2$. By default, the L2$ and L3$ (residing in the HNF) have their clusivity set to strict_inclusive and mostly_inclusive respectively (CHI_config.py): class CHI_L2Controller(CHI_Cache_Controller): ''' Default parameters for a L2 Cache controller ''' def __init__(self, ruby_system, cache, l2_clusivity, prefetcher): super(CHI_L2Controller, self).__init__(ruby_system) self.sequencer = NULL self.cache = cache self.use_prefetcher = False self.allow_SD = True self.is_HN = False self.enable_DMT = False self.enable_DCT = False self.send_evictions = False # Strict inclusive MOESI self.alloc_on_seq_acc = False self.alloc_on_seq_line_write = False self.alloc_on_readshared = True self.alloc_on_readunique = True self.alloc_on_readonce = True self.alloc_on_writeback = True self.dealloc_on_unique = False self.dealloc_on_shared = False self.dealloc_backinv_unique = True self.dealloc_backinv_shared = True class CHI_HNFController(CHI_Cache_Controller): ''' Default parameters for a coherent home node (HNF) cache controller ''' #def __init__(self, ruby_system, cache, prefetcher, addr_ranges): def __init__(self, ruby_system, cache, prefetcher, addr_ranges, hnf_enable_dmt, hnf_enable_dct, \ num_tbe, num_repl_tbe, num_snp_tbe, unified_repl_tbe, l3_clusivity): super(CHI_HNFController, self).__init__(ruby_system) self.sequencer = NULL self.cache = cache self.use_prefetcher = False self.addr_ranges = addr_ranges self.allow_SD = True self.is_HN = True #self.enable_DMT = True #self.enable_DCT = True self.enable_DMT = hnf_enable_dmt self.enable_DCT = hnf_enable_dct self.send_evictions = False # MOESI / Mostly inclusive for shared / Exclusive for unique self.alloc_on_seq_acc = False self.alloc_on_seq_line_write = False self.alloc_on_readshared = True self.alloc_on_readunique = False self.alloc_on_readonce = True self.alloc_on_writeback = True self.dealloc_on_unique = True self.dealloc_on_shared = False self.dealloc_backinv_unique = False self.dealloc_backinv_shared = False The simulations complete okay for the default clusivity of L2$ and L3$. However, if I change the L2$ clusivity to "mostly_inclusive" some of the benchmarks are failing with an assertion error. I took the default L3$ clusivity of mostly_inclusive to update the L2$ clusivity to be mostly_inclusive: class CHI_L2Controller(CHI_Cache_Controller): ''' Default parameters for a L2 Cache controller ''' def __init__(self, ruby_system, cache, l2_clusivity, prefetcher): super(CHI_L2Controller, self).__init__(ruby_system) self.sequencer = NULL self.cache = cache self.use_prefetcher = False self.allow_SD = True self.is_HN = False self.enable_DMT = False self.enable_DCT = False self.send_evictions = False # Strict inclusive MOESI if (l2_clusivity == "sincl"): self.alloc_on_seq_acc = False self.alloc_on_seq_line_write = False self.alloc_on_readshared = True self.alloc_on_readunique = True self.alloc_on_readonce = True self.alloc_on_writeback = True self.dealloc_on_unique = False self.dealloc_on_shared = False self.dealloc_backinv_unique = True self.dealloc_backinv_shared = True elif (l2_clusivity == "mincl"): # Mostly inclusive MOESI self.alloc_on_seq_acc = False self.alloc_on_seq_line_write = False self.alloc_on_readshared = True self.alloc_on_readunique = False self.alloc_on_readonce = True self.alloc_on_writeback = True self.dealloc_on_unique = True self.dealloc_on_shared = False self.dealloc_backinv_unique = False self.dealloc_backinv_shared = False The assertion error being: log_parsec_volrend_134_8rnf_1snf_4hnf_3_clust_all_priv_l2.txt:build/ARM/mem/ruby/protocol/Cache_Controller.cc:5477: panic: Runtime Error at CHI-cache-actions.sm:1947: assert failure. QS 1: Even though the L2$ is private, i am assuming that L2$ clusivity can be set to mostly_inclusive. Is that assumption correct? QS2: If the answer to QS 1 is yes, then it would seem that the "mostly_inclusive" settings for the L2$ (copied from the mostly_inclusive settings for L3$ residing in the HNF) could be the root cause of the problem. Any thoughts on this ? Thanks in advance JO
_______________________________________________ gem5-users mailing list -- gem5-users@gem5.org To unsubscribe send an email to gem5-users-le...@gem5.org %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s
