Hi Tushar, I wonder if all the nodes(controllers) need to have a router before them. This is what I have seen for all the topologies created in GEM5. I have tried topology like the one I attach here. If I run it with ruby_network_tester for simple network with Hammer protocol for 1000000000 sim-cycles. I will have deadlock. But if I run the same for Pt2Pt in GEM5 for example, then there is no deadlock. Do you think because controllers do not know how to handle multiple input links well? or there are some other reasons. Please help me figure it out. Thank you very much!
Best, Jinzhu On Thu, Aug 23, 2012 at 10:07 PM, gem5 gem5 <gem5.user....@gmail.com> wrote: > Thank you very much, Tushar:) > > Best, > > Jinzhu > > > On Wed, Aug 22, 2012 at 12:24 AM, Tushar Krishna <tus...@csail.mit.edu>wrote: > >> Hi Jinzhu, >> Do the orange routers guarantee that the message is delivered out of all >> output links simultaneously? >> I think something like the following could be the problem: >> Suppose Core B and Core C are sharing a line. >> Core A does a broadcast for GETX. At the orange routers, suppose it >> reaches Core B and Core C, but not Core D since that output link is busy >> and/or no free buffer at Core D. >> Now Core B looks at this message and invalidates its copy. Now it sends >> out a GETX. >> Suppose Core D receives this GETX (via Core B's orange router) before the >> older GETX from Core A (still stuck in Core A's router for some reason). >> This will violate ordering (because other cores saw Core A's broadcast >> before Core B's). >> >> (1) since the orange routers are modeling a "bus", any message being >> broadcast should wait till ALL output links are free and then be sent out >> of all output links simultaneously. >> (2) after that the blue routers should guarantee a FIFO ordering (to >> ensure that messages received from two different "buses" don't get >> reordered) when delivering messages to the core. >> >> I *think* (though you should check) that (2) is guaranteed automatically >> in the code since the input queues are implemented as MessageBuffers which >> order messages according to time of receipt, but I know (1) is not >> guaranteed by default in the simple router. >> See if that solves your issue. >> While a stronger condition of latency between two nodes always being the >> same will definitely solve your problem, I think a weaker condition of >> ensuring that ALL the blue routers receive messages from different buses in >> the same order should be the constraint. >> >> - Tushar >> >> >> On Aug 21, 2012, at 5:30 PM, gem5 gem5 wrote: >> >> Tushar, >> >> All the links used in my topology are single directional. The connection >> is as the attached image. It's Single writer multiple reader and I think >> this topology should be able to guarantee that certain messages only go on >> certain buses. >> >> I am thinking the problem is that the total order is achieved only when >> the latency between any two nodes is always the same, but the throttle link >> creates lots of delays(some messages got sent much later) so latency is not >> always the same.....And maybe I should also create one MessageBuffer/queue >> for every bus in the controller instead of just one MessageBuffer for one >> vnet. Then I can read messages from all the messagebuffer round robin. Is >> the simple network modeled cycle accurate? Any suggestions? Thank you very >> much! >> >> Best, >> Jinzhu >> >> On Mon, Aug 20, 2012 at 1:14 AM, Tushar Krishna <tus...@csail.mit.edu>wrote: >> >>> Hi Jinzhu, >>> While it is true that the a garnet router is more accurate than the >>> PerfectSwitch as it models contention in a detailed manner, I don't think >>> your problem has anything to do with that. >>> I think you should track messages through each router and see why/where >>> 2 messages from the same source get re-ordered. >>> >>> I am wondering if the problem could with the topology itself… >>> Are you modeling multiple buses as multiple links between routers? >>> How does a router guarantee that certain messages only go on certain >>> links (buses), and certain others go on other links? By default both >>> simple/garnet networks try to route using the shortest path, and here all >>> buses will be on the shortest path. >>> >>> - Tushar >>> >>> >>> On Aug 19, 2012, at 11:41 PM, gem5 gem5 wrote: >>> >>> Hi Tushar, >>> >>> I have tried to modify the PerfectSwitch to schedule messages according >>> to a fixed round robin order. And then I also did the same to the throttle >>> link by changing the wakeup order. However, I still cannot see the total >>> order in multiple logic buses. I wonder if that is because the >>> implementation of PerfectSwtich does not reflect the timing characteristic >>> of a real router. It just simply copies everything from the inport to the >>> outport. Although I scheduled according to a predefined order, but actually >>> it's still done right away. Maybe a more detailed model such as Garnet >>> router can solve the problem? I wonder if that's the issue or there is >>> something else I am missing here. >>> >>> What I did is just to take a few logic buses (I have tested a individual >>> bus with a broadcast protocol and it works). Each node has a router before >>> all the input links from the buses to model multiple reading channels for >>> the node. Each node only has one output link connecting to its dedicated >>> writing bus to model the single writing channel. In this way, it models >>> single write multi read bus-based crossbar....I am not sure if this is the >>> right way to model it and if this kind of design should provide total >>> order. Any comments? >>> >>> Thank you very much! >>> >>> Best, >>> >>> Jinzhu >>> >>> On Fri, Aug 10, 2012 at 9:58 PM, gem5 gem5 <gem5.user....@gmail.com>wrote: >>> >>>> Tushar, >>>> >>>> Thank you very much. Yes, I guess all the routers are connected to the >>>> same MessageBuffer and each router represents a logic bus. I think the >>>> controller does not listen to all the buses according to a particular >>>> order. Even if I connect a router before the controller, the router might >>>> not listen to all the buses according to a particular order. I thought the >>>> simple network's router arbitrates/allocates everything following the same >>>> round robin, but maybe that's not the case... >>>> >>>> Thanks for your suggestion. I will look at the simple router first and >>>> then check the event queue if necessary. >>>> >>>> Best, >>>> >>>> Jinzhu >>>> >>>> >>>> On Fri, Aug 10, 2012 at 12:19 AM, Tushar Krishna <tus...@csail.mit.edu >>>> > wrote: >>>> >>>>> Hi Jinzhu, >>>>> Hmm if I understand your topology correctly, each controller connects >>>>> to multiple routers: each router corresponding to one bus. >>>>> The problem is that all these routers connect to the same >>>>> MessageBuffer in the controller... ? >>>>> And I guess the problem occurs because the routers on each bus can >>>>> wake up in any order within a cycle which means the controller's >>>>> MessageBuffer does not see messages from each bus in a particular order. >>>>> >>>>> If this is the problem, connecting all buses/routers to *one* router >>>>> which then connects to the controller, and then reading each bus in a >>>>> particular order should solve the problem. But you say you tried that? You >>>>> should probably debug why that doesn't work.. >>>>> >>>>> If your problem can be solved by a particular wakeup order, you could >>>>> look into the event queue to see if you can create some order between >>>>> events with the same time. I don't have any experience with that though... >>>>> >>>>> - Tushar >>>>> >>>>> >>>>> On Aug 9, 2012, at 5:37 PM, gem5 gem5 <gem5.user....@gmail.com> wrote: >>>>> >>>>> > Hi, >>>>> > >>>>> > I have created a logic bus to run a broadcast protocol which >>>>> requires total ordering in ruby simple network. It works. Now I need to >>>>> have multiple logic buses. To achieve total ordering among different >>>>> buses, >>>>> each cache/directory controller needs to listen to all the buses according >>>>> to a predefined order. I have tried several methods, such as, set the >>>>> interface MessageBuffer order to be true, connect a router between all the >>>>> input links and a controller, but none of them works. I wonder what 's the >>>>> right way to achieve the goal. Thanks. >>>>> > >>>>> > Best, >>>>> > >>>>> > Jinzhu >>>>> > _______________________________________________ >>>>> > gem5-users mailing list >>>>> > gem5-users@gem5.org >>>>> > http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>> _______________________________________________ >>>>> gem5-users mailing list >>>>> gem5-users@gem5.org >>>>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>>>> >>>> >>>> >>> _______________________________________________ >>> gem5-users mailing list >>> gem5-users@gem5.org >>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>> >>> >>> >>> _______________________________________________ >>> gem5-users mailing list >>> gem5-users@gem5.org >>> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >>> >> >> <SWMR.jpg>_______________________________________________ >> >> gem5-users mailing list >> gem5-users@gem5.org >> http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users >> >> >> >
<<attachment: topology.jpg>>
_______________________________________________ gem5-users mailing list gem5-users@gem5.org http://m5sim.org/cgi-bin/mailman/listinfo/gem5-users
