[RADIATOR] Radiator performance problem with specific hardware
Hello, We are having problems with Radiator performance while using specific hardware and software. The hardware we're using is: CPU: Sun SPARC T5140 - having 2 x SunSparc 1.2Ghz CPU, each CPU having 8 cores, and each core simulates 8 virtual processors, so system has 128 virtual (logical) processors The software we're using: OS: Solaris 10 Radiator 4.5.1. The performance problem appears when using more than 64 workers defined in FarmSize parameter. If we use more than 64 workers, number of requests per second drops drastically (measured values: with 64 workers- 5400 requests/second, 128 workers - 1200 requests/second). By doing some specific tests we've come to conclusion that problem lies somewhere in simultaneous multiple read/write to socket (UDP queue) mechanism. Has enyone encountered some simmilar problems? Where to look for solution? Any suggestion is welcome. Kind regards, Damjan Kukas The information contained in this e-mail message is privileged and confidential and is for the exclusive use of the addressee. The person who receives this message and who is not the addressee, one of his employees or an agent entitled to hand it over to the addressee, is informed that he may not use, disclose or reproduce the contents thereof, and is kindly asked to notify the sender and delete the e-mail immediately. ___ radiator mailing list radiator@open.com.au http://www.open.com.au/mailman/listinfo/radiator
Re: [RADIATOR] Radiator performance problem with specific hardware
Hi, On Wed, 1 Sep 2010, Kukas Damjan wrote: > Hello, > > We are having problems with Radiator performance while using specific > hardware and software. > The hardware we're using is: > CPU: Sun SPARC T5140 - having 2 x SunSparc 1.2Ghz CPU, each CPU having 8 > cores, and each core simulates 8 virtual processors, so system has 128 > virtual (logical) processors > > The software we're using: > OS: Solaris 10 > Radiator 4.5.1. > > The performance problem appears when using more than 64 workers defined in > FarmSize parameter. If we use more than 64 workers, number of requests per > second drops drastically (measured values: with 64 workers- 5400 > requests/second, 128 workers - 1200 requests/second). By doing some specific > tests we've come to conclusion that problem > lies somewhere in simultaneous multiple read/write to socket (UDP queue) > mechanism. from your description this does sound a lot like your are hitting a lock contention issue in the operating system ( solaris ). If you have enough Requests to saturate that many cores you might try splitting the radiator into for example 4 instances on separate ports with each instance having a farm of 32 workers. That would give you 4 separate udp queues to 4 separate farms and might perhaps get you around your operating system issue. Of course you would have to distribute the load to the 4 instances either directly from your radius clients or via other means. I do not know if this is an option in your situation. Greetings Christian Kratzer CK Software GmbH -- Christian Kratzer CK Software GmbH Email: c...@cksoft.de Schwarzwaldstr. 31 Phone: +49 7452 889 135 D-71131 Jettingen Fax: +49 7452 889 136 HRB 245288, Amtsgericht Stuttgart Web: http://www.cksoft.de/ Geschaeftsfuehrer: Christian Kratzer ___ radiator mailing list radiator@open.com.au http://www.open.com.au/mailman/listinfo/radiator
Re: [RADIATOR] Radiator performance problem with specific hardware
Hello Kukas, Hello Christian - I agree with Christian - in my consulting practice I almost always find that it is preferable to set up frontend / multiple backend instances of Radiator designed to break up processing into separate processes running on different ports. At the very least you should be running separate instances for authentication and accounting. regards Hugh On 1 Sep 2010, at 08:20, Christian Kratzer wrote: > Hi, > > On Wed, 1 Sep 2010, Kukas Damjan wrote: > >> Hello, >> >> We are having problems with Radiator performance while using specific >> hardware and software. >> The hardware we're using is: >> CPU: Sun SPARC T5140 - having 2 x SunSparc 1.2Ghz CPU, each CPU having 8 >> cores, and each core simulates 8 virtual processors, so system has 128 >> virtual (logical) processors >> >> The software we're using: >> OS: Solaris 10 >> Radiator 4.5.1. >> >> The performance problem appears when using more than 64 workers defined in >> FarmSize parameter. If we use more than 64 workers, number of requests per >> second drops drastically (measured values: with 64 workers- 5400 >> requests/second, 128 workers - 1200 requests/second). By doing some specific >> tests we've come to conclusion that problem >> lies somewhere in simultaneous multiple read/write to socket (UDP queue) >> mechanism. > > from your description this does sound a lot like your are hitting a lock > contention > issue in the operating system ( solaris ). > > If you have enough Requests to saturate that many cores you might try > splitting the > radiator into for example 4 instances on separate ports with each instance > having a > farm of 32 workers. > > That would give you 4 separate udp queues to 4 separate farms and might > perhaps get you around your operating system issue. > > Of course you would have to distribute the load to the 4 instances > either directly from your radius clients or via other means. I do not know > if this is an option in your situation. > > Greetings > Christian Kratzer > CK Software GmbH > > -- > Christian Kratzer CK Software GmbH > Email: c...@cksoft.de Schwarzwaldstr. 31 > Phone: +49 7452 889 135 D-71131 Jettingen > Fax: +49 7452 889 136 HRB 245288, Amtsgericht Stuttgart > Web: http://www.cksoft.de/ Geschaeftsfuehrer: Christian Kratzer > ___ > radiator mailing list > radiator@open.com.au > http://www.open.com.au/mailman/listinfo/radiator NB: Have you read the reference manual ("doc/ref.html")? Have you searched the mailing list archive (www.open.com.au/archives/radiator)? Have you had a quick look on Google (www.google.com)? Have you included a copy of your configuration file (no secrets), together with a trace 4 debug showing what is happening? -- Radiator: the most portable, flexible and configurable RADIUS server anywhere. Available on *NIX, *BSD, Windows, MacOS X. Includes support for reliable RADIUS transport (RadSec), and DIAMETER translation agent. - Nets: internetwork inventory and management - graphical, extensible, flexible with hardware, software, platform and database independence. - CATool: Private Certificate Authority for Unix and Unix-like systems. ___ radiator mailing list radiator@open.com.au http://www.open.com.au/mailman/listinfo/radiator