Wasn't the container testing on Power done using Juju? I thought they hit something like 60k virtual machines.
I have scale tested Juju and the number of agents it can handle to effectively 5000 machines, though this was just agents interacting with the API server, and not a lot of actual workload. I did do stuff like "reconfigure all 5000 units at the same time" and things performed just fine. I was also observing the behavior of "if you kill the API server, how long before the system recovers", which was about 3 minutes at 2k "machines". I've also done some testing for "if change the charm so that it tries to change its relation data as fast as it possibly can, how many can you run at the same time", and there I think I got to about 500 or so before the database bottlenecked. I started a bit of scale testing in an HA scenario, and initial results showed the performance was going to improve (not linearly, but better than a single machine), but at the time I was testing it we ran into some other bugs that have since been fixed. It certainly is the sort of thing that we should be doing on a regular basis (every 3 months?) I haven't done it since back in April. I have started putting together some scripts to make it a little bit easier to spin them up. (It currently takes about 20 minutes to spin up 15 real machines, and add 1000 units onto those machines, mostly because we don't allow you to specify "-n" with "--to".) The latest bottleneck that I'm aware of is that we changed our logging infrastructure so every agent tries to connect directly to rsyslogd to send its log messages, and rsyslog doesn't really like 10,000 TLS connections. (By default rsyslog restricts to 200 concurrent connections, bumping that up to 1000, and I was maxing 1 CPU for rsyslogd, and I was still getting some connections rejected.) John =:-> On Thu, Oct 2, 2014 at 6:13 PM, Mark Ramm-Christensen (Canonical.com) < [email protected]> wrote: > However we did do similar testing on Go juju a while back. > > That said we need to publish scale testing numbers again. Something to > talk about next week! > > On Thu, Oct 2, 2014 at 10:03 AM, Kapil Thangavelu < > [email protected]> wrote: > >> Unfortunately that's not very representative of the current >> implementation as it was based on pyjuju while the current implementation >> is in go and utilizing mongodb instead of zookeeper. >> >> -kapil >> >> On Thu, Oct 2, 2014 at 9:40 AM, Charles Butler < >> [email protected]> wrote: >> >>> There's this article which was published a while ago: >>> >>> >>> https://maas.ubuntu.com/2012/06/04/scaling-a-2000-node-hadoop-cluster-on-ec2ubuntu-with-juju/ >>> >>> Hope this helps, >>> >>> Charles >>> >>> On Thu, Oct 2, 2014 at 9:02 AM, Mike Sam <[email protected]> wrote: >>> >>>> I was wondering what is the largest vm count that has been provisioned >>>> and deployed with juju in testing so far? In other words, what is the >>>> demonstrated scale that juju has proven to handle well so far? >>>> >>>> Thanks, >>>> Mike >>>> >>>> -- >>>> Juju-dev mailing list >>>> [email protected] >>>> Modify settings or unsubscribe at: >>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev >>>> >>>> >>> >>> -- >>> Juju-dev mailing list >>> [email protected] >>> Modify settings or unsubscribe at: >>> https://lists.ubuntu.com/mailman/listinfo/juju-dev >>> >>> >> >> -- >> Juju-dev mailing list >> [email protected] >> Modify settings or unsubscribe at: >> https://lists.ubuntu.com/mailman/listinfo/juju-dev >> >> > > -- > Juju-dev mailing list > [email protected] > Modify settings or unsubscribe at: > https://lists.ubuntu.com/mailman/listinfo/juju-dev > >
-- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
