Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-14 Thread Azoff, Justin S
> On Sep 13, 2018, at 5:57 PM, John Althouse wrote: > > Could anyone update Bro on docker to include 2.5.5, 2.6-beta and master? > https://hub.docker.com/r/broplatform/bro/tags/ > > We use this with our internal trybro instance which is fantastic for quickly > collaborating and testing script

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-13 Thread John Althouse
Could anyone update Bro on docker to include 2.5.5, 2.6-beta and master? https://hub.docker.com/r/broplatform/bro/tags/ We use this with our internal trybro instance which is fantastic for quickly collaborating and testing scripts. :) On Wed, Sep 12, 2018 at 3:26 PM, Jon Siwek wrote: > On Wed,

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-12 Thread Jon Siwek
On Wed, Sep 12, 2018 at 11:58 AM Alan Commike wrote: > Are there still parent/child processes handling comms/work? No. Single process, configurable number of threads (default 1). > Is there a mechanism today for per node type tuneables? One should be able to use an @if directive [1] to tune d

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-12 Thread Alan Commike
On Wed, Sep 12, 2018 at 8:21 AM Jon Siwek wrote: > > An idea in this type of situation could be to tune Broker::max_threads > per node type. E.g. leave at 1 for workers and bump to ~4 for > manager/logger since there's idle cores on their host and they're > inherently in a less-scalable/centrali

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-12 Thread Jon Siwek
On Wed, Sep 12, 2018 at 9:18 AM Azoff, Justin S wrote: > > Just finished the migration to master across the board, and it's looking > REALLY good. Great, thanks for helping test and provide performance data. > The manager box in this cluster only runs the manager and logger processes, > no pro

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-12 Thread Azoff, Justin S
On Sep 5, 2018, at 6:35 PM, Jon Siwek mailto:jsi...@corelight.com>> wrote: There's no significant code changes/features planned to get added to the master branch from now until the 2.6-beta gets released (maybe in about a week). Until that happens, please help test the latest master branch and

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-10 Thread Azoff, Justin S
> On Sep 7, 2018, at 4:41 PM, Azoff, Justin S wrote: > > Before, cpu maxed out but spending 60% in user and 30% in system > After, cpu maxed out but spending 12% in user and 80% in system > I did some more testing and profiling and figured out what is going on.. The new version is much more e

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-08 Thread Azoff, Justin S
> On Sep 8, 2018, at 11:20 AM, Azoff, Justin S wrote: > > Do many of those options do anything? I tried looking in the CAF source to > figure out how they are used, and it looks like they are all defined in > libcaf_core/caf/actor_system_config.hpp as Scratch that.. the deprecation refers to

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-08 Thread Azoff, Justin S
> On Sep 7, 2018, at 6:58 PM, Jon Siwek wrote: > > On Fri, Sep 7, 2018 at 3:41 PM Azoff, Justin S wrote: > >> One thing I'm still seeing when I switch from an old version to latest >> master is that huge spike >> in Content switches/interrupts and cpu spent in the kernel. > > I just updated

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-07 Thread Jon Siwek
On Fri, Sep 7, 2018 at 3:41 PM Azoff, Justin S wrote: > One thing I'm still seeing when I switch from an old version to latest master > is that huge spike > in Content switches/interrupts and cpu spent in the kernel. I just updated the default tuning parameters for CAF's scheduling policy and e

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-07 Thread Azoff, Justin S
> On Sep 6, 2018, at 7:40 PM, Jon Siwek wrote: > > On Thu, Sep 6, 2018 at 3:40 PM Azoff, Justin S wrote: > >> 8842 >> broker::topic+broker::internal_comma...@u32.bro/known/certs/<$>/data/clone > > Thanks, there was an unintended forwarding loop in data store > communication. It's fixed in

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-06 Thread Jon Siwek
On Thu, Sep 6, 2018 at 3:40 PM Azoff, Justin S wrote: >8842 > broker::topic+broker::internal_comma...@u32.bro/known/certs/<$>/data/clone Thanks, there was an unintended forwarding loop in data store communication. It's fixed in master now, but I've also just reverted to generally disabling

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-06 Thread Azoff, Justin S
> On Sep 6, 2018, at 4:19 PM, Jon Siwek wrote: > > On Thu, Sep 6, 2018 at 3:14 PM Azoff, Justin S wrote: > > >> I tested an almost stock local.bro (a few additional things disabled) and >> saw the same thing. >> >> fa7fa5aa is fine, but with 452eb0cb everything is working really hard to do

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-06 Thread Jon Siwek
On Thu, Sep 6, 2018 at 3:14 PM Azoff, Justin S wrote: > I tested an almost stock local.bro (a few additional things disabled) and saw > the same thing. > > fa7fa5aa is fine, but with 452eb0cb everything is working really hard to do > something. Thanks for that, I'll start looking into it, but

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-06 Thread Jon Siwek
On Thu, Sep 6, 2018 at 2:47 PM Azoff, Justin S wrote: > I just got 2 clusters upgraded from > > fa7fa5aa to > 452eb0cb > > And now everything is broken.. > > cpu and memory are through the roof across the board, as well as network > traffic, but it's not logging much. > > I may have created a me

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-06 Thread Azoff, Justin S
> On Sep 6, 2018, at 3:41 PM, Azoff, Justin S wrote: > > I just got 2 clusters upgraded from > > fa7fa5aa to > 452eb0cb > > > I may have created a message loop replacing the relay_rr stuff, but it's kind > of hard to tell. > > I'll do some more testing but so far this is the first issue I'

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-06 Thread Azoff, Justin S
> On Sep 5, 2018, at 6:35 PM, Jon Siwek wrote: > > There's no significant code changes/features planned to get added to > the master branch from now until the 2.6-beta gets released (maybe in > about a week). Until that happens, please help test the latest master > branch and provide any feedba

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-06 Thread Jon Siwek
On Wed, Sep 5, 2018 at 5:43 PM Michael Dopheide wrote: > > To be clear, Cluster::relay_rr() is gone forever? I’ll need to rewrite some > policies, but also update the blog to be correct. Yes. - Jon ___ bro-dev mailing list bro-dev@bro.org http://mai

Re: [Bro-Dev] Bro 2.6-beta plans

2018-09-05 Thread Michael Dopheide
To be clear, Cluster::relay_rr() is gone forever? I’ll need to rewrite some policies, but also update the blog to be correct. Dop On Wed, Sep 5, 2018 at 5:37 PM Jon Siwek wrote: > There's no significant code changes/features planned to get added to > the master branch from now until the 2.6-be

[Bro-Dev] Bro 2.6-beta plans

2018-09-05 Thread Jon Siwek
There's no significant code changes/features planned to get added to the master branch from now until the 2.6-beta gets released (maybe in about a week). Until that happens, please help test the latest master branch and provide any feedback about how it's working if you can. - Jon ___