Re: [Bro-Dev] Module prefix in sending and receiving Broker events

2018-09-27 Thread Matthias Vallentin
Jan: > Event handlers just don't seem to inherit the surrounding module > namespace, which kind of makes sense if you want to handle events > generated in the global namespace. I agree that it does make sense in that regard. It's certainly prudent - as mentioned - to always qualify your events. Th

Re: [Bro-Dev] Module prefix in sending and receiving Broker events

2018-09-27 Thread Matthias Vallentin
and the Bro script "foo". So far so good. If I remove the event declaration, the Python script prints [, , ['foo', []]] and the Bro script nothing. I hope this illustrates the issue a bit better. Matthias -- Matthias Vallentin CEO Tenzir GmbH c/o Regus Valent

Re: [Bro-Dev] Module prefix in sending and receiving Broker events

2018-09-13 Thread Matthias Vallentin
> Does the > suggestion [1] to always explicitly scope events by their > namespace/module address your problem? That's what I thought would work, but it's the opposite: when I add the module name as prefix, Bro silently ignores the event. I found a surprising solution though: if I declare the eve

[Bro-Dev] Module prefix in sending and receiving Broker events

2018-09-12 Thread Matthias Vallentin
When I receive events from Bro via Broker, they have the prefix of the enclosing module: module Foo; event foo() { ... } event bar() { ... } When I publish "foo" via Broker, it arrives as "Foo::foo". However, when I publish an event "Foo::bar" from Broker, Bro doesn't recognize it.

Re: [Bro-Dev] switch/case type recognition

2018-09-12 Thread Matthias Vallentin
> A 'vector of any' also qualifies as an 'any', so while the error > message of "duplicate case label" could possibly be improved, I think > it's still correct in that it is meant to prevent an ambiguous case > match. Yeah, with that reasoning anything qualifies as "any", so adding "any" should ne

Re: [Bro-Dev] How to use Broker::Data in an event handler?

2018-09-12 Thread Matthias Vallentin
> case type Broker::Data as d: > print "Broker::Data, expected to be nil", d?$data, d?$data ? > cat(d$data) : "nil"; > # or use the same logic from the is_nil() function above This is the logic I was looking for, thanks! The generic is_nil function might come in handy some othe

Re: [Bro-Dev] How to use Broker::Data in an event handler?

2018-09-11 Thread Matthias Vallentin
> This, plus a couple other bugs should now be fixed in bro + broker, so make sure to update both if trying the above examples. Great, it works now! One more question: how would I capture a default-constructed broker::Data() in a case statement? This would happen when I publish just "None" on the

[Bro-Dev] switch/case type recognition

2018-09-11 Thread Matthias Vallentin
Consider this snippet: function f(x: any) { switch (x) { case type any: print "any"; break; case type vector of any: print "vector of any"; break; } } event bro_init() { f(42); } I

[Bro-Dev] How to use Broker::Data in an event handler?

2018-09-10 Thread Matthias Vallentin
I'm trying to figure out if/how it is possible to use Broker::Data in an event handler as follows: event foo(x: Broker::Data) { print x; } I'm trying to send an event via the Python bindings: event = broker.bro.Event("foo", broker.Data(42)) endpoint.publish("/test"

Re: [Bro-Dev] Broker data layouts

2018-08-25 Thread Matthias Vallentin
> Agree. Right now a newly connecting peer gets a round of explicit > LogCreates, but that's probably not the best way forward for larger > topologies. Okay. In the future, we probably need some form of "serialization-free" batching mechanism to ship data more efficiently. There exist technologies

Re: [Bro-Dev] Broker data layouts

2018-08-24 Thread Matthias Vallentin
> I don't really see a way around that without substantially increasing > volume. We could send LogCreate updates regularly, so that it's easier > to synchronize with an ongoing stream. It sounds like this is critical also for regular operation: (1) when an endpoint bootstraps slowly and the LogCr

Re: [Bro-Dev] 'async' update and proposal

2018-03-13 Thread Matthias Vallentin
>I think we should instead just skip the "async" keyword altogether. >Requiring it at some places, but not others, hurts more than it helps >in my opinion. This sounds fine to me. Given that Bro is inherently an asynchronous language, it makes sense to for those semantics to trickle down to the

Re: [Bro-Dev] can I send an opaque of bloomfilter over Cluster::manager2worker_event ?

2017-05-01 Thread Matthias Vallentin
> I figured merging same bloom to itself won't make a difference, I > primarily want to copy it to blacklistbloom. That's correct behavioar. A bloomfilter merge is idempotent when executed in reflexive fashion (because it boils down to a bitwise OR of a bitvector with itself). Matthias _

Re: [Bro-Dev] early performance comparisons of CAF-based run loop

2017-04-19 Thread Matthias Vallentin
> Oh, sounds like that would be high-priority task then before we'd > consider moving to a CAF-based loop? I've added kqueue support in topic/kqueue, but it's still missing the final touch. (Unit tests are still failing.) Hopefully it's not a long way from here. Matthias _

Re: [Bro-Dev] early performance comparisons of CAF-based run loop

2017-04-14 Thread Matthias Vallentin
> If we can confirm that on at least Linux and FreeBSD for, say, the two > most recent major releases of each and also consider common > alternative capturing solutions (pfring, netmap, afnet?), I'd be > pretty comfortable switching. Just a quick comment here regarding FreeBSD: the native polling

Re: [Bro-Dev] Scaling out bro cluster communication

2017-02-10 Thread Matthias Vallentin
> What I want to make possible is client side load balancing and > failover for worker -> manager/datanode communication. This is an important part of future Bro deployments. Before delving into script code, I would like to get a better understanding of the underlying concepts and communication

Re: [Bro-Dev] Broker's remote logging (BIT-1784)

2017-01-31 Thread Matthias Vallentin
> I'm wondering if there's a reason that in the Broker case things > *have* to be this way. Is there something that prevents the Broker > manager from doing the same as the RemoteSerializer? Some background: when Broker sends to a log topic, the message has the structure of a pair (id, (x, y, z, .

Re: [Bro-Dev] Rethinking Broker's blocking API

2017-01-10 Thread Matthias Vallentin
> Could we then now also lift the error() method into the message class? > So "if (msg.error())" would be a shortcut for > "if(msg.status().error()? (And then we'd back be where we started I > believe. :-) Done. And yes, we've done the full circle. ;-) But at least we're in full agreement now. S

Re: [Bro-Dev] Rethinking Broker's blocking API

2017-01-10 Thread Matthias Vallentin
> I see the challenge but I think we need some way to differentiate > serious errors from expected updates, otherwise we're back at writing > switch statements that need to comprehensively list all cases. I agree that writing switch statements is not very productive. From a user perspective, it's

Re: [Bro-Dev] Rethinking Broker's blocking API

2017-01-10 Thread Matthias Vallentin
> Sorry, that was misleading. Statuses don't provide an operator bool. > They could, however, to distinguish them from errors. Mulling over this more, I'm not sure if it's clear which status codes constitute an error. For example, there's a peer_lost and peer_recovered status code. Is only the fir

Re: [Bro-Dev] dynamic-cast branch (Re: [Bro-Commits] [git/bro] topic/robin/dynamic-cast: Add experimental "is" and "as" operators. (dabe125))

2017-01-10 Thread Matthias Vallentin
> So not sure what the right solution is but for now: either upgrade > bison, or remove the line and keep an eye on if things work correctly. Upgrading Bison worked just fine, thanks. Matthias ___ bro-dev mailing list bro-dev@bro.org http://mailman.

Re: [Bro-Dev] Rethinking Broker's blocking API

2017-01-10 Thread Matthias Vallentin
> > To distinguish between the two status, I use operator bool. > > I don't see that in the "status and error handling" section. Would I > do "if (! status) { }"? That doesn't seem quite > intuitive. I think a method with a descriptive name would be better > here. Sorry, that was misleading. Sta

Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/dynamic-cast: Add experimental "is" and "as" operators. (dabe125)

2017-01-09 Thread Matthias Vallentin
> On branch : topic/robin/dynamic-cast > Link : > https://github.com/bro/bro/commit/dabe125fe8fab80ea1f678844b872b369764fd80 I've tried branching away from topic/robin/dynamic-cast for Broker integration, but get a compile error in parse.y: [ 86%] [BISON][Parser] Building parser with

Re: [Bro-Dev] Rethinking Broker's blocking API

2017-01-09 Thread Matthias Vallentin
> But not a big deal either way, any of these options sounds fine to me. This is the synopsis for the slightly adapted message class, no other changes: class message { public: /// Checks whether a message is a (topic, data) pair. /// @returns `true` iff the message contains a

Re: [Bro-Dev] Rethinking Broker's blocking API

2017-01-06 Thread Matthias Vallentin
> I think the name "error" is not just misleading but would also turn > out tricky to use correctly. Agreed. > auto msg = ep.receive(); > > if (msg) > return f(*msg); // unbox contained message > > if (msg.failed()) > cout << "Trouble: " << to_string(msg.status()) <

Re: [Bro-Dev] Rethinking Broker's blocking API

2017-01-05 Thread Matthias Vallentin
> Nice summary of the challenge! I agree that none of the options you > list sound really appealing. Here's an alternative idea: could we > change your option 1 (the variant) into always returning *both*, i.e., > tuple? You pushed me into a new direction. Broker already returns expected for opera

Re: [Bro-Dev] [Bro-Commits] [git/bro] topic/robin/dynamic-cast: Add experimental "is" and "as" operators. (dabe125)

2017-01-04 Thread Matthias Vallentin
> function check(a: any) > { > local s: string = "default"; > > if ( a is string ) > s = (a as string); Are the parenthesis around the expression required? Intuitively, operator "as" should have higher precedence. Matthias _

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2017-01-03 Thread Matthias Vallentin
> Yeah, I agree, don't like that version anymore either. Ok, good. :-) > case type count as c: > > > > What do you think of that? The additional "type" makes it visually > clear what's it's about, and also helps the parser figure it out. I find that there's too much going on

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2017-01-03 Thread Matthias Vallentin
> > [result, value] = Broker::lookup(h, 42) # Returns [Broker::Result, > > opaque of Broker::Data] > > > > if ( result == Broker::SUCCESS ) ... > > I would prefer this solution, as it feels more natural coming from other > languages like python. I like it as well because there's no ca

[Bro-Dev] Rethinking Broker's blocking API

2017-01-02 Thread Matthias Vallentin
Broker's current API to receive messages is as follows: context ctx; auto ep = ctx.spawn(); ep.receive([&](const topic& t, const data& x) { .. }); ep.receive([&](const status& s) { .. }); or the last two in one call: ep.receive( [&](const topic& t, const data& x) { .. }

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2017-01-02 Thread Matthias Vallentin
On Wed, Dec 14, 2016 at 04:17:26PM +, Siwek, Jon wrote: > > > On Dec 13, 2016, at 11:42 AM, Matthias Vallentin wrote: > > > >>> local r = put(store, key, test(lookup(store, key))); > > > > It's up to the user to check the result variable (here

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2017-01-02 Thread Matthias Vallentin
> Alternatively, we could leave it to frameworks to define their own > error types. So for Broker, we'd have Broker::NotFound, > Broker::Timeout, etc. And the opaque types would define internally > what they convert to, and how. It looks like this is the model you went with in the revised proposal

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2016-12-13 Thread Matthias Vallentin
> I don't really like using a record like that, as that would associate > specific semantics with what's really a user-definable type. It was only meant to illustrate the idea of error handling and function composition. These ideas still hold up when substituting the user-defined result type with

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2016-12-13 Thread Matthias Vallentin
> > local r = put(store, key, test(lookup(store, key))); > > For prototyping purposes, I see the convenience in that, but wonder if > the runtime will do something that’s useful and widely applicable > enough for that to translate directly into production code. What > exactly does the runtime do

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2016-12-13 Thread Matthias Vallentin
> I might prefer just doing the unpacking myself. Having > separate/explicit checks for each individual return value would > naturally occur over time anyway (e.g. when debugging, adding extra > logging, improving error handling). > How common do you expect async function chains to occur? Any spe

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2016-12-11 Thread Matthias Vallentin
> Personally, I see this more as a question of readability (as opposed > to typeability :). But it's a matter of taste, and I'd be fine with > using "as" instead of "cast<>". Probably aligned with that thought is consistency and intuition: we don't have C++-style angle brackets in Bro, so "as" fee

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2016-12-06 Thread Matthias Vallentin
> > Alternative to even providing “type(v)”, you could have a “v is T” > > I like that. That avoids the question of storing types altogether. I think we can cover all type-based dispatching with "is" and "switch." In fact, I see "x is T" as syntactic sugar for: function is(x: any) { switch

Re: [Bro-Dev] [Proposal] Language extensions for better Broker support

2016-12-02 Thread Matthias Vallentin
> Feedback welcome, this is just a first draft. I like this. Some initial feedback: - In the switch statement, I would require that a user either provide a default case or fully enumerates all cases. Otherwise it's too easy to cause harder-to-debug run-time errors. - For the abbrevia

Re: [Bro-Dev] bro-pkg -> bropkg

2016-09-16 Thread Matthias Vallentin
> I like Jon's idea to create a symlink for bro-pkg. One could also > install links for brocut and bro-ctl. PREFIX/bin/bro PREFIX/bin/bro-ctl PREFIX/bin/broctl PREFIX/bin/bro-cut PREFIX/bin/brocut PREFIX/bin/bro-pkg PREFIX/bin/bropkg .. That looks too confusing to

Re: [Bro-Dev] bro-pkg -> bropkg

2016-09-16 Thread Matthias Vallentin
> Bropkg is easier to type indeed. And then let's change the brocut. Yeah, all bro* utils should be consistent. If we change bro-pkg, then bro-cut must undergo the same changes. Personally, I never had trouble typing "bro-pkg" though. Can we do a Twitter poll to figure out what our users prefer?

Re: [Bro-Dev] package manager progress

2016-07-28 Thread Matthias Vallentin
> - Would suggest to rename “pkg.meta” to, say, “bro-pkg.meta”, just to > make it more explicit that it's a Bro package. That's something one > can also then search for on GitHub. Just throwing in two more permutations: bro.meta or bro.pkg. > - For our default package source, do we want to su

[Bro-Dev] New documentation via Sphinx

2016-07-27 Thread Matthias Vallentin
I'm in the process of documenting Broker with Sphinx. With minimal effort, I put up a scaffold that looks like this: http://bro.github.io/broker/ It's the bootstrap theme for sphinx, as an alternative to the classic read-the-docs theme. I've hacked the sidebar so that it shows the table of co

Re: [Bro-Dev] package manager progress

2016-07-27 Thread Matthias Vallentin
> I actually don't like this that much because some of these can cross > boundaries and do all sorts of different things in a single plugin. > It makes more sense to me to leave the naming open. I'm with Seth on this one. The reason why I think we should keep the naming open is that it's the job

Re: [Bro-Dev] package manager progress

2016-07-26 Thread Matthias Vallentin
> > I'm not 100% following. Isn't every package recorded as submodule? > > Every package within a package source is recorded as a git submodule > and that recording happens at the time the package author registers > their package with a source. The bro-pkg client itself makes no > changes to subm

Re: [Bro-Dev] package manager progress

2016-07-25 Thread Matthias Vallentin
> Right now, packages don’t get downloaded via the submodule, they are > cloned directly from the package’s full git URL (which git just > happens to encoded within the submodule). > > So this means only packages a user is interested in end up getting > downloaded. I'm not 100% following. Isn't

Re: [Bro-Dev] Broker data store API

2016-07-25 Thread Matthias Vallentin
> I can't speak to whether or not it is 'needed', but I have had desire > to use it in the past... The only thing preventing me from doing it > was the fact that Broker is currently a fast moving target. Good to know. Scott Campbell also uses the current version of Broker in his projects and menti

Re: [Bro-Dev] package manager progress

2016-07-25 Thread Matthias Vallentin
> The package manager client is at a point now where I think it would be > usable. Cool! > * Add a way for package’s to define “discoverability metadata”. > > E.g. following the original plan for this would involve putting > something like a “tags” field in each package’s pkg.meta file, but t

Re: [Bro-Dev] Broker data store API

2016-07-24 Thread Matthias Vallentin
> Not sure about the choice of RocksDB in particular — could have just > been that it happened to pop up on people’s radar at the right time. It's certainly an industrial-strength key-value, so I think it's solid choice for those with better performance when needing persistence. > Given those his

[Bro-Dev] Broker data store API

2016-07-22 Thread Matthias Vallentin
TL;DR: - Does anyone use Broker's RocksDB backend? - Brief overview of the revamped data store frontend API I've been working on the Broker data store API a bit, trying to come with the smallest denominator possible for an initial release. So far I have ported the in-memory SQLite backend ove

Re: [Bro-Dev] Some thoughts on the bro deep cluster, broker, and sumstats

2016-07-14 Thread Matthias Vallentin
> *tl;dr* Just a quick heads-up: thanks a bunch for summarizing your thoughts! We haven't forgotten your mail and will get back after we're done with our crunch with releasing Bro 2.5. Stay tuned, Matthias ___ bro-dev mailing list bro-dev@bro.org h

[Bro-Dev] [Broker] multi-topic subscriptions

2016-06-28 Thread Matthias Vallentin
If a broker endpoint subscribes to multiple topics, how many messages do you expect to receive? Consider this snippet: context ctx; auto e = ctx.spawn(); e.subscribe("/foo"); e.subscribe("/foo/bar"); e.subscribe("/foo/bar/baz"); e.publish("/foo/bar/baz", 4.2); Should the endpoint rece

Re: [Bro-Dev] New proposal (Re: CBAN naming)

2016-06-09 Thread Matthias Vallentin
> I like the “packages” + “package-manager” combo that Johanna suggests. +1 Matthias ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listinfo/bro-dev

Re: [Bro-Dev] New proposal (Re: CBAN naming)

2016-06-09 Thread Matthias Vallentin
> I see benefits in two separate repos: Yep. > client: bro-pkg > community packages: bro-pkg-community I'm not sure if I understand the -community suffix. The client bro-pkg makes sense to me. But the first association I have with bro-pkg-community is a community-version of bro-pkg (

Re: [Bro-Dev] New proposal (Re: CBAN naming)

2016-06-09 Thread Matthias Vallentin
> > Do you equate one package with one container, or does the plural > > "packages" signify something more collection-ish? > > I see them as one to one. Okay, that's what I was thinking as well. Matthias ___ bro-dev mailing list bro-dev@bro.org htt

Re: [Bro-Dev] New proposal (Re: CBAN naming)

2016-06-08 Thread Matthias Vallentin
> project name: Bro Package Manager > client name: bro-pkg > git repo name: bro-pkg > design spec/proposal: use Robin's/Johanna’s > top-level container name: packages > changes to existing naming conventions: none Looks good to me. One question though: what is the top-level container name? Do yo

Re: [Bro-Dev] CBAN naming

2016-06-06 Thread Matthias Vallentin
> A scenario that I see as being quite likely is that a developer starts such > a container as being script-only, but wants to expand it at some future > date. Say there's some new botnet with a domain-generation algorithm that > they would like Bro to predict and alert on. The script is functional

Re: [Bro-Dev] CBAN naming

2016-06-02 Thread Matthias Vallentin
> So with that said, I propose bro-pkg, but will leave this open for > another day if there are strong opinions. I like "bro-pkg," though I wonder whether it could be even shortened to "bpkg" or "bkg." This would be the name for the command line client. How would we call the whole thing? The Bro P

Re: [Bro-Dev] CBAN naming

2016-06-01 Thread Matthias Vallentin
> I like all these suggestions from Adam - and as a person who enjoys making > electronic music would humbly add 'bpm' (Bro Package Manager) to the list > ;-) We had a good discussion in the Bro Gitter channel about this yesterday. In fact, I suggested "bpm," but as it turns out, there exists alre

Re: [Bro-Dev] CBAN naming

2016-05-31 Thread Matthias Vallentin
> - boxer, boxes (sort of already taken by Broala's BroBox) > - bundler, bundles (though seems like Ruby has taken this name) > - bagger/bagboy, bags (also has the association w/ eye bags) > - tempest, drops (eye of the storm, rain drops, eye drops... the tears of > those trapped i

Re: [Bro-Dev] CBAN naming

2016-05-30 Thread Matthias Vallentin
> By the way: Are we talking about renaming the whole project? I don't know :-). We need a command line client and a project name. When they are the same, it's certainly easier to remember. My favorite name so far is also "brow." The eye brow of the Bro logo makes for a nice association. It didn

Re: [Bro-Dev] CBAN naming

2016-05-30 Thread Matthias Vallentin
> Besides naming and skipping ahead to implementation, I highly recommend > looking at how Rust manages crates. They are a pleasure to use and work > with. Rust's cargo [1] is indeed well thought through (I like the dependency specification especially [2]) and we should look at it in depth during

Re: [Bro-Dev] CBAN naming

2016-05-27 Thread Matthias Vallentin
> I like how Spicy was named, by choosing something completely different and > unrelated to the "bro" theme. I like that too. Do you have a suggestion? Matthias ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.berkeley.edu/mailman/listin

Re: [Bro-Dev] Broker status update

2016-05-27 Thread Matthias Vallentin
> > peer(x, y); // Create a peering between the two endpoints. > > peer(y, x); // Idempotent. Peerings are symmetric. > > > x.peer(y); // Create a peering between the two endpoints. > > y.peer(x); // Idempotent. Peerings are symmetric. > > I would prefer the 2nd wa

[Bro-Dev] CBAN naming

2016-05-27 Thread Matthias Vallentin
To find the new name for our CBAN project, it probably make sense to brainstorm separately from the existing technical thread. I'd say let's collect some candidates and then create survey to vote on them. Here are some ideas from the existing thread: - brow - broil - broom - bpk

Re: [Bro-Dev] CBAN design proposal

2016-05-26 Thread Matthias Vallentin
> - having both "upgrade" vs "update" commands sounds confusing, I'm > sure I would constantly mix up the two. Suggest to rename one of > them. I think this comes from Homebrew (and maybe other package managers as well). I kinda got used to it in this context, but occasionally still trip over

Re: [Bro-Dev] CBAN design proposal

2016-05-24 Thread Matthias Vallentin
(I will respond to the actual proposal in more depth later.) > That is a good point. I am more concerned about accumulating clutter. If I understold it correctly, I don't think that the central CBAN repository will accumulate clutter. The automatic checks will help simply age out repos that do no

[Bro-Dev] Broker status update

2016-05-21 Thread Matthias Vallentin
This is just a brief summary of where the Broker overhaul is currently at. The new Broker API will come two types of endpoints: blocking and nonblocking. The former provides a synchronous API to receive messages, whereas the latter operates fully asynchronous. Performance-wise, the asynchronous ver

Re: [Bro-Dev] Broker C++ namespace: broker -> bro?

2016-05-12 Thread Matthias Vallentin
> Not sure I like that. The library is called Broker, so "broker" seems > the natural namespace to me. All right, let's keep it as is. This is also aligns with the sentiment at our Gitter room. > I think the better solution would be finding a different name for > those things now called "broker"

[Bro-Dev] Broker C++ namespace: broker -> bro?

2016-05-11 Thread Matthias Vallentin
I'm considering renaming the C++ broker namespace from "broker" to "bro". After all, it's *Bro's* communication library. This would reduce noise in the source code (3 less characters for those who cannot or don't want to import the entire namespace) and also reduce ambiguity (we can now use bro::br

Re: [Bro-Dev] bloomfilter_counting_init parameterization ?

2016-05-03 Thread Matthias Vallentin
> global bloomfilter_counting_init: function(k: count , cells: count , > max: count , name: string &default=""): opaque of bloomfilter ; The counting Bloomfilter is very similar to a regular Bloom filter, except that the underlying bit vector now consists of "cells," i.e., sequences of bits that

[Bro-Dev] [JIRA] (BIT-1554) broker (bro 2.4.1) fails to build against Python 3.{3, 4, 5}

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1554: --- Assignee: Matthias Vallentin > broker (bro 2.4.1) fails to build against Pytho

[Bro-Dev] [JIRA] (BIT-1522) Broker listener takes a long time to shut down on cluster stop/restart

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1522: --- Assignee: Matthias Vallentin > Broker listener takes a long time to shut down

[Bro-Dev] [JIRA] (BIT-1554) broker (bro 2.4.1) fails to build against Python 3.{3, 4, 5}

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25709#comment-25709 ] Matthias Vallentin commented on BIT-1554: - Fixing this "soon" would pr

[Bro-Dev] [JIRA] (BIT-1556) bro-2.4.1 fails to compile broker with -march=i686

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1556: --- Assignee: Matthias Vallentin > bro-2.4.1 fails to compile broker with -march=i

[Bro-Dev] [JIRA] (BIT-1450) Improve Python API

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1450: --- Assignee: Matthias Vallentin > Improve Python

[Bro-Dev] [JIRA] (BIT-1555) aux/broker/bindings/python/CMakeLists.txt doesn't respect -DINSTALL_LIB_DIR

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1555: --- Assignee: Matthias Vallentin > aux/broker/bindings/python/CMakeLists.txt does

[Bro-Dev] [JIRA] (BIT-1447) Can't abort blocking Broker Python functions

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1447: --- Assignee: Matthias Vallentin > Can't abort blocking Broker Python f

[Bro-Dev] [JIRA] (BIT-1445) Broker crash when two stores go to the same SQLite DB

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1445: --- Assignee: Matthias Vallentin > Broker crash when two stores go to the same SQL

[Bro-Dev] [JIRA] (BIT-1388) Broker's integration in Bro's main/run loop

2016-04-18 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthias Vallentin reassigned BIT-1388: --- Assignee: Matthias Vallentin > Broker's integration in Bro's

[Bro-Dev] [JIRA] (BIT-1506) Bro fails to build on OS X 10.11 (El Capitan) due to OpenSSL header removal

2016-04-11 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25701#comment-25701 ] Matthias Vallentin commented on BIT-1506: - Right, ideally we avoid a new rel

[Bro-Dev] [JIRA] (BIT-1506) Bro fails to build on OS X 10.11 (El Capitan) due to OpenSSL header removal

2016-04-10 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25604#comment-25604 ] Matthias Vallentin commented on BIT-1506: - This looks like the right way to search

Re: [Bro-Dev] Broker: use of broker::peering

2016-04-04 Thread Matthias Vallentin
> I think you’re safe changing it as long as a user continues to have a > way to disconnect an endpoint from any one of its peers. Yep, that makes sense. Thanks for chiming in. Matthias ___ bro-dev mailing list bro-dev@bro.org http://mailman.icsi.be

[Bro-Dev] Broker: use of broker::peering

2016-03-31 Thread Matthias Vallentin
In Broker, what is the use case for having an explicit peering between two endpoints? Would it maybe suffice to provide endpoint introspection, i.e., the ability to iterate over an endpoint's peers? At least Bro doesn't use broker::endpoint for anything but that, but I was wondering if I am missing

Re: [Bro-Dev] Broker & CAF includes

2016-03-25 Thread Matthias Vallentin
> Providing stable-ish ABIs seems like something libraries often do, so > I tried to plan that in to Broker. Don’t know if I did that well, or > there’s better strategies to use, or I was the only one worried about > that to begin with, but thought I’d mention it just in case it wasn’t > even on y

[Bro-Dev] [JIRA] (BIT-1558) Bro's ascii formatter writing out scientific notation

2016-03-22 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=25102#comment-25102 ] Matthias Vallentin commented on BIT-1558: - Looks like a missing `std::fixed` in an

Re: [Bro-Dev] Broker & CAF includes

2016-03-21 Thread Matthias Vallentin
Thanks for chiming in, Jon. > [..] i.e. treat the use of CAF an implementation detail. This is the clean way to think about layering and creating abstractions. It applies to the API perspective, though. As long as CAF internals are hidden from a Broker user, we are good. The "implementation deta

Re: [Bro-Dev] Coding style enforcement

2016-03-19 Thread Matthias Vallentin
> I’m fine w/ any style or naming convention changes in order to cause > less friction for Matthias/others. Good to know Jon, thanks for chiming in. My goal is to leverage clang-format to the best degree possible such switching styles is not a big undertaking. Since no tool has (yet) good support

[Bro-Dev] Broker & CAF includes

2016-03-18 Thread Matthias Vallentin
During Broker refactoring, I noticed the following: all headers in broker/* include either standard library headers or Broker headers. This appears to be by design, which makes sense to me. As a library writer, one faces the tricky question of exposing headers from dependencies. For example, Brok

[Bro-Dev] Coding style enforcement

2016-03-11 Thread Matthias Vallentin
While porting Broker to the latest CAF version, I am realizing that the current pre C++11 coding style is not very conducive. Since the introduction of lambdas, and in particular with CAF's asynchronous and template-heavy programming model, the Whitesmiths style isn't very practical. Once can cons

Re: [Bro-Dev] Fw: Broker raw throughput

2016-03-11 Thread Matthias Vallentin
> To me, the performance numbers themselves don't matter as much as > managing expectations does: should I *expect* to be able to pass all > of my events through broker? This question depends on the event type and your concrete topology, and is hard to answer in general. We can say "in our point-t

Re: [Bro-Dev] Broker raw throughput

2016-03-09 Thread Matthias Vallentin
> The optimization I meant is to not wrap each integer in its own > message object, but rather make one message which then contains X > integers that are transparently interpreted by the receiver as X > messages. But this requires some form of "output queue" or lookahead > mechanism. I can see tha

Re: [Bro-Dev] Broker raw throughput

2016-03-08 Thread Matthias Vallentin
> You could additionally try to tweak > caf#middleman.max_consecutive_reads, which configures how many > new_data_msg messages a broker receives from the backend in a single > shot. It makes sense to have the two separated, because one configures > fairness in the scheduling and the other fairness

Re: [Bro-Dev] Broker raw throughput

2016-03-07 Thread Matthias Vallentin
> I have created a ticket for further progress tracking / discussion [1] > as this is clearly not a Bro/Broker problem. Thank you all for > reporting this and all the input you have provided. It's good to see the new commit improves performance. But I want to take again the perspective of Broker,

Re: [Bro-Dev] Broker raw throughput

2016-03-02 Thread Matthias Vallentin
> @Matthias: FYI, I have used a new feature in CAF that allows senders > to get feedback from the I/O layer for not overloading it. This allows > the sender to adapt to the send rate of the network. Great, it sounds like this would fix the stall/hang issues. I expect to port Broker to the actor-sy

Re: [Bro-Dev] Broker raw throughput

2016-02-25 Thread Matthias Vallentin
For better reproducibility, here's the Makefile that I used to drive the experiments: CC = cc CXX = c++ FLAGS = -O3 -g -std=c++11 -stdlib=libc++ LIBS = -lcaf_core -lcaf_io -ltcmalloc -lprofiler caf-client: caf-client.cpp $(CXX) $(FLAGS) $< -o $@ $(LIBS) caf-server

[Bro-Dev] Broker raw throughput

2016-02-24 Thread Matthias Vallentin
performance was lacking. This corresponds to your findings, but I don't remember it being factor 5-6 worse. Thanks for sending me the gperf graphs. I will come back to you after running a test series under Linux and digging through the code a bit. Dominik ----- Forwarded message from Ma

Re: [Bro-Dev] Broker updates?

2016-02-23 Thread Matthias Vallentin
> If there are updates to a Broker store, is there a way that I can get > evented notification that a key was modified? I'm not seeing anything > that provides that functionality yet, but I need it for something I'm > working on. That's currently not supported, but it's on the wish list [1].

[Bro-Dev] [JIRA] (BIT-1474) Improve signaling of deprecated code

2015-09-08 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=22009#comment-22009 ] Matthias Vallentin commented on BIT-1474: - Ah, I missed that. That's d

[Bro-Dev] [JIRA] (BIT-1411) SQL_Injection_Victim is a misleading name

2015-09-06 Thread Matthias Vallentin (JIRA)
[ https://bro-tracker.atlassian.net/browse/BIT-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=21973#comment-21973 ] Matthias Vallentin commented on BIT-1411: - I like this syntax and the prop

[Bro-Dev] [JIRA] (BIT-1474) Improve signaling of deprecated code

2015-09-06 Thread Matthias Vallentin (JIRA)
Matthias Vallentin created BIT-1474: --- Summary: Improve signaling of deprecated code Key: BIT-1474 URL: https://bro-tracker.atlassian.net/browse/BIT-1474 Project: Bro Issue Tracker Issue

  1   2   >