Changeset: ab1fe21d38f7 for MonetDB URL: http://dev.monetdb.org/hg/MonetDB?cmd=changeset;node=ab1fe21d38f7 Modified Files: sql/backends/monet5/datacell/actuator.c sql/backends/monet5/datacell/petrinet.c sql/backends/monet5/datacell/sensor.c Branch: default Log Message:
Added some documentation. diffs (290 lines): diff --git a/sql/backends/monet5/datacell/actuator.c b/sql/backends/monet5/datacell/actuator.c --- a/sql/backends/monet5/datacell/actuator.c +++ b/sql/backends/monet5/datacell/actuator.c @@ -2,7 +2,7 @@ * The contents of this file are subject to the MonetDB Public License * Version 1.1 (the "License"); you may not use this file except in * compliance with the License. You may obtain a copy of the License at - * http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html + * http://www.monetdb.org/Legal/MonetDBLicense * * Software distributed under the License is distributed on an "AS IS" * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the @@ -18,26 +18,11 @@ */ /* - * @' The contents of this file are subject to the MonetDB Public License - * @' Version 1.1 (the "License"); you may not use this file except in - * @' compliance with the License. You may obtain a copy of the License at - * @' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html - * @' - * @' Software distributed under the License is distributed on an "AS IS" - * @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the - * @' License for the specific language governing rights and limitations - * @' under the License. - * @' - * @' The Original Code is the MonetDB Database System. - * @' - * @' The Initial Developer of the Original Code is CWI. - * @' Portions created by CWI are Copyright (C) 1997-2008 CWI. - * @' All Rights Reserved. - * * @a M. Kersten, E. Liarou, R. Goncalves - * @* The Actuator Simulation program - * This program listens to an event stream on a particular port. - * It collects statistics on the events. +The Actuator Simulation program + +This program listens to an event stream on a particular port. +It collects statistics on the events. * @f actuator */ #ifndef ACTUATOR @@ -98,6 +83,26 @@ #define PASSIVE 2 static int mode = PASSIVE; +/* +The actuator. +Events leave the Datacell to trigger actions in the real world. It may be as simple +as being displayed on a screen, upto and including calling the mergency fighters +by telephone. Since, the mapping from event message to real world action is situation +specific, the default provided here is to only show the event message. + +Users interested in experimentation with real actuators, can hook up the event +stream with a SCADA system or directly sent the events in a proper format to +the channel where the actuators is listening. + +The actuator runs in two modes, active or passive. In the former case, the tool +contacts the server and establishes an UDF channel to pass CSV encoded event strings. +Optionally, the event can be tagged with a serial key and a time-stamp. + +Events are generated using a built-in random number generator. They are sent over the +channel with an optional delay, expressed in number of microseconds. + +The code is relatively straightforward to extend to create your favoured dummy sensor. +*/ void usage() { diff --git a/sql/backends/monet5/datacell/petrinet.c b/sql/backends/monet5/datacell/petrinet.c --- a/sql/backends/monet5/datacell/petrinet.c +++ b/sql/backends/monet5/datacell/petrinet.c @@ -2,7 +2,7 @@ * The contents of this file are subject to the MonetDB Public License * Version 1.1 (the "License"); you may not use this file except in * compliance with the License. You may obtain a copy of the License at - * http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html + * http://www.monetdb.org/Legal/MonetDBLicense * * Software distributed under the License is distributed on an "AS IS" * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the @@ -18,35 +18,58 @@ */ /* - * @' The contents of this file are subject to the MonetDB Public License - * @' Version 1.1 (the "License"); you may not use this file except in - * @' compliance with the License. You may obtain a copy of the License at - * @' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html - * @' - * @' Software distributed under the License is distributed on an "AS IS" - * @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the - * @' License for the specific language governing rights and limitations - * @' under the License. - * @' - * @' The Original Code is the MonetDB Database System. - * @' - * @' The Initial Developer of the Original Code is CWI. - * @' Portions created by CWI are Copyright (C) 1997-2008 CWI. - * @' All Rights Reserved. - * * @f petrinet * @a Martin Kersten * @v 1 * @+ Petri-net engine - * This module is a prototype for the implementation of a - * Petri-net interpreter for the DataCell. - * - * The example below accepts events at channel X - * and the continuous query move them to Y. - * Both query and emitter attached are scheduled. - * @example - * -see test directory - * @end example +The Datacell scheduler is based on the long-standing and mature Petri-net technology. For completeness, we +recap its salient points taken from Wikipedia. For more detailed information look at the science library. + +A Petri net (also known as a place/transition net or P/T net) is one of several mathematical modeling +languages for the description of distributed systems. A Petri net is a directed bipartite graph, +in which the nodes represent transitions (i.e. events that may occur, signified by bars) and +places (i.e. conditions, signified by circles). The directed arcs describe which places are pre- +and/or postconditions for which transitions (signified by arrows). +Some sources state that Petri nets were invented in August 1939 by Carl Adam Petri – +at the age of 13 – for the purpose of describing chemical processes. + +Like industry standards such as UML activity diagrams, BPMN and EPCs, Petri nets offer a +graphical notation for stepwise processes that include choice, iteration, and concurrent execution. +Unlike these standards, Petri nets have an exact mathematical definition of their execution semantics, +with a well-developed mathematical theory for process analysis. + +A Petri net consists of places, transitions, and directed arcs. Arcs run from a place to a transition or vice versa, +never between places or between transitions. The places from which an arc runs to a transition are called the input +places of the transition; the places to which arcs run from a transition are called the output places of the transition. + +Places may contain a natural number of tokens. A distribution of tokens over the places of a net is called a marking. +A transition of a Petri net may fire whenever there is a token at the start of all input arcs; when it fires, +it consumes these tokens, and places tokens at the end of all output arcs. A firing is atomic, i.e., a single non-interruptible step. + +Execution of Petri nets is nondeterministic: when multiple transitions are enabled at the same time, +any one of them may fire. If a transition is enabled, it may fire, but it doesn't have to. + +Since firing is nondeterministic, and multiple tokens may be present anywhere in the net (even in the same place), Petri nets are well suited for modeling the concurrent behavior of distributed systems. + +The Datacell scheduler is a fair implementation of a Petri-net interpreter. It models all continuous queries as transitions, +and the baskets as the places. The events are equivalent to tokens. Unlike the pure Petri-net model, all tokens in a place +are taken out on each firing. They may result into placing multiple tokens into receiving baskets. + +The scheduling amongst the transistions is currently deterministic. Upon each round of the scheduler, it determines all +transitions eligble to fire, i.e. have non-empty baskets, which are then actived one after the other. Future implementations +may relax this rigid scheme using a parallel implementation of the scheduler, such that each transition by itself can +decide to fire. However, when resources are limited to handle all complex continuous queries, it may pay of to invest +into a domain specif scheduler. + +For example, in the EMILI case, we may want to give priority to fire transistions based on the sensor type (is there fire) +or detection of emergency trends (the heat increases beyong model-based prediction). The software structure where to +inject this domain specific code is well identified and relatively easy to extend. + +The current implementation is limited to a fixed number of transitions. The scheduler can be stopped and restarted +at any time. Even selectively for specific baskets. This provides the handle to debug a system before being deployed. +In general, event processing through multiple layers of continous queries is too fast to trace them one by one. +Some general statistics about number of events handled per transition is maintained, as well as the processing time +for each continous query step. This provides the information to re-design the event handling system. */ #include "petrinet.h" #include "mal_builder.h" @@ -260,18 +283,18 @@ int i, k; mnstr_printf(PNout,"#scheduler status %s\n", statusnames[status]); for (i = 0; i < pnettop; i++) { - mnstr_printf(PNout, "#[%d]\t%s %s delay %d cycles %d events %d time %d ms\n", + mnstr_printf(PNout, "#[%d]\t%s %s delay %d cycles %d events %d time " LLFMT " ms\n", i, pnet[i].name, statusnames[pnet[i].status], pnet[i].delay, pnet[i].cycles, pnet[i].events, pnet[i].time/1000); if ( pnet[i].error) mnstr_printf(PNout,"#%s\n", pnet[i].error); for (k = 0; k < pnet[i].srctop; k++) - mnstr_printf(PNout, "#<--\t%s basket[%d] %d\n", + mnstr_printf(PNout, "#<--\t%s basket[%d] " SZFMT " " SZFMT "\n", pnet[i].source[k].table, pnet[i].source[k].bskt, pnet[i].source[k].lastcount, pnet[i].source[k].consumed); for (k = 0; k < pnet[i].trgttop; k++) - mnstr_printf(PNout, "#-->\t%s basket[%d] %d %d\n", + mnstr_printf(PNout, "#-->\t%s basket[%d] " SZFMT " " SZFMT "\n", pnet[i].target[k].table, pnet[i].source[k].bskt, pnet[i].target[k].lastcount, diff --git a/sql/backends/monet5/datacell/sensor.c b/sql/backends/monet5/datacell/sensor.c --- a/sql/backends/monet5/datacell/sensor.c +++ b/sql/backends/monet5/datacell/sensor.c @@ -2,7 +2,7 @@ * The contents of this file are subject to the MonetDB Public License * Version 1.1 (the "License"); you may not use this file except in * compliance with the License. You may obtain a copy of the License at - * http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html + * http://www.monetdb.org/Legal/MonetDBLicense * * Software distributed under the License is distributed on an "AS IS" * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the @@ -18,34 +18,39 @@ */ /* - * @' The contents of this file are subject to the MonetDB Public License - * @' Version 1.1 (the "License"); you may not use this file except in - * @' compliance with the License. You may obtain a copy of the License at - * @' http://monetdb.cwi.nl/Legal/MonetDBLicense-1.1.html - * @' - * @' Software distributed under the License is distributed on an "AS IS" - * @' basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the - * @' License for the specific language governing rights and limitations - * @' under the License. - * @' - * @' The Original Code is the MonetDB Database System. - * @' - * @' The Initial Developer of the Original Code is CWI. - * @' Portions created by CWI are Copyright (C) 1997-2008 CWI. - * @' All Rights Reserved. - * - * @a M. Kersten, E. Liarou, R. Goncalves - * @* The Sensor Simulation program - * This program generates an event stream on a particular port. - * The controlling parameters are the host (-h), the port id (-p). - * The events either come from a pre-cooked ascii file (-f) - * or they are generated internally (-l). In the latter case - * it assumes integer fields only and the number of columns - * should explicity be set (-C). - * Sending the events to the DataCell can be batched (-B) or delayed (-D) - * a number of milliseconds. - * @f sensor - */ + * @a M. Kersten, F.Groffen, E. Liarou, R. Goncalves +The Sensor Simulation program + +The sensor tool can be used to simulate a simple sensor, such as an on/off sensor or +a heat sensor. It is an independent tool, which runs on any platform where a C-compiler +is available for compilation, and where the fire-wall settings permit connection to +specific host:port channels. + +The sensor generator runs in two modes, active or passive. In the former case, the tool +contacts the server and establishes an UDF channel to pass CSV encoded event strings. +In the latter case, it is the datacell server that periodically contacts the sensor to +deliver an event. +Optionally, the event can be tagged with a serial key and a time-stamp. + +Events are generated using a built-in random number generator or a file. +The latter can be used to re-play a scenario, precisely mimicking the time +an event was raised. +The events are sent over the channel with an optional delay, expressed in number of microseconds. + +To stress the system under various situations, the events can be sent in batches, +where between each batch an optional delay is obeyed. The delay is specified in +number of milliseconds. Setting the delay to zero results in an event stream burst. + +The sensor program provides a template to experiment with your own sensors. +The structure has been set up to generate MonetDB tuple formatted events, +comma-separated-input (CSV) and tab-separated-input (TSV). The protocol +can readily be extended with SOAP and JSON based schemes. + +More esotheric solutions, e.g. to mimick real sensor systems, calls for +development of a simple event constructor and serialization scheme. +Moreover, in those cases a receptor should be constructed, which understands +the event message protocol. +*/ #ifndef SENSOR #define SENSOR #include "monetdb_config.h" @@ -94,6 +99,12 @@ return se; } +/* +Defaults: +The default settings for the sensor simulator are single millisecond delay, +sending each event in isolatio, e.g. no batching, using a single payload columns. +The datacell is presumed to life on the same host and listens to the port 50500. +*/ static int delay = 1; /* intra batch delay in ms, use 1 to avoid loosing too many */ static int batchsize = 1; static int events = -1; @@ -516,7 +527,7 @@ if (timestamp) { currenttsmp = GDKusec(); - snprintf(tuple + tlen, maxtuple - tlen, "%s%lld", (autoincrement?separator[protocol]:""), currenttsmp); + snprintf(tuple + tlen, maxtuple - tlen, "%s" LLFMT "", (autoincrement?separator[protocol]:""), currenttsmp); tlen += (int)strlen(tuple + tlen); if (tlen >= maxtuple) { mnstr_printf(SEout, "Buffer not large enough to handle request.\n"); _______________________________________________ Checkin-list mailing list Checkin-list@monetdb.org http://mail.monetdb.org/mailman/listinfo/checkin-list
