MonetDB: monetdburl - In tls tests, show stderr when mclient doe...

2023-11-06 Thread Joeri van Ruth via checkin-list
Changeset: 676010558b3a for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/676010558b3a
Modified Files:
clients/mapilib/Tests/systemcertificates.py
clients/mapilib/Tests/tlssecurity.py
Branch: monetdburl
Log Message:

In tls tests, show stderr when mclient doesn't exit with code 2.

(For example if LeakSanitizer triggers)


diffs (30 lines):

diff --git a/clients/mapilib/Tests/systemcertificates.py 
b/clients/mapilib/Tests/systemcertificates.py
--- a/clients/mapilib/Tests/systemcertificates.py
+++ b/clients/mapilib/Tests/systemcertificates.py
@@ -25,7 +25,10 @@ HOST = 'python.org'
 # Run mclient
 cmd = ['mclient', '-L-', '-d', f"monetdbs://{HOST}:443/demo"]
 proc = subprocess.run(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
-assert proc.returncode == 2, f"mclient is supposed to exit with status 2, not 
{proc.returncode}"
+if proc.returncode != 2:
+msg = str(proc.stderr, 'utf-8')
+print(f"mclient is supposed to exit with status 2, not 
{proc.returncode}.\n--- stderr ---\n{msg}\n---end stderr ---", file=sys.stderr)
+exit(1)
 
 # After the TLS handshake succeeds we expect the server to send something like
 # 'HTTP/1.1 400 Bad Request' because we're sending \x00\x00 instead of an HTTP
diff --git a/clients/mapilib/Tests/tlssecurity.py 
b/clients/mapilib/Tests/tlssecurity.py
--- a/clients/mapilib/Tests/tlssecurity.py
+++ b/clients/mapilib/Tests/tlssecurity.py
@@ -75,7 +75,10 @@ def attempt(experiment: str, portname: s
 logging.debug(f"cmd={cmd}")
 proc = subprocess.run(cmd, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
 logging.debug(f"mclient exited with code {proc.returncode}, 
err={proc.stderr}")
-assert proc.returncode == 2, f"mclient is supposed to exit with status 2, 
not {proc.returncode}"
+if proc.returncode != 2:
+msg = str(proc.stderr, 'utf-8')
+print(f"mclient is supposed to exit with status 2, not 
{proc.returncode}.\n--- stderr ---\n{msg}\n---end stderr ---", file=sys.stderr)
+assert proc.returncode == 2, f"mclient is supposed to exit with status 
2, not {proc.returncode}"
 output = str(proc.stderr, 'utf-8').rstrip()
 actual_error = None if 'Sorry, this is not' in output else output
 
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: monetdburl - Plug memory leak

2023-11-06 Thread Joeri van Ruth via checkin-list
Changeset: a9f64cfeb344 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/a9f64cfeb344
Modified Files:
clients/mapilib/connect_openssl.c
Branch: monetdburl
Log Message:

Plug memory leak


diffs (33 lines):

diff --git a/clients/mapilib/connect_openssl.c 
b/clients/mapilib/connect_openssl.c
--- a/clients/mapilib/connect_openssl.c
+++ b/clients/mapilib/connect_openssl.c
@@ -134,11 +134,9 @@ verify_server_certificate_hash(Mapi mid,
unsigned char *buf = NULL;
int buflen = i2d_X509(x509, &buf);
if (buflen <= 0) {
-   X509_free(x509);
return croak_openssl(mid, __func__, "could not convert server 
certificate to DER");
}
assert(buf);
-   X509_free(x509);
 
// Compute the has of the DER using the deprecated API so we stay
// compatible with OpenSSL 1.1.1.
@@ -307,14 +305,17 @@ wrap_tls(Mapi mid, SOCKET sock)
BIO_free_all(bio);
return croak_openssl(mid, __func__, "Server did not send a 
certificate");
}
+   // be careful when to free server_cert
if (verify_method == verify_hash) {
const char *required_prefix = 
msettings_connect_certhash_digits(settings);
msg = verify_server_certificate_hash(mid, server_cert, 
required_prefix);
+   X509_free(server_cert);
if (msg != MOK) {
BIO_free_all(bio);
return msg;
}
} else {
+   X509_free(server_cert);
long verify_result = SSL_get_verify_result(ssl);
if (verify_result != X509_V_OK) {
BIO_free_all(bio);
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - try running ctest also on windows

2023-11-06 Thread Niels Nes via checkin-list
Changeset: dc920c0ec4f4 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/dc920c0ec4f4
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

try running ctest also on windows


diffs (47 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -42,6 +42,8 @@ jobs:
   - os: ubuntu-latest
 c_compiler: cl
 runs-on: ${{ matrix.os }}
+env:
+  CTEST: ${{ runner.os == 'Windows' && 'RUN_TESTS' || 'test' }}
 steps:
   - name: Checkout
 uses: actions/checkout@v3
@@ -86,17 +88,6 @@ jobs:
   make install -j3
 if: runner.os == 'macOS'
 
-  - name: ctest 
-run: |
-  cd build 
-  cmake --build . --target test
-if: runner.os != 'Windows'
-
-  - name: mtest 
-run: |
-  PATH=$HOME/MDB/bin:$PATH $HOME/MDB/bin/Mtest.py -r --debug=0 --ci 
--no-html --TSTTRGBASE=.
-if: runner.os != 'Windows'
-
   - name: choco packages
 run: |
   choco install winflexbison3
@@ -112,6 +103,16 @@ jobs:
   cmake --build . --target install
 if: runner.os == 'Windows'
 
+  - name: ctest 
+run: |
+  cd build 
+  cmake --build . --target ${{ CTEST }}
+
+  - name: mtest 
+run: |
+  PATH=$HOME/MDB/bin:$PATH $HOME/MDB/bin/Mtest.py -r --debug=0 --ci 
--no-html --TSTTRGBASE=.
+if: runner.os != 'Windows'
+
   - name: mtest 
 shell: pwsh
 run: |
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: new_rmt_opt - Closing branch new_rmt_opt.

2023-11-06 Thread Sjoerd Mullender via checkin-list
Changeset: d597a7918f40 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/d597a7918f40
Branch: new_rmt_opt
Log Message:

Closing branch new_rmt_opt.

___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - Merge new_rmt_opt into default.

2023-11-06 Thread Sjoerd Mullender via checkin-list
Changeset: b22b663c3f2d for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/b22b663c3f2d
Modified Files:
sql/backends/monet5/sql_gencode.c
Branch: default
Log Message:

Merge new_rmt_opt into default.


diffs (truncated from 484 to 300 lines):

diff --git a/sql/backends/monet5/sql_gencode.c 
b/sql/backends/monet5/sql_gencode.c
--- a/sql/backends/monet5/sql_gencode.c
+++ b/sql/backends/monet5/sql_gencode.c
@@ -354,7 +354,8 @@ static int
Client c = MCgetClient(m->clientid);
MalBlkPtr curBlk = 0;
InstrPtr curInstr = 0, p, o;
-   sqlid table_id = prp->id;
+   tid_uri *tu = ((list*)prp->value.pval)->h->data;
+   sqlid table_id = tu->id;
node *n;
int i, q, v, res = -1, added_to_cache = 0, *lret, *rret;
size_t len = 1024, nr, pwlen = 0;
@@ -389,6 +390,7 @@ static int
 
sql_table *rt = sql_trans_find_table(m->session->tr, table_id);
const char *uri = mapiuri_uri(rt->query, m->sa);
+   assert(strcmp(tu->uri, uri) == 0);
if (!rt) {
sql_error(m, 10, SQLSTATE(HY013) MAL_MALLOC_FAIL);
goto cleanup;
@@ -928,8 +930,12 @@ static int
Symbol symbackup = c->curprg;
exception_buffer ebsave = m->sa->eb;
 
-   if (prp->id == 0) {
-   sql_error(m, 003, SQLSTATE(42000) "Missing property on the 
input relation");
+   if (list_empty(prp->value.pval)) {
+   sql_error(m, 003, SQLSTATE(42000) "Missing REMOTE property on 
the input relation");
+   goto bailout;
+   }
+   if (list_length(prp->value.pval) != 1) {
+   sql_error(m, 003, SQLSTATE(42000) "REMOTE property on the input 
relation is NOT unique");
goto bailout;
}
if (strlen(mod) >= IDLENGTH) {
diff --git a/sql/server/rel_distribute.c b/sql/server/rel_distribute.c
--- a/sql/server/rel_distribute.c
+++ b/sql/server/rel_distribute.c
@@ -15,6 +15,12 @@
 #include "rel_remote.h"
 #include "sql_privileges.h"
 
+typedef struct rmt_prop_state {
+   int depth;
+   prop* rmt;
+   sql_rel* orig;
+} rps;
+
 static int
 has_remote_or_replica( sql_rel *rel )
 {
@@ -66,7 +72,7 @@ has_remote_or_replica( sql_rel *rel )
 }
 
 static sql_rel *
-rewrite_replica(mvc *sql, list *exps, sql_table *t, sql_table *p, int 
remote_prop)
+do_replica_rewrite(mvc *sql, list *exps, sql_table *t, sql_table *p, int 
remote_prop)
 {
node *n, *m;
sql_rel *r = rel_basetable(sql, p, t->base.name);
@@ -96,11 +102,15 @@ rewrite_replica(mvc *sql, list *exps, sq
 
/* set_remote() */
if (remote_prop && p && isRemote(p)) {
-   sqlid id = p->base.id;
-   char *local_name = sa_strconcat(sql->sa, sa_strconcat(sql->sa, 
p->s->base.name, "."), p->base.name);
-   prop *p = r->p = prop_create(sql->sa, PROP_REMOTE, r->p);
-   p->id = id;
-   p->value.pval = local_name;
+   list *uris = sa_list(sql->sa);
+   tid_uri *tu = SA_NEW(sql->sa, tid_uri);
+   tu->id = p->base.id;
+   tu->uri = sa_strconcat(sql->sa, sa_strconcat(sql->sa, 
p->s->base.name, "."), p->base.name);
+   append(uris, tu);
+
+   prop *rmt_prop = r->p = prop_create(sql->sa, PROP_REMOTE, r->p);
+   rmt_prop->id = p->base.id;
+   rmt_prop->value.pval = uris;
}
return r;
 }
@@ -109,24 +119,33 @@ static sql_rel *
 replica_rewrite(visitor *v, sql_table *t, list *exps)
 {
sql_rel *res = NULL;
-   const char *uri = (const char *) v->data;
+   prop *rp = ((rps*)v->data)->rmt;
+   sqlid tid = rp->id;
+   list *uris = rp->value.pval;
 
if (mvc_highwater(v->sql))
return sql_error(v->sql, 10, SQLSTATE(42000) "Query too 
complex: running out of stack space");
 
-   if (uri) {
-   /* replace by the replica which matches the uri */
-   for (node *n = t->members->h; n; n = n->next) {
+   /* if there was a REMOTE property in any higher node and there is not
+* a local tid then use the available uris to rewrite */
+   if (uris && !tid) {
+   for (node *n = t->members->h; n && !res; n = n->next) {
sql_part *p = n->data;
sql_table *pt = find_sql_table_id(v->sql->session->tr, 
t->s, p->member);
 
-   if (isRemote(pt) && strcmp(uri, pt->query) == 0) {
-   res = rewrite_replica(v->sql, exps, t, pt, 0);
-   break;
+   if (!isRemote(pt))
+   continue;
+
+   for (node *m = uris->h; m && !res; m = m->next) {
+   if (strcmp(((tid_uri*)m->data)->uri, pt->query) 
== 0) {
+   res = do_replica_rewrite(v->sql, exps, 
t, pt, 0);
+   }
}
 

MonetDB: default - maybe runner is know within the steps only

2023-11-06 Thread Niels Nes via checkin-list
Changeset: b27cf097fa52 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/b27cf097fa52
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

maybe runner is know within the steps only


diffs (15 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -42,9 +42,9 @@ jobs:
   - os: ubuntu-latest
 c_compiler: cl
 runs-on: ${{ matrix.os }}
-env:
-  CTEST: ${{ runner.os == 'Windows' && 'RUN_TESTS' || 'test' }}
 steps:
+  env:
+CTEST: ${{ runner.os == 'Windows' && 'RUN_TESTS' || 'test' }}
   - name: Checkout
 uses: actions/checkout@v3
 with:
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: Jun2023 - Check snprintf result so that we don't need d...

2023-11-06 Thread Sjoerd Mullender via checkin-list
Changeset: 28af495bf842 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/28af495bf842
Modified Files:
tools/merovingian/client/monetdb.c
Branch: Jun2023
Log Message:

Check snprintf result so that we don't need dirty tricks to compile.


diffs (31 lines):

diff --git a/tools/merovingian/client/monetdb.c 
b/tools/merovingian/client/monetdb.c
--- a/tools/merovingian/client/monetdb.c
+++ b/tools/merovingian/client/monetdb.c
@@ -2509,8 +2509,11 @@ main(int argc, char *argv[])
if (mero_host == NULL)
mero_host = "/tmp";
/* first try the port given (or else its default) */
-   snprintf(buf, sizeof(buf), "%s/.s.merovingian.%d",
-mero_host, mero_port == -1 ? MAPI_PORT : mero_port);
+   if (snprintf(buf, sizeof(buf), "%s/.s.merovingian.%d",
+mero_host, mero_port == -1 ? MAPI_PORT 
: mero_port) >= (int) sizeof(buf)) {
+   fprintf(stderr, "monetdb: directory name too long\n");
+   exit(1);
+   }
if ((err = control_ping(buf, -1, NULL)) == NULL) {
mero_host = buf;
} else {
@@ -2530,9 +2533,11 @@ main(int argc, char *argv[])
while ((e = readdir(d)) != NULL) {
if (strncmp(e->d_name, 
".s.merovingian.", 15) != 0)
continue;
-   snprintf(buf, sizeof(buf), "%s/%s", 
mero_host, e->d_name);
-   if (stat(buf, &s) == -1)
+   if (snprintf(buf, sizeof(buf), "%s/%s", 
mero_host, e->d_name) >= (int) sizeof(buf) ||
+   stat(buf, &s) == -1) {
+   /* too long or doesn't exist */
continue;
+   }
if (S_ISSOCK(s.st_mode)) {
char *nerr;
if ((nerr = control_ping(buf, 
-1, NULL)) == NULL) {
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - Merge with Jun2023 branch.

2023-11-06 Thread Sjoerd Mullender via checkin-list
Changeset: f9bc37a6f40d for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/f9bc37a6f40d
Modified Files:
tools/merovingian/client/monetdb.c
Branch: default
Log Message:

Merge with Jun2023 branch.


diffs (31 lines):

diff --git a/tools/merovingian/client/monetdb.c 
b/tools/merovingian/client/monetdb.c
--- a/tools/merovingian/client/monetdb.c
+++ b/tools/merovingian/client/monetdb.c
@@ -2509,8 +2509,11 @@ main(int argc, char *argv[])
if (mero_host == NULL)
mero_host = "/tmp";
/* first try the port given (or else its default) */
-   snprintf(buf, sizeof(buf), "%s/.s.merovingian.%d",
-mero_host, mero_port == -1 ? MAPI_PORT : mero_port);
+   if (snprintf(buf, sizeof(buf), "%s/.s.merovingian.%d",
+mero_host, mero_port == -1 ? MAPI_PORT 
: mero_port) >= (int) sizeof(buf)) {
+   fprintf(stderr, "monetdb: directory name too long\n");
+   exit(1);
+   }
if ((err = control_ping(buf, -1, NULL)) == NULL) {
mero_host = buf;
} else {
@@ -2530,9 +2533,11 @@ main(int argc, char *argv[])
while ((e = readdir(d)) != NULL) {
if (strncmp(e->d_name, 
".s.merovingian.", 15) != 0)
continue;
-   snprintf(buf, sizeof(buf), "%s/%s", 
mero_host, e->d_name);
-   if (stat(buf, &s) == -1)
+   if (snprintf(buf, sizeof(buf), "%s/%s", 
mero_host, e->d_name) >= (int) sizeof(buf) ||
+   stat(buf, &s) == -1) {
+   /* too long or doesn't exist */
continue;
+   }
if (S_ISSOCK(s.st_mode)) {
char *nerr;
if ((nerr = control_ping(buf, 
-1, NULL)) == NULL) {
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - Remove dirty trick.

2023-11-06 Thread Sjoerd Mullender via checkin-list
Changeset: 0ebe49f44f78 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/0ebe49f44f78
Modified Files:
tools/merovingian/client/monetdb.c
Branch: default
Log Message:

Remove dirty trick.


diffs (12 lines):

diff --git a/tools/merovingian/client/monetdb.c 
b/tools/merovingian/client/monetdb.c
--- a/tools/merovingian/client/monetdb.c
+++ b/tools/merovingian/client/monetdb.c
@@ -2381,7 +2381,7 @@ command_snapshot(int argc, char *argv[])
 int
 main(int argc, char *argv[])
 {
-   char buf[1024+10];
+   char buf[1024];
int i;
int retval = 0;
 #ifdef TIOCGWINSZ
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - env not allowed in steps..

2023-11-06 Thread Niels Nes via checkin-list
Changeset: f04055373ba7 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/f04055373ba7
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

env not allowed in steps..


diffs (21 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -43,8 +43,6 @@ jobs:
 c_compiler: cl
 runs-on: ${{ matrix.os }}
 steps:
-  env:
-CTEST: ${{ runner.os == 'Windows' && 'RUN_TESTS' || 'test' }}
   - name: Checkout
 uses: actions/checkout@v3
 with:
@@ -104,6 +102,8 @@ jobs:
 if: runner.os == 'Windows'
 
   - name: ctest 
+env:
+  CTEST: ${{ runner.os == 'Windows' && 'RUN_TESTS' || 'test' }}
 run: |
   cd build 
   cmake --build . --target ${{ CTEST }}
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - inline the conditional variable usage

2023-11-06 Thread Niels Nes via checkin-list
Changeset: de8b9bdc445f for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/de8b9bdc445f
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

inline the conditional variable usage


diffs (16 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -102,11 +102,9 @@ jobs:
 if: runner.os == 'Windows'
 
   - name: ctest 
-env:
-  CTEST: ${{ runner.os == 'Windows' && 'RUN_TESTS' || 'test' }}
 run: |
   cd build 
-  cmake --build . --target ${{ CTEST }}
+  cmake --build . --target ${{ runner.os == 'Windows' && 'RUN_TESTS' 
|| 'test' }}
 
   - name: mtest 
 run: |
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - Adds tests with replica join

2023-11-06 Thread stefanos mavros via checkin-list
Changeset: 5295bbd56e5b for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/5295bbd56e5b
Added Files:
sql/test/rel-optimizers/Tests/replicas-base.test
sql/test/rel-optimizers/Tests/replicas-join.reqtests
sql/test/rel-optimizers/Tests/replicas-join.test
Branch: default
Log Message:

Adds tests with replica join


diffs (104 lines):

diff --git a/sql/test/rel-optimizers/Tests/replicas-base.test 
b/sql/test/rel-optimizers/Tests/replicas-base.test
new file mode 100644
--- /dev/null
+++ b/sql/test/rel-optimizers/Tests/replicas-base.test
@@ -0,0 +1,28 @@
+statement ok
+create table foo_local (n int, m text)
+
+statement ok
+insert into foo_local values (1, 'hello'), (2, 'world'), (3, '!!')
+
+statement ok
+create remote table foo_r1 (n int, m text) on 
'mapi:monetdb://localhost:50002/node2'
+
+statement ok
+create remote table foo_r2 (n int, m text) on 
'mapi:monetdb://localhost:50003/node3'
+
+
+statement ok
+create table bar_local (n int, m text)
+
+statement ok
+insert into bar_local values (10, 'alice'), (2, 'bob'), (3, 'tom'), (4, 
'jerry')
+
+statement ok
+create remote table bar_r1 (n int, m text) on 
'mapi:monetdb://localhost:50002/node2'
+
+statement ok
+create remote table bar_r2 (n int, m text) on 
'mapi:monetdb://localhost:50003/node3'
+
+statement ok
+create remote table buz_rmt (l int) on 'mapi:monetdb://localhost:50002/node2'
+
diff --git a/sql/test/rel-optimizers/Tests/replicas-join.reqtests 
b/sql/test/rel-optimizers/Tests/replicas-join.reqtests
new file mode 100644
--- /dev/null
+++ b/sql/test/rel-optimizers/Tests/replicas-join.reqtests
@@ -0,0 +1,1 @@
+replicas-base
diff --git a/sql/test/rel-optimizers/Tests/replicas-join.test 
b/sql/test/rel-optimizers/Tests/replicas-join.test
new file mode 100644
--- /dev/null
+++ b/sql/test/rel-optimizers/Tests/replicas-join.test
@@ -0,0 +1,60 @@
+statement ok
+create replica table foo_rpl (n int, m text)
+
+statement ok
+alter table foo_rpl add table foo_local
+
+statement ok
+alter table foo_rpl add table foo_r1
+
+statement ok
+alter table foo_rpl add table foo_r2
+
+statement ok
+create replica table bar_rpl (n int, m text)
+
+statement ok
+alter table bar_rpl add table bar_local
+
+statement ok
+alter table bar_rpl add table bar_r1
+
+statement ok
+alter table bar_rpl add table bar_r2
+
+query T nosort
+plan select * from foo_rpl
+
+project (
+| table("sys"."foo_local") [ "foo_local"."n" as "foo_rpl"."n", "foo_local"."m" 
as "foo_rpl"."m", "foo_local"."%TID%" NOT NULL UNIQUE as "foo_rpl"."%TID%" ]
+) [ "foo_rpl"."n", "foo_rpl"."m" ]
+
+query T nosort
+plan select * from foo_rpl, bar_rpl
+
+project (
+| crossproduct (
+| | table("sys"."foo_local") [ "foo_local"."n" UNIQUE as "foo_rpl"."n", 
"foo_local"."m" as "foo_rpl"."m", "foo_local"."%TID%" NOT NULL UNIQUE as 
"foo_rpl"."%TID%" ],
+| | table("sys"."bar_local") [ "bar_local"."n" as "bar_rpl"."n", 
"bar_local"."m" as "bar_rpl"."m", "bar_local"."%TID%" NOT NULL UNIQUE as 
"bar_rpl"."%TID%" ]
+| ) [  ]
+) [ "foo_rpl"."n", "foo_rpl"."m", "bar_rpl"."n", "bar_rpl"."m" ]
+
+# In the next test and given the implentation of the remote optimizer we 
+# expect the replicas to be resolved to the remote tables. The optimizer 
+# could be in principle be adapted to resolve the replica crossproduct
+# to local tables
+query T nosort
+plan select * from foo_rpl, bar_rpl, buz_rmt
+
+table (
+| project (
+| | crossproduct (
+| | | crossproduct (
+| | | | REMOTE("sys"."foo_r1") [ "foo_r1"."n" as "foo_rpl"."n", "foo_r1"."m" 
as "foo_rpl"."m", "foo_r1"."%TID%" NOT NULL UNIQUE as "foo_rpl"."%TID%" ],
+| | | | REMOTE("sys"."bar_r1") [ "bar_r1"."n" as "bar_rpl"."n", "bar_r1"."m" 
as "bar_rpl"."m", "bar_r1"."%TID%" NOT NULL UNIQUE as "bar_rpl"."%TID%" ]
+| | | ) [  ],
+| | | REMOTE("sys"."buz_rmt") [ "buz_rmt"."l" ]
+| | ) [  ]
+| ) [ "foo_rpl"."n", "foo_rpl"."m", "bar_rpl"."n", "bar_rpl"."m", 
"buz_rmt"."l" ] REMOTE mapi:monetdb://localhost:50002/node2
+) [ "foo_rpl"."n", "foo_rpl"."m", "bar_rpl"."n", "bar_rpl"."m", "buz_rmt"."l" ]
+
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - Fixes bug with rewrite_remote opt

2023-11-06 Thread stefanos mavros via checkin-list
Changeset: 98a7373050f8 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/98a7373050f8
Modified Files:
sql/server/rel_distribute.c
Branch: default
Log Message:

Fixes bug with rewrite_remote opt


diffs (27 lines):

diff --git a/sql/server/rel_distribute.c b/sql/server/rel_distribute.c
--- a/sql/server/rel_distribute.c
+++ b/sql/server/rel_distribute.c
@@ -376,8 +376,9 @@ rel_rewrite_remote_(visitor *v, sql_rel 
l->p = prop_remove(l->p, pl);
r->p = prop_remove(r->p, pr);
if (!find_prop(rel->p, PROP_REMOTE)) {
-   /* remove any local tids */
-   pl->id = 0;
+   /* remove local tid ONLY if no subtree 
has local parts */
+   if (pl->id == 0 || pr->id == 0)
+   pl->id = 0;
/* set the new (matching) uris */
pl->value.pval = uris;
/* push the pl REMOTE property to the 
list of properties */
@@ -424,8 +425,9 @@ rel_rewrite_remote_(visitor *v, sql_rel 
l->p = prop_remove(l->p, pl);
r->p = prop_remove(r->p, pr);
if (!find_prop(rel->p, PROP_REMOTE)) {
-   /* remove any local tids */
-   pl->id = 0;
+   /* remove local tid ONLY if no 
subtree has local parts */
+   if (pl->id == 0 || pr->id == 0)
+   pl->id = 0;
/* set the new (matching) uris 
*/
pl->value.pval = uris;
/* push the pl REMOTE property 
to the list of properties */
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - Enables replicas-join test

2023-11-06 Thread stefanos mavros via checkin-list
Changeset: bcf8a36a5801 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/bcf8a36a5801
Modified Files:
sql/test/rel-optimizers/Tests/All
Branch: default
Log Message:

Enables replicas-join test


diffs (9 lines):

diff --git a/sql/test/rel-optimizers/Tests/All 
b/sql/test/rel-optimizers/Tests/All
--- a/sql/test/rel-optimizers/Tests/All
+++ b/sql/test/rel-optimizers/Tests/All
@@ -2,3 +2,5 @@ split-select
 groupby-cse
 groupjoin
 join-merge-remote-replica
+replicas-base
+replicas-join
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - merges second head

2023-11-06 Thread stefanos mavros via checkin-list
Changeset: 339fc301fd75 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/339fc301fd75
Branch: default
Log Message:

merges second head


diffs (16 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -102,11 +102,9 @@ jobs:
 if: runner.os == 'Windows'
 
   - name: ctest 
-env:
-  CTEST: ${{ runner.os == 'Windows' && 'RUN_TESTS' || 'test' }}
 run: |
   cd build 
-  cmake --build . --target ${{ CTEST }}
+  cmake --build . --target ${{ runner.os == 'Windows' && 'RUN_TESTS' 
|| 'test' }}
 
   - name: mtest 
 run: |
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - try windows ctest with extended path (vcpkg)

2023-11-06 Thread Niels Nes via checkin-list
Changeset: 1f777d4f4753 for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/1f777d4f4753
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

try windows ctest with extended path (vcpkg)


diffs (34 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -16,7 +16,7 @@ jobs:
 strategy:
   fail-fast: false  # don't stop other jobs
   matrix:
-branch: [ master ]
+branch: [ ${{ GITHUB_REF }} ]
 os: [ ubuntu-latest, macos-latest, windows-latest ]
 c_compiler: [ gcc, clang, cl ]
 include:
@@ -105,12 +105,21 @@ jobs:
 run: |
   cd build 
   cmake --build . --target ${{ runner.os == 'Windows' && 'RUN_TESTS' 
|| 'test' }}
+if: runner.os != 'Windows'
 
   - name: mtest 
 run: |
   PATH=$HOME/MDB/bin:$PATH $HOME/MDB/bin/Mtest.py -r --debug=0 --ci 
--no-html --TSTTRGBASE=.
 if: runner.os != 'Windows'
 
+  - name: ctest 
+shell: pwsh
+run: |
+  $env:PATH = 
'C:\vcpkg\installed\x64-windows\bin;C:\vcpkg\installed\x64-windows\debug\bin;' 
+ $env:PATH
+  cd build 
+  cmake --build . --target ${{ runner.os == 'Windows' && 'RUN_TESTS' 
|| 'test' }}
+if: runner.os == 'Windows'
+
   - name: mtest 
 shell: pwsh
 run: |
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - docs aren't clear on this

2023-11-06 Thread Niels Nes via checkin-list
Changeset: 0e1186759e9f for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/0e1186759e9f
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

docs aren't clear on this


diffs (27 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -16,7 +16,6 @@ jobs:
 strategy:
   fail-fast: false  # don't stop other jobs
   matrix:
-branch: [ ${{ GITHUB_REF }} ]
 os: [ ubuntu-latest, macos-latest, windows-latest ]
 c_compiler: [ gcc, clang, cl ]
 include:
@@ -46,7 +45,7 @@ jobs:
   - name: Checkout
 uses: actions/checkout@v3
 with:
-  ref: ${{ matrix.branch }}
+  ref: ${{ github.ref }}
 
   - name: install pymonetdb cryptography
 run: pip3 install pymonetdb cryptography
@@ -132,5 +131,5 @@ jobs:
   - name: Publish mtest results
 uses: actions/upload-artifact@v3
 with:
-  name: mtest-${{ matrix.branch }}-${{ matrix.os }}-${{ 
matrix.c_compiler }}
+  name: mtest-${{ github.ref }}-${{ matrix.os }}-${{ matrix.c_compiler 
}}
   path: mtests.tar
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - use the sha instead

2023-11-06 Thread Niels Nes via checkin-list
Changeset: 06787b1cb3cc for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/06787b1cb3cc
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

use the sha instead


diffs (10 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -131,5 +131,5 @@ jobs:
   - name: Publish mtest results
 uses: actions/upload-artifact@v3
 with:
-  name: mtest-${{ github.ref }}-${{ matrix.os }}-${{ matrix.c_compiler 
}}
+  name: mtest-${{ github.sha }}-${{ matrix.os }}-${{ matrix.c_compiler 
}}
   path: mtests.tar
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org


MonetDB: default - full windows path's

2023-11-06 Thread Niels Nes via checkin-list
Changeset: 18e9a4f1516f for MonetDB
URL: https://dev.monetdb.org/hg/MonetDB/rev/18e9a4f1516f
Modified Files:
.github/workflows/linux.yml
Branch: default
Log Message:

full windows path's


diffs (12 lines):

diff --git a/.github/workflows/linux.yml b/.github/workflows/linux.yml
--- a/.github/workflows/linux.yml
+++ b/.github/workflows/linux.yml
@@ -114,7 +114,7 @@ jobs:
   - name: ctest 
 shell: pwsh
 run: |
-  $env:PATH = 
'C:\vcpkg\installed\x64-windows\bin;C:\vcpkg\installed\x64-windows\debug\bin;' 
+ $env:PATH
+  $env:PATH = 
'C:\MDB\lib;C:\MDB\lib\monetdb5;C:\MDB\bin;C:\vcpkg\installed\x64-windows\bin;C:\vcpkg\installed\x64-windows\debug\bin;'
 + $env:PATH
   cd build 
   cmake --build . --target ${{ runner.os == 'Windows' && 'RUN_TESTS' 
|| 'test' }}
 if: runner.os == 'Windows'
___
checkin-list mailing list -- checkin-list@monetdb.org
To unsubscribe send an email to checkin-list-le...@monetdb.org