On 17/02/18 16:40, Dan Prince wrote:
Thanks for the update Emilien. A couple of things to add:
1) This was really difficult to pin-point via the Heat stack error
message ('list index out of range'). I actually had to go and add
LOG.debug statements to Heat to get to the bottom of it. I aim to sync
with a few of the Heat folks next week on this to see if we can do
better here.
The message itself is pretty much all we get from yaql, even in its own
interpreter:
(py27) cat yaql_data.json
{"data": [{"foo": "bar"}]}
(py27) yaql -d yaql_data.json
Yet Another Query Language - command-line query tool
Version 1.1.3
Copyright (c) 2013-2017 Mirantis, Inc
yaql> dict($.data.where($ !=
null).flatten().selectMany($.items()).groupBy($[0], $[1], $.flatten()))
{
"foo": [
"bar"
]
}
yaql> dict($.data.where($ !=
null).flatten().selectMany($.items()).groupBy($[0], $[1], [$[0],
$[1].flatten()]))
Execution exception: list index out of range
yaql>
(Note that different lengths of data will give you different errors though.)
The big issue here though is that for failures in validation we report
the path in the template to the function that failed, but we don't do
the same for failures in actually resolving the function at runtime. A
comprehensive fix is challenging without breaking what is supposed to be
a stable third-party plugin API, but it might be possible. Was that the
information you needed to debug this?
We do report which resource failed, but for something with a huge
definition like allNodesConfig I can see why that might not help as much
as you'd hope.
2) I had initially thought it would have been much better to revert
the (breaking) change to python-yaql. That said it was from 2016! So I
think our window of opportunity for the revert is probably way too
large to consider that. Sounds like we need to publish the yaql
package more often in RDO, etc. So your patch to update our queries is
probably our only option.
I _think_ this should be OK for upgrades, as long as you never do a
stack update using the existing (Pike) templates after upgrading the
undercloud to Queens, but... sadface.
I think we need to either merge Thomas's patch that gets rid of this
function altogether (https://review.openstack.org/#/c/545856/) and
backport it to older versions of t-h-t, or make yaql itself
backward-compatible by doing something like
https://review.openstack.org/#/c/545996/
cheers,
Zane.
On Fri, Feb 16, 2018 at 8:36 PM, Emilien Macchi <emil...@redhat.com> wrote:
Upgrading YAQL from 1.1.0 to 1.1.3 breaks advanced queries with groupBy
aggregation.
The commit that broke it is
https://github.com/openstack/yaql/commit/3fb91784018de335440b01b3b069fe45dc53e025
It broke TripleO: https://bugs.launchpad.net/tripleo/+bug/1750032
But Alex and I figured (after a strong headache) that we needed to update
the query like this: https://review.openstack.org/545498
It would be great to avoid this kind of change within minor versions, please
please.
Happy weekend,
PS: I'm adding YAQL to my linkedin profile right now.
Be careful here. Do you really want to write YAQL queries all day!
Dan
--
Emilien Macchi
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
