Hi,
Concatenation through a letter looks like a good approach to me. As far as
killing all queries on a specific node, I would put it aside for now - this
looks like a separate command with possibly different parameters.
On Tue, Jan 15, 2019 at 1:30 PM Юрий <jury.gerzhedow...@gmail.com> wrote:
> Thanks Vladimir for your thoughts.
>
> Based on it most convenient ways are first and third.
> But with some modifications:
> For first variant delimiter should be a letter, e.g. 123X15494, then it
> could be simple copy by user.
> For 3rd variant can be used convert both numeric to HEX and use a letter
> delimiter not included to HEX symbols (ABCDEF), in this case query id will
> be shorter and also can be simple copy by user. e.g. 7BX3C86 ( it the same
> value as used for first variant), instead of convert all value as string to
> base16 due to it will be really long value.
>
> Possible realization for the cases:
> 1) Concatenation node order id with query id with a letter delimiter.
>
> query_id = 1234X8753 , where *1234* - node order, *8753* - local node query
> counter. *X* - delimeter
>
> 2) Converting both node order id and query id to HEX.
>
> query_id = 7BX3C86, value is concat(hex(node),"X",hex(queryID))
>
> For both variants we can use either simple or copmlex KILL QUERY syntax.
> Simple:
>
> KILL QUERY 7BX3C86 - for kill concrete query
> KILL QUERY 7B - for killing all queries on a node. May be need extra
> symbols for such queries to avoid fault of user and kill all queries by
> mistake, like KILL QUERY 7B*
>
> Complex:
>
> KILL QUERY WHERE queryId=7BX3C86 - for killing concrete query.
>
> KILL QUERY WHERE nodeId=37d7afd8-b87d-4aa1-b3d1-c1c033800000 - for kill
> all running queries on a given node.
>
>
>
> What do you think?
>
>
> вт, 15 янв. 2019 г. в 11:20, Vladimir Ozerov <voze...@gridgain.com>:
>
> > Hi Yuriy,
> >
> > I think all proposed approaches might work. The question is what is the
> > most convenient from user perspective. Encoded values without special
> > characters are good because they are easy to copy with mouse
> (double-click)
> > or keyboard (Ctrl+Shift+arrow). On the other hand, ability to identify
> > ID/name of suspicious node from query ID is also a good thing. Several
> > examples of query ID:
> >
> > CockroachDB: 14dacc1f9a781e3d0000000000000001
> > MongoDB: shardB:79014
> >
> > Also it is important that the same query ID is printed in various log
> > messages. This will be very useful for debugging purposes, e.g. grep over
> > logs. So ideally query ID should not have any symbols which interfere
> with
> > grep syntax.
> >
> >
> > On Mon, Jan 14, 2019 at 3:09 PM Юрий <jury.gerzhedow...@gmail.com>
> wrote:
> >
> > > Hi Igniters,
> > >
> > > Earlier we discuss about columns for running queries. Let's summarize
> it
> > > and continue discussion for not closed questions.
> > >
> > > What we had:
> > > *name of view**: *running_queries
> > > *columns and meaning*:
> > > query_id - unique id of query on node
> > > node_id - initial node of request.
> > > sql - text of query
> > > schema_name - name of sql schema
> > > duration - duration in milliseconds from start
> of
> > > execution.
> > >
> > > All of this columns are clear, except query_id.
> > > Let's keep in mind that the query_id column of the view coupled with
> KILL
> > > QUERY command.
> > >
> > > We have the following variants what is query_id:
> > > 1) It's string, internally with two parts separated by '.'(it can be
> > other
> > > separator): numeric node order and numeric query counter unique for
> local
> > > node, e.g. '172.67321'. For this case query id will be really unique
> > across
> > > a cluster, but can be looks a strange for a user, especially in case we
> > > will have ability to kill all queries on a node, when user should get
> > first
> > > part before separator to use it, e.g. KILL QUERY '172.*'.
> > >
> > > 2) Just single numeric id, unique for local node, e.g '127'. In this
> case
> > > we need more complicated syntax for further KILL QUERY command, which
> > lead
> > > to use two columns from the view, e.g. KILL QUERY WHERE nodeId=
> > > 37d7afd8-b87d-4aa1-b3d1-c1c033800000 and queryId=67321
> > >
> > > 3) Use base16String(concat(node,".",queryID) as query id, e.g. '
> > > 3132332E393337'. Then we hide internal structure of id and such id will
> > be
> > > unique across a cluster. However we will need use complicated syntax
> for
> > > KILL QUERY command as for 2nd case.
> > >
> > > 4) Just single numeric id, unique for local node, e.g '127'. But user
> > > should use two columns to merge it and create query id unique in a
> > cluster.
> > > Such approach use by Oracle:ALTER SYSTEM CANCEL SQL 'SID, SERIAL,
> > SQL_ID'.
> > > In this case user will know real meaning of each part of passed
> parameter
> > > for KILL QUERY command. But it hard to use.
> > >
> > > 5) Any other approach you can think of....
> > >
> > > If be honestly I prefer first variant, it looks simple to use by user
> (it
> > > require read a docs, but any case it required for any use cases).
> > >
> > > Lets discuss it again and chose better approach to expose query_id
> column
> > > for Ignite. Also confirm list of columns.
> > >
> > > вт, 27 нояб. 2018 г. в 11:00, Vladimir Ozerov <voze...@gridgain.com>:
> > >
> > > > Yes ("нуы")
> > > >
> > > > On Tue, Nov 27, 2018 at 10:56 AM Павлухин Иван <vololo...@gmail.com>
> > > > wrote:
> > > >
> > > > > I believe that the meaning was:
> > > > >
> > > > > > I propose to start with running queries VIEW first.
> > > > > вт, 27 нояб. 2018 г. в 10:47, Vladimir Ozerov <
> voze...@gridgain.com
> > >:
> > > > > >
> > > > > > I propose to start with running queries мшуц first. Once we have
> > it,
> > > it
> > > > > > will be easier to agree on final command syntax.
> > > > > >
> > > > > > On Fri, Nov 23, 2018 at 9:32 AM Павлухин Иван <
> vololo...@gmail.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > May be I am a little bit late with my thoughts about a command
> > > > syntax.
> > > > > > > How do I see it is going to be used:
> > > > > > > 1. A user is able to kill a query by unique id belonging only
> to
> > > this
> > > > > > > query.
> > > > > > > 2. A user is able to kill all queries started by a specific
> node.
> > > > > > > For killing a single query we must identify it by unique id
> which
> > > is
> > > > > > > going to be received directly from Ignite (e.g. running queries
> > > view)
> > > > > > > and not calculated by user. Internally the id is compound but
> why
> > > > > > > cannot we convert it to opaque integer or string which hides
> real
> > > > > > > structure? E.g. base16String(concat(nodeOrder.toString(), ".",
> > > > > > > queryIdOnNode.toString())) The syntax could be KILL QUERY '123'
> > or
> > > > > > > KILL QUERY WHERE queryId = '123'
> > > > > > > For killing all queries started by some node we need to use
> only
> > > node
> > > > > > > order (or id). It could be like KILL QUERY WHERE nodeOrder =
> 34.
> > > > > > > чт, 22 нояб. 2018 г. в 12:56, Denis Mekhanikov <
> > > > dmekhani...@gmail.com
> > > > > >:
> > > > > > > >
> > > > > > > > Actually, option with separate parameters was mentioned in
> > > another
> > > > > thread
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/proposed-design-for-thin-client-SQL-management-and-monitoring-view-running-queries-and-kill-it-tp37713p38056.html
> > > > > > > >
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > чт, 22 нояб. 2018 г. в 08:51, Vladimir Ozerov <
> > > > voze...@gridgain.com
> > > > > >:
> > > > > > > >
> > > > > > > > > Denis,
> > > > > > > > >
> > > > > > > > > Problems with separate parameters are explained above.
> > > > > > > > >
> > > > > > > > > чт, 22 нояб. 2018 г. в 3:23, Denis Magda <
> dma...@apache.org
> > >:
> > > > > > > > >
> > > > > > > > > > Vladimir,
> > > > > > > > > >
> > > > > > > > > > All of the alternatives are reminiscent of mathematical
> > > > > operations.
> > > > > > > Don't
> > > > > > > > > > look like a SQL command. What if we use a SQL approach
> > > > > introducing
> > > > > > > named
> > > > > > > > > > parameters:
> > > > > > > > > >
> > > > > > > > > > KILL QUERY query_id=10 [AND node_id=5]
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > > > On Wed, Nov 21, 2018 at 4:11 AM Vladimir Ozerov <
> > > > > > > voze...@gridgain.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Denis,
> > > > > > > > > > >
> > > > > > > > > > > Space is bad candidate because it is a whitespace.
> > Without
> > > > > > > whitespaces
> > > > > > > > > we
> > > > > > > > > > > can have syntax without quotes at all. Any
> non-whitespace
> > > > > delimiter
> > > > > > > > > will
> > > > > > > > > > > work, though:
> > > > > > > > > > >
> > > > > > > > > > > KILL QUERY 45.1
> > > > > > > > > > > KILL QUERY 45-1
> > > > > > > > > > > KILL QUERY 45:1
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Nov 21, 2018 at 3:06 PM Юрий <
> > > > > jury.gerzhedow...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Denis,
> > > > > > > > > > > >
> > > > > > > > > > > > Let's consider parameter of KILL QUERY just a string
> > with
> > > > > some
> > > > > > > query
> > > > > > > > > > id,
> > > > > > > > > > > > without any meaning for user. User just need to get
> the
> > > id
> > > > > and
> > > > > > > pass
> > > > > > > > > as
> > > > > > > > > > > > parameter to KILL QUERY command.
> > > > > > > > > > > >
> > > > > > > > > > > > Even if query is distributed it have single query id
> > from
> > > > > user
> > > > > > > > > > > perspective
> > > > > > > > > > > > and will killed on all nodes. User just need to known
> > one
> > > > > global
> > > > > > > > > query
> > > > > > > > > > > id.
> > > > > > > > > > > >
> > > > > > > > > > > > How it can works.
> > > > > > > > > > > > 1)SELECT * from running_queries
> > > > > > > > > > > > result is
> > > > > > > > > > > > query_id | node_id
> > > > > > > > > > > > | sql | schema_name | connection_id |
> > > > > duration
> > > > > > > > > > > > 123.33 | e0a69cb8-a1a8-45f6-b84d-ead367a00000 |
> > > > SELECT
> > > > > > > ... |
> > > > > > > > > ...
> > > > > > > > > > > > | 22 | 23456
> > > > > > > > > > > > 333.31 | aaa6acb8-a4a5-42f6-f842-ead111b00020
> |
> > > > > > > UPDATE... |
> > > > > > > > > > ...
> > > > > > > > > > > > | 321 | 3467777
> > > > > > > > > > > > 2) KILL QUERY '123.33'
> > > > > > > > > > > >
> > > > > > > > > > > > So, user need select query_id from running_queries
> view
> > > and
> > > > > use
> > > > > > > it
> > > > > > > > > for
> > > > > > > > > > > KILL
> > > > > > > > > > > > QUERY command.
> > > > > > > > > > > >
> > > > > > > > > > > > I hope it became clearer.
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > ср, 21 нояб. 2018 г. в 02:11, Denis Magda <
> > > > dma...@apache.org
> > > > > >:
> > > > > > > > > > > >
> > > > > > > > > > > > > Folks,
> > > > > > > > > > > > >
> > > > > > > > > > > > > The decimal syntax is really odd - KILL QUERY
> > > > > > > > > > > > > '[node_order].[query_counter]'
> > > > > > > > > > > > >
> > > > > > > > > > > > > Confusing, let's use a space to separate
> parameters.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also, what if I want to halt a specific query with
> > > > certain
> > > > > ID?
> > > > > > > > > Don't
> > > > > > > > > > > know
> > > > > > > > > > > > > the node number, just know that the query is
> > > distributed
> > > > > and
> > > > > > > runs
> > > > > > > > > > > across
> > > > > > > > > > > > > several machines. Sounds like the syntax still
> should
> > > > > consider
> > > > > > > > > > > > > [node_order/id] as an optional parameter.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Probably, if you explain to me how an end user will
> > use
> > > > > this
> > > > > > > > > command
> > > > > > > > > > > from
> > > > > > > > > > > > > the very beginning (how do I look for a query id
> and
> > > node
> > > > > id,
> > > > > > > etc)
> > > > > > > > > > then
> > > > > > > > > > > > the
> > > > > > > > > > > > > things get clearer.
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Denis
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Nov 20, 2018 at 1:03 AM Юрий <
> > > > > > > jury.gerzhedow...@gmail.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Hi Vladimir,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Thanks for your suggestion to use MANAGEMENT_POOL
> > for
> > > > > > > processing
> > > > > > > > > > > > > > cancellation requests.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > About your questions.
> > > > > > > > > > > > > > 1) I'm going to implements SQL view to provide
> list
> > > of
> > > > > > > running
> > > > > > > > > > > queries.
> > > > > > > > > > > > > The
> > > > > > > > > > > > > > SQL VIEW has been a little bit discussed earlier.
> > > > > Proposed
> > > > > > > name
> > > > > > > > > is
> > > > > > > > > > > > > > *running_queries* with following columns:
> query_id,
> > > > > node_id,
> > > > > > > sql,
> > > > > > > > > > > > > > schema_name, connection_id, duration. Currently
> > most
> > > of
> > > > > the
> > > > > > > > > > > information
> > > > > > > > > > > > > can
> > > > > > > > > > > > > > be retrieved through cache API, however it
> doesn't
> > > > > matter,
> > > > > > > any
> > > > > > > > > > case
> > > > > > > > > > > we
> > > > > > > > > > > > > > need to expose SQL VIEW. Seem's you are right -
> the
> > > > part
> > > > > > > should
> > > > > > > > > be
> > > > > > > > > > > > > > implemented firstly.
> > > > > > > > > > > > > > 2) Fully agree that we need to support all kind
> of
> > > SQL
> > > > > > > queries
> > > > > > > > > > > > > > (SLECT/DML/DDL, transactional, non transnational,
> > > > local,
> > > > > > > > > > > distributed).
> > > > > > > > > > > > I
> > > > > > > > > > > > > > definitely sure that it will possible for all of
> > > above,
> > > > > > > however
> > > > > > > > > I'm
> > > > > > > > > > > not
> > > > > > > > > > > > > > sure about DDL - need to investigate it deeper.
> > Also
> > > > > need to
> > > > > > > > > > > understand
> > > > > > > > > > > > > > that canceled DML operation can lead to partially
> > > > updated
> > > > > > > data
> > > > > > > > > for
> > > > > > > > > > > non
> > > > > > > > > > > > > > transational caches.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > пн, 19 нояб. 2018 г. в 19:17, Vladimir Ozerov <
> > > > > > > > > > voze...@gridgain.com
> > > > > > > > > > > >:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Hi Yuriy,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I think we can use MANAGEMENT_POOL for this. It
> > is
> > > > > already
> > > > > > > used
> > > > > > > > > > for
> > > > > > > > > > > > > some
> > > > > > > > > > > > > > > internal Ignite tasks, and it appears to be a
> > good
> > > > > > > candidate to
> > > > > > > > > > > > process
> > > > > > > > > > > > > > > cancel requests.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > But there are several things which are not
> clear
> > > > enough
> > > > > > > for me
> > > > > > > > > at
> > > > > > > > > > > the
> > > > > > > > > > > > > > > moment:
> > > > > > > > > > > > > > > 1) How user is going to get the list of running
> > > > > queries in
> > > > > > > the
> > > > > > > > > > > first
> > > > > > > > > > > > > > place?
> > > > > > > > > > > > > > > Do we already have any SQL commands/views to
> get
> > > this
> > > > > > > > > > information?
> > > > > > > > > > > > > > > 2) We need to ensure that KILL command will be
> > > > > processed
> > > > > > > > > properly
> > > > > > > > > > > by
> > > > > > > > > > > > > all
> > > > > > > > > > > > > > > kinds of SQL queries - SELECT/DML/DDL,
> > > > > non-transactional or
> > > > > > > > > > > > > > transactional,
> > > > > > > > > > > > > > > local queries and distributed queries. Will we
> be
> > > > able
> > > > > to
> > > > > > > > > support
> > > > > > > > > > > all
> > > > > > > > > > > > > > these
> > > > > > > > > > > > > > > modes?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Vladimir.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Mon, Nov 19, 2018 at 6:37 PM Юрий <
> > > > > > > > > > jury.gerzhedow...@gmail.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Hi Igniters,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Earlier we agreed about syntax KILL QUERY
> > > > > > > > > > > > > > '[node_order].[query_counter]',
> > > > > > > > > > > > > > > > e.g. KILL QUERY '25.123' for single query or
> > > KILL
> > > > > QUERY
> > > > > > > > > '25.*'
> > > > > > > > > > > for
> > > > > > > > > > > > > all
> > > > > > > > > > > > > > > > queries on the node. Which is part of IEP-29
> > > > > > > > > > > > > > > > <
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-29%3A+SQL+management+and+monitoring
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > .
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Now I want to discuss internal realization of
> > > KILL
> > > > > query
> > > > > > > > > > feature.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > My current vision is following:
> > > > > > > > > > > > > > > > After parsing, Ignite create KILL query
> command
> > > > with
> > > > > two
> > > > > > > > > > > > parameters:
> > > > > > > > > > > > > > > > nodeOrderId, nodeQryId. To determine that
> need
> > to
> > > > > kill
> > > > > > > all
> > > > > > > > > > > queries
> > > > > > > > > > > > > on a
> > > > > > > > > > > > > > > > node we can use negative value of query id,
> due
> > > to
> > > > > qry id
> > > > > > > > > > always
> > > > > > > > > > > > have
> > > > > > > > > > > > > > > > positive values.
> > > > > > > > > > > > > > > > The command process at IgniteH2Indexing as
> > native
> > > > > > > command.
> > > > > > > > > > > > > > > > By nodeOrderId we find node which initial for
> > the
> > > > > query
> > > > > > > and
> > > > > > > > > > send
> > > > > > > > > > > to
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > node new GridQueryKillRequest with nodeQryId
> to
> > > > > > > TOPIC_QUERY
> > > > > > > > > > with
> > > > > > > > > > > > not
> > > > > > > > > > > > > > > QUERY
> > > > > > > > > > > > > > > > POOL executor.
> > > > > > > > > > > > > > > > At GridReduceQueryExecutor we add support of
> > > > > processing
> > > > > > > new
> > > > > > > > > > > > > > > > GridQueryKillRequest
> > > > > > > > > > > > > > > > which just run already exists cancelQueries
> > > method
> > > > > with
> > > > > > > given
> > > > > > > > > > > qryId
> > > > > > > > > > > > > or
> > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > all qryIds which currently running at the
> node
> > in
> > > > > case at
> > > > > > > > > > initial
> > > > > > > > > > > > > KILL
> > > > > > > > > > > > > > > > QUERY parameters used star symbol.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I have a doubt which of thread pool we should
> > use
> > > > to
> > > > > > > process
> > > > > > > > > > > > > > > > GridQueryKillRequest.
> > > > > > > > > > > > > > > > My opinion it shouldn't be QUERY pool, due to
> > the
> > > > > pool
> > > > > > > can be
> > > > > > > > > > > fully
> > > > > > > > > > > > > > used
> > > > > > > > > > > > > > > by
> > > > > > > > > > > > > > > > executing queries, it such case we can't
> cancel
> > > > query
> > > > > > > > > > > immediately.
> > > > > > > > > > > > > May
> > > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > use one of already existed pool or create new
> > > one?
> > > > Or
> > > > > > > may be
> > > > > > > > > > I'm
> > > > > > > > > > > > > > mistaken
> > > > > > > > > > > > > > > > and it should use QUERY pool.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > What do you think about proposed plan of
> > > > > implementation?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > And please give comments about which of
> thread
> > > pool
> > > > > will
> > > > > > > be
> > > > > > > > > > > better
> > > > > > > > > > > > to
> > > > > > > > > > > > > > use
> > > > > > > > > > > > > > > > for kill query requests. It's small, but
> really
> > > > > important
> > > > > > > > > part
> > > > > > > > > > of
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > > realization.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Thanks.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > Живи с улыбкой! :D
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Живи с улыбкой! :D
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Живи с улыбкой! :D
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > >
> > >
> > >
> > > --
> > > Живи с улыбкой! :D
> > >
> >
>
>
> --
> Живи с улыбкой! :D
>
