[jira] [Created] (IGNITE-9307) Node is hang when it was stopping during eviction

2018-08-17 Thread Anton Kalashnikov (JIRA)
Anton Kalashnikov created IGNITE-9307: - Summary: Node is hang when it was stopping during eviction Key: IGNITE-9307 URL: https://issues.apache.org/jira/browse/IGNITE-9307 Project: Ignite

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Dmitriy Pavlov
Hi Igniters, It is not an easy fix, so I'm not sure it is possible to do in 2.7. Offheap size is not reported by VM (it returns -1). To implement it we need totally migrate off-heap memory metrics to durable memory data. I think this issue was reported and I'll find the duplicate. Sincerely, Dm

Re: Spark SQL Table Name Resolution

2018-08-17 Thread Stuart Macdonald
Hi Dmitriy, thanks - that’s done now, Stuart. On 16 Aug 2018, at 22:23, Dmitriy Setrakyan wrote: Stuart, can you please move the ticket into PATCH_AVAILABLE state? You need to click "Submit Patch" button in Jira. D. On Wed, Aug 15, 2018 at 10:22 AM, Stuart Macdonald wrote: > Here's the init

[jira] [Created] (IGNITE-9308) Add baseline topology command to REST API

2018-08-17 Thread ARomantsov (JIRA)
ARomantsov created IGNITE-9308: -- Summary: Add baseline topology command to REST API Key: IGNITE-9308 URL: https://issues.apache.org/jira/browse/IGNITE-9308 Project: Ignite Issue Type: Bug

[GitHub] ignite pull request #4564: IGNITE-9307 Added completing eviction future if n...

2018-08-17 Thread akalash
GitHub user akalash opened a pull request: https://github.com/apache/ignite/pull/4564 IGNITE-9307 Added completing eviction future if node was stopped You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-

[GitHub] ignite pull request #1479: IGNITE-4533 GridDhtPartitionsExchangeFuture store...

2018-08-17 Thread ezhuravl
Github user ezhuravl closed the pull request at: https://github.com/apache/ignite/pull/1479 ---

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Maxim Muzafarov
Igniters, Suppose, Dmitry is talking about IGNITE-5583 [1] - `Switch non-heap memory metrics to new page memory semantics` and related previous disscustions to it [4]. Also we have some additional improvements to CacheMetrics: IGNITE-5490 [2] - `Implement replacement for obsolete CacheMetrics#get

[GitHub] ignite pull request #4539: IGNITE-9264 Lost partitions raised twice if node ...

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4539 ---

[jira] [Created] (IGNITE-9309) LocalNodeMovingPartitionsCount metrics may calculates incorrect due to processFullPartitionUpdate

2018-08-17 Thread Maxim Muzafarov (JIRA)
Maxim Muzafarov created IGNITE-9309: --- Summary: LocalNodeMovingPartitionsCount metrics may calculates incorrect due to processFullPartitionUpdate Key: IGNITE-9309 URL: https://issues.apache.org/jira/browse/IGNITE

[GitHub] ignite pull request #4507: Ignite 8926 9200

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4507 ---

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Dmitriy Pavlov
Hi Maxim, thank you for stepping in and for finding these issues. Yes, these tickets are correct. I can move https://issues.apache.org/jira/browse/IGNITE-5583 to unassigned if someone would like to implement this change. I will not have enough time to complete it in 1 month (before 2.7 release).

Continuous queries and MVCC

2018-08-17 Thread Vladimir Ozerov
Folks, As you know we are developing multi-version concurrency control for Ignite caches, which is essentially new mode for transactional cache with snapshot semantics. One of the most important applications of this mode would be fully transactional SQL The question is how to implement continuous

Re: CacheStore and ignite.close

2018-08-17 Thread Ilya Kasnacheev
Hello! It turns out there's some problem with checkpointing-WAL interference. close(true) cancels current checkpoint, and upon restart some data is missing from the cache. We are investigating this. Regards, -- Ilya Kasnacheev 2018-08-16 10:34 GMT+03:00 Alexey Goncharuk : > Ilya, > > Can yo

[jira] [Created] (IGNITE-9310) SQL: throw exception when missing cache is attempted to be created inside a transaction

2018-08-17 Thread Vladimir Ozerov (JIRA)
Vladimir Ozerov created IGNITE-9310: --- Summary: SQL: throw exception when missing cache is attempted to be created inside a transaction Key: IGNITE-9310 URL: https://issues.apache.org/jira/browse/IGNITE-9310

[jira] [Created] (IGNITE-9311) Add missing @Override annotation

2018-08-17 Thread Maxim Muzafarov (JIRA)
Maxim Muzafarov created IGNITE-9311: --- Summary: Add missing @Override annotation Key: IGNITE-9311 URL: https://issues.apache.org/jira/browse/IGNITE-9311 Project: Ignite Issue Type: Bug

[jira] [Created] (IGNITE-9312) Remove unnecessary @SuppressWarnings annotation

2018-08-17 Thread Maxim Muzafarov (JIRA)
Maxim Muzafarov created IGNITE-9312: --- Summary: Remove unnecessary @SuppressWarnings annotation Key: IGNITE-9312 URL: https://issues.apache.org/jira/browse/IGNITE-9312 Project: Ignite Issue

[GitHub] ignite pull request #4565: IGNITE-9296 should not wait if walWriter is cance...

2018-08-17 Thread macrergate
GitHub user macrergate opened a pull request: https://github.com/apache/ignite/pull/4565 IGNITE-9296 should not wait if walWriter is cancelled You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9296 Al

[jira] [Created] (IGNITE-9313) ML TF integration: killed user script or chief processes didn't restart workers

2018-08-17 Thread Stepan Pilschikov (JIRA)
Stepan Pilschikov created IGNITE-9313: - Summary: ML TF integration: killed user script or chief processes didn't restart workers Key: IGNITE-9313 URL: https://issues.apache.org/jira/browse/IGNITE-9313

[jira] [Created] (IGNITE-9314) MVCC TX: Datastreamer operations

2018-08-17 Thread Igor Seliverstov (JIRA)
Igor Seliverstov created IGNITE-9314: Summary: MVCC TX: Datastreamer operations Key: IGNITE-9314 URL: https://issues.apache.org/jira/browse/IGNITE-9314 Project: Ignite Issue Type: Task

[GitHub] ignite pull request #4564: IGNITE-9307 Added completing eviction future if n...

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4564 ---

Re: QueryDetailMetrics for cache-less SQL queries

2018-08-17 Thread Vladimir Ozerov
Query is not executed on specific cache. It is executed on many caches. On Fri, Aug 17, 2018 at 6:10 AM Dmitriy Setrakyan wrote: > But internally the SQL query still runs on some cache, no? What happens to > the metrics accumulated on that cache? > > D. > > On Thu, Aug 16, 2018, 18:51 Alexey Kuz

Re: Metrics for MVCC caches

2018-08-17 Thread Павлухин Иван
Hi team, I need you opinion again. I wrote some tests for metrics and felt a confusion with metrics count for the same key update during transaction. Imagine following case: BEGIN; INSERT INTO person(id, name) VALUES(1, 'ivan'); UPDATE person SET name = 'vanya' WHERE id = 1; COMMIT; My intuition

Re: Continuous queries and MVCC

2018-08-17 Thread Nikolay Izhikov
Hello, Vladimir. > 1) *Event ordering*. I vote to keep things as is. > 2) *Initial query* > With MVCC it is now > technically possible to query data on certain snapshot and then receive > only events happened after this snapshot I think it a usefull extension for current implementation. В Пт,

[GitHub] ignite pull request #4445: IGNITE-7701 SQL system view for node attributes

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4445 ---

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Vladimir Ozerov
Folks, We already have this: >>> PageMemory [pages=6997377] Then we can multiply it by page size and get occupied memory. Am I wrong? On Fri, Aug 17, 2018 at 12:56 PM Dmitriy Pavlov wrote: > Hi Maxim, > > thank you for stepping in and for finding these issues. Yes, these tickets > are correct.

[GitHub] ignite pull request #4543: IGNITE-9268 Check interrupted flag during locking...

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4543 ---

[GitHub] ignite pull request #4566: IGNITE-3653 Fix P2P class loading for remote filt...

2018-08-17 Thread dmekhanikov
GitHub user dmekhanikov opened a pull request: https://github.com/apache/ignite/pull/4566 IGNITE-3653 Fix P2P class loading for remote filter and filter factory in CQs. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/ap

Re: StandaloneWalRecordsIterator: support iteration from custom pointer

2018-08-17 Thread Ivan Rakov
Dmitriy, Thanks a lot for stepping in! Your help is appreciated. I've looked through your changes and left a comment in JIRA issue. Best Regards, Ivan Rakov On 16.08.2018 23:56, Dmitriy Govorukhin wrote: Ivan, I implemented this issue, please review my changes. https://reviews.ignite.apache.o

[GitHub] asfgit closed pull request #1: MTCGA-002 Build trigger timeout.

2018-08-17 Thread GitBox
asfgit closed pull request #1: MTCGA-002 Build trigger timeout. URL: https://github.com/apache/ignite-teamcity-bot/pull/1 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull r

Re: Metrics for MVCC caches

2018-08-17 Thread Andrey Mashenkov
Hi. There is another similar case, what if update actually change nothing? BEGIN; INSERT INTO person(id, name) VALUES(1, 'ivan'); UPDATE person SET name = 'ivan' WHERE id = 1; COMMIT; Should it we take into account both operations or not? I'd think current implementation is correct one as TX wi

[GitHub] ignite pull request #4281: IGNITE-8859 - Open upper Java verison bounds

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4281 ---

[jira] [Created] (IGNITE-9315) Eviction meta from near cache may appear in DHT entries

2018-08-17 Thread Stanislav Lukyanov (JIRA)
Stanislav Lukyanov created IGNITE-9315: -- Summary: Eviction meta from near cache may appear in DHT entries Key: IGNITE-9315 URL: https://issues.apache.org/jira/browse/IGNITE-9315 Project: Ignite

[GitHub] ignite pull request #4567: IGNITE-9220 Uncomment tests from internal test su...

2018-08-17 Thread alamar
GitHub user alamar opened a pull request: https://github.com/apache/ignite/pull/4567 IGNITE-9220 Uncomment tests from internal test suites. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-9220 Alter

[GitHub] ignite pull request #4549: IGNITE-9277

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4549 ---

[GitHub] ignite pull request #4477: IGNITE-9169: Cache (Deadlock Detection) suite han...

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4477 ---

[GitHub] ignite pull request #4568: IGNITE-9315: Eviction meta from near cache may ap...

2018-08-17 Thread slukyano
GitHub user slukyano opened a pull request: https://github.com/apache/ignite/pull/4568 IGNITE-9315: Eviction meta from near cache may appear in DHT entries. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite i

[GitHub] ignite pull request #4557: IGNITE-9278 Fix TensorFlow integration: Can't fin...

2018-08-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/4557 ---

Re: Continuous queries and MVCC

2018-08-17 Thread Valentin Kulichenko
Vladimir, 1. Continuous queries are asynchronous in general case, so I don't think it's even possible to provide transactional ordering, especially for the case of distributed transactions. I would leave current guarantees as-is. 2. This one might be pretty useful. If it's not very hard to do, le

Re: Continuous queries and MVCC

2018-08-17 Thread Dmitriy Setrakyan
On Fri, Aug 17, 2018 at 3:18 AM, Vladimir Ozerov wrote: > Folks, > > As you know we are developing multi-version concurrency control for Ignite > caches, which is essentially new mode for transactional cache with snapshot > semantics. One of the most important applications of this mode would be >

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Alex Plehanov
To be more precise we need to multiply it by page size with system overhead (systemPageSize). If we want to print only used offheap memory, this will solve the problem. If we want to print, for example, currently allocated (commited) offheap memory (if persistence is disabled for data region then o

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Dmitriy Pavlov
Yes, I agree. To calculate free (currently -1) we need to know total & used. Used=sysPageSize*usedPages, but overall should be obtained from all segments and chunks. So it would be a remarkable part of all mem.metrics migration (required by IGNITE-5583). пт, 17 авг. 2018 г. в 23:55, Alex Plehanov

Re: Wrong off-heap size is reported for a node

2018-08-17 Thread Denis Magda
Vova, the things are even simpler - we have this ignite.dataRegionMetrics().getPhysicalMemorySize() that returns the number equal/comparabel to pageNumber X pageSize. Igniters, if you believe that we need to do more work here then let's do it iteratively. Let's fix the off-heap occupied size the

Release policy updates

2018-08-17 Thread Denis Magda
Peter, Anton V, Igniters, The board communicated the following release policy changes: -- for new releases : -- you MUST supply a SHA-256 and/or SHA-512 file -- you SHOULD NOT supply MD5 or SHA-1 files Are we good? More details are below. *2 Release Dist Policy Changes (Q? us...@

[MTCGA]: new failures in builds [1676757] needs to be handled

2018-08-17 Thread dpavlov . tasks
Hi Ignite Developer, I am MTCGA.Bot, and I've detected some issue on TeamCity to be addressed. I hope you can help. *New Critical Failure in master Cache 4 https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache4&branch=%3Cdefault%3E&tab=buildTypeStatusDiv Changes

[MTCGA]: new failures in builds [1677097] needs to be handled

2018-08-17 Thread dpavlov . tasks
Hi Ignite Developer, I am MTCGA.Bot, and I've detected some issue on TeamCity to be addressed. I hope you can help. *New Critical Failure in master Cache 4 https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache4&branch=%3Cdefault%3E&tab=buildTypeStatusDiv Changes

[MTCGA]: new failures in builds [1677738] needs to be handled

2018-08-17 Thread dpavlov . tasks
Hi Ignite Developer, I am MTCGA.Bot, and I've detected some issue on TeamCity to be addressed. I hope you can help. *New Critical Failure in master Cache 4 https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache4&branch=%3Cdefault%3E&tab=buildTypeStatusDiv No chang

[MTCGA]: new failures in builds [1677843] needs to be handled

2018-08-17 Thread dpavlov . tasks
Hi Ignite Developer, I am MTCGA.Bot, and I've detected some issue on TeamCity to be addressed. I hope you can help. *New Critical Failure in master Cache 4 https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache4&branch=%3Cdefault%3E&tab=buildTypeStatusDiv No chang

[MTCGA]: new failures in builds [1677948] needs to be handled

2018-08-17 Thread dpavlov . tasks
Hi Ignite Developer, I am MTCGA.Bot, and I've detected some issue on TeamCity to be addressed. I hope you can help. *New Critical Failure in master Cache 4 https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Cache4&branch=%3Cdefault%3E&tab=buildTypeStatusDiv No chang