Page Locking vs Entry-level Locking

Hi, Ignite documentation talks about entry-level locks and the page structure has a LOCK_OFFSET that I assume is used to store tag info. I have these questions. 1. Does Ignite use a lock-free implementation to lock pages and/or entries? 2. When is a page locked and when is it released?

Optimistic Locking and the Prepare Phase

Hi, The design doc below states: *" In optimistic transactions, locks are acquired on primary nodes during the "prepare" phase, then promoted to backup nodes and released once the transaction is committed. Depending on an isolation level, if Ignite detects that a version of an entry has been chan

Re: What happens if Primary Node fails during the Commit Phase

I got the answer for #3 here https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Durable+Memory+-+under+the+hood#IgniteDurableMemory-underthehood-Pages&links. I will post the remaining questions in a separate thread. On Mon, Feb 12, 2018 at 8:03 PM, John Wilson wrote: > You're always helpf

Re: IgniteSet implementation: changes required

Indeed, all sets, regardless of whether they collocated or not, share single cache, and also use onheap data structures irresistable to checkpointing/recovery. 2018-02-13 2:14 GMT+03:00 Dmitriy Setrakyan : > On Fri, Feb 9, 2018 at 6:26 PM, Andrey Kuznetsov > wrote: > > > Hi all, > > > > Current

Re: What happens if Primary Node fails during the Commit Phase

You're always helpful Val. Thanks! I have a question regarding Optimistic Locking 1. The documentation here, https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Key-Value+Transactions+Architecture, states that locks, for optimistic locking, are acquired during the "prepare" p

Re: Username/password authentication for thin clients

Taras, From what I see this license complies with Apache 2.0 and allows to do everything we want with the code. Just the license notice has to be preserved. So, I would add the source file to Ignite. — Denis > On Feb 12, 2018, at 7:57 AM, Taras Ledkov wrote: > > Colleagues, Denis, > > It w

Re: [SparkDataFrame] Query Optimization. Prototype

The testing of the SQL 99 *Core* specification is done and this page will be released once 2.4 goes live: https://apacheignite-sql.readme.io/v2.3/docs/sql-conformance However, Wiki page is already live: https://en.wikipedia.org/wiki/SQL_compliance — Denis > On Feb 12, 2018, at 12:20 AM, Vladimi

What happens if Primary Node fails during the Commit Phase

Hi, Assume the Prepare phase has completed and that the primary node has received a commit message from the coordinator. Two questions: 1. A primary node commits a transaction before it forwards a commit message to the backup nodes. True? 2. What happens if a Primary Node fails while it

Re: Ignite diagnostic (SQL system views)

Alex P, sounds like a good plan for me. Vladimir, do you have any suggestions or corrections? — Denis > On Feb 12, 2018, at 4:57 AM, Alex Plehanov wrote: > > The views engine and the first view are almost ready to merge (review > comments are resolved). Which views should we take next? My prop

Re: Username/password authentication for thin clients

Taras, How about to take a look at https://shiro.apache.org/ ? Apache Shiro™ is a powerful and easy-to-use Java security framework. It also supports Blowfish. https://shiro.apache.org/static/1.3.1/apidocs/org/apache/shiro/crypto/BlowfishCipherService.html And has Apache license. -- Alexey Kuzne

Re: IgniteSet implementation: changes required

On Fri, Feb 9, 2018 at 6:26 PM, Andrey Kuznetsov wrote: > Hi all, > > Current set implementation has significant flaw: all set data are > duplicated in onheap maps on _every_ node in order to make iterator() and > size(). For me it looks like simple yet ineffective implementation. > Currently, th

Re: Make Teamcity Green Again

Dmitriy, thanks for pushing this! We have to bring the number of the failing tests to zero. I hope that the community gets behind this effort. D. On Mon, Feb 12, 2018 at 8:26 AM, Dmitry Pavlov wrote: > Hi Folks, > > I want to resurrect this process and start to fix tests. Currently approx. > 60

Re: Apache Ignite 2.4 release

On Mon, Feb 12, 2018 at 9:22 AM, Dmitry Pavlov wrote: > Hi, > > Unfortunately, a quick fix did not give us too much performance boost. > > I'm going to implement a complete algorithm change for storing the page > identifier. But this change is quite significant and will require > re-testing. I su

[jira] [Created] (IGNITE-7681) SQL COPY: measure and fix performance if needed

Kirill Shirokov created IGNITE-7681: --- Summary: SQL COPY: measure and fix performance if needed Key: IGNITE-7681 URL: https://issues.apache.org/jira/browse/IGNITE-7681 Project: Ignite Issue

Re: [SparkDataFrame] Query Optimization. Prototype

Nikolay, When you're talking about join optimization, what exactly are you referring to? Since other parts of data frames integration are already merged, I think it's a good time to resurrect this thread? Does it make sense to review it right now? Or you want to make some more changes? -Val On

[GitHub] ignite pull request #3484: IGNITE-7451: Make Linear SVM for multi-classifica...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3484 ---

[GitHub] ignite pull request #3512: IGNITE-7626 Unify code in test which clean up per...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3512 ---

[GitHub] ignite pull request #3512: IGNITE-7626 Unify code in test which clean up per...

GitHub user EdShangGG opened a pull request: https://github.com/apache/ignite/pull/3512 IGNITE-7626 Unify code in test which clean up persistence directories You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite

[GitHub] ignite pull request #3511: Ignite 7202 1: Direct IO test problem locating

GitHub user dspavlov opened a pull request: https://github.com/apache/ignite/pull/3511 Ignite 7202 1: Direct IO test problem locating You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7202-1 Alternati

[jira] [Created] (IGNITE-7680) Ignite PDS 1 (Direct IO): failed test IgnitePdsCheckpointSimulationWithRealCpDisabledTest.testDirtyFlag()

Dmitriy Pavlov created IGNITE-7680: -- Summary: Ignite PDS 1 (Direct IO): failed test IgnitePdsCheckpointSimulationWithRealCpDisabledTest.testDirtyFlag() Key: IGNITE-7680 URL: https://issues.apache.org/jira/browse/

Re: Apache Ignite 2.4 release

Hi, Unfortunately, a quick fix did not give us too much performance boost. I'm going to implement a complete algorithm change for storing the page identifier. But this change is quite significant and will require re-testing. I suggest including https://issues.apache.org/jira/browse/IGNITE-7638 in

[GitHub] ignite pull request #1921: IGNITE-5151: Add some warning when offheap evicti...

Github user vinx13 closed the pull request at: https://github.com/apache/ignite/pull/1921 ---

[GitHub] ignite pull request #1947: IGNITE-1456: CPP: Implements Cache.Rebalance()

Github user vinx13 closed the pull request at: https://github.com/apache/ignite/pull/1947 ---

[GitHub] ignite pull request #1919: IGNITE-5179: Inconsistent return type in javadoc ...

Github user vinx13 closed the pull request at: https://github.com/apache/ignite/pull/1919 ---

[GitHub] ignite pull request #1917: IGNITE-5011: Add validation on args and show prop...

Github user vinx13 closed the pull request at: https://github.com/apache/ignite/pull/1917 ---

Re: Plugins in tests

Yakov, Agree, I created the issue for this. I will try to do it in the near future. IGNITE-7679 On Tue, Jan 30, 2018 at 8:40 AM, Yakov Zhdanov wrote: > Guys, > > When running tests from core module I see that Ignite has 2 plugins > configur

[jira] [Created] (IGNITE-7679) Move all test plugins to a separate module

Dmitriy Govorukhin created IGNITE-7679: -- Summary: Move all test plugins to a separate module Key: IGNITE-7679 URL: https://issues.apache.org/jira/browse/IGNITE-7679 Project: Ignite Issue

[GitHub] ignite pull request #3477: IGNITE-7626 Unify code in test which clean up per...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3477 ---

Re: Make Teamcity Green Again

Hi Folks, I want to resurrect this process and start to fix tests. Currently approx. 60-80 tests are failed each run on master. As one from the first point I suggest to use TC investigations for brief research of test failures. If you can help with research please create and assign investigation

[GitHub] ignite pull request #3510: IGNITE-7676 Add affinity version to snapshot plug...

GitHub user EdShangGG opened a pull request: https://github.com/apache/ignite/pull/3510 IGNITE-7676 Add affinity version to snapshot plugin stub You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7676

[jira] [Created] (IGNITE-7678) Persistence: newly created cache creates partitions in MOVING state

Alexey Goncharuk created IGNITE-7678: Summary: Persistence: newly created cache creates partitions in MOVING state Key: IGNITE-7678 URL: https://issues.apache.org/jira/browse/IGNITE-7678 Project:

[GitHub] ignite pull request #3509: GG-13266 Node which not in baseline participating...

GitHub user EdShangGG opened a pull request: https://github.com/apache/ignite/pull/3509 GG-13266 Node which not in baseline participating in snapshot GG-13344 CANCEL of RESTORE could hang You can merge this pull request into a Git repository by running: $ git pull https://githu

Re: Username/password authentication for thin clients

Colleagues, Denis, It will be great to use bcrypt for password hashing in Ignite. Could you suggest the right way to use bcrypt: 1. add 'jbcrypt' maven dependency; 2. include the single 'BCrypt.java' file to our project [1]. Does the license allow to include 'BCrypt.java' ? [1]. https://github

[GitHub] ignite pull request #3509: GG-13266 Node which not in baseline participating...

Github user EdShangGG closed the pull request at: https://github.com/apache/ignite/pull/3509 ---

[jira] [Created] (IGNITE-7677) CacheBaselineTopologyTest flaky-fails on TC

Alexey Goncharuk created IGNITE-7677: Summary: CacheBaselineTopologyTest flaky-fails on TC Key: IGNITE-7677 URL: https://issues.apache.org/jira/browse/IGNITE-7677 Project: Ignite Issue Ty

[GitHub] ignite pull request #3508: ignite-7606: unit test fix

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3508 ---

[GitHub] ignite pull request #3508: ignite-7606: unit test fix

GitHub user dspavlov opened a pull request: https://github.com/apache/ignite/pull/3508 ignite-7606: unit test fix You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7606-2 Alternatively you can review

[jira] [Created] (IGNITE-7676) Add affinity version to snapshot plugin stub

Eduard Shangareev created IGNITE-7676: - Summary: Add affinity version to snapshot plugin stub Key: IGNITE-7676 URL: https://issues.apache.org/jira/browse/IGNITE-7676 Project: Ignite Issue

[GitHub] ignite pull request #3507: IGNITE-7652: ContinuousQueryWithTransformer examp...

GitHub user nizhikov opened a pull request: https://github.com/apache/ignite/pull/3507 IGNITE-7652: ContinuousQueryWithTransformer example SelfTest for ContinuousQuery examples added. You can merge this pull request into a Git repository by running: $ git pull https://github.co

[GitHub] ignite pull request #3506: Ignite 2.1.12

GitHub user slukyano opened a pull request: https://github.com/apache/ignite/pull/3506 Ignite 2.1.12 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-2.1.12 Alternatively you can review and apply the

[jira] [Created] (IGNITE-7675) Ignite Hadoop: test failures: "Stream has been closed: IgfsOutputStreamImpl"

Dmitriy Pavlov created IGNITE-7675: -- Summary: Ignite Hadoop: test failures: "Stream has been closed: IgfsOutputStreamImpl" Key: IGNITE-7675 URL: https://issues.apache.org/jira/browse/IGNITE-7675 Proj

Re: IGNITE-6005 is ready for review

Hi Nikolay, Thanks for contribution! I will try to review your pull request by tomorrow. On Mon, Feb 12, 2018 at 11:45 AM, Nikolay Izhikov wrote: > Hello, Igniters. > > I've fix some relatively small issue(100 loc) IGNITE-6005 [1], PR [2] > The fix in DataStructuresProcessor.java > > As far as

[GitHub] ignite pull request #3505: Ignite 2.3.3

GitHub user alamar opened a pull request: https://github.com/apache/ignite/pull/3505 Ignite 2.3.3 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-2.3.3 Alternatively you can review and apply these c

[jira] [Created] (IGNITE-7674) It is possible to create BinaryObject with wrong field type and that can lead to broken Transaction (TransactionHeuristicException)

Sergey Kosarev created IGNITE-7674: -- Summary: It is possible to create BinaryObject with wrong field type and that can lead to broken Transaction (TransactionHeuristicException) Key: IGNITE-7674 URL: https://issu

Re: IGNITE-4908 is ready for review (Ignite.reentrantLock looks much slower than IgniteCache.lock)

Nikolay, Could you please review changes? On Mon, Feb 12, 2018 at 3:23 PM, Александр Меньшиков wrote: > Hi, guys! I'm waiting (more than 2 months) for CR ignite-4908 ( > https://issues.apache.org/jira/browse/IGNITE-4908 all links inside). It's > about Ignite.reentrantlock. > > 2017-11-29 12:35

[GitHub] ignite pull request #3504: Ignite 7673 2.1.7

GitHub user dkarachentsev opened a pull request: https://github.com/apache/ignite/pull/3504 Ignite 7673 2.1.7 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7673-2.1.7 Alternatively you can review

[jira] [Created] (IGNITE-7673) Remove using of Collections.UnmodifiableCollection from GridTaskSessionImpl

Dmitry Karachentsev created IGNITE-7673: --- Summary: Remove using of Collections.UnmodifiableCollection from GridTaskSessionImpl Key: IGNITE-7673 URL: https://issues.apache.org/jira/browse/IGNITE-7673

Re: Ignite diagnostic (SQL system views)

The views engine and the first view are almost ready to merge (review comments are resolved). Which views should we take next? My proposal - NODES, NODE_ATTRIBUTES, NODE_METRICS, NODE_HOSTS and NODE_ADDRESSES, since these views are clear and all topology data available on each node. Any objections?

Re: IGNITE-4908 is ready for review (Ignite.reentrantLock looks much slower than IgniteCache.lock)

Hi, guys! I'm waiting (more than 2 months) for CR ignite-4908 ( https://issues.apache.org/jira/browse/IGNITE-4908 all links inside). It's about Ignite.reentrantlock. 2017-11-29 12:35 GMT+03:00 Александр Меньшиков : > Igniters, > > I’ve implemented new fast reentrant lock within an issue[1]. > > C

[GitHub] ignite pull request #3503: Log4j configuration with DEBUG logging for Object...

GitHub user sergey-chugunov-1985 opened a pull request: https://github.com/apache/ignite/pull/3503 Log4j configuration with DEBUG logging for ObjectBinaryProcessor You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-i

[jira] [Created] (IGNITE-7672) Fix Javadoc in Java 8

Peter Ivanov created IGNITE-7672: Summary: Fix Javadoc in Java 8 Key: IGNITE-7672 URL: https://issues.apache.org/jira/browse/IGNITE-7672 Project: Ignite Issue Type: Task Affects Versions:

[jira] [Created] (IGNITE-7671) Fix .gitignore files are tracked error

Peter Ivanov created IGNITE-7671: Summary: Fix .gitignore files are tracked error Key: IGNITE-7671 URL: https://issues.apache.org/jira/browse/IGNITE-7671 Project: Ignite Issue Type: Task

[jira] [Created] (IGNITE-7670) Errors Installing Ignite OSGi Karaf features on JBoss fuse

Stefano Violi created IGNITE-7670: - Summary: Errors Installing Ignite OSGi Karaf features on JBoss fuse Key: IGNITE-7670 URL: https://issues.apache.org/jira/browse/IGNITE-7670 Project: Ignite

[jira] [Created] (IGNITE-7669) java.lang.ClassNotFoundException: org.springframework.asm.commons.EmptyVisitor in session clustering

Sanjeet Jha created IGNITE-7669: --- Summary: java.lang.ClassNotFoundException: org.springframework.asm.commons.EmptyVisitor in session clustering Key: IGNITE-7669 URL: https://issues.apache.org/jira/browse/IGNITE-7669

[GitHub] ignite pull request #3502: IGNITE-7632 Fixed NPE in update igfs metrics

GitHub user pvinokurov opened a pull request: https://github.com/apache/ignite/pull/3502 IGNITE-7632 Fixed NPE in update igfs metrics You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7632 Alternative

[GitHub] ignite pull request #3501: ignite-7648

GitHub user ascherbakoff opened a pull request: https://github.com/apache/ignite/pull/3501 ignite-7648 Revert IGNITE_ENABLE_FORCIBLE_NODE_KILL system property. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite

[GitHub] ignite pull request #3500: ignite-7649 properly setting transformer instance...

Github user zstan closed the pull request at: https://github.com/apache/ignite/pull/3500 ---

IGNITE-6005 is ready for review

Hello, Igniters. I've fix some relatively small issue(100 loc) IGNITE-6005 [1], PR [2] The fix in DataStructuresProcessor.java As far as I can see from git log Ilya Lantukh maintains this piece of code. Ilya, can you please take a look at the fix? [1] https://issues.apache.org/jira/browse/IGNIT

[GitHub] ignite pull request #2858: IGNITE-6521 Update JVM options (default and recom...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/2858 ---

Re: [SparkDataFrame] Query Optimization. Prototype

Hi Nikolay, I am not sure if ticket for DECIMAL column metadata exists. If you haven't find one under "sql" component, please feel free to create it on your own. As far as testing of joins, I think it makes sense to start working on it when we finish ANSI compliance testing which is already in pro

Re: default SQL timeout

Makes sense. Going forward we should have fine-grained resource control: max duration (timeouts), max results, etc.. On Fri, Dec 22, 2017 at 1:41 AM, Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > To my knowledge, it's only possible to set timeout on query level which > actually ca