Re: Spark data frames

Ok, Igniters. I will do it in a few weeks. I need time to prepare to the talk. В Пт, 16/02/2018 в 07:16 +, Dmitry Pavlov пишет: > +1 for starting new topic from Nikolay when ' Community Meeting to > Introduce Ignite Spark Data Frames' is ready to be announced. > > пт, 16 февр. 2018 г. в 9:2

[jira] [Created] (IGNITE-7731) Restarted node can

Ksenia Rybakova created IGNITE-7731: --- Summary: Restarted node can Key: IGNITE-7731 URL: https://issues.apache.org/jira/browse/IGNITE-7731 Project: Ignite Issue Type: Bug Reporte

[GitHub] ignite pull request #3439: IGNITE-7192 :: JDBC support FQDN to multiple IPs ...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3439 ---

Re: Transport compression (not store compression)

Vladimir Ozerov, I also agree that your solution is good. I will check this flag before adding a client to map of clients. If one of the nodes have the flag then the session will be marked "compressed". At the nearest time, I will provide a solution. Dmitriy Setrakyan, I will implement and test

Re: Transport compression (not store compression)

Dmitry, IMO we should not compress individual messages. Instead, we should compress stream of enqueued messages. This way we will reduce number of array copying and improve performance. Makes sense? On Fri, Feb 16, 2018 at 12:05 AM, Dmitriy Setrakyan wrote: > Vova, I think your solution is fine

Re: Batch size parameter at DataStreamerCacheUpdaters.batched()

Val, I got what do you mean. Everything inside the org.apache.ignite.internal is considered to be not public API. Best Regards, Roman -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-7732) SQL COPY: ODBC support

Vladimir Ozerov created IGNITE-7732: --- Summary: SQL COPY: ODBC support Key: IGNITE-7732 URL: https://issues.apache.org/jira/browse/IGNITE-7732 Project: Ignite Issue Type: Task Comp

[jira] [Created] (IGNITE-7733) SQL COPY: native API support

Vladimir Ozerov created IGNITE-7733: --- Summary: SQL COPY: native API support Key: IGNITE-7733 URL: https://issues.apache.org/jira/browse/IGNITE-7733 Project: Ignite Issue Type: Task

[jira] [Created] (IGNITE-7734) SQL COPY: support single quotes for file name

Vladimir Ozerov created IGNITE-7734: --- Summary: SQL COPY: support single quotes for file name Key: IGNITE-7734 URL: https://issues.apache.org/jira/browse/IGNITE-7734 Project: Ignite Issue Ty

[jira] [Created] (IGNITE-7735) SQL COPY: expose streamer options to user

Vladimir Ozerov created IGNITE-7735: --- Summary: SQL COPY: expose streamer options to user Key: IGNITE-7735 URL: https://issues.apache.org/jira/browse/IGNITE-7735 Project: Ignite Issue Type:

Attention! TeamCity https://ci.ignite.apache.org/

TeamCity server https://ci.ignite.apache.org/ will be moved to another hardware server on Saturday 17.02.18 from 22:00 MSK to 6:00 MSK. It will be unavailable during this time. -- Kind Regards Vitaliy Osipov vosi...@gridgain.com *+7 (921) 397 27 68* *gridgain.com *Powered

[jira] [Created] (IGNITE-7736) SQL COPY: streaming model for network packets instead of request-response model

Vladimir Ozerov created IGNITE-7736: --- Summary: SQL COPY: streaming model for network packets instead of request-response model Key: IGNITE-7736 URL: https://issues.apache.org/jira/browse/IGNITE-7736

Re: Apache Ignite 2.4 release

Hi all, I'd like to suggest to change default WALMode. Currently we have: DEFAULT (write and fsync), LOG_ONLY (write without fsync), BACKGROUND, NONE. It turns out that fsyncs in current DEFAULT mode significantly restricts Ignite performance. Compared to LOG_ONLY, it offers additional guarantees

Re: Transport compression (not store compression)

On Fri, Feb 16, 2018 at 4:22 AM, Vladimir Ozerov wrote: > Dmitry, > > IMO we should not compress individual messages. Instead, we should compress > stream of enqueued messages. This way we will reduce number of array > copying and improve performance. Makes sense? > Yes, it does, as long as we c

Re: Semaphore Stuck when no acquirers to assign permit

Hi Vladislav, Can you please finalize the review? I would like to include the fix in one of the nightly Ignite builds for my own project. Thanks. --AG 2018-02-01 4:47 GMT+03:00 Dmitriy Setrakyan : > Hi Tim, Vlad, > > Any update on this ticket? I looked inside, but cannot understand the > status

Re: Apache Ignite 2.4 release

Naming one of the enum constants DEFAULT was a huge mistake. Not sure how it passed a code review, but let us all be more careful going forward. I agree with Ilya. The only remedy right now is to deprecate the DEFAULT constant. D. On Fri, Feb 16, 2018 at 5:37 AM, Ilya Lantukh wrote: > Hi all,

[jira] [Created] (IGNITE-7737) SQL COPY: rename "batch_size" to "packet_size"

Vladimir Ozerov created IGNITE-7737: --- Summary: SQL COPY: rename "batch_size" to "packet_size" Key: IGNITE-7737 URL: https://issues.apache.org/jira/browse/IGNITE-7737 Project: Ignite Issue T

Re: TeamCity. Ignite RDD tests

Hi Nikolay, Ok for me. In this case we have issue to fix. I suggest to mute with comment text with reference to issue in JIRA. Sincerely, Dmitriy Pavlov пт, 16 февр. 2018 г. в 7:35, Nikolay Izhikov : > Hello, Igniters. > > I'm working on issue [1]. > > Team City doesn't collect info about scal

Re: Apache Ignite 2.4 release

BTW, Ilya, why not name the enum value FULL_SYNC instead of STRICT? On Fri, Feb 16, 2018 at 5:43 AM, Dmitriy Setrakyan wrote: > Naming one of the enum constants DEFAULT was a huge mistake. Not sure how > it passed a code review, but let us all be more careful going forward. > > I agree with Ilya

Re: TeamCity. Ignite RDD tests

Thank you, Dmitry. Can you do it for me? Or I can do it by myself? 16 февр. 2018 г. 3:00 PM пользователь "Dmitry Pavlov" написал: > Hi Nikolay, > > Ok for me. In this case we have issue to fix. > > I suggest to mute with comment text with reference to issue in JIRA. > > Sincerely, > Dmitriy Pav

Re: TeamCity. Ignite RDD tests

I think you should be able to mute any test. If not, please write to me. пт, 16 февр. 2018 г. в 15:02, Nikolay Izhikov : > Thank you, Dmitry. > > Can you do it for me? > Or I can do it by myself? > > 16 февр. 2018 г. 3:00 PM пользователь "Dmitry Pavlov" < > dpavlov@gmail.com> > написал: > > >

[GitHub] ignite pull request #3533: IGNITE-7737

GitHub user devozerov opened a pull request: https://github.com/apache/ignite/pull/3533 IGNITE-7737 You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-7737 Alternatively you can review and apply these

Re: Apache Ignite 2.4 release

From my point of view, STRICT is the best option. The name signalizes to user that this mode provides optional strict guarantees. FULL_SYNC can be messed with CacheWriteSynchronizationMode#FULL_SYNC. I don't like the idea of naming different things with same names. Best Regards, Ivan Rakov On

Re: Apache Ignite 2.4 release

What about FULL_SYNC SYNC -> default BACKGROUND NODE ? On Fri, Feb 16, 2018 at 3:09 PM, Ivan Rakov wrote: > From my point of view, STRICT is the best option. The name signalizes to > user that this mode provides optional strict guarantees. > FULL_SYNC can be messed with CacheWriteSynchronization

Re: Apache Ignite 2.4 release

typo NODE -> NONE On Fri, Feb 16, 2018 at 3:21 PM, Anton Vinogradov wrote: > What about > FULL_SYNC > SYNC -> default > BACKGROUND > NODE > ? > > On Fri, Feb 16, 2018 at 3:09 PM, Ivan Rakov wrote: > >> From my point of view, STRICT is the best option. The name signalizes to >> user that this mo

Re: Apache Ignite 2.4 release

I had idea to name old default as FSYNC, but it would be too scientific. For old DEFAULT, STRICT or STRICT_SYNC - IMO are best options, so I agree with Ivan. пт, 16 февр. 2018 г. в 15:21, Anton Vinogradov : > typo > NODE -> NONE > > On Fri, Feb 16, 2018 at 3:21 PM, Anton Vinogradov < > avinogra.

[GitHub] ignite pull request #3534: IGNITE-7718 Fixed deserialization of binary objec...

GitHub user pvinokurov opened a pull request: https://github.com/apache/ignite/pull/3534 IGNITE-7718 Fixed deserialization of binary objects in singleton map … …and set. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain

Re: Apache Ignite 2.4 release

On Fri, Feb 16, 2018 at 6:26 AM, Dmitry Pavlov wrote: > I had idea to name old default as FSYNC, but it would be too scientific. > I like FSYNC, I do not think it is too scientific. Definitely not more scientific than LOG_ONLY. For old DEFAULT, STRICT or STRICT_SYNC - IMO are best options, so I

Re: Apache Ignite 2.4 release

>> I had idea to name old default as FSYNC, but it would be too scientific. So, then it can be FSYNC, SYNC, BACKGROUND and NONE! On Fri, Feb 16, 2018 at 3:49 PM, Dmitriy Setrakyan wrote: > On Fri, Feb 16, 2018 at 6:26 AM, Dmitry Pavlov > wrote: > > > I had idea to name old default as FSYNC, b

[GitHub] ignite pull request #3535: FIx flacky .Net test ServicesTest.TestCallJavaSer...

GitHub user apopovgg opened a pull request: https://github.com/apache/ignite/pull/3535 FIx flacky .Net test ServicesTest.TestCallJavaService You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-gg-13485

Re: Apache Ignite 2.4 release

Why create a new term to define something that has already been defined? That makes sense. I'm ok with FSYNC. Anton, I don't understand why we should rename LOG_ONLY to SYNC. We started this discussion with bad naming of DEFAULT, but this has nothing to do with LOG_ONLY (even though it may be s

Re: Batch size parameter at DataStreamerCacheUpdaters.batched()

Guys, I think, it makes sense to move this receiver implementation to public API. It has much better performance, than the default one, that performs single puts. I don't see any point in adding a batch size parameter to it though, since DataStreamer already has such setting itself. Denis пт, 1

Re: Apache Ignite 2.4 release

On Fri, Feb 16, 2018 at 7:02 AM, Ivan Rakov wrote: > Why create a new term to define something that has already been defined? >> > That makes sense. I'm ok with FSYNC. > Anton, I don't understand why we should rename LOG_ONLY to SYNC. We > started this discussion with bad naming of DEFAULT, but t

Re: TeamCity. Ignite RDD tests

Hello, Dmitriy. Seems, that I haven't suffictient access to do test mute. Can you give me access? Or please, mute it. В Пт, 16/02/2018 в 12:03 +, Dmitry Pavlov пишет: > I think you should be able to mute any test. If not, please write to me. > > пт, 16 февр. 2018 г. в 15:02, Nikolay Izhikov

[GitHub] ignite pull request #3523: IGNITE-7699 ObjectBinaryProcessor improvements

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3523 ---

[GitHub] ignite pull request #3510: IGNITE-7676 Add affinity version to snapshot plug...

Github user asfgit closed the pull request at: https://github.com/apache/ignite/pull/3510 ---

[GitHub] ignite pull request #3536: IGNITE-7717 Update discovery caches on topologies...

GitHub user Jokser opened a pull request: https://github.com/apache/ignite/pull/3536 IGNITE-7717 Update discovery caches on topologies after exchanges merge You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite i

Re: Apache Ignite 2.4 release

I am OK with either FSYNC or STRICT variant. LOG_ONLY name means "log without fsync". On Fri, Feb 16, 2018 at 4:05 PM, Dmitriy Setrakyan wrote: > On Fri, Feb 16, 2018 at 7:02 AM, Ivan Rakov wrote: > > > Why create a new term to define something that has already been defined? > >> > > That make

[jira] [Created] (IGNITE-7738) Allow 'multiple statements' in thin JDBC streaming mode

Alexander Paschenko created IGNITE-7738: --- Summary: Allow 'multiple statements' in thin JDBC streaming mode Key: IGNITE-7738 URL: https://issues.apache.org/jira/browse/IGNITE-7738 Project: Ignite

Re: Apache Ignite 2.4 release

Log only mode: flushes application buffers. So, in synced mode without fsync guarantee. That's why I propose to rename it as SYNC. On Fri, Feb 16, 2018 at 4:49 PM, Ilya Lantukh wrote: > I am OK with either FSYNC or STRICT variant. > > LOG_ONLY name means "log without fsync". > > On Fri, Feb 16,

Re: Apache Ignite 2.4 release

Igniters, just to clarify, does the term LOG_ONLY mean anything in the industry or is this just an Ignite term? D. On Fri, Feb 16, 2018 at 8:03 AM, Anton Vinogradov wrote: > Log only mode: flushes application buffers. > So, in synced mode without fsync guarantee. That's why I propose to rename

Re: Apache Ignite 2.4 release

Looks like it's an Ignite term - I've never heard of it outside Ignite scope. Though, renaming existing enum value requires keeping old as deprecated. DEFAULT is confusing enough to pay this price. As for LOG_ONLY, I think we can keep it as long as it has good and definitive javadoc. Best Reg

Re: Apache Ignite 2.4 release

Well, I cannot say that I like the name LOG_ONLY, but I would vote to keep it for now, given that it is already documented in many places, blogs, and examples. D. On Fri, Feb 16, 2018 at 8:13 AM, Ivan Rakov wrote: > Looks like it's an Ignite term - I've never heard of it outside Ignite > scope.

[GitHub] ignite pull request #3537: IGNITE-7537: [UNFINISHED] SQL COPY: CSV parsing o...

GitHub user gg-shq opened a pull request: https://github.com/apache/ignite/pull/3537 IGNITE-7537: [UNFINISHED] SQL COPY: CSV parsing options The CSV parser code is not yet finished, however the rest of the code (SQL COPY command parameters) is close to completion. You can merge thi

[GitHub] ignite pull request #3533: IGNITE-7737

Github user devozerov closed the pull request at: https://github.com/apache/ignite/pull/3533 ---

[jira] [Created] (IGNITE-7739) Write about Spring Injection and IgniteSpringBean in documentation

Ilya Kasnacheev created IGNITE-7739: --- Summary: Write about Spring Injection and IgniteSpringBean in documentation Key: IGNITE-7739 URL: https://issues.apache.org/jira/browse/IGNITE-7739 Project: Ign

Re: TeamCity. Ignite RDD tests

Nikolay, I raised your permissions to Ignite Committer status. Check it out, please. -- Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

[jira] [Created] (IGNITE-7740) IGFS caches returned by cache.context().queries().sqlMetadata()

Alexey Kuznetsov created IGNITE-7740: Summary: IGFS caches returned by cache.context().queries().sqlMetadata() Key: IGNITE-7740 URL: https://issues.apache.org/jira/browse/IGNITE-7740 Project: Ign

Re: Apache Ignite 2.4 release

Igniters, Sorry for pouring oil on the flames, but from database perspective moving from FSYNC to non-FSYNC mode appears to be a mistake. When you work with database, your main expectation is that it will save your data. All production database vendor make sure that you are safe, not that you are

Re: Apache Ignite 2.4 release

Hi Vladimir, What you saying is defenetely make sence. In the same time LOG_ONLY is also safe mode, user will be able to restore system after crash. If it is not true, we should create critical ticket and fix it. Do you know other databases defaults, such as Cassandra, Oracle, Postgre? Sincerel

Re: Apache Ignite 2.4 release

Log only mode is not safe - data might be corrupted in case of system crash. Oracle - fsync, Postgres - fsync, SQL Server - fsync, Cassandra - similar to our “background”. пт, 16 февр. 2018 г. в 19:11, Dmitry Pavlov : > Hi Vladimir, > > What you saying is defenetely make sence. > > In the same ti

Re: Apache Ignite 2.4 release

Thank you. Data can't be corrupted in case crash because of WAL replay (since completed checkpoint). Physical records are used to restore probably corrupted pages in persistent store (we overwrite so called 'grey zone' - pages we don't know for sure if they have been written). Only one effect is u

[jira] [Created] (IGNITE-7741) Fix javadoc for QR factorization

Yury Babak created IGNITE-7741: -- Summary: Fix javadoc for QR factorization Key: IGNITE-7741 URL: https://issues.apache.org/jira/browse/IGNITE-7741 Project: Ignite Issue Type: Bug Compo

Re: Apache Ignite 2.4 release

What is the reason to have DEFAULT mode at all if you claim LOG_ONLY to be completely safe? :) And how it could be safe provided that without fsync we loose part of WAL itself in case of crash? пт, 16 февр. 2018 г. в 19:32, Dmitry Pavlov : > Thank you. Data can't be corrupted in case crash becau

[GitHub] ignite pull request #3522: IGNITE-7452: Make Linear SVM example for multi - ...

Github user zaleslaw closed the pull request at: https://github.com/apache/ignite/pull/3522 ---

[jira] [Created] (IGNITE-7742) AssertionError in IgniteCacheOffheapManagerImpl when Iterate Cache with expire policy and persistence

Sergey Kosarev created IGNITE-7742: -- Summary: AssertionError in IgniteCacheOffheapManagerImpl when Iterate Cache with expire policy and persistence Key: IGNITE-7742 URL: https://issues.apache.org/jira/browse/IGNI

Re: Apache Ignite 2.4 release

Guys, While we're on this topic, what is the difference between BACKGROUND and NONE in terms of semantics and provided guarantees? To me it looks like both guarantee to recover the state since last checkpoint and anything else can potentially be lost, so from user perspective they are the same. Am

Re: Apache Ignite 2.4 release

Hi, all WAL modes except NONE protects from data consistency problem (B+Tree, pages, etc), which is why I suggest to avoid saying 'corrupted' about 'unapplied updates'. Log Only and Background may cause unapplied updates in case of OS/process failures. None mode IMO is not an option in case data

Re: Batch size parameter at DataStreamerCacheUpdaters.batched()

As far as I remember, it used to be public and then was moved to internal. The main issue with these updaters was that batching is dangerous because you can get deadlocks if keys are not sorted (which is the case for BATCHED). There is also BATCHED_SORTED, but it requires keys to be Comparable and

Re: Spark data frames

That's a great idea! Nikolay, let me know if you need any help with the presentation, I will be happy to help. -Val On Fri, Feb 16, 2018 at 12:19 AM, Nikolay Izhikov wrote: > Ok, Igniters. > > I will do it in a few weeks. > I need time to prepare to the talk. > > В Пт, 16/02/2018 в 07:16 +,

Re: .NET: ISslContextFactory should be empty

Since there are no objections I have removed unused interface members. Thank you for all the replies. On Thu, Feb 15, 2018 at 12:51 AM, Pavel Tupitsyn wrote: > Igniters, Alexey, Igor, > > I'd like to discuss recently added ISslContextFactory interface. > https://github.com/apache/ignite/commit/

Re: Apache Ignite 2.4 release

I suppose some approaches used by classic databases makes no sense for Ignite. FSYNC requirement for databases has the nature of single host solution. If you have corrupted db files you have corrupted (lost) data. For Ignite the enough number of backups and the failure detecting logic can provide

Re: Apache Ignite 2.4 release

Sergey, We do not have backups by default either, so essentially we are loosing data by default. Moreover, backups are less reliable option than fsync because a lot of users cannot afford putting servers into separate power circuits, so a single power failure may easily lead to poweroff of the who

Re: Apache Ignite 2.4 release

Vova, I hear your concerns, but at the same time I know that one of the largest banks in eastern Europe is using Ignite in LOG_ONLY mode with 3 backups to move money. The rational is that the probability of failure of 4 servers at hardware level at the same time is very low. However, if the JVM pr

[jira] [Created] (IGNITE-7743) JDBC driver allows to connect to non existent schema

Valentin Kulichenko created IGNITE-7743: --- Summary: JDBC driver allows to connect to non existent schema Key: IGNITE-7743 URL: https://issues.apache.org/jira/browse/IGNITE-7743 Project: Ignite

Re: Apache Ignite 2.4 release

Classic relational databases have no choice rather than to use FSYNC by default. RDBMS is all about consistency. Distributed databases try to balance consistency and performance. For instance, why to fsync every update if there is usually 1 backup copy? This is probably why VoltDB [1] and Cassandr

Berlin and S.F. Bay Area meetups next week featuring Apache Ignite

Igniters, Next week there are a number of events featuring Apache Ignite -- including meetups in Berlin and the San Francisco Bay Area. On Tuesday there will be an excellent webinar titled, “Redis Replaced: Why Companies Now Choose Apache® Ignite™ to Improve Application Speed and Scale.

[GitHub] ignite pull request #3538: Ignite 2.3.2 cloud streaming

GitHub user alexpaschenko opened a pull request: https://github.com/apache/ignite/pull/3538 Ignite 2.3.2 cloud streaming You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-2.3.2-gg-cloud-streaming Alt