Also it makes sense to use new 2.4 version.
2018-03-22 8:37 GMT+03:00 Vinokurov Pavel <[email protected]>:
> >> IgniteCache<String, String> igniteCache = ignite.getOrCreateCache("
> testCache ");
> please, change to ignite.cache("testCache") to be sure the we use
> configuration from the file.
>
> 2018-03-22 8:19 GMT+03:00 Vinokurov Pavel <[email protected]>:
>
>> You already showed the cache configuration, but could you show jdbc
>> connection initialization
>>
>> 2018-03-22 7:59 GMT+03:00 Vinokurov Pavel <[email protected]>:
>>
>>> Hi,
>>>
>>> Could you please show the "PATH/example-cache.xml" file.
>>>
>>> 2018-03-21 9:40 GMT+03:00 <[email protected]>:
>>>
>>>> Hi Vinokurov,
>>>>
>>>>
>>>>
>>>> Thanks for your reply.
>>>>
>>>> I try to write batches by 100 entries.
>>>>
>>>> And I got a worse result.
>>>>
>>>> The writing speed is down to 12.09 KB per second.
>>>>
>>>> Below is my code which I try to use putAll and writeAll to rewrite.
>>>>
>>>> Did I make some mistakes?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Main function:
>>>>
>>>> Ignite ignite = Ignition.start("PATH/example-cache.xml");
>>>>
>>>> IgniteCache<String, String> igniteCache =
>>>> ignite.getOrCreateCache("testCache ");
>>>>
>>>> for(int i = 0; i < 100; i++)
>>>>
>>>> {
>>>>
>>>> parameterMap.put(Integer.toString(i), "writeAll_val");
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> while(true)
>>>>
>>>> {
>>>>
>>>> igniteCache.putAll(parameterMap);
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Write all to PostgreSQL through JDBC:
>>>>
>>>> @Override
>>>>
>>>> public void writeAll(Collection<Cache.Entry<? extends String, ?
>>>> extends String>> entries) throws CacheWriterException {
>>>>
>>>> Iterator<Cache.Entry<? extends String, ? extends String>> it =
>>>> entries.iterator();
>>>>
>>>> Map<String, Object> parameterMap = new HashMap<>();
>>>>
>>>> int count = 1;
>>>>
>>>> while (it.hasNext()) {
>>>>
>>>> Cache.Entry<? extends String, ? extends String> entry = it.next();
>>>>
>>>> String valCount = "val";
>>>>
>>>> valCount += Integer.toString(count);
>>>>
>>>> parameterMap.put(valCount, entry.getValue());
>>>>
>>>> count++;
>>>>
>>>> it.remove();
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>> String sqlString = "INSERT INTO test_writeall(val) VALUES "
>>>>
>>>> + "(:val1),(:val2),(:val3),(:val
>>>> 4),(:val5),(:val6),(:val7),(:val8),(:val9),(:val10),"
>>>>
>>>> + "(:val11),(:val12),(:val13),(:
>>>> val14),(:val15),(:val16),(:val17),(:val18),(:val19),(:val20),"
>>>>
>>>> + "(:val21),(:val22),(:val23),(:
>>>> val24),(:val25),(:val26),(:val27),(:val28),(:val29),(:val30),"
>>>>
>>>> + "(:val31),(:val32),(:val33),(:
>>>> val34),(:val35),(:val36),(:val37),(:val38),(:val39),(:val40),"
>>>>
>>>> + "(:val41),(:val42),(:val43),(:
>>>> val44),(:val45),(:val46),(:val47),(:val48),(:val49),(:val50),"
>>>>
>>>> + "(:val51),(:val52),(:val53),(:
>>>> val54),(:val55),(:val56),(:val57),(:val58),(:val59),(:val60),"
>>>>
>>>> + "(:val61),(:val62),(:val63),(:
>>>> val64),(:val65),(:val66),(:val67),(:val68),(:val69),(:val70),"
>>>>
>>>> + "(:val71),(:val72),(:val73),(:
>>>> val74),(:val75),(:val76),(:val77),(:val78),(:val79),(:val80),"
>>>>
>>>> + "(:val81),(:val82),(:val83),(:
>>>> val84),(:val85),(:val86),(:val87),(:val88),(:val89),(:val90),"
>>>>
>>>> + "(:val91),(:val92),(:val93),(:
>>>> val94),(:val95),(:val96),(:val97),(:val98),(:val99),(:val100);";
>>>>
>>>>
>>>>
>>>> jdbcTemplate.update(sqlString, parameterMap);
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *From:* Vinokurov Pavel [mailto:[email protected]]
>>>> *Sent:* Wednesday, March 14, 2018 5:42 PM
>>>> *To:* [email protected]
>>>> *Subject:* Re: Performance of Ignite integrating with PostgreSQL
>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>>
>>>>
>>>> You could try to use igniteCache.putAll for write batches by 1000
>>>> entries.
>>>>
>>>> Use following script in PostgresDBStore#writeAll method to put data
>>>> into the database:
>>>>
>>>> String sqlString = "INSERT INTO test(val) VALUES
>>>> (:val1)(:val2)(:val3);";
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2018-03-14 11:58 GMT+03:00 <[email protected]>:
>>>>
>>>> Hi,
>>>>
>>>> I try to use Ignite to integrate with PostgreSQL.
>>>>
>>>> And I use “atop” to monitor the data write to PostgreSQL.
>>>>
>>>> Then observed that the writing speed is 1 MB per second.
>>>>
>>>> This performance is not really good. Below is my configuration and
>>>> code. Please help me to improve it.
>>>>
>>>> Thanks.
>>>>
>>>>
>>>>
>>>> There is my cache configuration:
>>>>
>>>> <bean class="org.apache.ignite.confi
>>>> guration.CacheConfiguration">
>>>>
>>>> <property name="name" value=
>>>> "testCache"/>
>>>>
>>>> <property name="cacheMode"
>>>> value="PARTITIONED"/>
>>>>
>>>> <property name="atomicityMode"
>>>> value=" ATOMIC"/>
>>>>
>>>> <property
>>>> name="atomicWriteOrderMode" value="PRIMARY"/>
>>>>
>>>> <property name="readThrough" value="true"/>
>>>>
>>>> <property name="writeThrough"
>>>> value="true"/>
>>>>
>>>> <property name="writeBehindEnabled"
>>>> value="true"/>
>>>>
>>>>
>>>>
>>>> <property name="writeBehindFlushThreadCount"
>>>> value="64"/>
>>>>
>>>> <property name="writeBehindBatchSize"
>>>> value="131072" />
>>>>
>>>> <property name="writeBehindFlushSize"
>>>> value="131072" />
>>>>
>>>>
>>>>
>>>> <property name="offHeapMaxMemory"
>>>> value="0" />
>>>>
>>>> <property name="cacheStoreFactory">
>>>>
>>>> <bean class="javax.cache.configurati
>>>> on.FactoryBuilder$SingletonFactory">
>>>>
>>>> <constructor-arg>
>>>>
>>>> <bean class="com.blu.imdg.jdbc.Postg
>>>> resDBStore">
>>>>
>>>> </bean>
>>>>
>>>> </constructor-arg>
>>>>
>>>> </bean>
>>>>
>>>> </property>
>>>>
>>>> <property name="backups"
>>>> value="0"/>
>>>>
>>>> <property name="indexedTypes">
>>>>
>>>> <list>
>>>>
>>>>
>>>> <value>java.lang.String</value>
>>>>
>>>>
>>>> <value>java.lang.String</value>
>>>>
>>>> </list>
>>>>
>>>> </property>
>>>>
>>>> </bean>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Main function:
>>>>
>>>> Ignite ignite = Ignition.start("PATH/example-cache.xml");
>>>>
>>>> IgniteCache<String, String> igniteCache =
>>>> ignite.getOrCreateCache("testCache ");
>>>>
>>>> int seqint = 0;
>>>>
>>>> while(true)
>>>>
>>>> {
>>>>
>>>> igniteCache.put(Integer.toString(seqint),
>>>> "valueString");
>>>>
>>>> seqint++;
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Write behind to PostgreSQL through JDBC:
>>>>
>>>> @Override
>>>>
>>>> public void write(Cache.Entry<? extends String, ? extends String>
>>>> entry) throws CacheWriterException {
>>>>
>>>> Map<String, Object> parameterMap = new HashMap<>();
>>>>
>>>> parameterMap.put(“val”, entry.getValue());
>>>>
>>>> String sqlString = "INSERT INTO test(val) VALUES (:val);";
>>>>
>>>> jdbcTemplate.update(sqlString, parameterMap);
>>>>
>>>> }
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain
>>>> confidential information. Please do not use or disclose it in any way and
>>>> delete it if you are not the intended recipient.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Regards
>>>>
>>>> Pavel Vinokurov
>>>>
>>>>
>>>> --
>>>> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain
>>>> confidential information. Please do not use or disclose it in any way and
>>>> delete it if you are not the intended recipient.
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Regards
>>>
>>> Pavel Vinokurov
>>>
>>
>>
>>
>> --
>>
>> Regards
>>
>> Pavel Vinokurov
>>
>
>
>
> --
>
> Regards
>
> Pavel Vinokurov
>
--
Regards
Pavel Vinokurov
