Hi, Could you please show the "PATH/example-cache.xml" file.
2018-03-21 9:40 GMT+03:00 <[email protected]>: > Hi Vinokurov, > > > > Thanks for your reply. > > I try to write batches by 100 entries. > > And I got a worse result. > > The writing speed is down to 12.09 KB per second. > > Below is my code which I try to use putAll and writeAll to rewrite. > > Did I make some mistakes? > > > > > > > > Main function: > > Ignite ignite = Ignition.start("PATH/example-cache.xml"); > > IgniteCache<String, String> igniteCache = > ignite.getOrCreateCache("testCache > "); > > for(int i = 0; i < 100; i++) > > { > > parameterMap.put(Integer.toString(i), "writeAll_val"); > > } > > > > while(true) > > { > > igniteCache.putAll(parameterMap); > > } > > > > > > Write all to PostgreSQL through JDBC: > > @Override > > public void writeAll(Collection<Cache.Entry<? extends String, ? extends > String>> entries) throws CacheWriterException { > > Iterator<Cache.Entry<? extends String, ? extends String>> it = > entries.iterator(); > > Map<String, Object> parameterMap = new HashMap<>(); > > int count = 1; > > while (it.hasNext()) { > > Cache.Entry<? extends String, ? extends String> entry = it.next(); > > String valCount = "val"; > > valCount += Integer.toString(count); > > parameterMap.put(valCount, entry.getValue()); > > count++; > > it.remove(); > > } > > > > String sqlString = "INSERT INTO test_writeall(val) VALUES " > > + "(:val1),(:val2),(:val3),(: > val4),(:val5),(:val6),(:val7),(:val8),(:val9),(:val10)," > > + "(:val11),(:val12),(:val13),(: > val14),(:val15),(:val16),(:val17),(:val18),(:val19),(:val20)," > > + "(:val21),(:val22),(:val23),(: > val24),(:val25),(:val26),(:val27),(:val28),(:val29),(:val30)," > > + "(:val31),(:val32),(:val33),(: > val34),(:val35),(:val36),(:val37),(:val38),(:val39),(:val40)," > > + "(:val41),(:val42),(:val43),(: > val44),(:val45),(:val46),(:val47),(:val48),(:val49),(:val50)," > > + "(:val51),(:val52),(:val53),(: > val54),(:val55),(:val56),(:val57),(:val58),(:val59),(:val60)," > > + "(:val61),(:val62),(:val63),(: > val64),(:val65),(:val66),(:val67),(:val68),(:val69),(:val70)," > > + "(:val71),(:val72),(:val73),(: > val74),(:val75),(:val76),(:val77),(:val78),(:val79),(:val80)," > > + "(:val81),(:val82),(:val83),(: > val84),(:val85),(:val86),(:val87),(:val88),(:val89),(:val90)," > > + "(:val91),(:val92),(:val93),(: > val94),(:val95),(:val96),(:val97),(:val98),(:val99),(:val100);"; > > > > jdbcTemplate.update(sqlString, parameterMap); > > } > > > > > > > > *From:* Vinokurov Pavel [mailto:[email protected]] > *Sent:* Wednesday, March 14, 2018 5:42 PM > *To:* [email protected] > *Subject:* Re: Performance of Ignite integrating with PostgreSQL > > > > Hi, > > > > You could try to use igniteCache.putAll for write batches by 1000 > entries. > > Use following script in PostgresDBStore#writeAll method to put data into > the database: > > String sqlString = "INSERT INTO test(val) VALUES (:val1)(:val2)(:val3);"; > > > > > > 2018-03-14 11:58 GMT+03:00 <[email protected]>: > > Hi, > > I try to use Ignite to integrate with PostgreSQL. > > And I use “atop” to monitor the data write to PostgreSQL. > > Then observed that the writing speed is 1 MB per second. > > This performance is not really good. Below is my configuration and code. > Please help me to improve it. > > Thanks. > > > > There is my cache configuration: > > <bean class="org.apache.ignite. > configuration.CacheConfiguration"> > > <property name="name" value= > "testCache"/> > > <property name="cacheMode" > value="PARTITIONED"/> > > <property name="atomicityMode" > value=" ATOMIC"/> > > <property name="atomicWriteOrderMode" > value="PRIMARY"/> > > <property name="readThrough" value="true"/> > > <property name="writeThrough" > value="true"/> > > <property name="writeBehindEnabled" > value="true"/> > > > > <property name="writeBehindFlushThreadCount" > value="64"/> > > <property name="writeBehindBatchSize" > value="131072" /> > > <property name="writeBehindFlushSize" > value="131072" /> > > > > <property name="offHeapMaxMemory" value="0" /> > > <property name="cacheStoreFactory"> > > <bean class="javax.cache. > configuration.FactoryBuilder$SingletonFactory"> > > <constructor-arg> > > <bean class="com.blu.imdg.jdbc. > PostgresDBStore"> > > </bean> > > </constructor-arg> > > </bean> > > </property> > > <property name="backups" value="0"/> > > <property name="indexedTypes"> > > <list> > > > <value>java.lang.String</value> > > > <value>java.lang.String</value> > > </list> > > </property> > > </bean> > > > > > > Main function: > > Ignite ignite = Ignition.start("PATH/example-cache.xml"); > > IgniteCache<String, String> igniteCache = > ignite.getOrCreateCache("testCache > "); > > int seqint = 0; > > while(true) > > { > > igniteCache.put(Integer.toString(seqint), > "valueString"); > > seqint++; > > } > > > > > > Write behind to PostgreSQL through JDBC: > > @Override > > public void write(Cache.Entry<? extends String, ? extends String> entry) > throws CacheWriterException { > > Map<String, Object> parameterMap = new HashMap<>(); > > parameterMap.put(“val”, entry.getValue()); > > String sqlString = "INSERT INTO test(val) VALUES (:val);"; > > jdbcTemplate.update(sqlString, parameterMap); > > } > > > > > > -- > 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain > confidential information. Please do not use or disclose it in any way and > delete it if you are not the intended recipient. > > > > > > -- > > Regards > > Pavel Vinokurov > > > -- > 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain > confidential information. Please do not use or disclose it in any way and > delete it if you are not the intended recipient. > -- Regards Pavel Vinokurov
