I have heard people creating 4096 but not more that. What's you reasoning behind creating such huge partitions. I have a table which contains 40+ million rows, but it has only 5 columns and I have kept the default partitions and getting good results out of it. You have 1450 columns, an interesting use case.
1) Is your app already in production? 2) Did you calculate the average size of single row. 3) How do you manage and monitor your big cluster. 4) Do you have any scripts or tools to stop,start, restart or rolling reastart the cluster or node of your cluster? I have cluster of 7 nodes, each node has a data region size of 40-50 Gb but whenever a segmentation happens i.e when Node goes of out cluster I have found it difficult to debug or investigate such issues. Most of the times I could not figure out the exact reason for sengmentation and always ended up with blaming network issue for the segmentation. If you have already figured out all these things then well and good, please share your ideas to tools with usergroup if possible. If you are doing POC and if you want 24/7 availability then I would suggest to figure out these things as well. Thanks, Prasad On Sun 27 Oct, 2019, 6:53 AM Yong Zhao <[email protected] wrote: > > Cool, thanks . > I have a quite big cluster with 100 nodes, so I guess 25600 will be good > partition? > I am using ignite as database. > I am creating a table with 20 million rows,each row has 1450 columns. > Is this a good idea? > > Thank you! > > On Sat, Oct 26, 2019 at 9:10 AM Prasad Bhalerao < > [email protected]> wrote: > >> Please check this link... >> >> >> https://apacheignite.readme.io/docs/affinity-collocation#section-affinity-function >> >> >> Example from the ignite doc. >> >> >> // Preparing Apache Ignite node configuration.IgniteConfiguration cfg = new >> IgniteConfiguration(); >> // Creating a cache configuration.CacheConfiguration cacheCfg = new >> CacheConfiguration("myCache"); >> // Creating the affinity function with custom >> setting.RendezvousAffinityFunction affFunc = new >> RendezvousAffinityFunction(); >> affFunc.setExcludeNeighbors(true); >> affFunc.setPartitions(2048); >> // Applying the affinity function >> configuration.cacheCfg.setAffinity(affFunc); >> // Setting the cache >> configuration.cfg.setCacheConfiguration(cacheCfg); >> >> >> >> On Sat 26 Oct, 2019, 7:42 PM Andrey Dolmatov <[email protected] wrote: >> >>> Try to implement you're own affinity function. If it map any key to >>> numbers from 1 to n, you have n partitions >>> >>> On Sat, Oct 26, 2019, 09:46 codeboyyong <[email protected]> wrote: >>> >>>> Hi can you please tell me how to config this ? >>>> "number of cache partitions" >>>> >>>> Thank You >>>> Yong >>>> >>>> >>>> >>>> -- >>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ >>>> >>>
