Thanks Jony. I've loaded up an embedded database to comparable capacity and while export isn't quick, it's not nearly as slow as you're experiencing.
Fro your process list the PDB appears to be running with a max heap size (Xmx) of 1024m. Perhaps increasing this could make a difference? The export process appears to be using 192m, but I think that should be using the same JAVA_ARGS as PuppetDB itself and so could be a bug. Wyatt On Tue, Oct 7, 2014 at 11:03 AM, JonY <ethrbu...@gmail.com> wrote: > https://gist.github.com/ce60c590a0531c0b09cd.git > > > # rpm -qa | grep puppet > puppet-server-3.7.1-1.el6.noarch > puppetlabs-release-6-11.noarch > puppetdb-terminus-2.2.0-1.el6.noarch > puppet-3.7.1-1.el6.noarch > vim-puppet-2.7.20-1.el6.rf.noarch > puppetdb-2.2.0-1.el6.noarch > > Am storing 30 days of data. Yes - it's a fair bit more then the default. > > > > > On Tuesday, October 7, 2014 9:00:17 AM UTC-7, Wyatt Alt wrote: >> >> Hey JonY, >> >> Sounds interesting. What version of PuppetDB are you using? Do you have >> reports, facts, and catalogs, or only some of those? Can you paste your >> config.ini? >> >> Also can you give the output of >> >> ps aux |grep java >> top -n1 >> free >> >> in a gist maybe? >> >> Wyatt >> >> >> >> >> >> On Tue, Oct 7, 2014 at 8:30 AM, JonY <ethr...@gmail.com> wrote: >> >>> (ok - terrribly unprofessional title.. I get it). >>> >>> I'm trying to 'do the right thing' and move from the embedded DB to >>> Postgres. Following the instructions I figured I would dump out the >>> contents of the embedded DB and import this into Postgres. >>> >>> So I start 'puppetdb export --outfile <someplace>'. >>> >>> Pour some coffee. >>> >>> Watch it dump one node. >>> >>> Have some more coffee. >>> >>> Wait. >>> >>> One more node. >>> >>> Hours pass. Glaciers melt. >>> >>> 3+ hours into the process and it has dumped 10 nodes (of > 100). At this >>> rate I'm looking at about 1.5 days to get this done. Really? >>> >>> System has 16 cores, 32Gb of RAM.. barely running at idle. Tell me I >>> missed some critical parameter. >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to puppet-users...@googlegroups.com. >>> To view this discussion on the web visit https://groups.google.com/d/ >>> msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/9b735460-1722-48dc-9ad5-f34f554567f4%40googlegroups.com > <https://groups.google.com/d/msgid/puppet-users/9b735460-1722-48dc-9ad5-f34f554567f4%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAJDiH3GYBs_UYBmC69D0HxqYbt71YJ6uCT2MgXS9rr6AmrGTTg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
