Hi, Director/PuppetDB module author here - so I might be biased ;-)
Am 26.05.2017 um 13:19 schrieb Jayapandian Ponraj:
* setup 1: I see that the import of all nodes from puppetdb (1K+) in our setup takes a huge amount of time and works only when php is given insane amount of memory(1G+). Is this normal ? has anyone tried it?
1000 nodes isn't much, but it is expensive to query the PuppetDB for all those details. It imports all facts and all assigned classes. One of those used to be very slow, I don't remember which one. Eventually the API query might be improved to benefit from new features in newer PuppetDB versions - not sure about this. I used to keep the "Nodes" import running long time ago for a while against a production environment. As far as I remember, Import took nearly 15 minutes all the times, and most time was spent on waiting for PuppetDB. This doesn't hurt Director at all as it usually runs as a background Job and not through the web frontend. We didn't experience such with a DB sized similar than your's, but of course this might put some burden on your PuppetDB. So I would suggest to not use the "Nodes" type of import, that's more kind of a playground.
* setup 2: I tried collecting the exported Host objects from puppetdb via director. In this case the import and sync is smooth but am unable to FILTER. Is there any documentation listing out the keys available to filter on? probably a few examples can help to get started
That's the way to go, I'm also doing so. A customer I work with from time to time is running it that way, exporting 2,5k hosts with quite some properties and custom vars (mostly from facts) and additionally exporting 10-20k single services the same way. Other 60k servives are then generated via Apply Rules. I do not suggest to filter at Sync time, that's a leftover for compatibility reasons from very old Director versions. Sure, it works - and the setup mentioned above is doing so. Just, some minor things might not work as you expect. Like tweaking the filter to get rid of some hosts would have no influence on the "purge" mechanism. Instead, having filters at the source is the way to go. This is easy for Import Sources like SQL/LDAP and similar, for PuppetDB this hasn't been implemented at the time being, as the initial thought was that in a perfect world it's Puppet that should filter objects by deciding whether to export them or not. So, to stop monitoring a host or a service, stop exporting it to PuppetDB and it would vanish. Our world isn't perfect, and people often show up with the desire to tweak things in Director, as a change in Puppet might take quite some time - depending on your environment. This would be worth a dedicated feature request, when creating such please also add some concrete examples of what you want to achieve and how. That would help to get a better picture of how your very own perfect world would look like ;-)
* I see that the preview when using puppetdb import ain't that great. Is there any improvement planned in this regard?
Not yet. It should however be more or less usable when working with exported resources and a defined set of properties. Cheers, Thomas NB: In case you're concerned about memory usage, please consider moving to PHP 7.x. The difference to 5.x when it comes to memory and performance is impressive. -- Thomas Gelf Principal Consultant NETWAYS GmbH | Deutschherrnstr. 15-19 | D-90429 Nuernberg Tel: +49 911 92885-0 | Fax: +49 911 92885-77 CEO: Julian Hein, Bernd Erk | AG Nuernberg HRB18461 http://www.netways.de | thomas.g...@netways.de ** OSMC 2017 - November - osmc.de ** ** OSBconf 2017 - September - osbconf.org ** _______________________________________________ icinga-users mailing list icinga-users@lists.icinga.org https://lists.icinga.org/mailman/listinfo/icinga-users
