Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-06 Thread Jeremy Stanley
On 2016-09-06 17:49:08 + (+), Randall, Nathan X wrote: > For the storage backing Elasticsearch data nodes, we have been > using one 500GB Cinder volume (backed by a Ceph cluster built from > DL380s filled with 1.2TB 10k SAS drives) per data node. However, > we've found that a VM with 8 vCPU

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-06 Thread Randall, Nathan X
dall From: Tim Bell mailto:tim.b...@cern.ch>> Date: Saturday, September 3, 2016 at 1:12 AM To: David Medberry mailto:openst...@medberry.net>> Cc: openstack-operators mailto:openstack-operators@lists.openstack.org>> Subject: Re: [Openstack-operators] ElasticSearch on OpenStack Thanks. Ho

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-04 Thread Nicholas Kuechler
As VW mentioned, we're running our ELK clusters on a nova undercloud. My elasticsearch cluster looks like this: * 3 master nodes * 6 client nodes (these nodes are running elasticsearch as clients only - no data storage. they handle all api requests. they're also running kibana) * 40-ish data node

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-03 Thread Tim Bell
-operators] ElasticSearch on OpenStack Nathan: The page at https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html gives you good advice on a maximum size for the elasticsearch VM's memory. Nathan: suggest you pick a flavor with 64GB RAM or less, then base other sizing t

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-02 Thread Jeremy Stanley
On 2016-09-02 12:36:27 + (+), Tim Bell wrote: > Has anyone had experience running ElasticSearch on top of OpenStack VMs ? The OpenStack community infrastructure includes a modest ElasticSearch cluster indexing CI job results, behind the http://logstash.openstack.org/ and http://status.open

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-02 Thread David Medberry
Nathan: The page at https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html gives you good advice on a maximum size for the elasticsearch VM's memory. Nathan: suggest you pick a flavor with 64GB RAM or less, then base other sizing things off of that (i.e. choose a flavor with

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-02 Thread David Medberry
omitted on more line: give java heap 30GB, and leave the rest of the memory to the OS filesystem cache so that Lucene can make best use of it. On Fri, Sep 2, 2016 at 2:15 PM, David Medberry wrote: > From Nathan (on TWC Cloud team): > > Nathan: The page at https://www.elastic.co/guide/en/elastic

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-02 Thread David Medberry
>From Nathan (on TWC Cloud team): Nathan: The page at https://www.elastic.co/guide/en/elasticsearch/guide/current/heap-sizing.html gives you good advice on a maximum size for the elasticsearch VM's memory. Nathan: suggest you pick a flavor with 64GB RAM or less, then base other sizing things off

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-02 Thread Van Leeuwen, Robert
Hi, I had some “interesting” issues in the past with sparse files on xfs with elasticsearch: http://engineering.spilgames.com/hypervisor-kernel-panics-hit-2014-sl6-5/ If you pre-allocate your files you should be good. Cheers, Robert van Leeuwen From: Tim Bell Date: Friday, September 2, 2016 a

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-02 Thread Matt Van Winkle
We are running entire ELK clusters on our undercloud. Let me see if I can point our resident metrics expert at this as well. VW Sent from my iPhone On Sep 2, 2016, at 7:50 AM, David Medberry mailto:openst...@medberry.net>> wrote: Hey Tim, We've just started this effort. I'll see if the guy r

Re: [Openstack-operators] ElasticSearch on OpenStack

2016-09-02 Thread David Medberry
Hey Tim, We've just started this effort. I'll see if the guy running the service can comment today. On Fri, Sep 2, 2016 at 6:36 AM, Tim Bell wrote: > > > Has anyone had experience running ElasticSearch on top of OpenStack VMs ? > > > > Are there any tuning recommendations ? > > > > Thanks > > Ti