On Fri, Jul 12, 2019 at 5:38 PM Marc Roos wrote:
>
>
> Thanks Ilya for explaining. Am I correct to understand from the link[0]
> mentioned in the issue, that because eg. I have an unhealthy state for
> some time (1 pg on a insignificant pool) I have larger osdmaps,
> triggering this issue? Or is j
On 2019-07-15T19:08:26, " Drobyshevskiy, Vladimir " wrote:
Hi Vladimir,
is this a replicated or EC pool?
> The cluster itself:
> nautilus
> 6 nodes, 7 SSD with 2 OSDs per SSD (14 OSDs in overall).
You mean 14 OSDs per node, right?
> Each node: 2x Intel Xeon E5-2665 v1 (governor = perf
Hi,
We are running ceph version 14.1.2 with cephfs only.
I just noticed that one of our pgs had scrub errors which I could repair
# ceph health detail
HEALTH_ERR 1 MDSs report slow metadata IOs; 1 MDSs report slow requests;
1 scrub errors; Possible data damage: 1 pg inconsistent
MDS_SLOW_METADAT
Can you give me a little pointer on how and where to do the verbose
logging? I am really interested in discovery why I am getting the large
osdmap message. Maybe it is because of the snapshot being created?
-Original Message-
From: Ilya Dryomov [mailto:idryo...@gmail.com]
Cc: paul.e
Hello - I have created 10 nodes ceph cluster with 14.x version. Can you
please confirm below:
Q1 - Can I create 100+ pool (or more) on the cluster? (the reason is -
creating a pool per project). Any limitation on pool creation?
Q2 - In the above pool - I use 128 PG-NUM - to start with and enable
Hi, Lars!
> is this a replicated or EC pool?
>
This is a replicated pool with size = 3.
>
> > The cluster itself:
> > nautilus
> > 6 nodes, 7 SSD with 2 OSDs per SSD (14 OSDs in overall).
>
> You mean 14 OSDs per node, right?
>
No, I mean it is 14 OSDs on 7 SSDs at all: 5 nodes with 1 SSD
On 2019-07-16T19:27:32, " Drobyshevskiy, Vladimir " wrote:
> This is a replicated pool with size = 3.
> Is it possible to check this anyhow? ping shows average latency ~0.147 ms
> which is pretty high for IB but might be reasonable (IPoIB).
This means that, barring any other overhead, you're li
Hello, Paul!
You are effectively measuring the latency with jobs=1 here (which is
> appropriate considering that the WAL of a DB is effectively limited by
> latency) and yeah, a networked file system will always be a little bit
> slower than a local disk.
>
> But I think you should be able to get
Den tis 16 juli 2019 kl 16:16 skrev M Ranga Swami Reddy <
swamire...@gmail.com>:
> Hello - I have created 10 nodes ceph cluster with 14.x version. Can you
> please confirm below:
> Q1 - Can I create 100+ pool (or more) on the cluster? (the reason is -
> creating a pool per project). Any limitation
On 7/16/19 4:11 PM, Dietmar Rieder wrote:
> Hi,
>
> We are running ceph version 14.1.2 with cephfs only.
>
> I just noticed that one of our pgs had scrub errors which I could repair
>
> # ceph health detail
> HEALTH_ERR 1 MDSs report slow metadata IOs; 1 MDSs report slow requests;
> 1 scrub erro
Den mån 15 juli 2019 kl 23:05 skrev Oscar Segarra :
> Hi Frank,
> Thanks a lot for your quick response.
> Yes, the use case that concerns me is the following:
> 1.- I bootstrap a complete cluster mons, osds, mgr, mds, nfs, etc using
> etcd as a key store
>
as a key store ... for what? Are you stu
Hi guys,
our ceph cluster is performing way less than it could, based on the disks we
are using. We could narrow it down to the storage controller (LSI SAS3008 HBA)
in combination with an SAS expander. Yesterday we had a meeting with our
hardware reseller and sale representatives of the hardwar
With the sas expander you are putting more drives on 'one port'. Just
make sure you do not create a bottle neck there, adding to many drives.
I guess this depends on the speed of the drives. Then you should be fine
not?
-Original Message-
From: Stolte, Felix [mailto:f.sto...@fz-ju
All that etcd stuff is specific to ceph-docker, it supports storing config
options in there. Most of these features are obselete with the mon config
store nowadays.
To answer the original question: all you need to recover your Ceph cluster
are the mon and osd disks, they have all the data you need
On Tue, Jul 16, 2019 at 5:43 PM Stolte, Felix
wrote:
>
> They told us, that "best practices" for ceph would be to deploy disks as
> Raid 0 consisting of one disk using a raid controller with a big writeback
> cache.
>
> Since this "best practice" is new to me, I would like to hear your opinion
>
Thanks a lot Janne,
Well, maybe I'm missunderstanding how ceph stores keyrings in etcd...
https://github.com/ceph/ceph-container/blob/master/src/daemon/config.kv.etcd.sh
bootstrap="bootstrap${array[1]}Keyring"
etcdctl "${ETCDCTL_OPTS[@]}" "${KV_TLS[@]}" set "${CLUSTER_PATH}"/"
${bootstrap}" < "$
100+ pools work fine if you can get the PG count right (auto-scaler helps,
there are some options that you'll need to tune for small-ish pools).
But it's not a "nice" setup. Have you considered using namespaces instead?
Paul
--
Paul Emmerich
Looking for help with your Ceph cluster? Contact us
Nah, it was me not running container versions.
The bootstrap keys are used to get daemons up without giving out too much
admin access to them, so my guess is that as soon as you have a cluster
going, you can always read out or create new bootstrap keys if needed
later, but they are not necessary f
Hi Paul,
That is the initial question, is it possible to recover my ceph cluster
(docker based) if I loose all information stored in the etcd...
I don't know if anyone has a clear answer to these questions..
1.- I bootstrap a complete cluster mons, osds, mgr, mds, nfs, etc using
etcd as a key s
Den tis 16 juli 2019 kl 18:15 skrev Oscar Segarra :
> Hi Paul,
> That is the initial question, is it possible to recover my ceph cluster
> (docker based) if I loose all information stored in the etcd...
> I don't know if anyone has a clear answer to these questions..
> 1.- I bootstrap a complete c
Hi,I concur with Paul. Any kind of RAID for OSD devices would be ill advised.Make sure the 3008 SAS controller is flashed with the IT firmware so it will operate in JBOD mode.Regards, KasparOp 16 juli 2019 om 17:56 schreef Paul Emmerich : On Tue, Jul 16, 2019 at 5:43 PM Stolte, Felix < f.sto...@f
Thanks for your reply..
Here, new pool creations and pg auto scale may cause rebalance..which
impact the ceph cluster performance..
Please share name space detail like how to use etc
On Tue, 16 Jul, 2019, 9:30 PM Paul Emmerich, wrote:
> 100+ pools work fine if you can get the PG count right (
Hi all,
I have multisite RGW setup with one zonegroup and two zones. Each zone has
one endpoint configured like below:
"zonegroups": [
{
...
"is_master": "true",
"endpoints": ["http://192.168.100.1:80";],
"zones": [
{
"name": "primary_1",
"endpoints": ["http://192.168.100.1:80";]
We used to have issues when a load balancer was in front of the sync
endpoints, because our http client didn't time out stalled connections.
Those are resolved in luminous, but we still recommend using the radosgw
addresses directly to avoid shoveling data through an extra proxy.
Internally, sy
Hi,
The main question is that /etc/ceph is not longer needed when you use
etcd...
docker run -d --net=host \
-v /var/lib/ceph:/var/lib/ceph \
-e MON_IP=192.168.0.20 \
-e CEPH_PUBLIC_NETWORK=192.168.0.0/24 \
-e KV_TYPE=etcd \
-e KV_IP=192.168.0.20 \
ceph/daemon mon
I'd like to know/predict the b
Greetings.
Before I reinvent the wheel has anyone written a script to maintain X number of
snapshots on a cephfs file system that can be run through cron?
I am aware of the cephfs-snap code but just wondering if there are any other
options out there.
On a related note which of these options wou
Went over this with a few clusters built last year, it seems best practices
( at least in the white papers I have read ) only use jbod configuration(
and so have we for all clusters ). I agree regarding the writeback cache
issues and not using it but that's more due to how we use hardware (
defini
On 17/7/19 1:12 am, Stolte, Felix wrote:
> Hi guys,
>
> our ceph cluster is performing way less than it could, based on the disks we
> are using. We could narrow it down to the storage controller (LSI SAS3008
> HBA) in combination with an SAS expander. Yesterday we had a meeting with our
> hardw
On 7/16/19 5:34 PM, Dietmar Rieder wrote:
> On 7/16/19 4:11 PM, Dietmar Rieder wrote:
>> Hi,
>>
>> We are running ceph version 14.1.2 with cephfs only.
>>
>> I just noticed that one of our pgs had scrub errors which I could repair
>>
>> # ceph health detail
>> HEALTH_ERR 1 MDSs report slow metadata
29 matches
Mail list logo
