Finally, I've changed the configuration to the following:
##
### MDS
##
[mds]
mds_cache_memory_limit = 792723456
mds_bal_mode = 1
##
### Client
##
[client]
client_cache_size = 32768
client_mount_timeout = 30
client_oc_max_obijects = 2
client_oc_size = 1048576000
client_permissio
Hi!
On a new cluster, I get the following error. All 3x mons are connected to
the same switch and ping between them works (firewalls disabled).
Mon-nodes are Ubuntu 16.04 LTS on Cep Luminous.
[ceph_deploy.mon][ERROR ] Some monitors have still not reached quorum:
[ceph_deploy.mon][ERROR ] mon03
On 02/23/2018 12:42 AM, Mike Lovell wrote:
was the pg-upmap feature used to force a pg to get mapped to a
particular osd?
Yes it was. This is a semi-production cluster where the balancer module
has been enabled with the upmap feature.
It remapped PGs it seems to OSDs on the same host.
r
Hi all,
Thanks for all your follow ups on this. The Samsung SM863a is indeed a very
good alternative, thanks!
We ordered both (SM863a & DC S4600) so we can compare.
Intel's response (I mean the lack of it) is not very promising. Allthough
we have very good experiences with Intel DC SSD's we still
I always see this:
[mon01][DEBUG ] "mons": [
[mon01][DEBUG ] {
[mon01][DEBUG ] "addr": "[fd91:462b:4243:47e::1:1]:6789/0",
[mon01][DEBUG ] "name": "mon01",
[mon01][DEBUG ] "public_addr": "[fd91:462b:4243:47e::1:1]:6789/0",
[mon01][DEBUG ] "rank": 0
[mon01]
I found a fix: It is *mandatory *to set the public network to the same
network the mons use.
Skipping this while the mon has another network interface, saves garbage to
the monmap.
- Kevin
2018-02-23 11:38 GMT+01:00 Kevin Olbrich :
> I always see this:
>
> [mon01][DEBUG ] "mons": [
> [mon01]
Sage Wrote( Tue, 2 Jan 2018 17:57:32 + (UTC)):
Hi Stefan, Mehmet,
Hi Sage,
Sorry for the *extremly late* response!
Are these clusters that were upgraded from prior versions, or fresh
luminous installs?
My Cluster was initialy installed with jewel (10.2.1) have seen some
minor updates
Hi,
at the moment we use ceph with one big rbd pool and size=4 and use a
rule to ensure that 2 copies are in each of our two rooms. This works
great for VMs. But there is some big data which should be stored online
but a bit cheaper. We think about using cephfs for it with erasure
coding and
Finally, the problem is solved by changing the whole hardware of failure
server except hard disks.
The last test which I have done before changing server was, cross
exchanging SSD disks between failure server(node A) and one of the healthy
servers(node B) and recreating the cluster.
In this test w
Hi All,
What would be the proper way to preventively replace a DB/WAL SSD (when it
is nearing it's DWPD/TBW limit and not failed yet).
It hosts DB partitions for 5 OSD's
Maybe something like:
1) ceph osd reweight 0 the 5 OSD's
2) let backfilling complete
3) destroy/remove the 5 OSD's
4) replace
A very interesting question and I would add the follow up question:
Is there an easy way to add an external DB/WAL devices to an existing
OSD?
I suspect that it might be something on the lines of:
- stop osd
- create a link in ...ceph/osd/ceph-XX/block.db to the target device
- (maybe run some
Below is ceph -s
> cluster:
> id: {id}
> health: HEALTH_WARN
> noout flag(s) set
> 260610/1068004947 objects misplaced (0.024%)
> Degraded data redundancy: 23157232/1068004947 objects degraded
> (2.168%), 332 pgs unclean, 328 pgs degraded, 328 pgs
Probably unrelated, but I do keep seeing this odd negative objects degraded
message on the fs-metadata pool:
> pool fs-metadata-ssd id 16
> -34/3 objects degraded (-1133.333%)
> recovery io 0 B/s, 89 keys/s, 2 objects/s
> client io 51289 B/s rd, 101 kB/s wr, 0 op/s rd, 0 op/s wr
Don’t mean
Here is a [1] link to a ML thread tracking some slow backfilling on
bluestore. It came down to the backfill sleep setting for them. Maybe it
will help.
[1] https://www.mail-archive.com/ceph-users@lists.ceph.com/msg40256.html
On Fri, Feb 23, 2018 at 10:46 AM Reed Dier wrote:
> Probably unrelat
On Fri, Feb 23, 2018 at 5:05 AM Dennis Benndorf <
dennis.bennd...@googlemail.com> wrote:
> Hi,
>
> at the moment we use ceph with one big rbd pool and size=4 and use a
> rule to ensure that 2 copies are in each of our two rooms. This works
> great for VMs. But there is some big data which should b
On Fri, Feb 23, 2018 at 12:54 AM, Daniel Carrasco wrote:
> client_permissions = false
Yes, this will potentially reduce checks against the MDS.
> client_quota = false
This option no longer exists since Luminous; quota enforcement is no
longer optional. However, if you don't have any quotas t
Hey,
We had one of our monitor servers die on us and I have a replacement
computer now. In between that time you have released 12.2.3 but we are
still on 12.2.2.
We are on Ubuntu servers
I see all the binaries are in the repo but your package cache only shows
12.2.3, is there a reason for not kee
i am starting a new thread. replied inlined.
On Thu, Feb 22, 2018 at 9:24 PM, Micha Krause wrote:
> Hi,
>
> Debian Packages for stretch have broken dependencies:
>
> The following packages have unmet dependencies:
> ceph-common : Depends: libleveldb1 but it is not installable
>De
The only communication on the cluster network is between osds. All other
traffic for clients, mons, MDS, etc is on the public network. The cluster
network is what the osds use to backfill, recover, sent replica copies of
your data to the secondary osds, read parts of EC objects before the
primary s
Your 6.7GB of DB partition for each 4TB osd is on the very small side of
things. It's been discussed a few times in the ML and the general use case
seems to be about 10GB DB per 1TB of osd. That would be about 40GB DB
partition for each of your osds. This general rule covers most things
except for
+1 for this. I messed up a cap on a cluster I was configuring doing this
same thing. Luckily it wasn't production and I could fix it quickly.
On Thu, Feb 22, 2018, 8:09 PM Gregory Farnum wrote:
> On Wed, Feb 21, 2018 at 10:54 AM, Enrico Kern
> wrote:
> > Hey all,
> >
> > i would suggest some ch
Caspar, it looks like your idea should work. Worst case scenario seems like
the osd wouldn't start, you'd put the old SSD back in and go back to the
idea to weight them to 0, backfilling, then recreate the osds. Definitely
with a try in my opinion, and I'd love to hear your experience after.
Nico,
There was another part to my suggestion which was to set the initial crush
weight to 0 in ceph.conf. after you add all of your osds, you could
download the crush map, weight the new osds to what they should be, and
upload the crush map to give them all the ability to take PGs at the same
time. With
23 matches
Mail list logo
