Hello,
Last week i got time to try debug crashes of these vms
Below log includes rados debug which i left last time
https://storage.habari.co.tz/index.php/s/AQEJ7tQS7epC4Zn
I have observed the following with these settings in openstack and ceph
disk_cachemodes="network=writeback" is set in op
On 11/09/2020 22:43, Shain Miley wrote:
Thank you for your answer below.
I'm not looking to reuse them as much as I am trying to control what unused
number is actually used.
For example if I have 20 osds and 2 have failed...when I replace a disk in one
server I don't want it to automatically
Hi,
is it correct that when using the orchestrator to deploy and manage a
cluster you should not use "ceph osd purge" any more as the orchestrator
then is not able to find the OSD for the "ceph orch osd rm" operation?
Regards
--
Robert Sander
Heinlein Support GmbH
Schwedter Str. 8/9b, 10119 Berl
On 11/09/2020 17:44, Mark Nelson wrote:
On 9/11/20 4:15 AM, George Shuklin wrote:
On 10/09/2020 19:37, Mark Nelson wrote:
On 9/10/20 11:03 AM, George Shuklin wrote:
...
Are there any knobs to tweak to see higher performance for
ceph-osd? I'm pretty sure it's not any kind of leveling, GC or
Well, I can see duplicate admin socket command
registration/de-registration (and the second de-registration asserts)
but don't understand how this could happen.
Would you share the full log, please?
Thanks,
Igor
On 9/11/2020 7:26 PM, Jean-Philippe Méthot wrote:
Here’s the out file, as requ
Hi David,
you might want to try to disable swap for your nodes. Look like there is
some implicit correlation between such read errors and enabled swapping.
Also wondering whether you can observe non-zero values for
"bluestore_reads_with_retries" performance counters over your OSDs. How
wide-
Hello Ceph-Users
after upgrading one of our clusters to Nautilus we noticed the x pgs not
scrubbed/deep-scrubbed in time warnings.
Through some digging we found out that it seems like the scrubbing takes place
at random and doesn't take the age of the last scrub/deep-scrub into
consideration.
I
On Mon, Sep 14, 2020 at 5:13 AM Lomayani S. Laizer wrote:
>
> Hello,
> Last week i got time to try debug crashes of these vms
>
> Below log includes rados debug which i left last time
>
> https://storage.habari.co.tz/index.php/s/AQEJ7tQS7epC4Zn
>
> I have observed the following with these settin
> mail/b875f40571f1545ff43052412a8e mtime 2020-09-06
> 16:25:53.00,
> size 63580
> mail/e87c120b19f1545ff43052412a8e mtime 2020-09-06
> 16:24:25.00,
> size 525
Hi David, How is this going. To me this looks more like deduplication
than compression. This
> mail/b875f40571f1545ff43052412a8e mtime 2020-09-06
> 16:25:53.00,
> size 63580
> mail/e87c120b19f1545ff43052412a8e mtime 2020-09-06
> 16:24:25.00,
> size 525
Hi David, How is this going? To me this looks more like deduplication
than compression. This
Hello!
We have a Ceph cluster with 30 HDD 4 TB in 6 hosts, only for RBD.
Now, we're receiving other 6 servers with 6 SSD 2 TB each and we want to
create a separate pool for RBD on SSD, and let unused and backup volumes
stays in HDD.
I have some questions:
As I am only using "replicated_ru
Samsung PM983 M.2
I want to have a separate disk for buckets index pool and all of my server
bays are full and I should use m2 storage devices. Also the bucket index
doesn't need much space so I plan to have a 6x device with replica 3 for it.
Each disk could be 240GB to not waste space but ther
There's also Micron 7300 Pro/Max. Please benchmark it like described here
https://docs.google.com/spreadsheets/d/1E9-eXjzsKboiCCX-0u0r5fAjjufLKayaut_FOPxYZjc/edit
(https://docs.google.com/spreadsheets/d/1E9-eXjzsKboiCCX-0u0r5fAjjufLKayaut_FOPxYZjc/edit)
and send me the results if you get one :)
Thanks for the sheet. I need a low space disk for my use case (around
240GB). Do you have any suggestions with M.2 and capacitors?
On Mon, Sep 14, 2020 at 6:11 PM wrote:
> There's also Micron 7300 Pro/Max. Please benchmark it like described here
> https://docs.google.com/spreadsheets/d/1E9-eXjzs
https://www.kingston.com/unitedkingdom/en/ssd/dc1000b-data-center-boot-ssd
look good for your purpose.
- Original Message -
From: "Seena Fallah"
To: "Виталий Филиппов"
Cc: "Anthony D'Atri" , "ceph-users"
Sent: Monday, September 14, 2020 2:47:14 PM
Subject: [ceph-users] Re: Choosing
I did the same, 1 or 2 years ago, creating a replicated_ruleset_hdd and
replicated_ruleset_ssd. Eventhough I did not have any ssd's on any of
the nodes at that time, adding this hdd type criteria made pg's migrate.
I thought it was strange that this happens on a hdd only cluster, so I
mention
Same happened to us two weeks ago using nautilus, although we added the rules
and storage classes.
- Am 14. Sep 2020 um 16:02 schrieb Marc Roos m.r...@f1-outsourcing.eu:
> I did the same, 1 or 2 years ago, creating a replicated_ruleset_hdd and
> replicated_ruleset_ssd. Eventhough I did not
Hi,
In an attempt to get a (test) Mimic cluster running on Ubuntu 20.04 we
are using docker with ceph-container images (ceph/daemon:latest-mimic).
Deploying monitors and mgrs works fine. If however a monitor container
gets stopped and started (i.e. docker restart) two out of three (with
exception
Thanks!
Now got the root cause. The fix is on its way...
Meanwhile you might want to try to workaround the issue via setting
"bluestore_hybrid_alloc_mem_cap" to 0 or using different allocator, e.g.
avl for bluestore_allocator (and optionally for bluefs_allocator too).
Hope this helps,
Igor
Exactly! I created a replicated-hdd rule and set it to an existing small
pool without any changes on OSDs (all HDD) and PGs starts migration...
It seems like new rules forces migrations...
El 14/9/20 a las 11:09, André Gemünd escribió:
Same happened to us two weeks ago using nautilus, although
Hello,
Please keep in mind that you can have significant operational problems if
you choose too small OSDs. Sometimes your OSDs require >40G for
osdmaps/pgmaps/... and the smaller you OSD, the more likely it will be a
problem as Ceph is totally unable to deal with full disks and break apart.
--
M
Yes I'm planning to use only 60% - 70% of my disks and pools like
buckets.index doesn't grow too much and don't need much space! I'm just
trying to make this pool faster because I see it sometimes needs 1Milion
iops and I think NVME is a good option for this pool. But finding a good
datacenter NVME
Oh, yes, I think this would have helped indeed!
Thanks for pointing it out.
Greetings
André
- Am 14. Sep 2020 um 16:48 schrieb Stefan Kooman ste...@bit.nl:
> On 2020-09-14 16:09, André Gemünd wrote:
>> Same happened to us two weeks ago using nautilus, although we added the rules
>> and sto
Thanks Stefan!
Compiling crush map by hand on production cluster makes me sweat
but we like to take risks, don't we?
El 14/9/20 a las 11:48, Stefan Kooman escribió:
On 2020-09-14 16:09, André Gemünd wrote:
Same happened to us two weeks ago using nautilus, although we added the rules
and s
Hi Igor
We'll take a look at disabling swap on the nodes and see if that improves
the situation.
Having checked across all osds we're not seeing
bluestore_reads_with_retries as anything other than a zero value. We get
the error anywhere from 3 - 10 occurrences of the error a week, but it's
usuall
Alright, here’s the full log file.
Jean-Philippe Méthot
Senior Openstack system administrator
Administrateur système Openstack sénior
PlanetHoster inc.
4414-4416 Louis B Mayer
Laval, QC, H7P 0G1, Canada
TEL : +1.514.802.1644 - Poste : 2644
FAX : +1.514.612.0678
CA/US : 1.855.774.4678
FR : 01 7
On Mon, Sep 14, 2020 at 11:40:22AM -, Johannes L wrote:
> Hello Ceph-Users
>
> after upgrading one of our clusters to Nautilus we noticed the x pgs not
> scrubbed/deep-scrubbed in time warnings.
> Through some digging we found out that it seems like the scrubbing takes
> place at random and
On 2020-09-14 16:09, André Gemünd wrote:
> Same happened to us two weeks ago using nautilus, although we added the rules
> and storage classes.
I think this post [1] from Wido den Hollander might be useful
information. That way you can avoid data movement if data is already on hdd.
Gr. Stefan
On Sun, Sep 13, 2020 at 1:26 PM wrote:
>
> Hi all,
>
> I'm using ceph Octopus version and deployed it using cephadm. The ceph
> documentation provides 2 ways for creating a new cephfs volume:
>
> 1. via "ceph fs volume create ..." - I can use this and it works fine with
> the MDS automatically
Hello.
I'm using the Nautilus Ceph version for some huge folder with approximately
1.7TB of files.I created the filesystem and started to copy files via rsync.
However, I've had to stop the process, because Ceph shows me that the new size
of the folder is almost 6TB. I double checked the repli
On 2020-09-14 17:51, Tecnologia Charne.Net wrote:
> Thanks Stefan!
> Compiling crush map by hand on production cluster makes me sweat
> but we like to take risks, don't we?
I the crushtool says it's OK, I guess it's OK ;-). But yeah, that's the
most powerful operation one can perform on a clus
On 2020-09-09 15:51, Eugen Block wrote:
> Hi Simon,
>
>> What about the idea of creating the cluster over two data centers?
>> Would it be possible to modify the crush map, so one pool gets
>> replicated over those two data centers and if one fails, the other one
>> would still be functional?
>
>
What about hardlinks, are there any of those? Are there lots of
directories or tiny (<4k) files?
Also, size=2 is not very safe. You want size=3, min_size=2 if you are
doing replication.
On Mon, Sep 14, 2020 at 6:15 PM wrote:
>
> Hello.
>
> I'm using the Nautilus Ceph version for some huge folder
Hi,
I have a 3-OSD-node Ceph cluster with 1 480GB SSD and 8 x 2TB
12Gpbs SAS HDD on each node, to provide storage to a OpenStack
cluster. Both public and cluster networks are 2x10G. WAL and DB
of each OSD is on SSD and they share the same 60GB partition.
I run fio with different combinations of o
Can you post the fio results with the ioengine using libaio? From what you
posted, it seems to me that the read test hit cache. And the write performance
was not good, the latency was too high (~35.4ms) while the numjobs and iodepth
both were 1. Did you monitor system stat on both side (VM/Compu
I suggest trying the rsync --sparse option. Typically, qcow2 files (tend to be
large) are sparse files. Without the sparse option, the files expand in their
destination.
September 14, 2020 6:15 PM, fotof...@gmail.com wrote:
> Hello.
>
> I'm using the Nautilus Ceph version for some huge folder
Here is the test inside VM.
# fio --name=test --ioengine=libaio --numjobs=1 --runtime=30 \
--direct=1 --size=2G --end_fsync=1 \
--rw=read --bs=4K --iodepth=1
test: (groupid=0, jobs=1): err= 0: pid=14615: Mon Sep 14 21:50:55 2020
read: IOPS=320
The cash app is otherwise called square cash, which is a companion to a friend
app that permits the clients to transfer assets by connecting their financial
balance. It is fundamentally a versatile app-based stage for installment
administrations. It is an a lot less difficult app when contrasted
What is your Ceph version? From the test results you posted, your environment's
performance is okay in regard of your setup. But there are definitely many
things that can be tuned to get you better number.
I normally use top, iostat, pidstat, vmstat, dstat, iperf3, blktrace, netmon,
ceph admin
39 matches
Mail list logo
