I mount cephfs in /cephfs and create a dir in it:
[root@test-0 guhailin]# ll -h
drwxr-xr-x 1 guhailin ghlclass 0 8月 13 15:01 a
scp a file into it:
[root@test-0 guhailin]# ls a/
hadoop.tar.gz
[root@test-0 guhailin]# pwd
/cephfs/user/guhailin
[root@test-0 guhailin]# stat a/
Dear community,
TL;DR: Cluster runs good with Kernel 4.13, produces slow_requests with Kernel
4.15. How to debug?
I'm running a combined Ceph / KVM cluster consisting of 6 hosts of 2 different
kinds (details at the end).
The main difference between those hosts is CPU generation (Westmere /
On Thu, Aug 9, 2018 at 6:34 PM Erik Schwalbe wrote:
>
> Hi,
>
> Unfortunately, I deleted a few files and would like to restore them.
> For ext4 I would take photorec but that not seem to work for cephfs.
>
> Is it possible to restore deleted files stored in cephfs?
In general, no. There is a sho
On Fri, Aug 10, 2018 at 10:40 AM Willem Jan Withagen wrote:
>
> Hi,
>
> The manual of dashboard suggests:
> ceph config-key set mgr/dashboard/server_addr ${MGR_IP}
>
> But that command is required after reboot.
config-key settings are persistent. The docs are probably just
telling you to
On 13/08/2018 10:51, John Spray wrote:
On Fri, Aug 10, 2018 at 10:40 AM Willem Jan Withagen wrote:
Hi,
The manual of dashboard suggests:
ceph config-key set mgr/dashboard/server_addr ${MGR_IP}
But that command is required after reboot.
config-key settings are persistent. The docs
1. Create snapshot of the image we want to backup
2. If there's a previous backup snapshot, export diff and apply it on the
backup image
3. If there's no older snapshot, just do a full backup of image
So you need incremental backup? Try look to "rbd2qcow2" [1]
[1] https://github.com/socketpai
Hi,
Can you benchmark your Optane 900P with `fio -fsync=1 -direct=1 -bs=4k
-rw=randwrite -runtime=60`?
It's really interesting to see how much iops it will provide for ceph :)
--
With best regards,
Vitaliy Filippov
___
ceph-users mailing list
ceph
On 08/11/2018 07:56 AM, Paweł Sadowski wrote:
On 08/10/2018 06:24 PM, Gregory Farnum wrote:
On Fri, Aug 10, 2018 at 4:53 AM, Paweł Sadowsk wrote:
On 08/09/2018 04:39 PM, Alex Elder wrote:
On 08/09/2018 08:15 AM, Sage Weil wrote:
On Thu, 9 Aug 2018, Piotr Dałek wrote:
Hello,
At OVH we're
Hi,
Already tried zapping the disk. Unfortunaltely the same segfaults keep
me from adding the OSD back to the cluster.
I wanted to open an issue on tracker.ceph.com but I can't find the
"new issue" button.
---
Alex Cucu
On Mon, Aug 13, 2018 at 8:24 AM wrote:
>
>
>
> Am 3. August 2018 12:03:17
Hi,
Recently, the cluster runs healthy, but I get warning messages everyday:
2018-08-13 17:39:23.682213 [INF] Cluster is now healthy
2018-08-13 17:39:23.682144 [INF] Health check cleared: MDS_CLIENT_RECALL
(was: 6 clients failing to respond to cache pressure)
2018-08-13 17:39:23.052022 [INF] MD
Hi Paul,
thanks, I'll give it a try.. do you think this might head to
upstream soon? for some reason I can't review comments for
this patch on github.. Is some new version of this patch
on the way, or can I try to apply this one to latest luminous?
thanks a lot!
nik
On Fri, Aug 10, 2018 at 06
On Wed, Aug 1, 2018 at 4:33 AM, Jake Grimmett wrote:
> Dear All,
>
> Not sure if this is a bug, but when I add Intel Optane 900P drives,
> their device class is automatically set to SSD rather than NVME.
Not sure if we can't really tell apart SSDs from NVMe devices, but you
can use the --crush-de
On 08/13/2018 01:22 PM, Zhenshi Zhou wrote:
> Hi,
> Recently, the cluster runs healthy, but I get warning messages everyday:
>
Which version of Ceph? Which version of clients?
Can you post:
$ ceph versions
$ ceph features
$ ceph fs status
Wido
> 2018-08-13 17:39:23.682213 [INF] Cluster is
Hi,
I created a rule with this command:
ceph osd crush rule create-replicated rrd default datacenter
Since chooseleaf type is 1, I expected it to distribute the copies
evenly on two datacenters with six hosts each. For example, six copies
would mean a copy on each host.
When I test the resultin
Any suggestion on this please
Regards
Surya Balan
On Fri, Aug 10, 2018 at 11:28 AM, Surya Bala
wrote:
> Hi folks,
>
> I was trying to test the below case
>
> Having pool with replication count as 1 and if one osd goes down, then the
> PGs mapped to that OSD become stale.
>
> If the hardware fa
On a recent Luminous cluster, with nvme*n1 devices, the class is
automatically set as "nvme" on "Intel SSD DC P3520 Series" :
~# ceph osd tree
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF
-1 2.15996 root default
-9 0.71999 roo
"Don't run with replication 1 ever".
Even if this is a test, it tests something for which a resilient cluster is
specifically designed to avoid.
As for enumerating what data is missing, it would depend on if the pool(s)
had cephfs, rbd images or rgw data in them.
When this kind of data loss happe
Dear ceph users,
I set up a new cluster:
- Debian stretch
- ceph 12.2.7
- 3 nodes with mixed mon/osd
- 4 hdd 4TB osd per nodes
- 2 SSDs per nodes shared among osds for db/wal
- each OSD alone in a raid0+WriteBack
Inside a VM I get really good writes(200MB/s, 5k iops for direct 4K rand
writes),
Is this a clean (new) cluster and RBD image you are using for your test or
has it been burned in? When possible (i.e. it has enough free space),
bluestore will essentially turn your random RBD image writes into
sequential writes. This optimization doesn't work for random reads unless
your read patt
Hi,
Finally, I got a running server with files /sys/kernel/debug/ceph/xxx/
[root@docker27 525c4413-7a08-40ca-9a98-0a6df009025b.client213522]# cat mdsc
[root@docker27 525c4413-7a08-40ca-9a98-0a6df009025b.client213522]# cat monc
have monmap 2 want 3+
have osdmap 4545 want 4546
have fsmap.user 0
have
On Mon, Aug 13, 2018 at 2:49 PM Nikola Ciprich
wrote:
>
> Hi Paul,
>
> thanks, I'll give it a try.. do you think this might head to
> upstream soon? for some reason I can't review comments for
> this patch on github.. Is some new version of this patch
> on the way, or can I try to apply this one
Le 13/08/2018 à 15:21, Jason Dillaman a écrit :
> Is this a clean (new) cluster and RBD image you are using for your
> test or has it been burned in? When possible (i.e. it has enough free
> space), bluestore will essentially turn your random RBD image writes
> into sequential writes. This optimiza
Hi,
On 08/13/2018 03:22 PM, Zhenshi Zhou wrote:
Hi,
Finally, I got a running server with files /sys/kernel/debug/ceph/xxx/
[root@docker27 525c4413-7a08-40ca-9a98-0a6df009025b.client213522]# cat mdsc
[root@docker27 525c4413-7a08-40ca-9a98-0a6df009025b.client213522]# cat monc
have monmap 2 want
Hi,
Depending on your kernel (memory leaks with CephFS) increasing the
mds_cache_memory_limit could be of help. What is your current setting
now?
ceph:~ # ceph daemon mds. config show | grep mds_cache_memory_limit
We had these messages for months, almost every day.
It would occur when hour
Hi Burkhard,
I'm sure the user has permission to read and write. Besides, we're not
using EC data pools.
Now the situation is that any openration to a specific file, the command
will hang.
Operations to any other files won't hang.
Burkhard Linke
于2018年8月13日周一 下午9:42写道：
> Hi,
>
>
> On 08/13/2018
On Mon, Aug 13, 2018 at 9:32 AM Emmanuel Lacour
wrote:
> Le 13/08/2018 à 15:21, Jason Dillaman a écrit :
> > Is this a clean (new) cluster and RBD image you are using for your
> > test or has it been burned in? When possible (i.e. it has enough free
> > space), bluestore will essentially turn you
Le 13/08/2018 à 15:55, Jason Dillaman a écrit :
>
>
>
> so problem seems located on "rbd" side ...
>
>
> That's a pretty big apples-to-oranges comparison (4KiB random IO to
> 4MiB full-object IO). With your RBD workload, the OSDs will be seeking
> after each 4KiB read but w/ your RADOS bench w
Hi Eugen,
The command shows "mds_cache_memory_limit": "1073741824".
And I'll increase the cache size for a try.
Thanks
Eugen Block 于2018年8月13日周一 下午9:48写道：
> Hi,
>
> Depending on your kernel (memory leaks with CephFS) increasing the
> mds_cache_memory_limit could be of help. What is your current
Hi Wido,
The server and client use the same version, 12.2.5. And the command shows:
# ceph versions
{
"mon": {
"ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a)
luminous (stable)": 3
},
"mgr": {
"ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a
On Mon, Aug 6, 2018 at 8:17 PM Ilya Dryomov wrote:
>
> On Mon, Aug 6, 2018 at 8:13 PM Ilya Dryomov wrote:
> >
> > On Thu, Jul 26, 2018 at 1:55 AM Alex Gorbachev
> > wrote:
> > >
> > > On Wed, Jul 25, 2018 at 7:07 PM, Alex Gorbachev
> > > wrote:
> > > > On Wed, Jul 25, 2018 at 6:07 PM, Alex Go
On Mon, Aug 13, 2018 at 10:01 AM Emmanuel Lacour
wrote:
> Le 13/08/2018 à 15:55, Jason Dillaman a écrit :
>
>
>
>>
>> so problem seems located on "rbd" side ...
>>
>
> That's a pretty big apples-to-oranges comparison (4KiB random IO to 4MiB
> full-object IO). With your RBD workload, the OSDs wil
Le 13/08/2018 à 16:29, Jason Dillaman a écrit :
>
> For such a small benchmark (2 GiB), I wouldn't be surprised if you are
> not just seeing the Filestore-backed OSDs hitting the page cache for
> the reads whereas the Bluestore-backed OSDs need to actually hit the
> disk. Are the two clusters simil
Le 13/08/2018 à 16:29, Jason Dillaman a écrit :
>
>
> For such a small benchmark (2 GiB), I wouldn't be surprised if you are
> not just seeing the Filestore-backed OSDs hitting the page cache for
> the reads whereas the Bluestore-backed OSDs need to actually hit the
> disk. Are the two clusters sim
Le 13/08/2018 à 16:44, Emmanuel Lacour a écrit :
> Le 13/08/2018 à 16:29, Jason Dillaman a écrit :
>>
>>
>> For such a small benchmark (2 GiB), I wouldn't be surprised if you
>> are not just seeing the Filestore-backed OSDs hitting the page cache
>> for the reads whereas the Bluestore-backed OSDs n
On Mon, Aug 13, 2018 at 10:44 AM Emmanuel Lacour
wrote:
> Le 13/08/2018 à 16:29, Jason Dillaman a écrit :
>
>
>
> For such a small benchmark (2 GiB), I wouldn't be surprised if you are not
> just seeing the Filestore-backed OSDs hitting the page cache for the reads
> whereas the Bluestore-backed
On Fri, Aug 10, 2018 at 8:29 AM Sage Weil wrote:
> On Fri, 10 Aug 2018, Paweł Sadowski wrote:
> > On 08/09/2018 04:39 PM, Alex Elder wrote:
> > > On 08/09/2018 08:15 AM, Sage Weil wrote:
> > >> On Thu, 9 Aug 2018, Piotr Dałek wrote:
> > >>> Hello,
> > >>>
> > >>> At OVH we're heavily utilizing sn
Hi Ilya,
hmm, OK, I'm not sure now whether this is the bug which I'm
experiencing.. I've had read_partial_message / bad crc/signature
problem occurance on the second cluster in short period even though
we're on the same ceph version (12.2.5) for quite long time (almost since
its release), so it'
On Sun, Aug 12, 2018 at 12:13 AM Glen Baars
wrote:
> Hello Jason,
>
>
>
> Interesting, I used ‘rados ls’ to view the SSDPOOL and can’t see any
> objects. Is this the correct way to view the journal objects?
>
You won't see any journal objects in the SSDPOOL until you issue a write:
$ rbd create
On Tue, Jul 31, 2018 at 11:10 AM Ilja Slepnev wrote:
> Hi,
>
> is it possible to establish RBD mirroring between replicated and erasure
> coded pools?
> I'm trying to setup replication as described on
> http://docs.ceph.com/docs/master/rbd/rbd-mirroring/ without success.
> Ceph 12.2.5 Luminous
>
On Mon, Aug 13, 2018 at 10:25 AM, Ilya Dryomov wrote:
> On Mon, Aug 6, 2018 at 8:17 PM Ilya Dryomov wrote:
>>
>> On Mon, Aug 6, 2018 at 8:13 PM Ilya Dryomov wrote:
>> >
>> > On Thu, Jul 26, 2018 at 1:55 AM Alex Gorbachev
>> > wrote:
>> > >
>> > > On Wed, Jul 25, 2018 at 7:07 PM, Alex Gorbachev
Hi,
I am in the process of deploying CEPPH mimic on a few DELL R620/630 servers
These servers have only 8 disk slots and the larger disk they accept is 2
TB
Should I share a SSD drive between OS and WAL/DB or ran OS on internal SD
cards and dedicated SSD to DB/WAL only ?
Another option is to use
Hi Steven,
If you are running OSDs on the SD card, there would be nothing technically
stopping this setup, but the main factors against would be the simple endurance
and performance of SD cards and the potential fallout when they inevitably
fail. If you factor time and maintenance as a cost
Am 7. August 2018 18:08:05 MESZ schrieb John Petrini :
>Hi All,
Hi John,
>
>Any advice?
>
I am Not sure but what i would do is to increase the PG Step by Step and always
with a value of "Power of two" i.e. 256.
Also have a look on the pg_-/pgp_num. One of this should be increased First -
Hi Steven,
Just to somewhat clarify my previous post, I mention OSDs in the sense that the
OS is installed on the OSD server using the SD card, I would absolutely
recommend against using SD cards as the actual OSD media. This of course misses
another point, which is for the Mons or other suc
Jewel is almost EOL.
It looks similar to several related issues, one of which is
http://tracker.ceph.com/issues/21826
On Mon, Aug 13, 2018 at 9:19 PM, Alexandru Cucu wrote:
> Hi,
>
> Already tried zapping the disk. Unfortunaltely the same segfaults keep
> me from adding the OSD back to the clust
Hello guys,
We are testing ceph on luminous. We found pg unexpected down, when osd down.
In one pg map 14.1f0，we got 3 osds 4,6,0. When we do up/down osds in this
order, pg will down at last. All step are waiting for peering over. We found
this happened only on luminous ,we are tested on jewel an
hi,guyswe have much snaps,and we want to clear.but can not know the create time of these snaps.I know luminous have this feature default.but for jewel and hammer,have some hack way to get the create time?or it is just impossible.Thanks
__
Den mån 13 aug. 2018 kl 23:30 skrev :
>
>
> Am 7. August 2018 18:08:05 MESZ schrieb John Petrini <
> jpetr...@coredial.com>:
> >Hi All,
>
> Hi John,
>
> >
> >Any advice?
> >
>
> I am Not sure but what i would do is to increase the PG Step by Step and
> always with a value of "Power of two" i.e. 25
48 matches
Mail list logo
