[ceph-users] Dont used fqdns in "monmaptool" and "ceph-mon --mkfs"

2015-07-16 Thread Goncalo Borges
onmap /tmp/monmap --keyring /tmp/ceph.mon.keyring That simply makes things crazy in 0.94.1. Once I substituted the fqdn by simply the hostname (without the domain) it worked. Cheers Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale

[ceph-users] CephFS - Problems with the reported used space

2015-08-02 Thread Goncalo Borges
dir.entries="67" ceph.dir.files="67" ceph.dir.layout="stripe_unit=524288 stripe_count=8 object_size=4194304 pool=cephfs_dt" ceph.dir.rbytes="549755863046" ceph.dir.rctime="1438569478.09995047310" ceph.dir.rentries="68" ceph.dir.rfile

Re: [ceph-users] CephFS - Problems with the reported used space

2015-08-02 Thread Goncalo Borges
afaiu, should have an immediate effect (let us say a couple of seconds) in the system. This is not what I am experiencing where sometimes, my perception is that sizes are never updated until a new operation is triggered Cheers Goncalo On 08/03/2015 01:20 PM, Goncalo Borges wrote: Dear CephFS

Re: [ceph-users] Ceph Tech Talk Today!

2015-08-02 Thread Goncalo Borges
today at 13:00 EDT (17:00 UTC). Please stop by and hear a technical deep dive on CephFS and ask any questions you might have. Thanks! http://ceph.com/ceph-tech-talks/ direct link to the video conference: https://bluejeans.com/172084437/browser -- Goncalo Borges Research Computing ARC Centre

Re: [ceph-users] CephFS - Problems with the reported used space

2015-08-03 Thread Goncalo Borges
Hey John... First of all. thank you for the nice talks you have been giving around. See the feedback on your suggestions bellow, plus some additional questions. However, please note that in my example I am not doing only deletions but also creating and updating files, which afaiu,

Re: [ceph-users] CephFS - Problems with the reported used space

2015-08-03 Thread Goncalo Borges
space reported by a df command in this case? My naive assumption would be that a df should show as used space 512KB x 3. Is this correct? Cheers Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of

Re: [ceph-users] CephFS - Problems with the reported used space

2015-08-07 Thread Goncalo Borges
Hi All... I am still fighting with this issue. It may be something which is not properly implemented, and if that is the case, that is fine. I am still trying to understand what is the real space occupied by files in a /cephfs filesystem, reported for example by a df. Maybe I did not expla

[ceph-users] ceph osd map question / bug?

2015-08-13 Thread Goncalo Borges
sdmap e341 pool 'cephfs_dt' (5) object 'thisobjectdoesnotexist' -> pg 5.28aa7f5a (5.35a) -> up ([24,21,15], p24) acting ([24,21,15], p24) Is this expected? Are those PGs actually assigned to something it does not exists? Cheers Goncalo -- Goncalo Borges Research Co

[ceph-users] ceph osd debug question / proposal

2015-08-19 Thread Goncalo Borges
access the filesystem? For example, I can imagine situation where the filesystem may be in read-only, for example. TIA Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +61 2 9351

Re: [ceph-users] ceph osd debug question / proposal

2015-08-23 Thread Goncalo Borges
Hi Jan... Thank for the reply. Yes, I did an 'umount -l' but I was sure that no I/O was happening at the time. So, I was almost 100% sure that there were no real incoherence in terms of open files in the OS. On 08/20/2015 07:31 PM, Jan Schermer wrote: Just to clarify - you unmounted the fil

Re: [ceph-users] ceph osd debug question / proposal

2015-08-24 Thread Goncalo Borges
hit the OSD again all those open files will start working and if more need to be opened it will start looking for them... Jan On 24 Aug 2015, at 03:07, Goncalo Borges wrote: Hi Jan... Thank for the reply. Yes, I did an 'umount -l' but I was sure that no I/O was happening at the t

Re: [ceph-users] ceph osd debug question / proposal

2015-08-24 Thread Goncalo Borges
;down' and 'out' Cheers Goncalo On 08/25/2015 01:06 PM, Shinobu wrote: So what is the situation where you need to do: # cd /var/lib/ceph/osd/ceph-23/current # rm -Rf * # df (...) I'm quite sure that is not normal. Shinobu On Tue, Aug 25, 2015 at 9:41 AM, Go

Re: [ceph-users] Ceph Day Raleigh Cancelled

2015-08-25 Thread Goncalo Borges
Ceph Day events for Shanghai, Tokyo, and Melbourne should all still be proceeding as planned, however. Feel free to contact me if you have any questions about Ceph Days. Thanks. -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics

Re: [ceph-users] Ceph Day Raleigh Cancelled

2015-08-25 Thread Goncalo Borges
should all still be proceeding as planned, however. Feel free to contact me if you have any questions about Ceph Days. Thanks. On 08/26/2015 10:28 AM, nigel.d.willi...@gmail.com wrote: On 26 Aug 2015, at 9:47 am, Goncalo Borges wrote: I am interested in the Melbourne one. Hi Gonçalo, where di

[ceph-users] Question regarding degraded PGs

2015-08-26 Thread Goncalo Borges
27 active+recovering+degraded+remapped 17 active+recovery_wait+degraded 8 active+remapped+backfilling 4 active+recovering+degraded recovery io 521 MB/s, 170 objects/s Cheers Goncalo -- Goncalo Borges Research Computing ARC Centre of

Re: [ceph-users] Question regarding degraded PGs

2015-08-27 Thread Goncalo Borges
15 at 2:54 AM, Goncalo Borges wrote: Hey guys... 1./ I have a simple question regarding the appearance of degraded PGs. First, for reference: a. I am working with 0.94.2 b. I have 32 OSDs distributed in 4 servers, meaning that I have 8 OSD per server. c. Our cluster is set with 'osd po

Re: [ceph-users] Can't mount Cephfs

2015-08-27 Thread Goncalo Borges
t;mailto:ceph-users@lists.ceph.com> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com _______ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Bo

[ceph-users] Question on cephfs recovery tools

2015-09-09 Thread Goncalo Borges
pg dump_stuck stale | grep ^[12] | awk '{print $1}'`; do ceph pg $pg query; done ok Error ENOENT: i don't have pgid 1.23 Error ENOENT: i don't have pgid 2.38b (...) --- * --- 6) Create the non existing PGs # for pg in `ceph pg dump_stuck stale | grep ^[12] | awk '{print $

Re: [ceph-users] Question on cephfs recovery tools

2015-09-09 Thread Goncalo Borges
order to run them. The reason for my email is to get some clarification on that. Cheers Quoting Shinobu Kinjo : Anyhow this page would help you: http://ceph.com/docs/master/cephfs/disaster-recovery/ Shinobu - Original Message - From: "Shinobu Kinjo" To: "Gonc

Re: [ceph-users] Question on cephfs recovery tools

2015-09-14 Thread Goncalo Borges
pg dump_stuck stale | grep ^[12] | awk '{print $1}'`; do ceph pg $pg query; done ok Error ENOENT: i don't have pgid 1.23 Error ENOENT: i don't have pgid 2.38b (...) --- * --- 6) Create the non existing PGs # for pg in `ceph pg dump_stuck stale | grep ^[12] | awk '{print

Re: [ceph-users] Question on cephfs recovery tools

2015-09-14 Thread Goncalo Borges
e the one I've created The whole idea is, given a scenario where there is loss of data and metadata, what can be done from the admin side to recover the cephfs. Nevertheless, since this email thread is already long, I'll try to send a new email more focused. Cheers and Thanks for the

[ceph-users] HowTo CephgFS recovery tools?

2015-09-15 Thread Goncalo Borges
ephfs-table-tool reset commands but just one or two? e./ In what circumstances we would do a reset of the filesystem with 'ceph fs reset cephfs --yes-i-really-mean-it'? Thank you in Advance Cheers -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at t

Re: [ceph-users] cant get cluster to become healthy. "stale+undersized+degraded+peered"

2015-09-16 Thread Goncalo Borges
e a thoughts of what might be wrong? Or if there is other info I can provide to ease the search for what it might be? Thanks! ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Bor

[ceph-users] CephFS: Question how to debug client sessions

2015-09-24 Thread Goncalo Borges
us, replacing* 2015-09-25 11:09:53.675605 7f8eaa775700 1 mds.0.19 ms_verify_authorizer: cannot decode auth caps bl of length 0 -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +6

Re: [ceph-users] CephFS: Question how to debug client sessions

2015-09-24 Thread Goncalo Borges
I found a partial answer to some of the questions: 5./ My questions: - Is there a simple command for me to check which sessions are active? 'cephfs-table-tool 0 show session' does not seem to work - Is there a way for me to cross check which sessions belong to which clients (IPs)? '

[ceph-users] Annoying libust warning on ceph reload

2015-09-30 Thread Goncalo Borges
(...) liblttng-ust.so.0 => /usr/lib64/liblttng-ust.so.0 (0x00337da0) - To fix this, I had to set 'export HOME=/root' in /usr/lib64/ceph/ceph_common.sh Cheers Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terasc

[ceph-users] ceph-fuse and its memory usage

2015-10-01 Thread Goncalo Borges
S clients to limit the amount of memory they can use? Cheers Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +61 2 93511937 ___ cep

Re: [ceph-users] Annoying libust warning on ceph reload

2015-10-12 Thread Goncalo Borges
Hi Ken Here it is: http://tracker.ceph.com/issues/13470 Cheers G. On 10/09/2015 02:58 AM, Ken Dreyer wrote: On Wed, Sep 30, 2015 at 7:46 PM, Goncalo Borges wrote: - Each time logrotate is executed, we received a daily notice with the message ibust[8241/8241]: Warning: HOME environment

Re: [ceph-users] v9.1.0 Infernalis release candidate released

2015-10-13 Thread Goncalo Borges
ploy, see http://ceph.com/docs/master/install/install-ceph-deploy ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School o

Re: [ceph-users] ceph-fuse and its memory usage

2015-10-13 Thread Goncalo Borges
715200 and I've decreased both by a factor of 4 but I kept seeing the same behavior. At this point, I do not have a clear idea why this is happening. Cheers Goncalo On 10/03/2015 04:03 AM, Gregory Farnum wrote: On Fri, Oct 2, 2015 at 1:57 AM, John Spray wrote: On Fri, Oct 2, 2015 at 2:

Re: [ceph-users] v9.1.0 Infernalis release candidate released

2015-10-14 Thread Goncalo Borges
: Hi Goncalo, On Wed, Oct 14, 2015 at 6:51 AM, Goncalo Borges wrote: Hi Sage... I've seen that the rh6 derivatives have been ruled out. This is a problem in our case since the OS choice in our systems is, somehow, imposed by CERN. The experiments software is certified for SL6 and the trans

Re: [ceph-users] Ceph PGs stuck creating after running force_create_pg

2015-10-15 Thread Goncalo Borges
trying to query osds that have been lost and removed from the cluster? ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Borges Research Computing ARC Centre of Excellence for Part

Re: [ceph-users] ceph-fuse and its memory usage

2015-10-22 Thread Goncalo Borges
tedly writes data to 8M files. The cache make multiple write assimilate into single OSD write Ugh, of course. I don't see a tracker ticket for that, so I made one: http://tracker.ceph.com/issues/13569 -Greg -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Phys

[ceph-users] Suggestion: Create a DOI for ceph projects in github

2015-11-05 Thread Goncalo Borges
://github.com/blog/1840-improving-github-for-science https://guides.github.com/activities/citable-code/ That would provide a unique / standard way to cite ceph project everywhere. Cheers -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of

[ceph-users] SL6/Centos6 rebuild question

2015-11-12 Thread Goncalo Borges
RPM build errors: Bad exit status from /var/tmp/rpm-tmp.3TtsUK (%build) I remember that systemd support was introduced in the latest infernalis release, and I just wonder if that, somehow, breaks the backward compatibility with older systems. Cheers Goncalo -- Goncalo Borges R

Re: [ceph-users] SL6/Centos6 rebuild question

2015-11-13 Thread Goncalo Borges
Well... I misinterpreted the error. It is not systemd related but selinux related. I must be missing some selinux component. Will investigate better. From: Goncalo Borges [goncalo.bor...@sydney.edu.au] Sent: 13 November 2015 16:51 To: ceph-users

Re: [ceph-users] SL6/Centos6 rebuild question

2015-11-17 Thread Goncalo Borges
h.spec to avoid building debug packages which I do not really need. Cheers Goncalo On 11/13/2015 08:27 PM, Goncalo Borges wrote: Well... I misinterpreted the error. It is not systemd related but selinux related. I must be missing some selinux comp

[ceph-users] ceph-disk activate Permission denied problems

2015-12-02 Thread Goncalo Borges
t # su ceph -s /bin/bash --session-command="ceph-disk activate /dev/sdh1" Problem opening /dev/sdh for reading! Error is 13. You must run this program as root or use sudo! mount: only root can use "--options" option ceph-disk: Mounting filesystem failed: Comman

Re: [ceph-users] ceph-disk activate Permission denied problems

2015-12-03 Thread Goncalo Borges
he partition (WARNING:ceph-disk:Journal /dev/sdc3 was not prepared with ceph-disk. Symlinking directly) [1] https://en.wikipedia.org/wiki/GUID_Partition_Table#Partition_type_GUIDs -- ----- Adrien GILLARD +33 (0)6 29 06 16 31 gilla

[ceph-users] Ceph 9.2 fails to install in COS 7.1.1503: Report and Fix

2015-12-08 Thread Goncalo Borges
ld be available in Centos CR repos. For Centos 7.1.1503, it provides libunwind-1.1-5.el7.x86_64) http://mirror.centos.org/centos/7.1.1503/cr Cheers Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney

[ceph-users] CEPH/CEPHFS upgrade questions (9.2.0 ---> 10.2.1)

2016-05-18 Thread Goncalo Borges
re in the same version?* Thank you for your answers Cheers Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +61 2 93511937 ___ ceph-user

Re: [ceph-users] Jewel CephFS quota (setfattr, getfattr)

2016-05-23 Thread Goncalo Borges
= -1 ENOENT (No such file or directory) open("/usr/share/locale/en_US/LC_MESSAGES/attr.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en.UTF-8/LC_MESSAGES/attr.mo", O_RDONLY) = -1 ENOENT (No such file or directory) open("/usr/share/locale/en.

Re: [ceph-users] CEPH/CEPHFS upgrade questions (9.2.0 ---> 10.2.1)

2016-05-24 Thread Goncalo Borges
10.2.1-0.el7.x86_64.rpm rbd-fuse-10.2.1-0.el7.x86_64.rpm rbd-mirror-10.2.1-0.el7.x86_64.rpm rbd-nbd-10.2.1-0.el7.x86_64.rpm On 05/25/2016 07:45 AM, Gregory Farnum wrote: On Wed, May 18, 2016 at 6:04 PM, Goncalo Borges wrote: Dear All... Our infrastructure is the following: - We use C

[ceph-users] Jewel 10.2.1 compilation in SL6/Centos6

2016-06-08 Thread Goncalo Borges
(python27)$ make (python27)$ make install Cheers G. -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +61 2 93511937 ___ ceph-users maili

[ceph-users] fsmap question

2016-06-27 Thread Goncalo Borges
e [mds.rccephmds2] host = rccephmds2 mds standby_for_rank = rccephmds mds standby replay = true Am I doing something particularly different than what is expected? Cheers G. -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Phys

[ceph-users] ceph-mon.target and ceph-mds.target systemd dependencies in centos7

2016-06-27 Thread Goncalo Borges
[Install] WantedBy=ceph-mon.target Am i the only one seeing the issue? Is it really an issue? Cheers G. -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +6

Re: [ceph-users] client did not provide supported auth type

2016-06-27 Thread Goncalo Borges
2235 & v27 here ? how to solve this problem ? Regards, XiuCai. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle

Re: [ceph-users] client did not provide supported auth type

2016-06-27 Thread Goncalo Borges
Cheers G On 06/28/2016 01:40 PM, Goncalo Borges wrote: Hi XiuCai Shouldn't you have, at least, 2 mons? Cheers G. On 06/28/2016 01:12 PM, 秀才 wrote: Hi, ther are 1 mon and 7 osds in my cluster now. but it seems something wrong, because `rbd -p test reate pet --size 1024` could not retu

[ceph-users] object size changing after a pg repair

2016-06-29 Thread Goncalo Borges
Dear Cephers... Today our ceph cluster gave us a couple of scrub errors regarding inconsistent pgs. We just upgraded from 9.2.0 to 10.2.2 two days ago. # ceph health detail HEALTH_ERR 2 pgs inconsistent; 2 scrub errors; crush map has legacy tunables (require bobtail, min is firefly) pg 6.39c i

Re: [ceph-users] object size changing after a pg repair

2016-06-29 Thread Goncalo Borges
te": "INACTIVE", "scrubber.start": "MIN", "scrubber.end": "MIN", "scrubber.subset_last_update": "0'0", "scrubber.deep": false, &q

Re: [ceph-users] object size changing after a pg repair

2016-06-29 Thread Goncalo Borges
changing on the fly. Does this make sense? G. From: Shinobu Kinjo [shinobu...@gmail.com] Sent: 30 June 2016 15:10 To: Goncalo Borges Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] object size changing after a pg repair On Thu, Jun 30, 2016 at 1:48

Re: [ceph-users] object size changing after a pg repair

2016-06-30 Thread Goncalo Borges
Hi Greg Opened this one http://tracker.ceph.com/issues/16567 Let us see what they say. Cheers G. On 07/01/2016 04:09 AM, Gregory Farnum wrote: On Wed, Jun 29, 2016 at 10:50 PM, Goncalo Borges wrote: Hi Shinobu Sorry probably I don't understand your question properly. Is what y

[ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-04 Thread Goncalo Borges
5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/10 civetweb 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs 1/ 5 xio 1/ 5 compressor 1/ 5 newstore 1/ 5 bluestore 1/ 5 bluefs 1/ 3 bdev 1/ 5 kstore 4/

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-04 Thread Goncalo Borges
e code and recompile. Is this something safe to do? Cheers Goncalo On 07/05/2016 01:34 PM, Patrick Donnelly wrote: Hi Goncalo, I believe this segfault may be the one fixed here: https://github.com/ceph/ceph/pull/10027 (Sorry for brief top-post. Im on mobile.) On Jul 4, 2016 9:16 PM, "

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-04 Thread Goncalo Borges
, 2016 9:16 PM, "Goncalo Borges" wrote: Dear All... We have recently migrated all our ceph infrastructure from 9.2.0 to 10.2.2. We are currently using ceph-fuse to mount cephfs in a number of clients. ceph-fuse 10.2.2 client is segfaulting in some situations. One of the scenarios where

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-05 Thread Goncalo Borges
lock functions in two specific lines of src/client/Client.cc which, I imagine, were also not there in 9.2.0 (unless there was a big rewrite of src/client/Client.cc from 9.2.0 to 10.2.2) Cheers Goncalo On 07/05/2016 02:45 PM, Goncalo Borges wrote: Hi Brad, Shinobu, Patrick... Indeed if I run

Re: [ceph-users] multiple journals on SSD

2016-07-06 Thread Goncalo Borges
es Anything else? Best practice? P.S. I've done benchmarking: 3500 can support up to 16 10k-RPM HDD. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Borges Research Computing

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-06 Thread Goncalo Borges
My previous email did not go through because of its size. Here goes a new attempt: Cheers Goncalo --- * --- Hi Patrick, Brad... Unfortunately, the other user application breaks ceph-fuse again (It is a completely different application then in my previous test). We have tested it in 4 machi

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-08 Thread Goncalo Borges
rick Donnelly wrote: On Thu, Jul 7, 2016 at 2:01 AM, Goncalo Borges wrote: Unfortunately, the other user application breaks ceph-fuse again (It is a completely different application then in my previous test). We have tested it in 4 machines with 4 cores. The user is submitting 16 single cor

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-11 Thread Goncalo Borges
calo 20 0 1594m 83m 19m R 100.2 0.2 1:05.57 python 29312 goncalo 20 0 1594m 83m 19m R 99.9 0.2 1:05.01 python 31979 goncalo 20 0 1595m 82m 19m R 100.2 0.2 1:04.82 python 29333 goncalo 20 0 1594m 82m 19m R 99.5 0.2 1:04.94 python 29609 goncalo 20

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-11 Thread Goncalo Borges
On 07/11/2016 05:04 PM, Goncalo Borges wrote: Hi John... Thank you for replying. Here is the result of the tests you asked but I do not see nothing abnormal. Actually, your suggestions made me see that: 1) ceph-fuse 9.2.0 is presenting the same behaviour but with less memory consumption

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-12 Thread Goncalo Borges
ectcacher = 20' Cheers Goncalo From: Gregory Farnum [gfar...@redhat.com] Sent: 12 July 2016 03:07 To: Goncalo Borges Cc: John Spray; ceph-users Subject: Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2) Oh, is this one of your custom-built packages? A

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-14 Thread Goncalo Borges
ns will solve our problem. Does this make sense? The compilation takes a while but I will update the issue once I have finished this last experiment (in the next few days) Cheers Goncalo On 07/12/2016 09:45 PM, Goncalo Borges wrote: Hi All... Thank you for continuing to follow this alrea

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-14 Thread Goncalo Borges
your patch. Will report here afterwards. Thanks for the feedback. Cheers Goncalo On 07/15/2016 01:19 PM, Yan, Zheng wrote: On Fri, Jul 15, 2016 at 9:35 AM, Goncalo Borges wrote: Hi All... I've seen that Zheng, Brad, Pat and Greg already updated or made some comments on the bug issue.

Re: [ceph-users] ceph-fuse segfaults ( jewel 10.2.2)

2016-07-18 Thread Goncalo Borges
/ googling for tips, I saw an issue claiming that 'fuse_disable_pagecache' should be set to true in ceph.conf. Can you briefly explain is this is correct and what is the con of not using it? (just or me to understand it). Thank you in Advance Cheers Goncalo On 07/15/2016 01:3

[ceph-users] pgs stuck unclean after reweight

2016-07-19 Thread Goncalo Borges
;: 0, "dne": 0, "incomplete": 0, "last_epoch_started": 1016, "hit_set_history": { "current_last_update": "0'0", "history": [] } }, { "peer":

Re: [ceph-users] pgs stuck unclean after reweight

2016-07-19 Thread Goncalo Borges
ortbitwise flag during update? If YES, try to unset it. I faced with same problem when upgrade my ceph cluster from Hummer to Jewel. Maybe it's your: http://tracker.ceph.com/issues/16113 Среда, 20 июля 2016, 8:42 +05:00 от Goncalo Borges : Hi All... Today we had a warning r

Re: [ceph-users] pgs stuck unclean after reweight

2016-07-19 Thread Goncalo Borges
p_ osds are (i guess) the new ones it tries to use. However, in this new _up_ set, there is always one osd with the near full message. Maybe that is why re balancing is on hold? Maybe if I increase the thresold for the warning the rebalance will restart? Cheers G. On 07/20/2016 01:42 PM, Gonc

Re: [ceph-users] pgs stuck unclean after reweight

2016-07-19 Thread Goncalo Borges
Hi Swami. Did not make any difference. Cheers G. On 07/20/2016 03:31 PM, M Ranga Swami Reddy wrote: can you restart osd.32 and check the status? Thanks Swami On Wed, Jul 20, 2016 at 9:12 AM, Goncalo Borges wrote: Hi All... Today we had a warning regarding 8 near full osd. Looking to

Re: [ceph-users] Recovery stuck after adjusting to recent tunables

2016-07-23 Thread Goncalo Borges
Hi Kostis This is a wild guess but one thing I note is that your pool 179 has a very low pg number (100). Maybe the algorithm behind the new tunable need a higher pg number to actually proceed with the recovery? You could try to increase the pgs to 128 (it is always better to use powers of 2

Re: [ceph-users] pgs stuck unclean after reweight

2016-07-23 Thread Goncalo Borges
. Thanks for the help Goncalo From: Christian Balzer [ch...@gol.com] Sent: 20 July 2016 19:36 To: ceph-us...@ceph.com Cc: Goncalo Borges Subject: Re: [ceph-users] pgs stuck unclean after reweight Hello, On Wed, 20 Jul 2016 13:42:20 +1000 Goncalo Borges wrote

[ceph-users] mon_osd_nearfull_ratio (unchangeable) ?

2016-07-25 Thread Goncalo Borges
, it doesn't seem I am able to change this value. Can someone clarify exactly what is the happening here? Cheers G. -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +61 2 93511937 _

Re: [ceph-users] mon_osd_nearfull_ratio (unchangeable) ?

2016-07-25 Thread Goncalo Borges
Hi Brad Thanks for replying. Answers inline. I am a bit confused about the 'unchachable' message we get in Jewel 10.2.2 when I try to change some cluster configs. For example: 1./ if I try to change mon_osd_nearfull_ratio from 0.85 to 0.90, I get # ceph tell mon.* injectargs "--mon_osd

Re: [ceph-users] mon_osd_nearfull_ratio (unchangeable) ?

2016-07-26 Thread Goncalo Borges
last_osdmap_epoch 2546 last_pg_scan 2546 full_ratio 0.95 *nearfull_ratio 0.85* Cheers G. On 07/26/2016 12:39 PM, Brad Hubbard wrote: On Tue, Jul 26, 2016 at 12:16:35PM +1000, Goncalo Borges wrote: Hi Brad Thanks for replying. Answers inline. I am a bit confused about the 'unchach

[ceph-users] newly osds dying (jewel 10.2.2)

2016-07-26 Thread Goncalo Borges
Hi cephers... Our production cluster is running Jewel 10.2.2. We were running a production cluster with 8 servers each with 8 osds making a gran total of 64 osds. Each server also hosts 2 ssds for journals. Each sshd supports 4 journals. We had 1/3 of our osds above 80% occupied, and we decid

Re: [ceph-users] newly osds dying (jewel 10.2.2)

2016-07-26 Thread Goncalo Borges
the 10.100.1.0/24 being blocked. I think i had the firewall disabled when I bootstrapped the osds in the machines and that might explain why there was some transfer of data. Sorry for the entropy. Cheers G. On 07/27/2016 08:44 AM, Goncalo Borges wrote: Hi cephers... Our production

[ceph-users] ceph-fuse (jewel 10.2.2): No such file or directory issues

2016-07-27 Thread Goncalo Borges
to mitigate it. It is also worthwhile to mention that this seems to happen while we are adding a new storage server to the underlying ceph infrastructure, so there was some data movement happening in the background. Any suggestion on how to mitigate it? Cheers Goncalo and Sean -- Gon

Re: [ceph-users] ceph-fuse (jewel 10.2.2): No such file or directory issues

2016-07-27 Thread Goncalo Borges
lem. In this way you force ceph-fuse to read directly from osds, right?! We understand about the performance issues that it might imply but we are more concerned in having data coherence in the client. Thoughts? Cheers -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle P

Re: [ceph-users] mon_osd_nearfull_ratio (unchangeable) ?

2016-07-27 Thread Goncalo Borges
David wrote: Try: ceph pg set_nearfull_ratio 0.9 On 26 Jul 2016 08:16, "Goncalo Borges" <mailto:goncalo.bor...@sydney.edu.au>> wrote: Hello... I do not think that these settings are working properly in jewel. Maybe someone else can confirm. So, to summarize:

Re: [ceph-users] osd wrongly maked as down

2016-07-28 Thread Goncalo Borges
Firewall or communication issues? From: ceph-users [ceph-users-boun...@lists.ceph.com] on behalf of M Ranga Swami Reddy [swamire...@gmail.com] Sent: 28 July 2016 22:00 To: ceph-users Subject: [ceph-users] osd wrongly maked as down Hello, hello - I use 10

Re: [ceph-users] ceph-fuse (jewel 10.2.2): No such file or directory issues

2016-07-28 Thread Goncalo Borges
July 2016 06:54 To: Goncalo Borges Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] ceph-fuse (jewel 10.2.2): No such file or directory issues On Wed, Jul 27, 2016 at 6:37 PM, Goncalo Borges wrote: > Hi Greg > > Thanks for replying. Answer inline. > > > >>> Dear cephfs

[ceph-users] Tunables Jewel - request for clarification

2016-07-31 Thread Goncalo Borges
Dear cephers. I would like to request some clarification on migrating from legacy to optimal (jewel) tunables. We have recently migrated from infernalis to Jewel. However, we are still using legacy tunables. All our ceph infrastructure (mons. odss and mdss) are running 10.2.2 in Centos 7.2.15

Re: [ceph-users] Cephfs issue - able to mount with user key, not able to write

2016-08-03 Thread Goncalo Borges
ceph --version ceph version 10.2.2-1-g502540f (502540faf67308fa595e03f9f446b4ba67df731d) Any suggestion would be helpful. Thanks. Daleep Singh Bais ___________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-c

[ceph-users] Advice on migrating from legacy tunables to Jewel tunables.

2016-08-04 Thread Goncalo Borges
chooseleaf_vary_r=5 and then decrease it slowly to 1?) - then from firefly to hammer - then from hammer to jewel 2) or going directly to jewel tunables? Any advise on how to minimize the data movement? TIA Goncalo -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at

Re: [ceph-users] Advice on migrating from legacy tunables to Jewel tunables.

2016-08-08 Thread Goncalo Borges
arnum wrote: On Thu, Aug 4, 2016 at 8:57 PM, Goncalo Borges wrote: Dear cephers... I am looking for some advice on migrating from legacy tunables to Jewel tunables. What would be the best strategy? 1) A step by step approach? - starting with the transition from bobtail to firefly (and, in

Re: [ceph-users] how to debug pg inconsistent state - no ioerrors seen

2016-08-08 Thread Goncalo Borges
_ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 20

Re: [ceph-users] how to debug pg inconsistent state - no ioerrors seen

2016-08-09 Thread Goncalo Borges
-rw-r--r--1 ceph ceph 0 Aug 9 09:51 scrub\u6.107__head_0107__fff8 But it is empty.. Thanks! On 09/08/16 04:33, Goncalo Borges wrote: Hi Kenneth... The previous default behavior of 'ceph pg repair' was to copy the pg objects from the primary osd to others. Not sure i

Re: [ceph-users] CephFS quota

2016-08-13 Thread Goncalo Borges
Hi Willi If you are using ceph-fuse, to enable quota, you need to pass "--client-quota" option in the mount operation. Cheers Goncalo From: ceph-users [ceph-users-boun...@lists.ceph.com] on behalf of Willi Fehler [willi.feh...@t-online.de] Sent: 13 August

Re: [ceph-users] Cascading failure on a placement group

2016-08-13 Thread Goncalo Borges
Hi HP. I am just a site admin so my opinion should be validated by proper support staff Seems really similar to http://tracker.ceph.com/issues/14399 The ticket speaks about timezone difference between osds. Maybe it is something worthwhile to check? Cheers Goncalo

Re: [ceph-users] Cascading failure on a placement group

2016-08-13 Thread Goncalo Borges
The ticket I mentioned earlier was marked as a duplicate of http://tracker.ceph.com/issues/9732 Cheers Goncalo From: ceph-users [ceph-users-boun...@lists.ceph.com] on behalf of Goncalo Borges [goncalo.bor...@sydney.edu.au] Sent: 13 August 2016 22:23 To: Hein-Pieter van Braam; ceph-users

Re: [ceph-users] Cascading failure on a placement group

2016-08-13 Thread Goncalo Borges
ferent even if now is ok. It should be worthwhile to check if timezone is/was different in mind. Cheers From: Hein-Pieter van Braam [h...@tmm.cx] Sent: 13 August 2016 22:42 To: Goncalo Borges; ceph-users Subject: Re: [ceph-users] Cascading failure on a plac

Re: [ceph-users] Cascading failure on a placement group

2016-08-13 Thread Goncalo Borges
To: Goncalo Borges; ceph-users Subject: Re: [ceph-users] Cascading failure on a placement group Hi, The timezones on all my systems appear to be the same, I just verified it by running 'date' on all my boxes. - HP On Sat, 2016-08-13 at 12:36 +0000, Goncalo Borges wrote: > The ti

[ceph-users] Substitute a predicted failure (not yet failed) osd

2016-08-14 Thread Goncalo Borges
Hi cephfers I have a really simple question: the documentation always refers to the procedure to substitute failed disks. Currently I have a predicted failure in a raid 0 osd and I would like to substitute before it fails without having to go by replicating pgs once the osd is removed from crus

Re: [ceph-users] Substitute a predicted failure (not yet failed) osd

2016-08-14 Thread Goncalo Borges
users [ceph-users-boun...@lists.ceph.com] on behalf of Goncalo Borges [goncalo.bor...@sydney.edu.au] Sent: Sunday, August 14, 2016 5:47 AM To: ceph-us...@ceph.com Subject: [ceph-users] Substitute a predicted failure (not yet failed) osd Hi cephfers I have a really simple question: the documentation

Re: [ceph-users] PG is in 'stuck unclean' state, but all acting OSD are up

2016-08-15 Thread Goncalo Borges
@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Borges Research Computing ARC Centre of Excellence for Particle Physics at the Terascale School of Physics A28 | University of Sydney, NSW 2006 T: +61 2 93511937 ___ ceph-users

Re: [ceph-users] PG is in 'stuck unclean' state, but all acting OSD are up

2016-08-15 Thread Goncalo Borges
down-peering-failure [2] http://docs.ceph.com/docs/master/rados/troubleshooting/troubleshooting-pg/ On 08/16/2016 11:42 AM, Heller, Chris wrote: Output of `ceph pg dump_stuck` # ceph pg dump_stuck ok pg_stat state up up_primary acting acting_primary 4.2a8 down+peering

Re: [ceph-users] PG is in 'stuck unclean' state, but all acting OSD are up

2016-08-16 Thread Goncalo Borges
it returns to the network? -Chris *From: *Goncalo Borges *Date: *Monday, August 15, 2016 at 11:36 PM *To: *"Heller, Chris" , "ceph-users@lists.ceph.com" *Subject: *Re: [ceph-users] PG is in 'stuck unclean' state, but all acting OSD are up Hi Chris... The precis

Re: [ceph-users] ceph-fuse "Transport endpoint is not connected" on Jewel 10.2.2

2016-08-24 Thread Goncalo Borges
shows and I get kicked out of the ceph-fuse mount until I remount. Any suggestions? With regards, ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Goncalo Borges Research Computing ARC

[ceph-users] cephfs metadata pool: deep-scrub error "omap_digest != best guess omap_digest"

2016-08-29 Thread Goncalo Borges
uot;last_scrub_stamp": "2016-08-24 22:59:27.749260", "last_deep_scrub": "16645'96391", "last_deep_scrub_stamp": "2016-08-22 20:21:59.567449", "last_clean_scrub_stamp": "2016-08-24 22:59:27.

Re: [ceph-users] cephfs metadata pool: deep-scrub error "omap_digest != best guess omap_digest"

2016-08-30 Thread Goncalo Borges
in a discussion last time of exactly the same issue experienced by someone else in jewel Cheers Goncalo From: Brad Hubbard [bhubb...@redhat.com] Sent: 30 August 2016 17:13 To: Goncalo Borges Cc: ceph-us...@ceph.com Subject: Re: [ceph-users] cephfs metadata

  1   2   >