On Fri, 10 May 2013, Noah Watkins wrote:
> Mike,
>
> Thanks for the looking into this further.
>
> On May 10, 2013, at 5:23 AM, Mike Bryant wrote:
>
> > I've just found this bug report though: http://tracker.ceph.com/issues/3601
> > Looks like that may be the same issue..
>
> This definitely s
> There is already debugging present in the Java bindings. You can turn on
> client logging, and add 'debug javaclient = 20' to get client debug logs.
Ah, I hadn't noticed that, cheers.
> How many clients does HBase setup?
There's one connection to cephfs from the master, and one from each of
t
Mike,
Thanks for the looking into this further.
On May 10, 2013, at 5:23 AM, Mike Bryant wrote:
> I've just found this bug report though: http://tracker.ceph.com/issues/3601
> Looks like that may be the same issue..
This definitely seems like a candidate.
>> Adding some debug to the cephfs ja
Investigating further, there seems to be a large number of inodes with
caps, many of which are actually unlinked from the filesystem.
2013-05-10 13:04:11.270365 7f2d7f349700 2 mds.0.cache
check_memory_usage total 306000, rss 90624, heap 143444, malloc 53053
mmap 0, baseline 131152, buffers 0, max
Mhm, if that was the case I would expect it to be deleting things over
time. On one occurrence for example the data pool reached 160GB after 3 or
4 days, with a reported usage in cephfs of 12GB. Within minutes of my
stopping the clients, the data pool dropped by over 140GB.
I suspect the filehandle
Mike,
I'm guessing that HBase is creating and deleting its blocks, but that the
deletes are delayed:
http://ceph.com/docs/master/dev/delayed-delete/
which would explain the correct reporting at the file system level, but not the
actual 'data' pool. I'm not as familiar with this level of deta
Hi,
I'm experimenting with running hbase using the hadoop-ceph java
filesystem implementation, and I'm having an issue with space usage.
With the hbase daemons running, The amount of data in the 'data' pool
grows continuously, at a much higher rate than expected. Doing a du,
or ls -lh on a mounted
