[ceph-users] professional essay writing help
This professional help with writing essays for college not only provides flawless essays to the students but it also takes several initiatives to make the service comprehensive and inclusive for the students. Accordingly, the expert essay writer has published numerous history essay samples on the website. https://essayassignmenthelp.com.au/essay-writing-help.html ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Get World-Class Programming Assignment Help at a Reasonable budget.
I have been writing assignments for Myassignmenthelpau, since five years now. We have a large team of expert writers and editors who promise to re-define assignments of the best kind. Get the best Assignment help Perth Services at the cheapest price in Australia. I have been successfully offering 9+- across tools such as word counter, referencing generator and more. https://www.myassignmenthelpau.com/assignment-help-perth ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Speeding up reconnection
After debugging some more, I see it is indeed the kernel client closing the socket. Long story short: how do I get the socket to be closed in less than 10 minutes? ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Geek squad services
Are you looking for Geek Squad Tech Support, Visit Geek Tech Support now, to compare the offers for Geek Squad Tech Support. All channel support plans. https://thegethelp.com/geek-squad-tech-support/ Get your devices protected under our geek protection plan. Check out our detailed guide on how egeek protection plan is better than the geek squad protection plan https://egeekonline.com/protection-plan/ ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Meaning of the "tag" key in bucket metadata
Recently we encountered an instance of bucket corruption of two varieties.
One in which the bucket metadata was missing and another in which the
bucket.instance metadata was missing for various buckets.
We have seemingly been successful in restoring the metadata by
reconstructing it from the remaining pieces of metadata and injecting it
using "radosgw-admin metadata put" for both bucket and bucket.instance
metadata
One piece of information that we could not determine was "tag"
"ver": {
"tag": "_iIMyX8XLf0HSTciEsrgLA7j",
"ver": 1
},
and so we tried reusing the tag in bucket.instance for the bucket metadata
and also spoofed the tag value to something random. It appears in both
situations the bucket's functionality was restored. I am however uncertain
of the function of this "tag" key and what situation I may be exposing
myself to by reusing or spoofing its value.
Respectfully,
*Wes Dillingham*
w...@wesdillingham.com
LinkedIn
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] DocuBetter Meeting Today 1630 UTC
There is a general documentation meeting called the "DocuBetter Meeting", and it is held every two weeks. The next DocuBetter Meeting will be on 12 Aug 2020 at 0830 PDT, and will run for thirty minutes. Everyone with a documentation-related request or complaint is invited. The meeting will be held here: https://bluejeans.com/908675367 This meeting will cover the reorganization of the Ceph website as well as Zac's recently-developed workflow that aims to make it possible to move the good ideas from stale or poorly-formed documentation PRs into the documentation more quickly. Send documentation-related requests and complaints to me by replying to this email and CCing me at zac.do...@gmail.com. The next DocuBetter meeting is scheduled for: 12 Aug 2020 0830 PDT 12 Aug 2020 1630 UTC 13 Aug 2020 0230 AEST Etherpad: https://pad.ceph.com/p/Ceph_Documentation Zac's docs whiteboard: https://pad.ceph.com/p/docs_whiteboard Meeting: https://bluejeans.com/908675367 ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: RGW unable to delete a bucket
Hi, What version are you using? There seems to exist a bug (https://tracker.ceph.com/issues/42358), we had the same problem using redhat 12.2.12-84. Te only way to stop the rgw logs to crash the machine was to restart the service. To remove the bucket we had to upgrade to 12.2.12.-115 - Mensagem original - De: "Andrei Mikhailovsky" Para: "ceph-users" Enviadas: Terça-feira, 4 de agosto de 2020 13:16:28 Assunto: [ceph-users] RGW unable to delete a bucket Hi I am trying to delete a bucket using the following command: # radosgw-admin bucket rm --bucket= --purge-objects However, in console I get the following messages. About 100+ of those messages per second. 2020-08-04T17:11:06.411+0100 7fe64cacf080 1 RGWRados::Bucket::List::list_objects_ordered INFO ordered bucket listing requires read #1 The command has been running for about 35 days days and it still hasn't finished. The size of the bucket is under 1TB for sure. Probably around 500GB. I have recently removed about a dozen of old buckets without any issues. It's this particular bucket that is being very stubborn. Anything I can do to remove it, including it's objects and any orphans it might have? Thanks Andrei ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] fio rados ioengine
Hi all, Does anyone know if using fio with the rados ioengine will produce similar results to rados bench, i.e. can the 2 be used interchangeably? thx Frank ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: 5 pgs inactive, 5 pgs incomplete
> Are the OSDs online? Or do they refuse to boot?
Yes. They are up and running and not marked as down or out of the cluster.
> Can you list the data with ceph-objectstore-tool on these OSDs?
If you mean the "list" operation on the PG works if an output for example:
$ ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-63 --pgid
22.11a --op list
["22.11a",{"oid":"1001c1ee04f.0007","key":"","snapid":-2,"hash":3825189146,"max":0,"pool":22,"namespace":"","max":0}]
["22.11a",{"oid":"1000448667f.","key":"","snapid":-2,"hash":4294951194,"max":0,"pool":22,"namespace":"","max":0}]
...
If I run "ceph pg ls incomplete" in the output only one PG has
objects... all others have 0 objects.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] CephFS clients waiting for lock when one of them goes slow
Hi all, I have a very small Ceph setup - 3 OSDs, 3 MDS, 3 MON, CephFS with ~20 ceph-fuse clients connected. All run version 14.2.9 (both servers and clients). Some clients work with a relatively large CephFS directory, few tens of thousands of files, with lots of files being added and deleted all the time. Almost every day I see exactly the same pattern when system goes under load: - One of the clients starts running slow (e.g. high CPU or out-of-memory, sometimes with OOM killer intervention) - A number of requests from other clients gets blocked for ~15 seconds waiting for the (r/w/x)lock on other files in the same directory - Objecter requests empty - No requests visibly running in the dump_ops_in_flight output, only those waiting for the lock MDS cache size is 1.5 gb, total capacity - around 1TB, usual load is ~300 [rw]ops, 3-4 mb/s, most of the config is the default one. Have somebody seen similar issues before? Best regards, Petr Belyaev ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Ceph not warning about clock skew on an OSD-only host?
My understanding is that the existing mon_clock_drift_allowed value of 50 ms (default) is so that PAXOS among the mon quorum can function. So OSDs (and mgrs, and clients etc) are out of scope of that existing code. Things like this are why I like to ensure that the OS does `ntpdate -b` or equivalent at boot-time *before* starting ntpd / chrony - and other daemons. Now, as to why Ceph doesn’t have analogous code to acomplain about other daemons / clients - I’ve wonder that for some time myself. Perhaps there’s the idea that one’s monitoring infrastructure should detect that, but that’s a guess. > Yesterday, one of our OSD-only hosts came up with its clock about 8 hours > wrong(!) having been out of the cluster for a week or so. Initially, ceph > seemed entirely happy, and then after an hour or so it all went South (OSDs > start logging about bad authenticators, I/O pauses, general sadness). > > I know clock sync is important to Ceph, so "one system is 8 hours out, Ceph > becomes sad" is not a surprise. It is perhaps a surprise that the OSDs were > allowed in at all... > > What _is_ a surprise, though, is that at no point in all this did Ceph raise > a peep about clock skew. Normally it's pretty sensitive to this - our test > cluster has had clock skew complaints when a mon is only slightly out, and > here we had a node 8 hours wrong. > > Is there some oddity like Ceph not warning on clock skew for OSD-only hosts? > or an upper bound on how high a discrepency it will WARN about? > > Regards, > > Matthew > > example output from mid-outage: > > root@sto-3-1:~# ceph -s > cluster: >id: 049fc780-8998-45a8-be12-d3b8b6f30e69 >health: HEALTH_ERR >40755436/2702185683 objects misplaced (1.508%) >Reduced data availability: 20 pgs inactive, 20 pgs peering >Degraded data redundancy: 367431/2702185683 objects degraded > (0.014%), 4549 pgs degraded >481 slow requests are blocked > 32 sec. Implicated osds > 188,284,795,1278,1981,2061,2648,2697 >644 stuck requests are blocked > 4096 sec. Implicated osds > 22,31,33,35,101,116,120,130,132,140,150,159,201,211,228,263,327,541,561,566,585,589,636,643,649,654,743,785,790,806,865,1037,1040,1090,1100,1104,1115,1134,1135,1166,1193,1275,1277,1292,1494,1523,1598,1638,1746,2055,2069,2191,2210,2358,2399,2486,2487,2562,2589,2613,2627,2656,2713,2720,2837,2839,2863,2888,2908,2920,2928,2929,2947,2948,2963,2969,2972 > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: Ceph not warning about clock skew on an OSD-only host?
On Wed, Aug 12, 2020 at 4:09 PM Anthony D'Atri wrote: > > My understanding is that the existing mon_clock_drift_allowed value of 50 ms > (default) is so that PAXOS among the mon quorum can function. So OSDs (and > mgrs, and clients etc) are out of scope of that existing code. This is correct — the monitors issue leases to each other, those leases are in absolute clock time, and we need some level of coherency to maintain our consistency guarantees there... > > Things like this are why I like to ensure that the OS does `ntpdate -b` or > equivalent at boot-time *before* starting ntpd / chrony - and other daemons. > > Now, as to why Ceph doesn’t have analogous code to a complain about other > daemons / clients - I’ve wonder that for some time myself. Perhaps there’s > the idea that one’s monitoring infrastructure should detect that, but that’s > a guess. ...but none of the rest of the Ceph stack that needs clocks to be anywhere near each other is the cephx rotating keys — and those need to be correct on the order of "within an hour" instead of "we do 5-second leases that need to agree on absolute time". Usually, that kind of vague agreement isn't an issue, and in that core RADOS code we usually try not to impose requirements on the environment, so there's not a clock sync check happening. Perhaps at this point it would be appropriate to add one; tracker tickets and PRs are welcome. ;) -Greg > > > Yesterday, one of our OSD-only hosts came up with its clock about 8 hours > > wrong(!) having been out of the cluster for a week or so. Initially, ceph > > seemed entirely happy, and then after an hour or so it all went South (OSDs > > start logging about bad authenticators, I/O pauses, general sadness). > > > > I know clock sync is important to Ceph, so "one system is 8 hours out, Ceph > > becomes sad" is not a surprise. It is perhaps a surprise that the OSDs were > > allowed in at all... > > > > What _is_ a surprise, though, is that at no point in all this did Ceph > > raise a peep about clock skew. Normally it's pretty sensitive to this - our > > test cluster has had clock skew complaints when a mon is only slightly out, > > and here we had a node 8 hours wrong. > > > > Is there some oddity like Ceph not warning on clock skew for OSD-only > > hosts? or an upper bound on how high a discrepency it will WARN about? > > > > Regards, > > > > Matthew > > > > example output from mid-outage: > > > > root@sto-3-1:~# ceph -s > > cluster: > >id: 049fc780-8998-45a8-be12-d3b8b6f30e69 > >health: HEALTH_ERR > >40755436/2702185683 objects misplaced (1.508%) > >Reduced data availability: 20 pgs inactive, 20 pgs peering > >Degraded data redundancy: 367431/2702185683 objects degraded > > (0.014%), 4549 pgs degraded > >481 slow requests are blocked > 32 sec. Implicated osds > > 188,284,795,1278,1981,2061,2648,2697 > >644 stuck requests are blocked > 4096 sec. Implicated osds > > 22,31,33,35,101,116,120,130,132,140,150,159,201,211,228,263,327,541,561,566,585,589,636,643,649,654,743,785,790,806,865,1037,1040,1090,1100,1104,1115,1134,1135,1166,1193,1275,1277,1292,1494,1523,1598,1638,1746,2055,2069,2191,2210,2358,2399,2486,2487,2562,2589,2613,2627,2656,2713,2720,2837,2839,2863,2888,2908,2920,2928,2929,2947,2948,2963,2969,2972 > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Can I install HP envy 4520 drivers correctly?
HP envy 4520 wireless printer is one of the most popular brands, which is specially known for its printing services. It is the best product in the features and functions, so millions of users prefer it for the best printing performance. Due to some technical problems, my HP 4520 drivers are fully corrupted, so I want to install the newest version of [url=https://www.hpprintersupportpro.com/blog/troubleshoot-hp-printer-envy-4520-errors/]HP envy 4520 drivers[/url] successfully. As per my technical experience, I am trying my skills to install HP envy 4520 printer driver. I am failing to complete installation process. So anyone can help me to install printer driver appropriately. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Can I install HP envy 4520 drivers correctly?
HP envy 4520 wireless printer is one of the most popular brands, which is specially known for its printing services. It is the best product in the features and functions, so millions of users prefer it for the best printing performance. Due to some technical problems, my HP 4520 drivers are fully corrupted, so I want to install the newest version of HP envy 4520 drivers successfully. As per my technical experience, I am trying my skills to install HP envy 4520 printer driver. I am failing to complete installation process. So anyone can help me to install printer driver appropriately. https://www.hpprintersupportpro.com/blog/troubleshoot-hp-printer-envy-4520-errors/ ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Heavy rocksdb activity in newly added osd
Hi All
We had a cluster (v13.2.4) with 32 osds in total. At first, an osd (osd.18)
in cluster was down. So, we tried to remove the osd and added a new one
(osd.32) with new ID. We unplugged the disk (osd.18) and plugged in a new
disk in the same slot and add osd.32 into cluster. Then, osd.32 was
booting, but, we found it takes much time (around 18 mins) for the osd to
change to up state. Diving into osd.32 logs, we see that there is much
rocksdb activity before osd.32 change to up state. Can anyone explain why
this happened or give me any advice about how to prevent from this. Thanks.
[osd.32 log]
2020-08-03 15:36:58.852 7f88021fa1c0 0 osd.32 0 done with init, starting
boot process
2020-08-03 15:36:58.852 7f88021fa1c0 1 osd.32 0 start_boot
2020-08-03 15:36:58.854 7f87db02b700 -1 osd.32 0 waiting for initial osdmap
2020-08-03 15:36:58.855 7f87e4ba0700 -1 osd.32 0 failed to load OSD map for
epoch 22010, got 0 bytes
2020-08-03 15:36:58.955 7f87e0836700 0 osd.32 22011 crush map has features
283675107524608, adjusting msgr requires for clients
2020-08-03 15:36:58.955 7f87e0836700 0 osd.32 22011 crush map has features
283675107524608 was 288232575208792577, adjusting msgr requires for mons
*2020-08-03 15:36:58.955* 7f87e0836700 0 osd.32 22011 crush map has
features 720859615486820352, adjusting msgr requires for osds
2020-08-03 15:37:31.182 7f87e1037700 4 rocksdb:
[/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/db_impl_write.cc:1346]
[default] New memtable created with log file: #16. Immutable memtables: 0.
2020-08-03 15:37:31.285 7f87e8045700 4 rocksdb: (Original Log Time
2020/08/03-15:37:31.183995)
[/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/db_impl_compaction_flush.cc:1396]
Calling FlushMemTableToOutputFile with column family [default], flush slots
available 1, compaction slots available 1, flush slots scheduled 1,
compaction slots scheduled 0
2020-08-03 15:37:31.285 7f87e8045700 4 rocksdb:
[/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/flush_job.cc:300]
[default] [JOB 3] Flushing memtable with next log file: 16
lots of rocksdb activity-
2020-08-03 15:54:21.704 7f87e8045700 4 rocksdb: (Original Log Time
2020/08/03-15:54:21.705680)
[/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/memtable_list.cc:397]
[default] Level-0 commit table #112: memtable #1 done
2020-08-03 15:54:21.704 7f87e8045700 4 rocksdb: (Original Log Time
2020/08/03-15:54:21.705704) EVENT_LOG_v1 {"time_micros": 1596441261705697,
"job": 51, "event": "flush_finished", "output_compression":
"NoCompression", "lsm_state": [1, 3, 0, 0, 0, 0, 0], "immutable_memtables":
0}
2020-08-03 15:54:21.704 7f87e8045700 4 rocksdb: (Original Log Time
2020/08/03-15:54:21.705721)
[/home/gitlab/rpmbuild/BUILD/ceph-13.2.4/src/rocksdb/db/db_impl_compaction_flush.cc:172]
[default] Level summary: base level 1 max bytes base 268435456 files[1 3 0
0 0 0 0] max score 0.75
*2020-08-03 15:54:38.567* 7f87e0836700 1 osd.32 502096 state: booting ->
active
2020-08-03 15:54:38.567 7f87d5820700 1 osd.32 pg_epoch: 502096 pg[1.17e(
empty local-lis/les=0/0 n=0 ec=11627/16 lis/c 501703/501703 les/c/f
501704/501704/0 502096/502096/502096) [32,26,28] r=0 lpr=502096
pi=[501703,502096)/1 crt=0'0 mlcod 0'0 unknown mbc={}] state:
transitioning to Primary
Best
Jerry
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Block Network IP Of Wireless HP Printer Using HP Support Assistant.
Yes, you should! If you are one of those who are have a word with the professionals who are capable of resolving the whole host of network problems. You can make use of HP Support Assistant who will solve the whole host of wireless printer problems. https://www.amiytech.com/hp-support-assistant/ ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Approach Epson Customer Service For Fixing Network Problems.
Yes, you are not only allowed to approach Epson Customer Service but also fetch the instant technical backing directly from the professionals. If you want to avail the quickest remedy along with the supervision regarding the problems and hurdles, pertaining to HP printer network, you should get the free assistance at anytime. https://www.epsonprintersupportpro.net/ ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
