Re: [gentoo-user] Gentoo on a cell?
well, without wanting to get political or anything ... but your point begs the question of what country yours is, or any other country, that's not grounded in greed? "Greed" is a stupid concept On 2020-03-01 02:49, antlists wrote: On 29/02/2020 17:40, james wrote: is if the US government returns to the fundamental christian value system, that made our country great. Greed, un-bridled, is changing the quality of our lives, regardless of your personal belief systems. Our country? I think you mean YOUR country. And seen from outside, that greed has been there pretty much from its birth ... I'll agree about greed being the problem, though. But it's pretty difficult to return to a place you've never been, imho ... Cheers, Wol
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
William Kenworthy wrote: > On 29/2/20 11:31 pm, Rich Freeman wrote: >> On Sat, Feb 29, 2020 at 10:17 AM Daniel Frey wrote: >>> Yes, I'm aware linux does VLANs... I set up netifrc to do this (I >>> already have some "smart" switches set up - not full layer 3.) I thought >>> about running containers but if I ever have to do something like >>> emergency maintenance on my server the whole LAN would be down. Seems >>> like a no-brainer to have a tiny device like an RPi to do this. >> Yup. It really depends on your requirements. >> >> My main LAN uses a Pi as a DHCP+DNS server, for exactly this reason. >> I don't want to be replacing a hard drive in my server and now my >> lights/TV/whatever don't work. OpenHab runs on a Pi for this reason >> as well. > Keep in mind that rpi are not the only cheap, capable arm hardware out > there. > > I am using a number of odroid devices, including an N2 with a gentoo > based kernel and a gentoo aarch64 userland. Its used for lxc containers > for asterisk, dns, webdav, mail, calendaring and web running on the N2 > backed by an Odroid HC2 moosefs cluster (though I am using an intel > powered Odroid H2 for the master). > > Its all working rather well now the initial install/config stages are > over. Part of the gain over the pi's is the use of eMMC storage over > sdcards - almost 5 times faster in my tests. The 4G of ram has proven > quite adequate so far - even the asterisk latency is better than my > previous QEMU/KVM on intel setup. > > I have currently have rpi 1B, 3B and a zero and while the specs for the > rpi4 are better ... its not the best out there. > > BillK > Could you share some links to some of these things? As I mentioned earlier, I'm thinking about building a NAS system. Later, I may build a mythTV system. Then I can access the NAS from it or my desktop, or cell phone now that I am somewhat more updated, past the Motorola Razr stage. Thanks much. Dale :-) :-)
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On 2020-03-01 11:46, Dale wrote: ... now that I am somewhat more updated, past the Motorola Razr stage. Thanks much. Dale :-) :-) Which Razr do you mean? Are you 9 years (https://en.wikipedia.org/wiki/Droid_Razr) or 16 years out of date? (https://en.wikipedia.org/wiki/Motorola_Razr) ;-) Although, it's amazing to think, today, that there was only 7 years between those two...
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On Sunday, 1 March 2020 10:46:17 GMT Dale wrote: > William Kenworthy wrote: > > On 29/2/20 11:31 pm, Rich Freeman wrote: > >> On Sat, Feb 29, 2020 at 10:17 AM Daniel Frey wrote: > >>> Yes, I'm aware linux does VLANs... I set up netifrc to do this (I > >>> already have some "smart" switches set up - not full layer 3.) I thought > >>> about running containers but if I ever have to do something like > >>> emergency maintenance on my server the whole LAN would be down. Seems > >>> like a no-brainer to have a tiny device like an RPi to do this. > >> > >> Yup. It really depends on your requirements. > >> > >> My main LAN uses a Pi as a DHCP+DNS server, for exactly this reason. > >> I don't want to be replacing a hard drive in my server and now my > >> lights/TV/whatever don't work. OpenHab runs on a Pi for this reason > >> as well. > > > > Keep in mind that rpi are not the only cheap, capable arm hardware out > > there. > > > > I am using a number of odroid devices, including an N2 with a gentoo > > based kernel and a gentoo aarch64 userland. Its used for lxc containers > > for asterisk, dns, webdav, mail, calendaring and web running on the N2 > > backed by an Odroid HC2 moosefs cluster (though I am using an intel > > powered Odroid H2 for the master). > > > > Its all working rather well now the initial install/config stages are > > over. Part of the gain over the pi's is the use of eMMC storage over > > sdcards - almost 5 times faster in my tests. The 4G of ram has proven > > quite adequate so far - even the asterisk latency is better than my > > previous QEMU/KVM on intel setup. > > > > I have currently have rpi 1B, 3B and a zero and while the specs for the > > rpi4 are better ... its not the best out there. > > > > BillK > > Could you share some links to some of these things? As I mentioned > earlier, I'm thinking about building a NAS system. Later, I may build a > mythTV system. Then I can access the NAS from it or my desktop, or cell > phone now that I am somewhat more updated, past the Motorola Razr stage. > > Thanks much. > > Dale > > :-) :-) Android has its optimal use cases for sure, but I have found embedded x86 APUs to be more capable for network and server tasks. As an example: https://pcengines.ch/ signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
n952162 wrote: > > On 2020-03-01 11:46, Dale wrote: >> ... now that I am somewhat more updated, past the Motorola Razr stage. >> Thanks much. >> >> Dale >> >> :-) :-) >> > > Which Razr do you mean? > > Are you 9 years (https://en.wikipedia.org/wiki/Droid_Razr) or 16 years > out of date? (https://en.wikipedia.org/wiki/Motorola_Razr) > > ;-) > > Although, it's amazing to think, today, that there was only 7 years > between those two... > > > I had the flip phone, bottom link I think. I wore out the charging port and it got to where it was hard to charge the battery. Otherwise, the thing still worked great. I admit, texting is much easier with the new Samsung. Also, circle a word puzzles when at the doctors office waiting helps pass the time too. :/ Dale :-) :-)
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
Michael wrote: > On Sunday, 1 March 2020 10:46:17 GMT Dale wrote: >> >> Could you share some links to some of these things? As I mentioned >> earlier, I'm thinking about building a NAS system. Later, I may build a >> mythTV system. Then I can access the NAS from it or my desktop, or cell >> phone now that I am somewhat more updated, past the Motorola Razr stage. >> >> Thanks much. >> >> Dale >> >> :-) :-) > Android has its optimal use cases for sure, but I have found embedded x86 > APUs > to be more capable for network and server tasks. As an example: > > https://pcengines.ch/ Interesting. I was thinking at first about just buying a really small computer MB, like a micro ATX or something, but reading this thread made think more about something like this. For one, it pulls a very small amount of power. The 'computer' and hard drives would likely pull well under 50 watts. Also generate very little heat, more like warmth. ;-) Basically, I'm looking for something that I can build a NAS with. I'd need a way to hook 4, 6 maybe 8 hard drives and maybe a couple ethernet ports. It seems to be easy to find them with ethernet ports. It's the SATA ports that are harder to find. I found one for the Raspberry thing that had four ports. So far, it is the only one I found. I'm not sure if it can expand past that either. It seems the cards are stackable based on some pics but I'm not sure on that. This is interesting info tho. The link you shared lead me to a tiny board, maybe a little more powerful than the Raspberry but still low powered. Still, the SATA ports is a bit difficult to find. Maybe I'm looking for the wrong thing??? Either way, I left a map light, tiny light that's up there with the dome light, on a few weeks ago in my car. I pulled out my hydrometer battery tester. It's almost always accurate. I got a few week cells in my car's battery. After someone else posted about those AGM batteries, I'm thinking of going down that route even for my car. They ain't cheap by no stretch. That said, those Raspberry things are almost disposable if you get a Chinese model. lol Dale :-) :-)
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On Sun, Mar 1, 2020 at 2:13 AM William Kenworthy wrote: > > Keep in mind that rpi are not the only cheap, capable arm hardware out > there. > I completely agree. Anytime I'm looking at an application I consider the SBCs available as options. Certainly the odroids are highly spoken of. Main advantage of the Pi is its ubiquity - just about anything you could want is already packaged and documented for it. It is also pretty cheap. > backed by an Odroid HC2 moosefs cluster (though I am using an intel > powered Odroid H2 for the master). I considered an HC2 for lizardfs. My problem with it is that it has a single SATA port, which means you're buying a $50 SBC for every hard drive in your cluster. For a single drive per node it is probably your best bet. However, my chunkservers are: ~$65 RockPro64 $20 used LSI HBA $5 wall wart $25 cheap ATX PSU $5 ATX power switch $5 extra SATA cables $5 powered 16x PCIe riser cable (these are a bit hard to find) That is ~$125, and will support 16 hard drives. You're saving money on the 3rd drive per node. If you want some kind of enclosure for the drives you'll pay maybe another $5/drive. The other option that might be worth considering if you don't mind losing some bandwidth to the drives is just using SATA3 and hubs/etc and external drives. I'm shucking external drives anyway. So, any SBC with a SATA3 port would work for that, with nothing else needed. I could see USB3 bandwidth (shared) being a constraint if you're rebuilding, but it would keep up with gigabit ethernet. Oh, and for any kind of NAS/etc solution make sure that whatever you get has gigabit ethernet. The Pi3s at least don't have that - not sure about the Pi4. Wouldn't help in a Pi3 anyway as I think the LAN goes through the internal USB2 bus - the Pi is pretty lousy for IO in general - at least conventional PC IO. That GPIO breakout is of course nice for projects. -- Rich
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On 3/1/20 6:33 AM, Rich Freeman wrote: On Sun, Mar 1, 2020 at 2:13 AM William Kenworthy wrote: Keep in mind that rpi are not the only cheap, capable arm hardware out there. I completely agree. Anytime I'm looking at an application I consider the SBCs available as options. Certainly the odroids are highly spoken of. Main advantage of the Pi is its ubiquity - just about anything you could want is already packaged and documented for it. It is also pretty cheap. backed by an Odroid HC2 moosefs cluster (though I am using an intel powered Odroid H2 for the master). I considered an HC2 for lizardfs. My problem with it is that it has a single SATA port, which means you're buying a $50 SBC for every hard drive in your cluster. For a single drive per node it is probably your best bet. However, my chunkservers are: ~$65 RockPro64 $20 used LSI HBA $5 wall wart $25 cheap ATX PSU $5 ATX power switch $5 extra SATA cables $5 powered 16x PCIe riser cable (these are a bit hard to find) That is ~$125, and will support 16 hard drives. You're saving money on the 3rd drive per node. If you want some kind of enclosure for the drives you'll pay maybe another $5/drive. The other option that might be worth considering if you don't mind losing some bandwidth to the drives is just using SATA3 and hubs/etc and external drives. I'm shucking external drives anyway. So, any SBC with a SATA3 port would work for that, with nothing else needed. I could see USB3 bandwidth (shared) being a constraint if you're rebuilding, but it would keep up with gigabit ethernet. Oh, and for any kind of NAS/etc solution make sure that whatever you get has gigabit ethernet. The Pi3s at least don't have that - not sure about the Pi4. Wouldn't help in a Pi3 anyway as I think the LAN goes through the internal USB2 bus - the Pi is pretty lousy for IO in general - at least conventional PC IO. That GPIO breakout is of course nice for projects. I was reading the Pi4 has true gigabit now, thanks to its USB3 ports. Dan
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On 2/29/20 11:13 PM, William Kenworthy wrote: Keep in mind that rpi are not the only cheap, capable arm hardware out there. I am using a number of odroid devices, including an N2 with a gentoo based kernel and a gentoo aarch64 userland. Its used for lxc containers for asterisk, dns, webdav, mail, calendaring and web running on the N2 backed by an Odroid HC2 moosefs cluster (though I am using an intel powered Odroid H2 for the master). Its all working rather well now the initial install/config stages are over. Part of the gain over the pi's is the use of eMMC storage over sdcards - almost 5 times faster in my tests. The 4G of ram has proven quite adequate so far - even the asterisk latency is better than my previous QEMU/KVM on intel setup. I have currently have rpi 1B, 3B and a zero and while the specs for the rpi4 are better ... its not the best out there. BillK I am aware of other devices but the RPi (afaict) is the only one sold within the country and not subject to import duty/fees/taxes and the related shipping delays because of those. :( Dan
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
"within the country"? :-) You must be American? On 2020-03-01 16:38, Daniel Frey wrote: On 2/29/20 11:13 PM, William Kenworthy wrote: Keep in mind that rpi are not the only cheap, capable arm hardware out there. I am using a number of odroid devices, including an N2 with a gentoo based kernel and a gentoo aarch64 userland. Its used for lxc containers for asterisk, dns, webdav, mail, calendaring and web running on the N2 backed by an Odroid HC2 moosefs cluster (though I am using an intel powered Odroid H2 for the master). Its all working rather well now the initial install/config stages are over. Part of the gain over the pi's is the use of eMMC storage over sdcards - almost 5 times faster in my tests. The 4G of ram has proven quite adequate so far - even the asterisk latency is better than my previous QEMU/KVM on intel setup. I have currently have rpi 1B, 3B and a zero and while the specs for the rpi4 are better ... its not the best out there. BillK I am aware of other devices but the RPi (afaict) is the only one sold within the country and not subject to import duty/fees/taxes and the related shipping delays because of those. :( Dan
Re: [gentoo-user] Gentoo on a cell?
On 2/29/20 8:49 PM, antlists wrote: On 29/02/2020 17:40, james wrote: is if the US government returns to the fundamental christian value system, that made our country great. Greed, un-bridled, is changing the quality of our lives, regardless of your personal belief systems. Our country? I think you mean YOUR country. And seen from outside, that greed has been there pretty much from its birth ... I'll agree about greed being the problem, though. But it's pretty difficult to return to a place you've never been, imho ... Cheers, Wol Sorry for the confusion, IMHO, each country belongs to the citizens, of that country; not the elites, wealthy or the politicians. All I was implying is that Gentoo, is a great 'game-changer' for each country. That uniqueness and 'midis-operandi' is up to the citizens to decide. Gentoo, is the best of all the linux/unix distro, imho, period. As users/devs of Gentoo, we are all on the same side of empowerment of the people. One-thousand apologies, if what I wrote, did not come seem congenial. hth, James
Re: [gentoo-user] Gentoo on a cell?
On 3/1/20 2:51 AM, n952162 wrote: well, without wanting to get political or anything ... but your point begs the question of what country yours is, or any other country, that's not grounded in greed? I'm currently a US citizen, but places like BC, Canada, Australia or Denmark, etc etc, could surely bribe me away from the USA, to part of a very, aggressive 'Gentoo for everybody' type of project That'd be my stipulation, is it is about Gentoo, but others, like (GEntoo->CoreOS->Redhat->IBM) could sponge off if the base gentoo distro. I'd insist of giving Gentoo the credit, but realize, when startup and large corps get involved, sure they have to create many jobs, and eventually make a profit. 5G everywhere, FAST, with all unlocked cell phones, would make me very, very happy. ymmv. hth James "Greed" is a stupid concept On 2020-03-01 02:49, antlists wrote: On 29/02/2020 17:40, james wrote: is if the US government returns to the fundamental christian value system, that made our country great. Greed, un-bridled, is changing the quality of our lives, regardless of your personal belief systems. Our country? I think you mean YOUR country. And seen from outside, that greed has been there pretty much from its birth ... I'll agree about greed being the problem, though. But it's pretty difficult to return to a place you've never been, imho ... Cheers, Wol Getting back on-topic, Rf tools on gentoo, did you guys check these aforementioned 2D tools out? https://github.com/chonyy/handoff-visualizer and https://github.com/chonyy/handoff-simulator I sure hope a 3D version, for certain terrains and wireless, autonomous vehicles, etc, grows out of these projects. Immense amounts of data could be handled, in Real-Time, if folks in many of the areas put up community servers to share data with mobile vehicles in an open network scenario. The Texas universities will surely be pioneers in this, just to get the kids and grad students thinking about what is now possible with 5G and justifying that almost complete multi (fiber Channel) 100 Gig/s educational network. If they open that bad-boy, to hitech startups, it'd cause another tech-job-explosion, in just that. A Texas water company, with 2,500 miles of right away is very interested in mirroring what the universities do, for commercial interest Also got a (personal) hit from a company in Vancouver BC. Very advanced, beautiful (and EXPENSIVE) there in Vancouver. It'd be a dream city/province to roll out 5G on with Gentoo toys.! over-excited about 5G, James
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On 3/1/20 7:40 AM, n952162 wrote: "within the country"? :-) You must be American? No. Dan
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
I am going to answer multiple points below: On 1/3/20 11:36 pm, Daniel Frey wrote: On 3/1/20 6:33 AM, Rich Freeman wrote: On Sun, Mar 1, 2020 at 2:13 AM William Kenworthy wrote: Keep in mind that rpi are not the only cheap, capable arm hardware out there. I am in Oz, delivery from HardKernel (the South Korean company behind the Odroid line) takes ~1 week. Shipping is mostly via FedEx, who are a bit pricy and for me means a 30m drive to get it from a delivery centre as I cant be home to take delivery - though the last delivery used a different, much more flexible delivery company. lizardfs and moosefs are very similar (originally a fork) - I went with moosefs as the community was better a few months ago - but lizardfs sounds like its getting back on track after some infighting (moosefs apparently went through the same thing) I find the moosefs documentation and community help are adequate, but ultimately they are are a commercial project with a free taster offering hence some limitations in design for the community version (such as single master/no shadow masters) that lizardfs doesn't have. I will move to lizardfs when I am satisfied they have their act together because of this - having a single master means taking the whole cluster offline when the master needs maintenance or fails which is painful For those wanting to run a lot of drives on a single host - that defeats the main advantage of using a chunkserver based filesystem - redundancy. Its far more common to have a host fail than a disk drive. Losing the major part of your storage in one go means the cluster is effectively dead - hence having a lot of completely separate systems is much more reliable - yes, I did try having 4 sata drives on an atom board and found it was easy to justify two more HC2's for the reliability. (note, its not the effect of the atom boards reliability I am pointing out, but the effect on the whole storage system of losing such a large percentage of capacity in one go - think maintenance, fail to startup etc. - its more common than you may think) Dale: there is no single reference - I just designed on the fly and did what was necessary to move from my existing KVM/Qemu architecture on a two powerfull intel systems, each with 8TB storage to an lxc container based system backed by moosefs. It uses an odroid arm based n2 for lxc, an arm based xu4 for management and software compilation (its not that powerfull, but is adequate), the HC2's (very similar to the xu4 - same arm architecture) and an H2 to run the mfsmaster and I am moving to ansible for management. Overall, I am finding less maintenance due to better backups and better reliability, considerably lower power consumption while overall performance seems better. Gotchas: 1. I originally got the 4gb ram N2 to run the mfsmaster software - tests were excellent - until I added multiple copies of a mailserver with nearly 20 years of history - hit swap and slowed to a crawl. So I now have an intel based Odroid H2 with 32GB ram - currently is using ~4.2GB ram for 7TB of 20TB used, but has hit over 32GbB and well into swap while converging. One admin in our local LUG is using 4xHC2 with the master running on one of the HC2's as a media server with no problems - its millions of small files added ina short time that causes the grief - there is a formula in the moosefs documentation allowing resources to be estimated - on the mailing list I saw a mention of a data centre using something like 150G ram and having problems! 2. The Odroid HC2 has a single sata port and a single usb port - I have 5 of them, two have a sata + a usb3 drive attached. I did try (on the N2) using multiple usb3 drives on the internal hub - disaster with way too much traffic through the hub - don't do it! 3. Storage options are an SD card (the faster the better - swap on an SD card ... sucks!) or eMMC (5xfaster than even a good SD card.) for the N2 and xu4 - HC2's can only do sdcard, or sata. The H2 can do sd card, sata, eMMC, or m2 NVME - this last really flies! Note that almost all arm system max out at 2 to 4GB of ram, so swap is usually needed for safety - depending on OOM killer resource management on a SAN type storage system is asking for corruption like I saw in one recent gentoo email. 4. xu4/HC2 are 32 bit arm v7, the N2 is 64 bit and runs aarch64 nicely - I copied my rpi images across repurposed them (hooray for emerge/portage!). I do not use, or have done any work on their graphics/multimedia capabilities 5. I want to move to all gentoo-sources kernels - the xu4/HC2's are still on the odroid kernel until I get around to it. The N2 was a lot of work, but ultimately successful, the H2 is standard amd64 EFI. Have fun! BillK
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On Sun, Mar 1, 2020 at 8:52 PM William Kenworthy wrote: > > For those wanting to run a lot of drives on a single host - that defeats > the main advantage of using a chunkserver based filesystem - > redundancy. Its far more common to have a host fail than a disk drive. > Losing the major part of your storage in one go means the cluster is > effectively dead - hence having a lot of completely separate systems is > much more reliable Of course. You should have multiple hosts before you start putting multiple drives on a single host. However, once you have a few hosts the performance improves by adding more, but you're not really getting THAT much additional redundancy. You would get faster rebuild times by having more hosts since there would be less data to transfer when one fails and more hosts doing the work. So, it is about finding a balance. You probably don't want 30 drives on 2 hosts. However, you probably also don't need 15-30 hosts for that many drives either. I wouldn't be putting 16 drives onto a single host until I had a fair number of hosts. As far as the status of lizardfs goes - as far as I can tell it is mostly developed by a company and they've wavered a bit on support in the last year. I share your observation that they seem to be picking up again. In any case, I'm running the latest stable and it works just fine, but it lacks the high availability features. I can have shadow masters, but they won't automatically fail over, so maintenance on the master is still a pain. Recovery due to failure of the master should be pretty quick though even if manual - just have to run a command on each shadow to determine which has the most recent metadata, then adjust DNS for my master CNAME to point to the new master, and then edit config on the new master to tell it that it is the master and no longer a shadow, and after restarting the daemon the cluster should be online again. The latest release candidate has the high availability features (used to be paid, is now free), however it is still a release candidate and I'm not in that much of a rush. There was a lot of griping on the forums/etc by users who switched to the release candidate and ran into bugs that ate their data. IMO that is why you don't go running release candidates for distributed filesystems with a dozen hard drives on them - if you want to try them out just run them in VMs with a few GB of storage to play with and who cares if your test data is destroyed. It is usually wise to be conservative with your filesystems. Makes no difference to me if they take another year to do the next release - I'd like the HA features but it isn't like the old code goes stale. Actually, the one thing that it would be nice if they fixed is the FUSE client - it seems to leak RAM. Oh, and the docs seem to hint at a windows client somewhere which would be really nice to have, but I can't find any trace of it. I only normally run a single client but it would obviously perform well as a general-purpose fileserver. There has been talk of a substantial rewrite, though I'm not sure if that will actually happen now. If it does I hope they do keep the RAM requirements low on the chunkservers. That was the main thing that turned me off from ceph - it is a great platform in general but needing 1GB RAM per 1TB disk adds up really fast, and it basically precludes ARM SBCs as OSDs as you can't get those with that much RAM for any sane price - even if you were only running one drive per host good luck finding a SBC with 13GB+ of RAM. You can tune ceph to use less RAM but I've heard that bad things happen if you have some hosts shuffle during a rebuild and you don't have gobs of RAM - all the OSDs end up with an impossible backlog and they keep crashing until you run around like Santa Claus filling every stocking with a handful of $60 DIMMs. Right now lizardfs basically uses almost no ram at all on chunkservers, so an ARM SBC could run dozens of drives without an issue. -- Rich
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
On 2/3/20 10:40 am, Rich Freeman wrote: On Sun, Mar 1, 2020 at 8:52 PM William Kenworthy wrote: For those wanting to run a lot of drives on a single host - that defeats the main advantage of using a chunkserver based filesystem - redundancy. Its far more common to have a host fail than a disk drive. Losing the major part of your storage in one go means the cluster is effectively dead - hence having a lot of completely separate systems is much more reliable Of course. You should have multiple hosts before you start putting multiple drives on a single host. However, once you have a few hosts the performance improves by adding more, but you're not really getting THAT much additional redundancy. You would get faster rebuild times by having more hosts since there would be less data to transfer when one fails and more hosts doing the work. So, it is about finding a balance. You probably don't want 30 drives on 2 hosts. However, you probably also don't need 15-30 hosts for that many drives either. I wouldn't be putting 16 drives onto a single host until I had a fair number of hosts. As far as the status of lizardfs goes - as far as I can tell it is mostly developed by a company and they've wavered a bit on support in the last year. I share your observation that they seem to be picking up again. In any case, I'm running the latest stable and it works just fine, but it lacks the high availability features. I can have shadow masters, but they won't automatically fail over, so maintenance on the master is still a pain. Recovery due to failure of the master should be pretty quick though even if manual - just have to run a command on each shadow to determine which has the most recent metadata, then adjust DNS for my master CNAME to point to the new master, and then edit config on the new master to tell it that it is the master and no longer a shadow, and after restarting the daemon the cluster should be online again. The latest release candidate has the high availability features (used to be paid, is now free), however it is still a release candidate and I'm not in that much of a rush. There was a lot of griping on the forums/etc by users who switched to the release candidate and ran into bugs that ate their data. IMO that is why you don't go running release candidates for distributed filesystems with a dozen hard drives on them - if you want to try them out just run them in VMs with a few GB of storage to play with and who cares if your test data is destroyed. It is usually wise to be conservative with your filesystems. Makes no difference to me if they take another year to do the next release - I'd like the HA features but it isn't like the old code goes stale. Actually, the one thing that it would be nice if they fixed is the FUSE client - it seems to leak RAM. Oh, and the docs seem to hint at a windows client somewhere which would be really nice to have, but I can't find any trace of it. I only normally run a single client but it would obviously perform well as a general-purpose fileserver. There has been talk of a substantial rewrite, though I'm not sure if that will actually happen now. If it does I hope they do keep the RAM requirements low on the chunkservers. That was the main thing that turned me off from ceph - it is a great platform in general but needing 1GB RAM per 1TB disk adds up really fast, and it basically precludes ARM SBCs as OSDs as you can't get those with that much RAM for any sane price - even if you were only running one drive per host good luck finding a SBC with 13GB+ of RAM. You can tune ceph to use less RAM but I've heard that bad things happen if you have some hosts shuffle during a rebuild and you don't have gobs of RAM - all the OSDs end up with an impossible backlog and they keep crashing until you run around like Santa Claus filling every stocking with a handful of $60 DIMMs. Right now lizardfs basically uses almost no ram at all on chunkservers, so an ARM SBC could run dozens of drives without an issue. Everything bad you hear about ceph is true ... and then some! I did try, but this was some years ago so hopefully its better now. The two biggies were excessive network requirements (bandwidth, separation) and recovery times with frequent crash and burn. There are ceph features I would really like to use (rbd, local copies with much simpler config, ...) but moosefs is a lot more bullet proof on lesser resource requirements though I did find properly pruned vlans on a smartswitch separating the intra-cluster from external requests made a noticeable difference. moosefs has a windows client but its only available with the paid version. The master/shadow-master and auto failover is only available through the paid version - for the community you have to stop the master, copy the files then change DNS etc. before restarting the new master - cant really do it online even when scripted - its painful with downtime and I had dns caching issues tha
Re: [gentoo-user] Rasp-Pi-4 Gentoo servers
Sorry for the presumption. On 2020-03-02 01:08, Daniel Frey wrote: On 3/1/20 7:40 AM, n952162 wrote: "within the country"? :-) You must be American? No. Dan
