Super cool idea - I too wanted to refer to blockchain methods to avoid data being tampered. Ceph would need a completely different distribution coded for such storage, however we could say that the fundamentals are already in place?
Best, Laimis J. > On 7 Apr 2025, at 18:23, Tim Holloway <t...@mousetech.com> wrote: > > Additional features. > > * No "master server". No Single Point of Failure. > > * Resource location. A small number of master servers kept in sync like DNS > with tiers of secondary resources. I think blockchains also have a similar > setup? > > * Resource identification. A scheme like LDAP. For example: > > cn=library,catalog=dewey,filing=504.3,... > > cn=library,country=us,catalog=libraryofcongress,... > > country=us,agency=nih,department=... > > cn=upc,isbn=... > > A document/document set should have a canonical name, but allow alternate > names for ease of location, such as author searches, general topics and the > like. > > I considered OIDs as an alternative, but LDAP names are more human-friendly > and easier to add sub-domains to without petitioning a master registrar. Also > there's a better option for adding attributes to the entry description. > > On 4/7/25 09:39, Tim Holloway wrote: >> Yeah, Ceph in its current form doesn't seem like a good fit. >> >> I think that what we need to support the world's knowledge in the face of >> enstupidification is some sort of distributed holographic datastore. so, >> like Ceph's PG replication, a torrent-like ability to pull from multiple >> unreliable sources, a good indexing mechanism and, protections against >> tampering. Probably with a touch of git as well. >> >> I'm sure there's more, but those are items that immediately occur to me. >> >> Tim >> >> On 4/7/25 09:10, Alex Buie wrote: >>> MooseFS is the way to go here. >>> >>> >>> I have it working on android SD cards and of course normal Linux servers >>> over the internet and over Yggdrasil-network. >>> >>> One of my in-progress anarchy projects is a global hard drive for all of >>> humanity’s knowledge. >>> >>> I would LOVE to get involved with this preservation project technically in >>> a volunteer capacity. I can build a cutting edge resilient distributed >>> storage system for cheaper than anything currently on the market. >>> >>> Please reach out or pass along my email. >>> >>> Alex >>> >>> >>> On Sun, Apr 6, 2025 at 11:08 PM Linas Vepstas <linasveps...@gmail.com> >>> wrote: >>> >>>> OK what you will read below might sound insane but I am obliged to ask. >>>> >>>> There are 275 petabytes of NIH data at risk of being deleted. Cancer >>>> research, medical data, HIPAA type stuff. Currently unclear where it's >>>> located, how it's managed, who has access to what, but lets ignore >>>> that for now. It's presumably splattered across data centers, cloud, >>>> AWS, supercomputing labs, who knows. Everywhere. >>>> >>>> I'm talking to a biomed person in Australias that uses NCBI data >>>> daily, she's in talks w/ Australian govt to copy and preserve the >>>> datasets they use. Some multi-petabytes of stuff. I don't know. >>>> >>>> While bouncing around tech ideas, IPFS and Ceph came up. My experience >>>> with IPFS is that it's not a serious contender for anything. My >>>> experience with Ceph is that it's more-or-less A-list. >>>> >>>> OK. So here's the question: is it possible to (has anyone tried) set >>>> up an internet-wide Ceph cluster? Ticking off the typical checkboxes >>>> for "decentralized storage"? Stuff, like: internet connections need to >>>> be encrypted. Connections go down, come back up. Slow. Sure, national >>>> labs may have multi-terabit fiber, but little itty-bitty participants >>>> trying to contribute a small collection of disks to a large pool might >>>> only have a gigabit connection, of which maybe 10% is "usable". >>>> Barely. So, a hostile networking environment. >>>> >>>> Is this like, totally insane, run away now, can't do that, it won't >>>> work idea, or is there some glimmer of hope? >>>> >>>> Am I misunderstanding something about IPFS that merits taking a second >>>> look at it? >>>> >>>> Is there any other way of getting scalable reliable "decentralized" >>>> internet-wide storage? >>>> >>>> I mean, yes, of course, the conventional answer is that it could be >>>> copied to AWS or some national lab or two somewhere in the EU or Aus >>>> or UK or where-ever, That's the "obvious" answer. I'm looking for a >>>> non-obvious answer, an IPFS-like thing, but one that actually works. >>>> Could it work? >>>> >>>> -- Linas >>>> >>>> >>>> -- >>>> Patrick: Are they laughing at us? >>>> Sponge Bob: No, Patrick, they are laughing next to us. >>>> _______________________________________________ >>>> ceph-users mailing list -- ceph-users@ceph.io >>>> To unsubscribe send an email to ceph-users-le...@ceph.io >>>> >>> _______________________________________________ >>> ceph-users mailing list -- ceph-users@ceph.io >>> To unsubscribe send an email to ceph-users-le...@ceph.io >> _______________________________________________ >> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe send an email to ceph-users-le...@ceph.io > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
