On 2024-01-28 18:32, Giovanni Biscuolo wrote: > Hi Nicolas, > > Nicolas Graves <ngra...@ngraves.fr> writes: > > [...] > >> This is not always true. Git-LFS also has the concept of Custom Transfer >> Agents, which in some cases do not need a running server. One example is >> lfs-folderstore, which can simply use a remote directory as a LFS >> remote. > > thanks, i didn't know about custom transfer agents, the use withous an > API server is documented here: > > --8<---------------cut here---------------start------------->8--- > > In some cases the transfer agent can figure out by itself how and where > the transfers should be made, without having to query the API server. In > this case it's possible to use the custom transfer agent directly, > without querying the server, by using the following config option: > > lfs.standalonetransferagent, lfs.<url>.standalonetransferagent > > Specifies a custom transfer agent to be used if the API server URL > matches as in "git config --get-urlmatch lfs.standalonetransferagent > <apiurl>". git-lfs will not contact the API server. It instead sets > stage 2 transfer actions to null. "lfs.<url>.standalonetransferagent" > can be used to configure a custom transfer agent for individual > remotes. "lfs.standalonetransferagent" unconditionally configures a > custom transfer agent for all remotes. The custom transfer agent must be > specified in a "lfs.customtransfer.<name>" settings group. > > --8<---------------cut here---------------end--------------->8--- > (https://github.com/git-lfs/git-lfs/blob/main/docs/custom-transfers.md#using-a-custom-transfer-type-without-the-api-server) > > some examples: > > 1. git-lfs-agent-scp: A custom transfer agent for git-lfs that uses scp > to transfer files. This transfer agent makes it possible to use > git-lfs in situations where the remote only speaks ssh. This is > useful if you do not want to install a git-lfs server. (MIT license, > written in C, URL: https://github.com/tdons/git-lfs-agent-scp) > > 2. git-lfs-rsync-agent: The rsync git-lfs custom transfer agent allows > transferring the data through rsync, for example using SSH > authentication. (MIT license, written in Go, URL: > https://github.com/excavador/git-lfs-rsync-agent) > > 3. git-lfs-agent-scp-bash: A custom transfer agent for git-lfs that uses > scp to transfer files. This is a self-contained bash script designed > for seamless installation, requiring no prerequisites with the > exception of the external command scp. It enables to use git-lfs even > if you can not use http/https but ssh only. (MIT License, written in > bash, URL: https://github.com/yoshimoto/git-lfs-agent-scp-bash) > > So yes: we could use git-lfs without a git-lfs server and set an rsync > or scp transfer agent for each remote (documenting it for users, since > this must be done client-side) > > It's not at all as powerful as the location tracking features of > git-annex but... doable :-)
One downside however : For some reason, Custom Transfer Agents are rarely well written, well supported projects. I had to write one myself for my use-case, I now have a good understanding of the protocol which is definitely simple, but it's a necessary thing to note that despite this rather easy extensibility, few CTAs are good/reliable in the long run. > > [...] > >>> Another important limitation of Git-LFS is that you cannot delete >>> (remotely stored) objects [1], with git-annex is very easy. >> >> Probably true, haven't encountered the use-case yet. > > IMHO this is a very important feature when you have to manage media > archives. Depends on the use-case! If you're just looking for an archival tool of a media with one unmutable version which you want to support for an indefinite amount of time, doesn't matter that much. Gitlab and Github documentation say it's possible with the git-filter-repo extension, but that is indeed not easy. https://docs.gitlab.com/ee/topics/git/lfs/#removing-objects-from-lfs > > [...] > >> Just a note on upsides of Git-LFS : >> - integration with git is better. A special magit extension to use >> git-lfs is not needed, whereas it is with git-annex. > > true :-D > >> - less operations: once I know which files will be my media files, I >> have less headaches (basically the exact git experience, you don't have >> to think about where I should `git add` or `git annex add` a file). > > it's the same with git-annex, you just have to configure/distribute a > .gitattributes file, i.e.: > > --8<---------------cut here---------------start------------->8--- > > * annex.largefiles=(largerthan=5Mb) > * annex.largefiles=(not(mimetype=text/*)) > > --8<---------------cut here---------------end--------------->8--- > > see https://git-annex.branchable.com/tips/largefiles/ for a description > of this feature Nice! I haven't experimented with this. Funny how it's one extension of a git extension that provides what LFS does natively. > >> It's indeed less copyleft though. Simpler, but also maybe less adapted >> to this use-case. > > With git-annex everyone can set up a "git-annex enabled" server > (although haskel dependency is a limitation since it's unsupported in > many architectures)... or use one of the available special remotes. > > Thanks! Gio' One upside I've forgotten that git-annex may also provide (does it ?) : if the CTA is well-written, progress update during upload/download is quite reassuring when sending heavy files. -- Best regards, Nicolas Graves
