Hi all! It follows a head's up for the plan of making it easier to work with our pve-manager git repository by rewriting its history to filter out huge artefacts.
This will only affect developers, nothing in the current pve-manager Debian package will change. # Background Our current pve-manager git repository is huge (> 500 MB) and this is mostly due to hosting various huge copies of ExtJS, both as ZIP archive and as extracted version directly in its git history. Nowadays, well since Q1 of 2017 (before Proxmox VE 5), those huge artefacts are not used anymore, as we slit the one still in use, like the ExtJS GPL source code, out to its own repo, without any ZIP archives. But, git being git and providing a full history of every change still needs to hold copies of those artefacts in its CAS object store, one cannot really mask those in any (for development) ergonomic way. # Proposed Solution I'll use the git filter-repo [0] tool, a replacement for filter-branch with better UX and less potential for getting it wrong, to rewrite the history, filtering out any problematic artefact or directory. For this I'll use the following file-list www/ext6 www/ext5 www/ext4 www/touch po glob:*.zip used as inverted match via the following command: git filter-repo --invert-paths --paths-from-file ~/pve-manager-inverted-filter-paths Then, I'd rename the current "pve-manager.git" hosted at git.proxmox.com to "pve-manager-legacy.git", so it will still be able as reference for ancient history, providing the possibility to build pre PVE 5 pve-manager packages (why ever one would want/needs to do that). A new repo, with the same name "pve-manager.git", would then get created and the now cleaned up git repo pushed to it. # Result The result of above command measured by .git disk usage: Before: 551 MB After: 26 MB So a huge reduction. # Fallout This naturally has some fallout for developers currently working patch series, similar to any force-push (which we normally avoid at all cost). Rebasing won't work IIUC, but as the source file layout won't change, you can simply use "git cherry-pick <rev-range>" if you have the before filter and after filter remotes & branches in the same git repo. Otherwise, one can always use "git format-patch -o ~/patches/ <rev-range>" in the old repo to export patches cleanly, and then use "git am -3 ~/patches/*.patch" in the new repo. Note that git commit hash references inside commit messages of pve-manager will get rewritten, so here won't notice anything. Commit references from other repos are naturally untouched, but pve-manager being a leave package means that it won't have that many in other repos. I'll safe a copy of the old -> new commit reference map that git filter-repo produces, ensuring we got full transparency. # Date of Change I'll probably carry above out tomorrow, Saturday 2023-05-27, sometimes between 10:00 CEST and day's end, but writing today for a short heads-up. For the record: this plan was discussed with Dietmar Maurer and Dominik, and as said, this is "only" affecting developers. And yes, it is a bit of a nuisance and generating some churn, but we talked about doing this every other year, and it won't get better on it's own, so let's just finally go for it. cheers Thomas _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
