> If your purpose is just to solve a small inconvenience of the `source' > builtin mixing the namespaces of local scripts, "libraries", and > executables, I think the suggested `source -i' or `source -l' would be > fine. I think no additional reasoning is needed.
That is my purpose! I think I went a bit too far when trying to advocate for the feature. In my mind, this solution already creates a module system. I certainly intend to use it as one should it be merged. So I thought it would be a good argument to justify its inclusion. However, it sparked a much wider discussion which I was unprepared for. I'm going to learn more about the systems in order to discuss them more coherently. I was thinking about creating a new thread in help-bash and inviting the maintainers of every project that was cited to discuss it in more detail. It's my intention to work with everyone. > The question is whether you would still value `source -i' if it were > not useful for module managers at all. It would still be an extremely valuable feature for me. > To me, you seem to raise thebar by yourself. Where is your bar? I was just trying to convince everyone that it would be a good feature for inclusion. I think I went a bit too far there. > Can I call what you describe the "module loader"? > Let me call the above the "module manager". Yes, I think those are more precise terms. I agree with them. > I understand that the purpose of the *present* patch is the module > loader, but I thought you tried to prepare the primitive useful for a > unique module manager. Yes. I think it was a mistake for me to pursue that line of reasoning. For my use case, this module loader is the only primitive needed. My module management consists of cloning my dotfiles repository which can be found here: https://github.com/matheusmoreira/.files This configures all my applications including bash and also installs all my scripts in ~/.local/bin. These scripts are growing in complexity so I thought about ways to modularize them, extract common and useful functions and make them independently loadable by other scripts. So I thought: I have ~/.local/bin, why not ~/.local/lib/bash? I can just put the function definitions in module files, and put those module files in ~/.local/lib/bash. Then I just gotta get bash to load those modules. Getting bash to load those modules led to this patch. That is the exact use case supported by this proposed feature. For a moment I actually thought it was so simple that it ought to work for everything and for everyone. I mean, it's the exact same approach used by bash to find commands and executables. It's also got precedent in other Linux and *nix software. The GNU dynamic linker and loader for example uses LD_LIBRARY_PATH to achieve the exact same goal. Simple can be deceptive though. I should probably at least learn about the existing solutions before trying to discuss them. Otherwise I'll just end up putting my foot in my mouth. > The source builtin is already the primitive feature that can be used > to develop a module manager on top of it. > For these reasons, most module managers actually resolve > the path by itself using their rules and specify *the absolute path* > to the source builtin. I think I understand now. Yeah, it looks like the existing module managers have quite complex name resolution requirements. My solution to the file extensions problem for example is to just not have file extensions at all, thereby eliminating the need to do any file name processing to begin with. This is compatible with a simple implementation that just sources the given file name. This is a good thing: the patch will not conflict with existing systems. Absolute path names will still get loaded properly. I was hoping that this feature would at least be of some use to module managers but depending on the complexity of their path resolution algorithms there might be no sensible way for them to make any use of it. It's probably just wishful thinking on my part. > However, in that case, I don't see much difference from > « PATH=<modpath> source » (as far as the module just defines a set of > functions). It's true that the code is more or less the same from the perspective of module managers. However, building this into bash does provide a subtle benefit: separation between executable and module paths. Chet Ramey outlined it in his email: https://lists.gnu.org/archive/html/bug-bash/2024-05/msg00038.html > PATH=$BASH_IMPORT_PATH:$PATH source > or > PATH=$BASH_IMPORT_PATH source > > You could force the latter if you are sure that the sourced files will > not attempt to run external commands. The builtin would allow forcing the latter while leaving PATH alone. No need to ensure that they'll not attempt to run external commands. The builtin is therefore less limited than a pure PATH solution. This would be most useful for a module manager that doesn't resolve the module path by itself. You noted that most of them already do that so I'm not sure how many would benefit from this. If they're resolving the paths themselves, they might even support this already. Perhaps some simpler module managers will benefit though. > the suggested feature essentially just reduces one line > « local PATH=<modpath> » from the manager code. > Does reducing one line from the manager code > deserve the new feature in the Bash language? I agree that a one line of code reduction is underwhelming but that's not the only reason to include it. The users are the main beneficiaries. This gives users a simple way to load modules without using any module managers at all. >From the perspective of the user, it potentially reduces thousands and thousands of lines of code. Still, I did and still do hope that it will make life easier for the maintainers of the module managers too. If it manages to improve the code even just a bit, I'll still call it a win. > While you say you just work on the module loader and not > intend to deny the module managers, you try to promote > not using the module manager here. Yes. I think module managers should not be strictly necessary. I think bash should have a functional module loading facility. Bash need not have fancy package management features but it should be able to load the modules/libraries all by itself. For a user like me, that's sufficient. > Would you try to exclude the possibility > of a module manager in the future of Bash? Certainly not! It is not my intention to exclude them. I just think that they should all use the same interface. My idea is that both interactive users and module authors would write `import x` in order to bring in a dependency. Alternatively, they would write `source --library x`. How exactly the module managers bring the `x` into bash's module load path should be defined by the implementation. My idea was that module managers would munge the BASH_LIBRARIES_PATH variable contents so that they contained the directories with the modules they managed. They could solve the file extension problem with symlinks: create extension-less and .sh symbolic links pointing to the .bash module file. Perhaps that could work. Having uniform interfaces is exactly what allows the implementation to be swapped out later without touching existing code. This article was very influential in the way I think and will probably explain what I'm trying to get at better than I ever will: https://journal.stuffwithstuff.com/2010/09/18/futureproofing-uniform-access-and-masquerades/ Anyway, that's what I had in mind when I wrote all these emails. It might turn out that it's impossible to implement things that way. It could also be that it is possible but simply too painful to do it. Maybe there are too many existing systems and changing them all would be a monumental effort. In that case it can't be helped and I'll just accept it. I'll just be happy that `source --library` was considered. I don't have any intention of excluding anyone at all. > If you start to talk about the vision about the Bash language, that > would certainly raise the bar. You'll have to tell a story about how > the flag `-i' or the import builtin changes the language in detail > using a concrete example of an ecosystem and why it is the best > approach among other different possibilities. To that end, I described in this reply the exact use case I have which inspired this proposal and patch. It's certainly nothing big or complex or even featureful but I believe it is simple and functional and that a lot of people might benefit from it. Those who need more would be well served by sophisticated module managers. Thanks for the feedback, Matheus
