> On Jun 28, 2024, at 10:12 AM, Rowan Tommins [IMSoP] <imsop....@rwec.co.uk > <mailto:imsop....@rwec.co.uk>> wrote: > Namespaces don't require autoloading, and autoloading doesn't require one > file per class.
No they do not, but the design of each was heavily intertwined with each other resulting in a less than optimal design, IMO. So, are you arguing to keep one and eject the other for modules, and if so which are you arguing we eject? Autoloading? Or are you arguing to keep both for modules, in which case your argument above is moot? > To compile a program with multiple source files, in any language, you need > one of two things: > > a) A list of files you want to compile. Maybe auto-generated, maybe done with > a recursive iteration over a directory, but ultimately the compiler needs a > file path to process. Recursion is only needed if modules are hierarchical in nature. > b) A way for the compiler to tell, based on some symbol it wants to resolve, > which file should be compiled. That presumes the compiler did not simply generate an AST from the list of files. > PHP originally provided only option (a), via the include and require > keywords. Autoloading adds option (b), where you provide a function which > takes a class name and does *whatever you want* to find the definition. And that "whatever you want" takes execution time (and tracing through when you are debugging.) But when you look at many other languages loading is an implementation detail that PHP chose to hoist onto userland developers when PHP could have established the rules to handle it more performantly without userland involvement. Or is there some aspect of autoloading that could not be handled by PHP itself? Note I am asking only within the propose scope of modules, which we could constrain to optimize their runtime use. > I think it might be time to re-visit the tooling around option (a), as > OpCache makes the cost of eagerly loading a list of files much lower than it > was when autoloading was added. Now you are getting somewhere. Imagine that each module — which could equal a single directory — could have a pre-compiled op-cache which is essentially what I proposed in other recent emails. > That could be as simple as include_all($directory), or as fancy as > include_from_manifest_file($some_crazy_binary_file_format); either could be > implemented right now in userland, because it all eventually comes down to > calling include or require. meh. That sounds like a way to avoid discussing the ways in which smartly designed modules could really improve PHP. >>> My opinions match Larry's almost exactly: I want package-level >>> optimisation, and package-private declarations. But I don't want to rewrite >>> my entire codebase to start using a completely different naming system. >> I can't see how package-privates would be of any value to you *unless* you >> rewrite your codebase. > Simple: I have private Composer packages, right now, that group all their > classes under a particular namespace prefix. I want to be able to mark some > classes in that namespace as "internal". Not simple, although I admit I am being pedantic about words used here, but for a reason. I asked about "package-privates," you responded with "namespace-privates." Adding private to namespaces is orthogonal to the discussion of packages. To require that packages be constrained to have all the same warts as namespaces and existing PHP code simply so you can have namespace-privates is short-sighted (and IMO a bit selfish.) Alternately, namespaces could get private scope in parallel to having modules be considered. That would allow modules to gain improvements that we could not get by having to maintain BC with namespaces. Which causes me to ask: If you have really wanted namespace private why has it been six years since it was even last mentioned on the list, and four years since last discussed? https://externals.io/message/101323 <https://externals.io/message/101323> Why has there not been an RFC since this one https://wiki.php.net/rfc/namespace-visibility <https://wiki.php.net/rfc/namespace-visibility> six years ago, that was not even voted on? Why is it that when the topic of addressing modules/packages comes up — which has been talked about numerous times in the past six years — do you now bring up namespace privates in a manner that would effectively torpedo goals of the modules discussion, at least from the perspective of the OP and myself? If namespace private were really something important to you, why haven't you championed it before, rather than hijack a discussion about the benefits we could get from modules if not constrained by namespaces? > I do not want to change every place that uses the existing classes to > reference "ModuleName@ClassName" instead of "NamespacePrefix\ClassName", or > change every "use" to "import from". Then don't. Champion this RFC https://wiki.php.net/rfc/namespace-visibility <https://wiki.php.net/rfc/namespace-visibility> and get what you want. But please don't argue against a discussion on modules because you want a feature that can be gotten orthogonally. (If you must argue against it, make arguments for which accommodations for your preferences cannot be found.) > Code doesn't existing in isolation; if Symfony Mailer is re-published as a > "module", every single application that uses it needs to change their code > from referencing namespaced classes, to having "import" statements. And that is bad, how? But before you answer, it just means that instead of a `use` statement in your existing code you change to a `import` statement. You'd then of course need to changes — if applicable — to call the new Symphony Mailer, but you'd have to do that with or without modules. Or is there something else I am missing? >> As for package-level optimisation, you'll need to give examples of what you >> mean there as I don't want to wrongly assume. > > Currently, OpCache only optimises per file, because it can't guarantee how > files will be used together. > > A simple example is function fallback: if you could declare a package as > "completely loaded", OpCache could replace all references to "strlen" with > "\strlen", knowing that no namespaced function with that name could be added > later. Thank you for elaborating on that. So, champion an RFC to improve OpCache for namespaces. That need not impose on the discussion about modules. Further, and this is what is nice about being able to discuss modules not having to be compatible with namespaces, if there are aspect of namespaces that make optimization hard or impossible then we could potentially set up rules of modules that make similar optimizations easy and/or possible. EVEN further, consider the fact that in PHP all class members are public by default. One thing we could have in modules is to go back to short `var` and eliminate both `private` and `protected` modifiers and only have `public` with the default behavior being what is `private` outside of modules. `protected` would no longer be needed as we would have module scope which is defacto-`protected`. Classes could be `final` by default in modules and then we could modify them with an `open` keyword (thanks to Lynn for that one.) And so on. In other words, if we could treat modules as their own sandbox, we could get fix many of the regrettable former design choices of the PHP language — some of which are to make PHP be beginner friendly — and potentially re-energize people who once looked at PHP and dismissed it to give it another look. > What I meant was: we can't just treat namespaces and modules as completely > separate things, and assume that every code file will be using one style or > the other. We have to imagine the user experience when there is a mix of the > two. Why can we not just treat namespaces and modules as completely separate things? > I can't imagine it being pleasant to have a mix of "import" and "use" > statements at the top of a file, with different and even conflicting > semantics. That feels like a frivolous concern when compared to the benefits we could see with modules, especially when there would be ways to mitigate your stated concerns here. If you don't like to see imports, but your imports in a namespace and then "use" that namespace. Or we could allow "use module" instead of (or in addition to) "import" and then it could look more pleasant for you. As for conflicting semantics: 1.) I'm not seeing how those could be significant in the using/importing file, and 2.) Isn't dealing with conflicting semantics just a part of programming? 3.) Don't "use" and "use function" have conflicting semantics? God knows that "use" by itself has many confusing semantics, which "import" could avoid. > Perhaps I didn't word the question well. What I'm really asking, as someone > who's never used JS or Go modules, is why I'd want to write "import", rather > than referencing a global name in a hierarchy. "use module" would work just as well as "import"; the "import" is not special, the module scoping and features are what is valuable here. For specifics see my other recent emails on the subject. If they do not explain, please ask again with specifics. > That's really all I mean by "making it compatible with namespace": I want > "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by > having a way to mark all classes under "\Foo\Bar" as belonging to a > particular package. And that is what I am trying to get away from. First the backslash — because when using in reflection or other dynamic programming they have to be escaped which can lead to escaping errors. I know you don't care, but I and others do. Second, the hierarchy. Because there is no constraint on hierarchy PHP subtly encourages developers — as if sirens of the Odyssey — to create large hierarchies. I even find myself doing it as I fighting myself against it. The reasons hierarchy is bad is: 1.) larger hierarchies grow conceptual complexity, 2.) they place no limit on package growth as you can always create subdirectories, 3.) they make it harder to "see" all the code files in one place (a single directory), 4.) they constrain where code is located when there are benefits to a different layout > That's really all I mean by "making it compatible with namespace": I want > "new \Foo\Bar\Baz;" to be able to refer to a "packaged" class, probably by > having a way to mark all classes under "\Foo\Bar" as belonging to a > particular package. Revisiting this, why is it important to you that "new \Foo\Bar\Baz" refer to a "packaged" class vs a namespaced class, assuming you had namespace-private and OpCache improvements? Why can't you still just use the namespaces you prefer and let "packages" (modules) improve in other ways? I am trying my best not to make this ad-hominem so forgive me but I do have to ask if this is just not a case of "I am comfortable doing it the way I have been doing it and do not want to consider changing," maybe? Note I am asking that question limited to the one statement I quoted above, not on the broader discussion. >> 6. Modules and packages are typically small-scoped to a single directory and >> it is a code smell to have many different packages tightly coupled, as is >> the case with namespaces. Forcing modules to munge with namespaces would >> mean most modules would be written with those code smells for years because >> that will be how everyone doing a quick shift from namespace to module will >> write their modules. > > Again, this is entirely about code style, and not something the language can > control. A language cannot control it, but a language can encourage or discourse it. And the PHP language encourages a large amount of file and directory bloat. One only need to compare the number of files in most PHP libraries to the number of files in JS or Go package to see that the nature of a language clearly does not influence. To bring stats vs. opinion I asked ChatGPT what the two equivalent packages are to Symphony for JS and Go respectively and it suggested ExpressJS and Gin. So I cloned them to see the number of files and directories each has. From the root of each repo: Project Files Dirs Symfony: 12,504 2,162 ExpressJS: 259 87 Gin(GoLang): 145 30 The comparison might not be completely fair given how much longer Symfony has been around, but they all target the same use-case so even if there is less functionality in ExpressJS or Gin. Given that I think that well over an order of magnitude more files is a really odiferous code smell, and is thanks to the language which admittedly cannot "control" layout, but definitely influences it. Am I wrong? Present any other relatively equivalent project comparisons you please. Here are the bash commands to count files and dirs: find /path/to/subdirectory -type f | wc -l find /path/to/subdirectory -type d | wc -l > Also, the JS insistence on having a separate package for every tiny function > is a common source of criticism, so personally I am very happy that PHP > packages are generally larger than that. I can't speak for the OP, but nothing I am proposing is advocating for separate packages for every tiny functions. Nothing. Instead I am advocating for packages that are mostly in a few directories instead of almost two magnitudes more! >> That said, maybe the best solution is to NOT put the stake in the ground >> right now and say "They must be namespace compatible" or "They must not be >> namespace compatible" but move forward with an open mind so that we can >> tease out exactly how namespaces would constrain modules and and then make >> the decision later for what would be in the best interest of moving PHP into >> the future. > > If and when an actual problem arises, let's discuss it. Not "problems" but instead "opportunities." I have already pointed out numerous opportunities in this email and one of my recent emails. >>> What specifically stops us doing all the things you've been discussing >>> around loading, and visibility, etc, in a way that's compatible with the >>> 400_000 packages available on Packagist, and billions of lines of existing >>> code? >> >> You speak as if I am proposing getting rid of namespaces and making those >> 400_000 packages available on Packagist, and billions of lines of existing >> code not work in PHP. Of course not. > > No, I'm saying that every one of those packages could benefit if we make > incremental changes. Maybe. What benefits can you envision you would get if PHP made namespaces==modules compared with the benefits I have mentioned for making modules not be constrained to compatibility with namespaces (besides private and OpCache as we already discussed you pursue for namespaces?) Can we get precompiling for modules in a directory and written to a `.php.module` file? We can't do that with namespaces because scanning recursively could take too long at runtime. Can we get default private for all symbols and class members in namespaces? No, that would be a huge BC break. Can we get namespaces to be first-class AST participants? If yes, why have we not done it before? I could go on, but this email is getting loooong. > I don't want to couple it so that you can't have "package private" without > also switching to some new "advanced" dialect of the language, and I don't > see any reason why we need to do so. And I am not advocating that. I am advocating you should get "namespace private." Hey RFC is already written! https://wiki.php.net/rfc/namespace-visibility <https://wiki.php.net/rfc/namespace-visibility> And most of the other benefits of modules as I am proposing would be BC breaks so you could not get them in namespaces anyway. Unless you can come up with something besides private and opCache I had not considered. > Maybe package scoped declares could allow opting in to certain checks, but I > don't think "is in a package" and "has been audited for a load of extra > breaking changes" should be set by the same flag. I am not aware of any discussion of opting in, flags, nor auditing with respect to modules. -Mike
