Thanks David for investigating the topic thoroughly! I wasn't expecting all that when I filed a one-line ticket six years ago :-) So, here's a bunch of opinions.
Before I start, I'd like to quote the intro to the Media class <https://docs.djangoproject.com/en/3.0/topics/forms/media/>: > Rendering an attractive and easy-to-use Web form requires more than just HTML > - it also requires CSS stylesheets, and if you want to use fancy « Web2.0 » > widgets, you may also need to include some JavaScript on each page. The exact > combination of CSS and JavaScript that is required for any given page will > depend upon the widgets that are in use on that page. That was the reasonable thing to do before single-page apps, asset pipelines, bundlers and code splitting. It's still a fairly reasonable thing to do on a website that relies on good old HTML forms but would benefit from "fancy « Web2.0 » widgets". 1. Django shouldn't do anything (besides what django.contrib.staticfiles already does) for projects that use webpack (or any other bundler) The correct approach with a bundler is to bundle all JavaScript code and, if needed, to optimize with code splitting. Attempting to include only what's needed on each page, like Media does, will usually be counterproductive, because it will general different bundle for different pages and defeat caching by the browser. Then I know two techniques for integrating the frontend code with Django: - the single page app <https://fractalideas.com/blog/making-react-and-django-play-well-together-single-page-app-model/> (website and API on separate domains) — this is clearly out of scope of this discussion as Django only provides an API in this scenario - what I call the hybrid app <https://fractalideas.com/blog/making-react-and-django-play-well-together-hybrid-app-model/> (website and API on the same domain) — here django.contrib.staticfiles helps; it comes after the JavaScript bundler in the deployment pipeline (and I prefer plain django.contrib.staticfiles over django-webpack-loader, but that's another story) (NB: while the blog posts I just linked to focus on create-react-app, the concepts apply to any modern JavaScript toolchain.) Regardless, Django already provides more than we need. For example, both webpack and django.contrib.staticfiles add hashes to file names to ensure cache invalidation. So my answer to questions 3, 4 and 5 is "no, except maybe documentation". Now, let's leave this brave new world and remember the jQuery era. 2. Media makes sense Although pluggable apps are fantastic, the concept doesn't work well for templates, which is where <link rel="stylesheet"> and <script> tags are written. As a consequence, there's no good way for a pluggable app to add CSS and JS for rendering a rich field, that is, a better <input>. Pluggable apps could provide custom template tags for rich fields — it's fairly easy with inclusion_tag <https://docs.djangoproject.com/en/3.0/howto/custom-template-tags/#django.template.Library.inclusion_tag> — but that would require writing a specific template for each form. That wouldn't support rendering all fields in a loop like the admin does. This is where Form Media fills the gap. Form Media <https://github.com/django/django/blob/ccb1cfb64e919e163c51995ed99bff3c92d7d006/django/forms/widgets.py#L46> is a data structure representing a list of <link rel="stylesheet"> tags and a list of <script> tags, nothing more, nothing less. It knows how to render itself to HTML. When merging two instances (or extending one), it deduplicates tags while maintaining their order. Form Media is the only place where Django renders <link> or <script> tags in Python. Everywhere else, these tags are written in Django templates. This is consistent with my observation that Django templates don't provide a convenient solution for pluggable rich fields. For websites that haven't adopted a site-wide asset pipeline, this is useful, self-contained, well-defined functionality. It doesn't have to fit into a bigger design. 3. Areas for improvement The name is wrong — it should be Static — which is why I originally filed the ticket. This is a backwards-incompatible change. I haven't thought about how a deprecation path could work. The docs describe a fallback to MEDIA_URL which, as far as I can tell, no longer exists. This paragraph should be removed. The API would be simpler if `css` was a list rather than a dict of lists keyed by media. Media-specific styles could be written inside a media query. Accepting lists in addition to the more verbose, dict-based format would be convenient and backwards-compatible. 4. Could Django do more? I don't believe Django should have the ability to compile and compress assets. It's a messy problem; we don't have the resources to tackle it. I'd rather leave it to third-party apps like we currently do. To me, the key question is — is there something useful that Django can do beyond what it currently does, short of providing an asset pipeline? The discussion often revolves about listing which assets to include in a page: - Media is basically a list of assets - Claude's work was about letting apps describe a list of assets - django-compressor extracts lists of assets by parsing HTML inside {% compress %} tags, if memory serves - django-pipeline asks the user to include a list of assets in a setting Perhaps Django could standardize a way to accumulate a list of CSS and JS assets to include in a page, which could then be rendered in HTML, perhaps after optimizations (provided by third party apps). Essentially this would solve the same problem as django-sekizai and could provide a better integration point for django-compressor. To move forwards, we'd need a better description of the problem to be solved and of the expected benefits of the proposed solution. This sounds a lot like a DEP ;-) If we did that, we'd still need the Media (or Static) class on forms, except it would feed its data into this new system instead of rendering HTML. Since it's here to stay, we should proceed with the renaming. I hope this helps! -- Aymeric. > On 22 Apr 2020, at 00:09, David Smith <[email protected]> wrote: > > Hi All, > > I hope you are all well. > > I've been thinking about static assets over the past week or so following my > email on widget media. I collated the past 6 year's worth of discussions into > a single source, with relevant extracts and links to sources. I've also set > out some options and given my thoughts (FWIW). I've not yet done any detailed > planning as I think we need to agree general principles first. Sorry this is > a bit long, may need to go get a cup of tea first :-) > > I hope this is helpful to aid discussion on this topic 🤞 > > > Summary > > The below sets out this historical context for ticket #22298 > <https://code.djangoproject.com/ticket/22298> (What to do with Widget Media) > and the wider discussions this generated on how Django should manage assets. > Whilst there are some detailed points below, these are to help set context > and to aid discussion. This is a wide and deep topic and therefore I suggest > at this stage we discuss the topic with an aim to agree the overall > direction. The principle I suggest we discuss is: > > In alignment with it's "batteries included" philosophy, should Django have an > aim to improve the management of static assets? > > In addition to seeking agreement to this broad principle I'd also like to > encourage discussion on the topics which fall under this category. Here are > some questions to aid a discussion, although I fully expect the conversation > to be wider than these initial suggestions. Some items may be easier to seek > agreement on than others; however, all will take time and effort of the > community to agree the correct design and implementation. > > 1. Should Django have an improved way of managing assets within apps > (something akin to #29586) > 2. Should Django have the ability to compile and compress assets? > 3. Should Django have Webpack support? > 4. Should Django have improved support for JS frameworks? > 5. Should Django have support for NPM? > > Based on evidence below my view is that there is enough support to improve > Django's support in this area generally. Assuming this is agreed I think the > a range of options are: > Provide improve awareness of static assets with Django core to help 3rd party > packages > In addition, integrate an asset pipeline > In addition, integration with webpack, and support JS frameworks. > > My personal view is that option one is fairly cautious, there are already 3rd > party packages which enable this functionality, and that the third one is too > ambitious. I suspect most people using JS frameworks are doing it anyway > irrespective of what is part of Django code. I think there the most value can > be added is for sites with mostly HTML / CSS and, but would like some help > compiling CSS and "sprinkling" a bit of JS here and there. > > I'd therefore suggest we start by revisiting work previously done by Claude > to understand if this is the right foundation. Once this is complete this > will enable phase two to integrate an asset pipeline / compressor into > Django. I would suggest that this is via a separate package to start with > under the Django umbrella. I suspect the pace of iteration will be too fast > for Django core, and we would like it to be more stable before merging it. > Would it be possible to use some of the existing work that the wider > community has already developed and explore building on (or 'just' use) an > existing 3rd party package? (I don't know how acceptable this is, or not) > > My personal recommendation is also that ticket #22298 is closed or marked as > someday / maybe. Widget Media may be useful in the future, we don't know yet. > I would not support the deprecation of this functionality without something > better to replace it. I also feel that a rename would have little benefit but > would cause disruption for those using it and numbers may be high given it's > been in place for many years. If agreement can be sought on this then it will > help other Media related tickets to be progressed (example : #21987). > > Below is an overview of various comments, on tickets, mailing list and pull > requests over the past 6 years. > > The ticket in question was on Widget Media, So what does Widget Media do? > > It is used to load CSS or JS to widgets / forms (e.g. add <script> tags). > This is useful when a certain widget can be enhanced by some extra JS. A > calendar would be a classic example of this. By using the media class the JS > for the calendar is only loaded on pages where the widget is present. > > How long has been around? > > Media classes have been in Django for a long time. They are included from at > least version 1.0. > > And why may we want to rename to static, what's the history here? > > In [Django 1.3] `django.contrib.staticfiles` was added "to handle static > files in a generic way". > > In [Django > 1.4](https://docs.djangoproject.com/en/3.0/releases/1.4/#django-contrib-admin > <https://docs.djangoproject.com/en/3.0/releases/1.4/#django-contrib-admin>) > "Django's admin's static files also followed this convention by removing > `ADMIN_MEDIA_PREFIX` and replaced it with `STATIC_URL`". > > So whilst most of the project moved away from using the phase 'media'; > Widgets continues to use this phrase. This is inconsistent and _could_ be > fixed. > > Now you've mentioned it, do we even need it? > > Whilst the ticket originally suggested a name chance should be considered the > discussion on the ticket is much deeper. This is where the complexity of this > ticket grows significantly with and a wide range of options is available. > Some of the options mentioned are: > > - Let's deprecate Widget Media, it's not best practice > - Tell people to use django-compressor / pipeline > - Should these tools be integrated? > - Should there be better support for front end JS frameworks > - Should we have support for webpack > - We should support NPM > > All of these items I think come under the question of "How should Django work > with assets?" > > Django ecosystem > > Currently Django has a number of 3rd party packages which help to manage > assets. A couple of options which have been mentioned in previous discussions > on this topic are django-compressor (>2k github stars, >250k downloads > p/month) and django-pipeline (>1k GitHub stars, >100k downloads p/month). > These both augment the standard Django functionality to "Compresses linked > and inline JavaScript or CSS into a single cached file." > > In addition there is also Django webpack loader (2k GitHub Stars, >300k > downloads p/month) "Use webpack to generate your static bundles without > django's staticfiles or opaque wrappers." > > > Some of these packages were also mentioned in the discussion on the django > forum under the "top 5 packages" thread [4], but they did not feature time > and time again. So whilst they have reasonable use, they don't seem to > feature at the top of a good proportion of people's lists. > > What do other frameworks do? > > Previous comments have been that rails includes the following from Carlton [1] > > > The static files story is a little different. It seems to me we don't tell > the best story there. > Rails has two things which we could be envious of, even if we didn't want to > copy exactly: > * The frontend framework integration that's already been mentioned. > * The very easy "Ajax your form", with controllers (i.e. for us "generic > views") automatically handling ajax form submissions. > Both these features get users further quicker in these aspects than we are > able to offer. > > For a more in depth overview of rails, this link was previously provided to > the mailing list [5]. Further a technical write up was provided [1a] > > Previous work > > Whilst I'm only seeking discussion on principles at this stage (should Django > have these features). I think it's useful to draw reference to previous > discussions and attempts to make progress in this area, as I feel it is > useful context. > > It came up again as another GSOC. However, it was felt that the proposal > itself wasn't feasible and that it was to big and complex to achieve through > GSOC. [3] > > It also got a mention on a separate GSOC proposal again in 2018, not > particularly aligned to this topic but generated some relevant topics. . This > one was a bit more complex, but did gain some comments on what may be useful > for Django. Tom's post from this thread is particularly helpful [6]. > > "I'm perhaps a bit biased but I would be very interested in anything that can > make JavaScript a real first class citizen in Django" > > Claude has also attempted to add a base structure to Django. Ticket was > closed as it was felt that developing this within Django itself was the wrong > approach. Florian: > > I am just a little bit worried about adding this without any concrete plan on > how 3rd party apps are going to use it. What speaks against trying this > outside of core like channels? (I'll happily put it under the django > umbrella, but core seems a little bit fast tracked to me). > > (https://code.djangoproject.com/ticket/29586 > <https://code.djangoproject.com/ticket/29586>) > (https://github.com/django/django/pull/10218 > <https://github.com/django/django/pull/10218>) > (https://groups.google.com/forum/#!topic/django-developers/KYmNnvwXDUI > <https://groups.google.com/forum/#!topic/django-developers/KYmNnvwXDUI>) > > > So does any one use Widget Media it anyway? > > Opinions in on the ticket are mixed, some comments of having never used it vs > some which think it is quite common. > > Examples : > > A ticket where a site used it extensively enough to encounter performance > issues [#30563](https://code.djangoproject.com/ticket/30563 > <https://code.djangoproject.com/ticket/30563>). The comment suggested 194,016 > media objects were being loaded but only 13 unique items. > > Wagtail makes use of it. [Wagtail > sourcecode](https://github.com/wagtail/wagtail/search?l=Python&p=1&q=.media > <https://github.com/wagtail/wagtail/search?l=Python&p=1&q=.media>) > > > References > [1] https://groups.google.com/d/msg/django-developers/KVAZkRCq9KU/AwnM5UbCGQAJ > [1a] https://blog.yourlabs.org/posts/2019-02-26-django-js-research-report/ > <https://blog.yourlabs.org/posts/2019-02-26-django-js-research-report/> > [3] > https://groups.google.com/forum/#!searchin/django-developers/media$20class%7Csort:date/django-developers/XF7F__mETrM/ECa4JpF-wRgJ > > <https://groups.google.com/forum/#!searchin/django-developers/media%2420class%7Csort:date/django-developers/XF7F__mETrM/ECa4JpF-wRgJ> > [4]https://forum.djangoproject.com/t/top-5-3rd-party-packages/391 > <https://forum.djangoproject.com/t/top-5-3rd-party-packages/391> > [5]https://medium.com/@hpux/rails-5-1-loves-javascript-a1d84d5318b > <https://medium.com/@hpux/rails-5-1-loves-javascript-a1d84d5318b> > [6] https://groups.google.com/d/msg/django-developers/ldhY8sAQc-0/h2RngIeZCQAJ > > -- > You received this message because you are subscribed to the Google Groups > "Django developers (Contributions to Django itself)" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] > <mailto:[email protected]>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-developers/bbfaffce-f6e2-4fea-b21c-34db7886d153%40googlegroups.com > > <https://groups.google.com/d/msgid/django-developers/bbfaffce-f6e2-4fea-b21c-34db7886d153%40googlegroups.com?utm_medium=email&utm_source=footer>. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/C0E3B578-DE44-42FD-870F-37A6E9828DDD%40polytechnique.org.
