Project

General

Profile

Feature #16091

Rethink our caching of static (CSS, JavaScript, more?) files

Added by sajolida about 1 year ago. Updated 7 months ago.

Status:
In Progress
Priority:
Normal
Assignee:
Category:
Infrastructure
Target version:
-
Start date:
11/02/2018
Due date:
% Done:

10%

Feature Branch:
Type of work:
Website
Blueprint:
Starter:
Affected tool:

Description

I don't know much about caching but in the past months we've had to fix 2 issues related to the long caching of CSS files. One is #16049 and the other one 4e30a3e59b. As we'll work on making our website look more fancy (and less like ikiwiki.info), I expect these issues to become more frequent.

While working on self-hosting our website, should we reconsider the caching policy for CSS to avoid or minimize such issues?


Related issues

Related to Tails - Feature #12408: Ensure our website is ready for temporary surge of new users Confirmed 03/29/2017
Related to Tails - Feature #16128: Move the verification JavaScript from the verification extension to the page itself Confirmed 11/15/2018
Blocked by Tails - Feature #14588: Self-host our website Resolved 10/03/2018

History

#1 Updated by intrigeri about 1 year ago

  • Parent task deleted (#14588)

#2 Updated by intrigeri about 1 year ago

#3 Updated by intrigeri about 1 year ago

  • Assignee changed from intrigeri to sajolida
  • QA Check set to Info Needed

While working on self-hosting our website, should we reconsider the caching policy for CSS to avoid or minimize such issues?

I'm totally open to do that if a better way, that I believe is the industry standard way to handle such issues, is not practical for us for some reason. That other/better way is to have a unique query parameter per revision of the cached resource, i.e. what u implemented. I don't know if doing things that way consistently for long-term cacheable resources is doable, hence "Info Needed".

FTR I'm reluctant about disabling caching because:

  • I'd like us to be ready for slashdotting or #12408: caching mostly static resources (JS, CSS, pictures) substantially cuts the number of requests our web server has to answer.
  • The only way to fully fix the problem you're mentioning purely via web server config is to fully disable caching, which is not exactly best practice on a popular website (e.g. it affects Google ranking negatively). But we could find a trade-off between resource needs, website performance, and temporary poor rendering in browsers by setting a not too long cache expiration delay.

Regarding Redmine semantics, I've turned this ticket into "blocked by #14588" (as many other things), because I see no reason why improving things should formally become a new blocker for a migration I committed to do with a fixed budget; hoping you don't mind. This being said, I care about this topic and I've already added a note about it in the description of #14588 when we had this problem a couple weeks ago (https://labs.riseup.net/code/journals/93698/diff?detail_id=108353): I'll have to port the caching settings from Apache to nginx anyway so I'll at least quickly check if there's a cheap way to improve things :)

#4 Updated by intrigeri about 1 year ago

  • Related to Feature #12408: Ensure our website is ready for temporary surge of new users added

#5 Updated by u about 1 year ago

If we don't want to disable caching, we should use a technique similar to the one used in 4e30a3e59b7c97cf90b5a87bc6df1ece568f4335 whenever we make important changes. I think that would be reasonable to update the "timestamp" if needed - but I agree it can be forgotten..

#6 Updated by sajolida about 1 year ago

  • Assignee changed from sajolida to intrigeri
  • QA Check deleted (Info Needed)

I created this ticket as a way to have our sysadmins reflect on how we can prevent such problems in the past. I don't know anything about caching, what the industry standard is and I don't want to learn about this because it's way out of my skillset and attributions in the project. If you think it's a bad idea please reject this ticket (or adjust the Redmine metadata as you please) but please don't count on me to take a technical decision on this.

#7 Updated by intrigeri about 1 year ago

I created this ticket as a way to have our sysadmins reflect on how we can prevent such problems in the past. I don't know anything about caching, what the industry standard is and I don't want to learn about this because it's way out of my skillset and attributions in the project. If you think it's a bad idea please reject this ticket (or adjust the Redmine metadata as you please) but please don't count on me to take a technical decision on this.

Fair enough.

The thing is, there's simply no single person/team in the project who is obviously responsible for this problem, because the typical solution requires collaboration between web developers (who know about this sort of things), sysadmins (who can bring complementary knowledge and operational requirements), and tech writers (who can bring another set of requirements because their work will be affected by the decision).

So:

  • Sysadmins may not be able to fix that alone, at least not without creating other kinds of problems. Still, I'll see what I can do :)
  • I'm sorry I am still operating under the flawed assumption that "tech writer" implies "responsible for all aspects of the HTML we generate from wiki/src", i.e. our tech writers are also web developers. I should really stop implicitly assuming that: it's unrealistic, unfair, and has caused all kinds of problems in the past, such as the kind of Redmine ping pong we're playing here :/
  • We should think about how to fix the "nobody feels skilled and responsible for the web development aspect of our website" problem, at the very least because even once I've fixed my flawed assumptions and stop reassigning you things that are indeed outside of your skillset & attributions, in order to address this kind of tickets I'll still need someone who's ready to do the web site part of the work. Perhaps it boils down to broadening the scope of our quest for a solution to "lack of JS developers". And perhaps we'll need to add this to the core budget somehow. Food for thought!

#8 Updated by intrigeri about 1 year ago

  • Status changed from New to Confirmed

#9 Updated by sajolida about 1 year ago

  • I'm sorry I am still operating under the flawed assumption that "tech writer" implies "responsible for all aspects of the HTML we generate from wiki/src", i.e. our tech writers are also web developers. I should really stop implicitly assuming that: it's unrealistic, unfair, and has caused all kinds of problems in the past, such as the kind of Redmine ping pong we're playing here :/

Right, technical writers are responsible for the documentation only.
Our description of "UX designer" also overlaps a bit with web dev as
they "do UX design (graphical interface, interactions, language, etc.)
on the maintenance or important improvements [...] to the core pages of
our website".

But we have nobody responsible for the good working of our website in
general. Though I sometime do it myself or even bend a bit my assigned
roles (eg. #14922 in UX as per fundraising.git:final/ISC/1/unofficial.mdwn).

Perhaps it boils down to broadening the scope of our quest for a solution to "lack of JS developers".
And perhaps we'll need to add this to the core budget somehow. Food
for thought!

Yeap. That's why I mentioned the need for "Front-end web development",
at the summit. See summit-2018.git:core_days/hiring_strategy.mdwn.

#10 Updated by u about 1 year ago

Yeap. That's why I mentioned the need for "Front-end web development",
at the summit. See summit-2018.git:core_days/hiring_strategy.mdwn.

All of this lies in my skillset and I'm happy to help out with these things.

#12 Updated by u about 1 year ago

To get back to the issue of this ticket: I think it is super useful to cache CSS files and not make people download them over and over again. However, sometimes we want to force them to redownload the files because, as in the case of the donation campaign, we made a change that should be visible to users.

https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching#invalidating-and-updating-cached-responses advises to use versioned filenames such as local.20181011.css.

In order not to change filenames by hand we could set up a htaccess that redirects versioned files to the original filename see https://css-tricks.com/strategies-for-cache-busting-css.

Or, as I did in my commit, use query strings, also mentioned in https://css-tricks.com/strategies-for-cache-busting-css/#article-header-id-1. This seems like an easy solution to me, although it requires manual intervention from frontend devs/fundraisers/tech writers.

#13 Updated by intrigeri about 1 year ago

  • Status changed from Confirmed to In Progress
  • % Done changed from 0 to 10
  • Type of work changed from Sysadmin to Website

As part of #14588, I've configured the webserver that will soon host our website so it allows browsers to cache CSS, JS, fonts and images for up to 1 day (instead of 1 week on the current production website); I've ensure ETag headers are sent so browsers can revalidate their cache cheaply. This is meant as a temporary mitigation until our website supports invalidating browser cache when needed (using one of the techniques u has mentioned earlier). Once these techniques and processes are in place, we should increase the expiration delay.

So next step is to evaluate our options on the website front. Happy to work on this (with my "I kind of know some ikiwiki stuff" hat on, not sysadmin) with u at some point.

#14 Updated by u about 1 year ago

So next step is to evaluate our options on the website front. Happy to work on this (with my "I kind of know some ikiwiki stuff" hat on, not sysadmin) with u at some point.

Sure! I think I'd prefer to do that in a call rather than on this ticket, and then report the result on the ticket.

#15 Updated by geb about 1 year ago

Hi,

intrigeri wrote:

As part of #14588, I've configured the webserver that will soon host our website so it allows browsers to cache CSS, JS, fonts and images for up to 1 day (instead of 1 week on the current production website); I've ensure ETag headers are sent so browsers can revalidate their cache cheaply. This is meant as a temporary mitigation until our website supports invalidating browser cache when needed (using one of the techniques u has mentioned earlier). Once these techniques and processes are in place, we should increase the expiration delay.

If I may, I speed some of time working on HTTP Caching and, while u's trick about versionned files names (or URL arguments like 4e30a3e59b) is great (did not thought about it for #16049), its a bit constraining if not included in the framework. However, HTTP offers some nice caching mecanisms you could use to force longer expiration delay while asking or forcing revalidation :

Actually on the website i see :

Expire: current + 1 day
Cache-Control: max-age=86400
Etags

The use of etags actually let the browser revalidate for every request (its not explicit, so its browser decision, my firefox revalidate every CSS/JSS, but don't always ask for revalidation of the images) but let it use stale content for up to one day, even if the website is unavailable, and should be equivalent to:

Expire: current + 1 day
Cache-Control: max-age=86400
Cache-Control: min-fresh=0
[NoEtags]
Depending what you want to do it should be possible to :
  • Let the browser use cached content for longer time without revalidating.
  • Let the browser use cached content for longer time after revalidation.
  • Let the browser use cached content for longer time if revalidation fails.

For example :

Expire: current + 1 month
Cache-Control: max-age= 1 month
Cache-Control: max-stale= 1 mouth
Cache-Control: min-fresh=0
[NoEtags]

Will still ask the client to revalidate the content for every request, but keep the file in cache for up to one month which would decrease the server load.

Or

Expire: current + 1 month
Cache-Control: max-age= 1 month
Cache-Control: max-stale= 1 mouth
Cache-Control: min-fresh= 1 day
[NoEtags]

Will let the browser use cached content (even if revalidation fails) for up to 1 month and ask only for revalidation after one day, which would also decrease the server load (but may be not desirable).

Good reading :
https://developer.mozilla.org/en-US/docs/Web/HTTP/Caching Not fully complete (some things like max-stale etc are missing)
https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching same
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control complete (even include useless experimental features)

If you would like to play a bit with HTTP caching, feel free to ping me.

(Side note: its less KISS, but maybe images could be cached for longer than CSS)

#16 Updated by geb about 1 year ago

Another side note :

Cache theory and web users habits make it clear than even if a really small/short cache of one or two hours would have a huge impact on performances while allowing quick updates.

Thus if I had to propose a policy, I would suggest something like :
  • Use files from cache without revalidation for 1 or 2 hours
  • Use files from cache after revalidation for 1 week or 1 month
It would allow :
  • Quick (1, or 2 hours) changes without complexes revalidation mechanisms like ikiwi changes (and the possibility to rely manually on them using URL arguments if really necessay)
  • Subsequents requests for a given visit (to different pages) to be served directly from cache, reducing the server load and giving user faster access
  • Use of cached files for longer time than actually, reducing the server load and giving user faster access.

Which would give thoses HTTP headers :

Expire: current + 1 week
Cache-Control: max-age= 1 week
Cache-Control: max-stale= 1 week
Cache-Control: min-fresh= 2 hours
[NoEtags]

Or
Expire: current + 1 week
Cache-Control: max-stale= 1 week
Cache-Control: max-age= 2 hours
[NoEtags]

Which as far i remind, may have the same scemantic (need testing) and is more KISS.

As images are less likely to change, (and URL tricks are still possible), another policy using 1-2 days before revalidation and 1 week or 1 month for after revalidation sounds also raisonnable.

#17 Updated by sajolida about 1 year ago

Thus if I had to propose a policy, I would suggest something like :
  • Use files from cache without revalidation for 1 or 2 hours
  • Use files from cache after revalidation for 1 week or 1 month

Thanks geb for joining the discussion with such detailed information!

#18 Updated by intrigeri 7 months ago

  • Subject changed from Rethink our caching of CSS files to Rethink our caching of static (CSS, JavaScript, more?) files

#19 Updated by u 7 months ago

  • Related to Feature #16128: Move the verification JavaScript from the verification extension to the page itself added

Also available in: Atom PDF