Cache Tuning

Task

Verify CDN Configuration parameters & modify cache behavior with Cache rules. The aim of this task is to serve a page (including html, which is not cached by default) from cache. We will use /services/luggages/ in this example.

Why

Cloudflare caches several different extensions by default unless your origin tells us not to (for example, with a cache-control header with a value of private). Before moving on, please take a quick look at Default Cache Behavior. In particular, look at the extensions and the cache responses.

Note that HTML is not cached by default.

Sometimes it’s desirable to override the default cache behavior to serve more from cache.

Steps

ℹ️

While the majority of examples are using curl, we do also provide Browser and Powershell examples. Please feel free to repurpose and reuse for the later steps if you prefer these tools.

1. Analyze default behavior

Using curl or Powershell let’s check the HTTP response headers (replace cfdemolab-zone-xxx with your own zone):

curl

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/luggages/"

< HTTP/2 200
< date: Thu, 26 Oct 2023 07:42:38 GMT
< content-type: text/html
< last-modified: Thu, 26 Oct 2023 03:55:52 GMT
< x-origin: origin-1
< accept-ranges: bytes
< cf-cache-status: DYNAMIC
< server: cloudflare
< cf-ray: 81c110540d6823ab-LHR

Powershell

Invoke-WebRequest -Uri "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/luggages/" -Method HEAD

StatusCode        : 200
StatusDescription : OK
Content           :
RawContent        : HTTP/1.1 200 OK
                    Date: Mon, 30 Oct 2023 09:02:08 GMT
                    Connection: keep-alive
                    X-Origin: origin-1
                    CF-Cache-Status: DYNAMIC
                    Age: 8
                    Cache-Control: public, max-age=14400
                    Accept-Ranges: bytes
                    Expect-CT: max-age=8…

The main header that we’re interested in here is cf-cache-status - note how the value is DYNAMIC - this tells Cloudflare that this resource is not considered eligible for cache, and the page is therefore pulled directly from the origin. We will change this behavior in the next task, and analyze the results again.

Note: If you do not have curl, open a new tab in your web browser and open Developer Tools (typically this can be accessed by pressing F12).

Browse to your /services/luggages/ page - then in the Developer tools window under Network select the page (1) and then you can see the headers (2) - make sure you disable cache using the Disable cache checkbox.

2. Create a cache rule for `/services/luggages`

We will now modify the default cache behavior to serve the html content of /services/luggages/ from cache.

In the Cloudflare Dashboard go to Caching ‣ Cache Rules and select Create rule

Enter a descriptive name for the rule, such as “Cache HTML on Luggages Page”
Set the Field to URI Path
Set the Operator to equals
Set the Value to /services/luggages/
Set Cache status to Eligible for cache
You can customize additional cache settings such as:
- Edge TTL: The time that Cloudflare caches the resource on the Cloudflare network.
- Browser TTL: The time that a visitor’s browser caches the resource.
- Cache Key: Customize what is included in the cache key.
- Serve Stale Content: Serve stale content while revalidating in the background.
- Respect Strong ETags: Respect strong ETags for cache revalidation.
- Origin Error Page Pass-thru: Pass through error pages from the origin.
To the right of where you see Edge TTL, Click + Add Setting
Choose Ignore cache-control header and use this TTL and set the duration to 30 seconds

You configuration should look like this:

Scroll down, leaving the other values as default (but feel free to explore them!) and click Deploy

This will create a cache rule that makes all content on the /services/luggages/ page eligible for cache, and will ignore origin cache control headers. The content will be cached for 30 seconds on the Cloudflare edge.

3. Test cache rule

Using curl (or via the browser or Powershell methods shared previously) let’s check the HTTP response headers:

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/luggages/"

< HTTP/2 200
< date: Thu, 26 Oct 2023 11:02:13 GMT
< content-type: text/html
< last-modified: Thu, 26 Oct 2023 03:55:52 GMT
< x-origin: origin-2
< cf-cache-status: MISS
< expires: Thu, 26 Oct 2023 15:02:13 GMT
< cache-control: public, max-age=14400
< accept-ranges: bytes
< server: cloudflare
< cf-ray: 81c234aabcf976c3-LHR
< cf-team: 1c0d3f3eaa000076c304734400000001

Note how you now see cf-cache-status with a value of MISS

MISS tells us that the resource is eligible for cache, but was not in cache and so came from the origin. Run the command immediately again:

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/luggages/"

< HTTP/2 200
< date: Thu, 26 Oct 2023 11:02:32 GMT
< content-type: text/html
< last-modified: Thu, 26 Oct 2023 03:55:52 GMT
< x-origin: origin-2
< cf-cache-status: HIT
< age: 19
< expires: Thu, 26 Oct 2023 15:02:32 GMT
< cache-control: public, max-age=14400
< accept-ranges: bytes
< server: cloudflare
< cf-ray: 81c2352449fe76c3-LHR
< cf-team: 1c0d3f8a9f000076c30495b400000001

Success! We have a cf-cache-status of HIT - the page was served from cache.

Note the age header - we see it’s been in cache for 19 seconds in the below example. Make sure you have waited at least 30 seconds, and then try again:

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/luggages/"

< HTTP/2 200
< date: Thu, 26 Oct 2023 11:06:01 GMT
< content-type: text/html
< last-modified: Thu, 26 Oct 2023 03:55:52 GMT
< x-origin: origin-2
< cf-cache-status: REVALIDATED
< expires: Thu, 26 Oct 2023 15:06:01 GMT
< cache-control: public, max-age=14400
< accept-ranges: bytes
< server: cloudflare
< cf-ray: 81c23a3fed9076c3-LHR
< cf-team: 1c0d42bbe9000076c306090400000001

The content has now been REVALIDATED.

If you try again, you’ll see another hit right away.

Try browsing to other pages, you will note that the cf-cache-status is still DYNAMIC elsewhere.

4. Expanding the Scope

When used appropriately, caching content has substantial performance benefits and offloads tasks from your origin servers. In the last example we created a rule to cache all content on /services/luggages/ - now let’s expand this to cover everything under /services/.

Modify the rule you already created, updating the match criteria as follows:

Set the Field to URI Path
Set the Operator to contains
Set the Value to /services/
Leave the other values as they already are and click Deploy

Test by browsing to another page, eg /services/arrows/ - run curl at least twice; you will see a cache HIT.

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/arrows/"

< HTTP/2 200
< date: Thu, 26 Oct 2023 11:17:12 GMT
< content-type: text/html
< last-modified: Thu, 26 Oct 2023 03:55:53 GMT
< x-origin: origin-2
< cf-cache-status: HIT
< age: 3
< expires: Thu, 26 Oct 2023 15:17:12 GMT
< cache-control: public, max-age=14400
< accept-ranges: bytes
< server: cloudflare
< cf-ray: 81c24a9e3fe776c3-LHR
< cf-team: 1c0d4cf6e1000076c30ab6c400000001

So far we have been focussing on pages that exist and give an HTTP 200 response. But what happens if we try to navigate to a page that matches a cache rule, but does not exist?

Attempt to reach any made up page within /services/, for example:

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/doesnotexist/"

< HTTP/2 404
< date: Thu, 26 Oct 2023 11:21:17 GMT
< content-type: text/html
< cf-cache-status: MISS
< expires: Thu, 26 Oct 2023 15:21:17 GMT
< cache-control: public, max-age=14400
< server: cloudflare
< cf-ray: 81c25097ca5a76c3-LHR
< cf-team: 1c0d50b2d7000076c30c6ed400000001

We have an HTTP 404 (File Not Found) response and a cf-cache-status of MISS

Re-run the command:

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/doesnotexist/"

< HTTP/2 404
< date: Thu, 26 Oct 2023 11:22:30 GMT
< content-type: text/html
< cf-cache-status: HIT
< age: 73
< expires: Thu, 26 Oct 2023 15:22:30 GMT
< cache-control: public, max-age=14400
< server: cloudflare
< cf-ray: 81c25261ae4876c3-LHR
< cf-team: 1c0d51d0fd000076c30cee8400000001

Study the output above for a moment, and replicate yourself, what do you notice that is strange?

We have a cache HIT despite being a 404
The age header is returning a value higher than our TTL
…why?

Cache by status code documentation shows us that the 404 error code has a default TTL of 3 minutes (or 180 seconds)

To fix this, we just need to modify our rule to override the default behaviour for a 404 response code.

Before we continue, let's purge the cache, otherwise you may still keep receiving hits due to the previous tests. Navigate to Caching ‣ Configuration ‣ Custom Purge

You are welcome to use any method you like. If you are unsure what these are then take a look at the Cache Purge Dev Docs

Head back to Cache Rules and modify your rule to set a Status code TTL:

Within the Edge TTL section that we already configured, select Add status code setting
Set the Scope to Single Code
Set the Status code to 404
Set Duration to 10 seconds

Make no other changes and deploy your rule again.

Rerun the test:

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/doesnotexist/"

< HTTP/2 404
< date: Thu, 26 Oct 2023 11:45:05 GMT
< content-type: text/html
< cf-cache-status: HIT
< age: 9
< expires: Thu, 26 Oct 2023 15:45:05 GMT
< cache-control: public, max-age=14400
< server: cloudflare
< cf-ray: 81c2737a7d6876c3-LHR
< cf-team: 1c0d668088000076c315b30400000001

In this example I happened to catch the age at 9 seconds. When testing this, try running the command a few times. You will see that you get a status of EXPIRED after 10 seconds, and then HIT again:

curl -I "https://cfdemolab-zone-xxx.cfdemolab.xyz/services/doesnotexist/"

< HTTP/2 404
< date: Thu, 26 Oct 2023 11:46:20 GMT
< content-type: text/html
< cf-cache-status: EXPIRED
< expires: Thu, 26 Oct 2023 15:46:20 GMT
< cache-control: public, max-age=14400
< server: cloudflare
< cf-ray: 81c2754cebc776c3-LHR
< cf-team: 1c0d67a404000076c3161cc400000001

<rerun immediately>

< HTTP/2 404
< date: Thu, 26 Oct 2023 11:46:21 GMT
< content-type: text/html
< cf-cache-status: HIT
< age: 1
< expires: Thu, 26 Oct 2023 15:46:21 GMT
< cache-control: public, max-age=14400
< server: cloudflare
< cf-ray: 81c27555ecfb76c3-LHR
< cf-team: 1c0d67a9ae000076c316293400000001

Understanding how cache varies by status code is important, and the scenario of cached 404 pages is not uncommon!

Summary

In this section we’ve explored how to tune and optimize Cloudflare cache

Next, we will take a look at Load Balancing.

Part I

Part II

Cache Tuning

Task

Why

Steps

1. Analyze default behavior

2. Create a cache rule for `/services/luggages`

3. Test cache rule

4. Expanding the Scope

Summary

Part I

Part II

Cache Tuning

Task

Why

Steps

1. Analyze default behavior

2. Create a cache rule for /services/luggages

3. Test cache rule

4. Expanding the Scope

Summary

2. Create a cache rule for `/services/luggages`