Nginx 410 maps and Custom Error Page

In order to bulk retire multiple URLs of your web site at once, you can use nginx maps to have the webserver reply with a 410 http code, which means: GONE and not coming back.

This can be common when migrating away from a specific CMS like WordPress that builds a bunch of taxonomy pages that you will no longer be supporting or similar.

Also, this method will work for a any sort of matching you can do, but we'll focus on 410 and a custom error page, since this is what I had to implement recently.

Lastly we will also make nginx load a custom 410 error page, instead of the boring default.

The full code can be found in the repository: https://github.com/JonathanMH/nginx-410-map

Why respond with 410?

Technically, it's the correct signal to send to both people and machines that you removed a page deliberately and that it's not missing (or not found) by accident. The most frequent reason to actually make sure a 410 error code is sent because search engine providers and SEO tools will alert you to sudden spikes of 404 errors on your site. To avoid that, we want to make it clear, that we intentionally removed the pages.

Nginx 410 maps in action

We can retire multiple different pages at once and even use wildcards to match URLs, like for example no longer supporting any /tags or /categories on our domain, we can use wildcards to match a whole set of URLs, no matter how many we originally submitted to a search engine.

Let's start with some entries that are a simple file and a directory:

map $request_uri $is_retired_url {
  /gone.html 1;
  /gone-dir/ 1;
}

server {
  listen 5000 default_server;
  port_in_redirect off;

  if ($is_retired_url) {
    return 410;
  }

  location / {
    root /var/www/html;
    try_files $uri $uri/ =404;
  }
}

Now if we were to access http://localhost:5000/gone.html or http://localhost:5000/gone-dir/ we would get a GONE 410 response!

However, trailing slashes make this a little more complicated. For URLs that work with and without trailing slashes, we would have to add two entries to the map:

  /gone-dir/ 1;
  /gone-dir 1;

which is not cool, so we can fortunately make use of wildcards and pattern matching (YEAH!):

  ~^/gone-dir/? 1;

which means:

• ~ this is a regex
• ^ this is the beginning of the $request_uri
• /gone-dir/ this is the $request_uri
• ? the previous character is optional

Great, now we have one line for the with and without trailing slash variant.

Moving Redirects to External File

Now all we have to do is to create a file, I'd create a directory called redirects next to the nginx.conf and copy the body of our map into it:

/gone.html 1;
~^/gone-dir/? 1;

and then we change our map in the default.conf from

map $request_uri $is_retired_url {
  /gone.html 1;
  ~^/gone-dir/? 1;
}

to

map $request_uri $is_retired_url {
  include ./conf.d/redirects/410.redirects;
}

and we will from now on load the external file and not bloat our config file with 40003 lines of redirects or no longer available pages (yay!).

In my case I'm adding WordPress specific routes that I want to send 410 codes for like:

~^/category/* 1;
~^/tag/* 1;

You might get errors like

Starting nginx: nginx: [emerg] could not build map_hash, you should increase map_hash_bucket_size: 64

at the default map size, you can edit the nginx.conf and increase the values in the http block:

http {
  map_hash_max_size 8000;
  map_hash_bucket_size 8000;
  # [...]
}

Nginx Custom 410 Pages

If you find the 410 GONE default error page a bit boring, we can obviously do something about that!

In our public directory, I will create another directory called errors and inside that a file called 410.html.

Next we can add these lines to the default.conf inside the server block

  error_page 410 @gone;

  location @gone {
    root /var/www/html;
    rewrite ^(.*)$ /errors/410.html break;
  }

It's important that we don't just specify a path, because we need to tell nginx the location of our error page, which lives in the dame directory as all our other HTML content, in /var/www/html.

Let's create a slightly more fun 410 page:

	<h1>What ever you're looking for...</h1>
	<p>it's no longer there.</p>
	<img src="https://media3.giphy.com/media/jUwpNzg9IcyrK/giphy.gif?cid=ecf05e47n01tblzjq8jvn00drz7ro3te9svz0470cgsqln0y&rid=giphy.gif&ct=g" alt="Homer disappearing into a hedge">

Now you can greet your stranded visitors with a GIF from a popular cartoon and some mysterious text, which is probably better. In reality you should probably provide navigational links, a contact form or a search field for them to find anything related to what they actually came to your page for.

Full Example

If you are interested in the full example, you can find it on github: https://github.com/JonathanMH/nginx-410-map and if you have docker installed, you can play around with the full example using

docker-compose up --build

remember to CTRL+C and run it again after changes.

Let me know if this post was useful and what you're doing with nginx. I love hearing what sort of projects people work on!