Reading Time: 5 minutes
If you’re not checking your client’s HTTP headers, you’re not giving them good service.
I’m not talking about the stuff in between the <head> and </head> tags, either. I’m talking about the server response that you get before you get all that nice HTML, or that fancy PDF, or whatever else your client’s website is slinging.
That’s because, well, your client’s website isn’t slinging anything. It’s being slung by a server, and the server’s HTTP response is the first thing a web browser – or a web robot like Google’s crawler – will see. There’s information there you may not be properly reviewing.
Implementing important SEO items in the response header is more common than you may think, and most of the major SEO tools don’t check them. HTTP headers are so critical to the SEO efforts that we built a tool to use internally called Head Exam. It checks the HTTP response for SEO factors and lets us bulk check a list of URLs so we can audit a whole site.
Head Exam is so important to our SEO efforts that we’re releasing it as a free tool.
Tool updated 4/25/18, current version 1.16b
The canonicals in the above screenshot aren’t being pulled from the web page source. Those canonicals are coming from the HTTP header response. Use this tool to check your clients’ headers right now. You might be surprised at what you’ve missed.
[toggle title=”Watch the video walkthrough” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]
Here’s a scenario – let’s pretend you’re working on a technical audit for a client, and you call out an absence of a canonical tag (which is traditionally found in the source code). You compile your findings in your SEO report and send to your client. The client responds to your recommendation and sets forth with implementing a canonical tag to live in the source code. (Keep in mind, for some clients there might be a cost involved in this implementation.) But what if the HTTP header was already properly displaying a canonical URL request? Google reads this HTTP request and may have already honored it. You could be wasting your client’s time and money, and displaying a degree of your own negligence.
Why HTTP headers are absolutely critical for every SEO
Did you know that you can declare canonical URLs in the HTTP header? Did you know that you can declare canonical tags in the response header and not have to declare them in the HTML code? If not, then I’m glad you’re reading this now before you audit a client’s website and tell them to implement canonicals, and they look at you blankly and say that they have. In the HTTP headers. That you didn’t even check. The bigger the client, the more likely they’ve done this. Like who? Well, whitehouse.gov for example. Just take a look at their HTTP header:
In this snippet of a header (pulled from our Head Exam tool), we can see that the Whitehouse is declaring canonicals, short links and even share images in the header!
This is because the Whitehouse.gov site is built with Drupal, which is used by a full 6% of the Quantcast top 10k sites. In fact, Builtwith is tracking 617,090 live top websites using Drupal right now. Drupal puts canonicals in the HTTP header by default.
OK, but is anyone really putting this stuff in their HTTP headers?
Um, yes. I plopped in a random sampling from Alexa top sites… prepare to have your socks blown off:
Not only do all of these huge websites use canonicals, short links and other SEO elements in the HTTP headers, but two of them have huge glaring mistakes:
– washingtonexaminer.com declares a canonical that’s a 404. Further, the domain newsok.com in the canonical is for The Oklahoman, not the Washinton Examiner.
– radaronline.com has a short link declared that redirects to WordPress. Not their WordPress, but straight up the WordPress.com homepage.
The fact is this: WordPress and Drupal both include a lot of important SEO information in the HTTP headers by default, and together those platforms make up something like 70% of the top sites on the internet according to Quantcast. That makes looking at HTTP headers pretty damn important for you as an SEO.
What else can be hiding in the HTTP header?
Tons of stuff. In addition to canonical tags, some of the most important things for SEO experts are:
- HREFLANG tags -tells Google which language you are using on a specific page, so the search engine can serve that result to users searching in that language.
- Shortlinks -a shorter version of a URL you can use to distribute content online.
- Share image
- Content length
- File type
Is your client using one or more of these? Have you ever checked their HTTP response headers? If and when your client calls you out on this, “I didn’t know to check there” isn’t going to cut it.
5-minute HTTP header audit checklist
Use this checklist, along with our Head Exam tool above, to do a quick 5-minute health check of your clients’ HTTP headers. Click the items to expand and see what you should check if an item applies to your site.
[toggle title=”Are there canonical tags in the HTTP header?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]
☑ Are they correct?
☑ Do they contradict any on-page canonicals?[/toggle][toggle title=”Are there hreflang tags in the HTTP header?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]
☑ Is the hreflang tag chain unbroken?
☑ Do they contradict with any on-page hreflang tags?[/toggle][toggle title=”Is there a shortlink in the HTTP header?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]
☑ Is it right?[/toggle][toggle title=”Does the HTTP header declare a share image?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]
☑ Is it formatted correctly?
☑ Does it resolve?[/toggle][toggle title=”Is a language declared?” bgcolor=”” textcolor=”” bordercolor=”” opacity=””]
☑ Is it right?
☑ Does it match the language declared in page source?
☑ Does it match hreflang tags (if applicable)?[/toggle]
How to use Head Exam for Bulk HTTP Header Checks
Using head exam is really easy. You have two tabs:
Use the Bulk SEO HTTP header check tab to check HTTP headers for a bulk list of URLs. Use the Single HTTP header examiner tab to take an in-depth look at the full HTTP headers for a URL.
Bulk SEO HTTP header check tab
On this tab, simply paste a list of URLs into Column B, starting at row 6. The tool will pull each URL, split up the HTTP headers, and parse out relevant SEO metrics from the headers into the cells to the right. You can see canonicals, shortlinks, share images, and more.
Be careful not to change any of the other cells, as they will populate automatically.
Single HTTP Header Examiner tab
On this tab, you can see the full header response for a single URL, parsed out with important stuff (like link rel tags) highlighted. Simply enter your URL in cell C4, hit enter, and wait for the magic to happen.
That’s it! Hope you start checking HTTP headers every time now. Any questions on how to better use Head Exam, tweet the @greenlane_team!