Making sure your JavaScript frameworks don't harm Search Engine Optimization (SEO)

JavaScript frameworks are becoming more popular these days. I'm not a big fan of them in general for a lot of reasons, but today I'd like to focus on how these frameworks can have a detrimental effect on Search Engine Optimization (SEO). Before I get too far into SEO, though, I'd like to talk about these frameworks and why they're used.

Why use a JavaScript framework?

My non-technical readers might be asking, what is a JavaScript framework? Essentially, the frameworks that I'm referring to in this post allow the developer to bind data from your data source onto your page in an intuitive fashion. A secondary benefit is that you can do so without needing to completely refresh the page if new data is needed. This particular page is a good example: if you were to click on any of the navigation links to your right (after you're done reading this, I hope), you can see a new blog post without needing to reload the entire page. Even better is that the framework allows the developer to do this in a relatively short amount of development time, saving time and money.

The problem in a nutshell

I just don't trust that crawlers will correctly pull in content that is loaded using one of the JavaScript frameworks. Part of my concern comes from the fact that not all experts agree that the crawlers are doing a consistently good job with this type of content. Part of my concern comes from the fact that if you make a mistake, such as putting your JavaScript files in a folder and asking your crawler not to access that folder, the crawler cannot do its job. Either way, if there is a problem, instead of seeing a page a user sees like this:

the search engine crawling your site may be seeing this:

There are two problems with the screenshot that the search engines see. The first is that the content of the blog isn't on the page. It's impossible for your content to show up in search results if they can't see it! The second problem is that the links to other blogs are also missing. Since they're also being pulled in by Angular, the search engines may not see them, and so they might as well not exist as far as the search engines are concerned.

What you can do about it

Assuming that you'd like to use the JavaScript framework, there are two things that you can do to fix the problem. The first is to make sure that your website is not dependent upon Angular (or any other JavaScript framework) to load content. Developers won't like that solution because it means that they have to do double the work (i.e. put the content on the page in server code as well as in JavaScript), but it's the only way I know of to be certain that the search engines see the content.

Using Angular as an example, if you're a developer, this means that instead of placing your code in curly brackets, you place them in an attribute, and then also bind the correct text on the server.

For example, instead of <span>{{currentblog.BlogTitle}}</span>

• In Angular 1.x use <span ng-model="currentblog.BlogTitle"> Insert server code here </span>
• In Angular 2.x, use <span [innerText]="currentblog.BlogTitle"> Insert server code here </span>

The second thing you need to do is to make sure that there is a link to all blog posts somewhere where the search engines can find them. The most search engine-friendly way I know of doing this is to make sure that your robots.txt file has a link to a sitemap file, then in your sitemap file list all of your blog posts. For example:

Robots.txt for scottnorberg.com:

user-agent: *
sitemap: http://www.scottnorberg.com/sitemap.xml

Sitemap.xml for scottnorberg.com:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.scottnorberg.com/blog/Do-programmers-need-a-degree-in-Computer-Science</loc>
</url> ...

(For more information on the sitemap.xml file, please visit http://www.bruceclay.com/seo/advanced/xmlsitemap.htm.)

Now you've not only made sure search engines can see you blog posts regardless of whether executing JavaScript is working, but you've also made your content available to people who don't have JavaScript turned on.