A few months ago I started learning the Go programming language. As part of the learning process I decided to build a SEO tool so I could practice on a real life project, this project had a limited scope in time and features as any real life project would have. Along the way I learned a few things about SEO which I'm sharing here, this is all based on this project's limited experience and the opinions I formed from talking to other people working or interested in SEO.
- There are tons of SEO tools and most of them are quite expensive, which makes sense when you think of them as professional tools. If you are a hobbist or a small shop who can't pay the price you're going to have a hard time competing with the professionals who can.
- Most people I spoke with seem to agree that SEO tools are way too bloated, can be confusing and hard to use. We can see it this way, the more features we want in our tools the more complex they become. If your tool does tech SEO, keyword research, rank tracking and who knows what else, all that complexity is going to show up somewhere, it's going to have a learning curve.
- Charts and data visualization is important in SEO tools. Not all tools provide charts and they are more important than I first thought, charts help better understand and analyse the data reported by the tool, but agencies also use them in presentations for clients to show the current status of the project and the progress made over time.
- There's a lot of bullshit. During the last couple of months I've come across quite a few SEO "experts" who clearly don't even understand the basics of it. If you're going to hire someone to help you out with your site, make sure to do your homework and don't hire the first guy who claims to be an expert. SEO is expensive and takes time, some people will be happy to exploit that to rip you off.
My SEO tool was focused on tech SEO only, so this are some of the technical insights and learnings I got after analysing around one million pages:
- The most common issue I found is images with no alt text. This tag may help SEO, but most important, it is used by screen readers such as the ones used by visually impaired people, it makes the internet more accesible. Second most found issue was pages with no H1 title.
- Pages with incorrect H heading order are also common, some sites might be using H3 tags with no H1 and H2, or they are using H2 before the H1 tag.
- Soft 404 errors are a real issue. I've found that many websites are returning "Page not found" errors with a 200 status code. This one can be hard to detect because the website seems to work as expected, but underneath it is sending the wrong status code.
- Canonical tags are mostly ok. To be honest, I expected to see a lot of issues with the canonical tags but it turns out they where ok most of the time. The same goes for the hreflang tag, which is a tricky one, but it was mostly ok.
- Some automatically generated sitemap.xml files seem to include pages that should not be indexed, such as the ones with the noindex tag, canonicalized pages, even 301 and 404 pages.
- The robots.txt can be 301 redirected. When I first saw this one I though it was an error, but it turns out the robots file can be redirected as any other page, search engines will follow that redirect.
Overall it's been a fun learning exercise and I decided to keep exploring the possibilities of the tool I built. A few people also showed interest at some point, asking for collaboration or wanting to help out, so for those interested I open sourced the code and made it available for free under a very liberal license. Being a web based tool requires a bit of set up and tech knowledge, so I'm also keeping a free hosted version for smaller sites.
I'm not adding the links to the source code or the hosted version as I don't want this to be considered spam posting. Just reach me out if you're interested or just want to know the more geeky details of the project.