A Python Approach to SEO: Automating the Boring Stuff
SEO has become so much more than the keyword-stuffing, back link buying, content-marketing-frenzy that dominated the earlier half of the 2010s. What separates a website that is winging it from a data-driven, user-experience focused hub of relevant content is the tactics that the SEO and content teams are using to stay on the cutting edge. The demand for analytically minded SEOs who can think like a developer, a UX designer, a copywriter and your everyday user since the releases of Google’s Panda, RankBrain, and Fred. With the number of hats, the modern day SEO has to wear, the time it takes to do keyword research, report on trends, and address laborious, case-by-case auditing could be used for more tactile means.
Enter Python. For those who have been missing out on one of the fastest growing programming languages in the world, Python is a general-purpose language that is used by pretty much every major organization for automation, artificial intelligence, machine learning, and much more. I have been working with Python to automate the more manual parts of my job since day 1 in my role and a few simple scripts have already saved me hundreds of hours (which I have subsequently used to write articles, read articles on Search Engine Journal, and browse sneaker forums). Below are just a few real-world cases that have been cut down through the use of Python in the day-to-day workflow.
Speeding up Speed Testing
Libraries Used: Selenium, Pandas, BeautifulSoup, Time
Site speed is a major component in SEO ranking (just see Google’s recent update on mobile site speed). When one of my clients came to us to see how their mobile site would be affected by this update, we needed to establish the baseline for their mobile site speed as it currently stood. The Problem? Their site had over 1,000 URLs, each with various discrepancies, unique plugins and other variables which could potentially harm the speed of the site. We could have manually looked through each URL…or we could plug it into Python and see if we can speed up the process. Using Selenium Webdriver to automate browser actions, we built a script that would grab URLs directly from the sitemap, plug them into Google Pagespeed Insights and collect the mobile time-to-load, along with particular notes given by Pagespeed Insights. While that was in the running in the background, our team was able to draft the POV and report simultaneously, giving us an extra day and a half to present to our client the results.
Google Trends on the Fly
Libraries Used: PyTrends, Pandas, MatPlotLib
Being able to understand trends in keywords and search subjects has always been imperative to data-driven SEO strategy. Always wanting to make sure our recommendations are backed by hard numbers and not just a hunch, our trends analyzing script calls on Google Trends to give real-time reports on interest over time, related keywords, and geographic hotspots for search subjects. All you have to do is input the keywords you want to analyze (only 10 at a time), and run it and the script will generate graphs that detail trends faster and more efficiently than relying on Google Trend’s own UI.
Web Scraping: The New Copy/Paste
Libraries Used: BeautifulSoup, Pandas, Re, urllib
Communicating SEO copy recommendations has always been a difficult subject, especially if your client does already have copies of their content in transcripts. Our client wanted copy changes made to their site but wanted to see every change that we would be making to over 50 different pages. Rather than take half the day to enter the page URL, copy the content and paste onto a Microsoft Word Doc, this python web scraper parses through the page, locate SEO contributing content factors (metadata, <p> tag content, images, title tags) and formats them onto either an Excel spreadsheet or text document in a matter of minutes. This gave my team and I time to focus on the actual content optimizations without having to spend manual hours formatting the document to hand off to our client.
In short, automating SEO isn’t always about how to quickly gain more keywords or leverage a massive warehouse of search data. Rather, utilizing languages like Python to automate the time-consuming tasks of your day-to-day opens new opportunities for you and your team to focus on bigger picture objectives. Or, if you are like me, it gives you time to write about Python while your latest test runs in the background.
For examples of all the scripts listed above, see my GitHub for early stage workings of all projects (for full examples, please reach out directly via Linkedin)
Originally published at https://www.linkedin.com on August 14, 2018.
