Why Log File Analysis Is Essential for SEO - Here's Why #136 | Perficient Digital

Why Log File Analysis Is Essential for SEO – Here’s Why #136

Web server log files can yield a wealth of valuable information for the savvy SEO.

In this episode of our popular Here’s Why digital marketing video series, Perficient Digital’s Eric Enge explains why log files are so valuable for SEO analysis.

Don’t miss a single episode of Here’s Why. Click the subscribe button below to be notified via email each time a new video is published.

Subscribe to Here’s Why

Resources

Transcript

Mark: Okay, Eric. Say, I’m a site owner.

Eric: Okay. You’re a site owner.

Mark: Say, I’m a site owner and I’ve done all the basic tasks of an SEO audit. Is there anything people commonly overlook?

Eric: Well, often in my experience, Mark, site owners neglect to check their log files on a regular basis.

Mark: Their log files?

What Is a Log File?

Eric: A log file is a record maintained on most web servers that records all the incoming activity from humans, bots, and anything else that might have access to or even attempt to access your site. What’s particularly valuable about those logs for SEO purposes is the fact that they record every visit from a search engine to your site.

Mark: And you can learn valuable things from that?

Eric: Oh, yes. For example, over time, you can spot patterns of search engine crawler behavior on your site. Is a crawler’s visiting frequency tending up or down? Each could be communicating a message about how that search engine views your site.

Mark: Well, that’s interesting because that’s something you can’t see in your analytics, right?

Eric: That’s right. Another thing you can’t do in your analytics is go back in time any further than before when you installed the analytics in the first place. So, in contrast, with log files you can go back to the beginning of the site time since you set it up, provided that your server host preserves log files. And most do. And if they don’t, you can ask them to do that for you.

Mark: Good. Well, that sounds handy for investigating problems that might have been building up over a long time. Does log file analysis have any other benefits for SEO?

Eric: Sure. For one thing, it’s not a drag on page speed like JavaScript for analytics can be. And you don’t have to worry about messing it up by applying the code incorrectly.

Mark: Okay, well those are some of the advantages of log files. Now let’s get specific about what we can discover by using them.

What Can SEOs Discover in a Log File?

Eric: There are a number of useful things you can find out by examining your log files. One is knowing when and how often crawlers access your site. Of course, that means you can also see who might be trying to scrape your site or identify possible DOS, or denial of service, malicious attacks.

Mark: Well, what about the issue of crawl budget and priority?

Eric: Ah, yes. Search engine crawlers will only crawl so much of a site on any given day. We call how much effort they will give to a particular site its crawl budget. Knowing that, you want to direct as much as possible of that crawl budget to the pages that are highest priority for you to make sure that search crawlers are getting to your most important pages first and most often. Your log files help you determine all of this. And then, there are things you could do to redirect crawl priority to where you want it.

Mark: To learn more about how to do that and about some other uses of log file analysis, see Eric’s in-depth article on the topic on our blog. We’ll give you a link in the episode notes.

Don’t miss a single episode of Here’s Why. Click the subscribe button below to be notified via email each time a new video is published.

Subscribe to Here’s Why

See all of our Here’s Why Videos | Subscribe to our YouTube Channel

Contact Us

Uncertain about which services will best address your business challenge? Let our digital marketing experts help.

CONTACT US

8 responses to “Why Log File Analysis Is Essential for SEO – Here’s Why #136”

  1. Greg says:

    I use GoDaddy for hosting, but they only show me half of my logs for the month. Is there another hosting provider I should look at that is relatively affordable so I can get the whole month?

  2. WAJID AHMED says:

    Is this file really important for SEO?

  3. Eric Enge says:

    I use Pair.com, which is more expensive, but I have access to all my log files. You can also ask GoDaddy if they can give you access to more.

  4. Eric Enge says:

    As noted in my article, I think you can learn a lot that will help you with SEO. The log files themselves don’t have any direct impact on SEO.

  5. Greg says:

    I did ask GoDaddy yesterday, but they were unable to create a Cron Job for me to get the daily log file data. They only give monthly summary files to download that oddly enough only contains half the month’s logs

  6. Bill Scully says:

    I’ve used http://www.loggly.com in the past, what log viewer do you use?

  7. Eric Enge says:

    We use our own proprietary tool here at Perficient Digital. Our web crawler crawls sites, and also looks at the log files at the same time.

  8. Good Article For Newbies Eric. Thanks For Sharing this.

Leave a Reply