Initial Spike
At the beginning of February 2024, I was experiencing a spike in my Google Analytics (GA4) data for a couple of my blog posts. It was a little shocking to me because the article that I wrote wouldn’t have warranted such traffic. I’m no habitual writer that gets that much traffic. I started to Google “Can analytical spikes harm your SEO,” and “Can sudden spam spikes cause lower ranking on Google?” These were valid questions as a fledging web development owner. Eventually, after some research and a couple of virtual meetups, I learned about referral spam or referrer spam.
What is referrer spam you ask? Good question! Well, this is the definition from Wikipedia,
“The technique involves making repeated web site requests using a fake referrer URL to the site the spammer wishes to advertise. Sites that publish their access logs, including referrer statistics, will then inadvertently link back to the spammer’s site. These links will be indexed by search engines as they crawl the access logs, improving the spammer’s search engine ranking.“
This led me to believe that an external source was spamming my articles and sending the traffic over to them. You can see the spike before and after I tried to stem this issue with a possible plugin fix, or so I thought it would. More on that later!
As you can see, a jump in traffic spike from 3 to 590 users with a 2,169.2% increase is alarming. Technically it would be great if it was legit, but I knew this wasn’t the case.
So what did I do next? Well, I decided to give a Stop Referrer Spam plugin a try. As of this writing, this plugin no longer works, but it did at the time. I was able to place the anomaly URLs into a blacklist. The plugin would check these links and prevent them from hitting my site. This proved effective as the traffic died down by the beginning of March.
Time Goes On
As time went on, I didn’t think too much of the referrer spam. The plugin still ran on my site. I thought to myself, well, it is blocking the URLs on a blacklist, I shouldn’t remove it since it checking for newer links and increasing the database of bad spammy links.
Around September I noticed a major slowdown with my site. This raised alarm bells in my head again and I was starting to check all the usual suspect areas again. Is there referrer spam happening again, are my images too large on the site, am I loading unnecessary JavaScript and CSS on the site, were there any plugins that need deleting, are there new spam attacks on my site? Eventually, I pulled the trigger on something that I never thought I would have to, I blamed my hosting provider Bluehost for the slowdown. I asked for help from tech support which yielded no answers. Eventually, I gave up and left Bluehost for Rocket.net. Just on a bit of a spoiler here, it wasn’t their fault.
Moving To A New Home
With some helpful advice from the WordPress community and some long talks from virtual Meetup events, it was decided that I should move from Bluehost after 15 years. It was bittersweet really. I never had any issues with them, except for one odd issue. Like all hosting providers, they usually inject specific plugins from their service onto your side for control purposes, such as spam checks to purging cache abilities. I could never understand why the Bluehost Plugin that was installed on my site, would crash the site. My site never worked when it was activated. And so I just had it deactivated all the time. Anyways, back to the major slowdown with my site.
I made a new friend on Post Status a couple of months ago. His name was Andrew Wilder from NerdPress.net. Andrew was kind enough to check my site now and then and warned me of the extremely slow-down times he had encountered straight from the homepage alone. “I dunno man, 14 seconds to return your homepage HTML.“
Andrew and I speculated on a few key issues with my slow site, but ultimately he insisted that I give Log HTTP Requests plugin a try. I’ll be honest, I was hesitant at this point, having tried to delete plugins and remove as much as I could on the site to reduce the load time, was another plugin going to do much for me? Sometimes you want to trust your gut, but other times, you need to listen to a friend.
After I installed the plugin, I noticed some oddities with these requests that were happening every few seconds. Meanwhile, Andrew shared with me a screenshot of my site’s load time. Yay, we’re down from 14 seconds to 11 seconds. It’s progress, but there is still something fishy here.
Okay, so now what do I do? How do I mitigate this issue further? Is this still a hosting provider issue? I just moved to Rocket.net and I shouldn’t be experiencing any of these issues. Andrew suggested I reach out to Rocket’s tech support team and see what they have to say about the slowdown of my site. Rocket’s folks were awesome. Quick to respond in chat and was able to pinpoint an issue.
Okay, so what does an “autoload” mean? Is this something that I can fix on my end? Do I need to install yet another plugin, just to get rid of a pesky site issue? Turns out the answer was yes. I needed a “Developer’s eye” to figure this out. Well, I am a developer, not a seasoned one, but I know how to dive into the dark depths of the jungle and figure this one out. This just means it’s another side quest….hooray!
You Have Accepted Your Next Side Quest
Alright, now that we know the name of this issue, autoload, where do I begin? I started looking into my site health within the WordPress admin dashboard because that was suggested by Rocket’s tech support team.
At this point, the site is autoloading “284 options (size: 35MB) in the options table” This would mean I need to diagnose the database. Andrew now assumes that some artifacts might have been left behind from previous plugins. This got me wondering, how do I clean up the database? How do I clean up the autoloader data? The first article I came across was from Cloudways, they suggested that I use WP-CLI and install WP-doctor. With this technique, I should be able to view my tables and see what might be loading unnecessary issues. Fortunately, Rocket.net has a way for me to tap into the site through their web-based WP-CLI terminal. This was very nice, similar to my time using Gatsby and deploying things on Netlify.
After I installed wp-doctor, I was ready to use some of its commands. The weird part was that I couldn’t use some of them. The terminal kept flagging commands that I was writing – which I was following from the official GitHub repo – that they were “not registered commands.” At this point, I am thinking what do I need to do next? How do I get into the backend of this site?
I was a little confused at this point. How can I diagnose this issue without proper access? I reached out to Rocket’s tech support team again, and fortunately, Rocket.net allows you to SSH into your site via your own desktop terminal.
Alright, this is starting to feel like a side quest of a side quest. Such as the life of a developer. Time to get the machete out and dive deeper into this jungle.
You Have Accepted Another Side quest
It is time I crack open my desktop terminal and dive right into the issues. First things first, I needed to check if the wp-doctor commands work. Turns out they did and I was able to dive deeper into the database.
What I found as the usual suspect, yes the autoload option size was exceeding the threshold. This was considered a warning? But this was an issue that caused performance issues based on my WordPress Site Health stat. Further down the list, I noticed two errors. Looks like a cron-count and cron-duplicates were considered major issues.
I shared this bit of knowledge with some folks on Post Status and folks like Alex Standiford and Brian Henry suggested I look into the autoload tables to see what is pulling such a high threshold.
Since they had mentioned those lines of code, I looked further and found a nice article from Kinsta that highlighted specifics on how to parse out the table in a manner that made sense to me. Not that Alex and Brian’s suggestions didn’t help, I did fully understand the syntax that was given.
SELECT 'autoloaded data in KiB' as name, ROUND(SUM(LENGTH(option_value))/ 1024) as value FROM wp_options WHERE autoload='yes' UNION SELECT 'autoloaded data count', count(*) FROM wp_options WHERE autoload='yes' UNION (SELECT option_name, length(option_value) FROM wp_options WHERE autoload='yes' ORDER BY length(option_value) DESC LIMIT 10)
This yielded a huge clue to what I was tracking down in the first place. Something about the corn value spiking so high is definitely an issue here. Now I need to figure out what a cron means and why it is running up my database. Could this be the issue that is causing a slowdown?
So what is a cron? Well according to Wikipedia:
“Users who set up and maintain software environments use cron to schedule jobs (commands or shell scripts), also known as cron jobs….Cron is most suitable for scheduling repetitive tasks.”
If cron is a task, and a cron job is a scheduled task, is there something that is running within my database that I never had set it to run in the first place? Could the autoload be a result of some plugin trying to load and run a task every time I go to my website? If so, what is running?
At this point, I started to uninstall a bunch of plugins that I barely use. I deleted dormant themes that I have no use for. The odd thing I ran into was that the Stop Referrer Spam plugin wouldn’t deactivate. I went into the backend of Rocket.net and I had to deactivate it from there. However, the plugin was still showing that it was active on within my WordPress dashboard. I decided to go back into Rocket.net’s backend and delete the plugin from there. But it was STILL THERE?! If the fire alarms haven’t sounded off at this point, it has. Ultimately, I had to go through my terminal and into my plugins directory and do a complete rm -r of the directory. Okay, plugin moved. I radioed in Andrew from deep within the jungle, he then checked my site speed again. Still not great.
I decided to jump into a WordPress Meetup call to ask some further questions. See if anyone else could suggest a fix with this autoload and cron job issue. Sure enough I got some suggestions from Doug Yuen and Eric Leuthardt about how plugins do not delete their own data from the database even after you have removed it from your plugins folder. This does make me wonder why this is. Why don’t more plugins include a way to do away with their own dirty laundry. I get it though, sometimes you may need that data if you need to resinstall something, but please allow me remove whatever cruft that was left behind.
Doug later suggested that I install two specific plugins to help clean up the rampant database issues, Advance Database Cleaner and WP Crontrol. After installing these two plugins, I immediately saw some glaring warning signs. Yes, first of all the cron jobs were listed one that was called “wsrs_update_blacklist_twicedaily.” This cron job had a running event of over 200,464 times, but had no actions given. Meaning this was just sitting there, loading everytime and awaiting orders. I assume it times out when it had no where else to go.
If you hover over one of the event items in the list, it will also give you an option to “delete all hooks with this event” which basically did away with the repetitive wsrs_update_blacklist_twicedaily cron jobs. After I removed it, my lingering cron jobs went from 200,464 to 25! My only guess here is that this “blacklist” item was a leftover tsask from the Stop Referrer Spam plugin. It does checks again URLs that are harmful and it keeps track of them. However, if the URLs are no longer harming the site, it shouldn’t need to created another task for it again.
With the cron jobs cleanup, I guess there was a lot of level overs from past plugins, post articles, etc. So I gave the other plugin a once over cleaning as well.
Conclusion: Side Quests Completed
Well, we’re finally out of the jungle now. That wasn’t so hard was it? It only took about a week to diagnose the issue and pin the issue down and resolve it. Andrew checked the site again once more for me. This really tells me how much junk is left behind even after you delete something from your server. The database is so fragile.
At one point I did access the database from phpMyAdmin, but I was a little nervous about what to delete on the backend. I could have offloaded the site to my local and diagnose it that way. But imagine if this was an e-commerce site? How would you stop a site with running transactions while messing with the database? That’s some scary stuff.
After I solved this issue, I came across an articled where someone had faced the same issue! Well, if I Googled more, it would have came up. This person did do pretty much the same thing as me, but they didn’t have to download so many plugins in order to get ride of the cron jobs.
In the end though, the Stop Referrer Spam was the culprit. It generated too many redundant tasks that had no actions to perform. But the plugin was also messy at cleaning up its own work.
As a major takeaway from all this, once you delete your plugins, don’t just assume things have been cleaned up on your site. You need to dive deep into the database and do an internal cleanup as well. Start looking into the wp_options tables, go into the SQL tab, and type in the Kinsta code from above to verify what is still running on your database.