Lessons with LiteSpeed and Caching – How We Messed Up and Fixed ItTravis N.
First off, I’d like to apologize if you tried to access our main website between late afternoon of May 10 and today, May 11. Your browser may have asked you to download a file called “download” if you use Chrome or a Chrome-based browser (like Edge, Brave, etc.) or a bunch of random strings like “434gefGsYsFg” if you used Firefox. Let me begin by explaining the lead up.
Moving our website – for security & performance
As a hosting provider, we have to protect our customer’s data. We handle sensitive information such as personal details as well as payment details. This means that our website and customer backend has to be secured. We don’t host our website with customer websites. It’s actually isolated away. Our old server was giving up the ghost – it was slow, clunky, and we wanted to speed up our site. So we spun up a new web server with OpenLiteSpeed, the open source version of LiteSpeed. This is an Apache-compatible web server that is designed to be fast. There’s some differences between OpenLiteSpeed and regular LiteSpeed Enterprise, but from our testing, we can use the open source version to its maximum potential.
You might be wondering why not NGINX? Or NGINX + Apache? Well, we’ve tried that before. Both scenarios. Both didn’t perform to our expectations and trying to reverse proxy Apache through NGINX was breaking a few of our site’s plugins. So we stayed with plain Apache.
When caching causes pain
After we moved our website and did testing, we flipped the public DNS, patted ourselves on the back, and called it a day.
But here’s where our problems started. At this point, had you never visited our site before, you started getting prompted to download a file called “download”.
You can see on our test machine, we were working hard to fix it.
If you opened this file up (it’s safe), you’d see a bunch of garbage, like this:
This is the output of our web page compressed. Basically, this is was like downloading a ZIP file, opening it up with a text editor, and wondering why you can’t see the contents. What happened was our web server, in this case, OpenLiteSpeed, didn’t know how to handle the “ZIP” file of our web page (literally, that’s what it is… to help make things go a little faster, the server “zips” up the web page into a format called gzip and then your browser “unzips” it. Our page loads fast because just like how you share a zip file to make some files smaller, our server is doing the same thing only your browser displays the site instead). For example, say your sending your tax forms to your CPA and you call the file taxes.zip and you send it. Your CPA’s computer has a glitch and instead of seeing all your documents, they see the garbage. That’s what happened.
All three, from our end, looked like they were working properly… because we were bypassing Cloudflare, but not our local cache. Everything was great from our end!
You really don’t need to use WP Rocket if you’re using OpenLiteSpeed – just OpenLiteSpeed by itself or OpenLiteSpeed and LSCache (the LiteSpeed Cache plugin for WordPress) will work out of the box. WP Rocket though, adds a line that OpenLiteSpeed doesn’t like to your .htaccess file.
If you can’t tell, the fix is literally changing a _ to a –
That’s it. We cleared our caches, we called Team Members in other states to have them try, and whoohoo! No more downloaded files of garbage!
Now, the clean up begins. We lost a lot of site visitors and potential customers who probably thought we were hacked.
You can see the traffic just dwindling from the moment we flipped our DNS and caches expired. It will be a long road to recovery. We’re truly sorry – but we learned a valuable lesson about bringing over third-party caching plugins on WordPress to another web server. But now you know, too, so you can avoid the same mistakes we made.