The Internet Home Business Magazine for Moms & Dads

5 WordPress Hosting Lessons :: How to Survive the Digg Effect

 

Last June, I lived an internet entrepreneur’s nightmare as my site was up and down (mostly down) for a week due to one of my posts getting onto the front page of Digg.

This wasn’t my first trip to Digg Stardom - nor the second. My little WordPress blog had been there twice before. Sure, the site had a bit of a hard time under the traffic load, but for the most part, it was smooth sailing with my hosting company. But in June, when my post “The Top 10 Blogging Mistakes I Made in My First Year” hit it big, my site crumbled - as in, within the first 10 minutes of hitting the front page.

And while I’m here, I’ll humbly tip my hat to the Digg commenters who had a FIELD DAY with the sheer irony of this blogging mistakes post bringing my site down from the Digg Effect:

” #11 Not having enough bandwidth…” - oxdeltaxo

” Nah… #1 is, not choosing appropriate hosting. ;)” - malcam

” The host is another mistake as well!” - LeeVal

But I think the winner goes to Konstantino

” Well, it’s not successful anymore, “suspended page”. “

Digg Traffic Effect GraphBut it just didn’t make sense. How could my hosting company have sailed through two Diggs (one recent BIG one May 23), only to crash and burn on the third - under significantly less traffic (June 8)?!

Well, I learned more about hosting a WordPress blog through this experience than I EVER WANTED TO KNOW.

Yes, this post will get a little technical. But if you don’t think it applies to you now, bookmark it for later, because if you…

  • Run a WordPress blog in any kind of a shared hosting environment, and
  • Want your blog to continue to grow

Then you will need this information at some point down the road.

WordPress Hosting Lesson #1 - Bandwidth and disk space are irrelevant

Most hosting companies offer about 300GB of disk space and 3000GB of bandwidth. Neither of them matter for a WordPress blog. You will have an extremely hard time coming up with 300GB of stuff to put on a server. And bandwidth is only relevant if you are running lots of video or audio files off of your own server. The only thing that matters under high traffic loads is the amount of memory available to your account.

My server was down for that week because I didn’t have enough server memory anymore. See Lesson #2…

WordPress Hosting Lesson #2 - Regular shared hosting is highly unpredictable

In a shared hosting environment, you will be limited to a set percentage of server resources - usually about 20%. Once you exceed 20% of the server’s resources, they will shut you down. This is because you risk taking down hundreds of other sites that share your server with you.

If everyone on your server is having a slow traffic day, you have a LOT of server resources at your disposal. If even just one other person on your server is getting a little StumbleUpon traffic or bought a good ad, you are out of luck.

My earlier Diggs happened on days in which I was able to take up 20% of the entire server resources. On June 8th, I wasn’t so lucky and I was stuck with a fraction of that amount.

WordPress Hosting Lesson #3 - The topic of your Dugg story, your blog template, the day of the week and time of day that your story hits the front page can all be the cause of a server outage

Hot political news hitting on a Friday afternoon (when people are looking for work distractions) will crash your server much faster than a recipes article hitting in the middle of the week. This is just because of the interests of the Digg audience, which skews younger and male.

The factors that influence how quickly people come to your site at the same time are really hard on a WordPress blog. This is because every element on your page is a separate call to the database - every image, sidebar widget, plugin, headline, article, etc. Too many calls to the database at any one given moment is just too much for an unprepared server to handle.

There are a couple of solutions for this. Under heavy traffic loads:

  1. Always use the WPCache plugin
  2. Use a different, lighter-weight template with fewer images and sidebar widgets
  3. Turn off unnecessary plugins

WordPress Hosting Lesson #4 - A WordPress plugin can be the cause of a server outage

Poorly written plugins can do worse damage than crashing your blog during a Digg, so you should always be careful with the plugins you choose to install. If you are under a heavy traffic load, turn off all plugins that an author wrote and released recently in case they are problematic, unless the author has a long-standing history of writing solid working plugins.

WordPress Hosting Lesson #5 - Upgrading to a Virtual Private Server (VPS) is NOT the answer

When I outgrew my shared hosting plan, I had to do a lot of fast research to determine how and where to move my site to a more stable server. I was faced with paying $40-$90 a month for a Virtual Private Server (VPS), or going all out for a dedicated server for $250+ a month. I thought the dedicated server was probably overkill, so I went from $6.95 a month to $60 a month and tried out the VPS.

It was a disaster.

In a VPS environment, you DO have more memory resources available to your account, but you also are limited at the high end of what your server is capable of. I started at 512MB and upgraded to 768MB, and with WordPress, it still wasn’t enough - it could hardly handle StumbleUpon traffic, let alone a big Digg.

I ended up having to bite the bullet and I moved to a dedicated server this last weekend. Every month I was paying my friend Steve Johnson to fix my WordPress problems on the VPS anyway, I figured I might as well just pay for a solid server that isn’t going to give me headaches anymore (and a support staff that knows their head from a hole in the ground, but that’s another story ;) ).

Although I stand by WordPress as the blogging platform of choice, realize that as you grow, your hosting will have to grow with you. I even still stand by my first shared hosting company, (Blue Host) as they were extremely helpful in explaining how all of this works, and I was quite sad to leave them.

WordPress by nature is a memory hog. My new server has 6GB of RAM, and I’m fairly confident it will stay running throughout whatever I can throw at it.

Do you need this much memory for your own blog? Probably not. But if you keep at it, someday you will.

If you need to find a good WordPress Hosting Company, I can say from experience, don’t go with GoDaddy and DON’T GO WITH YAHOO (probably the worst WordPress hosting out there, actually). Go with Blue Host or one of these companies who offer both great tech support as well as Fantastico, which makes WordPress installation and setup a breeze

Share the Love... and the Traffic ;) These icons link to social bookmarking sites where readers can share and discover new web pages.
  • Digg
  • Netscape
  • StumbleUpon
  • del.icio.us
  • Ma.gnolia
  • Reddit
  • Technorati
  • Bumpzee
  • co.mments
add to sk*rt

If you liked this article, please...

Subscribe Via Email Subscribe Via RSS Add to Technorati

Or read these related articles...

  • Timely Link Love for Bloggy Friends
  • The Digg That Brought eMom Down
  • Hosting Transfer Complete!
  • A WordPress QuickStart Guide - What Plugins Should I Use?

  • Discussion

    What do you think? Leave a comment. Alternatively, write a post on your own weblog; this blog accepts trackbacks [trackback url].
    This blog has removed the "nofollow" tags, so each comment counts as an inbound link to your site. Comment Policy

    Comments

    1.
    On August 27th, 2007 at 1:24 pm, Andy Beard said:

    I am going a slightly different route with a clustered virtual dedicated solution.

    Thus whilst I only have a similar amount of memory on the virtual server, the MySQL is on another dedicated box.

    The negative side is I have to learn how to use HSphere rather than Cpanel.

    2.
    On August 27th, 2007 at 2:01 pm, Wendy Piersall said:

    Gosh - that sounds like a really complicated setup, Andy. WAY beyond my technical expertise. ;)

    3.
    On August 27th, 2007 at 4:48 pm, Matt said:

    The problem with most shared hosts, is that they cram way too many people onto 1 server, and the more people on 1 server, the less resources your site can use. I like my host (BlueFur), there’s maybe around 100 or less people on 1 server, and they can survive Digg’s no problem.

    Using WP-Cache is debatable whether it will help or not. Whether you’re getting it straight from the database, or from a cache, you’re still getting it off the hard drive. Which one is faster, I have no idea, but, it’s probably not going to make a big difference. If you have any plugins that write to the database on each page load (like a counter), then you’ll have a problem. Database reads are cheap (in terms of server usage, not money :P ), writes are expensive.

    6GBs of memory is way overkill. You need to remember that it’s not all about RAM, you need a good CPU (processor) too. I don’t know how your traffic is against John Chow’s, but his server only has 2GBs of RAM, but it has 2 Dual-Core Xeon processors. His sites fast, too (granted, his theme is more optimised, and lighter-weight, than yours…).

    What’s the full specs on your server?

    4.
    On August 27th, 2007 at 4:53 pm, Wendy Piersall said:

    Matt - I have some upcoming projects that I needed to plan for - 6GB is definitely overkill for now, but 6 months from now, I’m hoping it’s just about right. ;)

    Also, WPCache serves up static pages, so even though it pulls from the database to create the page, once the page is created there are far less active calls to the database - got that info straight from The Lorelle herself.

    5.
    On August 27th, 2007 at 5:09 pm, Matt said:

    6GBs is a lot of RAM, and *nix is light, so I’m guessing it’ll still be overkill. ;)

    Yes, and you’re still pulling the page from the hard drive. I wonder if there’s been any studies on whether pulling a static page off the hard drive or reading from the database is quicker…?

    I don’t really believe that it’s necessary to run WP-Cache o a dedicated server anyways, but it’s all out personal opinion on the matter, I suppose.

    What CPU(s) are in your server (I’m curious :P )?

    6.
    On August 27th, 2007 at 5:13 pm, Tyler said:

    What hosting company are you using for your dedicated server?

    7.
    On August 27th, 2007 at 5:14 pm, Wendy Piersall said:

    OK, OK, you must be as geeky as I am when it comes to server stuff! ;)

    Dual Core P4 3.0GHZ
    Two SATA 80GB Hard Drives (RAID Available)
    cPanel/Webhost Manager
    Fantastico/RVskins
    600 GB bandwidth (on 100mbit port)
    8 IPs

    Did that satisfy your curiosity?! :P

    8.
    On August 27th, 2007 at 7:22 pm, JoLynn Braley said:

    Thanks Wendy, this is good info to keep in the back of my mind for later so that I can refer to it. ;)

    9.
    On August 27th, 2007 at 7:25 pm, Suzie Cheel said:

    I think I will aspire to having this Challenge. It is great that you share your learning- so valuable and helps us prepare for stardom

    10.
    On August 27th, 2007 at 8:20 pm, Elizabeth Potts Weinstein said:

    Thanks for sharing this information! I’m bringing on some major projects and did not understand why a dedicated server was different, and now I really get it. My husband (outside attorney for Verio) has been telling me for a while to do it, but I did not understand his deal.

    Hum, I may need to make the jump for a new project coming on this fall.

    One question — do your IT costs go up with a dedicated server, over shared? Right now I do almost everything myself (cPanel, mySQL, etc.) but with a dedicated server will I need to hire someone? Or, perhaps your IT costs go down b/c the hosting company has people that help, as part of the monthly fee?

    Thanks!
    Elizabeth

    11.
    On August 27th, 2007 at 8:26 pm, Wendy Piersall said:

    Elizabeth, my new hosting company still manages my server - they moved my site and set up everything for free, and will even install and support third-party software on my server (which absolutely astonishes me). They monitor and offer tech support 24/7/365 - and yes, I’m paying way more than on a shared plan - about $250 a month.

    My VPS company was SO SO SO bad that I just couldn’t take it anymore. This company also hosts EduBlogs.com - a WordPressMU site with more than 40,000 blogs, and they have had consistent up time for more than 4 years. After such a bad experience with my last company, it was worth it to me to know I don’t have to mess with this stuff again! :D

    12.
    On August 27th, 2007 at 8:29 pm, Yuri said:

    WPCache has been tested to significantly increase site loading speed.
    http://blog.forret.com/2006/09/wp-cache-speeds-up-your-wordpress/

    There were other benchmarks I can’t find at the moment.

    You’ll need to use other WP optimization stuff, such as PHP/MySQL caching, as well as updating WP and WP cache to their latest versions.

    13.
    On August 27th, 2007 at 8:37 pm, Wendy Piersall said:

    Tyler - A company called pSek - hadn’t heard of them before the founder of EduBlogs recommended them to me.

    14.
    On August 27th, 2007 at 8:46 pm, Tyler said:

    Thanks for the info Wendy!

    I haven’t heard of this company either, but considering the amount of traffic that EduBlogs handles, I will definitely have to look into them.

    As you discussed earlier, my shared server is working just fine for my current projects, but I’m working on a new project that should launch in 3-6 months, and if all goes well, I will definitely need a more robust hosting solution :)

    15.
    On August 27th, 2007 at 11:34 pm, Simonne said:

    This is very interesting and useful info. I always thought bandwidth mattered in such situations. I’m sorry you had to go through all that trouble in order to come to these conclusions. Thank you for sharing your learnings with us.

    16.
    On August 28th, 2007 at 9:04 am, Nathania - Bold Interactive said:

    Wendy, thanks for taking the time to write this post. I have been dealing with hosting problems recently, and this advice will help me make the right decisions in the future when my sites get big too :)

    17.
    On August 28th, 2007 at 10:20 am, Aruni said:

    This is exactly why we moved from a shared semi-ded plan to a dedicated plan at Rackspace (www.rackspace.com). They aren’t cheap but their cusomter service is amazing. If anyone is interested in working with Rackspace, let me know and you can get a discount when using our name, and we get a referral credit. :-)

    18.
    On August 28th, 2007 at 10:27 am, Aruni said:

    Forgot to mention that we pay about the same as Wendy about $270/month with Rackspace, but we have a few more things given that we run our web application sites on that server as well, not just our blog. They manage everything for us and our project manager on our development team can log in with relevant access rights.

    I need to figure out how to get one of our posts to show up on Digg! :-)

    19.
    On August 28th, 2007 at 11:01 am, Edward Mills said:

    Hi Wendy

    Did you see this bullet point on Lorelle’s recent post about WP 2.3?

    # A new cache proxy called WPCP is coming which will speed up caching. Benchmark tests so far are amazing and will definitely speed up your WordPress blog.

    http://lorelle.wordpress.com/2007/07/23/wordcamp-2007/

    That’s gotta be a good thing!

    20.
    On August 28th, 2007 at 1:39 pm, MichelleVan said:

    Wendy, Even though I don’t have traffic problems….I appreciate this post because it explains why one of my sites is sometimes slower to respond. I always thought it was bandwidth - but it is probably who else I’m sharing with and the traffic they are getting.. Did I understand that correctly?

    21.
    On August 28th, 2007 at 3:53 pm, derek said:

    You’ve got a few great lessons in there for dealing with a huge influx of traffic. As Matt mentioned earlier, I too use BlueFur and they have been great for me thus far. I did experience a front-page digg and had a minor hiccup where they asked me about the CPU spike but as soon as I told them it was a digg they had me back up and running.

    Thanks for sharing your lessons learned Wendy!

    22.
    On August 29th, 2007 at 3:46 am, Deborah said:

    Much of this is over my head, but I recently had problems with front page diggs as well.

    When the first one was close to hitting front page my webmaster called to tell me he had to take the post down. It was consuming 99% of the bandwidth of my shared hosting, with an average of 1500 visitors per minute.

    I was forced to find new hosting and didn’t want to go thru the process more than once. I was recommended to Media Temple, for which I receive 1000G of bandwidth per month.

    No sooner had we switched hosting, a post hit front page at Digg. Surviving the Digg effect, it consumed about 45% of my alotted bandwidth, and even more the next day once it had been submitted to numerous other social sites. We were forced to remove the post once again.

    Hours later, yet another post hit front page. Because we caught it early and deleted the post, it consumed less than the first, but I’ve been keeping a low profile since to ride out to the end of this billing cycle.

    So one other thing to consider if you want to survive the Digg effect. Media Temple hasn’t let me down since I first switched hosts. The good part is that it only costs about $20/mo for 1000G of bandwidth per month.

    23.
    On August 29th, 2007 at 4:11 am, Andy Beard said:

    Deborah

    With Media Temple you have to look at your GPU usage more than bandwidth for most Wordpress blogs.

    24.
    On August 29th, 2007 at 9:36 pm, derek said:

    @Deborah :: Any time you want to come over to my site and write a guest post, let me know! With all your digg magic, I’d love to test my host again. :)

    25.
    On September 2nd, 2007 at 1:31 pm, Cal said:

    Wendy, have you looked into “real” caching solutions (I’m not an expert, but pound comes to mind, and I think Apache has a few tricks too.)

    The whole page can be cached in memory, not on the hard drive. And while that obviously interferes with dynamic stuff and maybe the comments, it may be worth it for heavy diggs. I believe you can tell the server to refresh the page once every few minutes or so.

    Would be interesting to see if anyone has any experience with that and knows of problems.

    26.
    On September 5th, 2007 at 12:13 pm, Lorelle said:

    Excellent article and great tips, Wendy. Just a minor point. Not “every” element on your WordPress blog interacts with the database. A graphic image hits the server, not the database. Javascript hits the server, not the database. Some PHP commands don’t interact with the database, but WordPress template tags interact with the database and other files on the server.

    Cutting back on everything that hits the server, like images and scripts, helps a ton. Cutting back on needless WordPress Plugins lowers the hits on the database.

    If you are in a shared hosting situation and timing is against you, the hits on the database are critical as those are the most fought over elements on the hosted server. Killing off the rest just lowers the interaction with the server and it’s memory share.

    27.
    On September 10th, 2007 at 5:24 pm, David Chin said:

    Superb article! As I write this, Hostgator had shut me down about three hours ago, and all I ever did was to run about, oh, thirty or so Wordpress installations with very light traffic. No problems before, but it’s probably my ever-increasing number of Wordpress blogs that are running into the memory limit of my account right now.

    Mentions on other sites...

    1. Self Made Minds » Blog Archive » That’s a wrap! - Friday 31st August on August 31st, 2007 at 4:48 am
    2. Timely Link Love for Bloggy Friends - eMoms at Home - The Internet Home Business Blog for Moms & Dads on September 21st, 2007 at 9:48 am


    Leave a Reply



    If you liked this article, take more To Go...

     

    Subscribe to Daily Posts Via Email
    Sign Up for the Weekly Email Newsletter
    I'll never sell or rent your info. Period.
     
    Subscribe Via an RSS Reader