A junior web developer often has a capability which in other professions is often reserved for exceptionally qualified individuals: the ability to completely fuck-up an entire business.
Many businesses are run completely online these days or their online presence is at least a major factor so anything that disrupts a website can be catastrophic. The very nature of the web, and how it works, means that it’s very easy when you’re making a change to a website to break the site completely.
Times when you are most likely to make a fuck-up include:
- Moving servers e.g. updating DNS records
- Uploading edited files to the server
- Editing a server configuration file e.g. Apache’s httpd.conf or PHP’s php.ini file
How it can be avoided
Everyone, has to learn and making mistakes is a very large part of that. I’ve worked on live servers before on massive sites and produced some lovely PHP errors messages plus some completely white screens of death that 1000s of people have seen and it’s nearly cost me everything. I’ve wiped-out entire email systems for clients for half a day and it’s scared the bejesus out of me.
The errors that I’ve made are avoidable (not always, but mostly) by using best practices which involve:
- create testing servers for every website (e.g. dev.example.com or test.example.com) which probably should be password protected/have restricted access
- create fully working local copies of websites and test everything on you local machine before uploading anything – even to the test server.
- run queries in a local copy of phpMyAdmin/MySQL Query Browser before uploading anything
- before changing any DNS settings fully plan every move. Check where the MX records are pointing. Don’t ever do what I’ve done and move the name servers without checking if the MX server was different to the A record. That really fucks things up.
- put up a holding page when making major changes
A quick word about DNS
DNS is a massive bug bear, if your boss doesn’t know anything about DNS make sure you explain to him on day 1 of your job that any changes to DNS can take days to fully work. Also make sure they know that if they purchase a domain name on 9am Monday morning you won’t be able to have a holding page up on that domain by 10am!
I’ve known people get royally told-off because of DNS and to no fault of their own. You can’t predict DNS, so you can’t say this brand new website will go live at 1pm on the dot if DNS is involved.
Local copies/testing servers
Test, test, test and test some more. Sometimes, even when you have exhaustively tested there may be a tiny little difference between your local server and the testing server and/or the live server which could send everything falling down when you go live. Therefore it’s always best to over-estimate to your client/boss how long it will take and factor in any problems – that way if it all goes smoothly, they’ll think you’re a hero for getting through it so quickly.
I use Webserv on my PC in order to run a local server – installing it takes around 10 minutes and then you’re good to go. Set up your hosts file and virtual hosts in your httpd.conf and you’re away.
The holding page, a thing of beauty
Nothing screams professional like a well created holding page (actually nothing screams professional like a holding page appearing while updates are performed at 3am while you get triple time, oh yeah!) Putting up a holding page stating that the site is going through temporary maintenance lets the site’s visitors know what’s going on and stops them from seeing ugly error messages or broken layouts which may prevent them from returning.
Test your queries
Every now and again an SQL query can take down the MySQL engine and grind the site to a halt so make sure you test those queries on a local machine using either phpMyAdmin or MySQL Query Broswer (download query broswer) that way you can see if you’ve made any catastrophic typos before it’s too late.
For instance if you forget to type out a join on a large relational database you can bring back a huge resultset that takes forever and slows down the server significantly.
Summary
Mistakes happen, end-of. The difference between an experienced developer and a junior is for an experienced developer the mistakes happen less often and when they do they’ll panic less and rectify them much quicker – but only because they’ve made those mistakes before when they were juniors.
When the case occurs that I need to do updates to a live piece of a site, I’ll usually use some user-agent cloaking to test the new bit of code before unleashing it on the general public.
Provided you don’t make any syntax errors, this is a nice way to test things out behind the scenes.
I do a sort of quick and dirty version of that, Mike; if what I’m testing is pretty simple, I just wrap it in an if statement looking for a querystring (for example, “?test=true”) so I can tell if it will break anything else on the page. This also makes it easy to send preview links to the client without having to explain user-agent strings to them.
They both seem like good ideas that I wasn’t at all aware of.
Mike: sadly, I am the king of the syntax error so this approach may not suit me.
Hey Matthew, nice idea. I’ve tried having them set a cookie for site wide previews (rather then cloak), but even that gets some people confused.
@phil – Yeah, that is one downside. I do tend to check locally/on the dev version first when I can. Other times I just try to be quick with the ‘undo’ and ‘upload’ :-)
Nice little article – not much outside the standard but always good to have it reinforced. I like you honesty to. “scared the bejesus out of me” … its true! and mistakes never get any easier to deal with :-)
BACK EVERYTHING UP BEFORE YOU GO HOME!
This might seem excessive but just last week ive lost about 4 months worth of hard labour due to a catastrophic hard disk faliure. (luckily it was just a few projects i’d been juggling for my portfolio. Its not drastic enough to have to consider remortgaging/suicide) Its now part of my routine that i leave 10 mins a session to whack everything on a DVD-RW.
@Tony C: Great advice. I now use an external HDD with Tim Machine (OS X) but backing up is so important and it’s a really good idea to work it into your schedule like you have.