The UK servers is synchronised with the US server, only but a few websites are active on the the UK server, in addition, an off-site backup is also planned to be used to save UK server content. This will take place in a few days.
If test are conclusive and the backup can hold all important data, we may do away with the US server. But there are no plans for this at present.
The UK server is being synchronized daily with the websites of the USA server, apart from a few exceptions, most website are now fully backed up. Some websites are already being served by the UK server and a synchronisation with the USA server will follow shortly. The big migration is planned for week 33 (Sept 11-15 2008). after that date, a full syncronisation will be taking place to either websites, (some will stay in the US and a mirror will be in action in the UK).
Plans are going really well.
Both UK and US web servers now have a status page providing useful information about the their status, the US server has it’s info of the UK server and vice versa. This allows our clients to find out possible down time and causes of the interruption, scheduled or not.
Following the early June Server Farm fire and subsequent down time here is a follow up on the progress so far and short terms plans.
The new server in the UK is nearly ready for production and to become the primary server, this will have a impact on server response as there are fewer nodes to go through to reach UK/EU customer base.
However this will be very minimal gains and US, rest of the world visitors will not see a difference. I will also operate a fail over policy for dns, emails and websites:
All websites will be synchronised on an daily basis between the 2 servers, in case of lasting downtime from the UK server, we will be able to reverse to the US server within hours (the time it will take to manually change the DNS records)
For email service, the UK server will handle emails. However, as email is a bit more resilient than websites in a way that if emails cannot be delivered they will be queued until they can, a fail over is less critical.
I am currently experimenting with the possibility of having the two servers able to receive the same email message, if one is not working, the other should. This would require having two email accounts for the same address and access these two accounts using the IP address of the mail server instead of the domain name, it is fairly technical and needs to be tested for feasibility first.
As for DNS, I will also have more than one name server (ie phone book) as DNS protocol require at least one of these to minimise interruption, I will have the record stored in 6 name servers located in different locations. The websites will be unreachable only if the Internet goes entirely down.
Following the early June Server Farm fire and subsequent down time here is a follow up on the progress so far and short terms plans.
The new server in the UK is nearly ready for production and to become the primary server, this will have a impact on server response as there are fewer nodes to go through to reach UK/EU customer base.
However this will be very minimal gains and US, rest of the world visitors will not see a difference. I will also operate a fail over policy for dns, emails and websites:
All websites will be synchronised on an daily basis between the 2 servers, in case of lasting downtime from the UK server, we will be able to reverse to the US server within hours (the time it will take to manually change the DNS records)
For email service, the UK server will handle emails. However, as email is a bit more resilient than websites in a way that if emails cannot be delivered they will be queued until they can, a fail over is less critical.
I am currently experimenting with the possibility of having the two servers able to receive the same email message, if one is not working, the other should. This would require having two email accounts for the same address and access these two accounts using the IP address of the mail server instead of the domain name, it is fairly technical and needs to be tested for feasibility first.
As for DNS, I will also have more than one name server (ie phone book) as DNS protocol require at least one of these to minimise interruption, I will have the record stored in 6 name servers located in different locations. The websites will be unreachable only if the Internet goes entirely down.
Using encrypted keys for authentication offers two main benefits. Firstly, it is convenient as you no longer need to enter a password (unless you encrypt your keys with password protection) if you use public/private keys. Secondly, once public/private key pair authentication has been set up on the server, you can disable password authentication completely meaning that without an authorized key you can’t gain access – so no more password cracking attempts.
It’s a relatively simple process to create a public/private key pair and install them for use on your ssh server.
First, create a public/private key pair on the client that you will use to connect to the server (you will need to do this from each client machine from which you connect):
ssh-keygen -t rsa
This will create two files in your (hidden) ~/.ssh directory called id_rsa and id_rsa.pub. id_rsa is your private key and id_rsa.pub is your public key.
If you don’t want to still be asked for a password each time you connect, just press enter when asked for a password when creating the key pair. It is up to you to decide whether or not you should password encrypt your key when you create it. If you don’t password encrypt your key, then anyone gaining access to your local machine will automatically have ssh access to the remote server. Also, root on the local machine has access to your keys although one assumes that if you can’t trust root (or root is compromised) then you’re in real trouble. Encrypting the key adds additional security at the expense of eliminating the need for entering a password for the ssh server only to be replaced with entering a password for the use of the key.
Now set permissions on your private key:
chmod 700 ~/.ssh ; chmod 600 ~/.ssh/id_rsa
Copy the public key (id_rsa.pub) to the server and install it to the authorized_keys list (or open the authorized_keys list file and paste the content of id_rsa.pub into it) :
cat id_rsa.pub >> ~/.ssh/authorized_keys
Note: once you’ve imported the public key, you can delete it from the server.
and finally set file permissions on the server:
chmod 700 ~/.ssh ; chmod 600 ~/.ssh/authorized_keys
The above permissions are required if StrictModes is set to yes in /etc/ssh/sshd_config (the default).
A complete memo of Sharpnet UK disaster recovery strategy will be laid out here. This will prove a valuable source of information to other Dedicated server administrators, especially new players.
Comments welcome!