HotSanic – Thumbnail generation problem

My new server has never been able to show the main HotSanic index page with thumbnails images, all I got was resized images. This problem had never happened on the old server.

I eventually gave up, then I really wanted to find out why I could not get these thumbnails, I had ImageMagick installed after all and the log files were not showing any errors, and no specific resources are available on the web about this problem.

After some investigating I found out the cause.

In the main HotSanic settings file, there is a CONVERTMETHOD variable.

The options available are explained above this variable.

My settings was CONVERTMETHOD=”HTML” all I had to do is change it to CONVERTMETHOD=”ImgMgck”

Simple.

After that I ran the makeindex.pl and convert.pl and presto, my thumbnails were there.

http://www.scothorse.net/HotSanic/

Backup strategy – off site backups

The various elements of the backup strategy having been defined and documented earlier, the last step involves the off site backup of the local backups.

This is done using the rsync command to mirror the entire content of the local backups directory as all backups are stored under this directory.

In case of disaster the off site backup could either be burned on a series of DVD’s and posted to the DC to have them restored or sent back to the dedicated server via ftp or other, this is where it is important to have a good backup server capable of coping well with large files.

If I were to rsync all the backup content on my own internal network backup server, this would be quite fast to pull, however ADSL push would need a serious amount of time to upload the content back, even though it is probably the safest way to keep data. And having over 500GB of available space it makes sense.

So I might very well do that as a disaster recovery option…

Have yet to decide. It’ll probably a combination of local network and other remote backup site…

Backup strategy – Websites

Each individual websites are now backed up within Plesk’s schedule once a week and stored on the local backups/dumps/* directory ( Note that this path has been linked from the default Plesk path to increase disk space)

All the sites have a rolling backup where 2 backups are kept, except for a few very large website generating several GigaBytes backups.

Backup strategy – Databases

The mysql database is backed up on a seven days rolling dump. these files are stored under the local backups/mysql directory.

In addition, the classifieds database is backed up weekly and stored for ever… Just in case we’re facing problem with past advertisers, legal challenges etc…

These backups are performed via a Plesk root scheduled cron and the Perl script invoked handles the database sanitation and backup on the fly. The Perl script is stored within the sharpnet private area so it’s self contained and will work out of the box in case of a restore…

AOL and emails not getting through

Okay, this problem always seems to happen every time I move to a new server, so if I document it I might actually get it sorted before it starts happening

My server cannot send emails to AOL. The problems comes from the fact that the IP address seen by AOL when receiving emails is apparently xxx.xxx.xxx.xxx (obfuscated to protect address, throughout this post)

I use the instructions from this URL:

http://postmaster.aol.com/Postmaster.Troubleshooting.html#vrfIP

and the email I get back from AOL using webmail on my dedi is:

Your connecting IP is: xxx.xxx.xxx.xxx

Please visit our web page at: http://postmaster.aol.com for more information about AOL Email Policies and methods to fix delivery issues.

Postmaster Group
America Online, Inc.

If I test this IP on this page:

http://postmaster.aol.com/cgi-bin/plugh/rdns.pl

I get an error

Here is the email I receive on my AOL email address (after several hours and having added the sender’s email address in my list of contacts, as AOL seems to tell to do that):

Return-Path:
Received: from mtain-mb03.r1000.mx.aol.com (mtain-mb03.r1000.mx.aol.com [1×2.x9.96.×3]) by air-mc04.mail.aol.com (v127_r1.1) with ESMTP id MAILINMC042-a9304b7dd0981d4; Thu, 18 Feb 2010 18:43:20 -0500
Received: from srv–.——-.co.uk (unknown [92.xx.69.187])
by mtain-mb03.r1000.mx.aol.com (Internet Inbound) with ESMTP id C6E7B3800009A
for ; Thu, 18 Feb 2010 18:43:04 -0500 (EST)
Received: (qmail 18299 invoked by uid 48); 18 Feb 2010 18:46:36 +0000
Received: from —–.gotadsl.co.uk (—–.gotadsl.co.uk
[213.xxx.112.119]) by webmail.pastelpals.co.uk (Horde MIME library) with
HTTP; Thu, 18 Feb 2010 18:46:36 +0000
Message-ID: <20100218184636.8cd1edr640og0gcg@webmail.pastelpals.co.uk>
Date: Thu, 18 Feb 2010 18:46:36 +0000
From: —–@—–.co.uk
To: —–@aol.com
Subject: TEST
MIME-Version: 1.0
Content-Type: text/plain;
charset=ISO-8859-1;
DelSp=”Yes”;
format=”flowed”
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
User-Agent: Internet Messaging Program (IMP) H3 (4.1.6)
x-aol-sid: 3039ac1d60174b7dd0884a98
X-AOL-IP: 92.xx.69.187
X-Mailer: Unknown (No Version)
X-Length: 1203
X-UID: 24645412

As one can see, the IP AOL sees (92.xx.69.187) is not the IP of the sending Server (92.xx.68.118)

I have still got a ticket opened with the DC regarding this…

Tbc…

Backup strategy – Appliance

The First stage of the backup strategy is to have a full clean backup of the web server content, including emails.

This creates a rather large file (around 16GB), that take about 10 hours to be created, therefore you don’t want to do this every day. This file will be kept in addition of the monthly backups

The current plan is to have these dumped in the local /backups/full directory once a month and then rsync it to the backup server (I haven’t yet decided which one).

These backups are performed via a Plesk root scheduled cron and the Perl script invoked handles the database sanitation and backup on the fly. The Perl script is stored within the sharpnet private area so it’s self contained and will work out of the box in case of a restore…

Local plesk backups – change dump location

All Plesk “local” backups are stored in the following path:

/var/lib/psa/dumps

The problem resides in the fact that if the partition containing /var is small, this can create big problems. Luckily Plesk is happy with symlinks…

So in order to raise the space available for dumps, simply move the content of the dumps directory to the new location and create a symlink:

Commands:

cd /var/lib/psa
mv dumps /home/backups/
ln -s /home/backups/dumps dumps

This creates a symbolic link to the new dump path which has plenty of disk space.

Backups – the strategy

Hello me.

As the new Sharpnet strategy is being put in place. I will do the harduous task of documenting all the steps involved in the new backup strategy adopted to protect our asset (the server data).

After a few brainstorm followed by a few restful nights, I have arrived to the following strategy:-

  • Remove all unnecessary content from the websites hosted on the server
  • Backup the appliance (using pleskbackup)
  • FTP the content of every website monthly
  • Rsync the httpdocs of every website daily
  • Rsync the content of the most important databases, daily, on a sever days rolling backup
  • Backup the most important databases weekly and keep them for as long as possible.

This is the plan.

Watch this space…

I have no life…

There, I said it, I have no life.

I don’t know why but even though I am quite open to new technologies, when they’re useful that is, I feel that the web is turning into a very strange hurly whirly of virtual “I need to be online all the time with everyone I know and let them know what I do every seconds of my life” thingy, stuff etc..

Now I find that very intrusive and pardon if I might be “Gay” (I am not, but my son use this all the time) but I cannot begin to understand the buzz (and I’ll come to this later) behind this.

I must be getting too old for this stuff (I am 44)

Back in the days, about 15 years ago, the Web was great, things were simple; you didn’t get (much) spam, most things were free and the information was more or less useful.

Today the web is turning into an ugly beast, [much too] full of useless stuff drowning the useful stuff. The useful stuff is usually no longer free though, this can be a good thing, but it tends to become a little silly most times.

In 1996 I watched a TV program about the web, and there was an interview of an artist from around L.A. I think, and he said (roughly):

“The creativity that is allowed to be broadcast to the world via the web will soon be locked down and that will be the death of the Web; the commercial world will take over and that will be it! Snap! All will no longer be free, that will be it, the web will be dead, and the good times will be gone forever”

This, I think, happened more than 5 years ago.

Today the web can be and still is very useful, but it’s getting harder to sort the weed from the chaff.

I have problems with this and I am a veteran, so imagine what it does to people who’ve never been on the web until today, like 3-4-5 years old kids or even people in their 60-70’s who can barely understand a Computer to start with, and could potentially find themselves in front of a page that says “Your computer is infected”* on their first hour of web browsing.

What kind of message is that giving out? The web is crap… and don’t trust it, but then it’s usually too late, I am afraid to say, the poor sod is reduced to pay criminals.

That frightens/worries me slightly.

I do have kids, 3 boys and they are, fortunately, in their teens now and quite frankly very nice chaps; they were given unrestricted and, god forbid, unsupervised access to the Web since they were about that age. But I am glad it was 10 years ago! Now I would not be so sure. All they were interested about was, and still is, gaming.

To come back to the Buzz, about all this, and the reason that prompted me to write this, is the arrival of Google “Buzz” in my Gmail, which I promptly hid.

You see a couple of months ago, I signed up to Facebook and other Plaxo “social websites”, the latter to reconnect with long lost friend in my country of origin (Switzerland) and the former just to see what all the fuss was about.

Plaxo was useful to initiate contact, and Facebook, Well, I stuck a couple of photos on, talked about my hobby (Motor biking) and lost interest pretty much after 3 weeks, I since disabled it. I find it an utter waste of time and, as I previously said, having no life does not help, sorry, my wife uses it for business and she’s got more of a life than me.

I also opened a Twitter account, this also has been falling in the way side, after the only thing I was writing on it is “Typing this on this site” (Well it asks you what you’re doing RIGHT NOW isn’t it?). It became quite boring after a short while, so apart from posting some links explaining what I think of Twitter/Facebook/Bebo etc. I don’t bother.

Maybe one day when I have a life and more friends I will use these again, but at present I don’t find them at all interesting, but that’s just me.

I have no life, remember?

What I do find still a little useful are Discussion Forums, so I guess I am finding myself being part of a forgotten generation that died many, many Web Moons ago.

I don’t know if it’s just me that believe all this? Surely not! There must be people that have better things to do out there than Tweeting and Book their face? Maybe it’s a fad?

Yes, it’s a fad.

It’s got to be…

On to the next thing, I might start “Blogging” more, something I really never though much about… You never know.

Boot note:

Please Google don’t mess up the Web, it’s still a great place to be on.

Will it still be in 10 years time? I guess I’ll have to wait and see.

Maybe I’ll get that lost life back as well.

* You know the bogus website tricking you to think you’re really infected, when you’re not… This is actually quite comical when I discover that my Linux machine has got 100’s of infected “dll” and “exe” files! (I have been using Linux as my OS of choice since 2003)

All websites moved

All the websites and email accounts have now been moved. There was a little hicup as the pleskbackup (8.6) did not like me, so I had to backup and restore each websites manually.

There was also a problem with the/usr/ partition overfilling and crashed the Plesk CP by filling up /usr/local/psa/PMM/logs, I had to move some restore logs as they were huge (3.4GB). After that, panic over and all went ok. (I took the safety option not to delete the files as you never know what Plesk can throw up at you, but it did not seem to affect my work, so I don’t know what these files are for except after action review? yes I fancy looking at a 3.4GB log file!)

Plesk does seem to have some issues regarding disk space…

Anyway, the site are all moved and all is working well again. The DNS will be propagated by tonight, as I can already ping the new IP…

The Urchin stats will be replaced by Google Analytics, it seems quite good.