Plesk 11 to VM – Part 3: Customisation and migration

Intro

In this series of Best Practice articles I talk about migrating websites and emails from a Plesk 11 run server to a Virtualmin run server. It describes in details the steps involved, including the eventual pitfalls and other thing I came across whilst doing it. Note that this documentation is addressed to people with medium expertise in linux/system administration.

Useful customisation

Most of VM defaul settings are fine, however in order to make it a little easier to migrate websites, the following mods are useful.

Server Template – Apache directives

As Plesk websites are hosted under $Documentroot/httpdocs this can be set by default before creating website by editing the Apache website default config:

VM Menu > Server Templates > Default Settings > Edit template section: apache website

Change: DocumentRoot ${HOME}/public_html
To: DocumentRoot ${HOME}/httpdocs

Change: <Directory ${HOME}/public_html>
To: <Directory ${HOME}/httpdocs>

Under “Users’ website subdirectory to create“.

Change: Default (public_html)
To: Directory under home [httpdocs]

Account Plans

One may customise them to match previous Plesk hosting “plans”, however this is not mandatory.

Virtual website creation – Admin username

The only requirement in this process is to create the Administration username as a “Custom username” giving it the domain.tld name (i.e. mydomain.com). This way the websites paths will end up being: /home/domain.tld, which makes it easier to do a path change from /var/www/vhosts to /home wherever there are scripts configurations containing full paths.

Perl

I am a perl script writer thus I use perl extensively, over the years I have created scripts that require a few additional modules. I shall not go into details with them (yet), however there’s an important bit I must mention, some perl scripts I use have a different #!. Some point to the sometimes non-existent /usr/local/bin/perl, so in order to avoid these script throwing an error I simply link it to the whereis perl (usally /usr/bin/perl or /bin/perl):

ln -s /usr/bin/perl /usr/local/bin/perl

Website content transfer

To transfer website content from Plesk to VM it is a matter of rsyncing each website with the rsync command, to minimise the load, the following commands are run at nice 20 from the source server, note that the target server must have the virtual server already configured prior to doing this, otherwise an error will be thrown:

rsync -avz --rsh='ssh -p{port}' /var/qmail/mailnames/{domain}/{user}/  root@{server}:/home/{domain}/homes/{user}/

Where:

port is the ssh port (if different than 22, which is recommended for security)
domain is the domain to be transferred
target is is the target server IP or domain

Depending on the size of the website this might take some time. Note that I am not really needing to transfer statistical information over. most if not all the required files are under httpdocs and cgi-bin in my case.

After the transfer, on the target server, ownership of the directories must be set:

chown -R {username}:{username} /home/{domain}/httpdocs /home/{domain}/cgi-bin

Where:

username is the owner of the httpd and cgi-bin content (in my case I created a username matching the domain, making it easier to find the correct directory under /home when working over the CLI.
domain is the domain

Email content transfer

The operation of transferring emails to the target server are pretty much the same as website transfer, with a couple of exceptions.

nice -20 rsync -avz --rsh='ssh -p{port}' /var/qmail/mailnames/{domain} root@{target}:/home/{domain}/home/

Where:

port is the ssh port (if different than 22, which is recommended for security)
domain is the domain to be transferred
target is is the target server IP or domain

After the transfer, on the target server, ownership of the directories must be set:

IMPORTANT: note user is user.domain (not just domain)

chown -R {user}.{domain}:{domain} /home/{domain}/homes/{user}/

Where:

user is the name of the user
domain is the name of the domain

IMAP changes (At client level, here using Thunderbird)

Server name: no change
Username: replace @ with .
Connection security: from SSL/TLS to STARTTLS
Authentication method: from encrypted password to normal password

SMTP changes (At client level, here using Thunderbird)

Server name: no change
Port from 25 to 587
Connection security: from SSL/TLS to STARTTLS
Authentication method: from encrypted password to normal password
Username: replace @ with .

Database content transfer

Source server

mysqlcheck --optimize {database} --auto-repair  -uadmin -p`cat /etc/psa/.psa.shadow`
mysqldump -uadmin -p`cat /etc/psa/.psa.shadow` --flush-logs --opt {database} > {database}.sql

Where:

database is the database name

Target server (after rsyncing all the dbs dumps on it)

VM Menu > Edit Databases > (manage current database usually called {domain}_{tld}) or create a new one if required.
click Manage …
Click Execute SQL
Run SQL from file Tab
From local file (browse to file)
Click Execute

Special characters

Sites with accents (like French’s)

The charset must be edited in the following Module:

Select Domain > Menu > Services > Configure Website > Languages > Character set for documents (x) iso-8859-1

Save, apply changes.

Plesk 11 to VM – Part 1: Why move?

Intro

Fed up with Plesk?

Since 2003 I have been using two brands of Dedicated Servers Control Panels: Ensim (RIP) and Plesk. So it’s died 🙁 oh well, Ensim was an awful CP, never liked it, I liked Plesk (from 8 to 11) a bit more and it has been my selected Control Panel for many years. However the time has come to make way for a relatively new comer: Virtualmin or VM for short, a very mature and simpler CP indeed with a vibrant Community.

Deciding to select another Control Panel is a big decision, especially when your bottom line is at stake, so after having extensively tested and used VM for the last few months I finally decided to make the move, VM is slick, simple and easy to use, it’s like Ronseal, it does what it says on the tin.

Continue reading Plesk 11 to VM – Part 1: Why move?

Reverting to passworded SSH (root) logins

My normal ssh logins are with public keys authentication (aka passwordless), however, under certain circumstances I’d like to revert back to good old passwords logins. Like when I want to transfer a Plesk server across another server using the Plesk Migration & Transfer manager.

After scouring the Net for some inspiration, which failed, and some tinkering, I finally found a neat solution to switch between the two (passwordless and normal authentication logins) quickly, the process is relatively simple* but frankly puzzling IMHO. I shall explain why: Continue reading Reverting to passworded SSH (root) logins

Plesk panel, backup strategy when low in space

Plesk backup is a real pain in the backside, if your server disk space is limited, even when choosing to backup in a ftp repository. All the files are created locally and then send over via ftp.

Prerequisites

Note that in order to implement this backup strategy, one must have an external backup space available like rsync.net (which I use) so that you can use the “rsync” command to transfer your files.

Minimising disk usage

In order to exclude any gigantic compressed (or not) archives being stored in your local hard drive, here is a little guide to help you avoid filling your server disk.

The first thing to backup regularly is the Plesk server configuration, this backup will not save the websites/email/database data but the content/configuration of your Panel, which is the first thing to restore if your server packs up.

Backing up Plesk (11) configuration

Open your Plesk Panel as Admin
Choose Tools & Settings from the Server Management sub-menu
Click on Backup Manager
Select Scheduled Backup Settings
Activate the scheduled backup
Select Store backup in: Server repository (or you can chose ftp there also as the files aren’t big, a couple of MB’s)
I have set the Maximum number of backups in repository to “3”, but you can choose another value here
Under the Backup content section, select “Server configuration“
Press OK

Obviously the choice of Schedule and prefix is at your convenience, I personally back up every day at 00:10 and the prefix is set to “configuration”.

If you aren’t using ftp, the content of this backup is stored in the /var/lib/psa/dumps/ directory. So in order to backup this on another server, simple rsync this location with a command that could be:

rsync -avz /var/lib/psa/dumps/ user@domain.rsync.net:var/lib/psa/dumps

If you don’t want to keep aggregated data add the –delete option:

rsync -avz --delete /var/lib/psa/dumps/ user@domain.rsync.net:var/lib/psa/dumps

To automate

To automate all this simply stick it in a cron (one can use the “Scheduled Tasks” under “Tools & Settings” (select root user) with a command that could look like this:

nice -20 rsync -avz --stats --delete /var/lib/psa/dumps/ user@domain.rsync.net:var/lib/psa/dumps 2>&1 | mail -s "backups dir rsync report" you@youremail

Databases

I will refer to MySQL databases in this section.

Database content is important, and you must back them up regularly, be it incrementally or not. I have Perl scripts that are in charge of dumping the content of the entire dataset and compress it. The file for my setup ends up being about 300MB, which is acceptable. I proceed to dump the content of the dataset via a “schedule” (root cron) and another cron simply “rsync’s” this file over to the backup server.

Email

All the email accounts are in /var/qmail/mailnames, the process is the same as above, use a rsync via a cron:

rsync -avz /var/lib/qmail/mainames/ user@domain.rsync.net:var/qmail/mailnames

Website content

Same as email but the directory is /var/www/vhosts/ so:

rsync -avz /var/www/vhosts/ user@domain.rsync.net:var/www/vhosts

There you have it, obviously I am providing this entirely as guidance and will in no way be liable for any loss you may get using these instructions. Even if you are on a cloud server (like me) it is a good idea to be able to extract files, especially databases or web content should a mistake be made that a cloud server cannot prevent like deleting a site by mistake etc…

Feel free to comment if you have any questions or if I have missed some important bits to backups that a Plesk backup may make which aren’t in this article.

qq trouble in home directory (#4.3.0) Plesk 10

A recent problem appeared on my system, an emails address was refusing emails sent internally and a message was coming back immediately (if using a mail client like Kmail or deferred if using a webmail service like Gmail):

451 451 qq trouble in home directory (#4.3.0) (state 17)

Searching for this problem on Google does not return much. however I tried to do the steps of switching from Qmail to Postfix based on BStar’s Code Snippets experience.

After changing from Qmail to Postfix, I could not do the reverse immediately (Plesk install returned an error I was using the CP)

This changed the problem to state that the mailbox was full. Yet funnily enough the recipient in question had no Mailbox, only a redirect.

In the end I had to remove the mail account and re-create it as before. since then all is back to normal.

I also managed to switch back to Qmail using the CP afterward.

File access PHP on Plesk

On certain occasions, and if you’re running Plesk on a dedicated server (like I do), there are certain problems with file access and other upload permissions which aren’t allowed unless you turn file permission to 777!

Obviously this isn’t the recommended way to operate a website, turning directories and files all open with rwx is kind of risky.

So I have found a solution that works everytime, simply change the “PHP Support” to run as “FastCGI Application” instead of the default “Apache”. This option is found under the Hosting setting of the domain.

I must point out that this hack only works if you’re running Plesk (10+) on Linux Servers.

Setting up a remote connection to a MYSQL database on Plesk 10

A new project’s required me to have access to a MYSQL database from a REMOTE IP address.

There are conflicting information about this, so in order to get the thing done correctly, here’s a reminder of the various steps:

1.Check /etc/my.cnf, normally the file’s already correct (Since Plesk 8), Reference here

2. open mysql shell session with:

# mysql -uadmin -p`cat /etc/psa/.psa.shadow` mysql

3. add credential to allow remote IP to connect to specific database:

GRANT ALL ON foo.* TO bar@'192.168.0.1' IDENTIFIED BY 'PASSWORD';

REVOKE GRANT OPTION ON foo.* FROM bar@'192.168.0.1';

where:

“foo” is the database name

“bar” is the database user name

“192.168.0.1” is the remote IP address

“PASSWORD” is the password, a random value (NOT PASSWORD!)

4. Open the Mysql port( 3306) on the Plesk Firewall (in the Modules section)

5. Test the connection using:

telnet host 3306

where:

“host” is the Plesk server to access.

A connection should then be able to be made.

Plesk Migration Manager (PMM)

Just to share my experience with this application. If you are migrating Plesk (8.x) to Plesk (8.x) (ie another server running plesk) and you have little space left on the /home (or /) hard disk partition on the source server.

You must also have a partition that has at least enough space to hold the /var/www/vhost content…

Plesk migration manager stores dumped file under /home/migration. To alleviate the risk of filling the / or /home partition do this:

rmdir /home/migration
ln -s {otherpartitiondir} /home/migration

where {otherpartitiondir} is a full path on the partition that has enough space.

Plesk is not really telling you where the stuff is stored (or I did not find it)