Nov 28

COMPLETED: Planned Maintenance – CPanel09

Planned Maintenance: cPanel09 Hosting Server

Planned Maintenance: cPanel09 Hosting Server

Replacement Drives, Migration of Data and Hardware Upgrades

What is being done:

Replacement of SSD Hard Drives and Migration of all existing data to the new disks, following the check/repair/analysis of other hosting servers (see Emergency Maintenance – CPanel06 on 16/November) we are continuing the rollout of new SSD drive arrays for shared-hosting, reseller hosting, secure hosting and email hotsing services

Who is affected:

Clients with cPanel/WHM Linux Hosting Services (web, email, personal, business hosting packages) on server:

  • cPanel09
cpanel logo

cpanel logo

The server will need to be completely offline during the maintenance to physically swap hard-drives, and restore all data as quickly as possible.

During the maintenance window you will not be able to access any aspect of your hosting accounts through the Client Portal, via WHM/cPanel or by http/ftp/pop3/smtp/imap etc

The servers will need to be powered down, have all drives removed, duplicated and replaced, and other hardware changed before bringing up with limited access whilst services are started in a controlled manner and the reported disk-io and load investigations are completed.

This will involve a maximum of 6 hours of outage of all services (mail, web, databases etc) on server cp09 and some limited access for an additional 24 hours whilst the impact and improvements are monitored and tweaked.

We are unable to provide an exact time for you to regain any access to your hosting account as it will depend on the type of services you use – email only will be back first, followed by ftp, web, databases – if any ‘prioritisation’ of clients is necessary, then those with the advanced-support levels will be 1st, followed by Enterprise & E-Commerce clients, then Business, Personal and finally any trial and subsidised/discounted/internal services.

Incoming email during this time will ‘queue’ at the sender, Outbound emails will stay on your local machine (where using a mail client) – webmail access will be unavailable until the work is completed.

Websites will be down until the data has been restored, Websites which rely on a database will start to work once the mysql databases are restored and repaired.

FTP upload/download access will return once we have confirmed all data is available.

Access to other services/ports where supported will then be made available again.

When is this maintenance:

Starting Saturday 29 November 2014 22:00
Completing Monday 01 December 2014 06:00

Why do we need to do this:

Several hosting servers have reported disk-corruptions over recent months, which required previous maintenance and downtime to repair on some. On completing a full scan of all drive arrays on all servers, datacentre technicians decided that a full replacement would be necessary at some point in the future for each machine (around a 1 year anniversary from last upgrade). In order to minimise any potential loss of client data, we have decided to do that in a controlled manner as soon as possible.

We are therefore going to be migrating to new drives to increase the IO capabilities, add more ram to upgrade caching for overall performance benefits, and some reconfiguration – so that the server(s) can be returned to service at full capacity and eliminate the ‘pending fail’ possibility of the drive array.

updates

2014-11-03 03:15 – Data restore completed – server returned to service with staff access only
2014-11-03 05:40 – All Email Services returned to service – imap/pop3/smtp – exc. webmail
2014-11-03 06:25 – All static websites returned to service
2014-11-03 06:55 – Webmail returned to service
2014-11-30 07:15 – All MySQL DB services returned to service after full mysqlcheck

Maintenance is completed – if you are currently seeing any issues with your service on hosting server cpanel09 please open a ticket through the client portal

Nov 16

COMPLETED: Planned Maintenance – CPanel06

Planned Maintenance: cPanel06 Hosting Server

Emergency Maintenance: cPanel06 Hosting Server

Replacement Drives, Migration of Data and Hardware Upgrades

What is being done:

Replacement of SSD Hard Drives and Migration of all existing data to the new disks, following the check/repair/analysis from Emergency Maintenance – CPanel06 on 16/November, and reconfiguration of caching, intrusion-detection systems, extra firewalling and other tweaks.

Who is affected:

Clients with cPanel/WHM Linux Hosting Services (web, email, personal, business hosting packages) on server:

  • cPanel06

The server will need to be completely offline during the maintenance to physically swap hard-drives, and restore all data as quickly as possible.

During the maintenance window you will not be able to access any aspect of your hosting accounts through the Client Portal, via WHM/cPanel or by http/ftp/pop3/smtp/imap etc

The servers will need to be powered down, have all drives removed, duplicated and replaced, and other hardware changed before bringing up with limited access whilst services are started in a controlled manner and the reported disk-io and load investigations are completed.

This will involve a maximum of 6 hours of outage of all services (mail, web, databases etc) on server cp06 and some limited access for an additional 24 hours whilst the impact and improvements are monitored and tweaked.

We are unable to provide an exact time for you to regain any access to your hosting account as it will depend on the type of services you use – email only will be back first, followed by ftp, web, databases – if any ‘prioritisation’ of clients is necessary, then those with the advanced-support levels will be 1st, followed by Enterprise & E-Commerce clients, then Business, Personal and finally any trial and subsidised/discounted/internal services.

Incoming email during this time will ‘queue’ at the sender, Outbound emails will stay on your local machine (where using a mail client) – webmail access will be unavailable until the work is completed.

Websites will be down until the data has been restored, Websites which rely on a database will start to work once the mysql databases are restored and repaired.

FTP upload/download access will return once we have confirmed all data is available.

Access to other services/ports where supported will then be made available again.

When is this maintenance:

Starting Monday 17 November 2014 05:00
Completing Tuesday 18 November 2014 11:00

Why do we need to do this:

A disk-corruption has occurred, which required previous maintenance and downtime to repair. On completing a full scan of all drive arrays, datacentre technicians decided that a full replacement woudl be necessary at some point in the future. In order to minimise any potential loss of client data, we have decided to do that in a controlled manner as soon as possible.

We are therefore going to be migrating to new drives to increase the IO capabilities, add more ram to upgrade caching for overall performance benefits, and some reconfiguration – so that the server(s) can be returned to service at full capacity and elimiate the ‘pending fail’ possibility of the drive array.

updates

2014-11-17 07:55 – Data restore completed – server being returned to service with staff access only
2014-11-17 08:00 – All Email Services returned to service – imap/pop3/smtp – exc. webmail
2014-11-17 08:10 – All static websites returned to service
2014-11-17 08:20 – Webmail returned to service
2014-11-17 09:20 – All MySQL DB services returned to service after full mysqlcheck
2014-11-17 22:00 – All services have been fully operational for 4+ hours load is down, peformance is up

Maintenance is completed – if you are currently seeing any issues with your service on hosting server cpanel06 please open a ticket through the client portal

Nov 16

COMPLETED: Emergency Maintanence – CPanel6

Completed: Emergency Maintenance: cPanel11 Hosting Server

What is being done:

At present the drives are being checked for consistency and being reorganised due to potential issues being reported by the drives.

Who is affected:

Clients with cPanel/WHM Linux Hosting Services (web, email, personal, business hosting packages) on server:

  • cPanel6

 

When is this maintenance:

Starting Sunday 16 November 2014 19:00
Completing – Presently unknown

Why do we need to do this:

The current disk drives in this server are reporting potential problems.  We are taking this action at this stage to prevent further issues later on.  Depending on the outcome of the initial work, the drives may need to be replaced.

Updates:

  • 16/11/2014 20:53 – Services have been restored at present.  Another window will be scheduled to replaced the disks.

Nov 01

COMPLETED: Emergency Maintenance – CPanel11

Emergency Maintenance: cPanel11 Hosting Server

Replacement Drives, Migration of Data and Hardware Upgrades

What is being done:

Replacement of SSD Hard Drives and Migration of all existing data to the new disks, in order to stabilise an intermittent Disk-IO/Load issue, addition of more RAM and reconfiguration of caching, intrusion-detection systems, extra firewalling and other tweaks.

Who is affected:

Clients with cPanel/WHM Linux Hosting Services (web, email, personal, business hosting packages) on server:

  • cPanel11

The server will need to be completely offline during the maintenance to physically swap hard-drives, and restore all data as quickly as possible.

During the maintenance window you will not be able to access any aspect of your hosting accounts through the Client Portal, via WHM/cPanel or by http/ftp/pop3/smtp/imap etc

The servers will need to be powered down, have all drives removed, duplicated and replaced, and other hardware changed before bringing up with limited access whilst services are started in a controlled manner and the reported disk-io and load investigations are completed.

This will involve a maximum of 12 hours of outage of all services (mail, web, databases etc) on server cp11 and some limited access for an additional 48 hours whilst the impact and improvements are monitored and tweaked.

We are unable to provide an exact time for you to regain any access to your hosting account as it will depend on the type of services you use – email only will be back first, followed by ftp, web, databases – if any ‘prioritisation’ of clients is necessary, then those with the advanced-support levels will be 1st, followed by Enterprise & E-Commerce clients, then Business, Personal and finally any trial and subsidised/discounted/internal services.

Incoming email during this time will ‘queue’ at the sender, Outbound emails will stay on your local machine (where using a mail client) – webmail access will be unavailable until the work is completed.

Websites will be down until the data has been restored, Websites which rely on a database will start to work once the mysql databases are restored and repaired.

FTP upload/download access will return once we have confirmed all data is available.

Access to other services/ports where supported will then be made available again.

When is this maintenance:

Starting Saturday 01 November 2014 22:00
Completing Monday 03 November 2014 10:00

Why do we need to do this:

Clients are reporting issues with ‘load’ related problems with MySQL DB driven websites and intermittent mail access at some times.

Investigations into these issues have found a number of causes, some which have been mitigated to the extent they can be, by a variety of means:
– attempts to ddos certain clients
– client setup issues triggering the IDS
– customers with ‘service abusing’ processes
– use of technologies banned from our shared hosting service
etc

However there appears to be some which will require more advanced analysis, which itself is increasing the load and IO, so inadvertently extending the problems – a ‘fix’ is to make hardware improvements to solves many of the underlying issues, and to allow a much greater suite of tools to be used simultaneously to find ‘abusers’

We are therefore going to be migrating to new drives to increase the IO capabilities, add more ram to upgrade caching for overall performance benefits, and some reconfiguration – so that whilst further improvements, checks, logging and investigation go on, services continue to operate for our clients at the usual high-availabilty ultra-fast levels we aim to always provide.

updates

2014-11-02 10:00 – Server returned to service with staff access only
2014-11-02 10:30 – All Email Services returned to service – imap/pop3/smtp – exc. webmail
2014-11-02-12:00 – All static websites returned to service
2014-11-02 13:00 – Webmail returned to service
2014-11-02 13:30 – All MySQL DB services returned to service after full mysqlcheck

Sep 11

COMPLETED: Planned Maintenance: cPanel Hosting Servers

Planned Maintenance: cPanel Hosting Servers

Upgrades to latest control panel and standardisation of mailserver configurations

What is being done:

Upgrade of cPanel/WHM to latest version – fixes bugs with …
* under-reported quotas
* mailman mailing list urls/ownership
* missing disk/bw stats
* standardisation of imap mailserver configurations

Who is affected:

Clients with Linux Hosting Services (web, email, personal, business, enterprise and reseller packages) on servers:

  • cPanel01
  • cPanel02
  • cPanel03
  • cPanel04
  • cPanel05
  • cPanel06
  • cPanel07
  • cPanel08
  • cPanel09
  • cPanel10
  • cPanel11
  • all cPanel/WHM servers from acquisitions

The servers will remain on/up during the upgrades, installations and updates.

During the maintenance window you will not be able to upgrade/downgrade your hosting accounts through the Client Portal or directly in WHM) or check storage quotas.

The servers will need to have all services stopped and restarted at the end of their upgrade cycle – this will involve a maximum of 15 minutes of outage of all services (mail, web, databases etc) on a server by server basis.
We are unable to provide an exact time of the 15 minutes of disruption as each server will take a variable amount of time to update depending on the number of emails/ftp users/website visitors active during the day.

Clients with IMAP folders that are syncronised to local mail programs (like outlook) will need your mail program to collect the complete list of emails again (this should happen seamlessly when the mail service on your server restarts) and takes an average of 3 minutes per mailbox.

There are bugs in some email clients which following a re-sync incorrectly report all previously read emails as ‘new’ mails – we apologise in advance if this happens, but it’s a ‘feature’ of your local email program (instead of the mailservers that we control) so beyond our ability to stop or influence.

When is this maintenance:

Starting Saturday 20 September 2014 12:00
Completing Saturday 20 September 2014 22:00

Why do we need to do this:

Increasing the anti-spam and anti-virus capabilities.
Fixing some bugs which have been reported by clients with the control panel.
Standardising configurations to make maintenance more automated and less disruptive in future.
Applying latest linux, cpanel/whm, apache, ftp and mysql security patches.

Sep 09

COMPLETED: Planned Maintenance: Media Hosting Servers – Replacing backup target drives

Planned Maintenance: Media Hosting Servers

Replacing backup target drives

What is being done:

Replacing backup target drives.

Who is affected:

Clients with Media Hosting Services (video hosting, ffmeg hosting, radio station hosting, shoutcast streaming) on servers:

  • media1
  • media2
  • cPanel07
  • legacy server cheapffmeg

The servers will be unavailable for around 15 minutes during the maintenance window.

When is this maintenance:

Starting Friday 12 September 2014 18:00
Completing Friday 12 September 2014 22:00

Why do we need to do this:

Increasing available storage for backups.

Aug 12

COMPLETED: Emergency Maintenance – W2K8DNP1

Emergency Maintenance: Disk Check
What: Following a forced reboot we are running a disk check.
Affects: All services provided by W2K8DNP1 server.
When: Tuesday, 12th August 2014 19:45
All hosting (web/email) services hosted on W2K8DNP1 server will be unavailable whilst the disk check is running.

Jul 25

COMPLETED: Upgrades to CPanel/WHM Shared Hosting Servers

All CPanel/WHM Linux Shared Hosting Servers have been updated to the latest

  • CPanel/WHm Control Panel 11.44.0.x
  • Softaculous Script Installer
  • Astutium Website Builder – rvsitebuilder

The cpanel/whm updates and patches include a number of security fixes and bug fixes plus performance enhancements to the latest version of PHP

The softaculous upgrade brings the ability to install the lastest versions:

  1. WordPress
  2. CSCart
  3. Drupal
  4. GeekLog
  5. Joomla
  6. Magento
  7. Moodle
  8. MyFAQ
  9. Plogger
  10. Prestashop
  11. Tiki
  12. and many others !

The rvsitebuilder upgrade adds an additional 40 user selectable templates with 120+ different style/colour combinations, as well as performance improvements to the speed of publishing new site-edits

Jun 18

COMPLETED: Emergency Maintenance – HSphere/Win6

Emergency Maintenance: Win6
What: Software/Failure
Affects: HSphere Web Hosting Services Provided By Win6
When: Wednesday 18th June 2014 10:00 – ongoing

Unfortunately thorugh several security issues, both with HSphere itself and customer applications, web hosting services provided by this server cannot be immediately recovered.  As such we are accelerating our plans to migrate customers to more appropriate platforms as soon as possible.  HSphere is no longer appropriately supported by its authors and we have been continuing to support it for as long as we have been able to.

This process will unfortunately involve some downtime for the affected accounts.

May 23

Notification: Domain Transfer Emails to SBH Registrants

If you have today (23/May/2014) received unexpected emails about Domain Name Transfers being in progress, please accept my apologies for any concern/worry caused.

An internal tidy-up process of putting all the domains from recent Mergers & Acquisitions onto our automated systems and to allow us to reduce your price for domain renewals has triggered these emails, which can all be safely ignored.

Some of your domain(s) in the Astutium Client Portal will temporarily show as ‘Pending Transfer’ and then change back a few minutes later to ‘Active’ once the processes complete.

As soon as that happens then changes to your domain (nameservers, renewals etc) will all be fully automated, near-instant (not all registries are real-time) and processed while-you-wait online giving you an improved service level.

If you have any specific questions, please open a support ticket through the client portal, and once again, my apologies for any alarm caused.

Rob Golding
Astutium Ltd