Nov 01

COMPLETED: Emergency Maintenance – CPanel11

Emergency Maintenance: cPanel11 Hosting Server

Replacement Drives, Migration of Data and Hardware Upgrades

What is being done:

Replacement of SSD Hard Drives and Migration of all existing data to the new disks, in order to stabilise an intermittent Disk-IO/Load issue, addition of more RAM and reconfiguration of caching, intrusion-detection systems, extra firewalling and other tweaks.

Who is affected:

Clients with cPanel/WHM Linux Hosting Services (web, email, personal, business hosting packages) on server:

  • cPanel11

The server will need to be completely offline during the maintenance to physically swap hard-drives, and restore all data as quickly as possible.

During the maintenance window you will not be able to access any aspect of your hosting accounts through the Client Portal, via WHM/cPanel or by http/ftp/pop3/smtp/imap etc

The servers will need to be powered down, have all drives removed, duplicated and replaced, and other hardware changed before bringing up with limited access whilst services are started in a controlled manner and the reported disk-io and load investigations are completed.

This will involve a maximum of 12 hours of outage of all services (mail, web, databases etc) on server cp11 and some limited access for an additional 48 hours whilst the impact and improvements are monitored and tweaked.

We are unable to provide an exact time for you to regain any access to your hosting account as it will depend on the type of services you use – email only will be back first, followed by ftp, web, databases – if any ‘prioritisation’ of clients is necessary, then those with the advanced-support levels will be 1st, followed by Enterprise & E-Commerce clients, then Business, Personal and finally any trial and subsidised/discounted/internal services.

Incoming email during this time will ‘queue’ at the sender, Outbound emails will stay on your local machine (where using a mail client) – webmail access will be unavailable until the work is completed.

Websites will be down until the data has been restored, Websites which rely on a database will start to work once the mysql databases are restored and repaired.

FTP upload/download access will return once we have confirmed all data is available.

Access to other services/ports where supported will then be made available again.

When is this maintenance:

Starting Saturday 01 November 2014 22:00
Completing Monday 03 November 2014 10:00

Why do we need to do this:

Clients are reporting issues with ‘load’ related problems with MySQL DB driven websites and intermittent mail access at some times.

Investigations into these issues have found a number of causes, some which have been mitigated to the extent they can be, by a variety of means:
– attempts to ddos certain clients
– client setup issues triggering the IDS
– customers with ‘service abusing’ processes
– use of technologies banned from our shared hosting service
etc

However there appears to be some which will require more advanced analysis, which itself is increasing the load and IO, so inadvertently extending the problems – a ‘fix’ is to make hardware improvements to solves many of the underlying issues, and to allow a much greater suite of tools to be used simultaneously to find ‘abusers’

We are therefore going to be migrating to new drives to increase the IO capabilities, add more ram to upgrade caching for overall performance benefits, and some reconfiguration – so that whilst further improvements, checks, logging and investigation go on, services continue to operate for our clients at the usual high-availabilty ultra-fast levels we aim to always provide.

updates

2014-11-02 10:00 – Server returned to service with staff access only
2014-11-02 10:30 – All Email Services returned to service – imap/pop3/smtp – exc. webmail
2014-11-02-12:00 – All static websites returned to service
2014-11-02 13:00 – Webmail returned to service
2014-11-02 13:30 – All MySQL DB services returned to service after full mysqlcheck

Sep 11

COMPLETED: Planned Maintenance: cPanel Hosting Servers

Planned Maintenance: cPanel Hosting Servers

Upgrades to latest control panel and standardisation of mailserver configurations

What is being done:

Upgrade of cPanel/WHM to latest version – fixes bugs with …
* under-reported quotas
* mailman mailing list urls/ownership
* missing disk/bw stats
* standardisation of imap mailserver configurations

Who is affected:

Clients with Linux Hosting Services (web, email, personal, business, enterprise and reseller packages) on servers:

  • cPanel01
  • cPanel02
  • cPanel03
  • cPanel04
  • cPanel05
  • cPanel06
  • cPanel07
  • cPanel08
  • cPanel09
  • cPanel10
  • cPanel11
  • all cPanel/WHM servers from acquisitions

The servers will remain on/up during the upgrades, installations and updates.

During the maintenance window you will not be able to upgrade/downgrade your hosting accounts through the Client Portal or directly in WHM) or check storage quotas.

The servers will need to have all services stopped and restarted at the end of their upgrade cycle – this will involve a maximum of 15 minutes of outage of all services (mail, web, databases etc) on a server by server basis.
We are unable to provide an exact time of the 15 minutes of disruption as each server will take a variable amount of time to update depending on the number of emails/ftp users/website visitors active during the day.

Clients with IMAP folders that are syncronised to local mail programs (like outlook) will need your mail program to collect the complete list of emails again (this should happen seamlessly when the mail service on your server restarts) and takes an average of 3 minutes per mailbox.

There are bugs in some email clients which following a re-sync incorrectly report all previously read emails as ‘new’ mails – we apologise in advance if this happens, but it’s a ‘feature’ of your local email program (instead of the mailservers that we control) so beyond our ability to stop or influence.

When is this maintenance:

Starting Saturday 20 September 2014 12:00
Completing Saturday 20 September 2014 22:00

Why do we need to do this:

Increasing the anti-spam and anti-virus capabilities.
Fixing some bugs which have been reported by clients with the control panel.
Standardising configurations to make maintenance more automated and less disruptive in future.
Applying latest linux, cpanel/whm, apache, ftp and mysql security patches.

Sep 09

COMPLETED: Planned Maintenance: Media Hosting Servers – Replacing backup target drives

Planned Maintenance: Media Hosting Servers

Replacing backup target drives

What is being done:

Replacing backup target drives.

Who is affected:

Clients with Media Hosting Services (video hosting, ffmeg hosting, radio station hosting, shoutcast streaming) on servers:

  • media1
  • media2
  • cPanel07
  • legacy server cheapffmeg

The servers will be unavailable for around 15 minutes during the maintenance window.

When is this maintenance:

Starting Friday 12 September 2014 18:00
Completing Friday 12 September 2014 22:00

Why do we need to do this:

Increasing available storage for backups.

Aug 12

COMPLETED: Emergency Maintenance – W2K8DNP1

Emergency Maintenance: Disk Check
What: Following a forced reboot we are running a disk check.
Affects: All services provided by W2K8DNP1 server.
When: Tuesday, 12th August 2014 19:45
All hosting (web/email) services hosted on W2K8DNP1 server will be unavailable whilst the disk check is running.

Jul 25

COMPLETED: Upgrades to CPanel/WHM Shared Hosting Servers

All CPanel/WHM Linux Shared Hosting Servers have been updated to the latest

  • CPanel/WHm Control Panel 11.44.0.x
  • Softaculous Script Installer
  • Astutium Website Builder – rvsitebuilder

The cpanel/whm updates and patches include a number of security fixes and bug fixes plus performance enhancements to the latest version of PHP

The softaculous upgrade brings the ability to install the lastest versions:

  1. WordPress
  2. CSCart
  3. Drupal
  4. GeekLog
  5. Joomla
  6. Magento
  7. Moodle
  8. MyFAQ
  9. Plogger
  10. Prestashop
  11. Tiki
  12. and many others !

The rvsitebuilder upgrade adds an additional 40 user selectable templates with 120+ different style/colour combinations, as well as performance improvements to the speed of publishing new site-edits

Jun 18

COMPLETED: Emergency Maintenance – HSphere/Win6

Emergency Maintenance: Win6
What: Software/Failure
Affects: HSphere Web Hosting Services Provided By Win6
When: Wednesday 18th June 2014 10:00 – ongoing

Unfortunately thorugh several security issues, both with HSphere itself and customer applications, web hosting services provided by this server cannot be immediately recovered.  As such we are accelerating our plans to migrate customers to more appropriate platforms as soon as possible.  HSphere is no longer appropriately supported by its authors and we have been continuing to support it for as long as we have been able to.

This process will unfortunately involve some downtime for the affected accounts.

May 23

Notification: Domain Transfer Emails to SBH Registrants

If you have today (23/May/2014) received unexpected emails about Domain Name Transfers being in progress, please accept my apologies for any concern/worry caused.

An internal tidy-up process of putting all the domains from recent Mergers & Acquisitions onto our automated systems and to allow us to reduce your price for domain renewals has triggered these emails, which can all be safely ignored.

Some of your domain(s) in the Astutium Client Portal will temporarily show as ‘Pending Transfer’ and then change back a few minutes later to ‘Active’ once the processes complete.

As soon as that happens then changes to your domain (nameservers, renewals etc) will all be fully automated, near-instant (not all registries are real-time) and processed while-you-wait online giving you an improved service level.

If you have any specific questions, please open a support ticket through the client portal, and once again, my apologies for any alarm caused.

Rob Golding
Astutium Ltd

May 12

COMPLETED: Maintenance – Fibre Migration / Recabling GS2-THN

Planned Maintenance: Fibre Migration / Recabling GS2-THN

What: Migration of Fibre Routes Global Switch 2 < - > TeleHouse North

Affects: All Equipment in Global Switch 2

When: Sunday, 1825th May 2014 08:00 – 16:00

In order to allow for expansion within our GS2 DataCentre Facilities, the fibre-routes between GS2 and THE are being replaced/recabled and reconfigured.

No client services should be affected as we have multiple diverse routes – although the primary docklands fibre ring will be considered ‘at-risk’ during these works.

Astutium technicians will be onsite during the entire process to ensure minimal impact to the network and to test the new cabling structures meet our very high quality standards.

May 12

COMPLETED: Maintenance – Fibre Migration / Recabling GS2-THE

Planned Maintenance: Fibre Migration / Recabling GS2-THE

What: Migration of Fibre Routes Global Switch 2 < - > TeleHouse East

Affects: All Equipment in Global Switch 2

When: Saturday, 1724th May 2014 08:00 – 16:00

In order to allow for expansion within our GS2 DataCentre Facilities, the fibre-routes between GS2 and THE are being replaced/recabled and reconfigured.

No client services should be affected as we have multiple diverse routes – although the primary docklands fibre ring will be considered ‘at-risk’ during these works.

Astutium technicians will be onsite during the entire process to ensure minimal impact to the network and to test the new cabling structures meet our very high quality standards.

May 07

COMPLETED: Maintenance – Domain Registration : [*.uk]

At 07:00 (BST) on Tuesday 10 June 2014 Nominet will have a systems outage lasting about one hour when they will be deploying a software upgrade. During this time all .uk registration systems will be unavailable. The Automaton will not process email requests, but they will be received and queued for processing when it restarts.