Jul 04

COMPLETED: Emergency Maintenance: OVZBudget02

What: Urgent Replacement of : ovz-budget-02.vps.astutium.com
Load based problems causing VPS Host Node random restarts on the ovz-budget-02 server
This host/node has suffered 3 issues in recent weeks, and again today is under extreme load.

We are therefore performing an emergency migration of all Active VPS to new hardware to prevent further servcie interruptions whilst the device is investigated

Affects:
All customers with Budget OVZ virtual machines on ovz-budget-02.vps.astutium.com

When:
Ongoing

Migrations are in progress – ETA on ALL VPS returned to service is 06:00 5th July 2016

We apologise for the inconvenience caused, and the short notice, but due to the nature of the problem, replacement rather than wait-for-fail has been determined the best course of action for clients affected

Update 05:50 5th July – 50% of the migrations have completed and all VPS are back online, due to clients needing access to their virtual machines, we will compete the migration to new hardware overnight on 5th, 6th and 7th July

Update 17:00 6th July – over 80% of VPS have been successfully moved to new host-node hardware with no loss of data and minimal overall impact. Migrations are continuing at off-peak access times – If you’d like to be transferred immediately, please open a ticket.

Jun 11

COMPLETED: Emergency Maintenance – Plesk 8-9

Outage: plesk8-9.hosting.astutium.com

What: Unable to access services running on Plesk Hosting Server 8-9

Affects: All customers with Plesk based shared hosting in plsk8 or plsk9 machines

When: Sat 11/June/2016 9:15am

Plesk hosting platform became unresponsive at approx 9.15am this morning.

Support Staff attempted to restart the server remotely but access was not restored.

DataCentre Technicians despatched to site to investigate further.

This issue is under active investigation, initial findings indicate complete failure of the hardware, so it will need to be replaced and data restored to a new system

We apologise for the inconvenience caused, and assure all affected clients that we are working on solving the issue.

Update 1 (2016-06-11 11:45)
The failed server could not be repaired and has been replaced with a new machine, data recovery is in progress, this is expected to complete around 14:00 today

May 31

COMPLETED: Emergency Maintenance: OVZBudget03

Outage: ovz-budget-03.vps.astutium.com

What:
Intermittent connectivity issues accessing Virtual Private Servers on the ovz-budget-03 server
caused by excessive CPU Load and Data-Transfer levels

Affects:
All customers with OVZ virtual machines on ovz-budget-03.vps.astutium.com.

When:
Ongoing

This issue is under investigation, initial findings indicate one-or-more exploited virtual servers causing connectivity problems to other virtual machines on the same host-node

We apologise for the inconvenience caused, and assure all affected clients that we are working on isolating the errant virtual machines in order to returning the vps host server to operational status

Update 2016-05-31 13:43
The problematic processes have all been stopped and removed from the server, all client VPS have been restarted and returned to service.

Load is still higher than usual (due to the ongoing RAID rebuilds) but CPU/RAM usage and Connectivity has returned to normal levels

We will continue to monitor this server for unusual traffic & cpu patterns

May 29

COMPLETED: Emergency Maintenance: OVZBudget03

Outage: ovz-budget-03.vps.astutium.com

What:
The issue is currently undergoing active investigation.
See updates further down.

Affects:
All customers with OVZ virtual machines on ovz-budget-03.vps.astutium.com.

When:
This condition was detected at 07:50 2016/05/29.

We do not yet have an ETA for service restoration but this post will be updated as further information is available.

Update 2016-05-29 12:00

The RAID-10 SSHD Disk Array has failed drives, which are currently being replaced and rebuilt
Server will be in a degraded state for approximately 72 hours during the primary rebuild:
* performance will be slower than usual
* the server (and all VMs) will require at least 2 reboots during this time

Update 2016-06-01 11:31

We will shortly need to restart the machine to pickup the newly installed and initialised drives, virtual servers will be gracefully shut-down and after the reboot has completed, started again

We apologise for the inconvenience caused, and assure all affected clients that we are working on returning the vps host server to an optimal state and performance as soon as possible.

Apr 12

COMPLETED: cPanel Shared Hosting Services

Planned Maintenance: cPanel Shared Hosting Services

Control panel and operating system updates.

What is being done:

* update to new version of cpanel
* reboot to use new kernel
* upgrade anti-virus software

Who is affected:

All Shared Hosting, Reseller Hosting, Media Hosting, ShoutCAST Hosting, File Hosting, Email Hosting, WordPress Hosting and Ecommerce Hosting services on shared servers running:

  • cPanel/WHM

Incoming email during this time will ‘queue’ at the sender, Outbound emails will stay on your local machine (where using a mail client) – webmail access will be unavailable until the work is completed.

Websites will be unavailable until the server has completely restarted following the upgrades and patches.

FTP upload/download access will return once we have confirmed all data is available.

Streaming services will automatically resume to their previous state as soon as the maintenance on the specific machine has completed.

When is this maintenance:

Starting: Sunday 17th April 12:00
Completing: Sunday 17th April 18:00

Why do we need to do this:

It is important for the safety and security of your data that each of the server(s) providing your hosting has the most current security updates, operating system patches and control panel improvements.

Feb 01

COMPLETED: Planned Maintenance: Astutium Website and Client Portal

Planned Maintenance: Astutium Website and Client Portal

Rollout of new features and fixes

When is this maintenance:

Starting Monday 1st February 2016 10:00
Completing Monday 1st February 2016 18:00

What is being done:

* Upgrade to latest versions of Operating System
* Security enhancements to Client Portal
* Changes to new version of Smarty & PHP

Who is affected:

All visitors to our website and client portal

End-user services are not affected in any way

Our website server will need to be taken offline for a few short periods during the maintenance to update php, apache, mysql etc

During the maintenance window you will temporarily not be able to access the client portal to make changes to domains, hosting, vps etc

Why do we need to do this:

It is important for the safety and security of your data that we continually improve security to our services.
Additionally new features and enhancements to our client portal require a migration to a new OS and PHP version.

Dec 02

COMPLETED:Emergency Maintanence – UK1.mcftserver.com

Outage: UK1.mcftserver.com

What:
All Minecraft servers on UK1.mcftserver.com are currently unavailable due to a hardware failure.

Affects:
All customers with Minecraft servers on UK1.mcftserver.com.

When:
This condition was detected at 21:50 2015/12/02.

We do not yet have an ETA for service restoration but this post will be updated as further information is available.

Update: December 2nd

The hardware issue has been resolved and the minecraft servers are back online, if you still have any issues with your minecraft server please create a support ticket.

Dec 01

COMPLETED: Planned Maintenance: HSPhere Hosting Services

Planned Maintenance: HSphere Hosting Services

Patches for OS and Replacement of Hardware

When is this maintenance:

Starting Thursday 3rd December 2015 18:00
Completing Monday 7th December 2015 06:00

What is being done:

* Upgrade to latest versions of Operating Systems where supported
* Additional RAM to motherboard maximum supported levels for all shared-hosting services
* Standardisation of hosting server specifications across all platforms to 8 CPU Cores | 32GB Ram | SSD Disks

Who is affected:

All Shared Hosting, Reseller Hosting, Email Hosting on legacy H-Sphere Clusters

Dedicated Servers, Virtual Private Servers, Colocated Servers etc are not affected

Each server will need to be taken offline for a short period during the maintenance to execute a physical to virtual conversion of the operating system for deployment onto new servers with more ram and faster (SSD) drives.

Due to the cluster nature of hsphere, whilst each individual server outage is expected to be around 2 hours, they are interdependant, and so mail/web/etc will not work while the control panel or sql servers are updated – any shutdown/restart/reboot/virtualisation events needing to be done are planned for between 8pm and 8am

During the maintenance window you will temporarily not be able to access aspects of your hosting accounts through the Client Portal, via Control Panel or by http/ftp/pop3/smtp/imap etc

Incoming email during this time will ‘queue’ at the sender, Outbound emails will stay on your local machine (where using a mail client) – webmail access will be unavailable until the work is completed.

Websites will be unavailable until the server has completely restarted following the upgrades and patches.

FTP upload/download access will return once we have confirmed all data is available.

Why do we need to do this:

It is important for the safety and security of your data that each of the server(s) providing your hosting has the most current security updates, operating system patches and control panel improvements. The constant attacks of all internet-connected machines means an ongoing security update is needed, and pre-emptive server replacement prevents overall downtime due to failed components

Additionally by bringing all the shared hosting servers up-to the same level of specification (following our recent switch to ALL-SSD based hosting for performance improvements) it makes keeping spares more manageable, as well as reducing “fix-times” should there be any hardware failures.

Nov 19

COMPLETED: Planned Maintenance: Shared Hosting Services

Planned Maintenance: Shared Hosting Services

Patches for OS / Control Panels and Upgrades to Hardware

What is being done:

* Upgrade to latest versions of Operating Systems, Control Panels, Script Installers and System Addons
* Additional RAM to motherboard maximum supported levels for all shared-hosting services
* Standardisation of hosting server specifications across all platforms to 8 CPU Cores | 32GB Ram | SSD Disks

Who is affected:

All Shared Hosting, Reseller Hosting, Media Hosting, ShoutCAST Hosting, File Hosting, Email Hosting, WordPress Hosting and Ecommerce Hosting services on shared servers running:

  • cPanel/WHM
  • DirectAdmin
  • DotNet/Website Panel
  • Legacy Ensim
  • Legacy Ensim Windows
  • Legacy Plesk/Parallels/Odin
  • Legacy HSphere

Dedicated Servers, Virtual Private Servers, Colocated Servers etc are not affected. If you have a Management Service for your server with Astutium, you will be contacted directly regarding a timeslot for your system to be updated and restarted.

Each server will need to be taken offline for a short period during the maintenance to physically add additional RAM (where supported) and then to be restarted with the latest security patches to operating system, control panel and related items.

Whilst the individual server outage is expected to be around 1 hour, they will be done in stages over the maintenance period, with any shutdown/restart/reboot events done planned for between 8pm and 8am

During the maintenance window you will temporarily not be able to access aspects of your hosting accounts through the Client Portal, via Control Panel or by http/ftp/pop3/smtp/imap etc

We are unable to provide an exact time for you as it will depend on the type of services you use, the amount of work needed to be completed on each machine, and the specification of each server (average boot-time of a shared hosting server from ‘off’ is 18 minutes for it to count+test the memory and other devices, load the operating system and start serving clients)

The basic procedure will be:
* Stop all services and shutdown machine – 10 minutes
* Pull from rack and put on workbench – 10 minutes
* Upgrade hardware and test – 20 minutes
* Apply patches and updates – 10 minutes
* Return to rack and power on – 10 minutes
* Check all services started ok – 20 minutes

Incoming email during this time will ‘queue’ at the sender, Outbound emails will stay on your local machine (where using a mail client) – webmail access will be unavailable until the work is completed.

Websites will be unavailable until the server has completely restarted following the upgrades and patches.

FTP upload/download access will return once we have confirmed all data is available.

Streaming services will automatically resume to their previous state as soon as the maintenance on the specific machine has completed.

When is this maintenance:

Starting Thursday 26 November 2015 18:00
Completing Monday 30 November 2015 06:00

*EDIT* work was postponed 1 week from original dates published due to late delivery of parts

Why do we need to do this:

It is important for the safety and security of your data that each of the server(s) providing your hosting has the most current security updates, operating system patches and control panel improvements. A number of our control panel providers have released major new versions in recent weeks following security audits, and those changes need to urgently be applied to protect your websites.

Additionally by bringing all the shared hosting servers up-to the same level of specification (following our recent switch to ALL-SSD based hosting for performance improvements) it makes keeping spares more manageable, as well as reducing “fix-times” should there be any hardware failures.

Nov 13

COMPLETED: Emergency Maintenance: Win6/DC Hosting

Outage: Win6 (DC Hosting) due to RAID Array Degredation

What:
All customer websites served from Win6 are currently unavailable.  Mail and other servers are still available.

Affects:
All customers with websites served from Win6 (DC Hosting).

Please note; this does not affect any part of the Astutium Network, or the MiniVPS Maidenhead or Manchester based services.

When:
This condition was detected at 07:35 2015/11/13.
We do not yet have an ETA for service restoration, disk maintenance cannot provide an estimate.

Update Nov 14
All affected client sites have been migrated to a new server and all services restored to full operational status