[15/02/11] Venus unavailable

NetHosted Staff Joined: 24 Jun 2005 Posts: 1013

Hi,

Venus is currently being restarted, due to a possible raid issue this is taking longer than usual, we are working to get this back up and running ASAP.

Regards,

Darryl

NetHosted Staff Joined: 24 Jun 2005 Posts: 1013

Venus is currently running a file system integrity check and should be back up once this finishes.

Regards,

Darryl

NetHosted Staff Joined: 24 Jun 2005 Posts: 1013

The file integrity check is not running as expected unfortunately, we are working on alternative methods to get the server back up and running.

Regards,

Darryl

NetHosted Staff Joined: 24 Jun 2005 Posts: 1013

The integrity check is currently running in rescue mode and appears to be progressing correctly, we are unsure as to how long this will take to complete at present but wanted to provide an update.

Regards,

Darryl

NetHosted Staff Joined: 24 Jun 2005 Posts: 1013

Venus is back up and running, there may be some unexpected side-effects of the file system repairs, we are verifying many things manually now. If you notice any issues that did not exist prior to this please submit a support ticket and we will investigate.

Regards,

Darryl

NetHosted Staff Joined: 24 Jun 2005 Posts: 1013

We will be posting an incident report in this thread once we have had time to analyse the cause of this, currently the priority is to ensure the server is running correctly and stable, once this has been fully established we will be investigating the cause of this, thanks for your understanding.

Regards,

Darryl

NetHosted Staff Joined: 22 Mar 2004 Posts: 7017

Hi,

Here is the full incident report of last evening's events:

I was personally working in the datacentre when this happened. Upon attaching a console to Venus I could see numerous file system errors and took the decision to power off the server. Upon starting up again the RAID card had problems detecting the drives in place. Once this issue was fixed the operating system did not successfully complete it's standard file system integrity checks so I mounted the disks from a live CD and manually forced a "deep" file system inspection. This is what caused the bulk of the downtime that was seen as this process takes a long time to complete.

After this deep inspection the system was bootable and only required minor fixes to bring back all services. Since then the server has been running as expected and we believe this issue to be fully resolved.

Apologies to anyone that was affected by this unscheduled downtime.

Thanks,

Andrew

Client Support	Community	Server Status	Contact Us	Client Login
Email Hosting	Website Hosting	Reseller Hosting	VPS Hosting	Dedicated Servers