Order VPS Hosting
Order a VPS, Semi- dedicated or Dedicated server in Dallas, London or Australia.

Get Assistance
Ask our support team about your hosting requirements.


Host where the staff takes pride in making customers happy

I really appreciate your patience and effort. It seems like you created a company that would treat unix nerds the way we always wanted to be treated, so I'm happy to be a customer.

- Lawrence (#228/298)
Home > Support > Maintenance Notices

Related Links

Maintenance Notice Links:

Server Maintenance Notices

host102.rimuhosting.com Restart

PostedWednesday, July 1, 2009 11:37:11 PM UTC
Last UpdateFriday, July 3, 2009 2:25:00 AM UTC (89 minutes ago)
StatusCurrent/Ongoing
Affected Servershost102.rimuhosting.com, host119.rimuhosting.com

host102.rimuhosting.com is showing some unusual output in the kernel logs, and customer VPS management there is currently offline.

We are preping a replacement server with a more recent version of Xen and LVM. Once that is ready we will schedule a migration of the all VPSs.

Update 0050 UTC: Replacement host server is ready. We will start moving customers to the new machine around 0230am UTC.

Update 0230 UTC: Starting on the migration now.

#

Dallas Network Issue

PostedWednesday, July 1, 2009 2:13:44 PM UTC
Last UpdateWednesday, July 1, 2009 10:39:18 PM UTC (29 hours ago)
StatusCurrent/Ongoing
Affected Data CenterDallas (TX)

Summary

There was a power outage affecting some servers (about 20%) at the Dallas data center.  The power came back on quickly, but one switch did not work correctly after the issue causing one cabinet of servers to be inaccessible.  That switch/cabinet issue is now resolved.

We are working on collecting more information about what happened and will post that information here as soon as we have it.

We note that Rackspace (also in Dallas) had a power outage yesterday.  This outage appears completely unrelated.  Other than the fact that providing reliable power to buildings filled with power hungry servers is a technically challenging task.

We sincerely apologize for any inconvenience caused to affected customers in Dallas.  We work hard to provide a reliable service and part of that is to be very careful choosing the data centers we host our servers with.  We note that this is the first power issue we have experienced in the 5 years we have been at this data center.  We are confident that they are taking the outage seriously and will do all they can to avoid issues in the future.

Initial data center report

The UPS failed.  Right now all parts are here for the UPS and they are installed.  They will be bringing it up shortly.  The UPS vendors feel everything is fine, but when you switch load back to the UPS you always have a slight, very slight, chance that there will be an issue.

That UPS has worked flawlessly for 6 years and had all maintenance performed as recommended by the vendor.  The UPS failed.  And the UPS failure caused power interruption for about 10 minutes, then we had to turn PDUs back on. This far exceeds the expectation for all UPS manufacturers.  To totally avoid the chance that it can happen again is to get redundant feeds for all power feeds off different UPSs and PDUs. We already have different Generator and Services at the entrance.

Detail

We've detected a networking issue in the Dallas datacenter.  Multiple servers are affected and we're investigating the problem right now.  Updates are forthcoming.

@Update @1420UTC: It seems to be an electrical problem at the Dallas data center. We're expecting more updates soon.

@Update @1433UTC: We are starting to see some of the host servers in Dallas booting up.

@Update @1455UTC: The power is completely back up at the data center. The technicians are cleaning up after the networking issues left behind.

@Update @1710UTC: It appears there is one cabinet/switch that is not accessible.  We believe it is a switch configuration issue.  And we are working on resolving that.

@Update @1845UTC: The switch issue is resolved.  There are a handful of serves still not pinging and we are investigating these individually.

Affected host servers

The following servers were affected (does not included dedicated servers, just VPS host servers)

host32.rimuhosting.com
host102.rimuhosting.com
host103.rimuhosting.com
host104.rimuhosting.com
host109.rimuhosting.com
host111.rimuhosting.com
host112.rimuhosting.com
host113.rimuhosting.com
host114.rimuhosting.com
host117.rimuhosting.com
host118.rimuhosting.com
host121.rimuhosting.com
host125.rimuhosting.com
host130.rimuhosting.com
host131.rimuhosting.com
host132.rimuhosting.com
host133.rimuhosting.com
host134.rimuhosting.com
host173.rimuhosting.com
host186.rimuhosting.com
host193.rimuhosting.com
host531.rimuhosting.com
host534.rimuhosting.com
host556.rimuhosting.com
host557.rimuhosting.com
host716.rimuhosting.com
host721.rimuhosting.com

#

Network Problems with Host596

PostedThursday, July 2, 2009 3:42:54 PM UTC
Last UpdateThursday, July 2, 2009 3:50:20 PM UTC (12 hours ago)
StatusMarked as resolved
Affected Serverhost596.rimuhosting.com

We are seeing heavy packet loss to host596. We are currently investigating this and will post updates here.

UPDATE: It turned out to be a DDOS attack on the network. It should be under control and your VPSs should all be accessible now.

#

Dallas Network Issue (July 1, 0200UTC)

PostedWednesday, July 1, 2009 2:16:09 AM UTC
Last UpdateWednesday, July 1, 2009 2:33:25 AM UTC (49 hours ago)
StatusMarked as resolved

Our monitoring has detected that some machines behind a particular switch are not responding. We are investigating.

Update: The switch was having trouble negotiating a trunk, causing loss of connectivity to the machines attached to it. This has been fixed and all machines are pinging.

#

host44.rimuhosting.com Restart

PostedWednesday, July 1, 2009 2:16:42 AM UTC
Last UpdateWednesday, July 1, 2009 2:16:42 AM UTC (50 hours ago)
StatusMarked as resolved
Affected Serverhost44.rimuhosting.com

host44.rimuhosting.com restarted itself.  We are investigating.

Update: Server is up and VPSs are all running. Seeing a disk repair completed on startup and raid array resyncing. We will continue to monitor the server health.

#

host103.rimuhosting.com Restart

PostedTuesday, June 30, 2009 5:57:10 AM UTC
Last UpdateTuesday, June 30, 2009 6:58:31 AM UTC (69 hours ago)
StatusMarked as resolved
Affected Serverhost103.rimuhosting.com

host103.rimuhosting.com is showing a device issue on it backup drive. We have scheduled a restart for this at 615am UTC to clear the issue.

Update 0635 UTC: Server is back up and VPSs are restarting.
Update 0650 UTC: All VPSs are back up and responding normally. We will continue to monitor the server.

#

host122.rimuhosting.com Restart

PostedSunday, June 28, 2009 11:47:37 PM UTC
Last UpdateMonday, June 29, 2009 1:18:15 AM UTC (99 hours ago)
StatusMarked as resolved
Affected Serverhost122.rimuhosting.com

host122.rimuhosting.com is not responding normally.  We are investigating and may have it restarted.

Update: The PSU appears to have failed in Host122. We are having it swapped out.
Update: It took longer than expected to get the PSU swapped (we had trouble finding a suitable spare). The swap has now been completed and VPSs are coming back up.
Update: All VPSs are back up and running.

#

Hosting

Why RimuHosting

RimuHosting