Network interruption and system stop

Multiple nodes and instances stopped.
11 November 2020 by
Network interruption and system stop
Stephen Schmidt
| No comments yet

Update on network interruptions 11 Nov 2020

At approx 17:03 we experienced a network failure that affected multiple nodes in the Cloudworkz network.   As a result multiple customer instances were unavailable via internet.  It took some time to determine the cause of the failure and to identify which systems were affected. 

To resolve the issue we were forced to shut some nodes temporarily and reallocate IP addresses.  This is a time-consuming process and some instances were not available until approximately 22:15.

We apologize for the interruption in service.  We will now analyse what steps can be taken to reduce the risk of this kind of failure in the future.

As part of our service to customers, we are implementing an SMS notification service. This is a new service and we note that in some cases we do not have correct or appropriate telephone numbers for notification to system owners.  In the coming days, you may be contacted to confirm your preferred contact details in the event of failure.

The failure was limited to network connections and no servers failed and no databases were affected.  However, some systems were unavailable for a longer period and some instances required a restart and or were allocated new IP addresses and resources.  We do not expect any system to be adversely affected by the stop.  

If you have any questions or concerns do not hesitate to contact your responsible Application Consultant or Salesperson for more information.

Additional information will be available on a separate news page shortly. 

Network interruption and system stop
Stephen Schmidt 11 November 2020
Share this post
Archive
Sign in to leave a comment