08/26/2023
There was a fiber cut, interfering with our ISP customers, that started at 9 am on 8/26/2023. Crews from multiple companies tested, located, excavated the ground, and repaired the fiber by laying and splicing thousands of feet of new fiber by 1:30am Sunday morning after a massive day and night undertaking. This fiber is what feeds our towers and delivers Internet to Compunet Internet customers.
UPDATE History:
** All sites report back online as of 1:30am Sunday Aug 27 **
If you unplugged any devices during this event, make sure to plug them back in to the same ports that they were originally plugged into.
During this outage:
It is okay to "Power Cycle" (Unplug and Plug back in) your equipment.
No NOT perform a factory reset of equipment (IE: do NOT use a pin/pen to press the reset button on any equipment), as you will lose your wifi name, and other settings that will set your equipment to default and your internet may not come back online when the fiber cut is repaired.
----------
Linked Outages: Sublette County 911 Service, Union Cellular, Verizon Cellular (Cell towers along Hwy 191), and landlines are being effected by fiber outage: https://www.facebook.com/SubletteEMA/ -- for updates
2023-08-27 06:40:45 GMT
The additional damage has been exposed and the slack cable pulled into place. The splicers have returned to the site with the additional trailer, and splicing is now in progress. An estimated time of restoral (ETR) of 09:00 GMT has been provided. As such, the next update will be provided at that time or as new information is received.
2023-08-27 05:31:06 GMT - The 1300 feet of cable has been successfully placed, and splicers are currently en route to obtain a second splice trailer for splicing to be performed on both ends of the span in tandem. Field Operations remain at the fault site and report that 3 separate points of damage have been identified. Currently, slack cable is being pulled from other handholes as excavation continues to expose the damaged cable.
2023-08-27 03:31:19 GMT - Crews suspect 6000 feet of cable may be damaged. Tests are ongoing at this time.
2023-08-27 03:06:26 GMT - Further cable tests suggest more extensive fiber damage than previously suspected. Tests are ongoing in the field to determine the extent of the fiber damage.
2023-08-27 02:10:56 GMT - The fiber has been found damaged at multiple points. As such, 1300 feet of cable will be pulled into place. Splicers are preparing the first end of the cable for splicing as the far end is being pulled into place.
2023-08-27 01:13:01 GMT - Crews are continuing to pull slack fiber into position to allow for repairs.
2023-08-27 00:12:24 GMT - Locates have been completed, and crews are excavating the fiber to pull slack into position for splicing.
2023-08-26 23:28:16 GMT - Crews are working to obtain an ETA for emergency locates.
2023-08-26 22:40:15 GMT - Crews have located broken fiber conduit in the area, sticking out of the ground. Emergency locates are being directed to that area.
2023-08-26 21:49:22 GMT - Field Operations report the construction crew has arrived on location. The team is awaiting emergency locates prior to excavation beginning.
2023-08-26 21:09:54 GMT - Field Operations are reaching out to the splice team for status of dispatch and estimated arrival time.
2023-08-26 20:17:12 GMT - Field Operations reports the fiber team has been engaged and is gathering equipment to dispatch to the location and investigate.
2023-08-26 20:09:13 GMT - Field Operations arrived at the suggested distance and reports the area has been plowed however, no fiber cables are visibly exposed. He has contacted his manager to get a fiber team dispatched to further investigate and physically locate the fault.
2023-08-26 19:09:49 GMT - Light level measurements indicate a break 8.67 miles out from the Rock Springs central office. Field Operations have dispatched to drive to the suggested area and further isolate the fault.
2023-08-26 18:56:20 GMT - Field Operations is engaged and are conducting light level measurements at this time.
2023-08-26 18:17:29 GMT - The dispatch center continues working to coordinate and authorize dispatch with the duty supervisor.
2023-08-26 17:25:18 GMT - The dispatch center continues working to contact the duty supervisor so that dispatch can be authorized and technicians assigned.
2023-08-26 16:41:36 GMT - The dispatch center is working to coordinate dispatch of Field Operations for light level measurements.
2023-08-26 15:46:14 GMT - Initial investigation suggest a fiber cut causing the service impact. As such Field Operations have been engaged to dispatch to the node in alarm. Upon arrival on-site fiber optic light measurements will be taken to identify the exact point of failure.
2023-08-26 15:37:32 GMT - On August 26, 2023 at 14:57 GMT, We identified a service impact in Rock Springs, WY. As this network fault is impacting multiple clients in western Wyoming, the event has increased visibility with leadership. As such, client trouble tickets associated to this fault have been automatically escalated to higher priority.
The Network Operations Center (NOC) is engaged and investigating in order to isolate the cause. Please be advised that updates for this event will be relayed at a minimum of hourly unless otherwise noted. The information conveyed hereafter is associated to live troubleshooting effort and as the discovery process evolves through to service resolution, ticket closure, or post incident review, details may evolve.
Emergency Management of Sublette County is dedicated to preparedness and the protection of Life, Property and Environment in all hazards response.
