Incident report - Network


Incident report

Systems affected

12:17 - Internal alerting of high latencies
12:20 - Ops Team investigates
12:47 - Issue identified, our network is having slowness overall and packets are being
13:00 - Some emergency re-routing of traffic is done to mitigate the issue
13:35 - More re-routing done, traffic is slowly coming back to normal
15:21 - All possible re-routing is done, metrics show that everything is back to normal
18:26 - Switches and firewalls have been rebooted

Executive Summary
A malfunction in our network stack caused traffic to be slow and sometimes dropped
entirely. Ops team implemented mitigation as quickly as possible to bring everything back
to normal.

Actions taken
We implemented some additional safety layers in our network to split internal and external
traffic. By applying DDOS protection and various firewall rules to the external traffic only, we
reduced drastically the pressure our some core component of our infrastructure.

Thibault Miclo

16 Affected Services:

Visit Group |  Kungsgatan 34-36 |  411 19 Göteborg |  Sweden |  +46 (0)31 38 06 000