iTicket

Systems affected
iTicket

Timeline

• 09:55 – Monitoring alerts triggered for iTicket following deployment.
• 09:56 – Customer reports issues with bookings.
• 09:57 – Validation errors observed in iTicket; rollback considered.
• 10:08 – Root cause identified: during deployment, a hotfix branch that was not
  intended for release was mistakenly merged with the sprint deploy.
• 10:25 – Re-deploy of corrected version initiated and tested.
• 10:30 – Booking creation and modification confirmed working.
• 10:44 – Affected customers confirm resolution.
• 11:00 – Incident closed after stakeholder confirmation.

Root Cause Analysis
The incident was caused by human error during deployment. A hotfix branch containing
unverified changes — which was never intended to be part of the release — was
inadvertently merged into the sprint deployment. This introduced validation errors that
disrupted booking creation and modification.

Investigation / Resolution Actions

• Initial Response: Errors were identified quickly via our monitoring and customer
  reports.
• Rollback & Re-deploy: The team initiated a rollback and deployed a corrected
  version.
• Testing: Booking operations were tested across environments and confirmed
  working.
• Customer confirmation: Affected customers confirmed resolution.

Conclusion
The disruption to booking operations stemmed from an unintended deployment of code
due to a merge error during release. The issue was resolved through redeployment,
verification, and customer confirmation.

Preventive measures
Strengthen and automate deployment procedures to ensure only intended and tested code
is released.
Ongoing Improvements: An initiative to improve and streamline the path to live for iTicket
deployments is already underway. This will reduce deployment risk and improve overall
system resilience.

Status: Resolved
Resolution Time: 11:01 AM