IT Resilience: The Gap Between Theory and Practice

A Look Back at water lab live 

Encrypted servers on a Saturday morning, 500 “stranded” passengers at the airport, and the realization that a 278-page emergency manual is of no help to anyone in a crisis situation. The water lab live event on March 31 in Düsseldorf focused entirely on the question: How resilient is our company really when a crisis strikes?

In theory, everything goes according to plan. But what happens when the ERP system suddenly crashes or a critical alert puts an end to the weekend? Together, we took a look behind the scenes of crisis management—unvarnished and practical—“the arcitects’ way”

The Anatomy of a Crisis

Martin Swierkot (COO of arcitects) opened the event with a gripping reality check. Using a disturbingly realistic story, he examined a ransomware attack from three different perspectives and provided us with fascinating insights:

• The Admin: The first alert comes in on Saturday morning. Without clear escalation procedures and after the issue had been active for over 24 hours, it takes nearly three hours before it is escalated.
• The Management: Chaos reigns in the operations center. There are 40 people in the room, no clear division of roles, and the grim news: the backups are 13 months old. The experienced manager can only watch helplessly as the crisis unfolds.
• The Investigators (LKA): Day 4 of the crisis. The ransom demand is exorbitant, and the backups seem unusable at first. Taking a neutral look at the facts helps bring the situation under control.

Incident Response: From the “Red Zone” to the “Green Zone”

Eric Soldierer (Principal Cloud Engineer) debunked some of the myths surrounding incident response and shared practical insights from his years of experience as an incident responder. His core principle: strictly separating compromised systems from newly deployed ones. During recovery, a clean “green zone” must be established, rather than simply trying to patch up the “red zone.”

He described blind actionism as particularly critical in an emergency:
“Simply shutting down systems in an emergency erases valuable forensic artifacts in RAM.” His clear advice: “Disconnect the network, yes; shutdown, no!”

ARC Live-Demo

Alex Benoit (CEO of water) impressively demonstrated how simple and quick it is to activate the ARC emergency platform. The platform is based on Microsoft Azure Stack and provides users with access to a standalone tenant featuring all Office applications in just a few minutes. ARC thus offers the ideal conditions for quickly resuming productivity in a crisis situation, including with Outlook, Teams, and SharePoint—because maximum productivity can only be achieved in familiar environments.

BCM Without the Paperwork Nightmare: Iterative Rather Than an Ivory Tower

During my presentation on resilience in practice, I was then able to highlight our iterative approach to business continuity management (BCM)—because, let’s be honest: in an emergency, no one is going to read a 278-page manual that’s been sitting in a drawer for three years. If we can mentally let go of the “perfect impact analysis” and instead develop practical emergency plans, we’ll create real added value for our organization.

Practice is the foundation of practical skills

Together with Martin, I explored the importance of emergency drills and presented various types. The combination of standard tabletop exercises and crisis simulations emerged as particularly important. The theoretical emergency plans developed during table-top exercises are put to the test during crisis simulations—with the difference that the survival of the company is not at stake in the exercise scenario. By specifically supplementing tabletop exercises with crisis simulations, training focuses not only on process plans and flowcharts but also on key areas of action during a crisis, such as methodical decision-making approaches and behavioral patterns in stressful situations.

Customer-Panel

The event was rounded out by engaging guest presentations and a concluding panel discussion, for which I would like to take this opportunity to once again extend my sincere thanks to all who participated. For me, this serves as yet another reminder that while the challenges and obstacles are unique to each organization, the common goal is clear: IT resilience is one of the most critical building blocks for business continuity during a crisis.

Conclusion & Key-Takeaways

• The event made it clear: Prevention is always cheaper than reaction. Those who fail to invest in their IT security and the training of their teams today risk the very survival of their company in the event of a crisis.
• A crisis often fails not because of technology, but due to a lack of leadership, unclear responsibilities, and the “we’ve always done it this way” mindset.
• Practical emergency plans for individual processes offer greater value during a crisis than a comprehensive, in-depth analysis of the entire corporate landscape 
• Tabletop exercises help, but only under the real stress of a crisis simulation does it become clear whether the protocols work or whether the brain reverts to pure “reaction mode.”
• Start simple: A single page listing the most important names, roles, and numbers is better in an emergency than being left empty-handed