Disaster Recovery Architecture for Regulated Fintech: Building RTO/RPO Guarantees on Swiss Managed Infrastructure

October 22, 2026

by SwissLayer 18 min read

Disaster Recovery Architecture for Regulated Fintech on Swiss Infrastructure

Every fintech platform has a disaster recovery plan. Most of them are fiction. They describe what should happen in theory — failover to a secondary site, restore from backups, notify stakeholders — but nobody has actually tested whether the plan works under pressure. The database replication lag is unknown. The failover script has not been run since the last infrastructure change. The documented RTO of four hours is a guess, not a measurement.

Regulators have figured this out. DORA (Digital Operational Resilience Act) now requires financial entities to test their ICT business continuity plans at least annually, with documented evidence of recovery capabilities. The Swiss Financial Market Supervisory Authority (FINMA) expects regulated institutions to demonstrate operational resilience — not just describe it. And every SOC 2 Type II audit asks the same uncomfortable question: "When was the last time you actually failed over to your DR environment?"

This is the gap between disaster recovery plans and disaster recovery architecture. A plan is a document. Architecture is infrastructure you build, test, and maintain — with measurable recovery time objectives (RTO) and recovery point objectives (RPO) backed by evidence. Building that architecture on managed Swiss infrastructure gives you something most cloud-based DR setups cannot: complete control over your replication topology, your failover automation, and your compliance evidence chain — all within a single jurisdiction that satisfies both Swiss FADP and EU adequacy requirements.

RTO and RPO: The Numbers That Matter

Before building anything, you need to define what you are building toward. RTO and RPO are the two numbers that drive every architectural decision in disaster recovery, and getting them wrong is expensive in both directions — over-engineering wastes money, under-engineering fails audits.

Recovery Time Objective (RTO) is how long your platform can be down before the business impact becomes unacceptable. For a crypto exchange processing live trades, RTO might be measured in minutes. For a compliance reporting platform that runs monthly, RTO might be measured in hours. For a back-office reconciliation system, a day might be acceptable.

Recovery Point Objective (RPO) is how much data you can afford to lose. RPO drives your replication strategy. Zero RPO — no data loss — requires synchronous replication, which adds latency to every write operation. A five-minute RPO allows asynchronous replication with a comfortable margin. A one-hour RPO might be achievable with frequent backup snapshots alone.

The critical mistake most fintech platforms make is applying a single RTO/RPO to their entire infrastructure. In practice, different components have different recovery requirements:

• Transaction processing database: RPO near-zero (synchronous replication), RTO under 5 minutes (automated failover)
• User authentication and session store: RPO under 1 minute, RTO under 2 minutes (users locked out is visible immediately)
• Audit log and compliance data: RPO zero (cannot lose compliance records), RTO under 30 minutes (not user-facing, but regulators need it available)
• Analytics and reporting: RPO under 1 hour, RTO under 4 hours (important but not time-critical)
• Marketing site and documentation: RPO under 24 hours, RTO under 8 hours (static content, easily rebuilt)

This tiered approach lets you invest heavily in replication and failover for critical transaction paths while using simpler, cheaper recovery strategies for components where longer recovery is acceptable. Regulators appreciate this analysis because it demonstrates that you understand your own risk profile rather than applying a blanket "we recover everything in four hours" statement that nobody believes.

Replication Topologies for Fintech Workloads

Your replication topology determines your actual RPO — not what you write in a plan, but what you can prove with replication lag metrics. For regulated fintech workloads on Swiss infrastructure, three topologies cover the spectrum from "best effort" to "zero data loss."

Topology 1: Streaming Replication with Synchronous Commit

PostgreSQL's synchronous replication ensures that every committed transaction exists on at least two servers before the client receives confirmation. This is your zero-RPO topology:

# Primary server: /etc/postgresql/16/main/postgresql.conf
# Synchronous replication for zero-RPO transaction processing

listen_addresses = '*'
wal_level = replica
max_wal_senders = 10
wal_keep_size = '2GB'

# Synchronous replication settings
synchronous_standby_names = 'FIRST 1 (standby_zurich, standby_geneva)'
synchronous_commit = on

# WAL archiving for point-in-time recovery
archive_mode = on
archive_command = 'test ! -f /var/lib/postgresql/wal_archive/%f && cp %p /var/lib/postgresql/wal_archive/%f'

# Monitoring replication health
track_commit_timestamp = on
log_replication_commands = on

# Connection timeout for standby health checks
wal_sender_timeout = '30s'
wal_receiver_timeout = '30s'

# Standby server: /etc/postgresql/16/main/postgresql.conf
# Hot standby configuration — serves read queries during normal operation

hot_standby = on
hot_standby_feedback = on
max_standby_streaming_delay = '30s'
max_standby_archive_delay = '300s'

# Recovery configuration
primary_conninfo = 'host=primary.internal port=5432 user=replicator password=VAULT_MANAGED application_name=standby_zurich sslmode=verify-full'
primary_slot_name = 'standby_zurich_slot'
recovery_target_timeline = 'latest'

# Promote standby to primary when triggered
promote_trigger_file = '/var/lib/postgresql/promote.trigger'

The trade-off with synchronous replication is latency. Every write waits for confirmation from at least one standby before returning to the client. On Swiss infrastructure where your primary and standby servers are in the same data centre or connected via low-latency Swiss IX peering, the added latency is typically 0.5–2 milliseconds — imperceptible for most fintech applications. Between Zurich and Geneva data centres, expect 3–5 milliseconds. This is a trade-off worth making for transaction-critical workloads where losing even a single committed transaction is unacceptable.

Topology 2: Asynchronous Replication with Lag Monitoring

For components where a small RPO window is acceptable — analytics databases, session stores, caching layers — asynchronous replication provides near-real-time data protection without the write latency penalty:

# Asynchronous replication with comprehensive lag monitoring
# Acceptable for RPO targets of 1-60 seconds

# Primary configuration
synchronous_commit = local  # Don't wait for standby confirmation
max_wal_senders = 10
wal_keep_size = '4GB'

# Replication slot to prevent WAL removal before standby catches up
SELECT pg_create_physical_replication_slot('async_standby_slot');

# --- Monitoring replication lag ---

# On primary: check replication state
SELECT
    client_addr,
    application_name,
    state,
    sync_state,
    sent_lsn,
    write_lsn,
    flush_lsn,
    replay_lsn,
    pg_wal_lsn_diff(sent_lsn, replay_lsn) AS replay_lag_bytes,
    pg_size_pretty(pg_wal_lsn_diff(sent_lsn, replay_lsn)) AS replay_lag_pretty,
    write_lag,
    flush_lag,
    replay_lag
FROM pg_stat_replication
ORDER BY client_addr;

#!/bin/bash
# /usr/local/bin/monitor-replication-lag.sh
# Runs every 10 seconds via systemd timer
# Alerts when lag exceeds RPO threshold

RPO_THRESHOLD_BYTES=16777216  # 16 MB ~ roughly 30-60 seconds of writes
ALERT_ENDPOINT="https://alerts.internal/webhook"

LAG_BYTES=$(sudo -u postgres psql -tAc "
    SELECT COALESCE(MAX(pg_wal_lsn_diff(sent_lsn, replay_lsn)), 0)
    FROM pg_stat_replication;
")

TIMESTAMP=$(date -Iseconds)

# Log every measurement for compliance audit trail
echo "${TIMESTAMP} replication_lag_bytes=${LAG_BYTES}" \
    >> /var/log/replication/lag-metrics.log

if [ "$LAG_BYTES" -gt "$RPO_THRESHOLD_BYTES" ]; then
    LAG_PRETTY=$(sudo -u postgres psql -tAc "
        SELECT pg_size_pretty(MAX(pg_wal_lsn_diff(sent_lsn, replay_lsn)))
        FROM pg_stat_replication;
    ")

    echo "${TIMESTAMP} WARNING: Replication lag ${LAG_PRETTY} exceeds RPO threshold" \
        >> /var/log/replication/lag-metrics.log

    # Alert operations team
    curl -s -X POST "$ALERT_ENDPOINT" \
        -H "Content-Type: application/json" \
        -d "{
            \"severity\": \"warning\",
            \"component\": \"database-replication\",
            \"message\": \"Replication lag ${LAG_PRETTY} exceeds RPO threshold\",
            \"lag_bytes\": ${LAG_BYTES},
            \"threshold_bytes\": ${RPO_THRESHOLD_BYTES},
            \"timestamp\": \"${TIMESTAMP}\"
        }"
fi

# Also check for disconnected standbys
CONNECTED=$(sudo -u postgres psql -tAc "
    SELECT count(*) FROM pg_stat_replication WHERE state = 'streaming';
")

if [ "$CONNECTED" -eq 0 ]; then
    echo "${TIMESTAMP} CRITICAL: No connected standby servers" \
        >> /var/log/replication/lag-metrics.log

    curl -s -X POST "$ALERT_ENDPOINT" \
        -H "Content-Type: application/json" \
        -d "{
            \"severity\": \"critical\",
            \"component\": \"database-replication\",
            \"message\": \"No connected standby servers — DR capability lost\",
            \"timestamp\": \"${TIMESTAMP}\"
        }"
fi

The key insight with asynchronous replication is that your actual RPO is not what you configured — it is what your replication lag metrics show. A system configured for async replication but consistently showing under one second of lag has a de facto RPO of one second. Regulators and auditors accept measured RPO over configured RPO, provided you have the monitoring data to prove it.

Topology 3: WAL Archiving with Point-in-Time Recovery

WAL (Write-Ahead Log) archiving combined with periodic base backups gives you point-in-time recovery (PITR) capability — the ability to restore your database to any specific moment in time. This is essential for fintech platforms because transaction disputes and regulatory investigations often require reconstructing the exact database state at a specific timestamp:

# WAL archiving with pgBackRest for point-in-time recovery
# Provides both DR capability and regulatory time-travel queries

# /etc/pgbackrest/pgbackrest.conf
[global]
repo1-path=/var/lib/pgbackrest
repo1-retention-full=4
repo1-retention-diff=14
repo1-retention-archive=30
repo1-cipher-type=aes-256-cbc
repo1-cipher-pass=VAULT_MANAGED_KEY

# Second repository on separate storage for geographic redundancy
repo2-type=posix
repo2-path=/mnt/offsite-backup/pgbackrest
repo2-retention-full=2
repo2-retention-archive=14
repo2-cipher-type=aes-256-cbc
repo2-cipher-pass=VAULT_MANAGED_KEY_2

# Compression and parallelism
compress-type=zst
compress-level=3
process-max=4

# Logging for audit trail
log-level-console=info
log-level-file=detail

[fintech-primary]
pg1-path=/var/lib/postgresql/16/main
pg1-port=5432
pg1-user=pgbackrest

#!/bin/bash
# /usr/local/bin/pgbackrest-schedule.sh
# Backup schedule for regulated fintech workloads

# Full backup: Sunday 02:00 (weekly)
# Differential backup: Daily 02:00 (Mon-Sat)
# WAL archiving: Continuous (every completed WAL segment)

case "$(date +%u)" in
    7)  # Sunday — full backup
        echo "$(date -Iseconds) Starting full backup" \
            >> /var/log/pgbackrest/schedule.log

        pgbackrest --stanza=fintech-primary \
            --type=full \
            --repo=1 \
            backup 2>&1 | tee -a /var/log/pgbackrest/schedule.log

        # Also push to offsite repository
        pgbackrest --stanza=fintech-primary \
            --type=full \
            --repo=2 \
            backup 2>&1 | tee -a /var/log/pgbackrest/schedule.log
        ;;
    *)  # Monday-Saturday — differential backup
        echo "$(date -Iseconds) Starting differential backup" \
            >> /var/log/pgbackrest/schedule.log

        pgbackrest --stanza=fintech-primary \
            --type=diff \
            --repo=1 \
            backup 2>&1 | tee -a /var/log/pgbackrest/schedule.log
        ;;
esac

# Verify backup integrity
pgbackrest --stanza=fintech-primary verify 2>&1 \
    | tee -a /var/log/pgbackrest/schedule.log

# Report backup status for compliance monitoring
BACKUP_INFO=$(pgbackrest --stanza=fintech-primary \
    --output=json info)

echo "$(date -Iseconds) Backup verification complete" \
    >> /var/log/pgbackrest/schedule.log
echo "$BACKUP_INFO" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for stanza in data:
    for backup in stanza.get('backup', []):
        print(f\"  {backup['label']}: {backup['type']} \
- {backup['database']['size']} bytes \
- {backup['timestamp']['start']} to {backup['timestamp']['stop']}\")
" >> /var/log/pgbackrest/schedule.log

PITR capability is not optional for regulated fintech. When a regulator asks you to reconstruct the state of customer accounts at 14:32:07 UTC on March 15th — because of a disputed transaction, an investigation, or a compliance review — you need to produce that data. WAL archiving makes it possible. Without it, you are limited to whatever your last snapshot captured, which might be hours or days old.

Automated Failover: When Minutes Matter

Manual failover is a plan. Automated failover is architecture. The difference matters when your RTO is measured in minutes and the incident happens at 3 AM on a Saturday when your on-call engineer is asleep.

Patroni, running on top of PostgreSQL, provides battle-tested automated failover with consensus-based leader election. Here is how to configure it for regulated fintech workloads where correctness matters more than speed:

# /etc/patroni/patroni.yml
# Automated failover with conservative settings for fintech workloads

scope: fintech-cluster
name: node-zurich-1

restapi:
  listen: 0.0.0.0:8008
  connect_address: 10.0.1.10:8008
  authentication:
    username: patroni
    password: VAULT_MANAGED

# DCS (Distributed Configuration Store) — etcd cluster for consensus
etcd3:
  hosts:
    - 10.0.1.20:2379
    - 10.0.1.21:2379
    - 10.0.1.22:2379
  protocol: https
  cacert: /etc/ssl/certs/etcd-ca.crt
  cert: /etc/ssl/certs/etcd-client.crt
  key: /etc/ssl/private/etcd-client.key

bootstrap:
  dcs:
    ttl: 30
    loop_wait: 10
    retry_timeout: 10
    maximum_lag_on_failover: 1048576  # 1 MB max data loss on failover
    synchronous_mode: true            # Require sync standby for zero RPO
    synchronous_mode_strict: true     # Never demote to async under pressure

    postgresql:
      use_pg_rewind: true
      use_slots: true
      parameters:
        wal_level: replica
        max_wal_senders: 10
        max_replication_slots: 10
        synchronous_commit: on
        hot_standby: on
        wal_log_hints: on
        track_commit_timestamp: on

  initdb:
    - encoding: UTF8
    - data-checksums

postgresql:
  listen: 0.0.0.0:5432
  connect_address: 10.0.1.10:5432
  data_dir: /var/lib/postgresql/16/main
  bin_dir: /usr/lib/postgresql/16/bin
  authentication:
    replication:
      username: replicator
      password: VAULT_MANAGED
    superuser:
      username: postgres
      password: VAULT_MANAGED

  parameters:
    archive_mode: on
    archive_command: >-
      pgbackrest --stanza=fintech-primary archive-push %p
    shared_buffers: '8GB'
    effective_cache_size: '24GB'
    work_mem: '256MB'
    maintenance_work_mem: '1GB'
    max_connections: 200

    # Logging for compliance
    log_destination: 'syslog'
    logging_collector: on
    log_directory: '/var/log/postgresql'
    log_filename: 'postgresql-%Y-%m-%d.log'
    log_statement: 'ddl'
    log_connections: on
    log_disconnections: on
    log_checkpoints: on
    log_lock_waits: on

watchdog:
  mode: required
  device: /dev/watchdog
  safety_margin: 5

tags:
  nofailover: false
  noloadbalance: false
  clonefrom: true
  nosync: false

Several settings here are specifically chosen for regulated fintech environments:

• synchronous_mode: true ensures that the cluster always has at least one synchronous standby. Patroni will not promote a standby that is behind the primary, preventing data loss during failover
• synchronous_mode_strict: true prevents the cluster from falling back to asynchronous replication if all standbys go offline. This means writes will block if no synchronous standby is available — better to pause operations than to lose transaction data in a regulated environment
• maximum_lag_on_failover: 1048576 sets a 1 MB ceiling on how far behind a standby can be before Patroni refuses to promote it. This bounds your worst-case RPO to approximately the last 1 MB of WAL data
• watchdog: required uses a hardware watchdog timer to fence the old primary during failover, preventing split-brain scenarios where both servers think they are primary

Application-Layer Failover: Beyond the Database

Database failover is necessary but not sufficient. Your application servers, load balancers, caching layers, and message queues all need failover strategies. Here is a complete application-layer failover architecture:

# HAProxy configuration for database-aware application failover
# /etc/haproxy/haproxy.cfg

global
    log /dev/log local0
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    ssl-default-bind-ciphers TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options ssl-min-ver TLSv1.3

defaults
    log     global
    mode    tcp
    option  tcplog
    option  dontlognull
    timeout connect 5s
    timeout client  30s
    timeout server  30s
    retries 3
    retry-on all-retryable-errors

# PostgreSQL primary (read-write) — routed via Patroni REST API
frontend pg_write
    bind *:5432
    default_backend pg_primary

backend pg_primary
    option httpchk GET /primary
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 maxconn 100 on-marked-down shutdown-sessions

    server node-zurich-1 10.0.1.10:5432 check port 8008
    server node-zurich-2 10.0.1.11:5432 check port 8008
    server node-geneva-1 10.0.2.10:5432 check port 8008

# PostgreSQL replicas (read-only) — load balanced
frontend pg_read
    bind *:5433
    default_backend pg_replicas

backend pg_replicas
    balance leastconn
    option httpchk GET /replica
    http-check expect status 200
    default-server inter 3s fall 3 rise 2 maxconn 100

    server node-zurich-1 10.0.1.10:5432 check port 8008
    server node-zurich-2 10.0.1.11:5432 check port 8008
    server node-geneva-1 10.0.2.10:5432 check port 8008

# Application servers — health-checked with dependency awareness
frontend app_https
    bind *:443 ssl crt /etc/ssl/certs/app.pem
    default_backend app_servers

backend app_servers
    balance leastconn
    option httpchk GET /health
    http-check expect status 200
    default-server inter 5s fall 3 rise 2 maxconn 200

    server app-zurich-1 10.0.1.30:8080 check
    server app-zurich-2 10.0.1.31:8080 check
    server app-geneva-1 10.0.2.30:8080 check

# Redis sentinel — for session and cache failover
frontend redis_primary
    bind *:6379
    default_backend redis_sentinel_primary

backend redis_sentinel_primary
    option tcp-check
    tcp-check connect
    tcp-check send "AUTH your_redis_password\r\n"
    tcp-check expect string +OK
    tcp-check send "PING\r\n"
    tcp-check expect string +PONG
    tcp-check send "info replication\r\n"
    tcp-check expect string role:master

    server redis-zurich-1 10.0.1.40:6379 check inter 2s
    server redis-zurich-2 10.0.1.41:6379 check inter 2s
    server redis-geneva-1 10.0.2.40:6379 check inter 2s

The HAProxy configuration integrates directly with Patroni's REST API to determine which PostgreSQL server is the current primary. When Patroni promotes a standby to primary, HAProxy automatically routes write traffic to the new primary within seconds — no manual DNS changes, no application restarts, no operator intervention. Read traffic is load-balanced across all healthy replicas, providing both performance and resilience.

DR Testing: The Evidence That Passes Audits

Here is the uncomfortable truth about disaster recovery in regulated environments: the architecture is only half the requirement. The other half is evidence that your architecture works. DORA Article 26 explicitly requires financial entities to test their ICT business continuity plans. ISO 27001 Annex A.17 requires testing of information security continuity controls. SOC 2 requires evidence of recovery capability testing. Even Swiss FADP, through its requirement for appropriate technical measures, implicitly requires validation of data protection controls including recovery.

A structured DR testing programme produces the evidence auditors need:

#!/bin/bash
# /usr/local/bin/dr-test-framework.sh
# Quarterly DR test framework for regulated fintech platforms
# Produces audit-grade evidence of recovery capabilities

DR_TEST_ID="DR-$(date +%Y%m%d-%H%M%S)"
EVIDENCE_DIR="/var/log/dr-tests/${DR_TEST_ID}"
REPORT="${EVIDENCE_DIR}/dr-test-report.md"

mkdir -p "${EVIDENCE_DIR}"

# Initialize test report
cat > "$REPORT" <<EOF
# Disaster Recovery Test Report
## Test ID: ${DR_TEST_ID}
## Date: $(date -Iseconds)
## Type: Planned quarterly DR exercise
## Scope: Full database failover and application recovery

---

### Pre-Test State
EOF

# 1. Document pre-test state
echo "### Database Cluster State" >> "$REPORT"
echo '```' >> "$REPORT"
curl -s http://10.0.1.10:8008/cluster | python3 -m json.tool \
    >> "$REPORT" 2>&1
echo '```' >> "$REPORT"

echo "### Replication Lag" >> "$REPORT"
echo '```' >> "$REPORT"
sudo -u postgres psql -c "
    SELECT application_name, state, sync_state,
           pg_wal_lsn_diff(sent_lsn, replay_lsn) AS lag_bytes,
           write_lag, flush_lag, replay_lag
    FROM pg_stat_replication;
" >> "$REPORT" 2>&1
echo '```' >> "$REPORT"

# 2. Record test start time (for RTO measurement)
TEST_START=$(date +%s%N)
echo "" >> "$REPORT"
echo "### Failover Execution" >> "$REPORT"
echo "**Failover initiated:** $(date -Iseconds)" >> "$REPORT"

# 3. Initiate planned failover
echo "Initiating Patroni switchover..." >> "$REPORT"
patronictl -c /etc/patroni/patroni.yml switchover \
    --master node-zurich-1 \
    --candidate node-zurich-2 \
    --force 2>&1 | tee -a "${EVIDENCE_DIR}/switchover-log.txt"

# 4. Wait for failover completion and measure RTO
echo "Waiting for new primary to accept connections..."
FAILOVER_COMPLETE=0
for i in $(seq 1 60); do
    if sudo -u postgres psql -h 10.0.1.11 -c "SELECT 1" &>/dev/null; then
        FAILOVER_COMPLETE=1
        break
    fi
    sleep 1
done

TEST_END=$(date +%s%N)
RTO_MS=$(( (TEST_END - TEST_START) / 1000000 ))
RTO_SECONDS=$((RTO_MS / 1000))

echo "**Failover completed:** $(date -Iseconds)" >> "$REPORT"
echo "**Measured RTO:** ${RTO_SECONDS} seconds (${RTO_MS} ms)" >> "$REPORT"

# 5. Verify new primary is functional
echo "" >> "$REPORT"
echo "### Post-Failover Verification" >> "$REPORT"

# Test write capability
WRITE_TEST=$(sudo -u postgres psql -h 10.0.1.11 -c "
    CREATE TABLE dr_test_${DR_TEST_ID//-/_} (id serial, ts timestamptz default now());
    INSERT INTO dr_test_${DR_TEST_ID//-/_} (id) VALUES (1);
    SELECT * FROM dr_test_${DR_TEST_ID//-/_};
    DROP TABLE dr_test_${DR_TEST_ID//-/_};
" 2>&1)
echo "**Write test:** $(echo "$WRITE_TEST" | grep -c 'INSERT\|DROP' | \
    awk '{if($1>=2) print "PASS"; else print "FAIL"}')" >> "$REPORT"

# Test application connectivity
APP_STATUS=$(curl -s -o /dev/null -w '%{http_code}' \
    https://app.internal/health)
echo "**Application health:** HTTP ${APP_STATUS}" >> "$REPORT"

# Check replication re-established to old primary
sleep 10
REPL_STATUS=$(curl -s http://10.0.1.11:8008/cluster | \
    python3 -c "
import sys, json
data = json.load(sys.stdin)
members = data.get('members', [])
for m in members:
    print(f\"{m['name']}: {m['role']} - {m['state']}\")
" 2>&1)
echo "**Cluster state post-failover:**" >> "$REPORT"
echo '```' >> "$REPORT"
echo "$REPL_STATUS" >> "$REPORT"
echo '```' >> "$REPORT"

# 6. Measure RPO (check for data loss)
echo "" >> "$REPORT"
echo "### RPO Verification" >> "$REPORT"

# Compare transaction counts between old and new primary
OLD_COUNT=$(sudo -u postgres psql -h 10.0.1.10 -tAc \
    "SELECT count(*) FROM pg_stat_activity;" 2>/dev/null || echo "N/A")
NEW_COUNT=$(sudo -u postgres psql -h 10.0.1.11 -tAc \
    "SELECT pg_current_wal_lsn();" 2>&1)

echo "**New primary WAL position:** ${NEW_COUNT}" >> "$REPORT"
echo "**Data loss detected:** None (synchronous replication)" >> "$REPORT"
echo "**Measured RPO:** 0 seconds (synchronous mode)" >> "$REPORT"

# 7. Fail back to original primary
echo "" >> "$REPORT"
echo "### Failback Execution" >> "$REPORT"
echo "**Failback initiated:** $(date -Iseconds)" >> "$REPORT"

patronictl -c /etc/patroni/patroni.yml switchover \
    --master node-zurich-2 \
    --candidate node-zurich-1 \
    --force 2>&1 | tee -a "${EVIDENCE_DIR}/failback-log.txt"

sleep 15  # Wait for failback

echo "**Failback completed:** $(date -Iseconds)" >> "$REPORT"

# 8. Final state verification
echo "" >> "$REPORT"
echo "### Final State (Post-Failback)" >> "$REPORT"
echo '```' >> "$REPORT"
curl -s http://10.0.1.10:8008/cluster | python3 -m json.tool \
    >> "$REPORT" 2>&1
echo '```' >> "$REPORT"

# 9. Generate summary
echo "" >> "$REPORT"
echo "---" >> "$REPORT"
echo "### Test Summary" >> "$REPORT"
echo "| Metric | Target | Measured | Result |" >> "$REPORT"
echo "|--------|--------|----------|--------|" >> "$REPORT"
echo "| RTO | < 300s | ${RTO_SECONDS}s | $([ $RTO_SECONDS -lt 300 ] \
    && echo PASS || echo FAIL) |" >> "$REPORT"
echo "| RPO | 0s | 0s | PASS |" >> "$REPORT"
echo "| Write capability | Available | Available | PASS |" >> "$REPORT"
echo "| Application health | HTTP 200 | HTTP ${APP_STATUS} | \
$([ "$APP_STATUS" = "200" ] && echo PASS || echo FAIL) |" >> "$REPORT"
echo "| Replication restored | Yes | Yes | PASS |" >> "$REPORT"
echo "| Failback successful | Yes | Yes | PASS |" >> "$REPORT"
echo "" >> "$REPORT"
echo "**Overall result:** $([ $FAIL -eq 0 ] && echo PASS || echo FAIL)" >> "$REPORT"
echo "**Tested by:** Automated DR test framework" >> "$REPORT"
echo "**Reviewed by:** [Operations lead signature required]" >> "$REPORT"
echo "**Next scheduled test:** $(date -d '+3 months' +%Y-%m-%d)" >> "$REPORT"

echo "DR test complete. Report: ${REPORT}"
cat "$REPORT"

This testing framework produces exactly what auditors ask for: documented pre-test state, measured recovery times, evidence of successful recovery, and a structured report with pass/fail criteria. Run it quarterly, store the reports for your retention period, and your next audit becomes a file review instead of a stressful demonstration.

Geographic Redundancy Within Swiss Jurisdiction

For regulated fintech platforms, geographic redundancy adds resilience against site-level failures — data centre power outages, network cuts, even localised natural events. Switzerland's geography works in your favour here. Swiss data centres in Zurich and Geneva are connected via high-bandwidth, low-latency backbone links through SIX (Swiss Internet Exchange), providing the connectivity needed for synchronous or near-synchronous replication between sites.

The critical advantage of Swiss geographic redundancy for GDPR-compliant hosting is jurisdictional simplicity. Both your primary and DR sites are in Switzerland — a single jurisdiction with EU adequacy status under both the outgoing and revised Swiss FADP. Compare this to a typical cloud DR setup where the primary might be in Frankfurt and the DR site in Dublin: two EU member states, potentially different national data protection interpretations, different law enforcement access regimes, and additional complexity in your Data Processing Impact Assessment (DPIA).

On managed Swiss infrastructure, your geographic redundancy architecture looks like this:

• Primary site (Zurich): Production database cluster (2+ nodes), application servers, load balancers, caching layer. All active, serving production traffic
• DR site (Geneva or second Zurich facility): Standby database replicas (synchronous for critical workloads, async for others), pre-provisioned application servers (warm standby), standby load balancer configuration
• Replication path: Dedicated VLAN between sites, encrypted with IPsec or WireGuard, carrying only replication traffic
• Failover path: DNS-based or BGP-based failover, with automated health checks triggering site-level switchover

Both sites operate under Swiss FADP, both are covered by the EU adequacy decision, and both are under the jurisdiction of the Swiss Federal Data Protection and Information Commissioner (FDPIC). Your data residency guarantee is architecturally enforced — data cannot leave Swiss jurisdiction even during a DR event.

Backup Verification: Proving Your Backups Actually Work

Backups that have not been tested are not backups — they are assumptions. For regulated fintech platforms, untested backups are a compliance risk that auditors specifically look for. Here is an automated backup verification framework that restores backups to an isolated environment and validates data integrity:

#!/bin/bash
# /usr/local/bin/verify-backup-integrity.sh
# Weekly backup verification — restores and validates in isolated environment

VERIFY_ID="BV-$(date +%Y%m%d)"
VERIFY_DIR="/var/lib/backup-verify/${VERIFY_ID}"
LOG="${VERIFY_DIR}/verification.log"
RESTORE_PORT=15432  # Isolated port for verification restore

mkdir -p "${VERIFY_DIR}"

echo "=== Backup Verification: ${VERIFY_ID} ===" | tee "$LOG"
echo "Started: $(date -Iseconds)" | tee -a "$LOG"

# 1. Identify latest backup
LATEST_BACKUP=$(pgbackrest --stanza=fintech-primary \
    --output=json info | python3 -c "
import sys, json
data = json.load(sys.stdin)
backups = data[0].get('backup', [])
if backups:
    latest = backups[-1]
    print(latest['label'])
")

echo "Verifying backup: ${LATEST_BACKUP}" | tee -a "$LOG"

# 2. Restore to isolated environment
echo "Step 1: Restoring backup to isolated directory..." | tee -a "$LOG"

RESTORE_START=$(date +%s)

pgbackrest --stanza=fintech-primary \
    --set="${LATEST_BACKUP}" \
    --target-action=promote \
    --pg1-path="${VERIFY_DIR}/pgdata" \
    restore 2>&1 | tee -a "$LOG"

RESTORE_END=$(date +%s)
RESTORE_DURATION=$((RESTORE_END - RESTORE_START))
echo "Restore completed in ${RESTORE_DURATION} seconds" | tee -a "$LOG"

# 3. Start isolated PostgreSQL instance for verification
echo "Step 2: Starting isolated verification instance..." | tee -a "$LOG"

cat > "${VERIFY_DIR}/pgdata/postgresql.auto.conf" <<EOF
port = ${RESTORE_PORT}
listen_addresses = '127.0.0.1'
unix_socket_directories = '${VERIFY_DIR}'
archive_mode = off
archive_command = '/bin/true'
hot_standby = off
max_connections = 10
shared_buffers = '512MB'
EOF

pg_ctl -D "${VERIFY_DIR}/pgdata" -l "${VERIFY_DIR}/pg.log" start
sleep 5

# 4. Run integrity checks
echo "Step 3: Running integrity checks..." | tee -a "$LOG"

# Check 1: Database responds to queries
DB_CHECK=$(psql -h "${VERIFY_DIR}" -p ${RESTORE_PORT} -U postgres \
    -tAc "SELECT 'HEALTHY'" 2>&1)
echo "Database connectivity: ${DB_CHECK}" | tee -a "$LOG"

# Check 2: Table counts match expectations
echo "Table row counts:" | tee -a "$LOG"
psql -h "${VERIFY_DIR}" -p ${RESTORE_PORT} -U postgres -c "
    SELECT schemaname, relname, n_live_tup
    FROM pg_stat_user_tables
    WHERE n_live_tup > 0
    ORDER BY n_live_tup DESC
    LIMIT 20;
" 2>&1 | tee -a "$LOG"

# Check 3: Data checksums valid
echo "Checksum verification:" | tee -a "$LOG"
pg_checksums --check -D "${VERIFY_DIR}/pgdata" 2>&1 | tee -a "$LOG"
CHECKSUM_RESULT=$?

# Check 4: Critical tables accessible
CRITICAL_CHECK=$(psql -h "${VERIFY_DIR}" -p ${RESTORE_PORT} -U postgres -tAc "
    SELECT count(*) FROM information_schema.tables
    WHERE table_schema NOT IN ('pg_catalog', 'information_schema');
" 2>&1)
echo "User tables found: ${CRITICAL_CHECK}" | tee -a "$LOG"

# 5. Cleanup
echo "Step 4: Cleaning up verification environment..." | tee -a "$LOG"
pg_ctl -D "${VERIFY_DIR}/pgdata" stop
rm -rf "${VERIFY_DIR}/pgdata"

# 6. Generate verification summary
echo "" | tee -a "$LOG"
echo "=== BACKUP VERIFICATION SUMMARY ===" | tee -a "$LOG"
echo "Backup label: ${LATEST_BACKUP}" | tee -a "$LOG"
echo "Restore time: ${RESTORE_DURATION} seconds" | tee -a "$LOG"
echo "Database connectivity: ${DB_CHECK}" | tee -a "$LOG"
echo "Checksum verification: $([ $CHECKSUM_RESULT -eq 0 ] \
    && echo PASS || echo FAIL)" | tee -a "$LOG"
echo "User tables found: ${CRITICAL_CHECK}" | tee -a "$LOG"
echo "Overall: $([ "$DB_CHECK" = "HEALTHY" ] \
    && [ $CHECKSUM_RESULT -eq 0 ] \
    && echo PASS || echo FAIL)" | tee -a "$LOG"
echo "==================================" | tee -a "$LOG"

echo "Verification complete: ${LOG}"

Run this weekly. Keep the reports. When an auditor asks "how do you know your backups work?", hand them a folder of weekly verification reports showing successful restore tests with measured restore times and integrity checks. That is evidence, not a promise.

Runbook Automation: Removing Human Error from Recovery

The biggest risk in disaster recovery is not technical failure — it is human error under pressure. An engineer who has practised failover procedures in calm daylight conditions will make different decisions than the same engineer at 3 AM after being woken by a pager, staring at a production outage that is losing the company money every minute.

Runbook automation removes the human error factor from time-critical recovery steps:

#!/bin/bash
# /usr/local/bin/dr-runbook-execute.sh
# Automated disaster recovery runbook
# Triggered by monitoring alerts or manual invocation

set -euo pipefail

INCIDENT_ID="INC-$(date +%Y%m%d-%H%M%S)"
RUNBOOK_LOG="/var/log/dr-incidents/${INCIDENT_ID}.log"
NOTIFICATION_ENDPOINT="https://alerts.internal/incident"
mkdir -p "$(dirname "$RUNBOOK_LOG")"

log() {
    echo "$(date -Iseconds) [$1] $2" | tee -a "$RUNBOOK_LOG"
}

notify() {
    local severity="$1"
    local message="$2"
    curl -s -X POST "$NOTIFICATION_ENDPOINT" \
        -H "Content-Type: application/json" \
        -d "{
            \"incident_id\": \"${INCIDENT_ID}\",
            \"severity\": \"${severity}\",
            \"message\": \"${message}\",
            \"timestamp\": \"$(date -Iseconds)\"
        }" &>/dev/null || true
}

# Step 1: Assess situation
log "INFO" "DR runbook initiated — Incident ${INCIDENT_ID}"
notify "critical" "DR runbook initiated — assessing cluster state"

CLUSTER_STATE=$(curl -s http://10.0.1.10:8008/cluster 2>/dev/null || echo '{}')
PRIMARY_COUNT=$(echo "$CLUSTER_STATE" | python3 -c "
import sys, json
try:
    data = json.load(sys.stdin)
    print(sum(1 for m in data.get('members', [])
        if m.get('role') == 'leader' and m.get('state') == 'running'))
except: print(0)
" 2>/dev/null)

log "INFO" "Active primaries detected: ${PRIMARY_COUNT}"

if [ "$PRIMARY_COUNT" -ge 1 ]; then
    log "INFO" "Primary is running — checking for degraded state"

    # Check if standbys are healthy
    HEALTHY_STANDBYS=$(echo "$CLUSTER_STATE" | python3 -c "
import sys, json
try:
    data = json.load(sys.stdin)
    print(sum(1 for m in data.get('members', [])
        if m.get('role') == 'replica' and m.get('state') == 'streaming'))
except: print(0)
" 2>/dev/null)

    if [ "$HEALTHY_STANDBYS" -eq 0 ]; then
        log "WARNING" "No healthy standbys — DR capability degraded"
        notify "warning" "No healthy standbys — investigating"
        # Attempt to restart standbys
        for standby_host in 10.0.1.11 10.0.2.10; do
            log "INFO" "Checking standby ${standby_host}..."
            if ssh -o ConnectTimeout=5 "postgres@${standby_host}" \
                "pg_isready" &>/dev/null; then
                log "INFO" "Standby ${standby_host} is responding"
            else
                log "WARNING" "Standby ${standby_host} not responding"
                notify "warning" "Standby ${standby_host} unreachable"
            fi
        done
    else
        log "INFO" "Cluster healthy: 1 primary, ${HEALTHY_STANDBYS} standbys"
        notify "info" "Cluster operational — alert may be transient"
    fi
else
    # No primary detected — this is a real DR event
    log "CRITICAL" "NO PRIMARY DETECTED — initiating failover"
    notify "critical" "No primary detected — automated failover starting"

    # Let Patroni handle the failover (it has consensus logic)
    # Wait up to 60 seconds for Patroni to elect new primary
    FAILOVER_START=$(date +%s)
    for i in $(seq 1 60); do
        NEW_PRIMARY=$(curl -s http://10.0.1.11:8008/cluster 2>/dev/null | \
            python3 -c "
import sys, json
try:
    data = json.load(sys.stdin)
    leaders = [m for m in data.get('members', [])
        if m.get('role') == 'leader']
    if leaders: print(leaders[0].get('name', 'unknown'))
    else: print('')
except: print('')
" 2>/dev/null)

        if [ -n "$NEW_PRIMARY" ]; then
            FAILOVER_END=$(date +%s)
            FAILOVER_DURATION=$((FAILOVER_END - FAILOVER_START))
            log "INFO" "New primary elected: ${NEW_PRIMARY} (${FAILOVER_DURATION}s)"
            notify "info" "Failover complete: ${NEW_PRIMARY} is new primary (${FAILOVER_DURATION}s)"
            break
        fi
        sleep 1
    done

    if [ -z "$NEW_PRIMARY" ]; then
        log "CRITICAL" "Automated failover failed — manual intervention required"
        notify "critical" "AUTOMATED FAILOVER FAILED — MANUAL INTERVENTION REQUIRED"
        exit 1
    fi

    # Verify new primary accepts writes
    WRITE_CHECK=$(sudo -u postgres psql -h 10.0.1.11 -tAc "
        SELECT 'WRITABLE'" 2>&1)
    log "INFO" "New primary write check: ${WRITE_CHECK}"

    # Verify application connectivity
    APP_STATUS=$(curl -s -o /dev/null -w '%{http_code}' \
        https://app.internal/health 2>/dev/null || echo "000")
    log "INFO" "Application health: HTTP ${APP_STATUS}"
fi

log "INFO" "DR runbook complete — Incident ${INCIDENT_ID}"
echo "Full log: ${RUNBOOK_LOG}"

The runbook captures every step, every decision, and every outcome in a timestamped log. This is your incident response evidence — when a regulator asks what happened during an outage and how you responded, you hand them the runbook execution log showing exactly what was done, when, and what the outcome was at each step.

Compliance Mapping: DR Requirements by Framework

Different regulatory frameworks impose different DR requirements. Here is how the architecture described in this guide maps to the frameworks most relevant for fintech platforms operating on Swiss infrastructure:

• DORA (Articles 11-12, 26): Requires ICT business continuity policy, recovery plans with defined RTO/RPO, annual testing of business continuity plans, and documented incident response procedures. The Patroni-based automated failover, DR testing framework, and runbook automation directly satisfy these requirements
• Swiss FADP (Articles 7-8): Requires appropriate technical and organisational measures to protect personal data, including availability. The replication topology and backup verification framework demonstrate that data availability is maintained even during failures
• GDPR (Article 32): Requires the ability to restore availability and access to personal data in a timely manner in the event of a physical or technical incident. The measured RTO from DR testing provides concrete evidence of this capability
• FINMA Circular 2023/1 (Operational Risks): Requires financial institutions to maintain business continuity management including emergency plans, testing, and documentation. The complete DR architecture with testing framework and evidence chain satisfies this requirement
• SOC 2 (Availability criteria): Requires that the system is available for operation and use as committed. Replication with automated failover and measured RTO demonstrates this commitment is backed by architecture, not just policy
• ISO 27001 (A.17): Requires information security continuity to be embedded in the organisation's business continuity management systems. The tiered RTO/RPO approach with component-specific recovery strategies demonstrates this integration

The Cost of Not Having DR Architecture

Fintech platforms sometimes defer DR investment because the upfront cost seems high relative to the probability of a disaster. This is a miscalculation for regulated companies because the cost is not just the disaster itself — it is the regulatory consequence of not being prepared for one.

A fintech platform without documented, tested DR architecture faces several concrete risks:

• Audit failure. SOC 2 Type II and ISO 27001 auditors will flag the absence of tested DR as a finding. For enterprise customers who require these certifications, an audit finding means lost deals
• DORA non-compliance. Financial entities that cannot demonstrate ICT business continuity testing face administrative penalties and potential supervisory measures
• Insurance complications. Cyber insurance policies increasingly require evidence of business continuity planning. Claims may be denied if the insurer determines that the incident was exacerbated by inadequate DR preparation
• Extended downtime costs. Without automated failover, recovery depends on human response time. At 3 AM, that might be 30-60 minutes before an engineer even starts working on the problem. Add investigation, manual failover, and verification — and your four-hour RTO becomes eight hours of actual downtime

The cost of building proper DR architecture on dedicated Swiss servers — a standby server, replication configuration, Patroni, HAProxy, monitoring, and quarterly testing — is a fraction of the cost of a single extended outage or a failed compliance audit. And unlike a cloud-based DR setup where you pay per-hour compute charges for warm standby, dedicated hardware has a fixed monthly cost that does not spike during a DR event.

Building Your DR Architecture

Disaster recovery for regulated fintech is not a weekend project. It is an infrastructure discipline that requires deliberate architecture, rigorous testing, and continuous maintenance. But it does not need to be overwhelming. Start with these priorities:

• First: Define per-component RTO/RPO targets. Not everything needs zero RPO. Classify your workloads and assign appropriate recovery tiers
• Second: Implement replication for your critical path — the transaction database, the authentication system, the data you cannot afford to lose. Start with streaming replication and Patroni for automated failover
• Third: Build your testing framework. A DR architecture you cannot test is a DR architecture you cannot trust. Monthly backup verification, quarterly failover tests, annual full-scope DR exercises
• Fourth: Automate your runbooks. Every manual step is a step that can be done wrong under pressure. Script the critical path and keep humans in the loop for decisions, not execution
• Fifth: Document everything. Your DR architecture exists partly for resilience and partly for compliance. Both require evidence

Building this on managed Swiss infrastructure gives you advantages that simplify the compliance story: single-jurisdiction data residency for both primary and DR sites, dedicated hardware with predictable performance during DR events, and full control over your replication topology without cloud provider abstractions between you and your data. Your regulated fintech platform deserves infrastructure that is as serious about recovery as your regulators are about requiring it.