Reboot SAP HANA on node 1
Description — Simulate a crash of the primary node running the primary SAP HANA database.
Run node: Primary SAP HANA database node
Run steps:
-
Crash the primary database system using the following command as root:
[root@prihana ~]# pcs status Cluster name: rhelhanaha Stack: corosync Current DC: sechana (version 1.1.19-8.el7_6.5-c3c624ea3d) - partition with quorum Last updated: Tue Nov 10 17:54:13 2020 Last change: Tue Nov 10 17:53:48 2020 by root via crm_attribute on prihana 2 nodes configured 6 resources configured Online: [ prihana sechana ] Full list of resources: clusterfence (stonith:fence_aws): Started prihana Clone Set: SAPHanaTopology_HDB_00-clone [SAPHanaTopology_HDB_00] Started: [ prihana sechana ] Master/Slave Set: SAPHana_HDB_00-master [SAPHana_HDB_00] Masters: [ prihana ] Slaves: [ sechana ] hana-oip (ocf::heartbeat:aws-vpc-move-ip): Started prihana Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled [root@prihana ~]# echo 'b' > /proc/sysrq-trigger
Note
To simulate a system crash, you must first ensure that
/proc/sys/kernel/sysrq
is set to 1.
Expected result:
-
The cluster detects the failed node (node 1), declares it “
UNCLEAN
”, and sets the secondary node (node 2) to status “partition WITHOUT quorum
”. -
The cluster fences node 1, promotes the secondary SAP HANA database, and registers it against the new primary when the EC2 instance is back up. Node 1 is currently in a stopped state because it is being rebooted.
[root@sechana ~]# pcs status Cluster name: rhelhanaha Stack: corosync Current DC: sechana (version 1.1.19-8.el7_6.5-c3c624ea3d) - partition with quorum Last updated: Tue Nov 10 18:17:24 2020 Last change: Tue Nov 10 18:17:06 2020 by root via crm_attribute on sechana 2 nodes configured 6 resources configured Online: [ prihana sechana ] Full list of resources: clusterfence (stonith:fence_aws): Started sechana Clone Set: SAPHanaTopology_HDB_00-clone [SAPHanaTopology_HDB_00] Started: [ prihana sechana ] Master/Slave Set: SAPHana_HDB_00-master [SAPHana_HDB_00] Masters: [ sechana ] OFFLINE: [ prihana ] hana-oip (ocf::heartbeat:aws-vpc-move-ip): Started sechana Daemon Status: corosync: active/enabled pacemaker: active/enabled pcsd: active/enabled [root@sechana ~]#
-
The overlay IP address is migrated to the new primary (on node 2).
-
Because
AUTOMATIC_REGISTER
=true
, the cluster restarts the failed HANA database and registers it against the new primary when the EC2 instance is back up.
Recovery procedure:
-
Start node 1 (EC2 Instance) using Amazon Web Services Management Console or Amazon CLI tools.