diff --git a/cts/scheduler/summary/remote-connection-unrecoverable.summary b/cts/scheduler/summary/remote-connection-unrecoverable.summary index 3cfb64565a..727dad2b29 100644 --- a/cts/scheduler/summary/remote-connection-unrecoverable.summary +++ b/cts/scheduler/summary/remote-connection-unrecoverable.summary @@ -1,54 +1,54 @@ Current cluster status: * Node List: * Node node1: UNCLEAN (offline) * Online: [ node2 ] * RemoteOnline: [ remote1 ] * Full List of Resources: * remote1 (ocf:pacemaker:remote): Started node1 (UNCLEAN) * killer (stonith:fence_xvm): Started node2 * rsc1 (ocf:pacemaker:Dummy): Started remote1 * Clone Set: rsc2-master [rsc2] (promotable): * rsc2 (ocf:pacemaker:Stateful): Promoted node1 (UNCLEAN) * Promoted: [ node2 ] * Stopped: [ remote1 ] Transition Summary: - * Fence (reboot) remote1 'resources are active and the connection is unrecoverable' + * Fence (reboot) remote1 'resources are active but connection is unrecoverable' * Fence (reboot) node1 'peer is no longer part of the cluster' * Stop remote1 ( node1 ) due to node availability * Restart killer ( node2 ) due to resource definition change * Move rsc1 ( remote1 -> node2 ) * Stop rsc2:0 ( Promoted node1 ) due to node availability Executing Cluster Transition: * Pseudo action: remote1_stop_0 * Resource action: killer stop on node2 * Resource action: rsc1 monitor on node2 * Fencing node1 (reboot) * Fencing remote1 (reboot) * Resource action: killer start on node2 * Resource action: killer monitor=60000 on node2 * Pseudo action: rsc1_stop_0 * Pseudo action: rsc2-master_demote_0 * Resource action: rsc1 start on node2 * Pseudo action: rsc2_demote_0 * Pseudo action: rsc2-master_demoted_0 * Pseudo action: rsc2-master_stop_0 * Resource action: rsc1 monitor=10000 on node2 * Pseudo action: rsc2_stop_0 * Pseudo action: rsc2-master_stopped_0 Revised Cluster Status: * Node List: * Online: [ node2 ] * OFFLINE: [ node1 ] * RemoteOFFLINE: [ remote1 ] * Full List of Resources: * remote1 (ocf:pacemaker:remote): Stopped * killer (stonith:fence_xvm): Started node2 * rsc1 (ocf:pacemaker:Dummy): Started node2 * Clone Set: rsc2-master [rsc2] (promotable): * Promoted: [ node2 ] * Stopped: [ node1 remote1 ] diff --git a/cts/scheduler/summary/remote-recover-all.summary b/cts/scheduler/summary/remote-recover-all.summary index 18d10730bf..5a7d3ce3fa 100644 --- a/cts/scheduler/summary/remote-recover-all.summary +++ b/cts/scheduler/summary/remote-recover-all.summary @@ -1,146 +1,146 @@ Using the original execution date of: 2017-05-03 13:33:24Z Current cluster status: * Node List: * Node controller-1: UNCLEAN (offline) * Online: [ controller-0 controller-2 ] * RemoteOnline: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * Full List of Resources: * messaging-0 (ocf:pacemaker:remote): Started controller-0 * messaging-1 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * messaging-2 (ocf:pacemaker:remote): Started controller-0 * galera-0 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * galera-1 (ocf:pacemaker:remote): Started controller-0 * galera-2 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * Clone Set: rabbitmq-clone [rabbitmq]: * Started: [ messaging-0 messaging-1 messaging-2 ] * Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 ] * Clone Set: galera-master [galera] (promotable): * Promoted: [ galera-0 galera-1 galera-2 ] * Stopped: [ controller-0 controller-1 controller-2 messaging-0 messaging-1 messaging-2 ] * Clone Set: redis-master [redis] (promotable): * redis (ocf:heartbeat:redis): Unpromoted controller-1 (UNCLEAN) * Promoted: [ controller-0 ] * Unpromoted: [ controller-2 ] * Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * Clone Set: haproxy-clone [haproxy]: * haproxy (systemd:haproxy): Started controller-1 (UNCLEAN) * Started: [ controller-0 controller-2 ] * Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 * stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-1 (UNCLEAN) Transition Summary: - * Fence (reboot) messaging-1 'resources are active and the connection is unrecoverable' - * Fence (reboot) galera-2 'resources are active and the connection is unrecoverable' + * Fence (reboot) messaging-1 'resources are active but connection is unrecoverable' + * Fence (reboot) galera-2 'resources are active but connection is unrecoverable' * Fence (reboot) controller-1 'peer is no longer part of the cluster' * Stop messaging-1 ( controller-1 ) due to node availability * Move galera-0 ( controller-1 -> controller-2 ) * Stop galera-2 ( controller-1 ) due to node availability * Stop rabbitmq:2 ( messaging-1 ) due to node availability * Stop galera:1 ( Promoted galera-2 ) due to node availability * Stop redis:0 ( Unpromoted controller-1 ) due to node availability * Move ip-172.17.1.14 ( controller-1 -> controller-2 ) * Move ip-172.17.1.17 ( controller-1 -> controller-2 ) * Move ip-172.17.4.11 ( controller-1 -> controller-2 ) * Stop haproxy:0 ( controller-1 ) due to node availability * Move stonith-fence_ipmilan-5254005bdbb5 ( controller-1 -> controller-2 ) Executing Cluster Transition: * Pseudo action: messaging-1_stop_0 * Pseudo action: galera-0_stop_0 * Pseudo action: galera-2_stop_0 * Pseudo action: galera-master_demote_0 * Pseudo action: redis-master_pre_notify_stop_0 * Pseudo action: stonith-fence_ipmilan-5254005bdbb5_stop_0 * Fencing controller-1 (reboot) * Pseudo action: redis_post_notify_stop_0 * Resource action: redis notify on controller-0 * Resource action: redis notify on controller-2 * Pseudo action: redis-master_confirmed-pre_notify_stop_0 * Pseudo action: redis-master_stop_0 * Pseudo action: haproxy-clone_stop_0 * Fencing galera-2 (reboot) * Pseudo action: galera_demote_0 * Pseudo action: galera-master_demoted_0 * Pseudo action: galera-master_stop_0 * Pseudo action: redis_stop_0 * Pseudo action: redis-master_stopped_0 * Pseudo action: haproxy_stop_0 * Pseudo action: haproxy-clone_stopped_0 * Fencing messaging-1 (reboot) * Resource action: galera-0 start on controller-2 * Pseudo action: rabbitmq_post_notify_stop_0 * Pseudo action: rabbitmq-clone_stop_0 * Pseudo action: galera_stop_0 * Resource action: galera monitor=10000 on galera-0 * Pseudo action: galera-master_stopped_0 * Pseudo action: redis-master_post_notify_stopped_0 * Pseudo action: ip-172.17.1.14_stop_0 * Pseudo action: ip-172.17.1.17_stop_0 * Pseudo action: ip-172.17.4.11_stop_0 * Resource action: stonith-fence_ipmilan-5254005bdbb5 start on controller-2 * Resource action: galera-0 monitor=20000 on controller-2 * Resource action: rabbitmq notify on messaging-2 * Resource action: rabbitmq notify on messaging-0 * Pseudo action: rabbitmq_notified_0 * Pseudo action: rabbitmq_stop_0 * Pseudo action: rabbitmq-clone_stopped_0 * Resource action: redis notify on controller-0 * Resource action: redis notify on controller-2 * Pseudo action: redis-master_confirmed-post_notify_stopped_0 * Resource action: ip-172.17.1.14 start on controller-2 * Resource action: ip-172.17.1.17 start on controller-2 * Resource action: ip-172.17.4.11 start on controller-2 * Resource action: stonith-fence_ipmilan-5254005bdbb5 monitor=60000 on controller-2 * Pseudo action: redis_notified_0 * Resource action: ip-172.17.1.14 monitor=10000 on controller-2 * Resource action: ip-172.17.1.17 monitor=10000 on controller-2 * Resource action: ip-172.17.4.11 monitor=10000 on controller-2 Using the original execution date of: 2017-05-03 13:33:24Z Revised Cluster Status: * Node List: * Online: [ controller-0 controller-2 ] * OFFLINE: [ controller-1 ] * RemoteOnline: [ galera-0 galera-1 messaging-0 messaging-2 ] * RemoteOFFLINE: [ galera-2 messaging-1 ] * Full List of Resources: * messaging-0 (ocf:pacemaker:remote): Started controller-0 * messaging-1 (ocf:pacemaker:remote): Stopped * messaging-2 (ocf:pacemaker:remote): Started controller-0 * galera-0 (ocf:pacemaker:remote): Started controller-2 * galera-1 (ocf:pacemaker:remote): Started controller-0 * galera-2 (ocf:pacemaker:remote): Stopped * Clone Set: rabbitmq-clone [rabbitmq]: * Started: [ messaging-0 messaging-2 ] * Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 messaging-1 ] * Clone Set: galera-master [galera] (promotable): * Promoted: [ galera-0 galera-1 ] * Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ] * Clone Set: redis-master [redis] (promotable): * Promoted: [ controller-0 ] * Unpromoted: [ controller-2 ] * Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-2 * ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-2 * ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-2 * Clone Set: haproxy-clone [haproxy]: * Started: [ controller-0 controller-2 ] * Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 * stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-2 diff --git a/cts/scheduler/summary/remote-recover-no-resources.summary b/cts/scheduler/summary/remote-recover-no-resources.summary index d7d9ef942c..eb94266763 100644 --- a/cts/scheduler/summary/remote-recover-no-resources.summary +++ b/cts/scheduler/summary/remote-recover-no-resources.summary @@ -1,137 +1,137 @@ Using the original execution date of: 2017-05-03 13:33:24Z Current cluster status: * Node List: * Node controller-1: UNCLEAN (offline) * Online: [ controller-0 controller-2 ] * RemoteOnline: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * Full List of Resources: * messaging-0 (ocf:pacemaker:remote): Started controller-0 * messaging-1 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * messaging-2 (ocf:pacemaker:remote): Started controller-0 * galera-0 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * galera-1 (ocf:pacemaker:remote): Started controller-0 * galera-2 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * Clone Set: rabbitmq-clone [rabbitmq]: * Started: [ messaging-0 messaging-1 messaging-2 ] * Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 ] * Clone Set: galera-master [galera] (promotable): * Promoted: [ galera-0 galera-1 ] * Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ] * Clone Set: redis-master [redis] (promotable): * redis (ocf:heartbeat:redis): Unpromoted controller-1 (UNCLEAN) * Promoted: [ controller-0 ] * Unpromoted: [ controller-2 ] * Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * Clone Set: haproxy-clone [haproxy]: * haproxy (systemd:haproxy): Started controller-1 (UNCLEAN) * Started: [ controller-0 controller-2 ] * Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 * stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-1 (UNCLEAN) Transition Summary: - * Fence (reboot) messaging-1 'resources are active and the connection is unrecoverable' + * Fence (reboot) messaging-1 'resources are active but connection is unrecoverable' * Fence (reboot) controller-1 'peer is no longer part of the cluster' * Stop messaging-1 ( controller-1 ) due to node availability * Move galera-0 ( controller-1 -> controller-2 ) * Stop galera-2 ( controller-1 ) due to node availability * Stop rabbitmq:2 ( messaging-1 ) due to node availability * Stop redis:0 ( Unpromoted controller-1 ) due to node availability * Move ip-172.17.1.14 ( controller-1 -> controller-2 ) * Move ip-172.17.1.17 ( controller-1 -> controller-2 ) * Move ip-172.17.4.11 ( controller-1 -> controller-2 ) * Stop haproxy:0 ( controller-1 ) due to node availability * Move stonith-fence_ipmilan-5254005bdbb5 ( controller-1 -> controller-2 ) Executing Cluster Transition: * Pseudo action: messaging-1_stop_0 * Pseudo action: galera-0_stop_0 * Pseudo action: galera-2_stop_0 * Pseudo action: redis-master_pre_notify_stop_0 * Pseudo action: stonith-fence_ipmilan-5254005bdbb5_stop_0 * Fencing controller-1 (reboot) * Pseudo action: redis_post_notify_stop_0 * Resource action: redis notify on controller-0 * Resource action: redis notify on controller-2 * Pseudo action: redis-master_confirmed-pre_notify_stop_0 * Pseudo action: redis-master_stop_0 * Pseudo action: haproxy-clone_stop_0 * Fencing messaging-1 (reboot) * Resource action: galera-0 start on controller-2 * Pseudo action: rabbitmq_post_notify_stop_0 * Pseudo action: rabbitmq-clone_stop_0 * Resource action: galera monitor=10000 on galera-0 * Pseudo action: redis_stop_0 * Pseudo action: redis-master_stopped_0 * Pseudo action: haproxy_stop_0 * Pseudo action: haproxy-clone_stopped_0 * Resource action: stonith-fence_ipmilan-5254005bdbb5 start on controller-2 * Resource action: galera-0 monitor=20000 on controller-2 * Resource action: rabbitmq notify on messaging-2 * Resource action: rabbitmq notify on messaging-0 * Pseudo action: rabbitmq_notified_0 * Pseudo action: rabbitmq_stop_0 * Pseudo action: rabbitmq-clone_stopped_0 * Pseudo action: redis-master_post_notify_stopped_0 * Pseudo action: ip-172.17.1.14_stop_0 * Pseudo action: ip-172.17.1.17_stop_0 * Pseudo action: ip-172.17.4.11_stop_0 * Resource action: stonith-fence_ipmilan-5254005bdbb5 monitor=60000 on controller-2 * Resource action: redis notify on controller-0 * Resource action: redis notify on controller-2 * Pseudo action: redis-master_confirmed-post_notify_stopped_0 * Resource action: ip-172.17.1.14 start on controller-2 * Resource action: ip-172.17.1.17 start on controller-2 * Resource action: ip-172.17.4.11 start on controller-2 * Pseudo action: redis_notified_0 * Resource action: ip-172.17.1.14 monitor=10000 on controller-2 * Resource action: ip-172.17.1.17 monitor=10000 on controller-2 * Resource action: ip-172.17.4.11 monitor=10000 on controller-2 Using the original execution date of: 2017-05-03 13:33:24Z Revised Cluster Status: * Node List: * Online: [ controller-0 controller-2 ] * OFFLINE: [ controller-1 ] * RemoteOnline: [ galera-0 galera-1 messaging-0 messaging-2 ] * RemoteOFFLINE: [ galera-2 messaging-1 ] * Full List of Resources: * messaging-0 (ocf:pacemaker:remote): Started controller-0 * messaging-1 (ocf:pacemaker:remote): Stopped * messaging-2 (ocf:pacemaker:remote): Started controller-0 * galera-0 (ocf:pacemaker:remote): Started controller-2 * galera-1 (ocf:pacemaker:remote): Started controller-0 * galera-2 (ocf:pacemaker:remote): Stopped * Clone Set: rabbitmq-clone [rabbitmq]: * Started: [ messaging-0 messaging-2 ] * Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 messaging-1 ] * Clone Set: galera-master [galera] (promotable): * Promoted: [ galera-0 galera-1 ] * Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ] * Clone Set: redis-master [redis] (promotable): * Promoted: [ controller-0 ] * Unpromoted: [ controller-2 ] * Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-2 * ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-2 * ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-2 * Clone Set: haproxy-clone [haproxy]: * Started: [ controller-0 controller-2 ] * Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 * stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-2 diff --git a/cts/scheduler/summary/remote-recover-unknown.summary b/cts/scheduler/summary/remote-recover-unknown.summary index 4f3d045284..e04e969988 100644 --- a/cts/scheduler/summary/remote-recover-unknown.summary +++ b/cts/scheduler/summary/remote-recover-unknown.summary @@ -1,139 +1,139 @@ Using the original execution date of: 2017-05-03 13:33:24Z Current cluster status: * Node List: * Node controller-1: UNCLEAN (offline) * Online: [ controller-0 controller-2 ] * RemoteOnline: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * Full List of Resources: * messaging-0 (ocf:pacemaker:remote): Started controller-0 * messaging-1 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * messaging-2 (ocf:pacemaker:remote): Started controller-0 * galera-0 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * galera-1 (ocf:pacemaker:remote): Started controller-0 * galera-2 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN) * Clone Set: rabbitmq-clone [rabbitmq]: * Started: [ messaging-0 messaging-1 messaging-2 ] * Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 ] * Clone Set: galera-master [galera] (promotable): * Promoted: [ galera-0 galera-1 ] * Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ] * Clone Set: redis-master [redis] (promotable): * redis (ocf:heartbeat:redis): Unpromoted controller-1 (UNCLEAN) * Promoted: [ controller-0 ] * Unpromoted: [ controller-2 ] * Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN) * Clone Set: haproxy-clone [haproxy]: * haproxy (systemd:haproxy): Started controller-1 (UNCLEAN) * Started: [ controller-0 controller-2 ] * Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 * stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-1 (UNCLEAN) Transition Summary: - * Fence (reboot) galera-2 'resources are in an unknown state and the connection is unrecoverable' - * Fence (reboot) messaging-1 'resources are active and the connection is unrecoverable' + * Fence (reboot) galera-2 'resources are in unknown state and connection is unrecoverable' + * Fence (reboot) messaging-1 'resources are active but connection is unrecoverable' * Fence (reboot) controller-1 'peer is no longer part of the cluster' * Stop messaging-1 ( controller-1 ) due to node availability * Move galera-0 ( controller-1 -> controller-2 ) * Stop galera-2 ( controller-1 ) due to node availability * Stop rabbitmq:2 ( messaging-1 ) due to node availability * Stop redis:0 ( Unpromoted controller-1 ) due to node availability * Move ip-172.17.1.14 ( controller-1 -> controller-2 ) * Move ip-172.17.1.17 ( controller-1 -> controller-2 ) * Move ip-172.17.4.11 ( controller-1 -> controller-2 ) * Stop haproxy:0 ( controller-1 ) due to node availability * Move stonith-fence_ipmilan-5254005bdbb5 ( controller-1 -> controller-2 ) Executing Cluster Transition: * Pseudo action: messaging-1_stop_0 * Pseudo action: galera-0_stop_0 * Pseudo action: galera-2_stop_0 * Pseudo action: redis-master_pre_notify_stop_0 * Pseudo action: stonith-fence_ipmilan-5254005bdbb5_stop_0 * Fencing controller-1 (reboot) * Pseudo action: redis_post_notify_stop_0 * Resource action: redis notify on controller-0 * Resource action: redis notify on controller-2 * Pseudo action: redis-master_confirmed-pre_notify_stop_0 * Pseudo action: redis-master_stop_0 * Pseudo action: haproxy-clone_stop_0 * Fencing galera-2 (reboot) * Fencing messaging-1 (reboot) * Resource action: galera-0 start on controller-2 * Pseudo action: rabbitmq_post_notify_stop_0 * Pseudo action: rabbitmq-clone_stop_0 * Resource action: galera monitor=10000 on galera-0 * Pseudo action: redis_stop_0 * Pseudo action: redis-master_stopped_0 * Pseudo action: haproxy_stop_0 * Pseudo action: haproxy-clone_stopped_0 * Resource action: stonith-fence_ipmilan-5254005bdbb5 start on controller-2 * Resource action: galera-0 monitor=20000 on controller-2 * Resource action: rabbitmq notify on messaging-2 * Resource action: rabbitmq notify on messaging-0 * Pseudo action: rabbitmq_notified_0 * Pseudo action: rabbitmq_stop_0 * Pseudo action: rabbitmq-clone_stopped_0 * Pseudo action: redis-master_post_notify_stopped_0 * Pseudo action: ip-172.17.1.14_stop_0 * Pseudo action: ip-172.17.1.17_stop_0 * Pseudo action: ip-172.17.4.11_stop_0 * Resource action: stonith-fence_ipmilan-5254005bdbb5 monitor=60000 on controller-2 * Resource action: redis notify on controller-0 * Resource action: redis notify on controller-2 * Pseudo action: redis-master_confirmed-post_notify_stopped_0 * Resource action: ip-172.17.1.14 start on controller-2 * Resource action: ip-172.17.1.17 start on controller-2 * Resource action: ip-172.17.4.11 start on controller-2 * Pseudo action: redis_notified_0 * Resource action: ip-172.17.1.14 monitor=10000 on controller-2 * Resource action: ip-172.17.1.17 monitor=10000 on controller-2 * Resource action: ip-172.17.4.11 monitor=10000 on controller-2 Using the original execution date of: 2017-05-03 13:33:24Z Revised Cluster Status: * Node List: * Online: [ controller-0 controller-2 ] * OFFLINE: [ controller-1 ] * RemoteOnline: [ galera-0 galera-1 messaging-0 messaging-2 ] * RemoteOFFLINE: [ galera-2 messaging-1 ] * Full List of Resources: * messaging-0 (ocf:pacemaker:remote): Started controller-0 * messaging-1 (ocf:pacemaker:remote): Stopped * messaging-2 (ocf:pacemaker:remote): Started controller-0 * galera-0 (ocf:pacemaker:remote): Started controller-2 * galera-1 (ocf:pacemaker:remote): Started controller-0 * galera-2 (ocf:pacemaker:remote): Stopped * Clone Set: rabbitmq-clone [rabbitmq]: * Started: [ messaging-0 messaging-2 ] * Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 messaging-1 ] * Clone Set: galera-master [galera] (promotable): * Promoted: [ galera-0 galera-1 ] * Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ] * Clone Set: redis-master [redis] (promotable): * Promoted: [ controller-0 ] * Unpromoted: [ controller-2 ] * Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-2 * ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-2 * ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0 * ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-2 * Clone Set: haproxy-clone [haproxy]: * Started: [ controller-0 controller-2 ] * Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ] * openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0 * stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0 * stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-2