(XCP-ng) Pool Recovery when Pool Master crashes

While researching XCP-ng’s High Availability feature, I realized that all Resource Pool operations depend on Pool Master. The reason is because Xen-Orchestra only connects and exchanges information directly with the Pool Master.

If the Pool Master has an inaccessibility problem (network drop, power outage, …), the remaining servers in the Pool will lose their connection to Xen Orchestra. At that time, the High Availability feature will not be able to work.

Here is my Resource Pool consisting of 3 servers, in which Pool Master is the server xcp-ng-17.

I tried shutting down the server xcp-ng-17 Go. Then back to Xen Orchestra, all servers disappeared from the system.

In the event that the Pool Master really has a problem that cannot be fixed soon, it is imperative to choose another server in the Pool as the Pool Master to re-establish Pool control on Xen Orchestra.

Here is a guide on how to recover the Resrouce Pool when the Pool Master crashes.

1. View pool and host information

SSH access to one of the remaining servers in the pool. Use the following command to view the host list

xe host-list params=uuid,name-label,host-metrics-live

Due to the server xcp-ng-17 I’m having problems, so the host-metrics-live parameter is displayed falsethe remaining 2 servers are working normally, showing true.

uuid ( RO)                 : edab6a91-4ea7-4adb-bdea-15c51e675d5f
           name-label ( RW): xcp-ng-18
    host-metrics-live ( RO): true


uuid ( RO)                 : e3cce885-f617-482e-b598-30a57df414a2
           name-label ( RW): xcp-ng-17
    host-metrics-live ( RO): false


uuid ( RO)                 : 8e52ca30-dcc0-4b28-af8c-ab009d2d5f10
           name-label ( RW): xcp-ng-16
    host-metrics-live ( RO): true

Check the pool information with the command xe pool-list

(09:44 xcp-ng-17 ~)# xe pool-list
uuid ( RO)                : c53ef1f0-9703-b811-a6d5-2f1b01a4203b
          name-label ( RW): My Pool
    name-description ( RW):
              master ( RO): e3cce885-f617-482e-b598-30a57df414a2
          default-SR ( RW): 57db5d2e-1ab0-0277-a15a-19eb6a06115a

Pool My Pool the master is xcp-ng-17 (uuid e3cce885-f617-482e-b598-30a57df414a2)

2. Change Pool Master

Type the following command to move the Pool Master to the new server in Pool

xe-toolstack-restart

Wait 2 minutes for the system to process and reboot. Then check the pool again with the command xe pool-listmaster has now been transferred xcp-ng-16 (uuid edab6a91-4ea7-4adb-bdea-15c51e675d5f)

(09:46 xcp-ng-17 ~)# xe pool-list
uuid ( RO)                : c53ef1f0-9703-b811-a6d5-2f1b01a4203b
          name-label ( RW): My Pool
    name-description ( RW):
              master ( RO): edab6a91-4ea7-4adb-bdea-15c51e675d5f
          default-SR ( RW): 57db5d2e-1ab0-0277-a15a-19eb6a06115a

3. Reconnect in Xen Orchestra

If the server xcp-ng-17 restart it will work under slave mode in pool.

• If the server xcp-ng-17 unrecoverable and must be reinstalled from scratch, we will remove it from the pool with the command xe pool-eject host-uuid=e3cce885-f617-482e-b598-30a57df414a2.

It is done. Resource Pool is now back to normal.