Redundant virtual routers on same host

[summer-abacus]

We are running CloudStack 4.22.0 and setup a redundant virtual router. There are three kvm-hosts running. After setting one kvm host into maintenance mode, we see that the primary and backup router are on the same host.

(cloudstackdev) 🐀 > list routers listall=true filter=name,state,redundantstate,hostname,
count = 5
router:
+-----------+---------+----------------+----------+
|   NAME    |  STATE  | REDUNDANTSTATE | HOSTNAME |
+-----------+---------+----------------+----------+
| r-8-VM    | Running | UNKNOWN        | srv1-215 |
| r-75-VM   | Running | UNKNOWN        | srv1-215 |
| r-981-VM  | Running | BACKUP         | srv1-215 |
| r-982-VM  | Running | PRIMARY        | srv1-215 |
| r-1063-VM | Running | UNKNOWN        | srv1-213 |
+-----------+---------+----------------+----------+

(cloudstackdev) 🕊 > list hosts filter=name,state,type,version,
count = 5
host:
+-----------+-------+--------------------+----------+
|   NAME    | STATE |        TYPE        | VERSION  |
+-----------+-------+--------------------+----------+
| srv1-214  | Up    | Routing            | 4.22.0.0 |
| srv1-215  | Up    | Routing            | 4.22.0.0 |
| v-1058-VM | Up    | ConsoleProxy       | 4.22.0.0 |
| s-1059-VM | Up    | SecondaryStorageVM | 4.22.0.0 |
| srv1-213  | Up    | Routing            | 4.22.0.0 |
+-----------+-------+--------------------+----------+

Is there a configuration to avoid having the primary and backup router on the same host?

[DaanHoogland]

I don’t think there is @summer-abacus . Sounds like a good feature. cc @weizhouapache @sureshanaparti

[weizhouapache]

I don’t think there is @summer-abacus . Sounds like a good feature. cc @weizhouapache @sureshanaparti

Actually the feature already exists.

ACS tries to deploy redundant VRs in different pod, then different cluster, then different host. If all fail, then deploy VRs on the same host.

Maybe share more logs to see what caused it.

[summer-abacus]

At the moment it looks like reproducable.

migrate virtual router r-981-VM to srv1-214

list routers listall=true filter=name,state,redundantstate,hostname,
count = 5
router:
+-----------+---------+----------------+----------+
|   NAME    |  STATE  | REDUNDANTSTATE | HOSTNAME |
+-----------+---------+----------------+----------+
| r-8-VM    | Running | UNKNOWN        | srv1-215 |
| r-75-VM   | Running | UNKNOWN        | srv1-215 |
| r-981-VM  | Running | BACKUP         | srv1-214 |
| r-982-VM  | Running | PRIMARY        | srv1-215 |
| r-1063-VM | Running | UNKNOWN        | srv1-214 |
+-----------+---------+----------------+----------+

set srv1-214 into maintenance mode

r-981-VM will be migrated to srv1-215

(cloudstackmsu) 🐞 > list routers listall=true filter=name,state,redundantstate,hostname,
count = 5
router:
+-----------+---------+----------------+----------+
|   NAME    |  STATE  | REDUNDANTSTATE | HOSTNAME |
+-----------+---------+----------------+----------+
| r-8-VM    | Running | UNKNOWN        | srv1-215 |
| r-75-VM   | Running | UNKNOWN        | srv1-215 |
| r-981-VM  | Running | BACKUP         | srv1-215 |
| r-982-VM  | Running | PRIMARY        | srv1-215 |
| r-1063-VM | Running | UNKNOWN        | srv1-214 |
+-----------+---------+----------------+----------+

Same happens, when we migrate the r-981-VM to srv1-213. And there is also nothing running on this cluster at the moment:

 list hostsmetrics filter=name,cpuallocated,memoryallocated
host:
count = 3
+----------+--------------+-----------------+
|   NAME   | CPUALLOCATED | MEMORYALLOCATED |
+----------+--------------+-----------------+
| srv1-214 | 0.00%        |               0 |
| srv1-215 | 1.03%        |      8053063680 |
| srv1-213 | 2.16%        |     22011707392 |
+----------+--------------+-----------------+

I've attached the management server logs but I do not see anything unexpected in there.
mgmt-srv-router.log

[weizhouapache]

ah ok, understood.

the VRs are deployed to different hosts, but the pod/cluster/host ids are not considered during the host maintenance.