Breakdown undesired allocations by shard routing role

[nicktindall]

Add a count of undesired shard allocations broken down by shard routing role.

This is so that we can check whether there are undesired shard allocations in a specific tier in serverless. We don't have an explicit concept of tier (search/indexing) in the code, but shard routing role is a good enough proxy for that.

Relates: ES-12221

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[pxsalehi]

If we're going to collect it all in a map of roles, then shouldn't it also replace totalAllocations and undesiredAllocationsExcludingShuttingDownNodes since those would be the values that the "default" role would have in stateful? but then you get into what to do for the values returned from desired balance API in serverless, and there you'd have to sum up index_only and search_only I guess, and sprinkle a bunch of asserts since index/search and default should be mutually exclusive in this map. Another option is to just add the specific break downs we need here? index/searchTierAllocations and index/searchTierUndesiredAllocations? I think you'd need both total and undesired since we're going to need the ratio in the autoscaler.

[pxsalehi]

also how do we get these two stats (index/TierAllocations and indexTierUndesiredAllocations?) out of the balancer in the autoscaler? Is there a getter for these?

[nicktindall]

Another option is to just add the specific break downs we need here? index/searchTierAllocations and index/searchTierUndesiredAllocations?

As far as I can tell we have no concept of "tier" in the ES codebase. There is role, but we seem to stop short of defining that as equivalent to a tier. I don't quite understand why but I don't want to make assumptions about what role means in regards to tiers (see co.elastic.elasticsearch.stateless.allocation.StatelessAllocationDecider#canAllocateShardToNode, it's not even done there)

if we make two specific fields, we're baking in the assumption that

• Role.INDEX_ONLY means indexing tier, Role.SEARCH_ONLY means search tier and Role.DEFAULT means the only tier in a stateful deployment
• the existing set of roles are fixed

So we'd probably need to add assertions to trigger if these assumptions were no longer valid. I think having a map of role to counts (possibly Role -> record RoleStats(int total, int undesired)) at least leaves the interpretation of the roles to the context they're used in.

Maybe these are valid assumptions? Perhaps there is some history regarding the modelling?

[nicktindall]

also how do we get these two stats (index/TierAllocations and indexTierUndesiredAllocations?) out of the balancer in the autoscaler? Is there a getter for these?

I added a getter and put up a serverless PR to illustrate that flow