Adding artifact cache entries to capz/container-registry for calico images to avoid throttling during large cluster deployments

[marosset]

I tested these changes in a personal azure sub

This is related to kubernetes-sigs/cluster-api-provider-azure#5688 and after these cache changes are in place we'll update that PR to use the new ACR image references which should prevent us from seeing image throttling issues during large cluster deployements

[marosset]

/assign @jackfrancis @nojnhuh @willie-yao

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: marosset
Once this PR has been reviewed and has the lgtm label, please ask for approval from jackfrancis. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• infra/azure/terraform/capz/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[marosset]

/hold
I'm working on some extra validation... will update...

[marosset]

I pushed some updates to the templates but am stuck at

terraform plan -out main.tfplan
Acquiring state lock. This may take a few moments...

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: building account: unable to configure ResourceManagerAccount: subscription ID could not be determined and was not specified
│
│   with provider["registry.terraform.io/hashicorp/azurerm"],
│   on main.tf line 17, in provider "azurerm":
│   17: provider "azurerm" {
│
╵
Releasing state lock. This may take a few moments...

I saw in the docs that there is terraform state manage stored in an azure storage account.
Is the expectation that we pull that down to validate updates with terraform init / plan?

[nojnhuh]

Is that error actually unrelated to the state, and it's only having trouble picking up your az login context?

I saw in the docs that there is terraform state manage stored in an azure storage account.
Is the expectation that we pull that down to validate updates with terraform init / plan?

It's been forever since I've actually run any terraform, but yes I think the general pattern would be that one canonical set of state is established and shared for all future operations. I haven't heard that we've set up any remote backend for that though, but I haven't been keeping super close tabs here either. Using a storage account in Azure might introduce a chicken-and-egg problem if the idea is that we should be able to migrate to a new sub.

[marosset]

Is that error actually unrelated to the state, and it's only having trouble picking up your az login context?

I saw in the docs that there is terraform state manage stored in an azure storage account.
Is the expectation that we pull that down to validate updates with terraform init / plan?

It's been forever since I've actually run any terraform, but yes I think the general pattern would be that one canonical set of state is established and shared for all future operations. I haven't heard that we've set up any remote backend for that though, but I haven't been keeping super close tabs here either. Using a storage account in Azure might introduce a chicken-and-egg problem if the idea is that we should be able to migrate to a new sub.

Is there a reason why we are keeping the state in the azure storage account instead of github?
Is there sensitive info in there?

[marosset]

Is that error actually unrelated to the state, and it's only having trouble picking up your az login context?

I saw in the docs that there is terraform state manage stored in an azure storage account.
Is the expectation that we pull that down to validate updates with terraform init / plan?

It's been forever since I've actually run any terraform, but yes I think the general pattern would be that one canonical set of state is established and shared for all future operations. I haven't heard that we've set up any remote backend for that though, but I haven't been keeping super close tabs here either. Using a storage account in Azure might introduce a chicken-and-egg problem if the idea is that we should be able to migrate to a new sub.

[marosset]

Is that error actually unrelated to the state, and it's only having trouble picking up your az login context?

I am logged into the correct tenant and account (verified with az account show) so I don't think it an az login issue (but it might be)

[nojnhuh]

Is there a reason why we are keeping the state in the azure storage account instead of github?

If we're actually maintaining the state anywhere outside of our local workstations, that's news to me.

I don't see anything in the docs along the lines of "you should commit this to version control" and instead see mentions of other remote storage backends, which suggests adding it to the repo might be an anti-pattern.

Is there sensitive info in there?

The docs mention encrypting the state, but I don't think any of what we have contains any secrets that would warrant encryption. I'd obviously want to look at it closely before pushing it anywhere.