Re-enable sync_active_users_remote_repositories

[ericholscher]

A placeholder for the next steps on this work.

I also think we could just run this task on .com, for .org we don't really care much about permissions (SSO on .com only). Well, we do care about permissions, but only to allow users to import/link a remote repo to a project, which we should probably have a check done at the moment the remote repo is going to be linked to the project, so it's more accurate.

Originally posted by @stsewd in #12402 (comment)

[stsewd]

#12404 should help to not OOM the process when syncing users with lots of repos, but the task will still be slow as each repo needs to be processed, so the task may still be killed by celery I guess.

[stsewd]

We could separate this task in two:

• One that only refreshes access to repositories linked to organizations with SSO enabled.
• One that refreshes repositories in general, so users don't need to click on "sync repositories" when they want to create a new project from a repo. Which I still think we should be fine asking users to just click on refresh, but I'm also fine running this task less frequently.

In addition, we also need to check for permissions using the API when a remote repo is going to be linked to a project.

[humitos]

The plan of splitting this task in two makes sense to me 👍🏼

The sync repositories for better UX is becoming less and less important due to the migration to GH App, where the user will need to authorize the repository manually before adding it -- so, removing this task will make the UX to be similar for both cases OAuth/GH App; which is probably good.

[stsewd]

Well, it looks like we have a separate task for SSO

readthedocs.org/readthedocs/settings/base.py

Lines 674 to 678 in 02dc6ae

	"every-day-resync-sso-organization-users": {
	"task": "readthedocs.oauth.tasks.sync_remote_repositories_organizations",
	"schedule": crontab(minute=0, hour=4),
	"options": {"queue": "web"},
	},

But we do still sync all repositories from the users, instead of just the repositories that are linked to projects.

[stsewd]

Wanted to use https://docs.github.com/en/enterprise-cloud@latest/rest/collaborators/collaborators?apiVersion=2022-11-28 for syncing permissions, but it only works if we have write access to the repository :_ which is true for .com, so maybe we can still use it for SSO only. My other idea was just to query the repo with each user.

[stsewd]

With the recent improvements, the task now completes in 3h36min... I think there are couple of reasons this is still high:

• We are syncing more than 2.4K users (active user within 90 days). I think we can reduce this to just active users within 30 days, that gives us around 1K users.
• We are triggering syncs for GitHub app installations several times, since we do it per-user. We should skip those, as GH app repos are kept in sync with webhooks.