custom domains - middleware and verification #2145

Soxasora · 2025-05-01T23:59:27Z

Description

Part of #1942
Focuses on custom domain insertion, providing a working form and verification process via pgboss.
Uses ACM with LocalStack to provide certificates bare CRUD simulation.

Screenshots

TBD

Additional Context

#1958 is being split to facilitate review and build greater stuff on top of it.
Doesn't contain Auth Sync or custom branding.

This PR provides:

custom domains
-- context
-- crud, resolvers, fragments
-- form with basic UI/UX
-- ACM certificates support
-- ACM certificate removal on STOPPED territories [context]; domain gets put on HOLD
middleware
-- custom domain detection via endpoint
-- doesn't apply referrals yet (no rewrites here)
domain verification
-- job every 5 minutes
-- robust AWS error handling
-- log every verification attempt and its steps
-- normalized schema for Domain, Record, Attempt, Certificate

Checklist

Are your changes backwards compatible? Please answer below:
Yes, everything is an addition. An exception is the way we handle S3 via localstack, now the container is called AWS and also allows ACM simulations other than S3.

On a scale of 1-10 how well and how have you QA'd this change and any features it might affect? Please answer below:
7, QA OK, domain verification handles errors correctly and follows its step, middleware correctly detects a custom domain and obtains its connected sub.

For frontend changes: Tested on mobile, light and dark mode? Please answer below:
Yes, it uses standard SN colors and styles that fits with theme changes.

Did you introduce any new environment variables? If so, call them out explicitly here:
An endpoint for localstack that can be contacted by our worker container
LOCALSTACK_ENDPOINT=http://aws:4566

DNS resolver for our DNS verification stage
DNS_RESOLVER=1.1.1.1

- ACM support - custom domains crud, resolvers, fragments - custom domains form, guidelines - custom domains context - domain verification every 5 minutes via pgboss - domain validation schema - basic custom domains middleware, to be completed - TODOs tracings

- CustomDomain -> Domain - DomainVerification table - CNAME, TXT, SSL verification types - WIP DomainVerification upsert

…ange status of a Record from its Attempt, multi-purpose dns verification

…tch as they can fail

- use DomainVerificationStatus enum for domains and records - adapt Territory Form UI to new schema - return 'records' as an object with its types - wip: prepare for attempts and certificate usage for prisma

…tion triggers

fix: - fix setDomain mutation transaction - fix schema typedefs enhance: - DNS records guidelines with flex-wrap for longer records cleanup: - add comments to worker - remove console.log on validation values

…main

huumn · 2025-05-14T20:00:05Z

https://github.com/stackernews/stacker.news/blob/c7321351bfef74dc2cfa9d0c2542bb3fbbc5db2c/docs/dev/custom-domains-base.md

So to QA this I need to use real DNS? I suspect we'll want to mock DNS somehow, like we mock ACM etc, going forward to make this trivial to iterate on.

huumn

I did a very casual high level first pass to get acquainted. It looks very good.

Thank you for separating it into its own PR. Also thanks to @ekzyis for his first review which I'm sure led to lots of improvements.

I left some questions and nits. I'll make another high level pass tomorrow. Then maybe again depending on when we can test this in a self-contained way.

huumn · 2025-05-14T23:42:53Z

api/acm/index.js

+  if (process.env.NODE_ENV === 'development') {
+    config.endpoint = process.env.LOCALSTACK_ENDPOINT
+  }


I think we can define this once in the file scoped config variable rather than checking it in every function

I feared AWS SDK (v2) reaction to undefined properties, but afaict it should ignore them, adjusted ^^

huumn · 2025-05-14T23:53:58Z

api/resolvers/domain.js

+              if (existing.certificate) {
+                await deleteDomainCertificate(existing.certificate.certificateArn)
+              }


Making network requests from inside an interactive tx is a bad idea. It consumes the db connection for the entire roundtrip.

I recommend deleting the certificate before entering the tx or after, depending on which makes sense.

Oh! You're right, totally missed this. Going to push changes in a bit :)

huumn · 2025-05-15T00:35:33Z

pages/api/domains/index.js

+import prisma from '@/api/models'
+
+// TODO: Authentication for this?
+// API Endpoint for getting all VERIFIED custom domains, used by a cachedFetcher
+export default async function handler (req, res) {
+  res.setHeader('Access-Control-Allow-Origin', '*')
+  res.setHeader('Access-Control-Allow-Methods', 'GET')
+  res.setHeader('Access-Control-Allow-Headers', 'Content-Type')
+
+  // Only allow GET requests
+  if (req.method !== 'GET') {
+    return res.status(405).json({ error: 'Method not allowed' })
+  }
+
+  try {
+    // fetch all VERIFIED custom domains from the database
+    const domains = await prisma.domain.findMany({
+      select: {
+        domainName: true,
+        subName: true
+      },
+      where: {
+        status: 'ACTIVE'
+      }
+    })
+
+    // map domains to a key-value pair
+    const domainMappings = domains.reduce((acc, domain) => {
+      acc[domain.domainName.toLowerCase()] = {
+        subName: domain.subName
+      }
+      return acc
+    }, {})
+
+    return res.status(200).json(domainMappings)
+  } catch (error) {
+    console.error('cannot fetch domains:', error)
+    return res.status(500).json({ error: 'Failed to fetch domains' })
+  }
+}


This endpoint just exists for the cachedFetcher call in lib/domains.js?

And that's because middleware can't import the prisma stuff, right?

Exactly that, I checked Prisma Edge but the changes it needs for it to work on the actual codebase, makes the endpoint much more bearable.

^{btw I just noticed that I didn't clean this endpoint, brb}

huumn · 2025-05-15T00:38:58Z

prisma/migrations/20250504202129_custom_domains_base/migration.sql

+-- !QUIRK: If someone else takes over the sub, the new guy needs to delete the domain
+CREATE OR REPLACE FUNCTION hold_domain_and_delete_certificate()


That is weird. I guess it's not clear to us whether the transfer is from an alt to another alt or to another person entirely.

I think we can actually just remove the custom domains settings and certificate if the owner is different on re-activation.

And, instead, on transfer, we retain settings

huumn · 2025-05-15T00:42:30Z

worker/territory.js

+      // make sure to delete the certificate from ACM if the sub is stopped, if we have it.
+      if (nextStatus === 'STOPPED' && sub.domain?.certificate?.certificateArn) {
+        await deleteDomainCertificate(sub.domain.certificate.certificateArn)
+      }
+
      await models.sub.update({
        include: { user: true },
        where: {
          name: subName
        },
        data: {
-          status: nextBillingWithGrace(sub) >= new Date() ? 'GRACE' : 'STOPPED',
+          status: nextStatus,


hmmm hard to tell if this is the right behavior unless we automatically re-add the cert when it's paid.

Hmm, a new certificate from ACM is fast and free, what's not really good (for the territory owner) is that they will be subjected again to ACM DNS validation.

Maybe we don't need to remove the certificate at all as long as the domain remains the same but this is also somewhat connected to what you said on the previous comment:

I guess it's not clear to us whether the transfer is from an alt to another alt or to another person entirely.

Soxasora added 2 commits May 1, 2025 18:38

Domains refactor, Domain Verification normalization

5e80c3f

- CustomDomain -> Domain - DomainVerification table - CNAME, TXT, SSL verification types - WIP DomainVerification upsert

Soxasora force-pushed the custom_domains_base branch from cd46aeb to 5e80c3f Compare May 2, 2025 23:33

Soxasora added 3 commits May 3, 2025 17:26

Domains normalization: Attempts, Records, Certificates

4d24845

Domain Verification worker adjusted to new schema; use triggers to ch…

b624c59

…ange status of a Record from its Attempt, multi-purpose dns verification

wip: Domain Verification worker, log all verification steps

89f1eb4

Soxasora closed this May 6, 2025

wip: clearer Domain Verification flow, surround AWS calls with try ca…

9d1c137

…tch as they can fail

Soxasora reopened this May 6, 2025

Soxasora added 9 commits May 6, 2025 14:51

Domain Verification schema updates

dc119a8

- use DomainVerificationStatus enum for domains and records - adapt Territory Form UI to new schema - return 'records' as an object with its types - wip: prepare for attempts and certificate usage for prisma

HOLD the domain and delete the certificate when a territory expires

67fb2c8

delete the certificate from ACM when we're about to STOP a territory

725ce81

Domain resolver refactor, use transactions, add comments

f3930f7

Stages for Domain Verification attempts logging, fix certificate dele…

01e319e

…tion triggers

separate ACM certificate requests and validation values

e132ad0

Domains UI/UX enhancements; core fixes to schema; general cleanup

cd9cb68

fix: - fix setDomain mutation transaction - fix schema typedefs enhance: - DNS records guidelines with flex-wrap for longer records cleanup: - add comments to worker - remove console.log on validation values

delete any existing domain verification jobs if we're updating the do…

9e96d7c

…main

Log AWS-related error messages; fix deleteCertificate recursion

82a71f5

Soxasora force-pushed the custom_domains_base branch from 775a966 to 82a71f5 Compare May 8, 2025 23:33

Soxasora and others added 2 commits May 8, 2025 18:47

fix missing await on async customDomainMiddleware

f95ab6a

Merge branch 'master' into custom_domains_base

4f49382

Soxasora marked this pull request as ready for review May 8, 2025 23:55

Soxasora and others added 2 commits May 8, 2025 19:04

hotfix: delete certificate from ACM also on domain removal

2382f3b

Merge branch 'master' into custom_domains_base

c732135

huumn reviewed May 15, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

custom domains - middleware and verification #2145

custom domains - middleware and verification #2145

Soxasora commented May 1, 2025 •

edited

Loading

huumn commented May 14, 2025 •

edited

Loading

huumn left a comment

huumn May 14, 2025

Soxasora May 15, 2025

huumn May 14, 2025

Soxasora May 15, 2025

huumn May 15, 2025

Soxasora May 15, 2025

huumn May 15, 2025

Soxasora May 15, 2025

huumn May 15, 2025

Soxasora May 15, 2025 •

edited

Loading

		-- !QUIRK: If someone else takes over the sub, the new guy needs to delete the domain
		CREATE OR REPLACE FUNCTION hold_domain_and_delete_certificate()

custom domains - middleware and verification #2145

Are you sure you want to change the base?

custom domains - middleware and verification #2145

Conversation

Soxasora commented May 1, 2025 • edited Loading

Description

Screenshots

Additional Context

Checklist

huumn commented May 14, 2025 • edited Loading

huumn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Soxasora May 15, 2025 • edited Loading

Choose a reason for hiding this comment

Soxasora commented May 1, 2025 •

edited

Loading

huumn commented May 14, 2025 •

edited

Loading

Soxasora May 15, 2025 •

edited

Loading