Skip to content

Automate release process with GitHub Actions and webhook-based gateway updates #1634

Open
@sanity

Description

@sanity

Summary

Implement an automated release process that triggers on GitHub release creation, publishes to crates.io, and updates gateway servers without manual intervention.

Current Process

The current release process is manual and involves:

  1. Manually bumping versions in multiple Cargo.toml files
  2. Running cargo publish for each crate (freenet, fdev, freenet-stdlib)
  3. Downloading artifacts from GitHub Actions cross-compile workflow
  4. Using scripts/update-remote-gws.sh to deploy pre-compiled binaries to gateways

Proposed Solution

1. GitHub Release Workflow

Create a new workflow that triggers on GitHub release creation:

  • Automatically bumps versions based on release tag (e.g., v0.1.5)
  • Publishes to crates.io in correct dependency order:
    1. freenet-macros
    2. freenet-stdlib
    3. freenet
    4. fdev
  • Downloads cross-compiled artifacts from existing workflow
  • Triggers gateway updates

2. Webhook-triggered Gateway Updates

Implement a minimal webhook endpoint on gateways that:

  • Receives update notification from GitHub Actions via HTTPS
  • Pulls binaries from GitHub release (not from the webhook payload)
  • Validates binary checksums
  • Updates binaries using the existing systemd service management
  • Restarts services

Benefits:

  • No SSH keys stored in GitHub
  • Gateway pulls binaries from trusted source (GitHub releases)
  • Webhook only triggers the update, doesn't transmit binaries

3. Remove freenet-stdlib Submodule

Migrate from git submodule to crates.io-only dependency:

  • Remove the stdlib submodule
  • Use version dependencies everywhere
  • Use cargo update for testing pre-release versions
  • Simplifies dependency management and release process

4. cargo-binstall Support

Add cargo-binstall configuration to Cargo.toml files:

  • Point to GitHub releases for pre-compiled binaries
  • Simplifies gateway update process
  • Could be used as fallback update mechanism

Implementation Steps

  1. Create release.yml workflow that:

    • Triggers on release creation
    • Extracts version from tag
    • Updates all Cargo.toml versions
    • Runs tests
    • Publishes to crates.io
    • Waits for cross-compile workflow
    • Uploads binaries to GitHub release
    • Triggers gateway webhooks
  2. Implement gateway webhook service:

    • Minimal HTTP endpoint (e.g., using Python/Rust)
    • Authentication via shared secret
    • Downloads binaries from GitHub release
    • Verifies checksums
    • Updates and restarts services
  3. Remove stdlib submodule:

    • Update all Cargo.toml to use crates.io version
    • Update CI/development docs
    • Remove .gitmodules entry
  4. Add cargo-binstall metadata:

    • Configure in Cargo.toml
    • Test installation process

Benefits

  • Automated: No manual version bumping or publishing
  • Secure: No SSH keys in GitHub, gateways pull from trusted source
  • Reliable: Automated process reduces human error
  • Fast: Gateway updates happen immediately after release
  • Simple: Developers just create a GitHub release

Gateway Information

  • vega.locut.us: x86_64 EC2 instance
  • technic.locut.us (ziggy): arm64 Raspberry Pi (port 23)

Both have limited resources, so compilation on-device is not viable.

Questions for Review

  1. Should we use a simple Python webhook listener or implement in Rust?
  2. What authentication mechanism for webhooks? (HMAC with shared secret?)
  3. Should we keep any manual override capability?
  4. Version numbering scheme preferences?

Related to closed issue #1558 about stdlib submodule removal.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions