Skip to content

Simplify Gradle build logic for making releases. #700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
name: Publish package to GitHub Packages
on:
push:
branches:
tags:
- '*-rc*'
jobs:
publish:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
with:
fetch-tags: true
- uses: actions/setup-java@v1
with:
java-version: 11
- name: Publish package
run: ./gradlew publishAllPublicationsToGitHubPackagesRepository
run: ./gradlew publishAllPublicationsToGitHubPackagesRepository -PpublishVersion=${{ github.ref_name }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

86 changes: 34 additions & 52 deletions MAINTAINING.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,67 +54,59 @@ sonatypePassword=$token

These are done more often, in irregular intervals. They are not considered stable and may break your application, so be cautious when using them.

The process is equal to the making of a release candidate, but without making any tags:
The process is identical to making a release candidate, but without making any tags:

1. build and upload the `master-SNAPSHOT`:
1. Switch to the `master` branch:
```
git pull; git checkout master;
git switch master
```
1. proceed as described in [Release candidate - Upload to Sonatype](https://github.com/metafacture/metafacture-core/wiki/Maintainer-Guidelines#upload-to-sonatype)
1. Proceed as described in [Release candidate - Upload to Sonatype](#upload-to-sonatype), but omit the `publishVersion` parameter in order to build and upload the `master-SNAPSHOT`.

## Release candidate

*Release candidates should be tested by different people before releasing!*

### Prepare your release candidate

1. Make an rc-branch (necessary for Gradle to pick up the proper name):
1. Make an annotated signed tag for the release candidate (necessary for Gradle to pick up the proper name):
```
git checkout -b A.B.C-rcN
git tag -s metafacture-core-A.B.C-rcN
```
1. When prompted, add a sensible commit message. For instance, something like:
```
Release candidate 5.7.0
```
(leave out the ` metafacture-core-` to avoid later the git "error: src refspec ... matches more than one" when you push the annotated git tag for having a tag named the same as the branch is not possible)
1. Optionally, you can now test the build locally by invoking a Gradle target:
```
./gradlew assemble
./gradlew assemble -PpublishVersion=A.B.C-rcN
```

### Upload to Sonatype

1. Now you can build and upload the release candidate to Sonatype (note that `./gradlew` should inform you to make a "snapshot build". If the version doesn't end with `-SNAPSHOT` the artifacts will not be uploaded to Sonatype's snapshot repository!):
```
./gradlew clean; ./gradlew publishToMavenLocal; ./gradlew publishToSonatype
```
1. Go to [Sonatype's snapshot repository](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture) and type in the correct `Version` to see if it is already available there (can take some minutes). [Example for `5.5.1-rc1-SNAPSHOT`](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture~~5.5.1*~~)(if you don't see a `5.5.1-rc1-SNAPSHOT.jar` there check it at https://oss.sonatype.org/content/repositories/snapshots/org/metafacture/metafacture-biblio/5.5.1-rc1-SNAPSHOT/).
1. Make an annotated signed tag (it's important to do that _after_ uploading to Sonatype's snapshot repository because otherwise the `-SNAPSHOT` will not be appended to the release candidate thus will not land in `snapshot repository`):
1. Make sure to have a *clean* Git directory (otherwise the build will fail with the error message `Working copy has modifications`):
```
git tag -s metafacture-core-A.B.C-rcN
git status
```
1. Push the annotated signed tag to GitHub:
1. Now you can build and upload the release candidate to Sonatype (note that `./gradlew` should inform you to make a "snapshot build". If the version doesn't end with `-SNAPSHOT` the artifacts will not be uploaded to Sonatype's snapshot repository!):
```
git push origin tag metafacture-core-A.B.C-rcN
./gradlew publishToSonatype -PpublishVersion=A.B.C-rcN
```
1. Go to [Sonatype's snapshot repository](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture) and type in the correct `Version` to see if it is already available there (can take some minutes). [Example for `5.5.1-rc1-SNAPSHOT`](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture~~5.5.1*~~)(if you don't see a `5.5.1-rc1-SNAPSHOT.jar` there check it at https://oss.sonatype.org/content/repositories/snapshots/org/metafacture/metafacture-biblio/5.5.1-rc1-SNAPSHOT/).

### Publish to [GitHub Packages](https://github.com/orgs/metafacture/packages?repo_name=metafacture-core)

1. Push your properly named branch to GitHub. Notice the `-rc` part of the branch's name:
1. Push the annotated signed tag to GitHub:
```
git push origin A.B.C-rcN
git push origin tag metafacture-core-A.B.C-rcN
```
Because there is `fetch --no-tags` in `actions/checkout@v2` the `-SNAPSHOT` suffix will always be appended (in comparison to doing `./gradlew publishAllPublicationsToGitHubPackagesRepository` locally, which will find the `SCM tag`). The publishing to GitHub packages is triggered then.

If we don't want `-SNAPSHOT` we may want to remove the `-SNAPSHOT` in `build.gradle`:
```
if (grgit.branch.current().name.contains('-rc')) { ...
return "${grgit.branch.current().name}-SNAPSHOT"
}
```
The publishing to GitHub packages is triggered then via the GitHub Actions workflow.

Note that `Packages` is not the same as [`Releases`](https://github.com/metafacture/metafacture-core/releases).

### Consume the SNAPSHOT

1. See e.g. [Example for 5.5.1-rc1-SNAPSHOT](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture~~5.5.1*~~) how to configure the dependency.
1. Configure your build system to use Sonatype's Snapshot Repository to be able to load the dependencies of the release candidate (or master-SNAPSHOT).
1. See e.g. [5.5.1-rc1-SNAPSHOT](https://oss.sonatype.org/index.html#nexus-search;gav~org.metafacture~~5.5.1*~~) for how to configure the dependency.
1. Configure your build system to use Sonatype's Snapshot Repository to be able to load the dependencies of the release candidate (or `master-SNAPSHOT`).
For Maven update your `pom.xml` (after `</dependencies>`):
```xml
<repositories>
Expand Down Expand Up @@ -149,43 +141,33 @@ Note that `Packages` is not the same as [`Releases`](https://github.com/metafact

a) It's going from your local Git repository to Sonatype to Maven Central. Each station requires some manual actions so you can double check that everything is ok. b) A release should also be published to GitHub.

1. Switch to `master` branch. Merge the approved `rc` into master:
```
git switch master; pull --no-ff origin A.B.C-rcN; git push origin master
```

1. Make sure you have a signed tag locally:
1. Ensure that the approved release candidate tag exactly matches `master` (should output `metafacture-core-A.B.C-rcN`):
```
git show metafacture-core-A.B.C
git switch master; git describe --tags --exact-match
```
If it doesn't exist yet, create it:
1. Make an annotated signed tag for the release:
```
git tag -s metafacture-core-A.B.C
```
1. When prompted, add a sensible commit message. For instance, something like:
```
Release 5.7.0
```
1. Make sure you have that signed tag pushed to GitHub:
```
git ls-remote --tags origin
```
If it is missing, push it with:
1. Push the annotated signed tag to GitHub:
```
git push origin metafacture-core-A.B.C
git push origin metafacture-core-A.B.C
```
1. Now the tag is available at GitHub. You can manually choose to [draft a new release on GitHub](https://github.com/metafacture/metafacture-core/releases/new). The signed `*dist*` files must be uploaded manually. They are produced like this:
1. Make sure to have a *clean* Git directory (otherwise the build will fail with the error message `Working copy has modifications`):
```
./gradlew metafacture-runner:signArchive
git status
```
and can be found in `metafacture-core/metafacture-runner/build/distributions/` (don't mind the `Source code` for that is created by GitHub automatically).
1. Make sure to have a *clean* Git directory (otherwise only a SNAPSHOT will be built):
1. Now the tag is available on GitHub. You can manually choose to [draft a new release on GitHub](https://github.com/metafacture/metafacture-core/releases/new). The signed `*-dist.*` files must be uploaded manually. They are produced like this:
```
git status
./gradlew metafacture-runner:signArchive -PpublishVersion=A.B.C
```
1. Let the release be built and uploaded (the SCM tag will be detected and the release be built):
and can be found in `metafacture-runner/build/distributions/` (don't mind the `Source code` for that is created by GitHub automatically).
1. Now you can build and upload the release to Sonatype:
```
./gradlew clean; ./gradlew publishToMavenLocal; ./gradlew publishToSonatype
./gradlew publishToSonatype -PpublishVersion=A.B.C
```
1. Finally, go to [oss.sonatype.org](https://oss.sonatype.org), log in, check the [Staging Repositories](https://oss.sonatype.org/#stagingRepositories) and when it's finished, click on `Close`. If everything is good publish with clicking on `Release` - attention, because once published it can't be removed. The artifacts are uploaded to Maven Central (which may take some time. Have a look e.g. [metafacture-biblio](https://repo1.maven.org/maven2/org/metafacture/metafacture-biblio/) ). You can check that it's actually in the publishing pipeline by clicking on `Views/Repositories->Releases`, then type in the `Path lookup` field `org/metafacture/` and click on version.

1. Finally, go to [oss.sonatype.org](https://oss.sonatype.org), log in, check the [Staging Repositories](https://oss.sonatype.org/#stagingRepositories) and when it's finished, click on `Close`. If everything is good, publish with clicking on `Release` - attention, because once published it can't be removed. The artifacts are uploaded to Maven Central (which may take some time; have a look at e.g. [metafacture-biblio](https://repo1.maven.org/maven2/org/metafacture/metafacture-biblio/)). You can check that it's actually in the publishing pipeline by clicking on `Views/Repositories->Releases`, then type in the `Path lookup` field `org/metafacture/` and click on version.
125 changes: 36 additions & 89 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ subprojects {
}

ext.scmInfo = getScmInfo()
logger.lifecycle("Making ${scmInfo.type()} build: ${scmInfo.version}")

allprojects {
group = 'org.metafacture'
Expand Down Expand Up @@ -270,7 +271,7 @@ subprojects {
password = System.getenv("GITHUB_TOKEN")
}
}
if (scmInfo.isRelease() && project.hasProperty('releaseRepositoryUrl')) {
if (scmInfo.isRelease && project.hasProperty('releaseRepositoryUrl')) {
maven {
url = releaseRepositoryUrl
credentials {
Expand All @@ -284,7 +285,7 @@ subprojects {

signing {
required {
scmInfo.isRelease()
scmInfo.isRelease
}
sign publishing.publications.mavenArtifacts
}
Expand All @@ -310,114 +311,60 @@ nexusPublishing {
class ScmInfo {
def version
def tag
def isRelease

ScmInfo(version, tag, isRelease) {
if (!isRelease) version += '-SNAPSHOT'

ScmInfo(version, tag) {
this.version = version
this.tag = tag
this.isRelease = isRelease
}

def isRelease() {
return tag != null
def type() {
return isRelease ? 'release' : tag != null ? 'release candidate' : 'snapshot'
}
}

def getScmInfo() {
def tag = getGitTag()
if (tag != null) {
logger.lifecycle('SCM tag found. Making a release build')
version = extractVersionFromTag(tag)
} else {
logger.lifecycle('No SCM tag found. Making a snapshot build')
version = getSnapshotVersion()
}
logger.lifecycle("Version is $version")
return new ScmInfo(version, tag)
}
def version = null
def tag = null
def isRelease = false

def getSnapshotVersion() {
if (grgit == null) {
logger.warn('No Git repository found')
return 'non-scm-build-SNAPSHOT'
}
if (grgit.branch.current().fullName == 'HEAD') {
logger.lifecycle('Detached HEAD found')
return "commit-${grgit.head().id}-SNAPSHOT"
}
if (grgit.branch.current().name == 'master') {
logger.lifecycle('On master branch')
return 'master-SNAPSHOT'
}
if (grgit.branch.current().name.startsWith('releases/')) {
logger.lifecycle('Release branch found')
return "${extractVersionFromBranch(grgit.branch.current().name)}-SNAPSHOT"
if (project.hasProperty('publishVersion')) {
version = publishVersion

def matcher = version =~ /\d+(?:\.\d+)+(-rc\d+)?/
if (matcher.matches()) {
tag = getGitTag(version)
isRelease = matcher.group(1) == null
}
}
if (grgit.branch.current().name.contains('-rc')) {
logger.lifecycle('Release candidate branch found')
return "${grgit.branch.current().name}-SNAPSHOT"
else {
version = grgit != null ? grgit.branch.current().name : rootProject.name
}
logger.lifecycle('Feature branch found')
return "feature-${grgit.branch.current().name}-SNAPSHOT"

return new ScmInfo(version, tag, isRelease)
}

def getGitTag() {
def getGitTag(version) {
if (grgit == null) {
logger.warn('No Git repository found')
return null
throw new GradleException('No Git repository found')
}
if (!grgit.status().isClean()) {
logger.warn('Working copy has modifications. Will not look for tags')
return null
}
def tags = getAnnotatedTags()
if (tags.isEmpty()) {
logger.lifecycle('HEAD has no annotated tags')
return null
}
if (tags.size() > 1) {
logger.warn("HEAD has ${tags.size()} annotated tags")
return null
throw new GradleException('Working copy has modifications')
}
def tag = tags[0]
logger.lifecycle("Found annotated tag $tag.name")
return tag.name
}

def getAnnotatedTags() {
def tags = []
def tagName = "${rootProject.name}-$version"
for (tag in grgit.tag.list()) {
if (tag.commit == grgit.head()
&& tag.tagger != null
&& tag.dateTime != null) {
tags.add tag
if (
tag.name == tagName && // matching name
tag.commit == grgit.head() && // pointing at HEAD
tag.tagger != null && tag.dateTime != null // annotated
) {
return tagName
}
}
return tags
}

def static extractVersionFromTag(tag) {
Matcher matcher =
tag =~ /metafacture-core-(\d+\.\d+\.\d+(-[-A-Za-z0-9]+)?)/
if (!matcher.matches()) {
throw new GradleException("""\
Unsupported tag format: $tag
Could not extract version from tag. Supported tag formats are
metafacture-core-X.Y.Z and
metafacture-core-X.Y.Z-QUALIFIER
""".stripIndent())
}
return matcher.group(1)
}

def static extractVersionFromBranch(branch) {
Matcher matcher =
branch =~ /releases\/metafacture-core-(\d+\.\d+\.\d+(-[-A-Za-z0-9]+)?)/
if (!matcher.matches()) {
throw new GradleException("""\
Unsupported branch format: $branch
Could not extract version from branch. Supported branch formats are
releases/metafacture-core-X.Y.Z and
releases/metafacture-core-X.Y.Z-QUALIFIER
""".stripIndent())
}
return matcher.group(1)
throw new GradleException("HEAD has no matching annotated tag: $tagName")
}