Release Process

Release process overview

Update: Since the 1.8.x series we have released from git master without branches. Skip to Performing a Release. We may or may not use branches again in the future.

Cutting a branch

  • When a release is ready to go, release manager (RM) puts forward a release plan as per standard Apache process, including dates. This gets VOTEd on by the committers. During this period master is still the only relevant source base.

  • As soon as a release is approved (or even before), RM should add the new version into JIRA as a target.

  • At the point where we would normally do the “code freeze” for a release, the RM cuts a branch named for the release. This branch is where the release candidates and releases will happen.

  • Ideally a release branch is only around for a week or maybe two before the release happens.

  • The only things that should EVER get checked into the release branch are - 1) bug fixes targeted at the release, 2) release-specific updates (documentation, SNAPSHOT removal, etc). In particular new functionality does not go here unless it is a solution to a JIRA report targeted at the release.

  • Normal development continues on master.

Dependencies and branches

  • Master should always be “cutting edge” and as such should usually be pointing at SNAPSHOT versions of all dependencies. This allows for continuous integration with our partner projects.

  • Soon after a release branch is cut, the RM is responsible for removing ALL dependencies on SNAPSHOT versions and replacing them with officially released versions. This change happens only on the release branch.

Managing change and issue resolution with a release branch

  • The RM goes through JIRA issues and sets “fix for” to point to both the next development version and the new branched release number for the fixes that are targeted for the release after the branch is cut.

  • In general, the assignee/coder fixes JIRA issues or makes other changes on master. If the JIRA issue is targeted at the release, or upon coder's discretion, they then merge the fix over to the release branch.

  • This way master is ALWAYS up-to-date, and we don't have to worry about losing fixes that have only been made on the release branch.

  • When the assignee resolves an issue, they confirm it's been fixed in both branches, if appropriate.

Checking changes into the branch

  • If bug fixes are needed later for a release which has long since happened (to fix user issues, etc), those fixes generally should also happen on master first assuming the problem still exists on master.

  • There are only two cases where we would ever check anything into the branch without first checking it into master. 1) Release specific items (release number references, release notes, removal of SNAPSHOTs), and 2) if master has moved on in some incompatible way.

Performing a release

Preparation

Verify that the code meets the basic requirements for being releasable:

  1. Check that the set of legal (legal/*.LICENSE) files corresponds to the set of third party JARs included in the binary distribution.

  2. Check that the apache-release profile works correctly and produces the required distributions. The profile can be executed as follows:

    mvn clean install -Papache-release
    

You may also execute a dry run of the release process: mvn release:prepare -DdryRun=true. In a dry run, the generated zip files will still be labled as SNAPSHOT. After this, you need to clean up using the following command: mvn release:clean

  1. Check that the Maven site can be generated and deployed successfully, and that it has the expected content.

    To generate the entire documentation in one place, complete with working inter-module links, execute the site-deploy phase (and check the files under target/staging). A quick and reliable way of doing that is to use the following command: mvn -Dmaven.test.skip=true clean package site-deploy

  2. Verify that src/site/site.xml includes the new release version in the Release Notes sidebar navigation. If missing, add an entry like:

    <item name="X.Y.Z" href="release-notes/X.Y.Z.html"/>
    

    Also create an empty placeholder release note for the next version under src/site/markdown/release-notes/ (e.g., 2.0.3.md). The distribution assembly references this file and the build will fail without it.

  3. The Spring Boot userguide samples under modules/samples/userguide/src/userguide/ inherit spring-boot-starter-parent, so they cannot be reactor modules and release:prepare does not rewrite their versions. Each declares a single <axis2.version> property, and the release plugin's preparationGoals/completionGoals invoke

    mvn antrun:run@sync-userguide-sample-versions
    

    which rewrites that property to the version being built. Nothing to do when releasing with release:prepare – but check the release commit contains the rewrite, and run the goal by hand if you are cutting a release any other way. Left unsynced, the shipped samples point at a SNAPSHOT that does not exist.

    Three things to know about it, all found by rehearsing release:prepare against a throwaway clone:

    • The execution is <inherited>false</inherited>, because preparationGoals runs across the whole reactor and an inherited copy resolves ${basedir} to each child module.
    • The target stages the rewritten poms with git add itself. The release plugin only transforms reactor poms, so without that the rewrite stays an unstaged working-tree change and the release commit and tag ship the old version. The commit phase takes whatever is staged, which is why staging is sufficient.
    • There is no completionGoals. Anything invoked there runs after the poms are bumped to the next SNAPSHOT and needs that SNAPSHOT's artifacts, which do not exist yet – the release aborts after tagging. So after a release the sample poms keep the released version rather than moving to the next SNAPSHOT. That is deliberate: master's samples then point at a real published artifact, and the next release rewrites them again.

    The whole body is skipped unless ${project.version} is a non-SNAPSHOT, which is what makes a -DdryRun=true run safe: a dry run never replaces pom.xml, so the goal sees the SNAPSHOT and does nothing. Without that guard a dry run would rewrite the samples off the last released version and stage them, and nothing reverts a dry run – the RM would be left with staged changes that any later commit would sweep up. It also means a dry run cannot verify the sync; only a real release:prepare transforms the poms first, which is why the check above is on the release commit.

    The same class of problem bites any sample kept out of the reactor: swagger-server sat at 2.0.1-SNAPSHOT through the whole 2.0.1 cycle because of it. It is a reactor module now, which is the preferred fix where the sample's parent allows it.

  4. Check that the source distribution is buildable.

  5. Check that the source tree is buildable with an empty local Maven repository.

If any problems are detected, they should be fixed on master (except for issues specific to the release branch) and then merged to the release branch.

Next update the release note found under src/site/markdown/release-notes. To avoid extra work for the RM doing the next major release, these changes should be done on master first and then merged to the release branch.

Pre-requisites

The following things are required to perform the actual release:

  • A PGP key that conforms to the requirements for Apache release signing. To make the release process easier, the passphrase for the code signing key should be configured in ${user.home}/.m2/settings.xml:

    <settings>
      ...
      <profiles>
        <profile>
          <id>apache-release</id>
          <properties>
            <gpg.passphrase><!-- key passphrase --></gpg.passphrase>
          </properties>
        </profile>
      </profiles>
      ...
    </settings>
    
  • The release process uses a Nexus staging repository. Every committer should have access to the corresponding staging profile in Nexus. To validate this, login to repository.apache.org and check that you can see the org.apache.axis2 staging profile. The credentials used to deploy to Nexus should be added to settings.xml:

    <servers>
      ...
      <server>
        <id>apache.releases.https</id>
        <username><!-- ASF username --></username>
        <password><!-- ASF LDAP password --></password>
      </server>
      ...
    </servers>
    

Release

In order to prepare the release artifacts for vote, execute the following steps:

If not yet done, export your public key and append it there.

If not yet done, also export your public key to the dev area and append it there.

The command to export a public key is as follows:

gpg –armor –export key_id

If you have multiple keys, you can define a ~/.gnupg/gpg.conf file for a default. Note that while ‘gpg –list-keys’ will show your public keys, using maven-release-plugin with the command ‘release:perform’ below requires ‘gpg –list-secret-keys’ to have a valid entry that matches your public key, in order to create ‘asc’ files that are used to verify the release artifcats. ‘release:prepare’ creates the sha512 checksum files.

  1. Start the release process using the following command - use ‘mvn release:rollback’ to undo and be aware that in the main pom.xml there is an apache parent that defines some plugin versions documented here.

    mvn release:prepare
    

    When asked for a tag name, accept the default value (in the following format: vX.Y.Z).

  2. Perform the release using the following command - though be aware you cannot rollback as shown above after that. That may need to happen if there are site problems further below. To start over, see the “Recovering from a failed release” section below.

    mvn release:perform
    
    The created artifacts i.e. zip files can be checked with, for example, 'sha512sum axis2-2.0.0-bin.zip' which should match the generated axis2-2.0.0-bin.zip.sha512 file. In that example, use 'gpg --verify axis2-2.0.0-bin.zip.asc axis2-2.0.0-bin.zip' to verify the artifacts were signed correctly.
    
  3. Login to Nexus and close the staging repository. For more details about this step, see here and here.

  4. Upload the source and binary distributions to the ASF dist area. The artifacts are in target/checkout/modules/distribution/target/. Before uploading, verify the GPG signatures and SHA512 checksums:

    cd target/checkout/modules/distribution/target
    for f in axis2-*.zip; do
      echo "=== $f ==="
      gpg --verify "$f.asc" "$f"
      sha512sum "$f" | cut -d' ' -f1
      cat "$f.sha512" | cut -d' ' -f1
    done
    

    Then copy to your local SVN checkout of the ASF dist area and commit:

    mkdir -p /path/to/dist/axis/axis2/java/core/X.Y.Z
    cp axis2-X.Y.Z-bin.zip axis2-X.Y.Z-bin.zip.asc axis2-X.Y.Z-bin.zip.sha512 \
       axis2-X.Y.Z-src.zip axis2-X.Y.Z-src.zip.asc axis2-X.Y.Z-src.zip.sha512 \
       axis2-X.Y.Z-war.zip axis2-X.Y.Z-war.zip.asc axis2-X.Y.Z-war.zip.sha512 \
       /path/to/dist/axis/axis2/java/core/X.Y.Z/
    cd /path/to/dist/axis
    svn add axis2/java/core/X.Y.Z
    svn commit -m "Stage Axis2 X.Y.Z release candidate"
    
  5. Create a staging area for the Maven site:

    git clone https://gitbox.apache.org/repos/asf/axis-site.git
    cd axis-site
    cp -r axis2/java/core/ axis2/java/core-staging
    git add  axis2/java/core-staging
    git commit -am "create core-staging dir as a prerequisite for the publish-scm plugin"
    git push
    
  6. Change to the target/checkout directory and prepare the site using the following commands:

    mvn site-deploy
    mvn scm-publish:publish-scm -Dscmpublish.skipCheckin=true
    

    Now go to the target/scmpublish-checkout directory (relative to target/checkout) and check that there are no unexpected changes to the site. Then commit the changes.

    Update: This plugin has a problem with specifying the remote core-staging dir, created above, with the git URL. See https://issues.apache.org/jira/browse/MSITE-1033 . For now, copy the output of the scmpublish-checkout dir listed above to the core-staging dir created earlier in this doc.

    The root dir of axis-site has a .asf.yaml file, referenced here at target/scmpublish-checkout/.asf.yaml, that is documented here.

  7. Start the release vote by sending a mail to java-dev@axis.apache.org. The mail should mention the following things:

If the vote passes, execute the following steps:

  1. Promote the artifacts in the staging repository. See here for detailed instructions for this step.

  2. Publish the distributions:

    svn mv https://dist.apache.org/repos/dist/dev/axis/axis2/java/core/x.y.z \
           https://dist.apache.org/repos/dist/release/axis/axis2/java/core/
    
  3. Publish the site:

    git clone https://gitbox.apache.org/repos/asf/axis-site.git
    cd axis-site
    git rm -r axis2/java/core
    git mv axis2/java/core-staging axis2/java/core
    git commit -am "Axis2 X.Y.Z site"
    git push
    

It may take several hours before everything has been synchronized. Before proceeding, check that

  • the Maven artifacts for the release are available from the Maven central repository;
  • the Maven site has been synchronized;
  • the distributions can be downloaded from the mirror sites.

Once everything is in place, send announcements to java-user@axis.apache.org (with copy to java-dev@axis.apache.org) and announce@apache.org. Since the two lists have different conventions, audiences and moderation policies, it is recommended to send the announcement separately to the two lists. Note that mail to announce@apache.org must be sent from an apache.org address and will always be moderated. The announcement sent to announce@apache.org also should include a general description of Axis2, because not everybody subscribed to that list knows about the project.

Post-release actions

  1. Update the DOAP file (etc/doap_Axis2.rdf) and add a new entry for the release.

  2. Update the status of the release version in JIRA.

  3. Remove old (archived) releases from https://dist.apache.org/repos/dist/release/axis/axis2/java/core/.

  4. Create an empty release note for the next release under src/site/markdown/release-notes.

Branch protection (.asf.yaml)

The repository has ASF-mandated branch protection rules in .asf.yaml that prevent force-push and branch deletion on master and release/* branches. This is a supply chain security measure applied across all ASF repositories (see PR #1200).

Under normal development, these rules are transparent — regular git push works as before. The protection only matters during release recovery (see below).

Recovering from a failed release

If mvn release:prepare or mvn release:perform fails and you need to start over, the recovery requires git push --force which is blocked by branch protection.

Recovery procedure:

  1. Temporarily disable force-push protection by editing .asf.yaml — remove or comment out restrict_force_push: true. Commit and push this change:

    # Edit .asf.yaml to remove restrict_force_push: true
    git add .asf.yaml
    git commit -m "Temporarily disable force-push protection for release recovery"
    git push
    

    Wait a minute for the GitHub ruleset to update.

  2. Perform the recovery:

    git reset --hard <last-commit-before-release-started>
    git push --force
    git push --delete origin vX.Y.Z
    
  3. Immediately restore protection by reverting the .asf.yaml change:

    git revert HEAD~1  # reverts the .asf.yaml disable commit
    git push
    

    Do not skip this step. The risk of leaving force-push unprotected (credential compromise, accidental history rewrite) is higher than the inconvenience of the toggle. If the .asf.yaml change is accidentally included in a release commit, force-push protection will be silently disabled until someone notices.

  4. Verify protection is restored: check the repository Settings → Rules → Rulesets page on GitHub to confirm force-push is blocked again.

Why not just leave force-push enabled? The ASF infrastructure team rolled out branch protection across all repositories after supply chain attacks (xz-utils, 2024) demonstrated the risk of history rewriting in open source projects. If a committer's credentials are compromised, force-push protection prevents an attacker from rewriting master to inject malicious code. The Axis2 project accepts this tradeoff: a small inconvenience during rare release failures in exchange for continuous protection against a real threat.

Known issue: The Maven site plugin has a bug with Git SCM URLs (MSITE-1033, status: IN PROGRESS) that can cause site deployment failures during the release process. This is a separate issue from branch protection but can compound the need for release restarts.