Difference between revisions of " Release Guide"

From Blazegraph
Jump to: navigation, search
(Checkpoint for new build process.)
Line 50: Line 50:
 
./scripts/release.sh 2.1.0
 
./scripts/release.sh 2.1.0
 
</pre>
 
</pre>
 +
 +
* Post Release
 +
** Verify the artifacts are on Maven Central
 +
** Verify the [https://blazegraph.github.io/database/apidocs/index.html Javadocs] are online with the correct version.
 +
  
 
* (LEGACY:  Now done in release.sh) Create a TAG for the release in GIT. ** The TAG is the immutable snapshot of the code as of the release.  Maintenance will be done on the BRANCH from which the release was issued.  The TAG should use the ''tags'' path and have the general pattern ''.../bigdata/'''tags'''/BIGDATA_RELEASE_X_Y_Z''.
 
* (LEGACY:  Now done in release.sh) Create a TAG for the release in GIT. ** The TAG is the immutable snapshot of the code as of the release.  Maintenance will be done on the BRANCH from which the release was issued.  The TAG should use the ''tags'' path and have the general pattern ''.../bigdata/'''tags'''/BIGDATA_RELEASE_X_Y_Z''.

Revision as of 04:51, 26 January 2016

This is meant to be a living document for performing releases of the bigdata project. See [1] for discussions about this page.

Project releases are done as branches in GIT. The branch name has the general form BIGDATA_RELEASE_X_Y_Z which corresponds to version X.Y.Z. This way the project name makes it into the tag and hence into the directory when it is checked out from GIT. Use a branch for the release since there is an expectation that we will have to support the branch.

Preconditions

  • Setup for Maven Central
  • notify developers that we are closing the branch down for a release and wait for the all clear.
  • merge all tickets into master
  • push snapshot to SF master
  • create release candidate branch. BLAZEGRAPH_X_Y_Z_RC
  • setup CI for RC branch
  • verify performance against standard benchmarks : SPARQL_Benchmarks
    • LUBM
    • BSBM
    • Govtrack (see bigdata-perf/CI/govtrack)
  • verify artifact deployments:
    • Verify executable jar
    • Verify WAR deployment (tomcat).
    • Verify Debian Deployer (Requires Ubuntu)
    • Verify RPM Deployer (Requires CentOS, RHEL, AMI)
    • Verify Tarball (unzip to local directory and run bin/blazegraph.sh start)
  • verify CI builds.
  • Review release notes in JIRA. Close or move any uncompleted tickets.
    • write (and commit) release notes in bigdata/src/releases (make sure the TAG URL is reflected in the release notes!). Copy and paste text version from JIRA and make any updates.
  • Update the tag in Github with the release notes and binary artifacts


  • Configured your git repository to be able to push to the remote sourceforge repository
$ git remote -v
origin	git@github.com:SYSTAP/bigdata.git (fetch)
origin	git@github.com:SYSTAP/bigdata.git (push)

git remote add sourceforge ssh://beebs@git.code.sf.net/p/bigdata/git

$ git remote -v
origin	git@github.com:SYSTAP/bigdata.git (fetch)
origin	git@github.com:SYSTAP/bigdata.git (push)
sourceforge	ssh://beebs@git.code.sf.net/p/bigdata/git (fetch)
sourceforge	ssh://beebs@git.code.sf.net/p/bigdata/git (push)

Release

  • Use the release.sh script with the new version number in the form X.Y.Z. This script builds, creates a new git branch and tag, updates the POM versions, deploys to maven central, pushes the javadocs, and pushes a new branch for the reverse merge.
./scripts/release.sh 2.1.0
  • Post Release
    • Verify the artifacts are on Maven Central
    • Verify the Javadocs are online with the correct version.


  • (LEGACY: Now done in release.sh) Create a TAG for the release in GIT. ** The TAG is the immutable snapshot of the code as of the release. Maintenance will be done on the BRANCH from which the release was issued. The TAG should use the tags path and have the general pattern .../bigdata/tags/BIGDATA_RELEASE_X_Y_Z.
git push sourceforge BIGDATA_RELEASE_X_Y_Z
git tag -a BIGDATA_RELEASE_X_Y_Z -m "Bigdata release X_Y_Z"
git push origin --tags
git push sourceforge --tags
  • Create a new directory on sourceforge for the release [2]. Upload the release notes into the directory on sourceforge and rename as "README" (they will be displayed at the bottom of the folder listing on sourceforge once you do this and refresh the page view).
  • Upload the artifacts generated by release.sh in the artifacts directory.
    • Download these artifacts and verify that the downloaded files are the same (md5/md5sum). There have been a lot of issues recently with sourceforge file uploads failing silently so you can not rely on the file being posted correctly the first time around.
    • "ant war" - generates bigdata.war (Standalone WAR deployment)
    • "ant executable jar" -- generates bigdata-X_Y_Z-bundled.jar; rename to bigdata-bundled.jar (executable jar deployment)
    • "ant deploy-artifact" - generates REL.bigdata.version.tgz (HA deployment)
    • (LEGACY) "ant javadoc" - generates javadoc. Upload to web site using rsync (see 'bigdata-project-release.txt' in dropbox for details), and verify the javadoc online.
      • (LEGACY) Rsync: rsync -qrzPl --perms --chmod=ugo+X,ugo+r -e ssh ant-build/docs www.blazegraph.com:/var/www/html/www.blazegraph.com/files/
    • (LEGACY) "ant ant-install-artifact" - generates a source code distribution. See 'bigdata-project-release.txt' in dropbox for important details on how and where to install this.
    • (LEGACY) Publish bigdata JAR to the systap maven repository:
      • NOTE: If the fastutil version has changed you will need to upload this to the maven repository. See comments in the POM.xml file.
      • (LEGACY) Change the version in pom.xml (bump version number, remove "-SNAPSHOT", do NOT commit).
      • (LEGACY) "mvn clean deploy" - deploys the new bigdata jar to the systap maven repository (alternatively try 'ant maven-deploy").
      • (LEGACY) Append "-SNAPSHOT" to the version pom.xml and commit to resume snapshot releases in CI.
    • Review the download page and edit as necessary.
    • TODO: Document process to snapshot user guide / wiki.
  • CAUTION: We maintain multiple releases. However, sourceforge will direct people to the most recently published release for which a download artifact exists. When publishing a release of a maintenance branch, you MUST (a) wait for the release to be distributed to enough mirrors that it will be assigned a download URL at which point it becomes the default download; and (b) go back and "touch" the most current release. Verify that the most current release is still the default download!!!

After actions

  • make sure that 'snapshot=true' and "#javadoc=" in build.properties and that pom.xml has -SNAPSHOT for the new version (and commit).
  • reverse merge to github (the release notes, build.properties and pom.xml are modified during the release)
    • Verify reverse merge, e.g., using Araxis Merge.
      • SF master, github master should be equal.
      • SF release tag, github release tag should be equal.
  • notify developers that the branch is open again.
  • blog it [3] and elsewhere.
  • post the release notes to the sourceforge project news (logged in as an admin, choose "News" from the tabs and then "Post New")
  • Update the website download link
  • Make sure the new files are set as the default download on Sourceforge
  • send out an email announcement (we maintain a mailing list).
  • send out a tinkerpop mailing list announcement
  • Update Wikipedia
  • update semanticweb.org (version and release date).
  • review W3 Wiki page for bigdata.
  • update the Roadmap

Maven

We publish maven artifacts for bigdata releases (starting with 1.2.3), for bigdata snapshots (from CI builds), and for bigdata dependencies that are not widely available in maven repositories. The maven repository for bigdata is:

Bigdata Maven Repository

There is a pom.xml file in the top-level of the bigdata project that controls the publication of the bigdata releases and snapshots. This pom must be updated any time a dependency version is updated or a new dependency is added. The pom also includes instructions (in XML comment blocks) on how to publish non-bigdata artifacts to the bigdata maven repository. This is used to bootstrap non-bigdata maven artifacts that are not otherwise widely available. In addition, any time we create a new release for one of the sub-projects (bigdata-ganglia, etc.), the new release artifact needs to be pushed to the maven repository.

Maven Central Guide

Maven Central Maven Deployment

PGP Key Setup

Here is a guide to moving existing PGP keys.

Encrypted Passwords

Setup your maven installation to have encrypted passwords Maven Password Encryption.