Release Guide

From Blazegraph
Revision as of 15:11, 25 September 2015 by Brad Bebee (Talk | contribs)

Jump to: navigation, search

This is meant to be a living document for performing releases of the bigdata project. See [1] for discussions about this page.

Project releases are done as branches in GIT. The branch name has the general form BIGDATA_RELEASE_X_Y_Z which corresponds to version X.Y.Z. This way the project name makes it into the tag and hence into the directory when it is checked out from GIT. Use a branch for the release since there is an expectation that we will have to support the branch.

Preconditions

  • notify developers that we are closing the branch down for a release and wait for the all clear.
  • merge all tickets into master
  • push snapshot to SF master
  • create release candidate branch. BLAZEGRAPH_X_Y_Z_RC
  • setup CI for RC branch
  • verify performance against standard benchmarks : SPARQL_Benchmarks
    • LUBM
    • BSBM
    • Govtrack (see bigdata-perf/CI/govtrack)
  • verify artifact deployments:
    • Verify executable jar
    • Verify WAR deployment (tomcat).
    • Verify "ant deploy-artifact" deployment (jetty).
  • verify CI builds.
  • Review release notes in JIRA. Close or move any uncompleted tickets.
    • write (and commit) release notes in bigdata/src/releases (make sure the TAG URL is reflected in the release notes!). Copy and paste text version from JIRA and make any updates.
  • Update the tag in Github with the release notes and binary artifacts


  • Configured your git repository to be able to push to the remote sourceforge repository
$ git remote -v
origin	git@github.com:SYSTAP/bigdata.git (fetch)
origin	git@github.com:SYSTAP/bigdata.git (push)

git remote add sourceforge ssh://beebs@git.code.sf.net/p/bigdata/git

$ git remote -v
origin	git@github.com:SYSTAP/bigdata.git (fetch)
origin	git@github.com:SYSTAP/bigdata.git (push)
sourceforge	ssh://beebs@git.code.sf.net/p/bigdata/git (fetch)
sourceforge	ssh://beebs@git.code.sf.net/p/bigdata/git (push)

Release

  • Bump the version in build.properties, make sure 'snapshot=false' and uncomment the "javadoc=" line so the javadoc will generate. Commit to GIT.
  • Create a TAG for the release in GIT. ** The TAG is the immutable snapshot of the code as of the release. Maintenance will be done on the BRANCH from which the release was issued. The TAG should use the tags path and have the general pattern .../bigdata/tags/BIGDATA_RELEASE_X_Y_Z.
git push sourceforge BIGDATA_RELEASE_X_Y_Z
git tag -a BIGDATA_RELEASE_X_Y_Z -m "Bigdata release X_Y_Z"
git push origin --tags
git push sourceforge --tags
  • Create a new directory on sourceforge for the release [2]. Upload the release notes into the directory on sourceforge and rename as "README" (they will be displayed at the bottom of the folder listing on sourceforge once you do this and refresh the page view).
  • (Re-)generate the release artifacts and upload into the directory on sourceforge. Be sure to click on the "i" icon and specify the necessary metadata for each release file (sourceforge will not update the "release" until you do this). DO NOT RENAME THESE DEPLOYMENT FILES. The names you give to SF are the actual names for the downloaded files....
    • Download these artifacts and verify that the downloaded files are the same (md5/md5sum). There have been a lot of issues recently with sourceforge file uploads failing silently so you can not rely on the file being posted correctly the first time around.
    • "ant war" - generates bigdata.war (Standalone WAR deployment)
    • "ant executable jar" -- generates bigdata-X_Y_Z-bundled.jar; rename to bigdata-bundled.jar (executable jar deployment)
    • "ant deploy-artifact" - generates REL.bigdata.version.tgz (HA deployment)
    • "ant javadoc" - generates javadoc. Upload to web site using rsync (see 'bigdata-project-release.txt' in dropbox for details), and verify the javadoc online.
    • "ant ant-install-artifact" - generates a source code distribution. See 'bigdata-project-release.txt' in dropbox for important details on how and where to install this.
    • Publish bigdata JAR to the systap maven repository:
      • NOTE: If the fastutil version has changed you will need to upload this to the maven repository. See comments in the POM.xml file.
      • Change the version in pom.xml (bump version number, remove "-SNAPSHOT", do NOT commit).
      • "mvn clean deploy" - deploys the new bigdata jar to the systap maven repository (alternatively try 'ant maven-deploy").
      • Append "-SNAPSHOT" to the version pom.xml and commit to resume snapshot releases in CI.
    • Review the download page and edit as necessary.
    • TODO: Document process to snapshot user guide / wiki.
  • CAUTION: We maintain multiple releases. However, sourceforge will direct people to the most recently published release for which a download artifact exists. When publishing a release of a maintenance branch, you MUST (a) wait for the release to be distributed to enough mirrors that it will be assigned a download URL at which point it becomes the default download; and (b) go back and "touch" the most current release. Verify that the most current release is still the default download!!!

After actions

  • make sure that 'snapshot=true' and "#javadoc=" in build.properties and that pom.xml has -SNAPSHOT for the new version (and commit).
  • reverse merge to github (the release notes, build.properties and pom.xml are modified during the release)
    • Verify reverse merge, e.g., using Araxis Merge.
      • SF master, github master should be equal.
      • SF release tag, github release tag should be equal.
  • for major releases (1.1.0, 1.2.0, etc).
    • create a new development branch (.../bigdata/branches/BIGDATA_RELEASE_X_Y_Z).
    • notify developers to switch to the new development branch.
    • update CI to run against the new development branch (create a new hudson job from the old development branch job).
  • notify developers that the branch is open again.
  • blog it [3] and elsewhere.
  • post the release notes to the sourceforge project news (logged in as an admin, choose "News" from the tabs and then "Post New")
  • send out an email announcement (we maintain a mailing list).
  • send out a tinkerpop mailing list announcement
  • Update Wikipedia
  • update semanticweb.org (version and release date).
  • review W3 Wiki page for bigdata.
  • update the Roadmap

Maven

We publish maven artifacts for bigdata releases (starting with 1.2.3), for bigdata snapshots (from CI builds), and for bigdata dependencies that are not widely available in maven repositories. The maven repository for bigdata is:

Bigdata Maven Repository

There is a pom.xml file in the top-level of the bigdata project that controls the publication of the bigdata releases and snapshots. This pom must be updated any time a dependency version is updated or a new dependency is added. The pom also includes instructions (in XML comment blocks) on how to publish non-bigdata artifacts to the bigdata maven repository. This is used to bootstrap non-bigdata maven artifacts that are not otherwise widely available. In addition, any time we create a new release for one of the sub-projects (bigdata-ganglia, etc.), the new release artifact needs to be pushed to the maven repository.