Difference between revisions of " Maven Notes"

From Blazegraph
Jump to: navigation, search
(Updated to move howto into a subsection.)
Line 5: Line 5:
 
The main goal of the "mavenization" process is to separate “heavy-weight” tests from the standard unit tests that a developer would run during a typical development cycle. Also included are tests that require some external infrastructure to be set up and torn down(e.g. a lookup service) around the execution of the tests. The term “integration tests” is not particularly accurate and is really a catch-all for anything that we don’t want to be part of the lighter-weight unit tests.  As the componentization of Bigdata proceeds, we will see some of these integration tests move back into their respective components but still be separated from the light-weight unit tests. Of course, the unit tests themselves will become part of the components they are meant to test. Maven’s separation of unit and integration tests into separate phases of the build lifecycle makes this fairly straightforward.
 
The main goal of the "mavenization" process is to separate “heavy-weight” tests from the standard unit tests that a developer would run during a typical development cycle. Also included are tests that require some external infrastructure to be set up and torn down(e.g. a lookup service) around the execution of the tests. The term “integration tests” is not particularly accurate and is really a catch-all for anything that we don’t want to be part of the lighter-weight unit tests.  As the componentization of Bigdata proceeds, we will see some of these integration tests move back into their respective components but still be separated from the light-weight unit tests. Of course, the unit tests themselves will become part of the components they are meant to test. Maven’s separation of unit and integration tests into separate phases of the build lifecycle makes this fairly straightforward.
  
 +
==Architecture==
 +
 +
For release 1.5.3, the existing project is broken into the artifacts below.  blazegraph-parent is the parent artifact that will build all of the dependencies and contains common configuration information.
 +
{|
 +
! style="border-style: solid; border-width: 20px"| Module
 +
! Description
 +
|-
 +
|    junit-ext
 +
|    Blazegraph extentions for unit tests.
 +
|-
 +
|    ctc-striterators
 +
|    Blazegraph CTC Striterators
 +
|-
 +
|    lgpl-utils
 +
|    Blazegraph LGPL Utils extensions
 +
|-
 +
|    dsi-utils
 +
|    Blazegraph DSI Utils extensions
 +
|-
 +
|    rdf-properties
 +
|    RDF properties common to multiple Blazegraph artifacts
 +
|-
 +
|    bigdata-util
 +
|    Utilities common to multiple Blazegraph artifacts
 +
|-
 +
|    bigdata-client
 +
|    Classes necessary build a Blazegraph client.
 +
|-
 +
|    bigdata-ganglia
 +
|    Blazegraph Ganglia package
 +
|-
 +
|    bigdata-gas
 +
|    Blazegraph Gather Apply Scatter (GAS) package
 +
|-
 +
|    bigdata-core
 +
|    bigdata, bigdata-rdf, bigdata-sails, and bigdata-gom source code.  Tests are in bigdata-core-test and bigdata-sails-test.  Future work will split this into separate artifacts as required.
 +
|-
 +
|    bigdata-jini
 +
|    Blazegraph HA and Scale-out packages requiring Apache River.  Includes tests.
 +
|-
 +
|    bigdata-war-html
 +
|    This version of the bigdata.war without lib files.
 +
|-
 +
|    bigdata-blueprints
 +
|    Blazegraph Embedded Server package
 +
|-
 +
|    bigdata-core-test
 +
|    Unit tests for bigdata, bigdata-rdf, and bigdata-gom
 +
|-
 +
|    bigdata-sails-test
 +
|    Unit tests for bigdata-sails
 +
|-
 +
|    bigdata-war
 +
|    bigdata.war distribution
 +
|-
 +
|    bigdata-jar
 +
|    Blazegraph executable jar for distribution.
 +
|}
 
==Maven Background and Help==
 
==Maven Background and Help==
 
===How to create a new maven project===
 
===How to create a new maven project===
Line 18: Line 76:
 
     <groupId>com.blazegraph</groupId>
 
     <groupId>com.blazegraph</groupId>
 
     <artifactId>blazegraph-parent</artifactId>
 
     <artifactId>blazegraph-parent</artifactId>
−
     <version>1.5.1-SNAPSHOT</version>
+
     <version>1.5.2-SNAPSHOT</version>
 
     <relativePath>../blazegraph-parent/pom.xml</relativePath>
 
     <relativePath>../blazegraph-parent/pom.xml</relativePath>
 
   </parent>
 
   </parent>

Revision as of 15:26, 8 August 2015

Here are a few notes on the ongoing effort to "Maven-ize" Bigdata. These notes apply to work currently being done in the maven_scaleout branch.

Overview

The main goal of the "mavenization" process is to separate “heavy-weight” tests from the standard unit tests that a developer would run during a typical development cycle. Also included are tests that require some external infrastructure to be set up and torn down(e.g. a lookup service) around the execution of the tests. The term “integration tests” is not particularly accurate and is really a catch-all for anything that we don’t want to be part of the lighter-weight unit tests. As the componentization of Bigdata proceeds, we will see some of these integration tests move back into their respective components but still be separated from the light-weight unit tests. Of course, the unit tests themselves will become part of the components they are meant to test. Maven’s separation of unit and integration tests into separate phases of the build lifecycle makes this fairly straightforward.

Architecture

For release 1.5.3, the existing project is broken into the artifacts below. blazegraph-parent is the parent artifact that will build all of the dependencies and contains common configuration information.

Module Description
junit-ext Blazegraph extentions for unit tests.
ctc-striterators Blazegraph CTC Striterators
lgpl-utils Blazegraph LGPL Utils extensions
dsi-utils Blazegraph DSI Utils extensions
rdf-properties RDF properties common to multiple Blazegraph artifacts
bigdata-util Utilities common to multiple Blazegraph artifacts
bigdata-client Classes necessary build a Blazegraph client.
bigdata-ganglia Blazegraph Ganglia package
bigdata-gas Blazegraph Gather Apply Scatter (GAS) package
bigdata-core bigdata, bigdata-rdf, bigdata-sails, and bigdata-gom source code. Tests are in bigdata-core-test and bigdata-sails-test. Future work will split this into separate artifacts as required.
bigdata-jini Blazegraph HA and Scale-out packages requiring Apache River. Includes tests.
bigdata-war-html This version of the bigdata.war without lib files.
bigdata-blueprints Blazegraph Embedded Server package
bigdata-core-test Unit tests for bigdata, bigdata-rdf, and bigdata-gom
bigdata-sails-test Unit tests for bigdata-sails
bigdata-war bigdata.war distribution
bigdata-jar Blazegraph executable jar for distribution.

Maven Background and Help

How to create a new maven project

See Maven in 5 minutes 

mvn archetype:generate -DgroupId=com.blazegraph -DartifactId=new-artifact -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

Edit new-artifact/pom.xml and add the lines: 

<parent>
    <groupId>com.blazegraph</groupId>
    <artifactId>blazegraph-parent</artifactId>
    <version>1.5.2-SNAPSHOT</version>
    <relativePath>../blazegraph-parent/pom.xml</relativePath>
  </parent>

Add the javadoc parameters and repositories: 

<reporting>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-javadoc-plugin</artifactId>
        <configuration>
          <stylesheetfile>${basedir}/src/main/javadoc/stylesheet.css</stylesheetfile>
          <show>public</show>
          <maxmemory>1000m</maxmemory>
          <author>true</author>
          <version>true</version>
          <doctitle><![CDATA[<h1>ctc-striterators</h1>]]></doctitle>
          <bottom> <![CDATA[<i>Copyright © 2006-2015 SYSTAP, LLC. All Rights Reserved.</i>
<script>
jQuery(document).ready(function(){
  jQuery('ul.sf-menu').superfish({
  pathClass: 'current',
  cssArrows: false
  });
});

(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-50971023-6', 'blazegraph.com');
ga('send', 'pageview');
</script>
]]></bottom>
        </configuration>
      </plugin>
    </plugins>
  </reporting>

  <repositories>
    <repository>
      <id>bigdata.releases</id>
      <url>http://www.systap.com/maven/releases/</url>
    </repository>
  </repositories>

How to run the integration tests

The bigdata-integ module is now referenced in the bigdata parent POM. If you run a build from the parent project, you’ll build bigdata-core, generate its artifacts including the deployment tarball, and run the integration tests. Note that the unit tests are currently not being run during the build, but this should change soon. 

mvn clean install

If you’ve already built bigdata-core and just want to run the integration tests, you can run this from the bigdata-integ directory: 


cd bigdata-integ
mvn clean integration-test (mvn clean install will also work)

How to not run the integration tests

If you don’t want to run the integration tests, it’s easy. You can:

  • Run the build from the bigdata-core project rather than the parent project.
  • OR from the parent project, add the –DskipITs argument on the command-line (this skips integration tests just as –DskipTests skips unit tests)
  • OR from the parent project, run mvn clean package instead of mvn clean install.

Since the integration-test build phase comes after the package build phase, the integration tests won’t be executed.

Creating a new project with archetype

 mvn -B archetype:generate \
 -DarchetypeGroupId=org.apache.maven.archetypes \
 -DgroupId=com.blazegraph.component \
 -DartifactId=blazegraph-component
Retrieved from "https://wiki.blazegraph.com/wiki/index.php?title=MavenNotes&oldid=1833"