GridShib Automated Tests

From Globus

This page describes both current and planned automated tests for the GridShib software.


Contents

Buildbot Tests

A more comprehensive set of tests is being developed that will test not only compilation, but will test the functionality of independent components as well as the complete system (e.g. certificates with SAML extensions will be generated by GS-ST and be used to test a GT container with GS4GT installed).

These tests are in development, both current functionality and planned are listed here. The tests are configured using gridshib-buidbot.cfg, which currently must be manually installed in ~buildmaster/Buildbot on buildbot.ncsa.uiuc.edu.

Results from latest tests can be see on buildbot.ncsa.uiuc.edu and are emailed to the gridshib-dev email.

gt-nightly-build

Test Results
http://buildbot.ncsa.uiuc.edu:8010/builders/gt-nightly-HEAD-build
http://buildbot.ncsa.uiuc.edu:8010/builders/gt-nightly-globus_4_0_branch-build
http://buildbot.ncsa.uiuc.edu:8010/builders/gt-nightly-globus_4_2_branch-build
Test Script
http://viewcvs.globus.org/viewcvs.cgi/gridshib/buildbot/build-gt-nightly.sh?view=log
  • Script takes version as argument.
Description
These tests are not really tests per-say. All they do is pull down and build the nightly installers from CVS Head, 4.0 branch head and 4.2 branch head respectively. These builds are then used in subsequent tests. If it does fail, and dependent GridShib tests will just use the build from the previous evening. The downloaded tarballs are cached in ~/dists for future tests as well.
So what should we be doing if it fails? Unless we caused the failure some how, nothing, it's just informational. The intent of notifying us if it fails (and it will only send email if it fails) is so that if repeatedly does so, we'll know we're not building against an up-to-date version of what is in CVS.
Email is sent only on failure.
Tarball urls:

gridshib-samltools-cvs-build

Test Results
http://buildbot.ncsa.uiuc.edu:8010/builders/gridshib-samltools-cvs-build
Test Script
http://viewcvs.globus.org/viewcvs.cgi/gridshib/buildbot/build-saml-tools-dist.sh?view=log
Description
  1. grab CVS HEAD
  2. run 'ant create-dist'
    • Copy resulting distribution to ~/dists
  3. unpack the tarball and run 'ant install' on the result
  4. run 'ant run-tests' on the resulting "install" directory
Future Plans
Install into temporary directory and create one or more SAML-enhance proxy certificate using result to test functionality. (Perhaps this is already being done by ant tests?)

GS4GT-Builds

Test Results
Test Script
http://viewcvs.globus.org/viewcvs.cgi/gridshib/buildbot/build-gs4gt-dist.sh?view=log
  • Script takes GT version as argument.
Description
Builds GS4GT distribution from CVS head against GT 4.0.6, GT 4.0.7, GT 4.0.8, GT 4.2.0 and GT nightly tarballs (head-of-cvs, head-of-4.0-branch and head-of-4.2-branch, using results of gt-nightly-build test).
Notes
The 4.2.0 build is occasionally failing with "Connection Refused" errors trying to download schemas (see Bug 6365)
Future Plans
  1. Run 'ant deploy-all undeploy-all' as substitute for unit tests.
  2. Start up resulting container, create a SAML-enhanced proxy certificate and test various aspects of GS4GT policy enforcement.

GS4GT-gpt-build

Test Results
http://buildbot.ncsa.uiuc.edu:8010/builders/gs4gt-gpt-build-HEAD
http://buildbot.ncsa.uiuc.edu:8010/builders/gs4gt-gpt-build-globus_4_2_branch
Test Script
http://viewcvs.globus.org/viewcvs.cgi/gridshib/buildbot/test-gs4gt-gpt-build.sh?view=markup
  • Script takes GT version as argument.
Description
  1. Builds GS4GT distribution in GT nightly tarball using gpt-build as it would be build as part of a normal GT installation.
  2. Checks filelist generated by GPT against what is in the tarball/CVS to ensure undeploys of GS4GT will function properly.
  • This is a superset to the old NMI builds.
  • A detailed discussion of this test can be found in the email archives.
  • This is not run against the 4.0 branch as the 4.0 branch does not have GS4GT.
Future Plans
None

gridshib-ca-build

Test Results
http://buildbot.ncsa.uiuc.edu:8010/builders/gridshib-ca-v0_5_BRANCH-build
http://buildbot.ncsa.uiuc.edu:8010/builders/gridshib-ca-HEAD-build
Test Script
http://viewcvs.globus.org/viewcvs.cgi/gridshib/buildbot/gridshib-ca-test.sh?view=markup
Description
This script tests building the GridShib-CA (including java code) and running test which test the build, install and basic certificate issuance. It does not test any of the web/GUI functionality.
These two tests cover the 0.5 branch and CVS HEAD.
Future Plans
It would be great to actually deploy the web pages and scripts into Apache and test user-facing functionality.

Forcing Tests to be Run

To force the running of a single test, go to http://buildbot.ncsa.uiuc.edu:8010/one_line_per_build and click on the test you want to run and then click "Force Build" at the bottom of the page.

Avoid forcing a run of all tests at once as this really puts a load on the buildbot system (the build schedule staggers the builds) and it will also run non-scheduled tests that are in the buildbot configuration but not scheduled to run

Other todo items

  • Run tests using all supported java versions (1.4.2, 1.50, others?)

Why are these tests on Buildbot and not NMI Build and Test?

Right now it's just easier for to develop and debug them there. There are two technical changes that would have to be implemented before they could be moved to Build and Test:

  1. Work out the details of starting a GT container as non-root. From conversations with Charles I believe this is possible by just having the grid-mapfile map any incoming request to the user under which the container is running.
  2. Having cached GT installs around against which we can build. Right now I have tarballs for GT 4.0.6, GT 4.0.7 and the GT nightly build (rebuilt each night) that I just untar, build GS4GT against and then throw the result away. Having to pull down and rebuild these tarballs from scratch each night seems like a waste of resources.

GS4GT NMI Tests

Update as of September 26, 2008: This test has been subsumed by GS4GT-gpt-build tests and has again been removed from the running tests.

Update as of September 15, 2008: This has has been resumed on buildbot as we've determined it has coverage not present in the bother buildbot tests as previously suspected. The test output can be found here.

Update as of July 30, 2008: Functionality of this test has been subsumed by the buildbot tests and it is no longer being run automatically.

This test simulates the building of GS4GT as part of a complete GT install using the latest GT tarball and GS4GT from head-of-CVS (details can be found in this thread). The results of this test can be seen on the NMI B&T webpage and they are emailed to the gridshib-dev email list.

GS-CA NMI Tests

While not automated, you can find build tests for the the GridShib CA here: [1]

Retrieved from "http://dev.globus.org/wiki/GridShib_Automated_Tests"
Personal tools
Execution Projects
Information projects
Distribution Projects
Documentation Projects
Deprecated