sun.com docs.sun.com | My Sun Worldwide Sites

|  | Products |  | Downloads |  | Service & Solutions |  | Support |  | Training |  | Research

|  | Previous | Contents |  | Next
Chapter 5 ### Executing Tests The JAX-WS TCK uses the JavaTest harness to execute the tests in the test suite. For detailed instructions that explain how to run and use JavaTest, see the JavaTest User's Guide and Reference in the documentation bundle. This chapter includes the following topics: 5.1 Overview 5.2 Starting JavaTest 5.3 Running a Subset of the Tests 5.4 Running the JAX-WS TCK against the Sun RI 5.5 Running the JAX-WS TCK Against a Vendor's Implementation 5.6 Running the Rebuilt JAX-WS TCK Against the Sun RI 5.7 Testing Interoperability Between a Vendor Implementation and the Sun Reference Implementation 5.8 Test Reports Note - The instructions in this chapter assume that you have installed and configured your test environment as described in Chapter 3 and Chapter 4, respectively. ### 5.1 Overview As explained in Appendix B, the JAX-WS TCK introduces the concept of rebuilding the TCK tests. To provide the user an ability to run the TCK against the Vendor Implementation and the Sun Reference Implementation, the use of the `keywords` feature (in GUI mode) or option `-Dkeywords=``value` (in command line mode) is provided. By default, the TCK is configured to run all tests in both directions for all the tests except the signature tests. Setting `keywords` allows the user to change which tests will be run. Setting the `keywords` property to `all` (the default) does not filter out any tests, and results in the Sun prebuilt tests to be run in the forward direction, and the vendor rebuilt tests to be run in the reverse direction. Setting `keywords` to `forward` causes the Sun prebuilt tests to be run in the forward direction only. Setting `keywords` to `reverse` causes the vendor rebuilt tests to be run in the reverse direction only. ###### Figure 5-1 JAX-WS TCK Test Paths Refer to the JavaTest User's Guide and Reference in the documentation bundle for information regarding how to configure the `keywords` feature in GUI mode. For command line mode, add the following to your command line `-Dkeywords=``value`, where `value` is either `forward`, `reverse`, or `all`. ### 5.2 Starting JavaTest There are two general ways to run the JAX-WS TCK using the JavaTest harness software: Through the JavaTest GUI From the command line in your shell environment Note - The `<ANT_HOME>/bin/ant` command referenced in the following two procedures and elsewhere in this guide is a wrapper around the Ant build tool, which is included in the JAX-WS TCK bundle. The `build.xml` file in `<TS_HOME>/bin` contains the various Ant targets for the JAX-WS TCK test suite #### To Start JavaTest in GUI Mode Change to the `<TS_HOME>/bin` directory and execute the `<ANT_HOME>/bin/ant gui` target:`cd <TS_HOME>/bin` `<ANT_HOME>/bin/ant gui` Configure the `keywords` feature within the JavaTest GUI. Note - Refer to the JavaTest User's Guide and Reference in the documentation bundle for information regarding how to configure the `keywords` feature in GUI mode. #### To Start JavaTest in Command-Line Mode Change to any subdirectory under `<TS_HOME>/src/com/sun/ts/tests`. Start JavaTest using the following command:`<ANT_HOME>/bin/ant [-Dkeywords=forward|reverse|all] runclient` Note - The `-Dkeywords` option is supported by the `runclient` command in batch mode only, not in GUI mode. ###### Example 5-1 Running JAX-WS TCK Signature Tests To run the JAX-WS TCK signature tests, enter the following commands:
    cd <TS_HOME>/src/com/sun/ts/tests/signaturetest/jaxws
    <ANT_HOME>/bin/ant [-Dkeywords=forward|reverse|all] runclient
###### Example 5-2 Running a Single Test Directory To run a single test directory in the `forward` direction, enter the following commands:
    cd <TS_HOME>/src/com/sun/ts/tests/jaxws/api/javax_xml_ws/Dispatch
    <ANT_HOME>/bin/ant -Dkeywords=forward runclient
###### Example 5-3 Running a Subset of Test Directories To run a subset of test directories in the `reverse` direction, enter the following commands:
    cd <TS_HOME>/src/com/sun/ts/tests/jaxws/api
    <ANT_HOME>/bin/ant -Dkeywords=reverse runclient
- ### 5.3 Running a Subset of the Tests #### To Run a Subset of Tests in GUI Mode Click Configure->Change Configuration->Tests to Run from the JavaTest main menu. The tabbed Configuration Editor dialog is displayed. Click Specify from the option list on the left. Select the tests you want to run from the displayed test tree, and then click Done. You can select entire branches of the test tree, or use Ctrl+Click or Shift+Click to select multiple tests or ranges of tests, respectively. Click Save File. Click Run Tests->Start to run the tests you selected. Alternatively, you can right-click the test you want from the test tree in the left pane of the JavaTest main window, and choose Execute These Tests from the popup menu. Click Report->Create Report. Specify the directory in which the JavaTest test harness will write the report, and then click OK. A report is created, and you are asked whether you want to view it. Click Yes to view the report. #### To Run a Subset of Tests in Command-Line Mode Change to the directory containing the tests you want to run. For example, `<TS_HOME>/src/com/sun/ts/tests/jaxws/api`. Start the test run by executing the following command: `<ANT_HOME>/bin/ant [-Dkeywords=forward|reverse|all] runclient` The tests in `<TS_HOME>/src/com/sun/ts/tests/jaxws/api` and its subdirectories are run in the direction(s) you specified with the `keywords` option. #### To Run a Subset of Tests in Batch Mode Based on Prior Result Status You can run certain tests in batch mode based on the test's prior run status by specifying the `priorStatus` system property when invoking `<ANT_HOME>/bin/ant`. Invoke `<ANT_HOME>/bin/ant` with the `priorStatus` property. The accepted values for the `priorStatus` property are any combination of the following: `fail` `pass` `error` `notRun` For example, you could run all the JAX-WS tests with a status of failed and error by invoking the following commands:
      cd <TS_HOME>/src/com/sun/ts/tests/jaxws
      <ANT_HOME>/bin/ant -DpriorStatus="fail,error" \
      [-Dkeywords=forward|reverse|all] runclient
Note that multiple `priorStatus` values must be separated by commas. ### 5.4 Running the JAX-WS TCK against the Sun RI This test scenario is ensures that the configuration and deployment of all the Sun prebuilt JAX-WS TCK tests against the Sun Reference Implementation are successful, and that the TCK is ready for compatibility testing against the Vendor and Sun Implementations. #### To Run the JAX-WS TCK Against the Sun RI Verify that you have followed the configuration instructions in 4.1 Configuring Your Environment to Run the JAX-WS TCK Against the Sun Reference Implementation. Specify `forward` for the `keywords` option. Verify that you have completed the steps in 4.6.2 Deploying the JAX-WS TCK Prebuilt Archives. Run the tests, as described in 5.2 Starting JavaTest and, if desired, 5.3 Running a Subset of the Tests. ### 5.5 Running the JAX-WS TCK Against a Vendor's Implementation This test scenario is one of the compatibility test phases that all Vendors must pass. This ensures that the Sun prebuilt JAX-WS TCK tests built against the Sun RI can be successfully run against the Vendors Implementation (VI). #### To Run the JAX-WS TCK Against a VI Verify that you have followed the configuration instructions in 4.2 Configuring Your Environment to Run the JAX-WS TCK Against the Vendor Implementation. Specify `forward` for the `keywords` option. Verify that you have completed the steps in 4.6.2 Deploying the JAX-WS TCK Prebuilt Archives Run the tests, as described in 5.2 Starting JavaTest and, if desired, 5.3 Running a Subset of the Tests. ### 5.6 Running the Rebuilt JAX-WS TCK Against the Sun RI This test scenario is one of the compatibility test phases that all Vendors must pass. This ensures that the JAX-WS TCK tests that are rebuilt using the Vendor's toolset can be successfully run against the Sun Reference Implementation. #### To Run the Rebuilt JAX-WS TCK Against the Sun RI Verify that you have followed the configuration instructions in 4.3 Configuring Your Environment to Rebuild and Run the JAX-WS TCK Against the Sun RI. Refer to Appendix B, Rebuilding the JAX-WS TCK Using the Vendor's Toolset to learn about rebuilding the JAX-WS TCK tests. Specify `reverse` for the `keywords` option. Verify that you have completed the steps in 4.6.3 Deploying the Rebuilt JAX-WS TCK Tests Against the Sun Reference Implementation. Run the tests, as described in 5.2 Starting JavaTest and, if desired, 5.3 Running a Subset of the Tests. ### 5.7 Testing Interoperability Between a Vendor Implementation and the Sun Reference Implementation #### To Test Interoperability Between a VI and the Sun RI Specify `all` for the `keywords` option. Verify that you have completed the steps in 4.6.2 Deploying the JAX-WS TCK Prebuilt Archives. Verify that you have completed the steps in 4.6.3 Deploying the Rebuilt JAX-WS TCK Tests Against the Sun Reference Implementation Run the tests, as described in 5.2 Starting JavaTest and, if desired, 5.3 Running a Subset of the Tests. ### 5.8 Test Reports A set of report files is created for every test run. These report files can be found in the report directory you specify. After a test run is completed, the JavaTest harness writes HTML reports for the test run. You can view these files in the JavaTest ReportBrowser when running in GUI mode, or in the web browser of your choice outside the JavaTest interface. To see all of the HTML report files, enter the URL of the `report.html` file. This file is the root file that links to all of the other HTML reports. The JavaTest harness also creates a `summary.txt` file in the report directory that you can open in any text editor. The `summary.txt` file contains a list of all tests that were run, their test results, and their status messages. #### 5.8.1 Creating Test Reports ##### To Create a Test Report in GUI Mode Click Report->Create Report from the JavaTest main menu. You are prompted to specify a directory to use for your test reports. The default location is `<TS_HOME>/src/com/sun/ts/tests/signaturetests/jaxws`. Specify the directory you want to use for your reports, and then click OK. Use the Filter drop-down list to specify whether you want to generate reports for the current configuration, all tests, or a custom set of tests. You are asked whether you want to view report now. Click Yes to display the new report in the JavaTest ReportBrowser. ##### To Create a Test Report in Command-Line Mode Specify where you want to create the test report. To specify the report directory from the command line at runtime, use:`<ANT_HOME>/bin/ant -Dreport.dir="<report_dir>"` Reports are written for the last test run to the directory you specify. The default location is `<TS_HOME>/src/com/sun/ts/tests/signaturetests/jaxws`. To specify the default report directory, set the `report.dir` property in `<TS_HOME>/bin/ts.jte`. For example, `report.dir="/home/josephine/reports"`. To disable reporting, set the `report.dir` property to `"none"`, either on the command line or in `ts.jte`. For example: `<ANT_HOME>/bin/ant -Dreport.dir="none"` #### 5.8.2 Viewing an Existing Test Report ##### To View an Existing Report in GUI Mode Click Report->Open Report from the JavaTest main menu. You are prompted to specify the directory containing the report you want to open. Select the report directory you want to open, and then click Open. The selected report set is opened in the JavaTest ReportBrowser. ##### To View an Existing Report in Command-Line Mode Use the Web browser of your choice to view the `report.html` file in the report directory you specified from the command line or in `ts.jte`.
|  | Previous | Contents |  | Next

Company Info Contact Terms of Use Privacy Copyright 1994-2009 Sun Microsystems, Inc.