<!-- Fragment document type declaration subset:
Arbortext, Inc., 1988-2008, v.4002
<!DOCTYPE SOLBOOK PUBLIC "-//Sun Microsystems//DTD SolBook 3.1//EN" [
<!ENTITY serverpath-threeway SYSTEM "figures/serverpath-threeway.eps" NDATA eps>
<!ENTITY % TCKentities SYSTEM "TCKentities.ent">
%TCKentities;
]>
-->
<?Pub UDT _bookmark _target>
<?Pub CX solbook(>
<chapter id="GBFWO"><title>Executing Tests</title>
<highlights>
<para>The &TCKShortName; uses the JavaTest harness to execute the tests in
the test suite. For detailed instructions that explain how to run and use
JavaTest, see the <citetitle>JavaTest User's Guide and Reference</citetitle> in
the documentation bundle.</para>
<para>This chapter includes the following topics:</para>
<itemizedlist remap="jumplist">
<listitem><para><olink targetptr="GCLRI">Overview</olink></para></listitem>
<listitem><para><olink targetptr="GBFUZ">Starting JavaTest</olink></para>
</listitem>
<listitem><para><olink targetptr="GBFWM">Running a Subset of the Tests</olink></para>
</listitem>
<listitem><para><olink targetptr="GCLRR">Running the JAX-WS TCK against the
Sun RI</olink></para></listitem>
<listitem><para><olink targetptr="GCLRZ">Running the JAX-WS TCK Against a
Vendor's Implementation</olink></para></listitem>
<listitem><para><olink targetptr="GCLQV">Running the Rebuilt JAX-WS TCK Against
the Sun RI</olink></para></listitem>
<listitem><para><olink targetptr="GBWUC">Testing Interoperability Between
a Vendor Implementation and the Sun Reference Implementation</olink></para>
</listitem>
<listitem><para><olink targetptr="GBFVK">Test Reports</olink></para></listitem>
</itemizedlist>
<note><para>The instructions in this chapter assume that you have installed
and configured your test environment as described in <olink type="custom-text" targetptr="GBFTP">Chapter 3</olink> and <olink type="custom-text" targetptr="GBFVV">Chapter 4</olink>, respectively.</para></note>
</highlights>
<sect1 id="GCLRI"><title>Overview</title>
<para>As explained in <olink type="custom-text" targetptr="GCLIZ">Appendix
B</olink>, the &TCKShortName; introduces the concept of rebuilding the TCK
tests. To provide the user an ability to run the TCK against the Vendor Implementation
and the Sun Reference Implementation, the use of the <literal>keywords</literal> feature
(in GUI mode) or option <command>-Dkeywords=</command><varname>value</varname> (in
command line mode) is provided.</para>
<para>By default, the TCK is configured to run all tests in both directions
for all the tests except the signature tests. Setting <literal>keywords</literal> allows
the user to change which tests will be run.</para>
<itemizedlist>
<listitem><para>Setting the <literal>keywords</literal> property to <literal>all</literal> (the
default) does not filter out any tests, and results in the Sun prebuilt tests
to be run in the forward direction, and the vendor rebuilt tests to be run
in the reverse direction.</para></listitem>
<listitem><para>Setting <literal>keywords</literal> to <literal>forward</literal> causes
the Sun prebuilt tests to be run in the forward direction only.</para>
</listitem>
<listitem><para>Setting <literal>keywords</literal> to <literal>reverse</literal> causes
the vendor rebuilt tests to be run in the reverse direction only.</para>
</listitem>
</itemizedlist>
<figure id="GCLRC"><title>&TCKShortName; Test Paths</title>
<mediaobject>
<imageobject><imagedata entityref="serverpath-threeway"></imageobject>
<textobject><simpara>&TCKShortName; Test Paths</simpara></textobject>
</mediaobject>
</figure>
<para>Refer to the <citetitle>JavaTest User's Guide</citetitle> and <citetitle>Reference
</citetitle> in the documentation bundle for information regarding how to
configure the <literal>keywords</literal> feature in GUI mode. For command
line mode, add the following to your command line <command>-Dkeywords=</command><varname>
value</varname>, where <varname>value</varname> is either <literal>forward</literal>, <literal>
reverse</literal>, or <literal>all</literal>.</para></sect1>
<sect1 id="GBFUZ"><title>Starting JavaTest</title>
<para>There are two general ways to run the &TCKShortName; using the JavaTest
harness software:</para>
<itemizedlist>
<listitem><para>Through the JavaTest GUI</para></listitem>
<listitem><para>From the command line in your shell environment</para>
</listitem>
</itemizedlist>
<note><para>The <command>&lt;ANT_HOME>/bin/ant</command> command referenced
in the following two procedures and elsewhere in this guide is a wrapper around
the Ant build tool, which is included in the &TCKShortName; bundle. The <filename>
build.xml</filename> file in <filename>&lt;TS_HOME>/bin</filename> contains
the various Ant targets for the &TCKShortName; test suite</para></note>
<task id="GBFWH"><title>To Start JavaTest in GUI Mode</title>
<procedure>
<step><para>Change to the <filename>&lt;TS_HOME>/bin</filename> directory
and execute the <command>&lt;ANT_HOME>/bin/ant gui</command> target:</para>
<literallayout><command>cd &lt;TS_HOME>/bin</command>
<command>&lt;ANT_HOME>/bin/ant gui</command></literallayout></step>
<step><para>Configure the <literal>keywords</literal> feature within the JavaTest
GUI.</para>
<note><para>Refer to the <citetitle>JavaTest User's Guide</citetitle> and <citetitle>
Reference</citetitle> in the documentation bundle for information regarding
how to configure the <literal>keywords</literal> feature in GUI mode.</para>
</note>
</step>
</procedure>
</task>
<task id="GBFVW"><title>To Start JavaTest in Command-Line Mode</title>
<procedure>
<step><para>Change to any subdirectory under <filename>&lt;TS_HOME>/src/com/sun/ts/tests
</filename>.</para></step>
<step><para>Start JavaTest using the following command:</para>
<literallayout><command>&lt;ANT_HOME>/bin/ant [-Dkeywords=forward|reverse|all] runclient
</command></literallayout>
<note><para>The <literal>-Dkeywords</literal> option is supported by the <command>
runclient</command> command in batch mode only, not in GUI mode.</para></note>
</step>
</procedure>
<example id="GCMCU">
<title>Running &TCKShortName; Signature Tests</title>
<para>To run the &TCKShortName; signature tests, enter the following commands:</para>
<screen>cd &lt;TS_HOME>/src/com/sun/ts/tests/signaturetest/jaxws
&lt;ANT_HOME>/bin/ant [-Dkeywords=forward|reverse|all] runclient</screen>
</example>
<example id="GCMBV">
<title>Running a Single Test Directory</title>
<para>To run a single test directory in the <literal>forward</literal> direction,
enter the following commands:</para>
<screen>cd &lt;TS_HOME>/src/com/sun/ts/tests/jaxws/api/javax_xml_ws/Dispatch
&lt;ANT_HOME>/bin/ant -Dkeywords=forward runclient</screen>
</example>
<example id="GCMCA">
<title>Running a Subset of Test Directories</title>
<para>To run a subset of test directories in the <literal>reverse</literal> direction,
enter the following commands:</para>
<screen>cd &lt;TS_HOME>/src/com/sun/ts/tests/jaxws/api 
&lt;ANT_HOME>/bin/ant -Dkeywords=reverse runclient</screen>
</example>
</task>
</sect1>
<sect1 id="GBFWM"><title>Running a Subset of the Tests</title>
<task id="GBFVT"><title>To Run a Subset of Tests in GUI Mode</title>
<procedure>
<step><para>Click <emphasis>Configure->Change Configuration->Tests to Run</emphasis> from
the JavaTest main menu.</para><para>The tabbed Configuration Editor dialog
is displa<?Pub Caret>yed.</para></step>
<step><para>Click <emphasis>Specify</emphasis> from the option list on the
left.</para></step>
<step><para>Select the tests you want to run from the displayed test tree,
and then click <emphasis>Done</emphasis>.</para><para>You can select entire
branches of the test tree, or use <keycap>Ctrl+Click</keycap> or <keycap>Shift+Click
</keycap> to select multiple tests or ranges of tests, respectively.</para>
</step>
<step><para>Click <emphasis>Save File</emphasis>.</para></step>
<step><para>Click <emphasis>Run Tests->Start</emphasis> to run the tests you
selected.</para><para>Alternatively, you can right-click the test you want
from the test tree in the left pane of the JavaTest main window, and choose <emphasis>
Execute These Tests</emphasis> from the popup menu.</para></step>
<step><para>Click <emphasis>Report->Create Report</emphasis>.</para></step>
<step><para>Specify the directory in which the JavaTest test harness will
write the report, and then click <emphasis>OK</emphasis>.</para><para>A report
is created, and you are asked whether you want to view it.</para></step>
<step><para>Click <emphasis>Yes</emphasis> to view the report. </para></step>
</procedure>
</task>
<task id="GBFWK"><title>To Run a Subset of Tests in Command-Line Mode</title>
<procedure>
<step><para>Change to the directory containing the tests you want to run.</para>
<para>For example, <filename>&lt;TS_HOME>/src/com/sun/ts/tests/jaxws/api</filename>.
</para></step>
<step><para>Start the test run by executing the following command:</para><para><command>
&lt;ANT_HOME>/bin/ant [-Dkeywords=forward|reverse|all] runclient</command></para>
<para>The tests in <filename>&lt;TS_HOME>/src/com/sun/ts/tests/jaxws/api</filename> and
its subdirectories are run in the direction(s) you specified with the <literal>keywords
</literal> option.</para></step>
</procedure>
</task>
<task id="GBFVL"><title>To Run a Subset of Tests in Batch Mode Based on Prior
Result Status</title>
<tasksummary>
<para>You can run certain tests in batch mode based on the test's prior run
status by specifying the <literal>priorStatus</literal> system property when
invoking <command>&lt;ANT_HOME>/bin/ant</command>.</para></tasksummary>
<procedure>
<step><para>Invoke <command>&lt;ANT_HOME>/bin/ant</command> with the <literal>priorStatus
</literal> property.</para><para>The accepted values for the <literal>priorStatus
</literal> property are any combination of the following:</para>
<itemizedlist>
<listitem><para><literal>fail</literal></para></listitem>
<listitem><para><literal>pass</literal></para></listitem>
<listitem><para><literal>error</literal></para></listitem>
<listitem><para><literal>notRun</literal></para></listitem>
</itemizedlist>
<para>For example, you could run all the &TechnologyShortName; tests with
a status of failed and error by invoking the following commands:</para>
<screen>cd &lt;TS_HOME>/src/com/sun/ts/tests/jaxws
&lt;ANT_HOME>/bin/ant -DpriorStatus="fail,error" \
[-Dkeywords=forward|reverse|all] runclient</screen>
<para>Note that multiple <literal>priorStatus</literal> values must be separated
by commas.</para></step>
</procedure>
</task>
</sect1>
<sect1 id="GCLRR"><title>Running the &TCKShortName; against the Sun RI</title>
<para>This test scenario is ensures that the configuration and deployment
of all the Sun prebuilt &TCKShortName; tests against the Sun Reference Implementation
are successful, and that the TCK is ready for compatibility testing against
the Vendor and Sun Implementations.</para>
<task id="GCLRT"><title>To Run the &TCKShortName; Against the Sun RI</title>
<procedure>
<step><para>Verify that you have followed the configuration instructions in <olink targetptr="GBFVU">Configuring Your Environment to Run the JAX-WS TCK Against
the Sun Reference Implementation</olink>.</para></step>
<step><para>Specify <literal>forward</literal> for the <literal>keywords</literal> option.
</para></step>
<step><para>Verify that you have completed the steps in <olink targetptr="GCLIW">
Deploying the JAX-WS TCK Prebuilt Archives</olink>.</para></step>
<step><para>Run the tests, as described in <olink targetptr="GBFUZ">Starting
JavaTest</olink> and, if desired, <olink targetptr="GBFWM">Running a Subset
of the Tests</olink>.</para></step>
</procedure>
</task>
</sect1>
<sect1 id="GCLRZ"><?Pub Dtl><title>Running the &TCKShortName; Against a Vendor's
Implementation</title>
<para>This test scenario is one of the compatibility test phases that all
Vendors must pass. This ensures that the Sun prebuilt &TCKShortName; tests
built against the Sun RI can be successfully run against the Vendors Implementation
(VI).</para>
<task id="GCLRM"><title>To Run the &TCKShortName; Against a VI</title>
<procedure>
<step><para>Verify that you have followed the configuration instructions in <olink targetptr="GCLHU">Configuring Your Environment to Run the JAX-WS TCK Against
the Vendor Implementation</olink>.</para></step>
<step><para>Specify <literal>forward</literal> for the <literal>keywords</literal> option.
</para></step>
<step><para>Verify that you have completed the steps in <olink targetptr="GCLIW">
Deploying the JAX-WS TCK Prebuilt Archives</olink></para></step>
<step><para>Run the tests, as described in <olink targetptr="GBFUZ">Starting
JavaTest</olink> and, if desired, <olink targetptr="GBFWM">Running a Subset
of the Tests</olink>.</para></step>
</procedure>
</task>
</sect1>
<sect1 id="GCLQV"><?Pub Dtl><title>Running the Rebuilt &TCKShortName; Against
the Sun RI</title>
<para>This test scenario is one of the compatibility test phases that all
Vendors must pass. This ensures that the &TCKShortName; tests that are rebuilt
using the Vendor's toolset can be successfully run against the Sun Reference
Implementation.</para>
<task id="GCLTJ"><title>To Run the Rebuilt &TCKShortName; Against the Sun
RI</title>
<procedure>
<step><para>Verify that you have followed the configuration instructions in <olink targetptr="GCLHF">Configuring Your Environment to Rebuild and Run the JAX-WS
TCK Against the Sun RI</olink>.</para></step>
<step><para>Refer to <olink type="custom-text" targetptr="GCLIZ">Appendix
B</olink>, <olink type="custom-text" targetptr="GCLIZ">Rebuilding the &TCKShortName; Using
the Vendor's Toolset</olink> to learn about rebuilding the &TCKShortName; tests.</para>
</step>
<step><para>Specify <literal>reverse</literal> for the <literal>keywords</literal> option.
</para></step>
<step><para>Verify that you have completed the steps in <olink targetptr="GCLIL">
Deploying the Rebuilt JAX-WS TCK Tests Against the Sun Reference Implementation</olink>.
</para></step>
<step><para>Run the tests, as described in <olink targetptr="GBFUZ">Starting
JavaTest</olink> and, if desired, <olink targetptr="GBFWM">Running a Subset
of the Tests</olink>.</para></step>
</procedure>
</task>
</sect1>
<sect1 id="GBWUC"><?Pub Dtl><title>Testing Interoperability Between a Vendor
Implementation and the Sun Reference Implementation</title>
<task id="GBWVW"><title>To Test Interoperability Between a VI and the Sun
RI</title>
<procedure>
<step><para>Specify <literal>all</literal> for the <literal>keywords</literal> option.
</para></step>
<step><para>Verify that you have completed the steps in <olink targetptr="GCLIW">
Deploying the JAX-WS TCK Prebuilt Archives</olink>.</para></step>
<step><para>Verify that you have completed the steps in <olink targetptr="GCLIL">
Deploying the Rebuilt JAX-WS TCK Tests Against the Sun Reference Implementation</olink></para>
</step>
<step><para>Run the tests, as described in <olink targetptr="GBFUZ">Starting
JavaTest</olink> and, if desired, <olink targetptr="GBFWM">Running a Subset
of the Tests</olink>.</para></step>
</procedure>
</task>
</sect1>
<sect1 id="GBFVK"><title>Test Reports</title>
<para>A set of report files is created for every test run. These report files
can be found in the report directory you specify. After a test run is completed,
the JavaTest harness writes HTML reports for the test run. You can view these
files in the JavaTest ReportBrowser when running in GUI mode, or in the web
browser of your choice outside the JavaTest interface.</para>
<para>To see all of the HTML report files, enter the URL of the <filename>report.html
</filename> file. This file is the root file that links to all of the other
HTML reports.</para>
<para>The JavaTest harness also creates a <filename>summary.txt</filename> file
in the report directory that you can open in any text editor. The <filename>summary.txt
</filename> file contains a list of all tests that were run, their test results,
and their status messages.</para>
<sect2 id="GBFWD"><title>Creating Test Reports</title>
<task id="GBFVH"><title>To Create a Test Report in GUI Mode</title>
<procedure>
<step><para>Click <emphasis>Report->Create Report</emphasis> from the JavaTest
main menu.</para><para>You are prompted to specify a directory to use for
your test reports. The default location is <filename>&lt;TS_HOME>/src/com/sun/ts/tests/signaturetests/jaxws
</filename>.</para></step>
<step><para>Specify the directory you want to use for your reports, and then
click <emphasis>OK</emphasis>.</para><para>Use the <emphasis>Filter</emphasis> drop-down
list to specify whether you want to generate reports for the current configuration,
all tests, or a custom set of tests.</para><para>You are asked whether you
want to view report now.</para></step>
<step><para>Click <emphasis>Yes</emphasis> to display the new report in the
JavaTest ReportBrowser.</para></step>
</procedure>
</task>
<task id="GBFVC"><title>To Create a Test Report in Command-Line Mode</title>
<procedure>
<step><para>Specify where you want to create the test report.</para>
<substeps>
<step><para>To specify the report directory from the command line at runtime,
use:</para>
<literallayout><command>&lt;ANT_HOME>/bin/ant -Dreport.dir="&lt;report_dir>"</command></literallayout>
<para>Reports are written for the last test run to the directory you specify.
The default location is <filename>&lt;TS_HOME>/src/com/sun/ts/tests/signaturetests/jaxws
</filename>.</para></step>
<step><para>To specify the default report directory, set the <literal>report.dir</literal> property
in <filename>&lt;TS_HOME>/bin/ts.jte</filename>.</para><para>For example, <literal>
report.dir="/home/josephine/reports"</literal>.</para></step>
<step><para>To disable reporting, set the <literal>report.dir</literal> property
to <literal>"none"</literal>, either on the command line or in <filename>ts.jte</filename>.
</para><para>For example:</para>
<literallayout><command>&lt;ANT_HOME>/bin/ant -Dreport.dir="none"</command></literallayout>
</step></substeps>
</step>
</procedure>
</task>
</sect2>
<sect2 id="GBFVB"><title>Viewing an Existing Test Report</title>
<task id="GBFVO"><title>To View an Existing Report in GUI Mode</title>
<procedure>
<step><para>Click <guimenu>Report->Open Report</guimenu> from the JavaTest
main menu.</para><para>You are prompted to specify the directory containing
the report you want to open.</para></step>
<step><para>Select the report directory you want to open, and then click <emphasis>
Open</emphasis>.</para><para>The selected report set is opened in the JavaTest
ReportBrowser.</para></step>
</procedure>
</task>
<task id="GBFWB"><title>To View an Existing Report in Command-Line Mode</title>
<procedure role="single-step">
<step><para>Use the Web browser of your choice to view the <filename>report.html</filename> file
in the report directory you specified from the command line or in <filename>ts.jte
</filename>.</para></step>
</procedure>
</task>
</sect2>
</sect1>
</chapter>
<?Pub *0000018983 0>
