You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
301 lines
13 KiB
301 lines
13 KiB
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=urf-8" />
|
|
<title>W3C QA - MUTAT developer information</title>
|
|
<meta name="Keywords"
|
|
content="qa, quality assurance, conformance, validity, test suite, test,tool,earl"
|
|
/>
|
|
<meta name="Description" content="W3C QA - MUTAT: test automating tool" />
|
|
<link rel="schema.DC" href="http://purl.org/dc" />
|
|
<meta name="DC.Subject" xml:lang="en" lang="en" content="test,tool,earl" />
|
|
<meta name="DC.Title" xml:lang="en" lang="en" content="MUTAT" />
|
|
<meta name="DC.Description.Abstract" xml:lang="en" lang="en"
|
|
content="MUTAT: test automating tool" />
|
|
<meta name="DC.Date.Created" content="2002-07-26" />
|
|
<meta name="DC.Language" scheme="RFC1766" content="en" />
|
|
<meta name="DC.Creator" content="Olivier Thereaux" />
|
|
<meta name="DC.Publisher"
|
|
content="W3C - World Wide Web Consortium - http://www.w3.org" />
|
|
<meta name="DC.Rights"
|
|
content="http://www.w3.org/Consortium/Legal/copyright-documents-19990405" />
|
|
<link rel="Stylesheet" href="/QA/2002/12/qa4.css" />
|
|
</head>
|
|
|
|
<body>
|
|
<!-- Header -->
|
|
|
|
<div id="Logo">
|
|
<a href="http://www.w3.org/"><img alt="W3C" src="/Icons/WWW/w3c_home" /></a>
|
|
<a href="http://www.w3.org/QA/"><img alt="QA" src="/QA/images/qa" width="161"
|
|
height="48" /></a> <!-- <div id="Header">Be strict to be cool</div> -->
|
|
|
|
|
|
<div>
|
|
<map name="introLinks" id="introLinks" title="Introductory Links">
|
|
|
|
<div class="banner">
|
|
<a class="bannerLink" title="W3C Activities" accesskey="A"
|
|
href="/Consortium/Activities">Activities</a> | <a class="bannerLink"
|
|
title="Technical Reports and Recommendations" accesskey="T"
|
|
href="/TR/">Technical Reports</a> | <a class="bannerLink"
|
|
title="Alphabetical Site Index" accesskey="S" href="/Help/siteindex">Site
|
|
Index</a> | <a class="bannerLink" title="Help for new visitors"
|
|
accesskey="N" href="/2002/03/new-to-w3c">New Visitors</a> | <a
|
|
class="bannerLink" title="About W3C" accesskey="B"
|
|
href="/Consortium/">About W3C</a> | <a class="bannerLink" title="Join W3C"
|
|
accesskey="J" href="/Consortium/Prospectus/Joining">Join W3C</a></div>
|
|
</map>
|
|
</div>
|
|
</div>
|
|
<!-- menuRight -->
|
|
|
|
<div id="Menu">
|
|
<p><a href="#about">About MUTAT</a><span class="dot">·</span><a
|
|
href="#Getting">Getting the Source</a><span class="dot">·</span><a
|
|
href="#files">The Files</a><span class="dot">·</span><a
|
|
href="#changelog">Change History</a><span class="dot">·</span></p>
|
|
<hr />
|
|
|
|
<p class="navhead">Nearby:</p>
|
|
|
|
<p><a href="bin/start">MUTAT demos</a><span class="dot">·</span> <a
|
|
href="/QA/"><abbr title="Quality Assurance">QA</abbr> Homepage</a><span
|
|
class="dot">·</span> <a href="/QA/#latest">Latest News</a><span
|
|
class="dot">·</span> <a href="/QA/#resources">QA Resources</a><span
|
|
class="dot">·</span> <a href="/QA/IG/">QA <abbr
|
|
title="Interest Group">IG</abbr></a><span class="dot">·</span> <a
|
|
href="/QA/WG/">QA <abbr title="Working Group">WG</abbr></a><span
|
|
class="dot">·</span> <a href="/QA/Agenda/">QA Calendar</a><span
|
|
class="dot">·</span></p>
|
|
</div>
|
|
<!-- content -->
|
|
|
|
<div id="Content">
|
|
<h1>MUTAT - Developer information</h1>
|
|
|
|
<p>MUTAT is a simple cgi script to demonstrate possible uses of <a
|
|
href="http://www.w3.org/2001/03/earl/">EARL</a>, and also use of RDF to
|
|
configure an application. It is designed to help perform Quality Assurance
|
|
evaluation tasks.</p>
|
|
|
|
<div>
|
|
<h2 id="use">Using it</h2>
|
|
|
|
<p>You can go straight to the <a href="bin/start">demonstration page</a> of
|
|
the testing tool which has been updated to be more user-friendly.</p>
|
|
|
|
<p>You can also make a URI that calls it directly, such as <a
|
|
href="bin/mutat?script_status=1&amp;TEST_url=http://www.w3.org/QA/Tools/MUTAT/wcag.n3">http://www.w3.org/QA/Tools/MUTAT/bin/mutat?script_status=1&TEST_url=http://www.w3.org/QA/Tools/MUTAT/wcag.n3</a></p>
|
|
</div>
|
|
|
|
<h2 id="about">About MUTAT</h2>
|
|
|
|
<p>QA experts have come up with a variety of solutions to handle entry and
|
|
retrieval of the data they need - sometimes little more than a paper and
|
|
pencil. The idea behind MUTAT was to demonstrate automating the process for
|
|
human-centered testing, and output the results in EARL. The script presents
|
|
the user with a series of questions: first a page of fill-in information on
|
|
their identity and testing environment, and then the actual tests. The tests
|
|
can be formatted either as a single page of text-based questions and links or
|
|
a succession of framed HTML pages, with form inputs for the results. Both the
|
|
initial information questions and the test questions are configured using an
|
|
RDF file. There is also a feature that allows filtering of test questions.</p>
|
|
|
|
<p>The script uses the Ginger Alliance's <a
|
|
href="http://www.gingerall.com/charlie/ga/xml/p_rdf.xml">RDF n3 parser</a>.
|
|
@@Is that still a sensible decision?</p>
|
|
|
|
<h3 id="source"><a name="Getting" id="Getting">Getting the Source
|
|
Code</a></h3>
|
|
|
|
<p>the code is available read-only, using the CVS server at W3C.</p>
|
|
<ul>
|
|
<li><p>Set the CVSROOT parameter:
|
|
<code>CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public</code><br />
|
|
with a unix shell (e.g bash) type :<br />
|
|
<code>me@myhost:~$ export
|
|
CVSROOT=:pserver:anonymous@dev.w3.org:/sources/public</code>.<br />
|
|
Or see the documentation for your CVS client.</p>
|
|
</li>
|
|
<li><p>login : <br />
|
|
<code>me@myhost:~$ cvs login</code><br />
|
|
(password:<code>anonymous</code>).</p>
|
|
</li>
|
|
<li><p>checkout the module <code>perl/modules/W3C/MUTAT</code>. <br />
|
|
with a unix shell (e.g bash) type :<br />
|
|
<code>me@myhost:~$ cvs checkout perl/modules/W3C/MUTAT</code></p>
|
|
</li>
|
|
</ul>
|
|
|
|
<p>You can also browse the corresponding <a
|
|
href="http://dev.w3.org/cvsweb/perl/modules/W3C/MUTAT">CVS repository</a>
|
|
with your favorite browser.</p>
|
|
|
|
<h3 id="todo">TODO</h3>
|
|
<ul>
|
|
<li>Update to new version of EARL as implemented by Hera, AccVerify,
|
|
axforms.</li>
|
|
<li>Document process for updating EARL versions in case the working group
|
|
restarts.</li>
|
|
<li>Restart discussion on creation and use of a public EARL store</li>
|
|
<li>Accept XML RDF. For instance it should be able to read the description
|
|
of WCAG at http://www.w3.org/2001/sw/Europe/200305/axforms/earlInst.rdf
|
|
at least.</li>
|
|
<li>Error-check on user input</li>
|
|
<li>Allow for completion of partial reports (should be possible with
|
|
working EARL store)</li>
|
|
<li>Document how to describe a test set more clearly</li>
|
|
</ul>
|
|
|
|
<h3><a name="files" id="files">The files @@incomplete</a></h3>
|
|
<dl>
|
|
<dt>mutat</dt>
|
|
<dd>A perl file that runs the mutat thing. Asks for a test file if called
|
|
with no parameters. There are parameters to make it ask for info like
|
|
tool etc (needs to be fixed to work for WCAG example), or to run the
|
|
tests themselves.</dd>
|
|
<dt>EARL.pm</dt>
|
|
<dd>Some PERL that allows for the creation of XML/RDF, and some specific
|
|
stuff for EARL. Needs commenting and documenting. Not sure that it is a
|
|
great thing to stick with - guess it's fine for PERL programmers.</dd>
|
|
<dt>RDF/Notation3/*</dt>
|
|
<dd>The RDF N3 parser from the Ginger Alliance. It probably makes sense
|
|
to replace this stuff over the medium term with an RDF/XML parser such
|
|
as Redland.</dd>
|
|
<dt>RDFcloud.pm</dt>
|
|
<dd>some stuff for collecting RDF assertions.</dd>
|
|
<dt>RDFtest.pm</dt>
|
|
<dd>some stuff for querying RDF.</dd>
|
|
<dt>create</dt>
|
|
<dd>A PERL script that helps you to create a new set of tests.
|
|
Unfortunately the key step, of adding an actual test case, seems not to
|
|
work at the moment. And it is not clear to me how it interprets the
|
|
other stuff.</dd>
|
|
</dl>
|
|
|
|
<h3>The text file of tests</h3>
|
|
|
|
<p>The text file is now in RDF. Namespace can be found <a href="ns">here</a>.
|
|
Right now the script only accepts files in n3, but there are tools to convert
|
|
between formats. The easiest way to figure out what's going on is to look at
|
|
the example file <a href="wcag.n3">wcag.n3</a> above for examples of all the
|
|
features. There is also a <a href="bin/create">script</a> to help automate
|
|
creation of test files. @@Check out script, documentation.</p>
|
|
|
|
<p>Features to play with:</p>
|
|
<ul>
|
|
<li><p>Add your own properties for the earl:Assertor or earl:TestSubject
|
|
(change the domain to the proper value)</p>
|
|
<pre>:thingy a rdf:Property;
|
|
rdfs:label "myThingy";
|
|
rdfs:domain earl:TestSubject; rdfs:range rdfs:Literal .</pre>
|
|
</li>
|
|
<li><p>Add default values to any property (sorry about the
|
|
re-declaration)</p>
|
|
<pre>earl:date a rdf:Property;
|
|
rdf:value "Wed 13 Mar 2002" .</pre>
|
|
</li>
|
|
<li><p>Add a limitless number of possible groupings to each TestCase to
|
|
filter which bits you would like to see - declare like below and make up
|
|
whatever values you'd like</p>
|
|
<pre>:priority a td:GroupProperty;
|
|
rdfs:label "priority" .</pre>
|
|
</li>
|
|
<li><p>Declare testcases with any properties you'd like to be copied into
|
|
the final report, even ones that aren't used by my script</p>
|
|
<pre>:test01 a earl:TestCase ;
|
|
rdfs:label "Test A";
|
|
:priority "C";
|
|
earl:id <HTTP: ATest.html Something www.w3.org> ;
|
|
td:text "This statement is false." .
|
|
:test02 a earl:TestCase ;
|
|
rdfs:label "Test Frog";
|
|
:priority "Argh";
|
|
earl:id <HTTP: somewhere some.domain> ;
|
|
td:text "My feet are green." .</pre>
|
|
</li>
|
|
<li><p>Make up your own choices for results to override earl's
|
|
passes/fails</p>
|
|
<pre>:C a td:ResultProperty; rdfs:label "Complete";
|
|
earl:validity earl:Pass;
|
|
earl:confidence earl:High .</pre>
|
|
</li>
|
|
</ul>
|
|
|
|
<h2 id="changelog">ChangeLog</h2>
|
|
<ul>
|
|
<li>2005-01-11: Started a new test set for SpecGL. Fixed the WCAG set to
|
|
ask for a page being tested. Tested create script and noted that it can't
|
|
add tests.</li>
|
|
<li>2005-01-03: reordered dates. Split off a page to start using it nicely
|
|
from this developer info. Started commenting the code.</li>
|
|
<li>2002-07-25 / 2002-07-26: finally moving the stuff to web/CVS space of
|
|
the QA activity at W3C.</li>
|
|
<li>2002-04-22: Recovered lost page from Google cache (yay!). Changed EARL
|
|
output to reflect change proposed in <a
|
|
href="http://lists.w3.org/Archives/Public/w3c-wai-er-ig/2002Mar/0017.html">w3c-wai-er-ig</a>
|
|
a month ago.</li>
|
|
<li>2002-04: Created script to automate creation of n3 test files</li>
|
|
<li>2002-03-12: Changed test format again... hopefully it's legal. Added
|
|
ability to read beginning questions in from earl spec, td spec, and test
|
|
file. Added ability to set default values with above. Added ability to
|
|
override earl's existing ResultProperties. Added ability to carry all
|
|
test information on so it can be printed in final earl report. Added
|
|
ability to create your own grouping properties for tests so they can be
|
|
filtered on any number of criteria. Abstracted a lot of stuff away from
|
|
mutat script and into odd data structures to holddata.</li>
|
|
<li>2002-03-04: Changed test format again in attempt to make it more
|
|
earl-like, added "group" property for tests that allows them to be
|
|
filtered, added ability to post to annotea database (but it'll give an
|
|
error if you do), added ability to post proper incomplete tests and added
|
|
a quick-out button for the frames view so you don't have to do all tests
|
|
until the end</li>
|
|
<li>2002-02-27: Made perl module to simplify EARL creation, added ability
|
|
to make comments</li>
|
|
<li>2002-02-25: Changed test question format to n3 (and use someone's n3
|
|
RDF parser to parse), changed script's data structure for tests and then
|
|
added object wrapper with pretty access methods</li>
|
|
<li>2002-02-18: Added access to <a href="http://jibbering.com/">Jim
|
|
Ley's</a> EARL database.</li>
|
|
<li>2002-02-15: Made source code public</li>
|
|
<li>2002-02-13: Created!</li>
|
|
</ul>
|
|
</div>
|
|
<!-- Footer -->
|
|
<hr />
|
|
|
|
<div class="disclaimer">
|
|
<a href="http://validator.w3.org/check/referer"><img
|
|
src="http://validator.w3.org/images/vxhtml10" alt="Valid XHTML 1.0!"
|
|
height="31" width="88" /></a>
|
|
<address class="author">
|
|
Document created Date: 2002-04-22 by <a
|
|
href="mailto:nadia@barbwired.com">Nadia Heninger</a><br />
|
|
Document moved to location Date: 2002-07-26 by <a
|
|
href="mailto:ot@w3.org">Olivier Thereaux</a><br />
|
|
Last modified $Date: 2008/01/14 15:55:07 $ by $Author:
|
|
charles $
|
|
</address>
|
|
|
|
<p class="policyfooter"><a rel="Copyright"
|
|
href="/Consortium/Legal/ipr-notice#Copyright">Copyright</a> © 2000-2003 <a
|
|
href="/"><acronym
|
|
title="World Wide Web Consortium">W3C</acronym></a><sup>®</sup> (<a
|
|
href="http://www.lcs.mit.edu/"><acronym
|
|
title="Massachusetts Institute of Technology">MIT</acronym></a>, <a
|
|
href="http://www.ercim.org/"><acronym
|
|
title="European Research Consortium for Informatics and Mathematics">ERCIM</acronym></a>,
|
|
<a href="http://www.keio.ac.jp/">Keio</a>), All Rights Reserved. W3C <a
|
|
href="/Consortium/Legal/ipr-notice#Legal_Disclaimer">liability</a>, <a
|
|
href="/Consortium/Legal/ipr-notice#W3C_Trademarks">trademark</a>, <a
|
|
rel="Copyright" href="/Consortium/Legal/copyright-documents">document use</a>
|
|
and <a rel="Copyright" href="/Consortium/Legal/copyright-software">software
|
|
licensing</a> rules apply. Your interactions with this site are in accordance
|
|
with our <a href="/Consortium/Legal/privacy-statement#Public">public</a> and
|
|
<a href="/Consortium/Legal/privacy-statement#Members">Member</a> privacy
|
|
statements.</p>
|
|
</div>
|
|
</body>
|
|
</html>
|