Markup Validator's test collection

Table of contents

This page lists a few test documents and test scenarios for the W3C Markup Validator make sure everything works fine after making changes to the validator. If you can think of interesting test cases that aren't on this page, let us know.

Valid Documents

Document types support

Below is a list of sample documents for a number of document types the validator is supposed to be supporting. The first validate link will attempt validation with the current instance, for test purposes. The v.w.o instance can be taken as reference/comparison.

• validate - with v.w.o - view: HTML 2.0
• validate - with v.w.o - view: HTML 2.0 Strict
• validate - with v.w.o - view: HTML 4.0 Strict
• validate - with v.w.o - view: HTML 4.0 Transitional
• validate - with v.w.o - view: HTML 4.0 Frameset
• validate - with v.w.o - view: HTML 4.01 Strict
• validate - with v.w.o - view: HTML 4.01 Transitional - (from the HTML 4.01 PR)
• validate - with v.w.o - view: HTML 4.01 Transitional - with iframe element (test for Bug 2329)
• validate - with v.w.o - view: ISO-HTML (aka ISO/IEC 15445:2000)
• validate - with v.w.o - view: XHTML 1.0 Strict
  validate - with v.w.o - view: XHTML 1.0 Strict (from XHTML Spec)
• validate - with v.w.o - view: XHTML Basic 1.0
• validate - with v.w.o - view: XHTML Basic 1.1
• validate - with v.w.o - view: XHTML 1.1
• validate - with v.w.o - view: XHTML/MathML2 (with FPI)
• validate - with v.w.o - view: MathML 2.0 (minimal test document)
• validate - with v.w.o - view: XHTML 1.1 served as application/xhtml+xml
• validate - with v.w.o - view: SMIL 1.0 minimal document
• validate - with v.w.o - view: SMIL 2.0
• validate - with v.w.o - view: SMIL 2.1
• validate - with v.w.o - view: SMIL 2.1 Mobile
• validate - with v.w.o - view: XHTML + MathML + SVG (from the 9 August 2002 WD)
• validate - with v.w.o - view: XHTML-Print
• validate - with v.w.o - view: SVG 1.0 Rec. minimal document
• validate - with v.w.o - view: SVG 1.1 Rec. minimal document
• validate - with v.w.o - view: SVG Tiny 1.1
• validate - with v.w.o - view: XHTML + RDFa

Document types outside the main catalogue

Below is a number of other types of documents that the validator does not have in its "hardcoded" catalogue, but should support anyway. All these documents should validate, without "pretty print" for the document type

• validate - with v.w.o - view: HTML4.0 plus blink ("Custom" SGML DTD)
• validate - with v.w.o - view: RDDL (with FPI)
• validate - with v.w.o - view: RDDL (no FPI)
• validate - with v.w.o - view: MusicXML (also test for Bug 5031)

Valid Doctypeless Documents

Some document types (SVG) do not require the presence of the DOCTYPE declaration, although a DTD can be used to validate. The validator can however determine the version to validate against by preparsing the root element and the presence of version, baseProfile attributes.

• validate - with v.w.o - view: Valid SVG 1.0, using a DOCTYPE
• validate - with v.w.o - view: Valid SVG 1.0, but no declared DOCTYPE. Version attribute present for auto-detection
• validate - with v.w.o - view: Valid SVG 1.1, using a DOCTYPE
• validate - with v.w.o - view: Valid SVG 1.1, no declared DOCTYPE. Version attribute present for auto-detection.
• validate - with v.w.o - view: Valid SVG Basic 1.1, using a DOCTYPE
• validate - with v.w.o - view: Valid SVG Basic 1.1, no declared DOCTYPE. Version and baseProfile attributes present for auto-detection.
• validate - with v.w.o - view: Valid SVG Tiny 1.1, using a DOCTYPE
• validate - with v.w.o - view: Valid SVG Tiny 1.1, no declared DOCTYPE. Version and baseProfile attributes present for auto-detection.
• validate - with v.w.o - view: Valid SVG 1.1, using a DOCTYPE. No version attribute (optional). Should use info given by doctype.
• validate - with v.w.o - view: Valid SVG 1.0, using a DOCTYPE. No version attribute (optional). Should use info given by doctype.
• validate - with v.w.o - view: SVG with neither DOCTYPE nor version attribute. The validator may use a default.

Valid Documents, with Warnings

• validate - with v.w.o - view: Reference to a non-existing ID (opensp message type X, See bug 2342).
• validate - with v.w.o - view: Non-SGML Chars
• validate - with v.w.o - view: HTML 4.01 Document with Ampersand as data
  (OK in SGML, not XML :see Bug798: Ampersand as data in XHTML)
• validate - with v.w.o - view: document served as text/html, using a custom DTD and for which parse mode is impossible to determine via generic heuristics (triggers warning W06)
• validate - with v.w.o - view - sample HTML 4 document using the (valid, but problematic) shorttags features

Valid Documents served with a wrong media type

• validate - with v.w.o - view: HTML 4.01 Strict served with an XHTML mime type
• validate - with v.w.o - View SVG 1.0 Rec. minimal document served as text/html
• validate - with v.w.o - view: XHTML-Print served as text/html
• validate - with v.w.o - view: MathML 2.0 as text/html

Bug Regression Tests: Valid documents

• validate - with v.w.o - view: XHTML+Math+SVG document with non-unix newlines, (See Bug 3992: non-unix newlines confuse the parser)
• validate - with v.w.o - view: XHTML with named entities on a line with colons and numbers, caused bogus error report in validator 0.8.0 (See Bug 4892)

Invalid Documents

Miscellaneous invalid docs

• validate - with v.w.o - view: XHTML1, bogus element (foo element not in DTD)
• validate - with v.w.o - view: XHTML1, bogus attribute
• validate - with v.w.o - view: XHTML1, bogus element (uppercase A element not in DTD), tends to cascade errors about each attribute
• validate - with v.w.o - view: us-ascii xhtml document with a latin1 character
• validate - with v.w.o - view: Invalid document (img without alt, invalid per).
  Also has non-sgml character (testing for warning output)
• validate - with v.w.o - view: invalid frames document (noframes in forbidden location). Also test for absence of newlines
• validate - with v.w.o - view: HTML 4.01 Strict - with iframe element (test for Bug 2329)
• validate - with v.w.o - view: XHTML1, blank first line (should be marked as invalid)
• validate - with v.w.o - view: Bug 798: Ampersand as data in XHTML (OK in SGML, not XML)
• validate - with v.w.o - view XHTML with bogus XML Decl, served as application/xhtml+xml (missing final "?")
• validate - with v.w.o - view XHTML with bogus XML Decl, served as text/html (missing final "?")
• validate - with v.w.o - view bogus FPI #1: lowercase "doctype".
• validate - with v.w.o - view: SMIL 2.0 with bogus xmlns and missing space between attributes
• validate - with v.w.o - view: SMIL 2.0 with bogus xmlns, xml ill-formedness and bogus elements (used to test sorting of errors)
• validate - with v.w.o - view: XHTML 1.0 missing space between attributes (Bug 2689)

Various often-invalid site

The following sites/pages are often invalid, but do not rely on the results.

• Yahoo!
• Microsoft
• Netscape
• XML 1.0

Missing Doctype

See also Docs that should be Well Formed for examples of doctype-less but well-formed XML documents.

• validate - with v.w.o - view: XHTML, no DOCTYPE, no xmlns (thus invalid)
• validate - with v.w.o - view: XML, no DOCTYPE, not well-formed (marked as "invalid XML" - FIXME should be reworded)
• validate - with v.w.o - view: SGML-ish document without html root. Served as text/html. No DOCTYPE. Correct SGML syntax. Would pass as well-formed XML if it were XML...
• validate - with v.w.o - view: SGML-ish document without html root. Served as text/html. No DOCTYPE. Bogus SGML syntax.

DTD-Valid but not conformant

For those, we will need to change the message, or hordes of SGML zombies will come down upon us ;)...

• validate - with v.w.o - view: XHTML 1.0 Strict missing xmlns attribute (valid)

Documents that are invalid, which the validator passes as OK

Wrong diagnosis

• bogus FPI #2: HTML 4.01 "Strict"
• bogus FPI #3: XHTML 1.1 "Strict"

Docs that should be Well Formed

well formed but no doctype. The validator could mark them as well formed, but not mention validity?

• validate - with v.w.o - view: Basic XML document, well formed. No doctype. Currently checked only for xmlwf. Passes.
• validate - with v.w.o - View: XML output from OpenOffice 5.2 ("Build/Tag number OpenOffice605"), many namespaces. Currently checked only for xmlwf. Passes.
• validate - with v.w.o - View: Very minimal SVG document, no document type. Currently checked only for xmlwf. Passes.
• validate - with v.w.o - View: XHTML document without doctype, served as XML. Fallback to XHTML 1.0, and invalid.

Various Encodings

• UTF-8 sample text (correct), more positive tests
• UTF-8 stress test (wrong cases), more negative tests
• iso-2022-jp encoded document
• windows-1250 encoded document (Slovenian)
• utf-8 encoded document
• XML file with a "Plane 20" character (bogus).
• validate - with v.w.o - view: Charset mismatch: HTTP Content-Type and XML Declaration (sends warning, proceed with HTTP and pass)
• validate - with v.w.o - view: Charset mismatch: HTTP Content-Type and meta http-equiv (sends warning, proceed with HTTP and pass)

Error conditions & strange cases

This should be split in UI testing and regression tests

The following documents trip up the parser pretty badly - finds errors way beyond the last line

• validate - with v.w.o - view: RDDL (with FPI, served as text/html - example of a custom XML language wrongly served as text/html)
• image/svg (is this a bug? I think this media type is not allowed any more)
• validate - with v.w.o - view: dereference relative URIs for DTDs

User Interface tests

The following tests do not test the binary "validity" results, but the proper handling of options, errors, various outputs, etc. "validate" links use this instance, with validator.w3.org links added for reference comparison

• validate - with v.w.o - a file: URL
• validate - with v.w.o - /check/referer (deprecated)
• validate - with v.w.o - /check/referer;verbose=1 (unsupported)
• validate - with v.w.o - /check/referer;verbose=1;No200=1 (unsupported)
• validate - with v.w.o - /check?uri=referer (supported, should validate this test collection)
• validate - with v.w.o - /check?uri=referer;verbose=1 (supported, should validate this test collection)
• validate - with v.w.o - /check?uri=refererr&verbose=1 (supported, should validate this test collection)
• validate - with v.w.o - /check?uri=referer;verbose=1;No200=1 (supported, should validate this test collection)
• validate - with v.w.o - Directory Redirect (should mention the redirect and validate the redirection target
• validate - with v.w.o - script invoked with no parameters (gives an error)
• validate - with v.w.o - uri parameter is empty
• validate - with v.w.o - uri is missing http:// (silent fix)
• validate - with v.w.o - 404 not found
• validate - with v.w.o - 404 not found, with "validate error pages" ON
• validate - with v.w.o - text/plain resource (refuses to validate)
• validate - with v.w.o - httpd on a non-standard port (should be working transparently if :8000 server is running)
• validate - a resource served as application/atom+xml, should be passed on to the Feed Validator.
• validate - a resource served as text/css, should be passed on to the CSS Validator.

Bugs and Regression Tests

These should be tied to a bug report on either mailing-list and/or bugzilla whenever possible.

• Long Lines (cause a core)
• A trailing ASCII NUL character (trips up HTML::Parser (not really))
• Undeclared namespace prefixes (colons ":" in element names (trips up parsing of onsgmls output))
• validate - with v.w.o - view: Test of doctype override for SVG document with no doctype, but an XML decl. The doctype should sneak between the XML decl and the root svg start tag. ( 857 - DOCTYPE Override)

Test for specific output

Output of preparse warnings

• validate - with v.w.o - view: text/xml, no charset (W01)
• validate - with v.w.o - view: text/html, no charset, fbc set (W02)
• validate - with v.w.o - view: charset override (W03)
• validate - with v.w.o - view: text/html, no charset, override set (W04)
• validate - with v.w.o - view: text/html, no charset (W04)
• validate - with v.w.o - view: XHTML, no DOCTYPE, override ON. (W05).
• validate - with v.w.o - view: HTML4.0 plus blink ("Custom" SGML DTD), parse mode fallback to SGML (W06)
• validate - with v.w.o - view: HTML 4.01 Strict served with an XHTML mime type. Parse mode conflict (W07)
• W08 is obsolete
• validate - with v.w.o - view: XHTML, no DOCTYPE, no xmlns (thus invalid). (W09).
• validate - with v.w.o - view: SGML-ish document with root element != html. Served as text/html. No DOCTYPE. (W09nohtml)
• validate - with v.w.o - view: XML, no DOCTYPE, not well-formed (W09xml)
• W10 needs test case
• validate - with v.w.o - view: XHTML with doctype override to HTML401. namespace will trigger (W11)
• validate - with v.w.o view: HTML 4.01 with xmlns attribute on root element. Should trigger a validation error (it does) and preparse warning W11 (it doesn't!)
• validate - with v.w.o - view: HTML 4.01 Strict, validated with HTML 4.01 Transitional override (W13)
• w14 needs a test case
• validate - with v.w.o - view: No doctype, override given, fallback fbd is off (W15)
• validate - with v.w.o - view: No doctype, fallback given (W16)
• W17 needs a test case. is it a DUP of W04 now?
• validate - with v.w.o - view: charset mismatch HTTP / XML decl (W18)
• validate - with v.w.o - view: charset mismatch HTTP / Meta (W19)
• W20 needs a test case. probably rare.
• validate - with v.w.o - view: UTF-8 document, with BOM (W21)
• W22 (bad encoding alias) needs test case
• validate - with v.w.o - view: MathML 2.0 as text/html (W23)
• validate - with v.w.o - view: supported, but rare, charset (W24)
• validate - with v.w.o - view: SVG with neither DOCTYPE nor version attribute (OK in SVG). (W25)