Tests for manifest item fallback, unclear expected outcome

[danielweck]

epub-tests/tests/pub-foreign_bad-fallback/EPUB/package.opf

Lines 20 to 27 in 9a023ce

	<manifest>
	<item id="content_001" href="foo.dmg" media-type="application/octet-stream" fallback="bar" />
	<item id="bar" href="bar.psd" media-type="image/psd" />
	<item id="nav" properties="nav" href="nav.xhtml" media-type="application/xhtml+xml" />
	</manifest>
	<spine>
	<itemref idref="content_001" />
	</spine>

In this case, the resulting reading order (i.e. computed spine, once the fallback chain is resolved) is empty if the reading system follows the cascade of unsupported media types. In some reading systems this may cause a crash / broken user interface, in others this may result in an empty viewport ... what consitutes a pass vs. fail?

[danielweck]

The expectation is clear with the test that has a specific XML content type:

epub-tests/tests/pub-foreign_xml-suffix-spine/EPUB/package.opf

Lines 20 to 27 in 9a023ce

	<manifest>
	<item id="content_primary" fallback="content_fallback" href="moby.xml" media-type="application/dtc+xml" />
	<item id="content_fallback" href="content_001.xhtml" media-type="application/xhtml+xml" />
	<item id="nav" properties="nav" href="nav.xhtml" media-type="application/xhtml+xml" />
	</manifest>
	<spine>
	<itemref idref="content_primary" />
	</spine>

... but conversely the "plain" application/xml test is problematic because XHTML itself is XML, so the description of the expected outcome needs to be clarified:

epub-tests/tests/pub-foreign_xml-spine/EPUB/package.opf

Lines 20 to 27 in 9a023ce

	<manifest>
	<item id="content_primary" fallback="content_fallback" href="moby.xml" media-type="application/xml" />
	<item id="content_fallback" href="content_001.xhtml" media-type="application/xhtml+xml" />
	<item id="nav" properties="nav" href="nav.xhtml" media-type="application/xhtml+xml" />
	</manifest>
	<spine>
	<itemref idref="content_primary" />
	</spine>

More specifically:

If the reading system does not support XML, it should display the HTML.

epub-tests/tests/pub-foreign_xml-spine/EPUB/package.opf

Lines 8 to 11 in 9a023ce

	<dc:description>
	This EPUB uses an ordinary XML content file with mimetype application/xml in the spine, with a manifest fallback to an
	HTML document. If the reading system does not support XML, it should display the HTML.
	</dc:description>

[danielweck]

Similarly with the JSON test, I struggle to understand the criteria for fail vs. pass:

epub-tests/tests/pub-foreign_json-spine/EPUB/package.opf

Lines 20 to 27 in 9a023ce

	<manifest>
	<item id="content_primary" fallback="content_fallback" href="moby.json" media-type="application/json" />
	<item id="content_fallback" href="content_001.xhtml" media-type="application/xhtml+xml" />
	<item id="nav" properties="nav" href="nav.xhtml" media-type="application/xhtml+xml" />
	</manifest>
	<spine>
	<itemref idref="content_primary" />
	</spine>

Let's say my reading system claims to support application/json by displaying its raw contents in the publication viewport, thereby ignoring the fallback XHTML. Is that a pass? In the absence of clear content processing model for JSON, it's hard to assert success.

(Naturally, in the real world I would expect most reading systems to follow the fallback chain to XHTML, but that's beside the point)

[gautierchomel]

see related discussion #245

Regarding pub-foreign_bad-fallback what happens is that Thorium open system dialog and ask where to save the file. I find it's a useful feature :)

I consider it "n/a" (no fallback is needed as access to the file is provided).

Same for pub-foreign_json-spine and pub-foreign_xml-spine

[iherman]

(Admin note: trying to keep the various repositories as clean as possible...)

The way I read these comments, a few tests may need an update, primarily in their description on what is pass, fail, or possibly n/a. Now that we have set up the environment for a future epub3.4 testing, could you create a PR with the proposed changes for the specific tests?

cc @gautierchomel

[gautierchomel]

@iherman I added it to my to-do list.