Improve SPARQL test coverage

[Tpt]

SPARQL test coverage has quite a few holes (please complete if you find other missing elements):

Syntax:

• no LANGDIR ("foo"@en--ltr) SPARQL 1.2: add a test with directional language string literals in the query #242
• no invalid LANGDIR ("foo"@en--foo) SPARQL: add negative syntax test for invalid language direction #247
• no DECIMAL_POSITIVE (+2.1)
• no DOUBLE_POSITIVE (+2e1)
• no StrReplaceExpression with 4 arguments (REPLACE("abab", "B.", "Z","i"))
• no BuiltInCall with SHA384 (SHA384("foo"))
• no ExprTripleTermSubject and ExprTripleTermObject with IRI, RDFLiteral, NumericLiteral or BooleanLiteral SPARQL 1.2: cover literals in triple terms #256
• no TripleTermDataObject with a RDFLiteral, NumericLiteral or BooleanLiteral SPARQL 1.2: cover literals in triple terms #256
• no TripleTermObject with RDFLiteral, NumericLiteral or BooleanLiteral SPARQL 1.2: cover literals in triple terms #256
• no ReifiedTripleObject with NumericLiteral or BooleanLiteral SPARQL 1.2: cover literals in triple terms #256
• no PathOneInPropertySet with a or ^a
• no HavingClause with multiple children (HAVING (COUNT(?s) > 1) (COUNT(?o) > 1))
• no GroupCondition with BuiltInCall (GROUP BY LANG(?s))
• no GroupCondition with FunctionCall (GROUP BY xsd:boolean(?value))

Evaluation:

• no DISTINCT aggregate
• no xsd:float - xsd:float
• no xsd:double - xsd:double
• no xsd:decimal - xsd:decimal
• no xsd:float * xsd:float
• no xsd:double * xsd:double
• no xsd:decimal * xsd:decimal
• no xsd:float / xsd:float
• no xsd:double / xsd:double
• no + xsd:float
• no + xsd:double
• no + xsd:decimal
• no - xsd:float
• no - xsd:double
• no - xsd:decimal
• no COALESCE() (without arguments)
• no DATATYPE("foo"@en--ltr) (on directional language string)
• no IRI(<ex:foo>) (called on an IRI)
• no ABS on xsd:float and xsd:double
• no CEIL on xsd:float and xsd:double
• no FLOOR on xsd:float and xsd:double
• no ROUND on xsd:float and xsd:double
• no testing of float and double special values (NaN, +Inf and -Inf), especially on comparison and ordering)
• no deep testing of regex syntax
• no testing of rdf:dirLangString on string manipulation functions
• no ORDER BY on xsd:dateTime
• Evaluation of SELECT ?s WHERE { ?s ex:p* ex:o } on the empty dataset (returns ex:o)

[rubensworks]

This is a very useful list.
Did you produce this in some automatic manner?

[Tpt]

Thanks! I ran test coverage on Oxigraph SPARQL parser and evaluator and listed the not executed code paths that were not special paths for optimizations, error propagation from the underlying storage or features not in SPARQL 1.2.

[afs]

SPARQL test coverage has quite a few holes

A requirement of the tests is complete the W3C process requirement to demonstrate features can be implemented; that's not total test coverage. The goal is to get as many implementation reports as possible (min 2 per feature).

We can add all those tests over time. I'm not sure how we should organise in the 1.0 and 1.1 test areas - one way is as a mixture in the current structure, another is to keep a demarcation of original compliance tests + extra tests.

For SPARQL 1.2, we (all) have a clean slate.

[Tpt]

I'm not sure how we should organise in the 1.0 and 1.1 test areas

Good point. Currently it seems we have a mixture in the current structure. But extra tests do not use the dawgt:approval predicate, allowing to distinguish them from the original tests. I tend to think it's fine, the original test suites are still available somewhere if people want to run them and not the "live" version from this CG.