2. Features §
This section is non-normative.
The JSON-LD 1.1 Syntax specification [JSON-LD11CG] defines a syntax to
express Linked Data in JSON. Because there is more than one way to
express Linked Data using this syntax, it is often useful to be able to
transform JSON-LD documents so that they may be more easily consumed by
specific applications.
To allow these algorithms to be adapted for syntaxes
other than JSON, the algorithms operate on the JSON-LD internal representation,
which uses the generic
concepts of arrays, dictionaries,
strings, numbers, booleans, and null to describe
the data represented by a JSON document. Algorithms act on this
internal representation with API entry points responsible for
transforming between the concrete and internal representations.
JSON-LD uses contexts to allow Linked Data
to be expressed in a way that is specifically tailored to a particular
person or application. By providing a context,
JSON data can be expressed in a way that is a natural fit for a particular
person or application whilst also indicating how the data should be
understood at a global scale. In order for people or applications to
share data that was created using a context that is different
from their own, a JSON-LD processor must be able to transform a document
from one context to another. Instead of requiring JSON-LD
processors to write specific code for every imaginable
context switching scenario, it is much easier to specify a
single algorithm that can remove any context. Similarly,
another algorithm can be specified to subsequently apply any
context. These two algorithms represent the most basic
transformations of JSON-LD documents. They are referred to as
expansion and compaction, respectively.
JSON-LD 1.1 introduces new features that are
compatible with JSON-LD 1.0 [JSON-LD],
but if processed by a JSON-LD 1.0 processor may produce different results.
In order to detect this JSON-LD 1.1 requires that the processing
mode be explicitly set to json-ld-1.1, either through the
processingMode API option, or using the
@version member within a context.
There are four major types of transformation that are discussed in this
document: expansion, compaction, flattening, and RDF serialization/deserialization.
2.1 Expansion §
This section is non-normative.
The algorithm that removes context is
called expansion. Before performing any other
transformations on a JSON-LD document, it is easiest to
remove any context from it and to make data structures
more regular.
To get an idea of how context and data structuring affects the same data,
here is an example of JSON-LD that uses only terms
and is fairly compact:
Example 3: JSON-LD documenet using only terms
{
"@context": {
"name": "http://xmlns.com/foaf/0.1/name",
"homepage": {
"@id": "http://xmlns.com/foaf/0.1/homepage",
"@type": "@id"
}
},
"@id": "http://me.markus-lanthaler.com/",
"name": "Markus Lanthaler",
"homepage": "http://www.markus-lanthaler.com/"
}The next input example uses one IRI to express a property
and an array to encapsulate another, but
leaves the rest of the information untouched.
Example 4: Sample JSON-LD document using an IRI instead of a term to express a property
{
"@context": {
"website": "http://xmlns.com/foaf/0.1/homepage"
},
"@id": "http://me.markus-lanthaler.com/",
"http://xmlns.com/foaf/0.1/name": "Markus Lanthaler",
"website": { "@id": "http://www.markus-lanthaler.com/" }
}Note that both inputs are valid JSON-LD and both represent the same
information. The difference is in their context information
and in the data structures used. A JSON-LD processor can remove
context and ensure that the data is more regular by employing
expansion.
Expansion has two important goals: removing any contextual
information from the document, and ensuring all values are represented
in a regular form. These goals are accomplished by expanding all properties
to absolute IRIs and by expressing all
values in arrays in
expanded form. Expanded form is the most verbose
and regular way of expressing of values in JSON-LD; all contextual
information from the document is instead stored locally with each value.
Running the Expansion algorithm
(expand)
operation) against the above examples results in the following output:
Example 5: Expanded JSON-LD document using an IRI
[
{
"@id": "http://me.markus-lanthaler.com/",
"http://xmlns.com/foaf/0.1/name": [
{ "@value": "Markus Lanthaler" }
],
"http://xmlns.com/foaf/0.1/homepage": [
{ "@id": "http://www.markus-lanthaler.com/" }
]
}
]The example above is the JSON-LD serialization of the output of the
expansion algorithm,
where the algorithm's use of dictionaries are replaced with JSON objects.
Note that in the output above all context definitions have
been removed, all terms and
compact IRIs have been expanded to absolute
IRIs, and all
JSON-LD values are expressed in
arrays in expanded form. While the
output is more verbose and difficult for a human to read, it establishes a
baseline that makes JSON-LD processing easier because of its very regular
structure.
2.3 Flattening §
This section is non-normative.
While expansion ensures that a document is in a uniform structure,
flattening goes a step further to ensure that the shape of the data
is deterministic. In expanded documents, the properties of a single
node may be spread across a number of different
dictionaries. By flattening a
document, all properties of a node are collected in a single
dictionary and all blank nodes
are labeled with a blank node identifier. This may drastically
simplify the code required to process JSON-LD data in certain applications.
For example, assume the following JSON-LD input document:
Example 9: JSON-LD document in compact form
{
"@context": {
"name": "http://xmlns.com/foaf/0.1/name",
"knows": "http://xmlns.com/foaf/0.1/knows"
},
"@id": "http://me.markus-lanthaler.com/",
"name": "Markus Lanthaler",
"knows": [
{
"name": "Dave Longley"
}
]
}Running the Flattening Algorithm
(flatten)
operation) with a context set to null to prevent compaction
returns the following document:
Example 10: Flattened sample document in expanded form
[
{
"@id": "_:t0",
"http://xmlns.com/foaf/0.1/name": [
{ "@value": "Dave Longley" }
]
},
{
"@id": "http://me.markus-lanthaler.com/",
"http://xmlns.com/foaf/0.1/name": [
{ "@value": "Markus Lanthaler" }
],
"http://xmlns.com/foaf/0.1/knows": [
{ "@id": "_:t0" }
]
}
]The example above is the JSON-LD serialization of the output of the
flattening algorithm,
where the algorithm's use of dictionaries are replaced with JSON objects.
Note how in the output above all properties of a node are collected in a
single dictionary and how the blank node representing
"Dave Longley" has been assigned the blank node identifier
_:t0.
To make it easier for humans to read or for certain applications to
process it, a flattened document can be compacted by passing a context. Using
the same context as the input document, the flattened and compacted document
looks as follows:
Example 11: Flattened and compacted sample document
{
"@context": {
"name": "http://xmlns.com/foaf/0.1/name",
"knows": "http://xmlns.com/foaf/0.1/knows"
},
"@graph": [
{
"@id": "_:t0",
"name": "Dave Longley"
}, {
"@id": "http://me.markus-lanthaler.com/",
"name": "Markus Lanthaler",
"knows": { "@id": "_:t0" }
}
]
}Please note that the result of flattening and compacting a document
is always a dictionary,
(represented as a JSON object when serialized),
which contains an @graph
member that represents the default graph.
8. RDF Serialization/Deserialization Algorithms §
This section describes algorithms to deserialize a JSON-LD document to an
RDF dataset and vice versa. The algorithms are designed for in-memory
implementations with random access to dictionary elements.
Throughout this section, the following vocabulary
prefixes are used in
compact IRIs:
| Prefix |
IRI |
| rdf |
http://www.w3.org/1999/02/22-rdf-syntax-ns# |
| rdfs |
http://www.w3.org/2000/01/rdf-schema# |
| xsd |
http://www.w3.org/2001/XMLSchema# |
8.1 Deserialize JSON-LD to RDF algorithm §
This algorithm deserializes a JSON-LD document to an RDF dataset.
Please note that RDF does not allow a blank node to be used
as a property, while JSON-LD does. Therefore, by default
RDF triples that would have contained blank nodes as properties are
discarded when interpreting JSON-LD as RDF.
8.1.2 Algorithm §
The algorithm takes a JSON-LD document element and returns an
RDF dataset. Unless the produceGeneralizedRdf option
is set to true, RDF triple
containing a blank node predicate
are excluded from output.
This algorithm generates new blank node identifiers
and relabels existing blank node identifiers.
The Generate Blank Node Identifier algorithm
keeps an identifier map and a counter to ensure consistent
relabeling and avoid collisions. Thus, before this algorithm is run,
the identifier map is reset and the counter is initialized
to 0.
- Expand element according to the
Expansion algorithm.
- Generate a node map according to the
Node Map Generation algorithm.
- Initialize an empty RDF dataset dataset.
- For each graph name and graph in node map
ordered by graph name:
- If graph name is a relative IRI, continue
with the next graph name-graph pair.
- Initialize triples as an empty array.
- For each subject and node in graph ordered
by subject:
- If subject is a relative IRI, continue
with the next subject-node pair.
- For each property and values in node
ordered by property:
- If property is
@type, then for each
type in values, append a triple
composed of subject, rdf:type,
and type to triples.
- Otherwise, if property is a keyword
continue with the next property-values pair.
- Otherwise, if property is a blank node identifier and
the
produceGeneralizedRdf option is not true,
continue with the next property-values pair.
- Otherwise, if property is a relative IRI,
continue with the next property-values pair.
- Otherwise, property is an absolute IRI or
blank node identifier. For each item
in values:
- If item is a list object, initialize
list triples as an empty array and
list head to the result of the List Conversion algorithm, passing
the value associated with the
@list key from
item and list triples. Append first a
triple composed of subject,
property, and list head to triples and
finally append all triples from
list triples to triples.
- Otherwise, item is a value object
or a node object. Append a triple
composed of subject, property, and
the result of using the
Object to RDF Conversion algorithm
passing item to triples, unless the result is
null, indicating a relative IRI that has
to be ignored.
- If graph name is
@default, add
triples to the default graph in dataset.
- Otherwise, create a named graph in dataset
composed of graph name and add triples.
- Return dataset.
8.2 Object to RDF Conversion §
This algorithm takes a node object or value object
and transforms it into an
RDF resource
to be used as the object of an RDF triple. If a
node object containing a relative IRI is passed to
the algorithm, null is returned which then causes the resulting
RDF triple to be ignored.
8.2.2 Algorithm §
The algorithm takes as its sole argument item which MUST be
either a value object or node object.
- If item is a node object and the value of
its
@id member is a relative IRI, return
null.
- If item is a node object, return the
IRI or blank node identifier associated
with its
@id member.
- Otherwise, item is a value object. Initialize
value to the value associated with the
@value
member in item.
- Initialize datatype to the value associated with the
@type member of item or null if
item does not have such a member.
- If value is
true or
false, set value to the string
true or false which is the
canonical lexical form as described in
section 8.6 Data Round Tripping
If datatype is null, set it to
xsd:boolean.
- Otherwise, if value is a number with a non-zero fractional
part (the result of a modulo‑1 operation) or value is a number
and datatype equals
xsd:double, convert value to a
string in canonical lexical form of
an xsd:double as defined in [XMLSCHEMA11-2]
and described in
section 8.6 Data Round Tripping.
If datatype is null, set it to
xsd:double.
- Otherwise, if value is a number with no non-zero
fractional part (the result of a modulo‑1 operation) or value
is a number and datatype
equals
xsd:integer, convert value to a
string in canonical lexical form of
an xsd:integer as defined in [XMLSCHEMA11-2]
and described in
section 8.6 Data Round Tripping.
If datatype is null, set it to
xsd:integer.
- Otherwise, if datatype is
null, set it to
xsd:string or rdf:langString, depending on if
item has an @language member.
- Initialize literal as an RDF literal using
value and datatype. If item has an
@language member, add the value associated with the
@language key as the language tag of literal.
- Return literal.
8.3 List to RDF Conversion §
List Conversion is the process of taking a list object
and transforming it into an
RDF collection
as defined in RDF Semantics [RDF11-MT].
8.3.2 Algorithm §
The algorithm takes two inputs: an array list
and an empty array list triples used for returning
the generated triples.
- If list is empty, return
rdf:nil.
- Otherwise, create an array bnodes composed of a
newly generated blank node identifier
for each entry in list.
- Initialize an empty array list triples.
- For each pair of subject from bnodes and item from list:
- Initialize object to the result of using the
Object to RDF Conversion algorithm
passing item to list triples.
- Unless object is
null, append a triple
composed of subject, rdf:first, and object.
- Set rest as the next entry in bnodes, or if that
does not exist,
rdf:nil. Append a
triple composed of subject,
rdf:rest, and rest to list triples.
- Return the first blank node from bnodes or
rdf:nil if bnodes is empty.
8.4 Serialize RDF as JSON-LD Algorithm §
This algorithm serializes an RDF dataset consisting of a
default graph and zero or more
named graphs into a JSON-LD document.
In the RDF abstract syntax, RDF literals have a
lexical form, as defined
in [RDF11-CONCEPTS]. The form of these literals is used when creating JSON-LD values based on these literals.
8.4.2 Algorithm §
The algorithm takes one required and two optional inputs: an RDF dataset dataset
and the two flags use native types and use rdf:type
that both default to false.
- Initialize default graph to an empty dictionary.
- Initialize graph map to a dictionary consisting
of a single member
@default whose value references
default graph.
- Initialize node usage map to an empty dictionary.
- For each graph in dataset:
- If graph is the default graph,
set name to
@default, otherwise to the
graph name associated with graph.
- If graph map has no name member, create one and set
its value to an empty dictionary.
- If graph is not the default graph and
default graph does not have a name member,
create such a member and initialize its value to a new
dictionary with a single member
@id
whose value is name.
- Reference the value of the name member in graph map
using the variable node map.
- For each RDF triple in graph
consisting of subject, predicate, and object:
- If node map does not have a subject member,
create one and initialize its value to a new dictionary
consisting of a single member
@id whose value is
set to subject.
- Reference the value of the subject member in node map
using the variable node.
- If object is an IRI or blank node identifier,
and node map does not have an object member,
create one and initialize its value to a new dictionary
consisting of a single member
@id whose value is
set to object.
- If predicate equals
rdf:type, the
use rdf:type flag is not true, and object
is an IRI or blank node identifier,
append object to the value of the @type
member of node; unless such an item already exists.
If no such member exists, create one
and initialize it to an array whose only item is
object. Finally, continue to the next
RDF triple.
- Set value to the result of using the
RDF to Object Conversion algorithm,
passing object and use native types.
- If node does not have an predicate member, create one
and initialize its value to an empty array.
- If there is no item equivalent to value in the array
associated with the predicate member of node, append a
reference to value to the array. Two JSON objects
are considered equal if they have equivalent key-value pairs.
- If object is a blank node identifier or IRI,
it might represent the list node:
- If the object member of node usage map does not exist,
initialize it to a new empty array.
- Append the value of the
@id member of node to
the object member of node usage map.
- If the object member of node map has no
usages member, create one and initialize it to
an empty array.
- Reference the
usages member of the object
member of node map using the variable usages.
- Append a new dictionary consisting of three
members,
node, property, and value
to the usages array. The node member
is set to a reference to node, property to predicate,
and value to a reference to value.
- For each name and graph object in graph map:
- If graph object has no
rdf:nil member, continue
with the next name-graph object pair as the graph does
not contain any lists that need to be converted.
- Initialize nil to the value of the
rdf:nil member
of graph object.
- For each item usage in the
usages member of
nil, perform the following steps:
- Initialize node to the value of the value of the
node member of usage, property to
the value of the property member of usage,
and head to the value of the value member
of usage.
- Initialize two empty arrays list
and list nodes.
- While property equals
rdf:rest,
the value of the @id member
of node is a blank node identifier,
the array value of the member of node usage map associated with the @id
member of node has only one member,
the value associated to the usages member of node has
exactly 1 entry,
node has a rdf:first and rdf:rest property,
both of which have as value an array consisting of a single element,
and node has no other members apart from an optional @type
member whose value is an array with a single item equal to
rdf:List,
node represents a well-formed list node.
Perform the following steps to traverse the list backwards towards its head:
- Append the only item of
rdf:first member of
node to the list array.
- Append the value of the
@id member of
node to the list nodes array.
- Initialize node usage to the only item of the
usages member of node.
- Set node to the value of the
node member
of node usage, property to the value of the
property member of node usage, and
head to the value of the value member
of node usage.
- If the
@id member of node is an
IRI instead of a blank node identifier,
exit the while loop.
- If property equals
rdf:first, i.e., the
detected list is nested inside another list
- and the value of the
@id of node equals
rdf:nil, i.e., the detected list is empty,
continue with the next usage item. The
rdf:nil node cannot be converted to a
list object as it would result in a list of
lists, which isn't supported.
- Otherwise, the list consists of at least one item. We preserve the
head node and transform the rest of the linked list to a
list object.
- Set head id to the value of the
@id
member of head.
- Set head to the value of the head id member of
graph object so that all it's properties can be accessed.
- Then, set head to the only item in the value of the
rdf:rest member of head.
- Finally, remove the last item of the list array
and the last item of the list nodes array.
- Remove the
@id member from head.
- Reverse the order of the list array.
- Add an
@list member to head and initialize
its value to the list array.
- For each item node id in list nodes, remove the
node id member from graph object.
- Initialize an empty array result.
- For each subject and node in default graph
ordered by subject:
- If graph map has a subject member:
- Add an
@graph member to node and initialize
its value to an empty array.
- For each key-value pair s-n in the subject
member of graph map ordered by s, append n
to the
@graph member of node after
removing its usages member, unless the only
remaining member of n is @id.
- Append node to result after removing its
usages member, unless the only remaining member of
node is @id.
- Return result.
8.5 RDF to Object Conversion §
This algorithm transforms an RDF literal to a JSON-LD value object
and a RDF blank node or IRI to an JSON-LD node object.
8.5.2 Algorithm §
This algorithm takes two required inputs: a value to be converted
to a dictionary and a flag use native types.
- If value is an IRI or a
blank node identifier, return a new dictionary
consisting of a single member
@id whose value is set to
value.
- Otherwise value is an
RDF literal:
- Initialize a new empty dictionary result.
- Initialize converted value to value.
- Initialize type to
null
- If use native types is
true
- If the
datatype IRI
of value equals
xsd:string, set
converted value to the
lexical form
of value.
- Otherwise, if the
datatype IRI
of value equals
xsd:boolean, set
converted value to true if the
lexical form
of value matches true, or false
if it matches false. If it matches neither,
set type to xsd:boolean.
- Otherwise, if the
datatype IRI
of value equals
xsd:integer or
xsd:double and its
lexical form
is a valid xsd:integer or xsd:double
according [XMLSCHEMA11-2], set converted value
to the result of converting the
lexical form
to a JSON number.
- Otherwise, if value is a
language-tagged string
add a member
@language to result and set its value to the
language tag of value.
- Otherwise, set type to the
datatype IRI
of value, unless it equals
xsd:string which is ignored.
- Add a member
@value to result whose value
is set to converted value.
- If type is not
null, add a member @type
to result whose value is set to type.
- Return result.
8.6 Data Round Tripping §
When deserializing JSON-LD to RDF
JSON-native numbers are automatically
type-coerced to xsd:integer or xsd:double
depending on whether the number has a non-zero fractional part
or not (the result of a modulo‑1 operation), the boolean values
true and false are coerced to xsd:boolean,
and strings are coerced to xsd:string.
The numeric or boolean values themselves are converted to
canonical lexical form, i.e., a deterministic string
representation as defined in [XMLSCHEMA11-2].
The canonical lexical form of an integer, i.e., a
number with no non-zero fractional part or a number
coerced to xsd:integer, is a finite-length sequence of decimal
digits (0-9) with an optional leading minus sign; leading
zeros are prohibited. In JavaScript, implementers can use the following
snippet of code to convert an integer to
canonical lexical form:
Example 20: Sample integer serialization implementation in JavaScript
(value).toFixed(0).toString()
The canonical lexical form of a double, i.e., a
number with a non-zero fractional part or a number
coerced to xsd:double, consists of a mantissa followed by the
character E, followed by an exponent. The mantissa is a
decimal number and the exponent is an integer. Leading zeros and a
preceding plus sign (+) are prohibited in the exponent.
If the exponent is zero, it is indicated by E0. For the
mantissa, the preceding optional plus sign is prohibited and the
decimal point is required. Leading and trailing zeros are prohibited
subject to the following: number representations must be normalized
such that there is a single digit which is non-zero to the left of
the decimal point and at least a single digit to the right of the
decimal point unless the value being represented is zero. The
canonical representation for zero is 0.0E0.
xsd:double's value space is defined by the IEEE
double-precision 64-bit floating point type [IEEE-754-2008] whereas
the value space of JSON numbers is not
specified; when deserializing JSON-LD to RDF the mantissa is rounded to
15 digits after the decimal point. In JavaScript, implementers
can use the following snippet of code to convert a double to
canonical lexical form:
Example 21: Sample floating point number serialization implementation in JavaScript
(value).toExponential(15).replace(/(\d)0*e\+?/,'$1E')
The canonical lexical form of the boolean
values true and false are the strings
true and false.
When JSON-native numbers are deserialized
to RDF, lossless data round-tripping cannot be guaranteed, as rounding
errors might occur. When
serializing RDF as JSON-LD,
similar rounding errors might occur. Furthermore, the datatype or the lexical
representation might be lost. An xsd:double with a value
of 2.0 will, e.g., result in an xsd:integer
with a value of 2 in canonical lexical form
when converted from RDF to JSON-LD and back to RDF. It is important
to highlight that in practice it might be impossible to losslessly
convert an xsd:integer to a number because
its value space is not limited. While the JSON specification [RFC7159]
does not limit the value space of numbers
either, concrete implementations typically do have a limited value
space.
To ensure lossless round-tripping the
Serialize RDF as JSON-LD algorithm
specifies a use native types flag which controls whether
RDF literals
with a datatype IRI
equal to xsd:integer, xsd:double, or
xsd:boolean are converted to their JSON-native
counterparts. If the use native types flag is set to
false, all literals remain in their original string
representation.
Some JSON serializers, such as PHP's native implementation in some versions,
backslash-escape the forward slash character. For example, the value
http://example.com/ would be serialized as http:\/\/example.com\/.
This is problematic as other JSON parsers might not understand those escaping characters.
There is no need to backslash-escape forward slashes in JSON-LD. To aid
interoperability between JSON-LD processors, forward slashes MUST NOT be
backslash-escaped.
D. Open Issues §
This section is non-normative.
The following is a list of issues open at the time of publication.
Thanks for the great work with JSON-LD! However, when trying to use JSON-LD for to present data in the company I'm working in, I noticed the following missing feature:
FEATURE PROPOSAL: ABILITY TO DEFINE ANY KEY AS AN INDEX KEY
In addition to JSON-LD's existing index container structure, I propose that any key under a JSON-LD node could be defined as a index key.
This would help clustering data under a node into coder friendly logical groups without messing up the Linked Data interpretation with e.g. blank nodes. I encountered the need for this feature at our company where our problem is that the amount of attributes a single JSON-LD node can have can potentially be quite many, say, tens or hundreds of attributes.
As far as I know, this can not be currently done with JSON-LD without 1) ending up with blank nodes or 2) the need to create a deeper JSON structure by using a separate index term (using "@container":"@index") which then contains the data underneath.
In addition, if a single key could be defined as a index term, this would make it more flexible to attach the JSON-LD Linked Data interpretation to even a wider amount of existing JSON data, without having to change the structure of such data (and without ending up with e.g. lots of blank nodes).
DEFINING AN INDIVIDUAL INDEX KEY IN @context
The "@context" definition could be done e.g. using the existing reserved keyword "@index" in the following way:
"indexkey":"@index"
which should be interpreted in the following way: 1) the "indexkey" is an index key and should be skipped when traversing the JSON tree while doing the JSON-LD to RDF interpretation, 2) any data directly under the "indexkey" should be interpreted as data directly attached to the node of the indexkey (same RDF subject).
EXAMPLE
To give a full example, in the following a single key "labels" is defined as an index index key to help grouping the data into coder friendly logical groups without messing up the Linked Data interpretation):
{
"@context": {
"labels":"@index",
"main_label":"http://example.org/my-schema#main_label",
"other_label":"http://example.org/my-schema#other_label",
"homepage":{ "@id":"http://example.org/my-schema#homepage", "@type":"@id"}
},
"@id":"http://example.org/myresource",
"homepage": "http://example.org",
"labels": {
"main_label": "This is the main label for my resource",
"other_label": "This is the other label"
}
}This example JSON-LD should generate the following RDF triplets:
<http://example.org/myresource> <http://example.org/my-schema#homepage> <http://example.org>.
<http://example.org/myresource> <http://example.org/my-schema#main_label> "This is the main label for my resource".
<http://example.org/myresource> <http://example.org/my-schema#other_label> "This is the other label".
This has already been discussed several times usingvarious terms.. the most recent request has come from David Janes on the mailing list. The basic idea is to support JSON values/subtrees that aren't mapped to an IRI in the context. They should survive algorithmic transformations (basically without being touched at all).
_This was raised by Fabian Steeg:_
The JSON-LD API document states: "Expansion has two important goals: removing any contextual information from the document, and ensuring all values are represented in a regular form."
Is there a way to achieve only the second goal, the regular form, but with compact terms? Using compaction with compactArrays=false is pretty close, but there is still at least one thing that is irregular and causing issues for me.
Given this input:
{
"http://example.com/foo": "foo-value",
"http://example.com/bar": {
"@value": "bar-value",
"@language": "en"
},
"@context": {
"foo": "http://example.com/foo",
"bar": "http://example.com/bar"
}
}
I get this from compaction with compactArrays=false:
{
"@graph": [{
"foo": ["foo-value"], <-- foo: array of strings
"bar": [{ <-- bar: array of objects
"@language": "en",
"@value": "bar-value"
}]
}],
"@context": {
"foo": "http://example.com/foo",
"bar": "http://example.com/bar"
}
}
But I'd like to get this (which is what expansion does to the values):
{
"@graph": [{
"foo": [{ <-- both foo and bar:
"@value" : "foo-value" array of objects
}],
"bar": [{
"@language": "en",
"@value": "bar-value"
}]
}],
"@context": {
"foo": "http://example.com/foo",
"bar": "http://example.com/bar"
}
}
So I guess I'm looking for something like a compactValues=false option.
Is there some way to get this output?
_Reported by @afs:_
We are encountering an issue when converting RDF Datasets to JSON-LD.
The problem is with blank nodes that are shared between graphs and lists.
In TriG (yes, this is a synthetic reduced test case that captures a
smaller example that might appear for real):
# Bnode references across graph and lists
PREFIX : <http://www.example.com/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
:G {
# Written in short form it would be:
# :z :q ("cell-A" "cell-B")
# but we want to share the tail ("cell-B")
:z :q _:z0 .
_:z0 rdf:first "cell-A" .
_:z0 rdf:rest _:z1 .
_:z1 rdf:first "cell-B" .
_:z1 rdf:rest rdf:nil .
}
:G1 {
# This references the tail ("cell-B")
:x :p _:z1 .
}
The triple in :G1 references into the list in :G.
But as we understand the conversion algorithm, section 4 only considers
each graph in turn and so does not see the cross graph sharing.
Is this a correct reading of the spec text?
Part 4 of the conversion algorithm has
"For each name and graph object in graph map: "
so 4.3.3.* walks back up the list in one graph only.
(Conversion generated by jsonld-java : it does not matter if compaction
is applied or not):
{
"@graph" : [ {
"@graph" : [ {
"@id" : ":z",
":q" : {
"@list" : [ "cell-A", "cell-B" ]
}
} ],
"@id" : ":G"
}, {
"@graph" : [ {
"@id" : ":x",
":p" : {
"@id" : "_:b1"
}
} ],
"@id" : ":G1"
} ],
"@context" : {
"@base" : "http://www.example.com/",
"" : "http://www.example.com/",
"rdf" : "http://www.w3.org/1999/02/22-rdf-syntax-ns#"
}
}
There is no _:b1 in :G to refer to because the algorith generated @list
and its implicit bNodes don't have labels.
This is a different dataset with no shared bNode.
If it is all the same graph (s/:G1/:G/), the RDF dataset structure is
correctly serialized.
Andy
See: digitalbazaar/jsonld.js#72
It would be helpful to have the ability to use @language within an object as a shorthand for "@context": {"@language": "..."} ... for instance... make:
{
"@language": "en",
"displayName": "foo"
}equivalent to:
{
"@context": {"@language": "en"},
"displayName": "foo"
}In the spirit of "Labeling Everything" (http://patterns.dataincubator.org/book/label-everything.html) ... it would be worthwhile, IMO, for JSON-LD to provide a basic @Label keyword for use both in @context and nodes. It's largely syntactic sugar but would be useful.
For example:
{
"@context": {
"@label": "An Example Context",
"displayName": "@label",
},
"displayName": "A Simple Label"
}Which would expand to:
_:c14n0 <http:
Problem description §
Many JSON specs existed before JSON-LD. A couple of these specs may not be compatible with JSON-LD as they contain multidimensional containers, such as GeoJSON.
Example of a multidimensional array:
[ [3.1,51.06,30],
[3.1,51.06,20] ]
This issue is a result from the discussion on the GeoJSON-LD repository: geojson/geojson-ld#32. If this issue will not get resolved, the GeoJSON-LD community would suggest creating custom JSON-LD parsers for JSON-LD dialects. This situation would be far from desirable.
Suggested solution §
Introduce a new @values keyword, which can be used to describe the values of a @set or a @list container in more detail.
When an array is given in the @values, then the precise amount of objects within this array corresponds with the array in the graph in this order.
When an object is given in the @values, each value of the array in the graph is mapped according to this template.
Example §
{
"@context": {
"coordinates": {
"@id": "geojson:coordinates",
"@container" : "@list",
"@values" : {
"@type" : "geojson:Coordinate",
"@container" : "@set",
"@values" : [
{"@type" : "xsd:double", "@id":"geo:longitude"},
{"@type" : "xsd:double", "@id":"geo:latitude"}
]
}
}
},
"@graph" : [{
"@id" : "ex:LineString1",
"coordinates" : [
[
3.1057405471801753,
51.064216229943476
],
[
3.1056976318359375,
51.063434090307574
]
]
}]
}Would transform to (and vice versa):
ex:LineString1 geojson:coordinates _:b0 .
_:b0 rdf:first _:b1 .
_:b1 a geojson:Coordinate ;
geo:longitude "3.105740547180175E0"^^xsd:double ;
geo:latitude "5.106421622994348E1"^^xsd:double .
_:b0 rdf:rest _:b2 .
_:b2 rdf:first a geojson:Coordinate ;
geo:longitude "3.1056976318359375"^^xsd:double ;
geo:latitude "51.063434090307574"^^xsd:double .
_:b2 rdf:rest rdf:nil .I want the following:
{
"@context": {
"type": "@type",
"profile": "@type"
},
"type": "cov:Coverage",
"profile": "cov:GridCoverage"
}However this is not allowed. The playground says "Invalid JSON-LD syntax; colliding keywords detected".
However, this one works:
{
"@context": {
"type": {"@id": "rdf:type", "@type": "@id" },
"profile": {"@id": "rdf:type", "@type": "@id" }
},
"type": "cov:Coverage",
"profile": "cov:GridCoverage"
}I understand that this restriction probably makes sense for other keywords, but could it do any harm for @type?
There have been some discussions on what it would take to be able to do a streaming parse of JSON-LD into Quads, and similarly to generate compliant JSON-LD from a stream of quads. Describing these as some kind of a profile would be useful for implementations that expect to work in a streaming environment, when it's not feasible to work on an entire document basis.
As currently stated, the JSON-LD to RDF algorithm requires expanding the document and creating a node map. A profile of JSON-LD which used a flattened array of node objects, where each node object could be independently expanded and no flattening is required could facilitate deserializing an arbitrarily long JSON-LD source to Quads. (Some simplifying restrictions on shared lists may be necessary). Outer document is an object, containing @context and @graph only; obviously, this only will work for systems that can access key/values in order, and for systems that ensure that @context comes lexically before @graph in the output. Obviously, only implementations that can read and write JSON objects with key ordering intact will be able to take advantage of such streaming capability.
Fo serializing RDF to JSON-LD, expectations on the grouping of quads with the same graph name and subject are necessary to reduce serialization cost, and marshaling components of RDF Lists is likely not feasible. Even if graph name/subject grouping is not maintained in the input, the resulting output will still represent a valid JSON-LD document, although it may require flattening for further processing. (Many triple stores will, in fact, generate statements/quads properly grouped, so this is likely not an issue in real world applications).
Hi there,
I was looking for a way to access properties in a JSON-LD document based on triples (to patch the document). This would mean having a view which creates a dictionary for a given document. The term Normalisation is already used, but this approach would be close to the way https://github.com/paularmstrong/normalizr. D3 uses https://github.com/d3/d3-hierarchy/blob/master/README.md#stratify in a slightly different way but with the same general intent.
The goal would be to be able to address document values with this syntax stratified_doc[triple.subject][triple.predicate] or even better stratified[triple.graph][triple.subject][triple.predicate].
This could also be a @stratified parameter for expansion.
Example §
For a document:
{
"@context": {
"dc": "http://purl.org/dc/elements/1.1/",
"ex": "http://example.org/vocab#",
"xsd": "http://www.w3.org/2001/XMLSchema#",
"ex:contains": {
"@type": "@id"
}
},
"@id": "http://example.org/graph/0",
"dc:creator": "Jane Doe",
"@graph": [
{
"@id": "http://example.org/library",
"@type": "ex:Library",
"ex:contains": "http://example.org/library/the-republic"
}
]
}Such a stratified would therefore look like:
{
"http://example.org/graph/0": {
"http://example.org/library": {
"@type": "http://example.org/vocab#Library",
"http://example.org/vocab#contains": {
"@id": "http://example.org/library/the-republic"
}
},
"http://example.org/library/the-republic": {}
},
"@graph": {
"http://example.org/graph/0": {
"http://purl.org/dc/elements/1.1/creator": "Jane Doe"
}
}
}This would therefore allow to do the following:
var creator = stratified['@graph']['http://example.org/graph/0']['http://purl.org/dc/elements/1.1/creator'];
var type = stratified['http://example.org/graph/0']['http://example.org/library']['@type'];
stratified['http://example.org/graph/0']['http://example.org/library/the-republic']['@type'] = 'http://example.org/vocab#Book';
var new_doc = {
...stratified,
'http://example.org/graph/0': {
...stratified['http://example.org/graph/0'],
'http://example.org/library/the-republic' : {
...stratified['http://example.org/graph/0']['http://example.org/library/the-republic'],
'@type': 'http://example.org/vocab#Book'
}
}
}Issue 507: JSON-LD 1.1: method to require {"id": "uri"} resource compaction resultapidefer Issue: The compaction algorithm prefers the most compact format, which for resources without relationships is a string containing the URI. This causes problems in systems that cannot handle arrays of mixed data types (for example ElasticSearch) when there are also resources that have relationships, resulting in both objects and strings in the same array.
For example:
"seeAlso": [
"http://example.org/reference1",
{"id": "http://example.org/reference2", "format": "text/html"}
]
would raise an error in Elastic.
Proposed solution: Provide a flag to the compaction algorithm to signal that the resulting JSON should always create objects for resources, even if there is only the URI available. This would instead render the example above as an array of objects:
"seeAlso": [
{"id": "http://example.org/reference1"},
{"id": "http://example.org/reference2", "format": "text/html"}
]
The purpose of the @container:@set functionality (AFAIU) is to ensure that the output is consistent in shape. Thus if there can ever be multiple values, the structure is always an array.
There are two situations in which this functionality could be desirable but is currently not possible:
@type As it's a keyword, we can only alias it (e.g. as type) but not define it to have @container:@set functionality. Meaning that there's a gotcha waiting to happen for ontologies that require or use multiple classes for a single resource instance. See playground@context Less useful, but @context will also compact to a single string/object when there might be multiple contexts. See playground
@context modifying itself seems particularly strange, but the inconsistency for @type seems solvable if the restrictions in its definition were loosened?
This is related to #235: When I have the following document:
{
"@context": {
"@vocab" : "http://vocab.getty.edu/",
"a" : "http://vocab.getty.edu/aaaaaaaaaat/"
},
"@id" : "http://vocab.getty.edu/aaaaaaaaaat/5001065997",
"@type": "http://vocab.getty.edu/aaaaaaaaaat/datatype"
}By point 3 of the spec, because http://vocab.getty.edu/aaaaaaaaaat/5001065997 contains the value of @vocab, it gets compacted as aaaaaaaaaat/5001065997 without even looking at the prefixes. I think this is not reasonable, in this case a:5001065997 would look much nicer IMO.
We have web application that needs to be able to modify RDF lists from a triple store and propagate the changes back. To do this, we are utilizing jsonld-java to serialize the RDF into JSON-LD, modifying it in the web app, and then sending it back to be deserialized and stored in the triple store. Originally, we were using blank nodes like the ones shown in Turtle below.
<http:
_:a a <http:
<http:
<http:
_:b a <http:
<http:
<http:
_:c a <http:
<http:
<http:
However, we discovered that blank node lists are collapsed during serialization thus losing all the blank node IDs.
[ {
"@id" : "http://example.com",
"http://example.com/property" : [ {
"@list" : [ {
"@value" : "a"
}, {
"@value" : "b"
}, {
"@value" : "c"
} ]
} ]
} ]With blank node IDs removed, we are no longer able to reference the existing RDF in the triple store to perform updates when the lists are modified in the web-app without much more complex logic. To avoid this, we skolemized the blank node IDs into IRIs.
<http://example.com> <http://example.com/property> <urn:a> .
<urn:a> a <http://www.w3.org/1999/02/22-rdf-syntax-ns#List> ;
<http://www.w3.org/1999/02/22-rdf-syntax-ns#first> "a" ;
<http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> <urn:b> .
<urn:b> a <http://www.w3.org/1999/02/22-rdf-syntax-ns#List> ;
<http://www.w3.org/1999/02/22-rdf-syntax-ns#first> "b" ;
<http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> <urn:c> .
<urn:c> a <http://www.w3.org/1999/02/22-rdf-syntax-ns#List> ;
<http://www.w3.org/1999/02/22-rdf-syntax-ns#first> "c" ;
<http://www.w3.org/1999/02/22-rdf-syntax-ns#rest> <http://www.w3.org/1999/02/22-rdf-syntax-ns#nil> .
However, when serializing the skolemized triples, the IRI for the last element in the RDF list is hidden, in this case urn:c. This leads to the same problem we were having when using blank node IDs.
[ {
"@id" : "http://example.com",
"http://example.com/property" : [ {
"@id" : "urn:a"
} ]
}, {
"@id" : "urn:a",
"@type" : [ "http://www.w3.org/1999/02/22-rdf-syntax-ns#List" ],
"http://www.w3.org/1999/02/22-rdf-syntax-ns#first" : [ {
"@value" : "a"
} ],
"http://www.w3.org/1999/02/22-rdf-syntax-ns#rest" : [ {
"@id" : "urn:b"
} ]
}, {
"@id" : "urn:b",
"@type" : [ "http://www.w3.org/1999/02/22-rdf-syntax-ns#List" ],
"http://www.w3.org/1999/02/22-rdf-syntax-ns#first" : [ {
"@value" : "b"
} ],
"http://www.w3.org/1999/02/22-rdf-syntax-ns#rest" : [ {
"@list" : [ {
"@value" : "c"
} ]
} ]
} ]Issue #277 seems to be the point where the implementation changed from serializing lists in the manner we expect to this new compact way. Is there any way we can get around this so that the last blank node of a list is not collapsed?
Currently it appears that properties are sorted into alphabetical order after any JSON-LD operation (compaction, framing).
In the context of framing, this is sometimes a nice feature, since it means that after framing multiple input JSON files, the JSON data is at least in a consistent order.
I understand that ordering is semantically meaningless, but as framing exists to turn the graph (which could correspond to multiple different trees) into a predictable JSON tree as a convenience for developers, it seems natural to me that if an explicit ordering is given in the frame, that the algorithm would respect that order rather than alphabetize. For example, if my data is:
{
"@context": "http://schema.org/",
"@id": "document",
"b": "text",
"a": "more text"
}
Under the frame:
{
"@context": "http://schema.org/",
"@id": "document",
"b": {},
"a": {}
}
(example in playground)
the returned document reverses the order of b and a (to be alphabetical), and not the order given in the frame. Framing is a really elegant way to specify the nesting order, but it would be nice for framing to also be able to dictate the ordering, so that the output data file really follows the exact structure of the given frame.
Related issue: there is no way to indicate that referenced nodes should occur before they are references (excluding circular references), which can be useful in streaming files. Having control of the node order via the frame would also give a mechanism to address that.
Hope this makes sense and apologies if I'm missing something fundamental here that makes alphabetizing the node order the only logical thing to do; or if I've misunderstood the expected behavior.
Comments at TPAC suggested that as our work is a breaking change (causing 1.0 processors that are not 1.1 compatible to intentionally break when they see "@version": 1.1), semantic versioning would suggest that we use a major release number, rather than a minor number.
This could impact a potential WG, which may want to make further changes, and then be in the place of using either 2.1 or 3.0, which is odd given that the previous recommendation is 1.0.
In some situations it is important/necessary to include the base direction of a text, alongside its language; see the “Requirements for Language and Direction Metadata in Data Formats” for further details. In practice, in a vanilla JSON, it would require something like:
"title": [ { "value": "Moby Dick", "lang": "en" },
{ "value": "موبي ديك", "lang": "ar" "dir": "rtl"}
]
(the example comes from that document).
At this moment, I believe the only way you can reasonably express that in JSON-LD is via cheating a bit:
"title": [ { "@value": "Moby Dick", "@language": "en" },
{ "@value": "موبي ديك", "@language": "ar" "dir": "rtl"}
]
and making sure that the dir term is not defined in the relevant @context so that, when generating the RDF output, that term is simply ignored. But that also means that there is no round-tripping, that term will disappear after expansion.
The difficulty lies in the RDF layer, in fact; RDF does not have any means (alas!) to express text direction. On the other hand, this missing feature is a general I18N problem whenever JSON-LD is used (there were issues when developing the Web Annotation Model, these issues are popping up in the Web Publication work, etc.).
Here is what I would propose as a non-complete solution
- Let us introduce a
@dir term, alongside @language. This means this term can be used in place of dir above, ie, it is a bona-fide part of a string representation, and would therefore be kept in the compaction/expansion steps, can also be used for framing.
- In JSON-LD 1.1,
@dir is ignored when transforming into RDF. I.e., only the language tag would be used.
- We may initiate some work in the RDF community to solve this issue. There may be several ways, each of them require the RDF community to chime in
3.1. Define a mechanism of "parametrized" standard datatypes that represent a (language,direction) pair. One would then get something like[] ex:title "موبي ديك"^^rdf:internationalText(ar,rtl) ;
3.2. Go for a "generalized" RDF where strings can also appear as subjects (that has been a matter of dispute for a long time...). That would give the possibility to add such attribute to texts like directions
3.3. Some other mechanisms that I cannot think about
- In a future JSON-LD 1.* the
@dir value can be properly mapped onto an RDF representing the right choices (if such choices are worked out)
Cc: @BigBlueHat @r12a
Per a suggestion by @danbri, we may want to add a container type, similar to @list for encoding schema:ItemList serializations, when the values are schema:ListItem and order is set through schema:position. ItemList can be used with text values as well, but this is already reasonably supported natively.
Markup might look like the following:
{
"@context": {
"@vocab": "http://schema.org/",
"itemListElement": {"@container": "@listItem"}
},
"@type": "ItemList",
"@url": "http://en.wikipedia.org/wiki/Billboard_200",
"name": "Top music artists",
"description": "The artists with the most cumulative weeks at number one according to Billboard 200",
"itemListElement": [
{"@type": "MusicGroup", "name": "Beatles"},
{"@type": "MusicGroup", "name": "Elvis Presley"},
{"@type": "MusicGroup", "name": "Michael Jackson"},
{"@type": "MusicGroup", "name": "Garth Brooks" }
]This would expand to the following:
[
{
"@id": "http://en.wikipedia.org/wiki/Billboard_200",
"@type": ["http://schema.org/ItemList"],
"http://schema.org/description": [{
"@value": "The artists with the most cumulative weeks at number one according to Billboard 200"
}],
"http://schema.org/itemListElement": [{
"@type": ["http://schema.org/ListItem"],
"http://schema.org/item": [{
"@type": ["http://schema.org/MusicGroup"],
"http://schema.org/name": [{"@value": "Beatles"}]
}],
"http://schema.org/position": [{"@value": 1}]
}, {
"@type": ["http://schema.org/ListItem"],
"http://schema.org/item": [{
"@type": ["http://schema.org/MusicGroup"],
"http://schema.org/name": [{"@value": "Elvis Presley"}]
}],
"http://schema.org/position": [{"@value": 2}]
}, {
"@type": ["http://schema.org/ListItem"],
"http://schema.org/item": [{
"@type": ["http://schema.org/MusicGroup"],
"http://schema.org/name": [{"@value": "Michael Jackson"}]
}],
"http://schema.org/position": [{"@value": 3}]
}, {
"@type": ["http://schema.org/ListItem"],
"http://schema.org/item": [{
"@type": ["http://schema.org/MusicGroup"],
"http://schema.org/name": [{"@value": "Garth Brooks"}]
}],
"http://schema.org/position": [{"@value": 3}]
}
],
"http://schema.org/name": [{"@value": "Top music artists"}]
}]Otherwise, it works like @list.
When compacting, the processor will re-order items based on position, and ignore any nextItem or previousItem entries.
Expansion shows 1-base position, but could be 0-base as well. Note that specific position values are lost when compacting, and duplicate values may lead to undefined relative ordering.
During the last meeting it was resolved to have one playground for 1.0 and 1.1 processing. Some notes on that related to jsonld.js:
- 0.4.x passes older 1.0 test suites
- 0.4.x does not pass more recent tests added for 1.0
- 0.5.x does pass all current 1.0 tests
- 0.5.x also includes a bit of 1.1 functionality, in particular some
@graph handling and a bit of @version handling
- 0.6.x should end up passing all 1.0 and current 1.1 draft tests
- there may be technical issues using multiple jsonld.js versions on one page
From an ease of site development viewpoint, I think we should just put the most recent jsonld.js on the playground and add a UI control to pick the processingMode API option. Due to practicalities of jsonld.js not having a full correct 1.0 only lib, it seems not worth the effort to try and deal with this any other way. There are edge cases where a 1.1 lib in 1.0 mode will produce different results than a 1.0 lib. My guess is that in practice this really doesn't matter. Or in any case, is not worth handling on the playground.