JSON-LD Framing allows developers to query by example and force a specific tree layout to a JSON-LD document.

This document is an experimental work in progress.

Introduction

A JSON-LD document is a representation of a directed graph. A single directed graph can have many different serializations, each expressing exactly the same information. Developers typically work with trees, represented as JSON objects. While mapping a graph to a tree can be done, the layout of the end result must be specified in advance. A Frame can be used by a developer on a JSON-LD document to specify a deterministic layout for a graph.

How to Read this Document

This document is a detailed specification for a serialization of Linked Data in JSON. The document is primarily intended for the following audiences:

To understand the basics in this specification you must first be familiar with JSON, which is detailed in [[!RFC7159]]. You must also understand the JSON-LD Syntax [[!JSON-LD]], which is the base syntax used by all of the algorithms in this document, and the JSON-LD API [[!JSON-LD-API]]. To understand the API and how it is intended to operate in a programming environment, it is useful to have working knowledge of the JavaScript programming language [[ECMA-262]] and WebIDL [[!WebIDL-2]]. To understand how JSON-LD maps to RDF, it is helpful to be familiar with the basic RDF concepts [[!RDF-CONCEPTS]].

Contributing

There are a number of ways that one may participate in the development of this specification:

Terminology

Typographical conventions

Features

Framing

Framing is used to shape the data in a JSON-LD document, using an example frame document which is used to both match the flattened data and show an example of how the resulting data should be shaped. Matching is performed by using properties present in in the frame to find objects in the data that share common values. Matching can be done either using all properties present in the frame, or any property in the frame. By chaining together objects using matched property values, objects can be embedded within one another.

A frame also includes a context, which is used for compacting the resulting framed output.

For example, assume the following JSON-LD frame:

    
    

This frame document describes an embedding structure that would place objects with type ex:Library at the top, with objects of type ex:Book that were linked to the library object using the ex:contains property embedded as property values. It also places objects of type ex:Chapter within the referencing ex:Book object as embedded values of the book object.

When using a flattened set of objects that match the frame components:

    
    

The Frame Algorithm can create a new document which follows the structure of the frame:

    
    

The Framing Algorithm does this by first expanding both the input frame and document. It then creates a map of flattened subjects. The outer-most node object within the frame is used to match objects in the map, in this case looking for node objects which have an @type of Library, and a contains property with another frame used to match values of that property. The input document contains exactly one such node object. The value of contains also has a node object, which is then treated as a frame to match the set of subjects which are contains values of the Library object, and so forth.

Default content

A frame may specify properties that don't exist in an input file. If the explicit inclusion flag is false, the framing algorithm will add a property and value to the result. The @default property in a node object or value object provides a default value to use in the resulting output document. If there is no @default value, the property will be output with a null value. (See for ways to avoid this).

The value of the property in the frame is not otherwise used in the output document. It's purpose is for frame matching and finding default values. Note the description value for Library in the following example.

    
    
    
    

Framing Flags

Framing can be controlled using API options, or by adding framing keywords within the frame as described in .

Framing flags set using keywords have effect only for the frame in which they appear, and for implicit frames which are created for objects where no frame object exists.

Object Embed Flag

The object embed flag determines if a referenced node object is embedded as a property value of a referencing object, or kept as a node reference. The initial value for the object embed flag is set using the embed option. Consider the following frame based on the default @last value of the object embed flag:

      
      

Because, the default for the object embed flag is @last (in addition to the explicit inclusion flag being false), non-listed properties are added two the output, and implicitly embedded using a default empty frame. As a result, the same output used in the Framed library objects above is generated.

However, if the @embed property is added explicitly with a value of @never, the values for Book and Chapter will be excluded.

      
      
      
      

Explicit inclusion flag

The explicit inclusion flag used to determine properties which will be included in the output document. The default value is false, which means that properties present in an input node object that are not in the associated frame will be included in the output object. The initial value for the explicit inclusion flag is set using the explicit option. If true, only properties present in the input frame will be placed into the output.

For example, take an expanded version of the library frame which include some properties from the input, but omit others.

      
      

The resulting output will exclude properties for Book which are not explicitly listed in the frame object:

    
    

Omit default flag

The omit default flag changes the way framing generates output when a property described in the frame is not present in the input document. The initial value for the omit default flag is set using the omitDefault option. See for a futher discussion.

Require all flag

The require all flag is used in frame matching to determine when a node object from an input document matches a frame. When matching, an object may include @type and other properties, a match is made when any property value in the object matches the node pattern in the frame object if the value of the require all flag is false (the default). If the flag value is true, then all properties in the frame object must be present in the node object for the node to match.

Reverse Framing

A frame may include @reverse, or a value of a term defined using @reverse to invert the relationships in the output object. For example, the Library example can be inverted using the following frame:

    
    

Using the flattened library example above, results in the following:

    
    

Framing Named Graphs

Frames can include @graph, which allows information from named graphs contained within a JSON-LD document to be exposed within it's proper graph context. By default, framing uses a merged graph, composed of all the node objects across all graphs within the input. By using @graph within a frame, the output document can include information specifically from named graphs contained within the input document.

The following example uses a variation on our library theme where information is split between the default graph, and a graph named http://example.org/graphs/books:

    
    
    
    
    
    

There is one class of products that can claim conformance to this specification: JSON-LD Processors.

A conforming JSON-LD Processor is a system which can perform the Framing operation in a manner consistent with the algorithms defined in this specification.

JSON-LD Processors MUST NOT attempt to correct malformed IRIs or language tags; however, they MAY issue validation warnings. IRIs are not modified other than conversion between relative and absolute IRIs.

The algorithms in this specification are generally written with more concern for clarity than efficiency. Thus, JSON-LD Processors MAY implement the algorithms given in this specification in any way desired, so long as the end result is indistinguishable from the result that would be obtained by the specification's algorithms.

Implementers can partially check their level of conformance to this specification by successfully passing the test cases of the JSON-LD test suite [[JSON-LD-TESTS]]. Note, however, that passing all the tests in the test suite does not imply complete conformance to this specification. It only implies that the implementation conforms to aspects tested by the test suite.

Framing Algorithms

All algorithms described in this section are intended to operate on language-native data structures. That is, the serialization to a text-based JSON document isn't required as input or output to any of these algorithms.

Reference to JSON data structures are interpreted using their abstract representation for the purpose of describing algorithms.

Syntax Tokens and Keywords

This specification adds a number of keywords (framing keywords) to the ones defined in the [[!JSON-LD]] specification:

@default
Used in Framing to set the default value for an output property when the framed node object does not include such a property.
@embed
Used in Framing to override the value of object embed flag within a specific frame. Valid values for @embed as the following:
@always
Always embed node objects as property values, unless this would cause a circular reference.
@last
Only the last value within a given node object should be embedded, previous values of other properties use a node reference. This is the default value if neither @embed nor object embed flag is not specified.
@link
In the in-memory abstract representation, nodes are linked directly, which allows for circular references. Always use a node reference when serializing matching values. Do we want to describe this here? Or, perhaps in a separate JSON-LD Linking document?
@never
Always use a node reference when serializing matching values.
true
(equivalent to @last).
false
(equivalent to @never).

Any other value for @embed is invalid and indicates that an invalid @embed value error has been detected and processing is aborted.

@explicit
Used in Framing to override the value of explicit inclusion flag within a specific frame.
@null
Used in Framing when a value of null should be returned, which would otherwise be removed when Compacting.
@omitDefault
Used in Framing to override the value of omit default flag within a specific frame.
Is this flag really useful? Easier to simply not have a default value, if it shouldn't match.
@requireAll
Used in Framing to override the value of require all flag within a specific frame.

All JSON-LD tokens and keywords are case-sensitive.

Framing

Framing is the process of taking a JSON-LD document, which expresses a graph of information, and applying a specific graph layout (called a Frame).

Framing makes use of the Node Map Generation algorithm to place each object defined in the JSON-LD document into a map of flattened subjects, allowing them to be operated upon by the Framing algorithm.

Framing Requirements

A valid JSON-LD Frame is a superset of a valid JSON-LD document, allowing additional content, which is preserved through expansion. The Grammar defined in [[JSON-LD]] is extended as follows:

Framing Algorithm

The framing algorithm takes an JSON-LD input (expanded input), which MUST be a JSON-LD document in expanded form, an input frame (expanded frame), which MUST be a JSON-LD frame in expanded form, a context (context), and a number of options and produces JSON-LD output.

If an error is detected in the expanded frame, a processor MUST generate a JsonLdFramingError with code set to invalid frame. Need more specifics as to what constitutes a valid frame.

Set graph map to the result of performing the Node Map Generation algorithm on expanded input.

If the frameDefault option is present with the value true, set graph name to @default. Otherwise, create merged node map using the the Merge Node Maps algorithm with graph map and add merged node map as the value of @merged in graph map and set graph name to @merged.

The recursive algorithm operates with a framing state (state), created initially using the object embed flag set to true, the explicit inclusion flag set to false, the require all flag set to true, the omit default flag set to false, graph map, graph name, along with map of flattened subjects set to the property associated with graph name in graph map, graph stack set to an empty array, and link set to an empty dictionary. The initial values of the object embed flag, require all flag, and omit default flag MUST be overridden by values set in options. Also initialize results as an empty array.

Processors MAY use other runtime options to set different framing state defaults for values of state.

Invoke the recursive algorithm using framing state (state), the keys from the map of flattened subjects as subjects, expanded frame (frame), result as parent, and null as active property.

The recursive algorithm adds elements to parent either by appending the element to parent, if it is an array, or by appending it to an array associated with active property in parent, if it is a dictionary. Note that if parent is an array, active property MUST be null, and if it is a dictionary, it MUST NOT be null.

The following series of steps is the recursive portion of the framing algorithm:

  1. If frame is an array, set frame to the first member of the array, which MUST be a valid frame.
  2. Initialize flags embed, explicit, and requireAll from object embed flag, explicit inclusion flag, and require all flag in state overriding from any property values for @embed, @explicit, and @requireAll in frame.
  3. Create a list of matched subjects by filtering subjects against frame using the Frame Matching algorithm with state, subjects, frame, and requireAll.
  4. Set link the the value of link in state associated with graph name in state, creating a new empty dictionary, if necessary.
  5. For each id and associated node object node from the set of matched subjects, ordered by id:

    Can we remove sorting, or make it subject to a processing flag? In general, sorting is a performance problem for JSON-LD, and inhibits stream processing.

    1. Initialize output to a new dictionary with @id and id and add output to link associated with id.
    2. If embed is @link and id is in link, node already exists in results. Add the associated node object from link to parent and do not perform additional processing for this node.
    3. Otherwise, if embed is @never or if a circular reference would be created by an embed, add output to parent and do not perform additional processing for this node.
    4. Otherwise, if embed is @last, remove any existing embedded node from parent accociate with graph name in state. Requires sorting of subjects. We could consider @sample, to embed just the first matched node. With sorting, we could also consider @first.
    5. If embed is @last or @always
      1. If graph map in state has an entry for id:
        1. If frame does not have the key @graph, set recurse to true, unless graph name in state is @merged and set subframe to a new empty dictionary.
        2. Otherwise, set subframe to the first entry for @graph in frame, or a new empty dictionary, if it does not exist, and set recurse to true, unless graph name in state is @merged or @default.
        3. If recurse is true:
          1. Push graph name from state onto graph stack in state.
          2. Set the value of graph name in state to id.
          3. Invoke the recursive algorithm using state, the keys from the graph map in state associated with id as subjects, subframe as frame, output as parent, and @graph as active property.
          4. Pop the value from graph stack in state and set graph name in state back to that value.
      2. For each property and objects in node, ordered by property:
        1. If property is a keyword, add property and objects to output.
        2. Otherwise, if property is not in frame, and explicit is true, processors MUST NOT add any values for property to output, and the following steps are skipped.
        3. For each item in objects:
          1. If item is a dictionary with the property @list, then each listitem in the list is processed in sequence and added to a new list dictionary in output:
            1. If listitem is a node reference, invoke the recursive algorithm using state, the value of @id from listitem as the sole member of a new subjects array, the first value from @list in frame as frame, list as parent, and @list as active property. If frame does not exist, create a new frame using a new dictionary with properties for @embed, @explicit and @requireAll taken from embed, explicit and requireAll. Could this use the list array, and null for active property?
            2. Otherwise, append a copy of listitem to @list in list.
          2. If item is a node reference, invoke the recursive algorithm using state, the value of @id from item as the sole member of a new subjects array, the first value from property in frame as frame, output as parent, and property as active property. If frame does not exist, create a new frame using a new dictionary with properties for @embed, @explicit and @requireAll taken from embed, explicit and requireAll.
          3. Otherwise, append a copy of item to active property in output.
      3. For each non-keyword property and objects in frame that is not in output:
        1. Let item be the first element in objects, which MUST be a frame object.
        2. Set property frame to the first item in objects or a newly created frame object if value is objects. property frame MUST be a dictionary.
        3. Skip property and property frame if property frame contains @omitDefault with a value of true, or does not contain @omitDefault and the value of the omit default flag is true.
        4. Add property to output with a new dictionary having a property @preserve and a value that is a copy of the value of @default in frame if it exists, or the string @null otherwise.
      4. If frame has the property @reverse, then for each reverse property and sub frame that are the values of @reverse in frame:
        1. Create a @reverse property in output with a new dictionary reverse dict as its value.
        2. For each reverse id and node in the map of flattened subjects that has the property reverse property containing a node reference with an @id of id:
          1. Add reverse property to reverse dict with a new empty array as its value.
          2. Invoke the recursive algorithm using state, the reverse id as the sole member of a new subjects array, sub frame as frame, null as active property, and the array value of reverse property in reverse dict as parent.
      5. Once output has been set are required in the previous steps, add output to parent.

Using result from the recursive algorithm, set compacted results to the result of using the compact method using results, context, and options.

If compacted results does not have a top-level @graph keyword, or if its value is not an array, modify compacted results to place the non @context properties of compacted results into a dictionary contained within the array value of @graph.

Recursively, replace all key-value pairs in compacted results where the key is @preserve with the value from the key-pair. If the value from the key-pair is @null, replace the value with null. If, after replacement, an array contains only the value null remove the value, leaving an empty array.

Return compacted results.

Frame Matching Algorithm

The Frame Matching Algorithm is used as part of the Framing algorithm to determine if a particular node object matches the critera set in a frame. In general, a node object matches a frame if it meets the matches on @type, or @id, or if it matches given one of several different properties (or all properties, if the require all flag is present.).

As matching is performed on expanded node objects, all values will be in the form of an array.

Node matching uses a combination of JSON constructs to match any, zero, or some specific values:

[] (match none)
An empty array matches no values, or a value which is, itself, an empty array.
{} (wildcard)
An array containing an empty object (after excluding any properties which are framing keywords) matches any value that is present, and does not match if there are no values.
[IRI+]
One or more strings in the form of an IRI, used for matching on @type and @id, which allows a match on any of the listed IRIs.
[frame object] (node pattern)
A non-empty frame object, used to match specific values using recursive node matching.
[value object] (value pattern)
A value object, used to match a specific value. Within a value object, the values for @value, @type, and @language may also be an array of one or more string values.

The frame matching algorithm takes the framing state (state), a list of subjects to match from the map of flattened subjects (subjects), a frame to match against (frame), and the requireAll flag and returns a list of matched subjects by filtering each node in subjects as follows:

Frame matching follows an order of precedence, first attempting to match on a particular @id, then a particular @type (or lack of @type), then by matching on any or all of a set of properties, if neither @id, nor @type are in the frame.

  1. Node matches if it has an @id property including any IRI or blank node in the @id property in frame.
    Framing works on map of flattened subjects, and the act of flattening ensures that all subjects have an `@id` property; thus the `"@id": []` pattern would never match any node object.
  2. Node matches if frame has no non-keyword properties.
  3. If requireAll is true, node matches if all non-keyword properties (property) in frame match any of the following conditions. Or, if requireAll is false, if any of the non-keyword properties (property) in frame match any of the following conditions. For the values of each property from frame in node:
    1. If property is @type:
      1. Property matches if the @type property in frame includes any IRI in values.
      2. Otherwise, property matches if values is not empty and the @type property in frame is wildcard.
      3. Otherwise, property matches if values is empty and the @type property in frame is match none.
      4. Otherwise, property does not match.
    2. Otherwise, the value of property in frame MUST be empty, or an array containing a valid frame.
    3. Property matches if values is empty, or non existant, the value of property in frame is a dictionary containing only the key @default with any value, and any other property in node has a non-default match.
    4. Node does not match if values is not empty and the value of property in frame is match none, and further matching is aborted.
    5. Otherwise, property matches if values is not empty and the value of property in frame is wildcard.
    6. Otherwise, if the value of property in frame is a value pattern (value pattern): property matching is determined using the Value matching algorithm.
    7. Otherwise, for any node pattern (node pattern) which is one of the values of property in frame:
      1. Let value subjects be the list of subjects from the map of flattened subjects matching the node object values from values.
      2. Let matched subjects be the result of calling this algorithm recursively using state, value subjects for subjects, node pattern for frame, and the requireAll flag.
      3. Property matches if matched subjects is not empty.
    8. Otherwise, property does not match.

Value Pattern Matching Algorithm

The Value Pattern Matching Algorithm is used as part of the Framing and Frame Matching algorithms. A value object matches a value pattern using the match none and wildcard patterns on @value, @type, and @language, in addition to allowing a specific value to match a set of values defined using the array form for each value object property.

The algorithm takes a value pattern (pattern) and value object (value) as parameters. Value matches pattern using the following algorithm:

  1. Let v1, t1, and l1 be the values of @value, @type, and @language in value, or null if none exists.
  2. Let v2, t2, and l2 be the values of @value, @type, and @language in value pattern, or null if none exists.
  3. Value matches pattern when pattern is wildcard, or:
    1. v1 is in v2, or v1 is not null and v2 is wildcard, and
    2. t1 is in t2, or t1 is not null and t2 is wildcard, or null, or t1 is null and t2 is null or match none, and
    3. l1 is in l2, or l1 is not null and l2 is wildcard, or null, or l1 is null and l2 is null or match none.

The Application Programming Interface

This API provides a clean mechanism that enables developers to convert JSON-LD data into a a variety of output formats that are easier to work with in various programming languages. If a JSON-LD API is provided in a programming environment, the entirety of the following API MUST be implemented.

JsonLdProcessor

The JSON-LD Processor interface is the high-level programming structure that developers use to access the JSON-LD transformation methods. The definition below is an experimental extension of the interface defined in the [[JSON-LD-API]].

The JsonLdProcessor interface frame method Frames the given input using frame according to the steps in the Framing Algorithm:

  1. Create a new Promise promise and return it. The following steps are then executed asynchronously.
  2. Set expanded input to the result of using the expand method using input and options.
  3. Set expanded frame to the result of using the expand method using frame and options with expandContext set to null and processingMode set to "json-ld-framing-1.1-expand-frame".
  4. Set context to the value of @context from frame, if it exists, or to a new empty context, otherwise.
  5. If frame has a top-level property which expands to @graph set the frameDefault option to options with the value true.
  6. Set framed to the result of using the Framing algorithm, passing expanded input, expanded frame, context, and options.
  7. Fulfill the promise passing framed.
input
The JSON-LD object or array of JSON-LD objects to perform the framing upon or an IRI referencing the JSON-LD document to frame.
frame
The frame to use when re-arranging the data of input; either in the form of an JSON object or as IRI.
options
A set of options that MAY affect the framing algorithm such as, e.g., the input document's base IRI.

Error Handling

JSON-LD Framing extends the error interface and codes defined in [[JSON-LD-API]].

invalid frame
The frame is invalid.
invalid @embed value
The value for @embed is not one recognized for the object embed flag.

Data Structures

This section describes datatype definitions used within the JSON-LD API.

JsonLdContext

The JsonLdContext type is used to refer to a value that that may be a JSON object, a string representing an IRI, or an array of JSON objects and strings.

See JsonLdContext definition in [[!JSON-LD-API]].

JsonLdOptions

The JsonLdOptions type is used to pass various options to the JsonLdProcessor methods.


    

In addition to those options defined in [[JSON-LD API]], framing defines these additional options:

embed
Sets the value object embed flag used in the Framing Algorithm. A boolean value of true sets the flag to @last, while an value of false sets the flag to @never.
explicit
Sets the value explicit inclusion flag used in the Framing Algorithm.
omitDefault
Sets the value omit default flag used in the Framing Algorithm
requireAll
Sets the value require all flag used in the Framing Algorithm
frameDefault
Instead of framing a merged graph, frame only the default graph.

See JsonLdOptions definition in [[!JSON-LD-API]].

IANA Considerations

This section is included merely for standards community review and will be submitted to the Internet Engineering Steering Group if this specification becomes a W3C Recommendation.

application/ld-frame+json

Type name:
application
Subtype name:
ld-frame+json
Required parameters:
None
Optional parameters:
None
Encoding considerations:
The same as the application/json MIME media type.
Security considerations:
Since a JSON-LD frame is intended to specify a deterministic layout for a JSON-LD graph, the serialization SHOULD NOT be passed through a code execution mechanism such as JavaScript's eval() function. It is RECOMMENDED that a conforming parser does not attempt to directly evaluate the JSON-LD frame and instead purely parse the input into a language-native data structure.
Interoperability considerations:
Not Applicable
Published specification:
The JSON-LD specification.
Applications that use this media type:
Any programming environment that requires the exchange of directed graphs. Implementations of JSON-LD have been created for JavaScript, Python, Ruby, PHP and C++.
Additional information:
Magic number(s):
Not Applicable
File extension(s):
.jsonldf
Macintosh file type code(s):
TEXT
Person & email address to contact for further information:
Manu Sporny <msporny@digitalbazaar.com>
Intended usage:
Common
Restrictions on usage:
None
Author(s):
Manu Sporny, Gregg Kellogg, Markus Lanthaler, Dave Longley
Change controller:
W3C

Fragment identifiers have no meaning with application/frame-ld+json resources.

Open Issues

The following is a list of open issues being worked on for the next release.

Acknowledgements

A large amount of thanks goes out to the JSON-LD Community Group participants who worked through many of the technical issues on the mailing list and the weekly telecons - of special mention are Niklas Lindström, François Daoust, and Zdenko 'Denny' Vrandečić. The editors would like to thank Mark Birbeck, who provided a great deal of the initial push behind the JSON-LD work via his work on RDFj. The work of Dave Lehn and Mike Johnson are appreciated for reviewing, and performing several implementations of the specification. Ian Davis is thanked for this work on RDF/JSON. Thanks also to Nathan Rixham, Bradley P. Allen, Kingsley Idehen, Glenn McDonald, Alexandre Passant, Danny Ayers, Ted Thibodeau Jr., Olivier Grisel, Josh Mandel, Eric Prud'hommeaux, David Wood, Guus Schreiber, Pat Hayes, Sandro Hawke, and Richard Cyganiak for their input on the specification.