Niklas Lindström: I'd like to mention the status of JSON-LD in RDFLib in the implementation report.
… I have toRdf covered except for 8 tests.
Topic: OData / JSON-LD Alignment
Manu Sporny: It's unfortunate that the two groups didn't get together sooner to collaborate, but we're here now, so let's see what we can do to align.
Michael Pizzo: I agree that it's unfortunate that the two groups didn't get together sooner, because we could have done a lot to align the formats.
… By opening up the conversation we may be able to align more closely in the future.
… This came about when I was in a schema.org and we were discussing how to represent such data in JSON-LD or OData.
… We're trying to do the same thing in either format. Forgetting about the keywords, and look at say a schema:Movie, it would look pretty much the same in either format.
… What would be different is how we would add type or other context information to the payload.
… We're trying to add info to the payload, but we're doing it differently.
… We both add type, context and aliasing information.
… It would be great if JSON-LD and OData could understand the same keyword. If there were a common way to add keywords, we could over time understand the meaning of those keywords.
… It would be nice if we could distinguish between different meanings for overlapping keywords.
… That was the genesis of the email by examining both forms and trying to see how we can manage them together by trying to look for keyword constructs and wouldn't confuse each other.
Manu Sporny: most of what you've said would have broad agreement with this group.
… I think the question we explored during the last call is if we don't do anything now, is there something we could do in the future, and I think there is.
… There's also some questions on the details of how you would do this.
… We understand that OData uses application/json, which could allow for conflicts with other keywords (e.g., "type" could conflict with other uses in application/json)
… One of the things JSON-LD did was use a different mime-type, so that the meaning of keywords (e.g., @id, @type) is well-understood.
… The question was, should there be some higher-level mime type to say that a doc could contain both JSON-LD and OData, or could there be a profile of OData for JSON-LD, so that you could use application/ld+json with an OData context so that both OData and JSON-LD keywords would have unambigious meaning.
… At this point, we have both specs in their final steps, and a lot of deployed data in JSON-LD, so that changing keywords in JSON-LD now would be problematic, but we think we could do something in a 1.1 or 2.0 release.
… Did the OData group discuss mime/type or the use of a JSON-LD context.
Michael Pizzo: We added format parameters added to application/json to specify the keywords that might be added. The default is to add a minimal number of keywords to the payload, or you can add all or no keywords.
… The OData spec at OASIS is really version 4; we had 3 previous versions developed in an open process outside of a standards community.
... In versions 1 and 2, we did something quite similar to JSON-LD. We had "metadata" with keywords prefaced by __. You can think of metadata like a JSON-LD context.
…We got feedback that there was too much stuff that didn't look like JSON. It was a JSON encoding, but didn't look like what people thought JSON should look like
… With version 3 and into the current version 4, we've tried to look as much as possible like normal JSON with minimal keywords, similar to what you've done with @context in the header that can represent out-of-band information about the representation of the payload.
… Part of doing this was to just follow the rules of JSON and stick with application/json; this poses the threat of overlap. Originally we though about using "." in the names to distinguish between them.
… We used "." as it wasn't implemented in a normal Serializer/deserializer
… We got feedback that eve though "." was a pain, there are other systems that also use ".", so we did end up adopting the "@" prefix in OData's JSON format. Someone could use "@odata.type" and it could overlap, but that seemed like a remote possibility.
… Your options for moving forward seem interesting; I like the idea that we could add OData information to a JSON-LD payload.
… We could also use a more pure-JSON convention. The fact that you're using unqualified keywords in your MIME type, doesn't prevent different keywords using a prefix from being used in a straight JSON format.
Manu Sporny: if I understand you, OData currently uses the "@" symbol, or you're thinking about that?
Michael Pizzo: I just pushed through a change which delayed the standard, but this adds the @ prefix. In the future, if we want to allow a convention for adding things to application/json, this might help.
Manu Sporny: this is something we seem to have agreement on.
… it feels like the JSON community might push back on having "@" prefix mean something special, but then we'd just need to move to a different mime type.
… We're both in very similar circumstances, and the JSON-LD group will continue to operate for the foreseeable future.
… We should push our respective specs out and look at alignment in the next iteration. Perhaps it's @odata. or a registry with both OData and JSON-LD keywords.
Gregg Kellogg: how would a registry work? something that tries to prevent conflicts with overlapping specifications? [scribe assist by Dave Longley]
… The idea is that if OData and JSON-LD deviated completely, the next thing we could do would be to create a common mime type extending application/json to use a set of keywords, and specify in some registry that there are some keywords that can be used for metadata expression and include both OData and JSON-LD keywords
… The third option is that JSON-LD could create an OData JSON-LD context that would specify all the OData keywords so that JSON-LD could be compatible with all OData documents.
Niklas Lindström: I find it a bit disconcerting that we're talking about registries, as the mime-type solution is already specified for doing things like this.
Markus Lanthaler: +1 what niklas just said
… I'd like when we continue with this to consider the actual use case and what a data-consumer needs to do to deal with this. JSON-LD is trying to reduce the number of keywords, and most importantly, be based on RDF to reduce the representation complexity.
… This makes the problem "done" from my perspective, but that the core purpose of JSON-LD is to encode the RDF, which has very well understood semantic meaning.
Gregg Kellogg: +1 to niklas
Michael Pizzo: In response to Niklas, I agree that we look at specific scenarios and solve real problems.
… That's why I prefaced my discussion with my use cases using schema.org.
… The problem with using a mime type is that it makes interoperability more difficult.
… I don't know if it's a registry, or namespacing, but we need to do something.
… I don't think the idea is that far-fetched, and there are advantages of being able to specify both keywords, and having a common way where there's overlap.
Dave Longley: Before we go too far in other directions, we should try solution #3 and see if we can do this with an OData contaxt in JSON-LD.
Niklas Lindström: +1 to dlongley (JSON-LD should be able to support the use cases of OData in and of itself)
Manu Sporny: I think this would be the simplest way to get alignment. If we can do this, then problem is solved.
… The remaining problem is you need to use application/ld+json, or the HTTP header.
Michael Pizzo: I'd love to do that; some document that said how to represent both JSON-LD and OData keywords would be hugely valuable.
… I'd love to work on some examples of how to do this.
Manu Sporny: just keep doing what you're doing, bring it up on a call, IRC or on the mailing list, and you'll get a lot of help.
Gregg Kellogg: I wanted to echo something Niklas said in his comments, about RDF being the underlying model here... the real value here. With respect to schema.org, it's an RDF vocabulary, it's just an exercise to how to lay that out in JSON-LD. [scribe assist by Manu Sporny]
Gregg Kellogg: Was there a consideration in OData to align with RDF? Could we use that as a rubric in how we resolve these things. Data model vs. syntactic model/overlay. [scribe assist by Manu Sporny]
Michael Pizzo: Certainly OData has been looking at RDF for a while and trying to understand the difference of what RDF/SPARQL provide and what OData provides. I view the two as being very complementary.
… RDF is very general, and OData is a bit more structured/constrained. When you have an Entity model, it makes it easy to expose and consume such data.
… Im speaking at ISWC in Sydney later this month on how to do this in a triple-store.
… I think it's interesting for having an RDF format for OData, we have both Atom and JSON formats; being able to describe this using say Turtle, it would work very naturally. Then the RDF would be a way of representing the OData information.
… I talked with TimBL and some others last year, and there was quite a bit of interest in getting triples out of OData.
… When the data starts out as triples, there's a natural way to expose it as an OData service.
… It's also interesting when something has some structured data to be able to expose that as OData, but make it available to the RDF community.
… We might already be there with where JSON-LD is.
Manu Sporny: In general, we're just talking about a way of exposing data and any format could potentially be sucked in to an RDF triple store. We're now getting out linked data formats that can be expressed in this way.
… The problem is that it's not clear which of the formats will work well in the future, but we need to be sure that whichever one does well has a way of being mapped to a universal data format, such as RDF.
… I view JSON-LD as being an object format which translates to RDF very well, but there's not necessarily a need to translate it to RDF.
Gregg Kellogg: it would be useful to highlight different use cases for OData and JSON-LD.
Gregg Kellogg: It would be useful to highlight where OData is better for some use cases and places where JSON-LD is better for other use cases. [scribe assist by Manu Sporny]
Michael Pizzo: OData is more than just a format, it describes formats for URLs, semantics for updates, and additional headers. There's more to it than just formats, in a sense, it allows pluggable modules.
Niklas Lindström: that sounds like an important path to explore.
… From an RDF perspective, I don't think of it too much as Triples, but the formation of expressions; RDF represents graph of relationships between things, and the triple store is just one of many ways to represent this, but so is MongoDB, for example.
… RDF is also format independent, note the Linked Data working group, and there are existing REST mechanisms for RDF, including Atom.
… That's a good direction, as it then becomes about the vocabularies that are used. That's where the real value of interoperability comes from.
Manu Sporny: there's a lot to discuss here, and it will take multiple weeks on the further work we want to collaborate on. In the mean time, we're at the end of the CR phase, and I don't think there's much we can do in the next three months to go into the REC.
… I think we have what we need for basic alignment; we can build on "@" as a way of identifying a keyword, and we can talk about mime types over time.
… We need a document that outlines when you might use one or the other.
… Mike, do you have some specific areas you think we should focus on in the short-term?
Michael Pizzo: I realize we're both trying to get things out, but I'd love to come up with a proof-of-concept that shows a JSON-LD document using OData keywords that can be interpreted by either system.
… I'm hopeful that that could be fairly quick, and get some pointers to how this could be easier in a JSON-LD 1.1, for example.
… If there's a way in JSON-LD to register a pattern or prefix that could be used for namespacing.
Manu Sporny: if you could markup Person, Place and Event in OData, a number of us could create a JSON-LD context to show how this might work.
Paul Kuykendall: Interesting that JSON-LD isn't used much as an RDF format, we are doing this and find it very useful with JSON-LD; we previously tried to work with OData, but ran into problems. I think there's a lot of interest in the industry for using JSON-LD as a serialization format.
… If we could bridge the gap, it would be a fantastic win.
Michael Pizzo: i'd love to work with people that have a foot in both camp. In particular, to see what you couldn't represent.
Paul Kuykendall: this was back at version 2-3.
Michael Pizzo: email me at mikep@microsoft.com
Manu Sporny: I think we have a concrete direction to move forward on based on working off of specific schema.org examples.
Michael Pizzo: I wanted to thank the group for discussing this.
Manu Sporny: likewise, I hope we can continue this type of collaboration.
Manu Sporny: there was a bug found in the spec for which a test case was added.
… I understand that we have 3 implementations, dlongley's markus' and gregg's
Paul Kuykendall: I've got permission to submit a report, but I'm working on issues.
… We could use a week to resolve the next 10 bugs.
Manu Sporny: we can hold off until you're ready with a report, but we're waiting on you guys.
Niklas Lindström: I suspect it would be valuable to have a report for RDFLib, although the processor is limited to fromRdf and toRdf. There is some support for compact/expand.
… Compaction is not really something that is important, so it always flattens.
… It does also expand and flatten. I could probably do some coverage on expand and flatten.
… If the deadline is extended to next Tuesday, that would help.
Manu Sporny: as long as you need, but we'd like to wrap this up.
… This is the stage that matters, because we can go to W3C management and show broad support, including reports we weren't expecting.
Paul Kuykendall: we can do ours by next week with no problems.
Gregg Kellogg: Gavin has done some work trying to deal with large JSON-LD graphs and has a problem w/ a number of the different implementations taking a long time to work through very large documents. [scribe assist by Manu Sporny]
Gregg Kellogg: We have some issues w/ sorting - mostly for bnode naming, so having a profile that doesn't require that would drop the time down considerably, for very large datasets, etc. [scribe assist by Manu Sporny]
Gregg Kellogg: That would be useful as a note. [scribe assist by Manu Sporny]
Manu Sporny: we tried to keep it streaming.
Gregg Kellogg: I'm also pushing to get Java JSON-LD folks to submit stuff. [scribe assist by Manu Sporny]
David I. Lehn: Can we add some tests for bugs that have been found?
Manu Sporny: if we add those tests, the risks are that implementations will break, and we'll need to re-submit reports.
… This is always a problem with extending CR, as there will always be bugs that are found.
Gregg Kellogg: I think generating a new report is easy, but this is a neverending thing. Do the tests point out flaws in the specifications? Something you couldn't implement correctly, rather than finding areas where there are inconsistencies where there is vagueness around the algorithms. [scribe assist by Manu Sporny]
Dave Longley: @base: "http://foo/bar" ... does that compact: "http://foo/bar" to "bar" or "" ?
David I. Lehn: basically path and base handling things.
David I. Lehn: This mostly has to do w/ path handling and handling with base. [scribe assist by Manu Sporny]
Markus Lanthaler: @base: "http://foo/bar" does compact "http://foo/bar" to "bar" IMO
Niklas Lindström: Setting @base to null, when the result is a "hard" relative IRI, RDFlib still consumes it as RDF. I would pass the test if I set a flag about being more strict about not explicitly dealing with relative IRIs.
… It expects to drop the triples with relative IRIs
Dave Longley: the "produce generalized RDF" flag should allow these triples to be emitted.
Niklas Lindström: The other problems are RDFLib limitations.