JSON-LD Community Group Telecon

Minutes for 2013-08-20

  1. @base and NQuad output
  2. Vikash update for GSoC
  3. Decision to go CR or PR
  1. Add an issue marker for the "@base": null feature in the JSON-LD API CR specification - it may be modified or removed.
  2. Drop relative IRI in the serialize to RDF algorithm. JSON-LD processors are free to include an option that preserves relative IRIs when serializing to RDF.
  3. Request that the RDF WG transition the JSON-LD 1.0 and JSON-LD 1.0 API specs to CR, using the shortest possible CR period (don't skip CR)
Manu Sporny
Dave Longley
Dave Longley, Manu Sporny, Gregg Kellogg, Markus Lanthaler, David I. Lehn, Vikash Agrawal
Audio Log
Dave Longley is scribing.
Manu Sporny: we may need to discuss @base a bit, but first an update from vikash on GSoC, and then a discussion about going to CR or skipping to PR

Topic: @base and NQuad output

Gregg Kellogg: the problem i came across was passing all tests but one
Gregg Kellogg: i didn't pass because the test is for the presence of relative IRIs in the output, it's effectively the same as the related expand test, where @base is set to null, so the relative IRIs are left alone
Gregg Kellogg: that's fine with JSON-LD (but perhaps dubious IMO), but not ok with RDF
Gregg Kellogg: i proposed a solution and implemented it, where the test rules would say that any resulting relative IRIs should be joined against some specific test base in order to get absolute IRIs out of it, markus, i believe objected to that because it required post processing
Gregg Kellogg: and pointed out the fact that the algorithm should just omit RDF, another proposal would be to drop relative IRIs but that wuold require a change to the algorithms or generate an error, etc.
Gregg Kellogg: but it would be illegal/illegitimate in my view to output relative IRIs in RDF
Gregg Kellogg: markus recommended we revisit @base: null
Gregg Kellogg: when i saw the effect of setting @base: null, it was counter-intuitive, i thought it was meant to set @base back to the original base
Gregg Kellogg: i didn't understand the use case
Manu Sporny: we wanted to have bundled apps with relative URLs in the app manifests, etc.
Manu Sporny: this came up but i forget who had the use case, but they wanted relative URLs for widgets/bundled apps
Manu Sporny: and not have that resolved
Manu Sporny: the other use case is for having something on the wire where some other application should decide the @base
Gregg Kellogg: this reminds me of n3 (i think) which lets you clear any base that was set
Gregg Kellogg: when you set that it does for back to the original document base, however, if there was no original document base, if it was a pure set of notation not living somewhere then it would continue as a relative IRI
Manu Sporny: that would still be illegal N-Quads right?
Gregg Kellogg: well, the tests wouldn't be a problem because there's always a document base
Gregg Kellogg: at least in my case there is parser behavior when there is no document base
Markus Lanthaler: i don't think there's any feature elsewhere that says, even though there is a base you override it
Markus Lanthaler: we have an additional feature where we say we have no base
Manu Sporny: so what you're proposing is falling back
Dave Longley: we're forgetting the discussion we had, we're rehashing the same arguments
Manu Sporny: i'm wary of changing this now
Gregg Kellogg: we need to be able to test this
Manu Sporny: yeah, there are two things, we need to fix the test, and the other thing is markus is questioning the feature
Manu Sporny: i think fixing the test is fairly easy and removing the feature is way more complicated, and we shouldn't try to fix it right before going into CR
Manu Sporny: we may just add an issue marker to remove the feature after implementation feedback
Gregg Kellogg: we could just reinterpret the feature to mean reset base
Dave Longley: we already have that feature with @context: null
Gregg Kellogg: ok
Manu Sporny: i think it's problematic that we don't remember how all this works and we need to all be on the same call to discuss
Gregg Kellogg: we could remove the test, change the toRDF algorithm, add instructions for how to handle output
Dave Longley: Within the same document, it might be useful to have relative IRIs, I don't think we can just drop it from the algorithm. [scribe assist by Manu Sporny]
Markus Lanthaler: You won't know the difference between the two relative IRIs. [scribe assist by Manu Sporny]
Dave Longley: The whole point is that what you would be digitally signing or asserting is that the bundle is the same - the bundle that you paid for is the same bundle that you're getting. [scribe assist by Manu Sporny]
PROPOSAL: Add an issue marker for the "@base": null feature in the JSON-LD API CR specification - it may be modified or removed.
Gregg Kellogg: +1
Manu Sporny: +1
Dave Longley: +1
Markus Lanthaler: +1
David I. Lehn: +0
Vikash Agrawal: +1
RESOLUTION: Add an issue marker for the "@base": null feature in the JSON-LD API CR specification - it may be modified or removed.
Gregg Kellogg: we need to resolve the test issue
Manu Sporny: what's wrong with gregg's solution?
Markus Lanthaler: we should be testing that there are relative IRIs and gregg's solution changes them so they are absolute IRIs
Gregg Kellogg: there is a turtle test like this where base is set to null that requires you to use a published base IRI that is part of the test case, so it's not unlike that, but the mechanism is different, so if that means changing the algorithm or massaging the test results
Markus Lanthaler: in turtle you would use the application-supplied base
Manu Sporny: what is your suggestion mlnt?
Markus Lanthaler: the simplest thing would be to extend N-Quads to also include relative IRIs
Manu Sporny: i think it's stupid that we can't put relative IRIs in there, it's up to the application to decide if the data is application or not, so i do think that N-Quads shoudl support that, but i think it will be a long drawn out discussion with the RDF group
Gregg Kellogg: it won't happen, there's no support for relative IRIs in RDF there
Markus Lanthaler: we could output a new different format for the test suite "generalized N-Quads"
Gregg Kellogg: this is about toRDF and this stuff won't be RDF
Gregg Kellogg: it's difficult to rationalize not having RDF
Markus Lanthaler: if that's the case, i think we have no choice but to discard those triples in the toRDF algorithm
Manu Sporny: so how do we ensure that implementations preserve those
Manu Sporny: in those cases where they need to
Gregg Kellogg: this stuff is tested in flattening and expansion
Dave Longley: dont' forget the graph comparison/hash/signing use case
Manu Sporny: a flag would be necessary here for preserving relative URLs
Gregg Kellogg: my turtle processor will generate relative IRIs if there is no base and a validate option is not provided
Gregg Kellogg: this allows me to ensure that i'm only outputting valid RDF but i can also output invalid RDF where apps need it
Gregg Kellogg: one alternative would be to turn this into an error case where a processor raises an exception
Gregg Kellogg: that would allow the behavior so a processor would generate it anyway
Gregg Kellogg: this would allow processors to continue if an error was detected
Markus Lanthaler: we use promises now that wouldn't allow a processor to continue if errors happens
Gregg Kellogg: we can't do that with promises?
Dave Longley: If we did the error test case, it would help, but it wouldn't test everything. You wouldn't be able to test what the output was. [scribe assist by Manu Sporny]
Markus Lanthaler: yes, but we removed that feature before we switched to promises
Gregg Kellogg: It would mean that it would eliminate that potential use case. [scribe assist by Manu Sporny]
Gregg Kellogg: We could add an at risk comment on the toRDF algorithm that could have us revisit relative IRIs after CR. [scribe assist by Manu Sporny]
Dave Longley: We could output the invalid RDF, but the way you would check it is to see if it's invalid, but matches something its supposed to match. [scribe assist by Manu Sporny]
Gregg Kellogg: I don't think a valid RDF processor could express relative IRIs. [scribe assist by Manu Sporny]
Gregg Kellogg: Some implementations might not support that. [scribe assist by Manu Sporny]
Dave Longley: What if we approach this the same way that we approach generalized RDF? By default, only allow absolute IRIs, but have a flag that allows relative IRIs. [scribe assist by Manu Sporny]
Dave Longley: By default it would work properly, but would still support this use case for specialized applications. [scribe assist by Manu Sporny]
Gregg Kellogg: That's reasonable. We need to do something like this in the test, we don't have flags to pass this stuff in the algorithms/API calls. [scribe assist by Manu Sporny]
Gregg Kellogg: We do need a test for generalized RDF... [scribe assist by Manu Sporny]
Gregg Kellogg: i think we also need to add a type that indicates it's an optional test
Gregg Kellogg: We do need a type for an optional test, or something else in the test manifest that we can use to say it's not required. [scribe assist by Manu Sporny]
Dave Longley: RDF processors might not be able to output valid NQuads. [scribe assist by Manu Sporny]
Gregg Kellogg: if you look at this from the way the turtle tests are done, this would be a negative evaluation tests and processors aren't required to pass those
Dave Longley: Are you saying "negative evaluation case" in the general case? Is the output supposed to be valid or invalid? [scribe assist by Manu Sporny]
Gregg Kellogg: The processor can be conforming, even if it doesn't pass negative tests. The current one is a negative test, it should not output the data using relative IRIs. [scribe assist by Manu Sporny]
Gregg Kellogg: You could fail that test and not generate an error and still be conforming. [scribe assist by Manu Sporny]
Dave Longley: I thought we'd also have a positive test with the option turned on. [scribe assist by Manu Sporny]
Gregg Kellogg: Yes, positive evaluation test would generate an error. [scribe assist by Manu Sporny]
Markus Lanthaler: If I run toRDF w/o this flag, do we drop those? [scribe assist by Manu Sporny]
Manu Sporny: Yes. [scribe assist by Manu Sporny]
Manu Sporny: Currently we don't have any developer asking for digitally signed app manifests with relative URLs. [scribe assist by Manu Sporny]
Dave Longley: we did have some devs that wanted relative URLs for widgets in JSON-LD, not sure if they needed RDF
Markus Lanthaler: So, can we just drop them and not specify a flag in the spec to include them.. implementations are free to do so but that is beyond the specified algorithm
Manu Sporny: so, we don't have to support the feature now in the spec (as Markus said) [scribe assist by Manu Sporny]
Manu Sporny: so the assumption here is that we will drop relative URLs in the toRDF algorithm and implementations can add a flag to keep relative URLs when converting to RDF
PROPOSAL: Drop relative IRI in the serialize to RDF algorithm. JSON-LD processors are free to include an option that preserves relative IRIs when serializing to RDF.
Gregg Kellogg: +1
Manu Sporny: +1
Vikash Agrawal: +1
Dave Longley: +1
Markus Lanthaler: +1
David I. Lehn: +0
RESOLUTION: Drop relative IRI in the serialize to RDF algorithm. JSON-LD processors are free to include an option that preserves relative IRIs when serializing to RDF.
Markus Lanthaler: just to be clear, we could drop an entire graph
Markus Lanthaler: for the record: we e.g. drop entire graphs if the graph name is a relative IRI
Manu Sporny: yes

Topic: Vikash update for GSoC

Vikash Agrawal: this week i added a bunch of things already, like fetching of data, manu commented the other day for using jsonld.js library for compaction, so this week i have the json-ld context for the linkedin data, [missed]
Vikash Agrawal: they may or may not get shown depending upon the user, they don't always encode valid xsd:dates, the date is broken into parts, i put a small piece of code on the mailin glist, i wnated to know if that's a good way of doing things
Manu Sporny: you're right that a valid xsd date needs a year, month, and a day
Manu Sporny: is there a type vikash can use for this?
Gregg Kellogg: i think there's a gmonth datatype
Manu Sporny: vikash are you asking us to pull and see if the code works?
Vikash Agrawal: yes, and look at the structure of the JSON
Manu Sporny: you will have to check to see if it's year month day and use xsd:date, and if it's just a month and a day you can use gMonthDay
Vikash Agrawal: user["birthDate"] = result.values[0].dateOfBirth.year+'-'+result.values[0].dateOfBirth.month+'-'+result.values[0].dateOfBirth.day;
Vikash Agrawal: i have year month date, i do the above to get the user's birthday
Vikash Agrawal: {... "birthDate" : "http://schema.org/birthDate", ...}
Vikash Agrawal: i can have a birthday that is different from the schema.org date, so i can have something like this in my context
Manu Sporny: yeah, so this is where it gets kind of complicated, schema.org doesn't use xsd:dates for the birth dates, they don't datatype it at all, there is no xsd datetime stuff associated with it
Vikash Agrawal: to simplify things you should probably do the same thing
Vikash Agrawal: ISO 8601
Gregg Kellogg: http://schema.org/Date
Gregg Kellogg: it does say that the value is of type Date, which the date is ISO8601
Vikash Agrawal: which is Date: 2013-08-19
Manu Sporny: except that they completely break that rule in a lot of their markup
Gregg Kellogg: that may be, and they will accept anything to figure out what it means
Vikash Agrawal: [missed] it's pretty weird to me, they have broken it into so much of chunks so writing something generic is hard
Vikash Agrawal: hard coding might be required as i've done for date
Manu Sporny: there may be times in the linkedin data where you have to do something that's a little easier for the JSON-LD processor to work with
Manu Sporny: so you'll have to do that at some points
Vikash Agrawal: so those were the essential issues i was facing, apart from that i was having similar problems with the email such as education, some may have [missed] of study, some will have start date, some end date, they are broken out into forms, hardcoding to convert to JSON-LD would be best
Manu Sporny: yes, in those instances you will have to do that
Manu Sporny: hopefully you won't have to do that for everything
Vikash Agrawal: that's why [missed] json one [missed] then i can use the compact to convert back to the final json that i receive resolving these issues
Manu Sporny: yes, sounds good
Vikash Agrawal: suppose i wanted to condense objects, i have multiple json objects again, what should be a good way to do that?
Vikash Agrawal: printing it and then using type id or something else?
Manu Sporny: are you asking how you should compact ... sorry, is the question you're asking: you have the linkedin data and you have their way of expressing dates and things like that are you asking how to transform that to send to the JSON-LD processor for compaction?
Vikash Agrawal: my question ... suppose you have hyperlinks, i have education, how should i make it into a JSON that i should use, and how should i express that in the @context
Manu Sporny: are you saying that linkedin has an education field
Manu Sporny: it's hard to figure this out without an example, so let's do this offline, we'll take it to the mailing list
Vikash Agrawal: i'll share a small gist right now on github, and if possible then we take it to the mailing list
Manu Sporny: anything else you need help with?
Manu Sporny: yes, i'm not sure how to map "educations" to schema.org
Vikash Agrawal: I will need to know to get everything happening has to be done
Markus Lanthaler: you could just create a declarative mapping
Gregg Kellogg: would indexing work? use values?
Gregg Kellogg: declare values to be a container index
Gregg Kellogg: would that allow you to pull those properties up, it may be something to play with?
Vikash Agrawal: https://gist.github.com/ivikash/6282468 [SampleJSON-LDOutput, after hard-coding]
Gregg Kellogg: certainly JSON-LD macros would help you turn this into something more reasonable
Gregg Kellogg: you could massage this to be what you want, to in schema.org representation
Manu Sporny: this linkedin data is not really compatible with JSON-LD, you could express it as JSON-LD, but it would be better to clean up the object quite a bit before mapping to schema.org
Vikash Agrawal: [missed] this is something that is just clearer and concise and acceptable
Manu Sporny: yes, the other gist is good
Gregg Kellogg: yes, that's fine
Manu Sporny: remember that you want to preserve as much of the school and location information, etc. as possible
Manu Sporny: the data is important to preserve
Gregg Kellogg: we do see other cases where we have this property that they use, for instance in microdata and microformats JSON, where there is "property" or "properties" and the values are contained in there, it's a shame we didn't handle that case yet and i think we will in the future
Manu Sporny: i think with this kind of data, the amount of hoop jumping in the @context will perhaps always be out of the scope of JSON-LD
Vikash Agrawal: github.com has a much more cleaner JSON [scribe assist by Vikash Agrawal]
Manu Sporny: yeah, we should do this in a way more compatible with JSON-LD than with linkedin
Manu Sporny: well, vikash, do the best that you can extracting the data out of the linkedin stuff and you may just have to hardcode a chunk of this stuff, or rather, a decent bit of this so you can get a JSON object to run the compaction code on
Manu Sporny: i think you have a pretty good idea of what you need to do now
Vikash Agrawal: These are the fields I'm retrieving: .fields(["id", "firstName", "lastName", "pictureUrl", "publicProfileUrl","location","headline","summary","specialties","positions","emailAddress","interests","publications","patents","languages","skills","certifications","educations","courses","volunteer","following","dateOfBirth","memberUrlResources","phoneNumbers","twitterAccounts","connections","network"])
Vikash Agrawal: this will be present in the linkedin, i just have a small thing to share
Vikash Agrawal: this is the data from linkedin, most of these things might not be available for my profile, i will be able to see in the output, depending upon the JSON i could implement those
Manu Sporny: you might want to get a couple of our linkedin profiles as tests
Vikash Agrawal: Sure
Manu Sporny: if you could print things out to the console and a couple of us could log in and give you our data as test data
Manu Sporny: we'll try and do that
Vikash Agrawal: ok
Vikash Agrawal: one more thing, could i get [missed] a link to the turtle processor that you were discussing
Vikash Agrawal: i just want to have to look at that for the understanding purpose [missed]
Gregg Kellogg: did you want a link to the processor?
Gregg Kellogg: if you want a processor you can look at my ruby one on github
Vikash Agrawal: Here's a link to the TURTLE spec - http://www.w3.org/TR/turtle/ [scribe assist by Manu Sporny]

Topic: Decision to go CR or PR

Vikash Agrawal: Thanks
Manu Sporny: i completely forgot about the waiting period between announcement and proposed call, which is for any stage after LC, so the way that this ends up happening is that we resolve to take it to CR here and then the RDF WG resolves to going to CR, they make the announcement and there's a 7 day lag for people to object if they want, if no objections, then there's a meeting with w3c and they decide if the doc should transition and there's a meeting and a call with editors, chairman, etc
Manu Sporny: the link in IRC is for all the things that go on in the discussion
Manu Sporny: then if director says its ok to transition we do that on a publish date
Manu Sporny: once we go to PR we have to do that all over again
Manu Sporny: we have two calls coming up where we'll have to do that
Manu Sporny: there's a question if we want to take a week and get all implementation reports together and skip CR and go directly to PR, which we can do
Manu Sporny: dan brickley said that doesn't give google a chance to respond
Manu Sporny: given google's track record, they never respond (never provide an implementation report), but i feel if we go ahead then people will complain that we rushed and didn't wait for major adopters of the technology
Manu Sporny: i think we could go to PR pretty easily, technologically speaking, but politically people could think we tried to railroad the standard through, etc.
Gregg Kellogg: a couple of thoughts, if we want to ensure we have time to do tests we need that, second i don't see why we need to rush through CR other than to get the process behind us, if this allows more space for more implementations that would be a good thing
Manu Sporny: there are some large organizations are waiting for an announcement but i don't know if that's a very strong reason
Markus Lanthaler: if we went straight to PR would that mean we'd need to resolve all at-risk now?
Dave Longley: I'm ok with waiting. I know we have other things we'd like to move on to, but we might cause more problems if we don't wait. We could shorten the period to the minimum amount. [scribe assist by Manu Sporny]
Manu Sporny: yes, we'd have to resolve all at-risk
Markus Lanthaler: the call is scheduled exactly when we wanted to leave CR
Manu Sporny: we wanted to exit september 19th
Markus Lanthaler: ok, you're right
Manu Sporny: it's one week here, one week there, i dont' think it makes too much of a difference
Manu Sporny: i think we can knock it down by a week
Markus Lanthaler: we've spent a lot of time between LC and now CR
Manu Sporny: what slows us down at this point is the W3C process (it's by design)
Manu Sporny: they want to make sure everyone understands we're getting ready to put this out as a REC
Manu Sporny: once we come out of CR we have to modify the docs based on feedback (1-2 weeks if lucky), resolve to go to PR, another 7 day waiting period, then 1-2 weeks waiting for a meeting, then a month of voting, then we have to clean up the docs (1-2 weeks) then we can finally publish as a REC after director approves
Manu Sporny: that's why i'm saying knocking 1-2 weeks off won't make too much of a difference in the long term
Markus Lanthaler: if we skip CR we knock off a month
Manu Sporny: google may be hostile to that approach
Gregg Kellogg: we don't need to antagonize them
Markus Lanthaler: i don't think google will submit a report anyway
Manu Sporny: i agree but they've used that as a stick to beat us over the head
Manu Sporny: they are big supporters of JSON-LD
Manu Sporny: so skipping CR wouldn't be a good idea
Manu Sporny: i'm fairly uneasy to go straight to PR, if google wasn't upset by it i think it would be the right thing to do
Manu Sporny: we all want to get this thing shipped because we think it's done, but other people haven't been involved and don't understand how quickly it's been moving
PROPOSAL: Request that the RDF WG transition the JSON-LD 1.0 and JSON-LD 1.0 API specs to CR, using the shortest possible CR period (don't skip CR)
Gregg Kellogg: +1
Dave Longley: +1
Manu Sporny: +1
Markus Lanthaler: +1
David I. Lehn: +1
Vikash Agrawal: +1
RESOLUTION: Request that the RDF WG transition the JSON-LD 1.0 and JSON-LD 1.0 API specs to CR, using the shortest possible CR period (don't skip CR)
Manu Sporny: We should specifically ask Google is they want more time for CR and extend the CR period based on feedback from them. They're the only implementers we're waiting on right now.