Manu Sporny: http://groups.drupal.org/node/249128

Manu Sporny: Lin has asked what JSON-LD features should be used in Drupal.

Markus Lanthaler: The main reason they were opened was because the algorithms haven't been updated, and we haven't discussed how compaction works.

Markus Lanthaler: I had some question about our support for property generators.

Manu Sporny: Perhaps we should discuss how compaction should work.

Markus Lanthaler: https://github.com/json-ld/json-ld.org/issues/142#issuecomment-7866816

Manu Sporny: It's not just for drupal, we have a similar mechanism in RDFa, and other companies have indicated their need for it too.

Niklas Lindström: I understand that the mutliple-property feature could be seen as "un-pure" from a theoretical perspective. You should rely on inference and other things, but those are all ideals.

Manu Sporny: I don't hear any objection to keeping the feature, we just need to work out details, specifically in compaction.

Gregg Kellogg: We may want to say right now that it's not addressed in compaction, it would allow us to close out this particular feature. [scribe assist by Manu Sporny]

Gregg Kellogg: We could say that if we /do not/ describe anything happening in compaction, then we may not do anything. [scribe assist by Manu Sporny]

Gregg Kellogg: The algorithm for compaction would be a bit different... more heavyweight. [scribe assist by Manu Sporny]

Gregg Kellogg: I don't think we want to wrap these things together, we don't want to message that just because we haven't wrapped some of the API details up, that we're not going to support the feature. [scribe assist by Manu Sporny]

Niklas Lindström: +1 for that

Manu Sporny: we do a proposal that says "support both property-generators and language-maps; close issues; create new issues for compaction implications".

Markus Lanthaler: you can't make use of these things without the API, so we need all parts.

Gregg Kellogg: I think we need to do the expansion... no controversy there, the issue comes in via compaction and in via framing... certainly, there is some open work to be done in the expansion algorithms. It's reasonable to link them to the existing issues. [scribe assist by Manu Sporny]

Gregg Kellogg: Our practice has been to keep issues open until the language has been updated to reflect what's done. It's a book-keeping exercise. [scribe assist by Manu Sporny]

Manu Sporny: we need to do something to tell them that there is a clear effort to support these features.

Manu Sporny: we could post on drupal that we're supporting it, the issue is resolved, but the work's not all done.

Manu Sporny: the reason we need to be more careful on how we use the issue tracker is because the the perception on outside groups.

Gregg Kellogg: +1

Manu Sporny: +1

Niklas Lindström: +1

Markus Lanthaler: +1

François Daoust: +1

Manu Sporny: Markus' point is that we've typically dealt with things all in one issue, why is this different.

Gregg Kellogg: I don't know where other people are wrt to this. The .compact() algorithm is getting to be almost unimplementably complex. [scribe assist by Manu Sporny]

Gregg Kellogg: It becomes almost unacceptable in an algorithm, when in-fact, most people typically generate JSON-LD through their own customized mechanisms. I do make use of compaction, but mostly as a way of doing fairly standard normalizations. [scribe assist by Manu Sporny]

Gregg Kellogg: I usually generate properties as arrays - so for example, dropping single value arrays. Making use of CURIEs, or other terms. I find compaction() useful for that stuff. I don't find .frame() useful at all. I usually do in-memory linking. [scribe assist by Manu Sporny]

Gregg Kellogg: It would be too much work to get the API to do what I wanted... noticed this in .frame(). If it doesn't work, it doesn't work and there is no insight into why. Very hard to debug frames. [scribe assist by Manu Sporny]

Gregg Kellogg: I think these things make it fairly useless, adding more complexity to .compact() is not something I'd like to do. [scribe assist by Manu Sporny]

Gregg Kellogg: Which term is selected for a specific property is one of those things. [scribe assist by Manu Sporny]

Gregg Kellogg: The ranking algorithm... the very act of splitting properties among multiple terms, is strange. [scribe assist by Manu Sporny]

Gregg Kellogg: if you have a property that has an equivalent IRI, the datatypes are split among those terms.... if you are trying to be entirely symmetric between compaction and expansion. It leads to unexpected results. [scribe assist by Manu Sporny]

Manu Sporny: At a high level, I agree on the complexity argument. That's one of the reasons in the beginning we said that when we talk about compaction, we leave a lot of the magic up the the implementation.

Manu Sporny: 2 questions: 1) do we expect compaction to be interoperable. 2) were do we draw the line on interoperability?

Niklas Lindström: I also agree on a high level, especially w.r.t. property generators.

Manu Sporny: would we be opposed to some implementations implementing it, and others not.

Gregg Kellogg: In RDFa, we have examples of optional features - implementing the feature is not required, but if it is required, it has interoperability requirements. Processor graphs is one such example. [scribe assist by Manu Sporny]

Gregg Kellogg: Maybe we have a flag that allows best-effort compaction... maybe we can have a set of parameters that are associated with these optional features. Processors can declare whether they are supported or not. They're supported if the appropriate options are passed to the algorithm. It also makes testing easier. It can also reduce the computational complexity to support odd cases that are more likely to never come about. [scribe assist by Manu Sporny]

Gregg Kellogg: In that case, maybe term selection is not optional, but it is a feature that is controlled through an API flag. [scribe assist by Manu Sporny]

Niklas Lindström: I think flags are good for optional features. It should be possible to query the implementation via a .hasFeature() method. [scribe assist by Manu Sporny]

Niklas Lindström: flags are good for optional features. We should have a "has feature" method, so you can take appropriate measure.

Manu Sporny: +1 to niklasl

Manu Sporny: I agree with the general direction of the discussion. We're discussing two things:

Markus Lanthaler: I'm wondering if, in this case, it might not be too difficult to support.

Manu Sporny: this would modify the feature because of complexity, and give poor signals to developers.

Markus Lanthaler: we currently silently ignore all other definitions for a prefix if it's also a term.

Niklas Lindström: we could actually say that things define with an expanded term definition are never used as prefixes.

Markus Lanthaler: then I would need to keep track what's a prefix or what's not.

Gregg Kellogg: We might say that if it uses an expanded definition, it only uses the @id key - if that, then it can be used as a prefix. [scribe assist by Manu Sporny]

Gregg Kellogg: A term can be used as a term if it matches. [scribe assist by Manu Sporny]

Manu Sporny: If we require a : at the end of a prefix this creates a micro-syntax.

Niklas Lindström: .. "foaf": {"@id": "…", "@prefix": true}

Gregg Kellogg: What are the issues where this happens in common practice? The only time this becomes an issue is in something like an OWL definition. [scribe assist by Manu Sporny]

Gregg Kellogg: There are very few cases here where this happens. [scribe assist by Manu Sporny]

Niklas Lindström: there were some issue when I did some RDFa/JSON-LD conversion; it's not clear from the context if I can use them for prefixes.

Gregg Kellogg: Maybe converting from RDFa to JSON-LD... [scribe assist by Manu Sporny]

Manu Sporny: the issue was how we would support property generator in compaction.

Manu Sporny: do we think that there may be optional features in compaction?

Markus Lanthaler: In General, I don't like this, but there may be some cases where it's useful.

Gregg Kellogg: There is a computational concern, if these features are enabled by default, it could really slow down your workflow. [scribe assist by Manu Sporny]

Gregg Kellogg: I have to do a combinatorial search over those properties for things like property generators... that is a computational burden. [scribe assist by Manu Sporny]

Gregg Kellogg: That's for somebody that doesn't even want property generator compaction. [scribe assist by Manu Sporny]

Gregg Kellogg: Perhaps these are features where the defaults can be defined by the presence of the feature. [scribe assist by Manu Sporny]

Manu Sporny: the more I think about it, the more I'm concerned about the memory/computational overhead of compacting property generators.

Gregg Kellogg: As a consumer of data, I don't specify a context... I use what is served to me. [scribe assist by Manu Sporny]

Markus Lanthaler: Then why are we concerned about compaction if people might not use it? [scribe assist by Manu Sporny]

Gregg Kellogg: Because people do use .compact() and we don't want the computational complexity in the default case. [scribe assist by Manu Sporny]

Niklas Lindström: I was thinking that if drupal started using a lot of properties for a given term, it could be that the consumers will react to that, and think that it's just noise. I wouldn't expect more than 3-5.

Manu Sporny: we're going to need to do some "least-resistance" proposal to make some progress.

Manu Sporny: Proposals on the table: 1) Support optional features in .compact(), 2) Make all features required in .compact()

Markus Lanthaler: we already support optional features - optimize

Manu Sporny: if we support optional features, we're likely to support property generation is high. If all are required, property generation support is low.

Manu Sporny: +1 to 1), -1 to 2)

Gregg Kellogg: +1 for 1), -1 to 2)

François Daoust: +1 to 1), -1 to 2)

Markus Lanthaler: -1 to 1), +1 to 2) (except optimizations)

Niklas Lindström: +1 to 1), -0.5 to 2)

Dave Longley: following along in IRC ... keep in mind that those that want to use the feature (eg: drupal) might shy away from JSON-LD anyway if they think it's "optional" (the compaction side of it)

Dave Longley: it needs to be sold that a processor that they can use will implement it ... otherwise what good is it?

Dave Longley: that said, i'm ok with the optional path, just pointing out (if it hasn't been) a possible issue.

Gregg Kellogg: +1

Manu Sporny: +1

Markus Lanthaler: -1

François Daoust: +1

Niklas Lindström: +1

Dave Longley: maybe a better solution would be to mark which term you want to use in the compaction, in the @context, so the compaction algorithm can just look there and pick it.

Dave Longley: that's really what i think people would want anyway ... instead of having to think through the algorithm in their minds

Dave Longley: and guess which term will be selected based on whichever combination is most efficient.

Dave Longley: {"@context": {"foo": {"compact: true", ... (other stuff here) ... }}

Dave Longley: that would make compaction super simple ... and it would do what you want.

Manu Sporny: you have a document with a property generator already in compact form.

Manu Sporny: Proposals for property generators and .compact(): 1) Support property generator compaction as an optional feature, 2) Do not support compaction of property generators at all, 3) Support property generator compaction as a required feature, 4) Compact only the first value in the enumeration of the array

Gregg Kellogg: +1 to 1), +0 to 2), -1 to 3), -1 to 4)

Dave Longley: {"@context": {"foo": {"compact: true", ... (other stuff here) ... }}

Dave Longley: the other option is to be explicit in the context for what to compact.

Dave Longley: "foo": {"@id": "whatever", ... property generator ..., "compact": true}

Dave Longley: the problem has always been that you need to understand the algorithm. This allows you to be more clear and not have deep understanding.

Manu Sporny: my concern is that it's pretty complex, in that it doesn't hide any of the complexity from the developer.

Markus Lanthaler: if you want to be specific, then you only need to specify one.

Dave Longley: "foo": {"@id": [any match in here], "compact": true}

Dave Longley: "foo": {"@id": [any match in here], "compact": 1}

Dave Longley: "foo": {"@id": [any match in here], "compact": 5}

Dave Longley: one of the issues is knowing what term to pick.

Manu Sporny: I thought the way the algorithm was supposed to work is that every id must exist with the same value. Then and only then do you use foo.

Manu Sporny: I think dave's option is interesting, but it is tangentially related to property generator compaction.

Dave Longley: do they really care about this?

Dave Longley: if the intention is to generate multiple properties when you see a single key, I don't see there's an issue.

Manu Sporny: it does mean that it doesn't cleanly round-trip.

Manu Sporny: +1 to 1), -1 to 2), -1 to 3), -1 to 4), +0 to 5)

Markus Lanthaler: +0.5 to 1), -1 to 2) +1 to 3), -1 to 4)

François Daoust: +1 to 1), -1 to 2), -1 to 3), -1 to 4)

François Daoust: [also like 5)]

Manu Sporny: this could send mixed messages to the drupal community.

Niklas Lindström: +1 to 1), -0.5 to 2), -1 to 3), -1 to 4), +0 to 5)

Dave Longley: it's more like we're fighting for their feature. We should ask them how they feel about 5)

Markus Lanthaler: don't really understand 5) to be honest... the way I would implement this is to rank a term for each of it @id's in compaction and leave all the rest the same

Manu Sporny: this takes us into entailment, and we should tread carefully.

Dave Longley: +1 to 1) -1 to 2) +0.5 to 3) +0 to 4) +1 to 5)

Manu Sporny: 1) seems to have the strongest support.

Dave Longley: I'm worried that people may feel this is license to not implement.

Manu Sporny: once people see an implementation (PHP), they'll be fine with it.

Markus Lanthaler: I can try an implementation of 1) to see how easy it is

Dave Longley: the only way for the algorithm to work is if you check both properties and values (for 1)

Markus Lanthaler: "foo": [ "a", "b" ] - foo: data - expand - a: data, b: data - compact - foo: data, data ??? Remove duplicates, round-trip OK?

Dave Longley: you could end up choosing the wrong term, and create more data.

Markus Lanthaler: we already have that problem. if you have two properties that map to the same IRI, they might not compact back the same.

Manu Sporny: maybe a callback could help.

Manu Sporny: for the drupal case, they definitely want to generate multiple properties on expansion.

Manu Sporny: we need to reach out to the drupal community about this.