Triply Use Case 1: Support for GLTF and other 3D formats

[wouterbeek]

Triply Use Case 1: Support for GLTF and other 3D formats

Description

There is good support for 3D objects in BIM tools, and there is good support for geo data in GeoSPARQL triple stores. But there is not yet a good way in which I can combine the two. As a data expert, I am able to find files that contain 3D objects in standardized formats (e.g. GLTF), but I do not know how to load these into my GeoSPARQL triple store. When I look this up on the web, I end up in academic papers on how GLTF could be integrated with GeoSPARQL. There are multiple approaches, none of which is supported out-of-the-box by contemporary triple stores.

Actor

• Data Expert

Preconditions

• A 3D object in a file with a popular format (e.g. GLTF).
• A geospatial/GeoSPARQL dataset, in which I can meaningfully position my file.

Postconditions

• A 3D object within my geospatial/GeoSPARQL dataset.

Steps

1. I have obtained a file that contains a 3D object in a popular format (e.g. GLTF).
2. I can upload that file in a triple store, that may contain geospatial/GeoSPARQL data.
3. In a query, I can retrieve the 3D object, in combination with other geospatial/GeoSPARQL data (from my local store and/or from a remote endpoint).

Implementation suggestions

3D objects could be serialized in GeoSPARQL literals, similar to the current support for geo serializations in GeoJSON/GML/WKT literals (the name geo:hasModel could be improved):

<my-feature>
  geo:hasGeometry <my-geometry>;
  geo:hasModel <my-model>.
<my-geometry> geo:asWKT 'MultiPolygon(...)'^^geo:wktLiteral.
<my-model> geo:asGltf '...'^^geo:gltfLiteral.

3D object serializations could be either text-based or binary, depending on their standardized or de facto serialization form in other contexts. There are antecedents for encoding binary data in literals, e.g. xsd:base64Binary (link).

[situx]

Hi @wouterbeek and thanks for your use case!

We were thinking about supporting links to 3D models as resources for now.
You seem to propose uploading them to the triple store, which I think might slow down querying.
Do you mean e.g. serializing e.g. a PLY file and storing it as a String literal or do you mean to upload it somewhere accessible to the triple store but not inside the knowledge graph?

[ar-chad]

@wouterbeek it would be great, if you could also try to add function signatures to your use cases.

[FransKnibbe]

@situx: at Triply, we would like to be able to use 3D geometry serialisations as literals, in the same way as WKT, GML or GeoJSON literals. It would be good to have support for at least one 3D geometry serialisation that is a general format for 3D geometry. We have used glTF, which is a format that is also used in the 3D Tiles standard, so it is already something common in GIS. But PLY or OBJ could also be candidates. I guess formats that allow the most pure geometry encodings are best suited, to have an easy option to use such a serialisation purely for geometry (and exclude data that are needed for visualisation). I don't have a good insight in the various general 3D graphics formats, but OBJ seems well suited for compact geometry representation, and can also be used for point clouds.

[FransKnibbe]

This issue is related to issue #586, because adding support for a general 3D data format like glTF or OBJ would require a way to georeference those geometry serialiations.

[situx]

@FransKnibbe Interesting!
It would be great to have an issue about exactly the literal types you would like to see in GeoSPARQL and especially for formats like PLY if you would like them binary encoded or not.

Out of curiosity:
How big are the 3D models you would like to include?
We typically scan 3D models as PLY with several millions of points that can be a big as several hundreds of megabit up to 1 GB. I would consider that too much for a text literal.
Could you maybe name an average size for the literal contents you encounter in your work that I believe would be representative for a lot of use cases we are about to encounter in 3D geodata?

[FransKnibbe]

@situx of course it helps if the 3D models are as small as possible. I can imagine a geo:Feature having multiple representations as a 3D model, with different generalisation levels (LODs). We have used 3D models primarily for visualisation and selection (clicking on an 3D object shows more information about the object), less for spatial analysis. If the main purpose is visualisation, the trick is not to use more coordinate data than is possible to display. So in a smooth working system there will probably be multiple 3D models associated with a feature, each having a different size: If many 3D models are to be displayed at the same time, each individual 3D model should have a low spatial resolution.

To answer your question, in practice I am seeing file sizes ranging from 10 to 0.5 MB. But I wouldn't dare saying that those sizes are representative for a lot of use cases.

(I wonder if it makes sense to introduce a new class in GeoSPARQL, one for 3D representations of features, specifically targeted at visualisation and selection, and perhaps with less capabilities for geometric processing and analysis, but that is out of scope for describing a use case)

[situx]

I think introducing the classes that you propose that makes a lot of sense and I would add:

• Encoding low-resolution 3D models meant for the use case of visualization as literals
• Representing high-resolution 3D models as linked file resources

Which also brings me to the point of representing metadata of 3D models.
In an ideal case, the metadata of the 3D models should already give you a lot of information about whether you want to consider the 3D model in your query so that you may filter out potential candidates beforehand.
I have worked on that before specifically with regards to 3D meshes and we ended up with the following catalog implemented in the Gigamesh Software Framework:
https://gigamesh.eu/ont/

Whether or not that should or should not be part of GeoSPARQL can of course be discussed, but we should have a look at how to described 3D models uniformly as well.

[wouterbeek]

@situx @ar-chad @FransKnibbe Thank you for your detailed feedback! I have added an "Implementation details" section in which I give suggestions for property and datatype IRIs. I have also given suggestions about how 3D objects could be serialized.

I have not included the following two feedback items, because of reasons that I will specify now:

Firstly, I have not included the interesting idea by @situx of allowing literal content to also be specified in an external web resource. I do see the value of this suggestion, but believe that this should be standardized at the level of the RDF standard. My reasoning here is that literals in the RDF standard can also have arbitrary length (e.g. rdf:HTML or rdf:XMLLiteral). I am also not convinced whether the length of literals requires a distinct standardization approach: offering this for literals may introduce an imbalance with IRIs, which are also allowed to have arbitrary length.

Secondly, I have not included the interesting idea by @FransKnibbe and @situx of introducing different properties and/or classes for 3D models of varying fidelity. I believe that this feedback is more useful to process in the context of #593.

Let me know in case you have any more feedback that I can process.

[ar-chad]

@wouterbeek it is possible to store geometries in multiple serialisation formats in the way you describe it in the implementation details. It is only a matter of adding additional geo:as* triple, where * corresponds to the serialisation format.

In additional to that there are geof:as* functions allowing to convert between currently supported serialisation formats. Although, those conversion functions may not be supported by all triple stores at the moment. It is a matter of implementation within a particular triple store.