Collaboration for terrain data.

[JonathanTurnock]

Hello 👋

I've kicked off a new repo to house DCS terrain data in the GeoJSON specification format and I was wandering if you may be interested in a bit of collaboration to set this up into a shared ETL pipeline.

There's a couple of other projects involved here also so it's an exercise in solving the common problem for everyone.

Any help or contributions from you fine experts would be most welcome and I just wanted to thankyou from the bottom of my heart for everything you do with dcs.

https://github.com/JonathanTurnock/dcs-global-terrain-database

Have a great day.

[DanAlbert]

Sorry for the slow response. I wanted to have a chance to look at the linked project before commenting, and it took me a while to find the time :)

I'm not very familiar with geojson. aiui we'd be able to share lua export from DCS, but we would not be able to use the geojson data in pydcs because the X/Y game coordinates are more important than lat/lon as a programming interface. We could always convert to/from, but since we need X/Y more often than lat/lon it'd be quite expensive for the source of truth to be lat/lon.

#247 probably covers what you're asking? The idea is that we'd have a set of export scripts that dumps as much of the raw lua data as possible, keep that in a common git repo, then use it to generate the content for whatever projects would need it (for pydcs that's python classes and data, for you it'd be the geojson data, etc).

What data are you planning to cover? One thing we sorely lack is elevation data for arbitrary locations at a resolution that's suitable for JDAM planning (that is, not more than a dozen or so meters between data points). If that's something you plan to include, and in a format that's small enough on disk (this is why we haven't just done a sampling export of each map; they'd be huge) Liberation would definitely use it :)

[JonathanTurnock]

This project is largely going to be used as a source for the upcoming Web Mission Editor and also a couple of other projects so it effectively will encompass everything, this repo is specifically centred around terrain data.

Height mapping is a hot topic indeed and something we are looking at right now.

We have started to assemble export scripts for some of the initial items, they run in dcs-fiddle which responds in JSON so thats why this format is really good for sharing in a common way.

Would you ever see a future where the data from pydcs comes from a CDN such as the github raw url of https://github.com/JonathanTurnock/dcs-global-terrain-database/blob/main/terrains/caucasus.json as opposed to each project having its own exporters and data sets.

My view on this is that there is only a finite amount of data in DCS but the number of times its exported must be ten fold haha.

We are still keeping x,y data simply as you say to not have to re-compute it and potentially end up slightly off the original export, here is an example of an airbase

{
  "geometry": {
    "coordinates": [38.925202300775, 45.087429883845, 30.01003074646],
    "type": "Point"
  },
  "properties": {
    "category": "AIRDROME",
    "id": "Krasnodar-Center",
    "name": "Krasnodar-Center",
    "point": {
      "x": 11652.913085938,
      "y": 30.01003074646,
      "z": 366766.3125
    },
    "type": "AIRBASE"
  },
  "type": "Feature"
}

Anyway, if anything you have seen looks useful then please feel free to reach out, I'm sure you have lots of awesome exporting experience and would be a great help.

[DanAlbert]

Would you ever see a future where the data from pydcs comes from a CDN such as the github raw url of https://github.com/JonathanTurnock/dcs-global-terrain-database/blob/main/terrains/caucasus.json as opposed to each project having its own exporters and data sets.

I guess I don't understand how that's different than what I already said here:

#247 probably covers what you're asking? The idea is that we'd have a set of export scripts that dumps as much of the raw lua data as possible, keep that in a common git repo, then use it to generate the content for whatever projects would need it (for pydcs that's python classes and data, for you it'd be the geojson data, etc).

Could you explain how what you're suggesting is different?

If you mean live CDN use in applications (e.g. Liberation fetches info directly from dcs-global-terrain-database), absolutely not :) It's an additional point of failure, source of data instability (consistently imperfect data is annoying, but data changing mid game is catastrophic), and I'm not sure there's much benefit.

If you mean pointing at that whenever we run the pydcs generators, sure, that's basically the idea in #247.

We are still keeping x,y data simply as you say to not have to re-compute it and potentially end up slightly off the original export, here is an example of an airbase

In that case, yeah. That's probably quite usable for us. The terrain export is definitely the worst part of maintaining pydcs.