Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add detections-vehicles dataset to backend #83

Merged
merged 7 commits into from
Jan 13, 2021

Conversation

leothomas
Copy link
Contributor

@leothomas leothomas commented Nov 17, 2020

What I did:

Added detections-vehicles dataset - addresses covid-dashboard issue #440

How I did it:

Extracted scene_ids for each feature in the two geojsons currently in S3 and built + uploaded a detections_scenes.csv file.

Created a new dataset metadata file called detections-vehicles.json which is very similar to detection-plane.json and detections-ship.json.

Added detections-vehicle to all datasets which has detections-ship and detection-plane as exclusive datasets

How you can test it:

Run docker-compose up --build

  • Hit http://localhost:8000/v1/datasets notice a new dataset for detections-vehicles is present (with {spotlightId} in url, since no spotlight has been specified)
  • Hit http://localhost:8000/v1/datasets/la notice that the detections-vehicles is still present
  • Hist http://localhost:8000/v1/datasets/ny notices that the detections-vehicles is no longer present

@olafveerman @drewbo would love some feedback on the detections-vehicles.json metadata files (I copy-pasted the description from detections-ships.json, not sure if the info field is missing anything)

@danielfdsilva tagging you because the format of the geojsons in s3 seems different from the other detection-* datasets. Let me know if you need anything in order to implement in the front end

@lillythomas fyi

@leothomas leothomas added enhancement New feature or request data labels Nov 17, 2020
@leothomas leothomas self-assigned this Nov 17, 2020
@olafveerman
Copy link
Contributor

Thanks @leothomas

  • detections-ships.json seems to check out. My only comment is that the interface shows detections in orange (not red), I can update the description for all ML detections
  • had a quick look at the GeoJSON, and didn't notice any structural difference that would trip up the frontend

Running this with 1.8.0 of the dashboard, I noticed a couple of issues. Unsure whether they should be solved on the API, or frontend, but raising them here:

  • background imagery is not available
  • there are vehicle detections for two dates. The domain in the API response is: "domain": [ "2020-01-09T00:00:00Z", "2020-07-08T00:00:00Z" ]. Instead of treating these as two discrete dates, the frontend interprets this as a range daily dates with the domain being the start and end date

danielfdsilva added a commit to NASA-IMPACT/covid-dashboard that referenced this pull request Nov 18, 2020
@danielfdsilva
Copy link
Contributor

Tested this locally and the data seems to show up correctly.
I encountered some errors, but I think it's because I don't have a planet key.

I did notice the domain error reported by @olafveerman, and this is a problem with the frontend. We were considering domains with 2 dates to be a range according to the time unit. This has been the case and worked until this moment. The very useful isPeriodic property of the api response was not being used.

Long story short, here's the PR with the fix: NASA-IMPACT/covid-dashboard#455

@leothomas
Copy link
Contributor Author

Fixed the issue with detections-vehicles missing background scenes (issue was due to the fact that detections-vehicles relies on Skysat imagery, which is not available through the Planet tile endpoint (detections-plane and detections-ship rely on Planet imagery which is available through the Planet tile endpoint).

@olafveerman re: observations colour - I'm not seeing anywhere on the backend to specify the colours of the observations. If that's not fixable through the front end, I'm happy to update the info field of each of the detections-* metadata files to specify that the observations are show in orange (instead of red)

@olafveerman
Copy link
Contributor

This looks good! Holding off on merging for a couple of days, until the Story feature has been launched and stable.

@leothomas
Copy link
Contributor Author

@olafveerman This PR has been open for a while. Can we merge, now that the Story feature has been launched?

@olafveerman olafveerman merged commit 3c2d9f7 into develop Jan 13, 2021
@olafveerman olafveerman deleted the feature/add-detections-vehicle-dataset branch January 13, 2021 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants