Skip to content

Commit

Permalink
still preparing for CRAN submission
Browse files Browse the repository at this point in the history
  • Loading branch information
ptaconet committed Jul 10, 2024
1 parent 1a127a2 commit 7e11dae
Show file tree
Hide file tree
Showing 8 changed files with 19 additions and 39 deletions.
1 change: 0 additions & 1 deletion .data_collections.csv
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,6 @@ MOD17A2H.061,MODIS,MODIS/Aqua Gross Primary Productivity 8-Day L4 Global 500 m S
MYD17A2H.061,MODIS,MODIS/Terra Gross Primary Productivity 8-Day L4 Global 500 m SIN Grid,Primary Productivity,NASA USGS LP DAAC,https://lpdaac.usgs.gov/products/myd17a2hv061,https://dx.doi.org/10.5067/MODIS/MYD17A2H.061,6.1,500,8,day,Global,2002-07-04,ongoing,12,https://search.earthdata.nasa.gov/search?q=MYD17A2H.061&ok=MYD17A2H.061,Implemented,earthdata,https://opendap.cr.usgs.gov/opendap/hyrax/,https://opendap.cr.usgs.gov/opendap/hyrax/MYD17A2H.061/contents.html,https://opendap.cr.usgs.gov/opendap/hyrax/MYD17A2H.061/h00v08.ncml,XDim,YDim,time,MOD_Grid_MOD17A2H_eos_cf_projection,+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
MOD17A2HGF.061,MODIS,MODIS/Terra Gross Primary Productivity Gap-Filled 8-Day L4 Global 500 m SIN Grid,Primary Productivity,NASA USGS LP DAAC,https://lpdaac.usgs.gov/products/mod17a2hgfv061,https://dx.doi.org/10.5067/MODIS/MOD17A2HGF.061,6.1,500,8,day,Global,2000-01-01,ongoing,49,https://search.earthdata.nasa.gov/search?q=MOD17A2HGF.061&ok=MOD17A2HGF.061,Implemented,earthdata,https://opendap.cr.usgs.gov/opendap/hyrax/,https://opendap.cr.usgs.gov/opendap/hyrax/MOD17A2HGF.061/contents.html,https://opendap.cr.usgs.gov/opendap/hyrax/MOD17A2HGF.061/h00v08.ncml,XDim,YDim,time,MOD_Grid_MOD17A2H_eos_cf_projection,+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
MYD17A2HGF.061,MODIS,MODIS/Aqua Gross Primary Productivity Gap-Filled 8-Day L4 Global 500 m SIN Grid,Primary Productivity,NASA USGS LP DAAC,https://lpdaac.usgs.gov/products/myd17a2hgfv061,https://dx.doi.org/10.5067/MODIS/MYD17A2HGF.061,6.1,500,8,day,Global,2002-01-01,ongoing,49,https://search.earthdata.nasa.gov/search?q=MYD17A2HGF.061&ok=MYD17A2HGF.061,Implemented,earthdata,https://opendap.cr.usgs.gov/opendap/hyrax/,https://opendap.cr.usgs.gov/opendap/hyrax/MYD17A2HGF.061/contents.html,https://opendap.cr.usgs.gov/opendap/hyrax/MYD17A2HGF.061/h00v08.ncml,XDim,YDim,time,MOD_Grid_MOD17A2H_eos_cf_projection,+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
MOD17A3.055,MODIS,MODIS/Terra Net Primary Production Yearly L4 Global 1 km SIN Grid,Primary Productivity,NASA USGS LP DAAC,,,5.5,500,365,day,Global,2000-01-01,ongoing,2235,https://search.earthdata.nasa.gov/search?q=MOD17A3.055&ok=MOD17A3.055,Implemented,earthdata,https://opendap.cr.usgs.gov/opendap/hyrax/,https://opendap.cr.usgs.gov/opendap/hyrax/MOD17A3.055/contents.html,https://opendap.cr.usgs.gov/opendap/hyrax/MOD17A3.055/h00v08.ncml,XDim,YDim,time,MOD_Grid_MOD17A3_eos_cf_projection,+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
MOD17A3HGF.061,MODIS,MODIS/Terra Net Primary Production Gap-Filled Yearly L4 Global 500 m SIN Grid,Primary Productivity,NASA USGS LP DAAC,https://lpdaac.usgs.gov/products/mod17a3hgfv061,https://dx.doi.org/10.5067/MODIS/MOD17A3HGF.061,6.1,500,365,day,Global,2000-02-18,ongoing,409,https://search.earthdata.nasa.gov/search?q=MOD17A3HGF.061&ok=MOD17A3HGF.061,Implemented,earthdata,https://opendap.cr.usgs.gov/opendap/hyrax/,https://opendap.cr.usgs.gov/opendap/hyrax/MOD17A3HGF.061/contents.html,https://opendap.cr.usgs.gov/opendap/hyrax/MOD17A3HGF.061/h00v08.ncml,XDim,YDim,time,MOD_Grid_MOD17A3H_eos_cf_projection,+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
MYD17A3HGF.061,MODIS,MODIS/Aqua Net Primary Production Gap-Filled Yearly L4 Global 500 m SIN Grid,Primary Productivity,NASA USGS LP DAAC,https://lpdaac.usgs.gov/products/myd17a3hgfv061,https://dx.doi.org/10.5067/MODIS/MYD17A3HGF.061,6.1,500,365,day,Global,2002-07-04,ongoing,409,https://search.earthdata.nasa.gov/search?q=MYD17A3HGF.061&ok=MYD17A3HGF.061,Implemented,earthdata,https://opendap.cr.usgs.gov/opendap/hyrax/,https://opendap.cr.usgs.gov/opendap/hyrax/MYD17A3HGF.061/contents.html,https://opendap.cr.usgs.gov/opendap/hyrax/MYD17A3HGF.061/h00v08.ncml,XDim,YDim,time,MOD_Grid_MOD17A3H_eos_cf_projection,+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
MODOCGA.061,MODIS,MODIS/Terra Ocean Reflectance Daily L2G-Lite Global 1 km SIN Grid ,Ocean Reflectance,NASA USGS LP DAAC,https://lpdaac.usgs.gov/products/modocgav061,https://dx.doi.org/10.5067/MODIS/MODOCGA.061,6.1,1000,1,day,Global,2000-02-24,ongoing,3,https://search.earthdata.nasa.gov/search?q=MODOCGA.061&ok=MODOCGA.061,Implemented,earthdata,https://opendap.cr.usgs.gov/opendap/hyrax/,https://opendap.cr.usgs.gov/opendap/hyrax/MODOCGA.061/contents.html,https://opendap.cr.usgs.gov/opendap/hyrax/MODOCGA.061/h00v08.ncml,XDim,YDim,time,MODIS_Grid_2D_eos_cf_projection,+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
Expand Down
2 changes: 1 addition & 1 deletion R/mf_get_opt_param.R
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
#' \describe{
#' \item{roiSpatialIndexBound}{OPeNDAP indices for the spatial coordinates of the bounding box of the ROI (minLat, maxLat, minLon, maxLon)}
#' \item{availableVariables}{Variables available for the collection of interest}
#' \item{roiSpatialBound }{The spatial coordinates of the bounding box of the ROI expressed in the CRS of the collection}
#' \item{roiSpatialBound}{The spatial coordinates of the bounding box of the ROI expressed in the CRS of the collection}
#' \item{OpenDAPXVector}{The X (longitude) vector}
#' \item{OpenDAPYVector}{The Y (longitude) vector}
#' \item{OpenDAPtimeVector}{The time vector, or NULL if the collection does not have a time vector}
Expand Down
6 changes: 3 additions & 3 deletions R/mf_get_url.R
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#' @name mf_get_url
#' @aliases mf_get_url
#' @title Build the URL(s) of the data to download
#' @description This is the main function of the package. It enables to build the OPeNDAP URL(s) of the spatiotemporal datacube to download, given a collection, variables, region and time range of interest.
#' @description Builds the OPeNDAP URL(s) of the spatiotemporal datacube to download, given a collection, variables, region and time range of interest.
#'
#' @param collection string. mandatory. Collection of interest (see details of \link{mf_get_url}).
#' @param variables string vector. optional. Variables to retrieve for the collection of interest. If not specified (default) all available variables will be extracted (see details of \link{mf_get_url}).
Expand All @@ -13,13 +13,13 @@
#' @param credentials vector string of length 2 with username and password. optional if the function \link{mf_login} was previously executed.
#' @param verbose boolean. optional. Verbose (default TRUE)
#'
#' @return a data.frame with one row for each dataset to download and 4 columns :
#' @return a data.frame with one row for each dataset to download and 5 columns :
#' \describe{
#' \item{id_roi}{Identifier of the ROI}
#' \item{time_start}{Start Date/time for the dataset}
#' \item{collection}{Name of the collection}
#' \item{name}{Indicative name for the dataset}
#' \item{url}{https URL (OPeNDAP) of the dataset}
#' \item{url}{https OPeNDAP URL of the dataset}
#' }
#'
#' @details
Expand Down
Binary file modified R/sysdata.rda
Binary file not shown.
10 changes: 5 additions & 5 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,15 @@ renvoyer erreur ou warning si le fichier n'existe pas
<!-- `r emo::ji("warning")` Package still under development ! -->

<!--
R package to access various spatiotemporal Earth science data collections in R using the [OPeNDAP framework](https://www.opendap.org/about). Currently implemented data collections are [MODIS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/), [VIIRS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/), [GPM](https://pmm.nasa.gov/GPM) and [SMAP](https://smap.jpl.nasa.gov/)).
R package to access various spatiotemporal Earth science data collections in R using the [OPeNDAP framework](https://www.opendap.org/about/). Currently implemented data collections are [MODIS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/), [VIIRS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/), [GPM](https://pmm.nasa.gov/GPM) and [SMAP](https://smap.jpl.nasa.gov/)).
Opendap (*Open-source Project for a Network Data Access Protocol*) is a data access protocol that enables to subset the data - spatially, temporally, etc. - directly at the downloading phase. Filters are provided directly within a http url. For example the following URL :
https://opendap.cr.usgs.gov/opendap/hyrax/MOD11A1.006/h17v08.ncml.nc4?MODIS_Grid_Daily_1km_LST_eos_cf_projection,LST_Day_1km[6093:6122][55:140][512:560],LST_Night_1km[6093:6122][55:140][512:560],QC_Day[6093:6122][55:140][512:560],QC_Night[6093:6122][55:140][512:560],time[6093:6122],YDim[55:140],XDim[512:560]
provides the MOD11A1.006 (MODIS/Terra Land Surface Temperature/Emissivity Daily L3 Global 1km SIN Grid V006) data in netCDF, subsetted for bands LST_Day_1km, LST_Night_1km, QC_Day, QC_Night, for each day between the 2017-01-01 and the 2017-01-30, and within the following bounding box (lon/lat): -5.41 8.84, -5.82 9.54.
This package enables to build OPeNDAP (https) URLs given input parameters such as a data collection, region and time range of interst . These URLs can then be used to either download the data to your workspace or computer, or access the datacube directly as an R object (of class `ndcf4`, `raster`, `stars`, etc.)
-->
**`modisfast`** (formerly `opendapr`) is an R package that provides functions to **speed-up** the **download** of time-series raster data products derived from some [**MODIS**](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/) and [**VIIRS**](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/) observations, as well as other widely-used satellite-derived environmental data (e.g. Global Precipitation Measurement Mission).

**`modisfast`** uses the abilities offered by the [OPeNDAP framework](https://www.opendap.org/about) (*Open-source Project for a Network Data Access Protocol*) to download a subset of Earth science data cube, along spatial, temporal or any other data dimension (depth, ...). This way, it reduces downloading time and disk usage to their minimum : no more 1° x 1° MODIS tiles with 10 bands when your region of interest is only 30 km x 30 km wide and you need 2 bands ! Moreover, `modisfast` enables parallel downloads of data.
**`modisfast`** uses the abilities offered by the [OPeNDAP framework](https://www.opendap.org/about/) (*Open-source Project for a Network Data Access Protocol*) to download a subset of Earth science data cube, along spatial, temporal or any other data dimension (depth, ...). This way, it reduces downloading time and disk usage to their minimum : no more 1° x 1° MODIS tiles with 10 bands when your region of interest is only 30 km x 30 km wide and you need 2 bands ! Moreover, `modisfast` enables parallel downloads of data.

`modisfast` is hence particularly suited for retrieving MODIS or VIIRS data **over long time series** and **over areas**, rather than short time series and points.

Expand Down Expand Up @@ -146,7 +146,7 @@ modisfast provides an entry point to some specific OPeNDAP servers (e.g. MODIS,
Apart from these performance aspects, ethical considerations have driven the development of this package :
* **Facilitating the access to Earth science data for R users in places where internet connection is slow or expensive** : Earth science products are generally huge files that can be quite difficult to download in places with slow internet connection, even more if large time series are needed. By enabling to download strictly the data that is needed, the products become more accessible in those places;
* **Caring about the environmental digital impact of our research work** : Downloading data has an impact on environment and to some extent contributes to climate change. By downloading only the data that is need (rather than e.g a whole MODIS tile, or a global SMAP or GPM dataset) we somehow promote digital sobriety.
* **Supporting the open-source-software movement** : The OPeNDAP is developed and advanced openly and collaboratively, by the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about) This open, powerfull and standard data access protocol is more and more used, by major Earth science data providers (e.g. NASA or NOAA). Using OPeNDAP means supporting methods for data access protocols that are open, build collaboratively and shared.
* **Supporting the open-source-software movement** : The OPeNDAP is developed and advanced openly and collaboratively, by the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about/) This open, powerfull and standard data access protocol is more and more used, by major Earth science data providers (e.g. NASA or NOAA). Using OPeNDAP means supporting methods for data access protocols that are open, build collaboratively and shared.
-->

<!--
Expand Down Expand Up @@ -205,7 +205,7 @@ mf_list_collections() %>%

There are other R packages available for accessing MODIS data, which may be more suitable if your requirements differ. These include :

* [`MODIS`](https://github.com/MatMatt/MODIS)
* [`MODIS`](https://github.com/fdetsch/MODIS)
* [`MODIStsp`](https://github.com/ropensci/MODIStsp)
* [`MODIStools`](https://github.com/ropensci/MODIStools)
* [`appeears`](https://github.com/bluegreen-labs/appeears)
Expand Down Expand Up @@ -254,4 +254,4 @@ The initial development and first release of this package were financed by the M

By enabling to download subsets of data cubes, `modisfast` facilites the access to Earth science data for R users in places where internet connection is slow or expensive and promotes digital sobriety for our research work.

The OPeNDAP, over which the package builds, is a project developed by the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about) and advanced openly and collaboratively. By using this data access protocol, `modisfast` support the open-source-software movement.
The OPeNDAP, over which the package builds, is a project developed by the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about/) and advanced openly and collaboratively. By using this data access protocol, `modisfast` support the open-source-software movement.
31 changes: 6 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ renvoyer erreur ou warning si le fichier n'existe pas
<!-- ⚠️ Package still under development ! -->

<!--
R package to access various spatiotemporal Earth science data collections in R using the [OPeNDAP framework](https://www.opendap.org/about). Currently implemented data collections are [MODIS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/), [VIIRS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/), [GPM](https://pmm.nasa.gov/GPM) and [SMAP](https://smap.jpl.nasa.gov/)).
R package to access various spatiotemporal Earth science data collections in R using the [OPeNDAP framework](https://www.opendap.org/about/). Currently implemented data collections are [MODIS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/modis-overview/), [VIIRS](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/s-npp-nasa-viirs-overview/), [GPM](https://pmm.nasa.gov/GPM) and [SMAP](https://smap.jpl.nasa.gov/)).
Opendap (*Open-source Project for a Network Data Access Protocol*) is a data access protocol that enables to subset the data - spatially, temporally, etc. - directly at the downloading phase. Filters are provided directly within a http url. For example the following URL :
https://opendap.cr.usgs.gov/opendap/hyrax/MOD11A1.006/h17v08.ncml.nc4?MODIS_Grid_Daily_1km_LST_eos_cf_projection,LST_Day_1km[6093:6122][55:140][512:560],LST_Night_1km[6093:6122][55:140][512:560],QC_Day[6093:6122][55:140][512:560],QC_Night[6093:6122][55:140][512:560],time[6093:6122],YDim[55:140],XDim[512:560]
provides the MOD11A1.006 (MODIS/Terra Land Surface Temperature/Emissivity Daily L3 Global 1km SIN Grid V006) data in netCDF, subsetted for bands LST_Day_1km, LST_Night_1km, QC_Day, QC_Night, for each day between the 2017-01-01 and the 2017-01-30, and within the following bounding box (lon/lat): -5.41 8.84, -5.82 9.54.
Expand All @@ -39,7 +39,7 @@ observations, as well as other widely-used satellite-derived
environmental data (e.g. Global Precipitation Measurement Mission).

**`modisfast`** uses the abilities offered by the [OPeNDAP
framework](https://www.opendap.org/about) (*Open-source Project for a
framework](https://www.opendap.org/about/) (*Open-source Project for a
Network Data Access Protocol*) to download a subset of Earth science
data cube, along spatial, temporal or any other data dimension (depth,
…). This way, it reduces downloading time and disk usage to their
Expand Down Expand Up @@ -157,7 +157,7 @@ modisfast provides an entry point to some specific OPeNDAP servers (e.g. MODIS,
Apart from these performance aspects, ethical considerations have driven the development of this package :
* **Facilitating the access to Earth science data for R users in places where internet connection is slow or expensive** : Earth science products are generally huge files that can be quite difficult to download in places with slow internet connection, even more if large time series are needed. By enabling to download strictly the data that is needed, the products become more accessible in those places;
* **Caring about the environmental digital impact of our research work** : Downloading data has an impact on environment and to some extent contributes to climate change. By downloading only the data that is need (rather than e.g a whole MODIS tile, or a global SMAP or GPM dataset) we somehow promote digital sobriety.
* **Supporting the open-source-software movement** : The OPeNDAP is developed and advanced openly and collaboratively, by the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about) This open, powerfull and standard data access protocol is more and more used, by major Earth science data providers (e.g. NASA or NOAA). Using OPeNDAP means supporting methods for data access protocols that are open, build collaboratively and shared.
* **Supporting the open-source-software movement** : The OPeNDAP is developed and advanced openly and collaboratively, by the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about/) This open, powerfull and standard data access protocol is more and more used, by major Earth science data providers (e.g. NASA or NOAA). Using OPeNDAP means supporting methods for data access protocols that are open, build collaboratively and shared.
-->
<!--
## Citation
Expand All @@ -166,7 +166,7 @@ We thank in advance people that use `modisfast` for citing it in their work / pu

## Collections available in modisfast

Currently `modisfast` supports download of 78 data collections,
Currently `modisfast` supports download of 77 data collections,
extracted from the following meta-collections :

- [MODIS land
Expand Down Expand Up @@ -753,25 +753,6 @@ Primary Productivity
</tr>
<tr>
<td style="text-align:left;">
MOD17A3.055
</td>
<td style="text-align:left;">
MODIS/Terra Net Primary Production Yearly L4 Global 1 km SIN Grid
</td>
<td style="text-align:left;">
MODIS
</td>
<td style="text-align:left;">
Primary Productivity
</td>
<td style="text-align:left;">
</td>
<td style="text-align:left;">
<https://opendap.cr.usgs.gov/opendap/hyrax/MOD17A3.055/contents.html>
</td>
</tr>
<tr>
<td style="text-align:left;">
MOD17A3HGF.061
</td>
<td style="text-align:left;">
Expand Down Expand Up @@ -1850,7 +1831,7 @@ series and points.
There are other R packages available for accessing MODIS data, which may
be more suitable if your requirements differ. These include :

- [`MODIS`](https://github.com/MatMatt/MODIS)
- [`MODIS`](https://github.com/fdetsch/MODIS)
- [`MODIStsp`](https://github.com/ropensci/MODIStsp)
- [`MODIStools`](https://github.com/ropensci/MODIStools)
- [`appeears`](https://github.com/bluegreen-labs/appeears)
Expand Down Expand Up @@ -1934,6 +1915,6 @@ connection is slow or expensive and promotes digital sobriety for our
research work.

The OPeNDAP, over which the package builds, is a project developed by
the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about) and
the non-profit [OPeNDAP, Inc.](https://www.opendap.org/about/) and
advanced openly and collaboratively. By using this data access protocol,
`modisfast` support the open-source-software movement.
Loading

0 comments on commit 7e11dae

Please sign in to comment.