Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create generalised extract_data function #116

Closed
wants to merge 82 commits into from

Conversation

ehwenk
Copy link
Collaborator

@ehwenk ehwenk commented Nov 11, 2024

  • Main item is a new, generalised extract_data function that makes it possible to subset a traits.build database using any column in any table.
    • The function works with single values or vectors of values
    • Does not return an error for empty searches
    • For context property values, searches across all context property categories simultaneously
  • The function from traits.build that is used to bind together datasets has been moved to austraits, allowing one to extract multiple pieces of a database, then bind them back together into a single database
  • Various other documentation fixes

fontikar and others added 30 commits January 12, 2023 16:04
…king for new version, waiting for austraits.build update
…rough the internal data subsets for version 3.0.2 and version 4.0.0. Pivot_ needs austraits.build fixes to implenment. Some minor fix to pivot_wider for dependencies #60
Some packages needed to make plots are included in suggests. This means that core functions of package may not work.
- reuse outputs from previous function calls to reduce runtime
- reduce dataset sizes for slow functions (summarise_trait_means, trait_pivot_wider, plot_locations)
- silence some outputs

closes #62
- As documented in #79 , the Zenodo API has changed, breaking our download feature. 
- This commit updates the internals to work with the latest changes. 

Specifically: 

- the way to access json for all versions has changed (changed url structure, and for id we now use one of the record ids, rather than the conceptid)
- the call to download file has changed
- format of the API json has changed

Also

- added record id to the table of versions
- put a check in to remove "v" from any version entered by user
* changes required for v5 austraits.build

* Removed original_name for trait_pivot_wider3 for v5.0.0

* Added trait_pivot_wider for v4.x.x and code for what_version

* Making new switches for join and as_wide_table based on new versioning

* Sub switch for extract_ and recreated internal data

* Sub switches for trait_pivot_longer

* Minor fix in join_methods

* Added vars a global vars

* Removed .data calls when not needed

* Update to work with latest zenodo API (#81)


- As documented in #79 , the Zenodo API has changed, breaking our download feature. 
- This commit updates the internals to work with the latest changes. 

Specifically: 

- the way to access json for all versions has changed (changed url structure, and for id we now use one of the record ids, rather than the conceptid)
- the call to download file has changed
- format of the API json has changed

Also

- added record id to the table of versions
- put a check in to remove "v" from any version entered by user

* Recreated data so extract is passing

* Update `treatment_id` with `treatment_context_id`

* Revert "Update `treatment_id` with `treatment_context_id`"

This reverts commit 3fc6717.

* minor column name changes

Changes column names, reflecting recent changes to traits.build output.

* Update as_wide_table.R

add `any_of` to column selection within `as_wide_table` to accommodate other traits.build databases that don't have the same columns in taxon_list.csv

* Fixed getting versions and load austraits with zenodo updates and minor update with as_wide_table with removal of variable

* Fixed minor bug in get_version_latest

---------

Co-authored-by: Elizabeth Wenk <[email protected]>
Co-authored-by: Daniel Falster <[email protected]>
Co-authored-by: yangsophieee <[email protected]>
* Created lites for all main versions of AusTraits

* Passing for as_wide_table

* Passing for as_wide_table and for extract_

* expanding test suite to all 3 majors, switches for method id adjusted

* expanding test suite to all 3 majors for summarise_D

* Expanding these for all 3 versions

* Added PR trigger for dev branch
ehwenk and others added 27 commits November 8, 2024 22:16
Specific context tables might be empty (null) if a series of extract functions are used in tandem.

Was fixing tests for austraits_lite5.0.0 and came across another issue... Unlike the old extract functions the new one doesn't work with vectors of values - line 123 in `extract_data`
The tests to do with extract_data are generally working, but there is a weird, oscillating bug with the order of tables in the database list that I can't work out.

And also there is a test on line 198 (expect_equal(trait_subset$traits %>% dplyr::distinct(dataset_id) %>% nrow(), 5)) alternates between giving 5 & 8 as output.

But bugs fixed:
- works with empty tibbles
- works with vectors of values to match
series of small errors causing tests to fail:
- needed to reorder tables in some of the austraits_lite versions
- recreated the austraits_lite for versions 3 & 4 that are empty, other than having the structure required to test compatibility
- plot_data needed an `as.numeric`
- the function `summarise_trait_means` was still being tests on v3.0 and indeed looking at the code, isn't going to summarise, because it is only searching for replicates within a single observation_id
The function `join_context_properties` was not being exported
Write tests for bind_databases (issue #114)
changed `plant_trait_name` to `trait_name` in the function `plot_trait_distribution_beeswarm` to generalise it to all traits.build databases.

Towards Issue #111
@dfalster
Copy link
Member

Hi @ehwenk, I think the PR has incorrectly been directed to master branch. Can you resubmit to develop branch?

@ehwenk ehwenk closed this Nov 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants