Ncbi metadata download #18
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
karinlag
reviewed
Nov 14, 2022
karinlag
reviewed
Nov 14, 2022
karinlag
reviewed
Nov 14, 2022
karinlag
reviewed
Nov 14, 2022
karinlag
reviewed
Nov 14, 2022
karinlag
reviewed
Nov 14, 2022
|
I think I sort of understand what you are doing, but let's chat when you are back |
|
@karinlag Can you have a second look ? |
|
It is almost there. You lose me with the "this"es where you introduce the yaml files :) sort that out, and I'll give it a final readthrough :) |
karinlag
reviewed
Feb 26, 2024
| conda activate ncbimeta | ||
| ``` | ||
|
|
||
| To use ncbimeta, you will need to use an API (application programming interface) key from NCBI. |
There was a problem hiding this comment.
An API is a way of asking a different computer to send you information. The key is a way for NCBI to associate you with the specific request.
Just concept sorting/clarification.
karinlag
reviewed
Feb 26, 2024
| per second than you would be able to perform without it. | ||
|
|
||
| To get an API key from NCBI : | ||
| - login (or create an account) |
karinlag
reviewed
Feb 26, 2024
| - login (or create an account) | ||
| - account > account settings : there you can create an API key you will use - copy it and keep it safe | ||
|
|
||
| We will need to create a configuration file (.yalm) for each metadata download |
karinlag
reviewed
Feb 26, 2024
| A trick is to create a simple search in NCBI ex: | ||
|  | ||
|
|
||
| This search field will help you build your "metadata download configuration file" (.yalm). |
karinlag
reviewed
Feb 26, 2024
|
|
||
| Compare it to the [.yalm configuration file](./providencia_metadata.yaml) | ||
|
|
||
| The `.yalm` file allow to define the destination of the download (OUTPUT_DIR), |
karinlag
reviewed
Feb 26, 2024
| your NCBI identification (EMAIL and API_KEY), the name of the sqlite database it will | ||
| download the metadata to (DATABASE), the tables it will create within the database | ||
| (TABLES) and the columns/fields it will create and download data to for each | ||
| table in the sqlite database (TABLE_COLUMNS, indentation: table name you want, |
karinlag
reviewed
Feb 26, 2024
|
|
||
| So, you see that you can define your own fields, that were not provided in the other examples found in the [Biosample example](https://github.com/ktmeaton/NCBImeta/tree/master/schema). | ||
|
|
||
| To define those fields you need to access at which hierarchy of the `.xml` file you downloaded as helper. |
There was a problem hiding this comment.
This sentence does not make sense to me.
There was a problem hiding this comment.
And on reread I am completely lost here. I have no idea of how to get to the yaml file I need for the search.
karinlag
reviewed
Feb 26, 2024
| conda activate NCBImeta | ||
| NCBImeta --config providencia_metadata.yaml --flat | ||
| ``` | ||
|
|
karinlag
reviewed
Mar 11, 2024
|
|
||
| To use ncbimeta, you will need to use an API (application programming interface) key from NCBI. | ||
| An API key is an authentication code that will serves to tell NCBI that you are the user | ||
| making requests to download metadata via ncbimeta software. Using an NCBI API key |
karinlag
reviewed
Mar 11, 2024
| run the software, but it is a configuration file that specifies the search/query | ||
| we are asking the software to do: to retrieve the available metadata we are interested in, | ||
| and then the specification of the specific of fields we want to download in a database/table. | ||
|
|
There was a problem hiding this comment.
I suggest you somehwere here show a screenshot of what this file should look like, not only link to it.
|
Are people still working on this? |
|
Yes , still need to do implement what Karin does want me to.modify ...
…On Thu, 19 Sept 2024, 17:58 novica, ***@***.***> wrote:
Are people still working on this?
—
Reply to this email directly, view it on GitHub
<#18 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AK3TRDPFCLFFIENAVNWN3GDZXLYCJAVCNFSM6AAAAAAR5W57ESVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNRRGQYTCOBQG4>
.
You are receiving this because you authored the thread.Message ID:
***@***.***
.com>
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please have a fast prettifying/spelling correction and pull this to the github