Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

auto-import from our Zenodo community #114

Merged
merged 18 commits into from
Nov 18, 2024
Merged

Conversation

haesleinhuepf
Copy link
Member

@haesleinhuepf haesleinhuepf commented Jul 11, 2024

I'm adding a Jupyter notebook and an "imported.yml" file. The notebook automatically imported all entries from our Zenodo community.

When the notebook is executed next time, it will not re-import entries. It will just add those which we were missing yet.

@SeverusYixin would you mind reviewing this PR? Let me know what you think!

This PR closes #47
Also related: #2

@haesleinhuepf
Copy link
Member Author

haesleinhuepf commented Jul 11, 2024

Interesting, the auto-importer added duplicate entries. Read details here:
https://github.com/NFDI4BIOIMAGE/training/actions/runs/9892381656/job/27324909692?pr=114#step:5:3043

@SeverusYixin
Copy link
Collaborator

Interesting, the auto-importer added duplicate entries. Read details here: https://github.com/NFDI4BIOIMAGE/training/actions/runs/9892381656/job/27324909692?pr=114#step:5:3043

I also see it

@haesleinhuepf
Copy link
Member Author

I just merged #113 . It is possible that then tests pass here.

@haesleinhuepf
Copy link
Member Author

Ok, this didn't help. I presume we need to identify DOI-links for each zenodo link, before adding the zenodo-link to our list. It is possible that we have the corresponding DOI link in our list and that's why the duplication-checker raises an error.

It might make sense to clean up the code a bit in advance.

"token = os.getenv('ZENODO_API_KEY')\n",
"community = 'nfdi4bioimage'\n",
"\n",
"response = requests.get('https://zenodo.org/api/records',\n",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be no theoretical error. What about add some error test like API rate limits, or other HTTP-related errors?

maybe like:
try:
response = requests.get('https://zenodo.org/api/records',
params={'communities': community,
'access_token': token})

# Raises an HTTPError for bad responses
response.raise_for_status()  
online_data = response.json()

except requests.exceptions.RequestException as e:
print(f"Error fetching data: {e}")

# Ensuring online_data is defined in case of an error
online_data = {} 

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

online_data = {}

If there is no online data, running the rest of the code is pointless.

But I agree some error handling would be nice

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I also mark this as viewed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. I never marked any files as viewed. Let me know what it does to the file :-)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

like this, viewed and approve

@haesleinhuepf haesleinhuepf marked this pull request as draft July 19, 2024 12:56
@haesleinhuepf
Copy link
Member Author

I was converting this into a draft. We need to fix the deployment (duplicate entries) before merging this

@SeverusYixin
Copy link
Collaborator

I was converting this into a draft. We need to fix the deployment (duplicate entries) before merging this

If you want me to review it again, just send it to me :)

@SeverusYixin
Copy link
Collaborator

Do I need to review it? Because all I got was a notification on the top right side, but no comment.

@haesleinhuepf haesleinhuepf marked this pull request as ready for review November 18, 2024 13:41
@haesleinhuepf haesleinhuepf merged commit 3d835da into main Nov 18, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add content from our zenodo community
2 participants