[REVIEW]: Empirical: A scientific software library for research, education, and public engagement #6617

editorialbot · 2024-04-13T21:13:21Z

Submitting author: @mmore500 (Matthew Andres Moreno)
Repository: https://github.com/devosoft/Empirical/
Branch with paper.md (empty if default branch): joss-paper
Version: v1.1.5
Editor: @mahfuz05062
Reviewers: @LTLA, @bramvandijk88
Archive: 10.5281/zenodo.11420797

Status

Status badge code:

HTML: <a href="https://joss.theoj.org/papers/9a9ff2d6232a7511878151c26a9433f4"><img src="https://joss.theoj.org/papers/9a9ff2d6232a7511878151c26a9433f4/status.svg"></a>
Markdown: [![status](https://joss.theoj.org/papers/9a9ff2d6232a7511878151c26a9433f4/status.svg)](https://joss.theoj.org/papers/9a9ff2d6232a7511878151c26a9433f4)

Reviewers and authors:

Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)

Reviewer instructions & questions

@LTLA & @bramvandijk88, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:

@editorialbot generate my checklist

The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @mahfuz05062 know.

✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨

Checklists

📝 Checklist for @bramvandijk88

📝 Checklist for @LTLA

The text was updated successfully, but these errors were encountered:

editorialbot · 2024-04-13T21:13:24Z

Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.

For a list of things I can do to help you, just type:

@editorialbot commands

For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:

@editorialbot generate pdf

editorialbot · 2024-04-13T21:13:55Z

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1145/3062341.3062363 is OK
- 10.1145/367701.367714 is OK
- 10.1145/2048147.2048224 is OK
- 10.1093/bib/bbp073 is OK
- 10.1186/1752-0509-6-96 is OK
- 10.1145/3205455.3205523 is OK
- 10.1162/artl_a_00284 is OK
- 10.1177/0037549712462620 is OK
- 10.1186/2194-3206-1-3 is OK
- 10.1145/3205651.3205780 is OK
- 10.5281/zenodo.4118608 is OK
- 10.1145/3185517 is OK
- 10.5281/zenodo.2575606 is OK
- 10.1038/nchem.1149 is OK
- 10.48550/ARXIV.2108.00382 is OK

MISSING DOIs

- No DOI given, and none found for title: Not so fast: Analyzing the performance of webassem...
- No DOI given, and none found for title: Squares: A Fast Counter-Based RNG
- No DOI given, and none found for title: LEARNING EVOLUTION AND THE NATURE OF SCIENCE USING...
- No DOI given, and none found for title: Netlogo: A simple environment for modeling complex...
- No DOI given, and none found for title: shiny: Web Application Framework for R
- No DOI given, and none found for title: shiny: Web Application Framework for R
- No DOI given, and none found for title: Suicide, signals, and symbionts: Evolving cooperat...
- No DOI given, and none found for title: Changing Environments Drive the Separation of Gene...
- No DOI given, and none found for title:  Open Science Philosophy

INVALID DOIs

- None

editorialbot · 2024-04-13T21:14:01Z

Software report:

github.com/AlDanial/cloc v 1.90  T=6.55 s (139.9 files/s, 26043.8 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C/C++ Header                   307          14616          17491          59901
C++                            384          10217           8478          38101
JavaScript                      10             99             82           9981
Markdown                        47           1029              0           3212
make                            61            908            392           2346
HTML                            51            118             24            841
Bourne Shell                    42            178            204            592
TeX                              2             44              0            443
Dockerfile                       1             27             16            241
YAML                             2              4             18            233
Python                           4             61            108            176
LESS                             1             26             41            133
reStructuredText                 1             63              8            124
CSV                              2              0              0             56
JSON                             2              0              0             25
-------------------------------------------------------------------------------
SUM:                           917          27390          26862         116405
-------------------------------------------------------------------------------

Commit count by author:

  7566	Charles Ofria
  1186	Matthew Andres Moreno
   746	Emily Dolson
   247	Alexander Lalejini
   207	rodsan0
   172	Alex Lalejini
   113	Katherine Perry
    97	Austin Ferguson
    88	kayakingCellist
    85	Jake Fenton
    66	Tait Weicht
    55	[email protected]
    45	Riley Hoffman
    41	Steven Jorgensen
    36	NateRiz
    31	emilydolson
    18	grenewode
    17	abbywlsn
     9	Robin Miller
     8	Oliver-BE
     7	anyaevostinar
     6	Jason Stredwick
     4	Replit user
     4	perryk12
     3	Jose
     3	Raheem Clemons
     2	Anya V
     2	Emily Louise Dolson
     2	Jory Schossau
     2	Luis Zaman
     2	Steven Patrick Jorgensen
     2	cgnitash
     2	mmore500
     1	Acacia Ackles
     1	Anya Johnson
     1	Anya Vostinar
     1	HackMD
     1	Santiago Rodriguez Papa
     1	c-moreno
     1	djrain
     1	leg2015
     1	ryan-moreno

editorialbot · 2024-04-13T21:14:05Z

Paper file info:

📄 Wordcount for paper.md is 1379

✅ The paper includes a Statement of need section

editorialbot · 2024-04-13T21:14:08Z

License info:

🟡 License found: Other (Check here for OSI approval)

editorialbot · 2024-04-13T21:14:31Z

👉📄 Download article proof 📄 View article proof on GitHub 📄 👈

mahfuz05062 · 2024-04-13T21:21:53Z

@LTLA and @bramvandijk88 - Thank you for agreeing to review this submission.

This is the review thread for the paper. All of our communications will happen here from now on.

As mentioned above, you can use the command @editorialbot generate my checklist to create your review checklist. As you go over the submission, please check any items that you feel have been satisfied.

There are also links to the JOSS reviewer guidelines (https://joss.readthedocs.io/en/latest/reviewer_guidelines.html)

The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#6617 so that a link is created to this thread for visibility. Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.

We aim for reviews to be completed within about 2-4 weeks. Please let me know if you require additional time. We can also use editorialbot (our bot) to set automatic reminders if you know you'll be away for a known period.

Please feel free to ping me (@mahfuz05062) if you have any questions/concerns.

bramvandijk88 · 2024-04-15T08:18:52Z

Review checklist for @bramvandijk88

Conflict of interest

I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

I confirm that I read and will adhere to the JOSS code of conduct.

General checks

Repository: Is the source code for this software available at the https://github.com/devosoft/Empirical/?
License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
Contribution and authorship: Has the submitting author (@mmore500) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

Installation: Does installation proceed as outlined in the documentation?
Functionality: Have the functional claims of the software been confirmed?
Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
State of the field: Do the authors describe how this software compares to other commonly-used packages?
Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

bramvandijk88 · 2024-04-15T08:43:29Z

Heya, I've started my review in between some teaching obligations. I generally dislike debugging paths and compiling C code, so I'll leave the problem-solving up to the authors:

In following the installation instructions on Mac OSX (Ventura 13.2.1) I get the following missing file:

Makefile:22: WARNING: the Cookiecutter Empirical Project, which you can find at https://github.com/devosoft/cookiecutter-empirical-project, should be preferred over ProjectTemplate g++ -O3 -DNDEBUG -Wall -Wno-unused-function -std=c++20 -I../../../Empirical/include/ source/native/project_name.cpp -o project_name In file included from source/native/project_name.cpp:21: In file included from ../../../Empirical/include/emp/config/command_line.hpp:42: In file included from ../../../Empirical/include/emp/config/../tools/string_utils.hpp:35: ../../../Empirical/include/emp/config/../tools/../base/array.hpp:28:10: fatal error: '../../../third-party/cereal/include/cereal/cereal.hpp' file not found #include "../../../third-party/cereal/include/cereal/cereal.hpp" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. make: *** [project_name] Error 1

LTLA · 2024-04-17T15:12:34Z

Review checklist for @LTLA

Conflict of interest

I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.

Code of Conduct

I confirm that I read and will adhere to the JOSS code of conduct.

General checks

Repository: Is the source code for this software available at the https://github.com/devosoft/Empirical/?
License: Does the repository contain a plain-text LICENSE or COPYING file with the contents of an OSI approved software license?
Contribution and authorship: Has the submitting author (@mmore500) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.

Functionality

Installation: Does installation proceed as outlined in the documentation?
Functionality: Have the functional claims of the software been confirmed?
Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)

Documentation

A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

Software paper

Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
State of the field: Do the authors describe how this software compares to other commonly-used packages?
Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?

LTLA · 2024-04-17T19:44:22Z

Alright, apologies for the delay. Here's my review:

This is a very interesting library. I have long wondered how to manipulate HTML from within C++, and Empirical provides an ergonomic way of doing so. It offers the possibility of writing web applications almost completely in C++, without any need to touch Javascript or associated frameworks. I can forsee this being useful for scientific programmers who want to build a simple application based on C++ tools without requiring any web development knowledge. In this respect, Empirical targets a similar market to Shiny (R) or dash (Python), but for C++ developers. That said, more experienced web developers will probably not use these features, as direct DOM manipulation is discouraged in frameworks like React.

The extra debugging information for the standard data structures is nice, especially for WebAssembly where debugging segfaults is even more tedious than usual. Perhaps the authors might consider checking whether a macro could be used to seamlessly switch between emp:: and std::. This would allow library developers to use Empirical for debug builds, but then switch back to the STL for release, in order to reduce dependencies for their own users. For example, if I was developing a header-only library that uses Empirical for debugging, I would prefer to avoid mandating Empirical as a compile-time dependency for all applications that use my library.

The paper itself is well-written and concise. Possibly too concise, actually: it might have benefited from one or two short code examples to demonstrate the ease-of-use of the UI manipulation, bounds-checking, etc. Then readers can get a better grasp of what the library actually does, before deciding whether to proceed to the documentation itself.

On some other minor points:

The README.md could be fleshed out with more motivation, e.g., by transplanting the statement of need from the paper. When I hit the repo landing page, I wasn't sure what this library was about, and when I trawled the codebase, I spent a while thinking that it was dedicated to evolution/phylo-related programming. Most people won't think that the interesting stuff is in base or web when they can instead be looking at a directory with an exciting name like Evolve.
Some instructions about the dependencies would be helpful during installation, see my comments about third-party requirements and jQuery.

Related issues:

mmore500 · 2024-04-19T14:08:32Z

Heya, I've started my review in between some teaching obligations. I generally dislike debugging paths and compiling C code, so I'll leave the problem-solving up to the authors:

In following the installation instructions on Mac OSX (Ventura 13.2.1) I get the following missing file:

Makefile:22: WARNING: the Cookiecutter Empirical Project, which you can find at https://github.com/devosoft/cookiecutter-empirical-project, should be preferred over ProjectTemplate g++ -O3 -DNDEBUG -Wall -Wno-unused-function -std=c++20 -I../../../Empirical/include/ source/native/project_name.cpp -o project_name In file included from source/native/project_name.cpp:21: In file included from ../../../Empirical/include/emp/config/command_line.hpp:42: In file included from ../../../Empirical/include/emp/config/../tools/string_utils.hpp:35: ../../../Empirical/include/emp/config/../tools/../base/array.hpp:28:10: fatal error: '../../../third-party/cereal/include/cereal/cereal.hpp' file not found #include "../../../third-party/cereal/include/cereal/cereal.hpp" ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1 error generated. make: *** [project_name] Error 1

Hi @bramvandijk88

Thanks for bringing this to our attention! We've updated the documentation to clarify a missing step of initializing the project subrepositories. If you do

git submodule update --init --recursive --depth 1 and then try compiling again it should hopefully work!

bramvandijk88 · 2024-04-22T09:06:52Z

If you do git submodule update --init --recursive --depth 1 and then try compiling again it should hopefully work!

Installation was running for ~ 10 minutes until I ran into this:

rm -rf test*.out
cd hardware && make test
mkdir -p temp
c++ -std=c++20 -g -pthread -Wall -Wno-unused-function -Wno-unused-private-field -I../../include/ -I../../ -I../../third-party/cereal/include/ -DCATCH_CONFIG_MAIN event_driven_gp.cpp -o test-event_driven_gp.out
In file included from event_driven_gp.cpp:25:
In file included from ../../include/emp/hardware/EventDrivenGP.hpp:36:
In file included from ../../include/emp/hardware/../matching/MatchBin.hpp:36:
../../include/emp/hardware/../matching/../../../third-party/robin-hood-hashing/src/include/robin_hood.h:54:14: fatal error: 'sys/auxv.h' file not found
#    include <sys/auxv.h> // for getauxval
             ^~~~~~~~~~~~
1 error generated.
make[2]: *** [test-event_driven_gp] Error 1
make[1]: *** [test-hardware] Error 2
make: *** [test-native-regular] Error 2

bramvandijk88 · 2024-04-22T09:09:21Z

On a different note (I'm sure the installation will eventually work out), I'm happy @LTLA went into depth on the C stuff (e.g. the smart pointer comments), as it's been over 6 years since I did anything with C. That leaves room for me to test the examples thoroughly, which I am quite fond of doing. I hope that together with @LTLA's comments, we'll cover all the bases.

mmore500 · 2024-04-22T15:00:48Z

Hi @bramvandijk88 ---

Apologies for the friction here!! Fortunately, I think we just fixed this issue as @mercere99 just ran into it last week. (This PR devosoft/Empirical#515).

The fix has been merged onto the master Empirical branch. It updates our git submodule of the robin-hood hashing repository. It can be a little involved to bump submodule versions on an existing clone, so the easiest thing might be to start a fresh clone of Empirical. When you clone now and initialize submodules, it will pull a version of robin-hood hashing that doesn't use sys/auxv.h.

Looking forward to getting your feedback on the examples, and please do let us know of any further issues you may run into.

bramvandijk88 · 2024-04-23T10:10:50Z

Hi!

I got through the installation now, and have completed the checklist. I have also tested some of the (basic) examples and tested some self-written code using WASM myself. Here's my review:

The authors clearly did all the due diligence when it comes to installation and automated testing. Barring a few minor hick-ups (which they resolved), everything installed smoothly and I got into the actual fun part. Similar to the other reviewer, I feel like the paper could have provided a bit more details. If anything, the current paper undersells the utility of this tool, which is definitely a wheel I am not going to reinvent in the future (in fact, I tagged some students to pick some of this up and build something cool with it).

I've always enjoyed explorable and sharable simulations, which is why I gradually moved from C/C++ to javascript/typescript. While I was made aware of Webassembly, getting all of this properly set up sounded like too much of a hassle. TLDR: the authors did the community a great service! I would say my only (minor) disappointment (which may be a misconception) is that running the code in a web browser still requires a server, meaning it will not be as easy for students that are just learning to code: e.g. they first need to learn how to install python and launch a local server using that. It's a minor annoyance I have when I am faced with 50 students all of which have their own errors in "trying to install stuff", but I suppose that's not always avoidable. It would be amazing if the authors considered to, in the future, porting their tool to something like WASMfiddle which would further enable educators such as myself to use their software in teaching students basic programming skills.

On the actual structures provided by Empirical, I did not have the time to test all the individual types (worlds, genomes) and their accompanying methods (GetNumOrgs, etc.). However, the provided examples in the "Built with Emperical Gallery" (all of which have a link to their original code) would give a starting coder plenty of room to work with. I am a little surprised by the performance of e.g. the cancer model still appearing quite slow, but perhaps there is a lot more going on under the hood than I realise. None of this invalidates the software anyway.

In other words, it seems like any enthusiast would get something running with relatively little effort. Congratulations to the authors for this amazing piece of software!

mahfuz05062 · 2024-04-27T03:18:53Z

@bramvandijk88 Thank you for completing the review! I see the other review is also going very well and I will check back later.

mahfuz05062 · 2024-05-04T01:37:21Z

@LTLA Please let us know if you have any questions or need any assistance on the remaining task of the checklist.

LTLA · 2024-05-07T00:11:48Z

Sorry for the late reply; lgtm. Some of my issues are still open but are trending in the right direction so not a blocker.

mmore500 · 2024-05-08T21:01:04Z

Pinging my co-maintainers and try to get a review on the de-jquerifyication PR that's open. Hopefully progress on that soon.

editorialbot · 2024-06-01T23:08:41Z

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1145/3062341.3062363 is OK
- 10.1145/367701.367714 is OK
- 10.1145/2048147.2048224 is OK
- 10.1093/bib/bbp073 is OK
- 10.5281/zenodo.5156931 is OK
- 10.1186/1752-0509-6-96 is OK
- 10.1145/3205455.3205523 is OK
- 10.1162/artl_a_00284 is OK
- 10.1177/0037549712462620 is OK
- 10.1186/2194-3206-1-3 is OK
- 10.1145/3205651.3205780 is OK
- 10.5281/zenodo.4118608 is OK
- 10.1145/3185517 is OK
- 10.5281/zenodo.2575606 is OK
- 10.1038/nchem.1149 is OK
- 10.48550/ARXIV.2108.00382 is OK
- 10.48550/ARXIV.2211.10897 is OK
- 10.48550/ARXIV.2405.09389 is OK

MISSING DOIs

- No DOI given, and none found for title: Not so fast: analyzing the performance of webassem...
- No DOI given, and none found for title: Squares: A Fast Counter-Based RNG
- No DOI given, and none found for title: LEARNING EVOLUTION AND THE NATURE OF SCIENCE USING...
- No DOI given, and none found for title: Netlogo: A simple environment for modeling complex...
- No DOI given, and none found for title: shiny: Web Application Framework for R
- No DOI given, and none found for title: Suicide, signals, and symbionts: Evolving cooperat...
- No DOI given, and none found for title: Changing Environments Drive the Separation of Gene...
- No DOI given, and none found for title:  Open Science Philosophy

INVALID DOIs

- None

editorialbot · 2024-06-01T23:09:56Z

👋 @openjournals/csism-eics, this paper is ready to be accepted and published.

Check final proof 👉📄 Download article

If the paper PDF and the deposit XML files look good in openjournals/joss-papers#5432, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

danielskatz · 2024-06-02T06:39:09Z

@mmore500 - As track editor, I've proofread the paper, which looks good, except for some minor issues in the references. Please merge devosoft/Empirical#520 or let me know what you disagree with.

danielskatz · 2024-06-02T06:41:01Z

@mmore500 - Also, I see the metadata in the zenodo repository doesn't match the paper, which we ask to happen. I see some discussion about this in #6617 (comment). How should we resolve this?

mmore500 · 2024-06-02T06:57:08Z

Merged 520, thanks for those.

danielskatz · 2024-06-02T06:57:53Z

@editorialbot recommend-accept

proofing again to make sure everything worked...

editorialbot · 2024-06-02T06:57:55Z

Attempting dry run of processing paper acceptance...

mmore500 · 2024-06-02T06:58:14Z

What is the nature of the mismatch in the Zenodo repository? I think the titles should exactly match now. Is it the author list?

editorialbot · 2024-06-02T06:59:16Z

👋 @openjournals/csism-eics, this paper is ready to be accepted and published.

Check final proof 👉📄 Download article

If the paper PDF and the deposit XML files look good in openjournals/joss-papers#5434, then you can now move forward with accepting the submission by compiling again with the command @editorialbot accept

danielskatz · 2024-06-02T06:59:19Z

What is the nature of the mismatch in the Zenodo repository? I think the titles should exactly match now. Is it the author list?

yes.

editorialbot · 2024-06-02T06:59:25Z

Reference check summary (note 'MISSING' DOIs are suggestions that need verification):

OK DOIs

- 10.1145/3062341.3062363 is OK
- 10.1145/367701.367714 is OK
- 10.1145/2048147.2048224 is OK
- 10.1093/bib/bbp073 is OK
- 10.5281/zenodo.5156931 is OK
- 10.1186/1752-0509-6-96 is OK
- 10.1145/3205455.3205523 is OK
- 10.1162/artl_a_00284 is OK
- 10.1177/0037549712462620 is OK
- 10.1186/2194-3206-1-3 is OK
- 10.1145/3205651.3205780 is OK
- 10.5281/zenodo.4118608 is OK
- 10.1145/3185517 is OK
- 10.5281/zenodo.2575606 is OK
- 10.1038/nchem.1149 is OK
- 10.48550/ARXIV.2108.00382 is OK
- 10.48550/ARXIV.2211.10897 is OK
- 10.48550/ARXIV.2405.09389 is OK

MISSING DOIs

- No DOI given, and none found for title: Not so fast: analyzing the performance of webassem...
- No DOI given, and none found for title: Squares: A Fast Counter-Based RNG
- No DOI given, and none found for title: LEARNING EVOLUTION AND THE NATURE OF SCIENCE USING...
- No DOI given, and none found for title: Netlogo: A simple environment for modeling complex...
- No DOI given, and none found for title: shiny: Web Application Framework for R
- No DOI given, and none found for title: Suicide, signals, and symbionts: Evolving cooperat...
- No DOI given, and none found for title: Changing Environments Drive the Separation of Gene...
- No DOI given, and none found for title:  Open Science Philosophy

INVALID DOIs

- None

mmore500 · 2024-06-02T07:00:18Z

I will release to zenodo with an udpated author list. Should take about 10 minutes I expect.

danielskatz · 2024-06-02T07:02:04Z

you should just be able to update the metadata - you don't need to make a new release or deposit

Also, I probably won't get back to this for 6-24 hours

mmore500 · 2024-06-02T07:06:40Z

There seems to be some issue with permissions in our organization on Zenodo that's not letting me update the metadata. (Or I'm just not using the interface correctly.)

mmore500 · 2024-06-02T07:09:49Z

The new version is v1.1.5 and the new Zenodo DOI is https://doi.org/10.5281/zenodo.11420797

danielskatz · 2024-06-02T15:55:44Z

@editorialbot set v1.1.5 as version

editorialbot · 2024-06-02T15:55:47Z

Done! version is now v1.1.5

danielskatz · 2024-06-02T15:56:02Z

@editorialbot set 10.5281/zenodo.11420797 as archive

editorialbot · 2024-06-02T15:56:05Z

Done! archive is now 10.5281/zenodo.11420797

danielskatz · 2024-06-02T15:56:15Z

@editorialbot accept

editorialbot · 2024-06-02T15:56:17Z

Doing it live! Attempting automated processing of paper acceptance...

editorialbot · 2024-06-02T16:03:33Z

Ensure proper citation by uploading a plain text CITATION.cff file to the default branch of your repository.

If using GitHub, a Cite this repository menu will appear in the About section, containing both APA and BibTeX formats. When exported to Zotero using a browser plugin, Zotero will automatically create an entry using the information contained in the .cff file.

You can copy the contents for your CITATION.cff file here:

CITATION.cff

cff-version: "1.2.0"
authors:
- family-names: Vostinar
  given-names: Anya
  orcid: "https://orcid.org/0000-0001-7216-5283"
- family-names: Lalejini
  given-names: Alexander
  orcid: "https://orcid.org/0000-0003-0994-2718"
- family-names: Ofria
  given-names: Charles
  orcid: "https://orcid.org/0000-0003-2924-1732"
- family-names: Dolson
  given-names: Emily
  orcid: "https://orcid.org/0000-0001-8616-4898"
- family-names: Moreno
  given-names: Matthew Andres
  orcid: "https://orcid.org/0000-0003-4726-4479"
doi: 10.5281/zenodo.11420797
message: If you use this software, please cite our article in the
  Journal of Open Source Software.
preferred-citation:
  authors:
  - family-names: Vostinar
    given-names: Anya
    orcid: "https://orcid.org/0000-0001-7216-5283"
  - family-names: Lalejini
    given-names: Alexander
    orcid: "https://orcid.org/0000-0003-0994-2718"
  - family-names: Ofria
    given-names: Charles
    orcid: "https://orcid.org/0000-0003-2924-1732"
  - family-names: Dolson
    given-names: Emily
    orcid: "https://orcid.org/0000-0001-8616-4898"
  - family-names: Moreno
    given-names: Matthew Andres
    orcid: "https://orcid.org/0000-0003-4726-4479"
  date-published: 2024-06-02
  doi: 10.21105/joss.06617
  issn: 2475-9066
  issue: 98
  journal: Journal of Open Source Software
  publisher:
    name: Open Journals
  start: 6617
  title: "Empirical: A scientific software library for research,
    education, and public engagement"
  type: article
  url: "https://joss.theoj.org/papers/10.21105/joss.06617"
  volume: 9
title: "Empirical: A scientific software library for research,
  education, and public engagement"

If the repository is not hosted on GitHub, a .cff file can still be uploaded to set your preferred citation. Users will be able to manually copy and paste the citation.

Find more information on .cff files here and here.

editorialbot · 2024-06-02T16:03:38Z

🐘🐘🐘 👉 Toot for this paper 👈 🐘🐘🐘

editorialbot · 2024-06-02T16:03:38Z

🚨🚨🚨 THIS IS NOT A DRILL, YOU HAVE JUST ACCEPTED A PAPER INTO JOSS! 🚨🚨🚨

Here's what you must now do:

Check final PDF and Crossref metadata that was deposited 👉 Creating pull request for 10.21105.joss.06617 joss-papers#5436
Wait five minutes, then verify that the paper DOI resolves https://doi.org/10.21105/joss.06617
If everything looks good, then close this review issue.
Party like you just published a paper! 🎉🌈🦄💃👻🤘

Any issues? Notify your editorial technical team...

danielskatz · 2024-06-02T16:18:56Z

Congratulations to @mmore500 (Matthew Andres Moreno) and co-authors on your publication!!

And thanks to @LTLA and @bramvandijk88 for reviewing, and to @mahfuz05062 for editing!
JOSS depends on volunteers and wouldn't be successful without your work

editorialbot · 2024-06-02T16:18:57Z

🎉🎉🎉 Congratulations on your paper acceptance! 🎉🎉🎉

If you would like to include a link to your paper from your README use the following code snippets:

Markdown:
[![DOI](https://joss.theoj.org/papers/10.21105/joss.06617/status.svg)](https://doi.org/10.21105/joss.06617)

HTML:
<a style="border-width:0" href="https://doi.org/10.21105/joss.06617">
  <img src="https://joss.theoj.org/papers/10.21105/joss.06617/status.svg" alt="DOI badge" >
</a>

reStructuredText:
.. image:: https://joss.theoj.org/papers/10.21105/joss.06617/status.svg
   :target: https://doi.org/10.21105/joss.06617

This is how it will look in your documentation:

We need your help!

The Journal of Open Source Software is a community-run journal and relies upon volunteer effort. If you'd like to support us please consider doing either one (or both) of the the following:

Volunteering to review for us sometime in the future. You can add your name to the reviewer list here: https://reviewers.joss.theoj.org/join
Making a small donation to support our running costs here: https://numfocus.org/donate-to-joss

editorialbot added C++ Makefile review Shell Track: 7 (CSISM) Computer science, Information Science, and Mathematics labels Apr 13, 2024

editorialbot assigned mahfuz05062 Apr 13, 2024

editorialbot mentioned this issue Apr 13, 2024

[PRE REVIEW]: Empirical: A scientific software library for research, education, and public engagement #6151

Closed

This was referenced Apr 17, 2024

Some of the quick start instructions need updating devosoft/Empirical#509

Closed

How smart is Ptr? devosoft/Empirical#510

Closed

Why rely on Jquery? devosoft/Empirical#511

Closed

editorialbot added the recommend-accept Papers recommended for acceptance in JOSS. label Jun 1, 2024

editorialbot added accepted published Papers published in JOSS labels Jun 2, 2024

danielskatz closed this as completed Jun 2, 2024

[REVIEW]: Empirical: A scientific software library for research, education, and public engagement #6617

[REVIEW]: Empirical: A scientific software library for research, education, and public engagement #6617

Comments

editorialbot commented Apr 13, 2024 • edited Loading

Status

Reviewer instructions & questions

Checklists

editorialbot commented Apr 13, 2024

editorialbot commented Apr 13, 2024

editorialbot commented Apr 13, 2024

editorialbot commented Apr 13, 2024

editorialbot commented Apr 13, 2024

editorialbot commented Apr 13, 2024

mahfuz05062 commented Apr 13, 2024

bramvandijk88 commented Apr 15, 2024 • edited Loading

Review checklist for @bramvandijk88

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

bramvandijk88 commented Apr 15, 2024

LTLA commented Apr 17, 2024 • edited Loading

Review checklist for @LTLA

Conflict of interest

Code of Conduct

General checks

Functionality

Documentation

Software paper

LTLA commented Apr 17, 2024 • edited Loading

mmore500 commented Apr 19, 2024

bramvandijk88 commented Apr 22, 2024

bramvandijk88 commented Apr 22, 2024

mmore500 commented Apr 22, 2024 • edited Loading

bramvandijk88 commented Apr 23, 2024

mahfuz05062 commented Apr 27, 2024

mahfuz05062 commented May 4, 2024

LTLA commented May 7, 2024

mmore500 commented May 8, 2024

editorialbot commented Jun 1, 2024

editorialbot commented Jun 1, 2024

danielskatz commented Jun 2, 2024

danielskatz commented Jun 2, 2024

mmore500 commented Jun 2, 2024

danielskatz commented Jun 2, 2024

editorialbot commented Jun 2, 2024

mmore500 commented Jun 2, 2024

editorialbot commented Jun 2, 2024

danielskatz commented Jun 2, 2024

editorialbot commented Jun 2, 2024

mmore500 commented Jun 2, 2024

danielskatz commented Jun 2, 2024

mmore500 commented Jun 2, 2024 • edited Loading

mmore500 commented Jun 2, 2024 • edited Loading

danielskatz commented Jun 2, 2024

editorialbot commented Jun 2, 2024

danielskatz commented Jun 2, 2024

editorialbot commented Jun 2, 2024

danielskatz commented Jun 2, 2024

editorialbot commented Jun 2, 2024

editorialbot commented Jun 2, 2024

editorialbot commented Jun 2, 2024

editorialbot commented Jun 2, 2024

danielskatz commented Jun 2, 2024

editorialbot commented Jun 2, 2024

editorialbot commented Apr 13, 2024 •

edited

Loading

bramvandijk88 commented Apr 15, 2024 •

edited

Loading

LTLA commented Apr 17, 2024 •

edited

Loading

LTLA commented Apr 17, 2024 •

edited

Loading

mmore500 commented Apr 22, 2024 •

edited

Loading

mmore500 commented Jun 2, 2024 •

edited

Loading

mmore500 commented Jun 2, 2024 •

edited

Loading