PyPI petl not up to date (1.7.4 vs 1.7.5)

[arturponinski]

I've noticed that the latest available version of petl on PyPi is 1.7.4: https://pypi.org/project/petl/
However, here on github the version is 1.7.5 release on Oct 15, 2021. Is there any particular reason, this has not been released? Would you be able to release the latest version at your earliest convenience?
Thanks in advance.

[juarezr]

Hi @arturponinski,

This happened because the release pipeline was using Travis-ci.org free service. But it was discontinued some time ago.

If I had the credentials to PyPI account, I would already have fixed with a new pipeline in github.com actions.

I sent a message to @alimanfoo but no he hasn't answered yet. Maybe he missed.

[alimanfoo]

Hi @juarezr, so sorry for slow reply, if you could send me your PyPI user name I'd be happy to add you as a maintainer.

[juarezr]

Hi @alimanfoo,

Glad to see you back here.

After travis-ci.org ceased working, all petl CD pipelines stopped working:

• Github release is at v1.7.5
• PyPI is at v1.7.4
• readthedocs is at v1.68
• Conda Forge is at v1.7.4

I see that we need to step in to fix all of them. I hope that with your help we can get all working.

Meanwhile:

• I have been creating an alternative CI using Github Actions.
• So far it has been testing the same set of tests as travis-ci.org did plus:
  • It runs tests in Windows like Appveyor does.
  • It runs additional tests for covering petl database integration using docker.
  • It runs additional tests for covering petl remote filesystem integration with fsspec using docker.
  • It runs additional tests for covering MacOS.
• Coverage stays at the same level as before.

Also, I would like to hear your opinion about:

• Migrating the deployment process for releases of PyPI, Conda Forge and readthedocs.org to Github Actions
• Guidelines for petl evolution, because we have some worthy contributions like:
  • @arturponinski migration from nosetests to pytest Migrate to pytest #584 ready to landing
  • POC for adding typing to petl functions on [POC] added typing to materialize (breaks py27 tests) #559
  • POC for dropping support for EOL Python 2.7 on Drop support for EOL Python 2.7 #563
• Onboard of new maintainers, as I'm busier than before. (@arturponinski looks like a promising volunteer if he would like 😄 )

Accounts

• PyPI username:
  • juarezr
• readthedocs.org:
  • juarezr
• Conda Forge
  • via petl-feedstock

[arturponinski]

I would be more than happy to be part of petl as a maintainer and push petl it to 2.0 according to roadmap posted by @alimanfoo here: https://groups.google.com/g/python-etl/c/0EOmiTOdrUE/m/c1vTuMGjDAAJ

[juarezr]

I would be more than happy to be part of petl as a maintainer and push petl it to 2.0 according to roadmap posted by @alimanfoo here: https://groups.google.com/g/python-etl/c/0EOmiTOdrUE/m/c1vTuMGjDAAJ

• Nice to hear that!
• I think that after your PR Migrate to pytest #584 we are getting pretty close to 2.0
• A killer feature to make the most of the compatibility break introduced by v2.0 would be Typing vs Python2.7 support #560 ( [POC] added typing to materialize (breaks py27 tests) #559 ) Resolves issue #179 (https://github.com/alimanfoo/petl/issues/179): sear... #274 😄
• Also @hugovk already explored the steps to drop py27 support in Drop support for EOL Python 2.7 #563

[alimanfoo]

Hi @juarezr,

• I have been creating an alternative CI using Github Actions.

• So far it has been testing the same set of tests as travis-ci.org did plus:

  • It runs tests in Windows like Appveyor does.
  • It runs additional tests for covering petl database integration using docker.
  • It runs additional tests for covering petl remote filesystem integration with fsspec using docker.
  • It runs additional tests for covering MacOS.

• Coverage stays at the same level as before.

All sounds great.

Also, I would like to hear your opinion about:

• Migrating the deployment process for releases of PyPI, Conda Forge and readthedocs.org to Github Actions

SGTM, whatever you think would be most convenient.

• Guidelines for petl evolution, because we have some worthy contributions like:

  • @arturponinski migration from nosetests to pytest Migrate to pytest #584 ready to landing
  • POC for adding typing to petl functions on [POC] added typing to materialize (breaks py27 tests) #559
  • POC for dropping support for EOL Python 2.7 on Drop support for EOL Python 2.7 #563

• Onboard of new maintainers, as I'm busier than before. (@arturponinski looks like a promising volunteer if he would like smile )

FWIW very happy for you to grant maintainer status to contributors as you see fit.

@arturponinski it's a pleasure to meet you!

Accounts

• PyPI username:

  • juarezr

• readthedocs.org:

  • juarezr

• Conda Forge

  • via petl-feedstock

I've added you to PyPI and RTFD, I think you're already on the conda-forge feedstock.

Please let me know if there's anything else I can help with 🌸

[juarezr]

Hi @alimanfoo,

I going to check everything and return if there is any other problem.
Thanks for your time.

Hi @ arturponinski,

I'll add you as maintainer of this repository also. Welcome!

[juarezr]

Update:

• Migrated release from discontinued travis-ci.org to a brand new CD using Github Action in cd: publish releases to PyPI with GA #593
• Released version v1.7.6 to PyPI
• Fixed readthedocs.org build

[juarezr]

TODO:

• Waiting for petl-feedstock trigger a new release in conda forge
• Remove hooks of:
  • travis-ci.org
  • AppVeyor
  • Both testing and release are now on Github Actions
• Update the badges on readme.md
• Cleanup outdated files and docs on repository

[arturponinski]

That's amazing news! Impressive work @juarezr We would need to release 1.7.5 to pypi as well (to be aligned with github releases)

[juarezr]

• After the release on PyPI was fixed, conda-forge/petl-feedstock detected and triggered its PR, as usual. 🎉
• So soon we may have v1.7.6 on Conda Forge too.
• Releasing v1.7.5 will demand manual intervention. But right now I'm a little busy for making this hanppen.

[juarezr]

OMG!

The release of conda-forge/petl-feedstock failed.

import: 'petl'
+ nosetests -v petl
Failure: ModuleNotFoundError (No module named 'pytest') ... ERROR

[arturponinski]

Still some nosetests leftovers https://github.com/conda-forge/petl-feedstock/blob/master/recipe/meta.yaml

[juarezr]

Investigating failure:

export SRC_DIR=/home/conda/feedstock_root/build_artifacts/petl_1643806652129/test_tmp
import: 'petl'
import: 'petl'
+ pytest --verbose petl
============================= test session starts ==============================
platform linux -- Python 3.10.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- $PREFIX/bin/python
cachedir: .pytest_cache
rootdir: $SRC_DIR
collecting ... collected 0 items

============================ no tests ran in 0.00s =============================
ERROR: file or directory not found: petl

Tests failed for petl-1.7.6-pyh6c4a22f_0.tar.bz2 - moving package to /home/conda/feedstock_root/build_artifacts/broken

[arturponinski]

@juarezr wouldn't this: https://github.com/petl-developers/petl/pull/596/files resolve the issue of missing 1.7.5 in PyPI ?
It would be beneficial in case the action failed for whatever reason.

[juarezr]

• It would be very interesting to have such a procedure
• I didn't look how it works yet
• I don't know if it works for conda-forge

[juarezr]

• Release v1.7.7 available in Conda Forge
• Fixed petl-feedstock on conda-forge build is working after pytest migration conda-forge/petl-feedstock#27
• Next release should work transparently by:
  • Tagging a SemVer version on this github repository (v1.7.8)
• Releases v1.7.5 and v1.7.6 missing in Conda Forge (v1.7.5 also missing on PyPI)

[juarezr]

Update:

• v1.7.5 uploaded to PyPI project repository
• Waiting for regro-cf-autotick-bot to trigger the conda-build build/import on conda-forge/petl-feedstock and uploading to Conda Forge