Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] Support for Python 3.12 - Missing distutils #176

Open
jessek opened this issue Jun 15, 2024 · 7 comments · May be fixed by #181
Open

[Bug] Support for Python 3.12 - Missing distutils #176

jessek opened this issue Jun 15, 2024 · 7 comments · May be fixed by #181
Labels
bug Something isn't working

Comments

@jessek
Copy link

jessek commented Jun 15, 2024 •
edited
Loading

Expected Behavior

The library should function under Python3.12

Actual Behavior

The library attempts to load the distutils module, which was removed in Python3.12. See https://docs.python.org/3.10/library/distutils.html for background on the deprecation and removal of distutils.

Steps to Reproduce the Problem

  1. Install mitreattack-python with python3 -m pip install mitreattack-python
  2. Run python3 and attempt to load library:
% python3 --version
Python 3.12.4

% python3
Python 3.12.4 (main, Jun  6 2024, 18:26:44) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

>>> from mitreattack.navlayers.core import Layer

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jessek/tmp/lib/python3.12/site-packages/mitreattack/__init__.py", line 3, in <module>
    from .collections import *
  File "/Users/jessek/tmp/lib/python3.12/site-packages/mitreattack/collections/__init__.py", line 3, in <module>
    from .stix_to_collection import *
  File "/Users/jessek/tmp/lib/python3.12/site-packages/mitreattack/collections/stix_to_collection.py", line 9, in <module>
    from stix2elevator.stix_stepper import step_bundle
  File "/Users/jessek/tmp/lib/python3.12/site-packages/stix2elevator/__init__.py", line 12, in <module>
    from stix.core import STIXPackage
  File "/Users/jessek/tmp/lib/python3.12/site-packages/stix/__init__.py", line 5, in <module>
    from .base import (Entity, EntityList, TypedCollection, TypedList,  # noqa
  File "/Users/jessek/tmp/lib/python3.12/site-packages/stix/base.py", line 20, in <module>
    from . import utils
  File "/Users/jessek/tmp/lib/python3.12/site-packages/stix/utils/__init__.py", line 382, in <module>
    from .parser import *  # noqa
    ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jessek/tmp/lib/python3.12/site-packages/stix/utils/parser.py", line 7, in <module>
    import mixbox.parser
  File "/Users/jessek/tmp/lib/python3.12/site-packages/mixbox/parser.py", line 5, in <module>
    from distutils.version import StrictVersion
ModuleNotFoundError: No module named 'distutils'

>>>

Possible Solution

This may be related to #163, and removing support for STIX1 will remove the dependency.
@jessek jessek added the bug Something isn't working label Jun 15, 2024
@stemarks
Copy link

stemarks commented Nov 4, 2024

I am also experiencing this issue, is there any ETA for a fix?

@billfitzgerald
Copy link

Would appreciate feedback on this -

I'm working locally exploring the dataset/tooling in this module and also encountered this issue.

I mitigated it by installing setuptools in the virtual environment where I'm working:

pip install setuptools

The other option was to create a virtual environment using an older version ( lesser than 3.12) of Python.

Is there a preferred way from the perspective of the maintainers to mitigate this?

@stemarks stemarks linked a pull request Nov 12, 2024 that will close this issue
Open
@jondricek
Copy link
Contributor

Maintainer here. Hello!

Personally I would like nothing more than to update this to work on Python 3.12, but my main focus has not been on that functionality yet. We are semi-stuck on Python 3.10 for a completely unrelated issue, but let me try to connect some dots.

  • mitreattack-python (this library) is a dependency of the ATT&CK website (attack.mitre.org) in that we build that website using python, and part of that process uses this library
  • Years ago we began using pyScss for doing some CSS magic with our jinja templates. This third party library broke in Python 3.11. We are tracking the issue in our website repository here: Should support python 3.11 onward attack-website#470
  • Considering that mitreattack-python is independent of that website build process, we just need to make sure that this continues to work for both Python 3.10 as well as the newest version of Python (3.12 right now), which should be do-able, but once again we're a bit time constrained in how we focus our efforts.

The issue is not STIX 1 support (#163). The script that is modified in PR #181 (stix_to_collection.py) uses the stix2elevator library (which is capable of STIX1 > STIX 2 conversion as I understand) to upgrade STIX 2.0 to STIX 2.1. Which is something that we do internally with some other tools/processes (related to https://github.com/center-for-threat-informed-defense/attack-workbench-frontend, but that's a whole separate topic). So because we have a different process, we don't use this functionality ourselves. So I guess that's a roundabout way of saying we can get rid of that script entirely.

I will work on this this week, and @stemarks thank you for the PR in the first place. I hope there are no hard feelings if I end up closing the PR and implement the same fix in a slightly different way as I rip out that and potentially some other content as well that we don't use, and using the cover of a breaking change might take the opportunity to do that in a release this week.

@billfitzgerald
Copy link

@jondricek - this is all super helpful and informative - thank you!

Based on this, my thinking is that the best short to medium term fix is to create a virtual environment that uses either 3.10 or 3.11 - is that accurate?

While installing setuptools "fixes" the issue, my read is that using v3.10 or 3.11 will provide a more consistent experience until this is addressed in a systematic way - would you agree?

@stemarks
Copy link

@jondricek no hard feelings at all, I totally understand. I modified this as we needed it to work with 3.12 so thought I would share what I had done in case it helped you guys time wise, and also in case the solution helps anyone else out they can see my changes 👍

@jondricek
Copy link
Contributor

@billfitzgerald yes - i recommend 3.10 for the time being due to the legacy reason above, but i think it should work in 3.11 as well. Honestly, if installing setuptools in the 3.12 environment fixes it for you, that should be fine - however i realize it is, i'll just say obnoxious from our end to force someone to do that. (But i would totally do that in a testing environment on my own machine if I needed to)

@stemarks thanks for lighting a fire under me to do our own version of PEP 594: Removing Dead Batteries (https://peps.python.org/pep-0594/). And thanks for sharing what you did to make it work!

@billfitzgerald
Copy link

@jondricek - super helpful, thank you!

RE: "obnoxious from our end"

I mean, it's 2024 - if running a one line command is the worst part of my day, it's an awesome day! :)

Thank you for your input here - very much appreciated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix!: error on py3.12 for distutils stemarks/mitreattack-python
4 participants
@billfitzgerald @jessek @jondricek @stemarks