Improper handling of bytes/str in split and join throws TypeError with subprocess.call()

[augustjd]

I'm using pyfakefs with some test cases that involve spinning up subprocesses, but unfortunately, the subprocess library doesn't appear to be supported yet, because of path resolution for the executables.

This may be harder than it looks, but on a cursory glance it seems that the only hangup is that os.path functions like split and join support both bytes and str to be passed as arguments, and return the corresponding type. The subprocess library internally calls os functions with bytes arguments, but this causes a TypeError because os.path.split() and the like in pyfakefs only support strings.

Here's a minimal example:

import subprocess
from pyfakefs import fake_filesystem_unittest

class TestSubprocess(fake_filesystem_unittest.TestCase):
    def setUp(self):
        self.setUpPyfakefs()
        self.copyRealFile("/bin/nc", "/bin/nc")

    def test_run_nc(self):
        subprocess.call("/bin/nc")  # throws TypeError

Running this throws:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/subprocess.py", line 1245, in _execute_child
    if os.path.dirname(executable):
  File "./env/lib/python3.6/site-packages/pyfakefs/fake_filesystem.py", line 2416, in dirname
    return self.split(path)[0]
  File "./env/lib/python3.6/site-packages/pyfakefs/fake_filesystem.py", line 2314, in split
    return self.filesystem.SplitPath(path)
  File "./env/lib/python3.6/site-packages/pyfakefs/fake_filesystem.py", line 970, in SplitPath
    path_components = path.split(self.path_separator)
TypeError: a bytes-like object is required, not 'str'

subprocess would be a great additional library to add to the stable of standard libs that work with pyfakefs.

[mrbean-bremen]

You are right - there may be quite some places where this is not supported correctly, as the initial implementation has been done in Python 2, and the Python 3 support is not complete.
I will have a look at this, at least at the subprocess call.

[mrbean-bremen]

Unfortunately, I cannot reproduce this concrete problem on my Windows machine, as this happens in the posix-specific part of subprocess, but I will look at it tomorrow anyway, if I find the time.
Thanks for reporting this!

[mrbean-bremen]

Ok, just looked into this - I did not get it right yesterday (was a little tired). Unfortunately, you seem to be out of luck here. There are indeed some str/byte issues that we may fix (I will have a look at these later), but the actual excecution code calls OS functionality that cannot be changed to use the fake file system, as far as I can see. Sorry for that...
Edit: My own approach would be to mock subprocess to be able to check the calling arguments and not try to run an executable in the test at all.

[augustjd]

Bummer. Thanks for looking into this.