Fix add_real_file() MacOS atime, prepare for release (#190)

* Update atime when CopyRealFile() accesses real file

This is a platform-specific issue for MacOS and BSD

* Deprecate CopyRealFile(), prepare for Release 3.2

CHANGES.md

@@ -1,26 +1,31 @@
 # pyfakefs Release Notes 
 The release versions are PyPi releases.
 
-## Version 3.2 (as yet unreleased)
+## Version 3.2
 
 #### New Features
-  * `io.open`, `os.open`: support for `errors` argument
-  * Added new methods to `fake_filesystem.FakeFilesystem` that make real files 
-    and directories appear within the fake file system: 
-    `add_real_file()`, `add_real_directory()` and `add_real_paths()`.
-    File contents are read from the real file system only when needed ([#170](../../issues/170)).
-  * Added the CHANGES.md release notes to the release manifest
+  * The `errors` argument is supported for `io.open()` and `os.open()`
+  * New methods `add_real_file()`, `add_real_directory()` and `add_real_paths()`
+    make real files and directories appear within the fake file system.
+    File contents are read from the real file system only as needed ([#170](../../issues/170)).
+    See `example_test.py` for a usage example.
+  * Deprecated `TestCase.copyRealFile()` in favor of `add_real_file()`.
+    `copyRealFile()` remains only for backward compatability.  Also, some
+    less-popular argument combinations have been disallowed.
+  * Added this file you are reading, `CHANGES.md`, to the release manifest
 
 #### Infrastructure
-  * `mox3` is no longer required - the relevant part has been integrated into pyfakefs ([#182](../../issues/182))
+  * The `mox3` package is no longer a prerequisite--the portion required by pyfakefs
+    has been integrated into pyfakefs ([#182](../../issues/182))
 
 #### Fixes
- * Corrected handling of byte/unicode paths in several functions ([#187](../../issues/187))
- * `FakeShutilModule.rmtree` failed for directory ending with path separator ([#177](../../issues/177))
- * Case incorrectly handled for added Windows drives 
+ * Corrected the handling of byte/unicode paths in several functions ([#187](../../issues/187))
+ * `FakeShutilModule.rmtree()` failed for directories ending with path separator ([#177](../../issues/177))
+ * Case was incorrectly handled for added Windows drives 
  * `pathlib.glob()` incorrectly handled case under MacOS ([#167](../../issues/167))
  * tox support was broken ([#163](../../issues/163))
- * Rename that only changes case was not possible under Windows ([#160](../../issues/160))
+ * On Windows it was not possible to rename a file when only the case of the file
+   name changed ([#160](../../issues/160))
 
 ## [Version 3.1](https://pypi.python.org/pypi/pyfakefs/3.1)
 
@@ -37,20 +42,20 @@ The release versions are PyPi releases.
 ## [Version 3.0](https://pypi.python.org/pypi/pyfakefs/3.0)
 
 #### New Features
- * support for path-like objects as arguments in fake `os`
+ * Support for path-like objects as arguments in fake `os`
    and `os.path` modules (Python >= 3.6)
- * some changes to make pyfakefs work with Python 3.6 
- * added fake `pathlib` module (Python >= 3.4) ([#29](../../issues/29))
- * support for `os.replace` (Python >= 3.3)
+ * Some changes to make pyfakefs work with Python 3.6 
+ * Added fake `pathlib` module (Python >= 3.4) ([#29](../../issues/29))
+ * Support for `os.replace` (Python >= 3.3)
  * `os.access`, `os.chmod`, `os.chown`, `os.stat`, `os.utime`:
    support for `follow_symlinks` argument (Python >= 3.3)
- * support for `os.scandir` (Python >= 3.5) ([#119](../../issues/119))
- * option to not fake modules named `path` ([#53](../../issues/53))
+ * Support for `os.scandir` (Python >= 3.5) ([#119](../../issues/119))
+ * Option to not fake modules named `path` ([#53](../../issues/53))
  * `glob.glob`, `glob.iglob`: support for `recursive` argument (Python >= 3.5) ([#116](../../issues/116))
- * support for `glob.iglob` ([#59](../../issues/59))
+ * Support for `glob.iglob` ([#59](../../issues/59))
 
 #### Infrastructure
- * added [auto-generated documentation](http://jmcgeheeiv.github.io/pyfakefs/)
+ * Added [auto-generated documentation](http://jmcgeheeiv.github.io/pyfakefs/)
 
 #### Fixes
  * `shutil.move` incorrectly moves directories ([#145](../../issues/145))
@@ -64,57 +69,55 @@ The release versions are PyPi releases.
 #### New Features
  * `io.open`, `os.open`: support for `encoding` argument ([#120](../../issues/120))
  * `os.makedirs`: support for `exist_ok` argument (Python >= 3.2) ([#98](../../issues/98))
- * support for fake `io.open()` ([#70](../../issues/70))
- * support for mount points ([#25](../../issues/25))
- * support for hard links ([#75](../../issues/75))
- * support for float times (mtime, ctime)
+ * Support for fake `io.open()` ([#70](../../issues/70))
+ * Support for mount points ([#25](../../issues/25))
+ * Support for hard links ([#75](../../issues/75))
+ * Support for float times (mtime, ctime)
  * Windows support:
      * support for alternative path separator
      * support for case-insensitive filesystems ([#69](../../issues/69))
      * support for drive letters and UNC paths
- * support for filesystem size ([#86](../../issues/86))
+ * Support for filesystem size ([#86](../../issues/86))
  * `shutil.rmtree`: support for `ignore_errors` and `onerror` arguments ([#72](../../issues/72))
- * support for `os.fsync()` and `os.fdatasync()` ([#73](../../issues/73))
+ * Support for `os.fsync()` and `os.fdatasync()` ([#73](../../issues/73))
  * `os.walk`: Support for `followlinks` argument
 
 #### Fixes
  * `shutil` functions like `make_archive` do not work with pyfakefs ([#104](../../issues/104))
- * file permissions on deletion not correctly handled ([#27](../../issues/27))
+ * File permissions on deletion not correctly handled ([#27](../../issues/27))
  * `shutil.copy` error with bytes contents ([#105](../../issues/105))
  * mtime and ctime not updated on content changes
 
 ## [Version 2.7](https://pypi.python.org/pypi/pyfakefs/2.7)
 
 #### Infrastructure
- * moved repository from GoogleCode to GitHub, merging 3 projects
- * added continuous integration testing with Travis CI
- * added usage documentation in project wiki
- * better support for pypi releases
+ * Moved repository from GoogleCode to GitHub, merging 3 projects
+ * Added continuous integration testing with Travis CI
+ * Added usage documentation in project wiki
+ * Better support for pypi releases
 
 #### New Features
- * added direct unit test support in `fake_filesystem_unittest` 
+ * Added direct unit test support in `fake_filesystem_unittest` 
    (transparently patches all calls to faked implementations)
- * added support for doctests
- * added support for cygwin
- * better support for Python 3
+ * Added support for doctests
+ * Added support for cygwin
+ * Better support for Python 3
 
 #### Fixes
  * `os.utime` fails to traverse symlinks ([#49](../../issues/49))
  * `chown` incorrectly accepts non-integer uid/gid arguments ([#30](../../issues/30))
  * Reading from fake block devices doesn't work ([#24](../../issues/24))
  * `fake_tempfile` is using `AddOpenFile` incorrectly ([#23](../../issues/23))
- * incorrect behavior of `relpath`, `abspath` and `normpath` on Windows.
- * cygwin wasn't treated as Windows ([#37](../../issues/37))
+ * Incorrect behavior of `relpath`, `abspath` and `normpath` on Windows.
+ * Cygwin wasn't treated as Windows ([#37](../../issues/37))
  * Python 3 `open` in binary mode not working ([#32](../../issues/32))
  * `os.remove` doesn't work with relative paths ([#31](../../issues/31))
  * `mkstemp` returns no valid file descriptor ([#19](../../issues/19))
  * `open` methods lack `IOError` for prohibited operations ([#18](../../issues/18))
- * incorrectly resolved relative path ([#3](../../issues/3))
+ * Incorrectly resolved relative path ([#3](../../issues/3))
  * `FakeFileOpen` keyword args do not match the `__builtin__` equivalents ([#5](../../issues/5))
- * relative paths not supported ([#16](../../issues/16), [#17](../../issues/17)))
+ * Relative paths not supported ([#16](../../issues/16), [#17](../../issues/17)))
 
 ## Older Versions
-As there have been three different projects that have been merged together 
-for release 2.7, no older release notes are given.
-The following versions are still available in PyPi:
+There are no release notes for releases 2.6 and below.  The following versions are still available on PyPi:
  * [1.1](https://pypi.python.org/pypi/pyfakefs/1.1), [1.2](https://pypi.python.org/pypi/pyfakefs/1.2), [2.0](https://pypi.python.org/pypi/pyfakefs/2.0), [2.1](https://pypi.python.org/pypi/pyfakefs/2.1), [2.2](https://pypi.python.org/pypi/pyfakefs/2.2), [2.3](https://pypi.python.org/pypi/pyfakefs/2.3) and [2.4](https://pypi.python.org/pypi/pyfakefs/2.4)

example_test.py

@@ -27,11 +27,10 @@
 and allows you to specify the contents or the size of the file.
 """
 
+import io
 import os
 import sys
 
-from pyfakefs.fake_filesystem_unittest import REAL_OPEN
-
 if sys.version_info < (2, 7):
     import unittest2 as unittest
 else:
@@ -65,6 +64,11 @@ def setUp(self):
           :py:class:`pyfakefs.mox3_stubout.StubOutForTesting`.  Use this if you need to
           define additional stubs.
         """
+
+        # This is before setUpPyfakefs(), so still using the real file system
+        with io.open(__file__, 'rb') as f:
+            self.real_contents = f.read()
+
         self.setUpPyfakefs()
 
     def tearDown(self):
@@ -146,12 +150,11 @@ def test_scandir(self):
 
     def test_real_file_access(self):
         """Test `example.file_contents()` for a real file after adding it using `add_real_file()`."""
-        real_file = __file__
-        with REAL_OPEN(real_file, 'rb') as f:
-            real_contents = f.read()
-        self.assertRaises(IOError, example.file_contents, real_file)
-        self.fs.add_real_file(real_file)
-        self.assertEqual(example.file_contents(real_file), real_contents)
+        filename = __file__
+        with self.assertRaises(IOError):
+            example.file_contents(filename)
+        self.fs.add_real_file(filename)
+        self.assertEqual(example.file_contents(filename), self.real_contents)
 
 
 if __name__ == "__main__":

fake_filesystem_unittest_test.py

@@ -191,22 +191,20 @@ def test_own_path_module(self):
 
 @unittest.skipIf(sys.version_info < (2, 7), "No byte strings in Python 2.6")
 class TestCopyOrAddRealFile(TestPyfakefsUnittestBase):
-    """Tests the `fake_filesystem_unittest.TestCase.copyRealFile()` method."""
+    """Tests the `fake_filesystem_unittest.TestCase.copyRealFile()` method.
+    Note that `copyRealFile()` is deprecated in favor of `FakeFilesystem.add_real_file()`.
+    """
     with open(__file__) as f:
         real_string_contents = f.read()
     with open(__file__, 'rb') as f:
         real_byte_contents = f.read()
-    # It is essential to do os.stat() after opening the real file, not before.
-    # This is because opening the file on MacOS and BSD updates access time
-    # st_atime.  Windows offers the option to enable this behavior as well.
     real_stat = os.stat(__file__)
 
     def testCopyRealFile(self):
-        '''Copy a real file to the fake file system'''
+        '''Typical usage of deprecated copyRealFile()'''
         # Use this file as the file to be copied to the fake file system
         real_file_path = __file__
-        fake_file_path = 'foo/bar/baz'
-        fake_file = self.copyRealFile(real_file_path, fake_file_path)
+        fake_file = self.copyRealFile(real_file_path)
 
         self.assertTrue('class TestCopyRealFile(TestPyfakefsUnittestBase)' in self.real_string_contents,
                         'Verify real file string contents')
@@ -216,25 +214,29 @@ def testCopyRealFile(self):
         # note that real_string_contents may differ to fake_file.contents due to newline conversions in open()
         self.assertEqual(fake_file.byte_contents, self.real_byte_contents)
 
-        self.assertEqual(fake_file.st_mode, self.real_stat.st_mode)
+        self.assertEqual(oct(fake_file.st_mode), oct(self.real_stat.st_mode))
         self.assertEqual(fake_file.st_size, self.real_stat.st_size)
         self.assertEqual(fake_file.st_ctime, self.real_stat.st_ctime)
-        self.assertEqual(fake_file.st_atime, self.real_stat.st_atime)
+        self.assertGreaterEqual(fake_file.st_atime, self.real_stat.st_atime)
+        self.assertLess(fake_file.st_atime, self.real_stat.st_atime + 10)
         self.assertEqual(fake_file.st_mtime, self.real_stat.st_mtime)
         self.assertEqual(fake_file.st_uid, self.real_stat.st_uid)
         self.assertEqual(fake_file.st_gid, self.real_stat.st_gid)
 
-        fake_file_path = '/nonexistent/directory/file'
-        with self.assertRaises(IOError):
-            self.copyRealFile(real_file_path, fake_file_path,
-                              create_missing_dirs=False)
-
-    def testCopyRealFileNoDestination(self):
+    def testCopyRealFileDeprecatedArguments(self):
+        '''Deprecated copyRealFile() arguments'''
         real_file_path = __file__
         self.assertFalse(self.fs.Exists(real_file_path))
-        self.copyRealFile(real_file_path)
+        # Specify redundant fake file path
+        self.copyRealFile(real_file_path, real_file_path)
         self.assertTrue(self.fs.Exists(real_file_path))
 
+        # Test deprecated argument values
+        with self.assertRaises(ValueError):
+            self.copyRealFile(real_file_path, '/different/filename')
+        with self.assertRaises(ValueError):
+            self.copyRealFile(real_file_path, create_missing_dirs=False)
+
     def testAddRealFile(self):
         '''Add a real file to the fake file system to be read on demand'''
 

pyfakefs/fake_filesystem.py

@@ -458,6 +458,8 @@ def byte_contents(self):
             self.contents_read = True
             with io.open(self.file_path, 'rb') as f:
                 self._byte_contents = f.read()
+        # On MacOS and BSD, the above io.open() updates atime on the real file
+        self.st_atime = os.stat(self.file_path).st_atime
         return self._byte_contents
 
     def IsLargeFile(self):
@@ -1772,29 +1774,39 @@ def CreateFile(self, file_path, st_mode=stat.S_IFREG | PERM_DEF_FILE,
             file_path, st_mode, contents, st_size, create_missing_dirs, apply_umask, encoding, errors)
 
     def add_real_file(self, file_path, read_only=True):
-        """Create file_path, including all the parent directories along the way, for a file
-        existing in the real file system without reading the contents, which will be read on demand.
+        """Create file_path, including all the parent directories along the way, for an existing
+        real file.  The contents of the real file are read only on demand.
         New in pyfakefs 3.2.
 
         Args:
-            file_path: path to the existing file.
-            read_only: if set, the file is treated as read-only, e.g. a write access raises an exception;
-                otherwise, writing to the file changes the fake file only as usually.
+            file_path: Path to an existing file in the real file system
+            read_only: If `True` (the default), writing to the fake file
+                raises an exception.  Otherwise, writing to the file changes
+                the fake file only.
 
         Returns:
             the newly created FakeFile object.
 
         Raises:
             OSError: if the file does not exist in the real file system.
             IOError: if the file already exists in the fake file system.
+
+        .. note:: On MacOS and BSD, accessing the fake file's contents will update \
+                  both the real and fake files' `atime.` (access time).  In this \
+                  particular case, `add_real_file()` violates the rule that `pyfakefs` \
+                  must not modify the real file system. \
+                  \
+                  Further, Windows offers the option to enable atime, and older \
+                  versions of Linux may also modify atime.
         """
         return self._CreateFile(file_path,
                                 read_from_real_fs=True,
                                 read_only=read_only)
 
     def add_real_directory(self, dir_path, read_only=True, lazy_read=True):
-        """Create fake directory for the existing directory at path, and entries for all contained
-        files in the real file system.
+        """Create a fake directory corresponding to the real directory at the specified
+        path.  Add entries in the fake directory corresponding to the entries in the
+        real directory.
         New in pyfakefs 3.2.
 
         Args:
@@ -1835,8 +1847,9 @@ def add_real_directory(self, dir_path, read_only=True, lazy_read=True):
         return new_dir
 
     def add_real_paths(self, path_list, read_only=True, lazy_dir_read=True):
-        """Convenience method to add several files and directories from the real file system
-        in the fake file system. See `add_real_file()` and `add_real_directory()`.
+        """This convenience method adds multiple files and/or directories from the
+        real file system to the fake file system. See `add_real_file()` and
+        `add_real_directory()`.
         New in pyfakefs 3.2.
 
         Args:
@@ -1861,7 +1874,8 @@ def _CreateFile(self, file_path, st_mode=stat.S_IFREG | PERM_DEF_FILE,
                     contents='', st_size=None, create_missing_dirs=True,
                     apply_umask=False, encoding=None, errors=None,
                     read_from_real_fs=False, read_only=True):
-        """Internal fake file creation, supports both normal fake files and fake files from real files.
+        """Internal fake file creator that supports both normal fake files and fake
+        files based on real files.
 
         Args:
             file_path: path to the file to create.