[Flaky test] commonlib-unit / xmodule.xmodule.modulestore.tests.test_assetstore::TestMongoAssetMetadataStorage::test_get_all_assets_with_paging

[timmc-edx]

This test fails intermittently and has been deleted according to the Flaky Test Process in #28037. (PR has some proposed fixes/alternatives.)

See docs on how to address flaky tests for why this should be fixed and how to go about it.

Builds where it failed, then passed on the same PR:

• https://build.testeng.edx.org/job/edx-platform-python-pipeline-pr/32033/testReport/junit/xmodule.xmodule.modulestore.tests.test_assetstore/TestMongoAssetMetadataStorage/Run_Tests___commonlib_unit___test_get_all_assets_with_paging_1/
• https://build.testeng.edx.org/job/edx-platform-python-pipeline-pr/32017/testReport/junit/xmodule.xmodule.modulestore.tests.test_assetstore/TestMongoAssetMetadataStorage/Run_Tests___commonlib_unit___test_get_all_assets_with_paging_2/

Failure output (for the two values of MODULESTORE_SETUPS):

Regression
Run Tests / commonlib-unit / xmodule.xmodule.modulestore.tests.test_assetstore.TestMongoAssetMetadataStorage.test_get_all_assets_with_paging_1 (from pytest)

Failing for the past 1 build (Since Failed#32033 )
Took 0.5 sec.
Error Message
AssertionError: assert 'demo.swf' == 'weather_patterns.bmp'
  - weather_patterns.bmp
  + demo.swf
Stacktrace
self = <xmodule.modulestore.tests.test_assetstore.TestMongoAssetMetadataStorage testMethod=test_get_all_assets_with_paging_1>
storebuilder = MongoModulestoreBuilder()

    @ddt.data(*MODULESTORE_SETUPS)
    def test_get_all_assets_with_paging(self, storebuilder):
        """
        Save multiple metadata in each store and retrieve it singularly, as all assets, and after deleting all.
        """
        with storebuilder.build() as (__, store):
            course1 = CourseFactory.create(modulestore=store)
            course2 = CourseFactory.create(modulestore=store)
            self.setup_assets(course1.id, course2.id, store)
    
            expected_sorts_by_2 = (
                (
                    ('displayname', ModuleStoreEnum.SortOrder.ascending),
                    ('code.tgz', 'demo.swf', 'dog.png', 'roman_history.pdf', 'weather_patterns.bmp'),
                    (2, 2, 1)
                ),
                (
                    ('displayname', ModuleStoreEnum.SortOrder.descending),
                    ('weather_patterns.bmp', 'roman_history.pdf', 'dog.png', 'demo.swf', 'code.tgz'),
                    (2, 2, 1)
                ),
                (
                    ('uploadDate', ModuleStoreEnum.SortOrder.ascending),
                    ('code.tgz', 'dog.png', 'roman_history.pdf', 'weather_patterns.bmp', 'demo.swf'),
                    (2, 2, 1)
                ),
                (
                    ('uploadDate', ModuleStoreEnum.SortOrder.descending),
                    ('demo.swf', 'weather_patterns.bmp', 'roman_history.pdf', 'dog.png', 'code.tgz'),
                    (2, 2, 1)
                ),
            )
            # First, with paging across all sorts.
            for sort_test in expected_sorts_by_2:
                for i in range(3):
                    asset_page = store.get_all_asset_metadata(
                        course2.id, 'asset', start=2 * i, maxresults=2, sort=sort_test[0]
                    )
                    num_expected_results = sort_test[2][i]
                    expected_filename = sort_test[1][2 * i]
                    assert len(asset_page) == num_expected_results
                    assert asset_page[0].asset_id.path == expected_filename
                    if num_expected_results == 2:
                        expected_filename = sort_test[1][(2 * i) + 1]
>                       assert asset_page[1].asset_id.path == expected_filename
E                       AssertionError: assert 'demo.swf' == 'weather_patterns.bmp'
E                         - weather_patterns.bmp
E                         + demo.swf

common/lib/xmodule/xmodule/modulestore/tests/test_assetstore.py:620: AssertionError

Regression
Run Tests / commonlib-unit / xmodule.xmodule.modulestore.tests.test_assetstore.TestMongoAssetMetadataStorage.test_get_all_assets_with_paging_2 (from pytest)

Failing for the past 1 build (Since Failed#32017 )
Took 0.5 sec.
Error Message
AssertionError: assert 'demo.swf' == 'weather_patterns.bmp'
  - weather_patterns.bmp
  + demo.swf
Stacktrace
self = <xmodule.modulestore.tests.test_assetstore.TestMongoAssetMetadataStorage testMethod=test_get_all_assets_with_paging_2>
storebuilder = MixedModulestoreBuilder([('draft', MongoModulestoreBuilder())], {})

    @ddt.data(*MODULESTORE_SETUPS)
    def test_get_all_assets_with_paging(self, storebuilder):
        """
        Save multiple metadata in each store and retrieve it singularly, as all assets, and after deleting all.
        """
        with storebuilder.build() as (__, store):
            course1 = CourseFactory.create(modulestore=store)
            course2 = CourseFactory.create(modulestore=store)
            self.setup_assets(course1.id, course2.id, store)
    
            expected_sorts_by_2 = (
                (
                    ('displayname', ModuleStoreEnum.SortOrder.ascending),
                    ('code.tgz', 'demo.swf', 'dog.png', 'roman_history.pdf', 'weather_patterns.bmp'),
                    (2, 2, 1)
                ),
                (
                    ('displayname', ModuleStoreEnum.SortOrder.descending),
                    ('weather_patterns.bmp', 'roman_history.pdf', 'dog.png', 'demo.swf', 'code.tgz'),
                    (2, 2, 1)
                ),
                (
                    ('uploadDate', ModuleStoreEnum.SortOrder.ascending),
                    ('code.tgz', 'dog.png', 'roman_history.pdf', 'weather_patterns.bmp', 'demo.swf'),
                    (2, 2, 1)
                ),
                (
                    ('uploadDate', ModuleStoreEnum.SortOrder.descending),
                    ('demo.swf', 'weather_patterns.bmp', 'roman_history.pdf', 'dog.png', 'code.tgz'),
                    (2, 2, 1)
                ),
            )
            # First, with paging across all sorts.
            for sort_test in expected_sorts_by_2:
                for i in range(3):
                    asset_page = store.get_all_asset_metadata(
                        course2.id, 'asset', start=2 * i, maxresults=2, sort=sort_test[0]
                    )
                    num_expected_results = sort_test[2][i]
                    expected_filename = sort_test[1][2 * i]
                    assert len(asset_page) == num_expected_results
                    assert asset_page[0].asset_id.path == expected_filename
                    if num_expected_results == 2:
                        expected_filename = sort_test[1][(2 * i) + 1]
>                       assert asset_page[1].asset_id.path == expected_filename
E                       AssertionError: assert 'demo.swf' == 'weather_patterns.bmp'
E                         - weather_patterns.bmp
E                         + demo.swf

common/lib/xmodule/xmodule/modulestore/tests/test_assetstore.py:620: AssertionError

(Previously filed as https://2u-internal.atlassian.net/browse/CR-3817.)