Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PYTHON-4786 - Fix UpdateResult.did_upsert TypeError #1878

Merged
merged 8 commits into from
Oct 1, 2024

Conversation

NoahStapp
Copy link
Contributor

No description provided.

blink1073
blink1073 previously approved these changes Sep 27, 2024
Copy link
Member

@blink1073 blink1073 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

assert self.__raw_result is not None
return len(self.__raw_result.get("upserted", {})) > 0
result = self.__raw_result.get("upserted")
return result is not None and len(str(result)) > 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't handle the None case correctly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a few more tests. For both client.bulk_write and collection.update_one() that tests upserting a document with "_id": None.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we expect something besides False if the result is None?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we allow upserting a document's id to be None?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes None is a valid id

@ShaneHarvey
Copy link
Member

ShaneHarvey commented Sep 27, 2024

For example:

>>> res = client.t.t.update_one({"_id": None}, {'$set': {'x': 1}}, upsert=True)
>>> res
UpdateResult({'n': 1, 'upserted': None, 'nModified': 0, 'ok': 1.0, 'updatedExisting': False}, acknowledged=True)
>>> res.upserted_id
>>> res.upserted_id is None
True
>>> res.did_upsert
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/shane/git/mongo-python-driver/pymongo/results.py", line 176, in did_upsert
    return len(self.__raw_result.get("upserted", {})) > 0
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

@NoahStapp
Copy link
Contributor Author

NoahStapp commented Sep 30, 2024

For example:

>>> res = client.t.t.update_one({"_id": None}, {'$set': {'x': 1}}, upsert=True)
>>> res
UpdateResult({'n': 1, 'upserted': None, 'nModified': 0, 'ok': 1.0, 'updatedExisting': False}, acknowledged=True)
>>> res.upserted_id
>>> res.upserted_id is None
True
>>> res.did_upsert
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/shane/git/mongo-python-driver/pymongo/results.py", line 176, in did_upsert
    return len(self.__raw_result.get("upserted", {})) > 0
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

I can't reproduce this locally, I get this result instead:

>>> from pymongo import MongoClient
>>> client = MongoClient()
>>> res = client.t.t.update_one({"_id": None}, {'$set': {'x': 1}}, upsert=True)
>>> res
UpdateResult({'n': 1, 'nModified': 0, 'ok': 1.0, 'updatedExisting': True}, acknowledged=True)
>>> res.upserted_id
>>> res.upserted_id is None
True
>>> res.did_upsert
False

The current change to did_upsert uses this:

        return result is not None and len(str(result)) > 0

@ShaneHarvey
Copy link
Member

ShaneHarvey commented Sep 30, 2024

The problem with your repro attempt is that it did not perform an upsert (see 'updatedExisting': True) which means your collection already had a document with _id:None.

Here's an example using the update command directly:

>>> client.t4.test.delete_many({})
DeleteResult({'n': 1, 'ok': 1.0}, acknowledged=True)
>>> client.t4.command({'update': 'test', 'updates': [{'q': {'_id': None}, 'u': {'$set': {'foo': 'bar'}}, 'upsert': True, 'multi': False}]})
{'n': 1, 'upserted': [{'index': 0, '_id': None}], 'nModified': 0, 'ok': 1.0}
>>> client.t4.command({'update': 'test', 'updates': [{'q': {'_id': None}, 'u': {'$set': {'foo': 'bar'}}, 'upsert': True, 'multi': False}]})
{'n': 1, 'nModified': 0, 'ok': 1.0}

@NoahStapp
Copy link
Contributor Author

The problem with your repro attempt is that it did not perform an upsert (see 'updatedExisting': True) which means your collection already had a document with _id:None.

Good catch--I tried again with a fresh DB and successfully reproduced an incorrect result without an error:

>>> from pymongo import MongoClient
>>> client = MongoClient()
>>> res = client.t.t.update_one({"_id": None}, {'$set': {'x': 1}}, upsert=True)
>>> res
UpdateResult({'n': 1, 'upserted': None, 'nModified': 0, 'ok': 1.0, 'updatedExisting': False}, acknowledged=True)
>>> res.did_upsert
False

I'll add a test to catch this case with a fix.

Copy link
Member

@ShaneHarvey ShaneHarvey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add similar tests for client.bulk_write with verbose=True and coll.update_one.

test/test_results.py Show resolved Hide resolved
@blink1073
Copy link
Member

drivers-pr-bot please backport to v4.9

blink1073
blink1073 previously approved these changes Oct 1, 2024
Copy link
Member

@blink1073 blink1073 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@@ -550,6 +550,47 @@ def test_returns_error_if_auto_encryption_configured(self):
"bulk_write does not currently support automatic encryption", context.exception._message
)

@client_context.require_version_min(8, 0, 0, -24)
@client_context.require_no_serverless
@unittest.skipUnless(_HAVE_PYMONGOCRYPT, "pymongocrypt is not installed")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why require _HAVE_PYMONGOCRYPT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoops, artifact of copy-and-paste.

@@ -171,9 +171,9 @@ def upserted_id(self) -> Any:

@property
def did_upsert(self) -> bool:
"""Whether or not an upsert took place."""
"""Whether an upsert took place."""
Copy link
Member

@ShaneHarvey ShaneHarvey Oct 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add .. versionadded:: 4.9

Copy link
Member

@ShaneHarvey ShaneHarvey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM once the tests pass.

@blink1073 blink1073 merged commit 7848feb into mongodb:master Oct 1, 2024
35 checks passed
@blink1073
Copy link
Member

drivers-pr-bot please backport to v4.9

blink1073 pushed a commit that referenced this pull request Oct 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants