Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Triplet Margin Loss] Issue 1118 #1120

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

cvnad1
Copy link

@cvnad1 cvnad1 commented Oct 26, 2024

@vroulet Hi Vincent, Added code and tests for the Triplet Margin Loss Function. Kindly review the code and please do comment in case of any changes.

@cvnad1
Copy link
Author

cvnad1 commented Oct 30, 2024

@vroulet May I know if there's anything that needs to be changed?

Copy link
Collaborator

@vroulet vroulet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @cvnad1 for doing this! Sorry for the delay. Here are some comments

anchor: The anchor embeddings. Shape: [batch_size, feature_dim].
positive: The positive embeddings. Shape: [batch_size, feature_dim].
negative: The negative embeddings. Shape: [batch_size, feature_dim].
margin: The margin value. Default: 1.0.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to put the default values since they are given in the signature.

by V. Balntas et al. Default: False.
reduction: Specifies the reduction to apply to the output:
'none' | 'mean' | 'sum'. Default: 'mean'.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add reference

margin: The margin value. Default: 1.0.
p: The norm degree for pairwise distance. Default: 2.
eps: Small epsilon value to avoid numerical issues. Default: 1e-6.
swap: Use the distance swap optimization from "Learning shallow
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use rst formatting for references (see e.g. the docstring of Adam)

swap: bool = False,
reduction: str = 'mean',
) -> chex.Array:
"""Triplet margin loss function.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add an example (doctest)

@@ -53,5 +53,41 @@ def test_batched(self):
)


class TripletMarginLossTest(chex.TestCase):

def setUp(self):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid using numerical values as expected returns.
They may fail depending on the backend for example.
You may consider simple test cases with a "handmade" function (see e.g. the lbfgs tests). You can check for specific inputs (like zeros or ones).

You may also add a test for some specific behaviors (like using swap here).

Also you should test this function under jit/vmap etc... (see the chex.all_variant utility in some other tests).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants