Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1. score_pairs refactor #333

Merged
merged 22 commits into from
Oct 21, 2021
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 0 additions & 23 deletions doc/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -123,26 +123,3 @@ to the following resources:
Survey <http://dx.doi.org/10.1561/2200000019>`_ (2012)
- **Book:** `Metric Learning
<http://dx.doi.org/10.2200/S00626ED1V01Y201501AIM030>`_ (2015)

.. Methods [TO MOVE TO SUPERVISED/WEAK SECTIONS]
.. =============================================

.. Currently, each metric learning algorithm supports the following methods:

.. - ``fit(...)``, which learns the model.
.. - ``get_mahalanobis_matrix()``, which returns a Mahalanobis matrix
.. - ``get_metric()``, which returns a function that takes as input two 1D
arrays and outputs the learned metric score on these two points
.. :math:`M = L^{\top}L` such that distance between vectors ``x`` and
.. ``y`` can be computed as :math:`\sqrt{\left(x-y\right)M\left(x-y\right)}`.
.. - ``components_from_metric(metric)``, which returns a transformation matrix
.. :math:`L \in \mathbb{R}^{D \times d}`, which can be used to convert a
.. data matrix :math:`X \in \mathbb{R}^{n \times d}` to the
.. :math:`D`-dimensional learned metric space :math:`X L^{\top}`,
.. in which standard Euclidean distances may be used.
.. - ``transform(X)``, which applies the aforementioned transformation.
.. - ``score_pairs(pairs)`` which returns the distance between pairs of
.. points. ``pairs`` should be a 3D array-like of pairs of shape ``(n_pairs,
.. 2, n_features)``, or it can be a 2D array-like of pairs indicators of
.. shape ``(n_pairs, 2)`` (see section :ref:`preprocessor_section` for more
.. details).
21 changes: 17 additions & 4 deletions doc/supervised.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,10 @@ Also, as explained before, our metric learners has learn a distance between
points. You can use this distance in two main ways:

- You can either return the distance between pairs of points using the
`score_pairs` function:
`pair_distance` function:

>>> nca.score_pairs([[[3.5, 3.6], [5.6, 2.4]], [[1.2, 4.2], [2.1, 6.4]]])
array([0.49627072, 3.65287282])
>>> nca.pair_distance([[[3.5, 3.6], [5.6, 2.4]], [[1.2, 4.2], [2.1, 6.4]], [[3.3, 7.8], [10.9, 0.1]]])
array([0.49627072, 3.65287282, 6.06079877])

- Or you can return a function that will return the distance (in the new
space) between two 1D arrays (the coordinates of the points in the original
Expand All @@ -82,6 +82,19 @@ array([0.49627072, 3.65287282])
>>> metric_fun([3.5, 3.6], [5.6, 2.4])
0.4962707194621285

- Alternatively, you can use `pair_score` to return the **score** between
pairs of points, the larger the **score**, the more similar the pair
and vice-versa. For Mahalanobis learners, it is equal to the opposite
mvargas33 marked this conversation as resolved.
Show resolved Hide resolved
of the distance.

>>> score = nca.pair_score([[[3.5, 3.6], [5.6, 2.4]], [[1.2, 4.2], [2.1, 6.4]], [[3.3, 7.8], [10.9, 0.1]]])
>>> score
array([-0.49627072, -3.65287282, -6.06079877])

This is useful because `pair_score` matches the **score** semantic of
scikit-learn's `Classification metrics
<https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics>`_.

.. note::

If the metric learner that you use learns a :ref:`Mahalanobis distance
Expand All @@ -93,7 +106,6 @@ array([0.49627072, 3.65287282])
array([[0.43680409, 0.89169412],
[0.89169412, 1.9542479 ]])

.. TODO: remove the "like it is the case etc..." if it's not the case anymore

Scikit-learn compatibility
--------------------------
Expand All @@ -105,6 +117,7 @@ All supervised algorithms are scikit-learn estimators
scikit-learn model selection routines
(`sklearn.model_selection.cross_val_score`,
`sklearn.model_selection.GridSearchCV`, etc).
You can also use some of the scoring functions from `sklearn.metrics`.

Algorithms
==========
Expand Down
31 changes: 21 additions & 10 deletions doc/weakly_supervised.rst
Original file line number Diff line number Diff line change
Expand Up @@ -160,9 +160,9 @@ Also, as explained before, our metric learner has learned a distance between
points. You can use this distance in two main ways:

- You can either return the distance between pairs of points using the
`score_pairs` function:
`pair_distance` function:

>>> mmc.score_pairs([[[3.5, 3.6, 5.2], [5.6, 2.4, 6.7]],
>>> mmc.pair_distance([[[3.5, 3.6, 5.2], [5.6, 2.4, 6.7]],
... [[1.2, 4.2, 7.7], [2.1, 6.4, 0.9]]])
array([7.27607365, 0.88853014])

Expand All @@ -175,6 +175,19 @@ array([7.27607365, 0.88853014])
>>> metric_fun([3.5, 3.6, 5.2], [5.6, 2.4, 6.7])
7.276073646278203

- Alternatively, you can use `pair_score` to return the **score** between
pairs of points, the larger the **score**, the more similar the pair
and vice-versa. For Mahalanobis learners, it is equal to the opposite
mvargas33 marked this conversation as resolved.
Show resolved Hide resolved
of the distance.

>>> score = mmc.pair_score([[[3.5, 3.6], [5.6, 2.4]], [[1.2, 4.2], [2.1, 6.4]], [[3.3, 7.8], [10.9, 0.1]]])
>>> score
array([-0.49627072, -3.65287282, -6.06079877])

This is useful because `pair_score` matches the **score** semantic of
scikit-learn's `Classification metrics
<https://scikit-learn.org/stable/modules/model_evaluation.html#classification-metrics>`_.

.. note::

If the metric learner that you use learns a :ref:`Mahalanobis distance
Expand All @@ -187,8 +200,6 @@ array([[ 0.58603894, -5.69883982, -1.66614919],
[-5.69883982, 55.41743549, 16.20219519],
[-1.66614919, 16.20219519, 4.73697721]])

.. TODO: remove the "like it is the case etc..." if it's not the case anymore

.. _sklearn_compat_ws:

Prediction and scoring
Expand Down Expand Up @@ -344,8 +355,8 @@ returns the `sklearn.metrics.roc_auc_score` (which is threshold-independent).

.. note::
See :ref:`fit_ws` for more details on metric learners functions that are
not specific to learning on pairs, like `transform`, `score_pairs`,
`get_metric` and `get_mahalanobis_matrix`.
not specific to learning on pairs, like `transform`, `pair_distance`,
mvargas33 marked this conversation as resolved.
Show resolved Hide resolved
`pair_score`, `get_metric` and `get_mahalanobis_matrix`.

Algorithms
----------
Expand Down Expand Up @@ -691,8 +702,8 @@ of triplets that have the right predicted ordering.

.. note::
See :ref:`fit_ws` for more details on metric learners functions that are
not specific to learning on pairs, like `transform`, `score_pairs`,
`get_metric` and `get_mahalanobis_matrix`.
not specific to learning on pairs, like `transform`, `pair_distance`,
mvargas33 marked this conversation as resolved.
Show resolved Hide resolved
`pair_score`, `get_metric` and `get_mahalanobis_matrix`.



Expand Down Expand Up @@ -859,8 +870,8 @@ of quadruplets have the right predicted ordering.

.. note::
See :ref:`fit_ws` for more details on metric learners functions that are
not specific to learning on pairs, like `transform`, `score_pairs`,
`get_metric` and `get_mahalanobis_matrix`.
not specific to learning on pairs, like `transform`, `pair_distance`,
mvargas33 marked this conversation as resolved.
Show resolved Hide resolved
`pair_score`, `get_metric` and `get_mahalanobis_matrix`.



Expand Down
Loading