Batch k-NN

[krstopro]

I am honestly not sure where should this be implemented. Right now I added linear_search (a better name might be needed, e.g. brute_force_search) inside Scholar.Neighbors.Utils. Like this it can be used inside other modules such as Scholar.Neighbors.KNearestNeighbors or dimensionality reduction algorithms (t-SNE, Trimap, PacMAP).

The function itself should be documented and tested. More distances are needed. I am just submitting a draft to get some feedback.

Closes #239.

[josevalim]

It looks good to me. For now it is a private module, so we don't have to sweat too much about the name. Does this function have any relationship with find_neighbour? If so, maybe we call one linear_search and the other batch_linear_search?

[krstopro]

I would rename find_neighbors to linear_search_with_candidates as that is what it really is: it performs linear search, but only on indices specified with candidates tensor.

scholar/lib/scholar/neighbors/utils.ex

Line 13 in 0d1bcc1

|> Nx.subtract(Nx.take(data, candidate_indices))

[msluszniak]

There is a function for handling multiple distances that are in t-SNE, NNDescent, and Trimap modules. I guess it might be worth to move it to Scholar.Shared module.

[msluszniak]

These lines are common for the main part of the search in the while loop, maybe it is worth moving this into a separate function, but it's up to you since it's just 3 lines

[krstopro]

I'm considering it. The code still needs some polishing.
I am also thinking of abstracting the whole thing as kind of map over batches. It might be useful for Task 1 of #246 as well.

[msluszniak]

Suggested change

	i = 0
	i = Nx.u64(0)

[krstopro]

This part might also need to be moved inside Scholar.Shared as a function that takes a tensor and batch size and returns {batches, leftover} where batches is a tensor of shape {num_batches, batch_size, dim} and leftover is a tensor of shape {leftover_size, dim}.
Might be relevant for #246, task 1.

[msluszniak]

Yes, definitely. In Python, there is even a function divmod that do that

[josevalim]

linear_search and linear_candidate_search (or linear_neighbour_search) are fine to me! As I said, it is your call, the whole API is private, so focus on what makes the code cleaner :)

[krstopro]

Closing this in favor of #257.