Custome data for bench marking

[praweenkumar]

Hey @erikbern and ANN Benchmarking team!,

Thanks for adding this clean implementation.

i have two query -

1. not a easy way to run test with custom data, i have data in .pkl format.
2. i dont see oracle 23AI added in engine.

Could you provide help with these 2 ?

[maumueller]

Hi @praweenkumar,

1. You will have to write some custom code to transform your dataset to the hdf5 file format that we are using. See the examples in https://github.com/erikbern/ann-benchmarks/blob/main/ann_benchmarks/datasets.py for inspiration on how to do that.

2. Feel free to contribute the dataset yourself as a PR. We are primarily interested in datasets that add some diversity to the existing ones, so a plot with the performance curves of some of our implementations would be appreciated to judge that.

Best,
Martin