Getting better at indexing

Week 4: Getting better at indexing 😼

Going on with the first approach to indexing of week 3, the purpose for this week was to reduce the time spent by the indexing task, for which we had to evaluate different code alternatives and choose one that fitted our code. (Alberto Villamayor helped us a bit 👀)

Session

Indexing review with Alberto

06/04/2022

Participants & Content Deliverers

Path Owner	Content Deliverer
Jesús Rebordinos	Alberto Villamayor

 

Key learning points

1. BulkRequest indexing
2. Technology and design recommendations

---

Exercise

With all the mapping and requests created, we were set to index every document in title.basics.tsv. Unfortunately for us, the indexing was very slow, so we had to find some sort of solution 🐌.

Going through the jungle of ElasticSearch Java API documentation, we discovered something that could do the trick: BulkRequest. BulkRequest is an API class that allows sending to ElasticSearch batches of documents to be indexed, dramatically increasing the performance. We struggled a bit with it to keep it working but as always, academy search team overcame the difficulties 😎👌.

After making everything work (:mage:) Alberto reviewed our code and told us about some code improvements that could be made and recommended us the usage of some coding tools such like SonarLint and Insomnia.

Author Daniel Barrientos

• Research regarding options for improving the indexing time
• Introduction of changes (BulkRequest) to improve indexing
• Swagger documentation
• Javadoc

Author Alberto Díez

• Research regarding options for improving the indexing time
• Introduction of changes (BulkRequest) to improve indexing
• Swagger documentation
• Javadoc

Author Marcos Tobías

• Research regarding options for improving the indexing time
• Introduction of changes (BulkRequest) to improve indexing
• Swagger documentation
• Javadoc