Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch provider: only highlight text fields #189

Merged
merged 3 commits into from
Jan 30, 2024

Conversation

stellarhoof
Copy link
Member

@stellarhoof stellarhoof commented Jan 29, 2024

Remove subFields configuration from the schema. Instead only send text fields for highlighting. This both simplifies the API and reduces payload to elastic, as well as fixing an issue where top-level keyword fields were being highlighted when they should not be.

Also improve performance (from ~70ms to ~10ms on a page size of 100 with large arrays each) when handling elastic responses.

@stellarhoof stellarhoof self-assigned this Jan 29, 2024
Copy link

changeset-bot bot commented Jan 29, 2024

🦋 Changeset detected

Latest commit: 77fce36

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
contexture-elasticsearch Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@stellarhoof stellarhoof marked this pull request as draft January 29, 2024 21:06
Copy link
Contributor

github-actions bot commented Jan 29, 2024

Messages
📖 You reduced the total lines of code! Awesome! 👍

Generated by 🚫 dangerJS against 77fce36

@stellarhoof stellarhoof force-pushed the feature/highlight-text-fields branch from 6841b8f to 2f4e503 Compare January 29, 2024 21:08
@stellarhoof stellarhoof marked this pull request as ready for review January 29, 2024 21:16
Copy link
Contributor

Coverage after merging feature/highlight-text-fields into main will be

90.59%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
packages/client/src
   exampleTypes.js93.78%85.71%40%97.80%109, 123, 126–133
   index.js98.96%93.85%100%100%211, 213, 284, 73
   lens.js100%100%100%100%
   listeners.js100%100%100%100%
   mockService.js95.65%83.33%100%100%22, 30
   node.js96.75%96%87.50%97.78%72, 76, 76
   reactors.js99.12%100%90.91%100%
   serialize.js100%100%100%100%
   subquery.js97.59%88.89%100%98.55%66, 66
   traversals.js100%100%100%100%
   types.js92.59%83.33%100%95.24%12–13
   validation.js78.79%83.33%100%77.78%20–26
packages/client/src/actions
   index.js100%100%100%100%
   wrap.js96.67%85.71%100%97.92%41, 41
packages/client/src/exampleTypes
   pivot.js82.87%74.29%87.50%83.70%10–13, 138–139, 14, 140–149, 15, 150–161, 167–168, 21–22, 227, 23, 243, 257–263, 48, 53–57, 8–9, 9
packages/client/src/util
   futil.js67.31%100%57.14%65.85%18–19, 24–28, 31, 36–41
   tree.js100%100%100%100%
packages/export/src
   csv.js92.73%80%50%95.83%25, 45–46
   index.js0%100%0%0%1–5
   utils.js100%100%100%100%
packages/export/src/nodes
   index.js0%100%0%0%1–5
   pivot.js98.43%82.35%100%100%52, 56, 89
   results.js100%100%100%100%
   terms_stats.js100%100%100%100%
packages/provider-elasticsearch/src
   index.js81.82%52%80%88.24%104–112, 12–13, 29, 53, 53–57, 62, 65–66, 73, 75, 87, 91, 91
   schema.js92.52%92.86%80%93.18%52–55, 85–87
   types.js100%100%100%100%
packages/provider-elasticsearch/src/example-types
   index.js100%100%100%100%
   schemaMapping.js100%100%100%100%
   testUtils.js98.88%92.31%100%100%12
packages/provider-elasticsearch/src/example-types/filters
   bool.js100%100%100%100%
   date.js84.62%66.67%50%88.24%16, 16, 6–8
   dateRangeFacet.js69.79%100%75%66.67%56–83
   exists.js100%100%100%100%
   facet.js92.80%86.67%66.67%94.39%39, 64, 64–69
   geo.js100%100%100%100%
   number.js52.05%100%50%50%22–29, 33–56
   query.js100%100%100%100%
   tagsQuery.js86.47%90.63%85.71%85.71%103–120, 132, 141–145, 151–153
   tagsText.js100%100%100%100%
   text.js95.73%86.67%100%98.82%11, 22, 40, 71, 71
packages/provider-elasticsearch/src/example-types/legacy
   cardinality.js93.33%100%50%100%
   dateHistogram.js95.45%100%50%100%
   esTwoLevelAggregation.js92.11%86.67%66.67%93.75%34–36, 83–87
   matchStats.js97.06%100%50%100%
   rangeStats.js100%100%100%100%
   smartIntervalHistogram.js97.44%100%50%100%
   statistical.js90.91%100%50%100%
   terms_stats.js97.73%100%66.67%100%
packages/provider-elasticsearch/src/example-types/metricGroups
   dateIntervalGroupStats.js100%100%100%100%
   dateRangesGroupStats.js98.28%100%75%100%
   fieldValuePartitionGroupStats.js87.76%100%40%92.31%34–36
   fieldValuesDelta.js97.44%100%66.67%100%
   fieldValuesGroupStats.js99.07%100%83.33%100%
   groupStatUtils.js86.05%100%75%85.71%20–24
   numberIntervalGroupStats.js100%100%100%100%
   numberRangesGroupStats.js97.87%100%66.67%100%
   percentilesGroupStats.js81.13%75%33.33%84.78%13–15, 21–23, 40–41
   pivot.js85.62%89.89%71.43%85.65%107, 151–152, 159, 166, 408–410, 460–470, 473, 478, 480, 483, 486, 534–535, 563–619, 75–81, 84–91
   stats.js86.96%100%50%94.12%16
   tagsQueryGroupStats.js100%100%100%100%
packages/provider-elasticsearch/src/example-types/metricGroups/pivotData
   columnResponse.js100%100%100%100%
   columnResult.js100%100%100%100%
   pivotResponse.js100%100%100%100%
   pivotResponseWithFilteredFieldValueGroup.js100%100%100%100%
packages/provider-elasticsearch/src/example-types/results
   index.js18.60%100%0%19.51%10–40, 8–9
packages/provider-elasticsearch/src/example-types/results/highlighting
   request.js98.80%97.87%100%98.97%40–42
   response.js100%100%100%100%
   search.js32.20%100%0%32.76%20–58
   testSchema.js100%100%100%100%
   util.js100%100%100%100%
packages/provider-elasticsearch/src/schema-data
   aliases.js100%100%100%100%
   mapping-with-non-objects.js100%100%100%100%
   mapping-with-types.js100%100%100%100%
   mapping-without-types.js100%100%100%100%
   schema-with-types.js100%100%100%100%
   schema-without-types.js100%100%100%100%
packages/provider-elasticsearch/src/utils
   dateUtil.js88.37%85.71%60%90.54%13–15, 18, 64–67
   elasticDSL.js100%100%100%100%
   fields.js100%100%100%100%
   futil.js99.39%100%92.86%100%
   keywordGenerations.js100%100%100%100%
   luceneQueryUtils.js100%100%100%100%
   regex.js100%100%100%100%
   results.js100%100%100%100%
   smartInterval.js83.33%33.33%100%92.86%11, 11, 9
packages/provider-mongo/src
   index.js93.33%71.43%83.33%96.77%10, 33, 45, 9
   types.js100%100%100%100%
packages/provider-mongo/src/example-types
   bool.js100%100%100%100%
   date.js79.34%90.91%55.56%80.20%54–58, 61–69, 7, 70–71, 8–9, 90, 90
   dateHistogram.js94.20%62.50%100%98.31%19, 19, 25, 25
   exists.js100%100%100%100%
   facet.js98.95%95.74%100%99.56%123–124, 124
   index.js100%100%100%100%
   mongoId.js100%100%100%100%
   number.js100%100%100%100%
   results.js97.74%92.31%100%99.01%180, 59, 61, 64–66
   statistical.js100%100%100%100%
   tagsText.js100%100%100%100%
   termsStats.js100%100%100%100%
   text.js100%100%100%100%
packages/react/src
   FilterAdder.js0%100%0%0%1, 10–19, 2, 20–29, 3, 30–39, 4, 40–49, 5, 50–52, 6–9
   FilterAdder.stories.js0%100%0%0%1, 10–19, 2, 20–23, 3–9
   FilterButtonList.js0%100%0%0%1, 10, 100–109, 11, 110–119, 12, 120–129, 13, 130–139, 14, 140–149, 15, 150–159, 16, 160–168, 17–19, 2, 20–29, 3, 30–39, 4, 40–49, 5, 50–59, 6, 60–69, 7, 70–79, 8, 80–89, 9, 90–99
   FilterButtonList.stories.js0%100%0%0%1, 10–19, 2, 20–29, 3, 30–39, 4, 40–49, 5, 50–56, 6–9
   FilterList.js0%100%0%0%1, 10, 100–109, 11, 110–119, 12, 120–129, 13, 130–139, 14, 140–149, 15, 150–159, 16, 160–169, 17,

@stellarhoof
Copy link
Member Author

I'm merging this PR as-is because it's a fix needed in the main product. I'll address any feedback in a follow-up.

@stellarhoof stellarhoof merged commit 719facb into main Jan 30, 2024
4 checks passed
@stellarhoof stellarhoof deleted the feature/highlight-text-fields branch January 30, 2024 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant