Update bool operator syntax for Elasticsearch simple_query_string and excludes

[patphongs]

Summary (required)

• Resolves Update bool operator syntax for Elasticsearch simple_query_string and excludes #6360

This updates the query string syntax that is used on the web UI as well as what is submitted in the API query string. Refer to the ticket for the syntax.

Required reviewers

1 Developer
1 UX

Impacted areas of the application

General components of the application that this PR will affect:

• More keyword search for all legal documents (MURs, ADRs, AFs, AOs, Statutes)
  • https://fec-feature-cms.app.cloud.gov/data/legal/search/murs/
  • https://fec-feature-cms.app.cloud.gov/data/legal/search/adrs/
  • https://fec-feature-cms.app.cloud.gov/data/legal/search/admin_fines/
  • https://fec-feature-cms.app.cloud.gov/data/legal/search/statutes/
  • https://fec-feature-cms.app.cloud.gov/data/legal/search/advisory-opinions/

Related PRs

Related PRs against other branches:

branch	PR
feature/add_case_respondents_and_exclude	link

How to test

• checkout this branch
• npm run build
• ./manage.py runserver
• Go to each of the legal searches and perform boolean search using the "more keyword options" box or constructing your own query.
  • http://127.0.0.1:8000/data/legal/search/murs/
  • http://127.0.0.1:8000/data/legal/search/adrs/
  • http://127.0.0.1:8000/data/legal/search/admin_fines/
  • http://127.0.0.1:8000/data/legal/search/statutes/
  • http://127.0.0.1:8000/data/legal/search/advisory-opinions/

[codecov]

Codecov Report

Attention: Patch coverage is 82.00000% with 9 lines in your changes missing coverage. Please review.

Project coverage is 80.59%. Comparing base (e5b7b1d) to head (5323974).
Report is 51 commits behind head on develop.

Files	Patch %	Lines
fec/legal/views.py	64.00%	9 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #6405      +/-   ##
===========================================
+ Coverage    80.54%   80.59%   +0.05%     
===========================================
  Files          233      233              
  Lines         5042     5077      +35     
===========================================
+ Hits          4061     4092      +31     
- Misses         981      985       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[cnlucas]

Working for me, thank you so much @patphongs!

[rfultz]

Maybe outside the scope of this work, but could we make some changes to the directions for symbols that will work in the main search box?

• The pipe requires spaces before and after it, right?
• The plus sign requires spaces before and after?
• The minus/dash requires a space in front and zero spaces after?
• If a user actually typed curly quotes, things would break, yes?
• Using a <pre> would apply a monospace font to what should be typed.
• Giving the <pre> a background color would visually demonstrate where spaces should and should not be included.

(the code in these: <pre style="background-color: rgba(211,211,211,.5);display:inline">)

I don't know if this is a better option than a space between the quotes.

[patphongs]

Maybe outside the scope of this work, but could we make some changes to the directions for symbols that will work in the main search box?

Thanks for your insights @rfultz, I think we can consider these changes more in depth in another ticket. Let me address each of your questions:

• The pipe requires spaces before and after it, right?

The pipe doesn't require spaces before and after, but I put it in there so the query is easier to read.

• The plus sign requires spaces before and after?

The plus sign doesn't require spaces before and after, but I put it in there so the query is easier to read also.

• The minus/dash requires a space in front and zero spaces after?

This is correct, the dash to exclude does require a space in front and 0 spaces after or else it will put in an additional dash in the space directly after.

• If a user actually typed curly quotes, things would break, yes?

Because we switched over to the standard analyzer and simple_query_string for the API, it's actually more forgiving with random chars like that. It would just ignore the curly quotes entirely. But you make a good point that a user may copy and paste and get confused. We shouldn't sacrifice styling for usability.

• Using a <pre> would apply a monospace font to what should be typed.
• Giving the <pre> a background color would visually demonstrate where spaces should and should not be included.

Interesting idea! I'd love to take a look at this further with you.

Other upcoming changes

• Remove symbol section for the more keyword options box as the user doesn't need it to fill out the inputs
• We need to add documentation for the symbols somewhere else after we remove it from the options box. It will be in the tooltip for now, but I think we need better documentation coming up.
• Possibly using commas to do the .split(',') in the future since the standard analyzer ignores most punctuation including commas. So it may be reasonable to ask the user to insert a comma between terms so that it's easier for the webpage to parse.

[rfultz]

Nice! I've been trying to break it and can't seem to!

For this ticket or a future one, could we add tests for gibberish, too, to make sure failures are handled properly? Mismatched parens (e.g. (, )(, ), (…(, )…) ), strange characters, too short, etc

[JonellaCulmer]

Tested on all effected pages, looks good. Thanks @patphongs.