Skip to content

6.2.0
Compare
Choose a tag to compare
@epugh epugh released this 11 Jun 19:43
v6.2.0
02e2a5e

6.2.0 - 2020-06-11

This release of Quepid finally introduces the classical search geek scorers like NDCG, DCG, Average Precision, and friends by default. We've also fixed a large number of bugs introduced by changes in Firefox and Chrome browser security models that happened in the first part of 2020, as well as recent updates to Elasticsearch and Solr.

The burgeoning suite of open source tools for relevancy all require judgement lists. We revamped Quepid to handle exporting and importing from your favorite tools like Quaerite and RRE, as well as a Learning to Rank formatted export. You can see this in action in Chorus, an ecommerce search focused stack that includes Quepid.

Ease of development has long been an issue with Quepid. In this release we've worked on developer happiness by auditing all of the dependencies we include. We're removed quite a few Ruby gems and Node packages that weren't used, and upgraded the rest (replacing PhantomJS with Puppeteer for headless testing). yarn.lock dropped from 6706 lines to 1525 and Gemfile.lock from 448 to 330! This allowed us to finally move to ECMAScript 6 and Node version 10. Lastly we have cut the production Quepid Docker image from almost 3 GB down to 2.3 GB.

Upgrade Notes

Follow the steps outlined at https://github.com/o19s/quepid/wiki/Installation-Guide#updating-quepid.

You will need to run some database migrations to update the database and insert the new classical scorers (NDCG, AP, etc).

There is a sql migration script in ./db/release_6_2_0_merge_default_scorer_into_scorer.sql that you might want to look if you have been using Quepid locally for a while. This script is primarily meant to support updating http://app.quepid.com database for the 6.2.0 schema.

QUEPID_DEFAULT_SCORER is a new environment variable specifying the name of the default scorer
for users when they create a new case. QUEPID_DEFAULT_SCORER=AP@5 is what app.quepid.com uses.

Features

  • Quepid Default Scorer Admin screen is buried in UI. Add it to the dropdown menu. #96 by @epugh.
  • Support Basic Auth in Quepid with Elasticsearch. #114 and o19s/splainer-search#79 by @CGamesPlay fixes #109.
  • When installing Quepid, use thor user:create to create an administrator! #112 by @epugh fixes #107.
  • Enhanced export ratings function that follows the standard query,docid,rating supports full export/import lifecycle and easier integration with other tools. #128 by @epugh fixes #126.
  • Export ratings in RRE and LTR file formats. Import ratings from RRE Judgement JSON file. #137 and #139 and #152 by @epugh fixes #133
  • Classical scorers like AP, DCG, and NDCG are now shipping by default. All new users who sign up on Quepid will start with AP@5. This effort is still in progress, for example right now the scorers are named AP@5, DCG@5, NDCG@5 because they are hard coded to only look at the first five ranks. We also need to work on the Q Score Graph's vertical axis handling. Huge effort in #100 by @nathancday and @epugh to get this done. This change also removes the individual query level scorer being assignable, which was a bad idea (#132) and restores the ability to write a custom unit test for a query in Javascript (#120) which is an interesting idea! Thanks to @janhoy for being the impetuous to get this work done!

Improvements

  • Community member reported the default demo TMDB Elasticsearch server on port 9200 had no data. Having seem random "drive by vandalism" of our demo Solr and ES servers that run on default ports, we want to move away. Yes, security by obscurity. #103 by @epugh changes default used in Wizard to port 8985 and 9206. #125 and #104 migrate existing URLs to the new ones.
  • Increase Developer Happiness by speeding up the bin/setup_docker script by caching RubyGems. 50% speed up! #105 by @nathancday.
  • Remove the concept of "communal" i.e shared with everyone scorers. This feature has been in Quepid for years, yet even @softwaredoug didn't remember it! #99 by @epugh fixes #98.
  • In the beginning of Quepid, before there was a Search Relevancy community, there was just one, slightly janky scorer, that wasn't like any of the standard search geek scorers (looking at you NDCG) that we use today. We are building towards supporting many scorers, so it's time to remove the DEFAULTS definition of a scorer, and the related scope in ActiveRecord. #97 by @epugh.
  • Solr 8.2 tightened up the security profile for accessing it that we depend on via JSONP to have Quepid work. Now the wizard provides you the command to run on your Solr if it can't connect. #95 by @epugh fixes #92.
  • Using the default tmdb dataset? Demonstrate the thumb:poster_path feature. #94 by @epugh fixed #72.
  • Rename user.username in database to user.email since that is what we use. Clean up API. #113 by @epugh fixes #111. You will need to run database migration for this release!
  • Audited code base to prune dependencies and remove unused code. #121, #119, #118, #116 all worked towards this goal.
  • Turns out Quepid has a Curate interface. It isn't supported, so in the interest of reducing our codebase, removing it. Points the way for the future however! #123.
  • Deal with sameSite cookie warnings from Firefox. #131 by @epugh fixes #130.
  • curator variables (i.e knobs and dials) that aren't used in query cause weird UI. #135 by @epugh fixes #64.
  • Saving two annotations in a row doesn't work, you need to rescore per annotation. Fixed in #136 by @epugh.
  • Users can opt out of community marketing emails. Add a /admin/users.csv export to make keeping track of that easier. Thanks @flaxsearch for the suggestion.
  • Inconsistent use of X icon in modal popups is frustrating. #148 and #149 by @worleydl fixes #146 and #145.
  • Swap the sorting of tries in the history tab to the newest first, going back in time. #151 by @epugh fixes #143 by @renekrie.
  • Be clearer that MySQL is the only supported database platform. #156 by @epugh fixes #155 by @janhoy.

Bugs

  • Wizard Autocomplete Didn't work well with Keyboard. Autocomplete suggestion had to be clicked with a Mouse. #94 by @epugh fixes this by upgrading package.
  • Multivalued and nest JSON fields didn't display well, you would get [object Object] instead. Now we display arrays and Json properly. #117 by @CGamesPlay fixes #52.
  • fixed highlighting throwing an error on Solr date fields by using hl.method=unified in Splainer-Search v2.5.9. #84 created by @janhoy.
  • fixed fields with a . like foo.bar failing to be rendered in UI in Splainer-Search v2.5.9. #106 created by @rjurney.
  • When a try number is not specified for a case, automatically redirect to the most recent try. #122 by @epugh fixes #110 by @binarymax.
  • Case shared via Team with me has NaN for the Try. #134 by @epugh fixes #83.
  • Not able to export cases from a Team page. Commit quepid@021bda8627463986f1ff04ffb1c290195c428c5b by @epugh fixed this.
  • Race condition in laying out the main Quepid Screen and the Tune Relevance screen. #147 by @worleydl fixes #144 by @epugh.
  • Using the "peek" function to page to deeper search results would lose your escapeQueries choice. o19s/splainer-search#87 by @jorgelbg fies this. Bump to splainer search 2.6.0 to include the fix.
Assets 2
Loading