Code

Adds Weaviate Engine#203

Code

Open

kosmikdc wants to merge 98 commits intomaintreygrainger/ai-powered-search:mainfrom weaviate_enginetreygrainger/ai-powered-search:weaviate_engineCopy head branch name to clipboard

+2,390-20,141

Commits

Commits on Aug 12, 2024

• - 4.7 - listing organization

  Show description for 48d848a
  

  Daniel Crouch

  committed
  48d848a

  Copy full SHA for 48d848a

Commits on Aug 15, 2024

• - Applies listing print standards to all chapters

  Show description for bb174c7
  

  Daniel Crouch

  committed
  bb174c7

  Copy full SHA for bb174c7

• SKG debug fix

  

  Daniel Crouch

  committed
  562349a

  Copy full SHA for 562349a

• Rework ch3 tf-idf calculations; sync with manuscript

  treygrainger

  committed
  a8f73dd

  Copy full SHA for a8f73dd

• Ch4, 5, 6: Listings and results synchronized

  

  Daniel Crouch

  committed
  5fd68b6

  Copy full SHA for 5fd68b6

Commits on Aug 16, 2024

• Ch7 and Ch8: Listings synced

  

  Daniel Crouch

  committed
  e71ccc4

  Copy full SHA for e71ccc4

Commits on Aug 18, 2024

• Ch9 mostly consistent

  

  Daniel Crouch

  committed
  ee4988f

  Copy full SHA for ee4988f

Commits on Aug 19, 2024

• Syncs listing 8.1

  

  Daniel Crouch

  committed
  776390a

  Copy full SHA for 776390a

Commits on Aug 20, 2024

• Ch10, Ch11 sync'd with manuscript and signed off

  

  Daniel Crouch

  committed
  37c4ed5

  Copy full SHA for 37c4ed5

• Ch8 sync, ch10 n => k

  

  Daniel Crouch

  committed
  e9a597b

  Copy full SHA for e9a597b

• ch9: Correct, Deterministic, sync'd with manuscript, signing off

  

  Daniel Crouch

  committed
  c696b2d

  Copy full SHA for c696b2d

Commits on Aug 24, 2024

• - ch12, 13, 14, and 15 finalized and sync with Manuscript

  Show description for 3d6b4ed
  

  Daniel Crouch

  committed
  3d6b4ed

  Copy full SHA for 3d6b4ed

Commits on Aug 26, 2024

• quantization WIP

  treygrainger

  committed
  7711b1f

  Copy full SHA for 7711b1f

Commits on Sep 4, 2024

• Intermediate Ch13 Quantization Faiss impl

  

  Daniel Crouch

  committed
  301e40d

  Copy full SHA for 301e40d

Commits on Sep 5, 2024

• Further improvementsQuantization code. Still needs cleanup and Listing 13.26 implemented

  

  Daniel Crouch

  committed
  595507e

  Copy full SHA for 595507e

Commits on Sep 6, 2024

• Another checkpoint. 3/5 working (PQ and engine left). Code 50% cleaned

  

  Daniel Crouch

  committed
  006e793

  Copy full SHA for 006e793

• Temp state. Just for reference. Mid refactor

  

  Daniel Crouch

  committed
  4bb1b81

  Copy full SHA for 4bb1b81

Commits on Sep 7, 2024

• Listing 13.21 int8 scalar quantization done

  

  Daniel Crouch

  committed
  d63cd9c

  Copy full SHA for d63cd9c

• Another checkpoint

  

  Daniel Crouch

  committed
  05bfc63

  Copy full SHA for 05bfc63

• Adds stubbed rerank recall method

  

  Daniel Crouch

  committed
  6634723

  Copy full SHA for 6634723

Commits on Sep 9, 2024

• Small refactors as I'm working on manuscript listings

  treygrainger

  committed
  34439b0

  Copy full SHA for 34439b0

• Another checkpoint.

  

  Daniel Crouch

  committed
  06c3428

  Copy full SHA for 06c3428

Commits on Sep 10, 2024

• Fix index writing error w/ PQ, increase M to get PQ recall up

  treygrainger

  committed
  303e98f

  Copy full SHA for 303e98f

• Checkpoint on evaluate rerank

  

  Daniel Crouch

  committed
  2785935

  Copy full SHA for 2785935

Commits on Sep 11, 2024

• Checkpoint, Only one bug in calculate_recall(). Picking up engine search in the morning

  

  Daniel Crouch

  committed
  2207ca8

  Copy full SHA for 2207ca8

• Cleanup quantization code (minus last engine example) + rename notebook

  treygrainger

  committed
  b34d239

  Copy full SHA for b34d239

• Include SolrEngine

  

  Daniel Crouch

  committed
  ff11448

  Copy full SHA for ff11448

• Reorganize to match manuscript listing flow

  treygrainger

  committed
  f083334

  Copy full SHA for f083334

• Refactor and cleanup quantization code for listings

  treygrainger

  committed
  972b09c

  Copy full SHA for 972b09c

Commits on Sep 16, 2024

• Quantization

  treygrainger

  committed
  883629f

  Copy full SHA for 883629f

• Merge in main

  treygrainger

  committed
  d4d622f

  Copy full SHA for d4d622f

• Revert back from quantization_type to quantization_size

  Show description for 9277191
  treygrainger

  authored
  9277191

  Copy full SHA for 9277191

Commits on Sep 18, 2024

• Finalizes quantization listings

  

  Daniel Crouch

  committed
  6902a83

  Copy full SHA for 6902a83

• Merge branch 'main' into quantization

  

  Daniel Crouch

  committed
  8768f23

  Copy full SHA for 8768f23

Commits on Sep 22, 2024

• Cleanup to sync with manuscript

  treygrainger

  committed
  ac3d2a2

  Copy full SHA for ac3d2a2

Commits on Oct 8, 2024

• - All chapters verified, functional and consistent (See lingering issues)

  Show description for aeb8645
  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  aeb8645

  Copy full SHA for aeb8645

• Finalizes cross encoder code

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  54fcb44

  Copy full SHA for 54fcb44

• Basic OS implementation

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  d658060

  Copy full SHA for d658060

• OpenSearch Spell checking

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  cf86ee6

  Copy full SHA for cf86ee6

• Vector search, schema improvements, spark

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  08f4dd1

  Copy full SHA for 08f4dd1

• UBI: UBI Plugin added to image, docker organization, opensearch configs, basic

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  5270da1

  Copy full SHA for 5270da1

• Generating Signals collection from UBI

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  399e093

  Copy full SHA for 399e093

• Unsaved

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  c9c655a

  Copy full SHA for c9c655a

• Fixes signals boosting

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  c63d1a9

  Copy full SHA for c63d1a9

• This is a checkpoint

  

  Daniel Crouch

  authored and

  kosmikdc

  committed
  5095ea9

  Copy full SHA for 5095ea9

• Make opensearch-docker-entrypoint.sh executable

  

  treygrainger

  authored and

  kosmikdc

  committed
  852529f

  Copy full SHA for 852529f

• Fixes build.

  kosmikdc

  committed
  5b9aa06

  Copy full SHA for 5b9aa06

• Fixes build and correctly integrates with ubi 1.0.0 for OS 2.14

  kosmikdc

  committed
  5b81cf9

  Copy full SHA for 5b81cf9

• Docker file updates

  kosmikdc

  committed
  ab31e48

  Copy full SHA for ab31e48

• check

  kosmikdc

  committed
  69a5889

  Copy full SHA for 69a5889

• check again

  kosmikdc

  committed
  9c72b15

  Copy full SHA for 9c72b15

• check again

  kosmikdc

  committed
  cefd8cf

  Copy full SHA for cefd8cf

• check again again

  kosmikdc

  committed
  df50a60

  Copy full SHA for df50a60

• Fixes keystroke error that broke 9.18

  kosmikdc

  committed
  1888d59

  Copy full SHA for 1888d59

• Check

  kosmikdc

  committed
  01cd4b8

  Copy full SHA for 01cd4b8

Commits on Oct 9, 2024

• check

  kosmikdc

  committed
  eb2edd6

  Copy full SHA for eb2edd6

Commits on Oct 10, 2024

• Updates, organizes and cleans dep stack. Updates applied throughout the stack

  kosmikdc

  committed
  9f10f84

  Copy full SHA for 9f10f84

• Merge branch 'deps_update' into quantization

  kosmikdc

  committed
  2753e5e

  Copy full SHA for 2753e5e

• Merge branch 'quantization' into os_engine

  kosmikdc

  committed
  d8b1555

  Copy full SHA for d8b1555

Commits on Oct 11, 2024

• OpenSearch integration:

  Show description for 144b158
  kosmikdc

  committed
  144b158

  Copy full SHA for 144b158

Commits on Oct 18, 2024

• Intermediate checkpoint. Still fixing Ch9 and cleaning

  kosmikdc

  committed
  d7516a2

  Copy full SHA for d7516a2

Commits on Oct 21, 2024

• - Adds optional engine override param

  Show description for a6d31e7
  kosmikdc

  committed
  a6d31e7

  Copy full SHA for a6d31e7

Commits on Oct 22, 2024

• Ch8 completely functional except index time boosting query side

  kosmikdc

  committed
  1ca531a

  Copy full SHA for 1ca531a

Commits on Oct 24, 2024

• Chapter 8 finished for Opensearch except with Spark Bulk Indexing failures in 8.8

  kosmikdc

  committed
  853956e

  Copy full SHA for 853956e

• UBI Checkpoint: 3 pending issues listed in notebook

  kosmikdc

  committed
  3dade7b

  Copy full SHA for 3dade7b

Commits on Oct 25, 2024

• Corrects two queries missing un-aliased variable references

  kosmikdc

  committed
  3604bb2

  Copy full SHA for 3604bb2

• Corrects 8.8

  kosmikdc

  committed
  2710556

  Copy full SHA for 2710556

Commits on Oct 26, 2024

• Fix index loader so 8.8 runs + improve 8.8 variables/revert line order

  treygrainger

  committed
  4bebcc1

  Copy full SHA for 4bebcc1

• Add back new line near end of listing 8.8

  treygrainger

  committed
  3823fc3

  Copy full SHA for 3823fc3

Commits on Oct 28, 2024

• Merge branch 'main' into os_engine

  kosmikdc

  committed
  3316d2d

  Copy full SHA for 3316d2d

Commits on Nov 1, 2024

• OS LTR all features except model idempotency and explore candidates. Needs cleaning

  kosmikdc

  committed
  95fa4e8

  Copy full SHA for 95fa4e8

• All LTR working except fuzzy/bigrams

  kosmikdc

  committed
  dd43938

  Copy full SHA for dd43938

Commits on Nov 4, 2024

• Merge branch 'main' into os_engine

  kosmikdc

  committed
  3843424

  Copy full SHA for 3843424

Commits on Nov 5, 2024

• OpenSearch LTR complete

  kosmikdc

  committed
  a084a9b

  Copy full SHA for a084a9b

Commits on Nov 7, 2024

• Corrects ch 13, 14, 15. improves 8.8

  kosmikdc

  committed
  61f19b2

  Copy full SHA for 61f19b2

Commits on Nov 16, 2024

• - Implements Opensearch Sparse Semantic Search completely semantic functions

  Show description for ab59120
  kosmikdc

  committed
  ab59120

  Copy full SHA for ab59120

Commits on Nov 18, 2024

• All chapter's verified

  kosmikdc

  committed
  71b1978

  Copy full SHA for 71b1978

Commits on Nov 19, 2024

• Corrects vector size for outdoors embedding collection

  kosmikdc

  committed
  739c9e2

  Copy full SHA for 739c9e2

• - Adds weaviate to build with spark

  Show description for 9783614
  kosmikdc

  committed
  9783614

  Copy full SHA for 9783614

Commits on Nov 20, 2024

• Weaviate checkpoint. GraphQL almost functional

  kosmikdc

  committed
  e452696

  Copy full SHA for e452696

Commits on Nov 26, 2024

• - Weaviate basic search functionality

  Show description for 90ff749
  kosmikdc

  committed
  90ff749

  Copy full SHA for 90ff749

Commits on Dec 8, 2024

• Weaviate Spark Reader

  Show description for 464664e
  kosmikdc

  committed
  464664e

  Copy full SHA for 464664e

Commits on Dec 11, 2024

• Standardizes spark session initialization to always have 8g available

  kosmikdc

  committed
  bc26ae9

  Copy full SHA for bc26ae9

Commits on Dec 13, 2024

• Merge branch 'main' into weaviate_engine

  kosmikdc

  committed
  53cdbb2

  Copy full SHA for 53cdbb2

Commits on Dec 17, 2024

• - Implements WV8 vector search

  Show description for fb43661
  kosmikdc

  committed
  fb43661

  Copy full SHA for fb43661

Commits on Dec 19, 2024

• Weaviate

  Show description for f109a5c
  kosmikdc

  committed
  f109a5c

  Copy full SHA for f109a5c

• - further leverages engine name for multi-methods

  Show description for c67b08d
  kosmikdc

  committed
  c67b08d

  Copy full SHA for c67b08d

Commits on Dec 25, 2024

• - Fixes recommendation indexing in Ch9 for weaviate

  kosmikdc

  committed
  39a2253

  Copy full SHA for 39a2253

Commits on Dec 27, 2024

• Fixes weaviate schema for recs

  kosmikdc

  committed
  b6a358e

  Copy full SHA for b6a358e

Commits on Jan 2, 2025

• - Adds support for an embedded solr engine for entity extraction and SKG

  Show description for b9a84f8
  kosmikdc

  committed
  b9a84f8

  Copy full SHA for b9a84f8

Commits on Jan 8, 2025

• - Semantic Search for weaviate implemented, quality is bad

  Show description for 444180e
  kosmikdc

  committed
  444180e

  Copy full SHA for 444180e

Commits on Jan 9, 2025

• - Mostly removes the existance of __weaviate_id, now used only in create_view_for_collection wv8 cursor loop.

  Show description for 9600dd3
  kosmikdc

  committed
  9600dd3

  Copy full SHA for 9600dd3

• Organizes system created files and docker image files

  kosmikdc

  committed
  1cbb6fa

  Copy full SHA for 1cbb6fa

Commits on Jan 10, 2025

• Merge branch 'main' into weaviate_engine

  kosmikdc

  committed
  afda4ab

  Copy full SHA for afda4ab

• Cleanup, verify

  kosmikdc

  committed
  8fb1e4b

  Copy full SHA for 8fb1e4b

Commits on Jan 11, 2025

• Fixes weaviate semantic query generation, reruns some chapters with solr

  kosmikdc

  committed
  3c08d23

  Copy full SHA for 3c08d23

Commits on Jan 12, 2025

• Reruns Chapter 8 w/ Solr

  kosmikdc

  committed
  ed57c9c

  Copy full SHA for ed57c9c

• Runs Ch8 again, again

  kosmikdc

  committed
  df59499

  Copy full SHA for df59499