Skip to content

The definitive guide to using Vector Search to solve your semantic search production workload needs.

Notifications You must be signed in to change notification settings

esteininger/vector-search

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

91 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Vector Search

Vector Search engines provide the ability for developers to store vectors structured around certain algorithms (i.e. KNN), and an engine to compute similar vectors (like cosine distance) to determine which vectors are related.

This repository provides a comprehensive overview of the vector search landscape inclusive of tutorials, guides, best-practices, and extended learning. Please review the Education section to learn more.

Here is how you may use a Vector Search engine within your application search architecture:

Topics

🧑‍🏫 Foundation - Learn the core concepts of vector-based information retrieval.

🎬 Use Cases - Understand where it makes sense to use vector search.

💵 Architecture - Guides on how to use vector search in your architecture.

Foundations

# Label Description
1 Keyword vs Vector Search The difference between standard (TF-IDF) text search and vector search and when to use each.
2 Sparse Vector Tutorial A walkthrough of building your own sparse vector feature extraction engine.
3 Dense Vector Tutorial A walkthrough of building your own dense vector feature extraction engine.
4 Atlas Vector Search Engine Guides that showcase MongoDB Atlas' vector search implementation.
5 Vector Search Comparisons A comparison of the most popular vector search engines.

Use Cases

# Label Description
1 Sentence Similarity Determination of how similar to texts are.
2 Token Classification Classification of text into pre-defined categories.
3 Question and Answering Building systems that automatically answer questions.
4 Personalization Using client data to personalize query results.
5 Automated Synonym Creation Enriching synonyms collection automatically.
6 Summarization Reconstruction of a corpus into less words.
7 Conversational Dialogue response generation.
8 File Search Search the contents of files across multiple modalities

Architecture

One-click model deployment that never leaves your AWS account

# Source Description
1 Reference Architecture Common best-practices for deploying vector search architecture in production.
2 Model Hosting Suggestions on how to host your vector models.
3 Model Versioning Common best-practices for versioning your models as they evolve.
4 Feedback Loops Query re-ranking, learn-to-rank and more.
5 Selecting Models Which model supports your domain-specific tasks best?

Education

Although a challenging topic to grasp, there's a myriad of educational resources at your disposal.

Information Retrieval

Overarching field of education.

Transformer Architecture

Latest breakthrough in the area of converting human content (text, images, etc.) into vector representations. Transformers are a deep learning model that utilize "self-attention", and differentially weigh the significance of each part of the input data.

Similarity Search

In order to determine what is deemed relevant, computers need to measure the distance between points, in this case vectors.

Gratitude

This repository wouldn't be possible without several key individuals:

Watch for Changes

This is a living repository and will evolve as I learn and the landscape changes. Please subscribe to changes accordingly via:

About

The definitive guide to using Vector Search to solve your semantic search production workload needs.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published