Getting Started with v2.0 #432
Replies: 8 comments 22 replies
-
|
Very excited to understand the new local models. Please can you expand how they compare to the model used in version 1? Can I get the same or better results using a local model as I could with the OpenAI API? |
Beta Was this translation helpful? Give feedback.
-
|
Great question! The most cited benefit of local models is the privacy factor. The performance will vary between models and even depending on the content you're embedding. A commonly used reference point for embedding model performance is the MTEB leaderboard.
The model used in Smart Connections v1 is currently ranked 27.
I hope that helps! 🌴 |
Beta Was this translation helpful? Give feedback.
-
|
Hesitant to have anything to do with AI partly do with it feeling like a scary tool with malicious intent and because of the privacy aspect. What does local models mean and am I by using your plugin releasing my vault to possibly nefarious influences?? |
Beta Was this translation helpful? Give feedback.
-
|
I'll keep messing around with it, seems good so far. I'm really excited for this though: |
Beta Was this translation helpful? Give feedback.
-
|
Ok got it, I'll play around with it for a while. |
Beta Was this translation helpful? Give feedback.
-
|
I thought embeddings were already note-level? Am I confused? |
Beta Was this translation helpful? Give feedback.
-
|
@quelsemme now you can specify different models for embedding full notes (note-level) and excerpts (block-level). Since excerpts are much shorter on average, it can make sense to use different models for each. 🌴 |
Beta Was this translation helpful? Give feedback.
-
|
That makes sense. I guess I just have always picked the one with the most dimensions and the most tokens, so I end up choosing both the same, for the purposes of trying to get the most complex embedding vector I can. |
Beta Was this translation helpful? Give feedback.
-
|
@quelsemme that's a common strategy 🌴 |
Beta Was this translation helpful? Give feedback.
-
|
I just upgraded to v2. By default, the Blocks embedding model "BGE-micro-v2" was selected. It embedded all the blocks, but Smart Chat wasn't generating any useful content from my notes when I asked it questions. So I've enabled "Jina-v2-small-2k" in Notes Embedding Model. Still, smart chat no longer generates useful content for me, and now every time I open Obsidian, it re-embeds all the smart notes (which takes half an hour). So, two issues here! I'm not an AI expert, I just know Smart Connections was very useful before the upgrade, and now there's a lot more options and the defaults don't work for me :) |
Beta Was this translation helpful? Give feedback.
-
|
@EpicPilgrim let me know if the results are still poor after updating There were some pretty awful logical errors until late last night (EST) and the results are looking much better today. 🌴 |
Beta Was this translation helpful? Give feedback.
-
|
It's much smoother now - startup is very quick! That said, I still can't do a generic query of my notes. For example, I have a note called "STITCH Assessments". I ask Smart Chat "Based on my notes, what is a STITCH assessment?" and it says it doesn't know. I have to reference [[STITCH Assessments]] explicitly for it to do anything with it. Previously, Smart Connections seemed to be able to work across all my notes when I asked questions. Am I missing something with v2? Or was it just chance that it even worked in v1? |
Beta Was this translation helpful? Give feedback.
-
|
@EpicPilgrim thanks for the update. It's most likely that it only worked by chance in v1, as the way embeddings work pretty much makes keyword-based queries ineffective. To better understand, you can imagine that embeddings convert your query into a point in space and retrieval using embeddings returns the nearest points. And while the words "STITCH assessment" occuring in both the query and note points may bring them closer together, that may still not be "close enough" to be included as context. In I hope that helps! 🌴 |
Beta Was this translation helpful? Give feedback.
-
|
That's going to be very powerful. Can't wait to check that out in |
Beta Was this translation helpful? Give feedback.
-
|
That makes great sense, and sounds amazing! I'll make sure to make my prompts more explicit for now but if that is where Smart Chat goes, it's going to be revolutionary. I played with Microsoft 365 Copilot (the paid version) and honestly, it drove me nuts with its inconsistency. This is fantastic. |
Beta Was this translation helpful? Give feedback.
-
|
@brianpetro I've upgraded to v.2 and set the plugin to openAI's most powerful model and embedding tool (accuracy is my most crucial consideration). But the plugin (that worked perfectly in version 1.xx doesn't work at all and, worse, freezes the entire vault where it is installed. How can I manage the problem other than disable or uninstall Smart Connections? Any suggestion? |
Beta Was this translation helpful? Give feedback.
-
|
Hmm, tough to say. Can you explain what's going on more detail. Is it a complete freeze where the entire program can no longer operate? Partial freeze, where you can still type and navigate etc, but typing in to search or AI chat etc does nothing, no text appears (usual indicating smart connections is not done doing its thing). Sometimes closing the program and reopening weirdly fixes that. But you probs already tried that. And by doesn't work at all, are you referring to the freezing? Or that the plugin fails to produce what you're expecting in terms of results? |
Beta Was this translation helpful? Give feedback.
-
|
Can you try using another model than the OpenAI large model? Or set the block model to None? The large model from OpenAI will use about twice as much memory as the v1 Ada model, so that could cause Obsidian to freeze. 🌴 |
Beta Was this translation helpful? Give feedback.
-
|
Absolutely loving the new upgrade! The suggested connections are more accurate, no loss of speed and chat is just as good. Great job Brian! |
Beta Was this translation helpful? Give feedback.
-
|
@Richie-Peak Perfect! Thanks for the feedback 😊 |
Beta Was this translation helpful? Give feedback.
-
|
Hi, Models options are:
|
Beta Was this translation helpful? Give feedback.
-
|
Following my previous message, this is what Mixtral 8x7b says: OUTPUTHi there! I hope you're doing well. Thank you for reaching out and considering the use of our Obsidian plugin. To help you make an informed decision on which embedding model to choose, here are some details about each option: BGE-micro-v2 (Local, 512 tokens, 384 dim): This is a lightweight model suitable for smaller datasets and devices with limited resources. It's designed to provide fast inference times at the cost of slightly lower accuracy compared to larger models. BGE-small (Local, 512 tokens, 384 dim): Similar to the micro version but with a higher capacity for handling more complex data and tasks. This model offers better performance than the micro variant while still maintaining reasonable inference times. Jina-v2-small-2K (Local, 2,048 tokens, 512 dim): A larger model that can handle longer sequences of text compared to the previous two options. It's designed for better accuracy at the expense of slightly slower inference times and higher memory usage. Jina-v2-small-4K (Local, 4,096 tokens, 512 dim): This model is even larger than the 2K version, allowing it to process longer sequences of text with improved accuracy. However, this comes at a cost of increased memory usage and slower inference times compared to smaller models. Jina-v2-small-8K (Local, 8,192 tokens, 512 dim): The largest model among the options provided, capable of handling very long sequences of text with high accuracy. This model requires significant memory resources and may have slower inference times compared to smaller models. Considering your M2 with 96 GB RAM, you should be able to handle any of these models without issue. However, if you're working on a large dataset or require the highest possible accuracy, I would recommend using either Jina-v2-small-4K or Jina-v2-small-8K. If inference speed is more important for your use case, BGE-micro-v2 might be a better choice. Is it true? |
Beta Was this translation helpful? Give feedback.
-
|
At a quick glance it looks right to me. |
Beta Was this translation helpful? Give feedback.
-
|
And for a M1 with 8Gb, what is the recommended local model? |
Beta Was this translation helpful? Give feedback.
-
|
@micuentadecasa I use BGE-micro for block-level and Jina-4k for note-level on my M2 8GB. |
Beta Was this translation helpful? Give feedback.
-
|
Notes Embedding Model and Blocks Embedding Mode is it free to choose? Or do I need additional Settings? For example, I need to download to a certain path to use |
Beta Was this translation helpful? Give feedback.
-
|
The note model is required, but the blocks are optional 🌼 |
Beta Was this translation helpful? Give feedback.
-
Everyone can now upgrade to the v2.0 early release in the Smart Connections settings.
Helping get Smart Connections ready for a version 2 general release:
Do you have questions about the new version?
Having trouble getting started?
Do you have any feedback on the new embedding models?
Do you have any other feedback that might be helpful?
I want to know 💡
Thanks for your help
🌴
Beta Was this translation helpful? Give feedback.
All reactions