How to get best performance

[Richie-Peak]

Hi all,

Given the miniscule costs of embeddings, it seems that for all but the poorest people with the biggest vaults, selecting the highest performance (and thus most expensive) embeddings models makes sense. (that is, assuming you use APIs, not local models which are of course free). It definitely seems to make sense for me.

Given this, shouldn't I just select OpenAI text-3 large for both notes and blocks? and GPT4 turbo 128k for chat?

I acknowledge I am responsible for any costs I incur using this plugin. I will not hold anyone responsible for costs I incur following their advice :)

[brianpetro]

Hey @Richie-Peak

If cost is a non-issue, computer performance is the only remaining thing to consider. The higher the number of dimensions (dims), the more work your computer has to do.

I'd suggest using higher dims for notes than blocks for that performance reason. But, if you have high memory and a good CPU, then this difference might not be noticeable.

If you can try out different models, it would be really helpful to the community if you could report back your experience and let us know which is optimal for your vault.

🌴

[GGFXwan]

Is there a thread anywhere yet that's been updating information on best performing models? If not, I'd be more than happy to start that process. I LLOOOOVVVEEE this plugin, and what it's capable of, and I'm very much looking forward to how my workflow is about to change, yugely.

[fabioscarsi]

Optimizing this choice seems absolutely relevant to getting the best results with Smart Connections. We could make conscious choices if we had some sort of Chatbot Arena rating of which models give the best results. Another interesting statistic could be the number of users who use a specific embedding model.

[brianpetro]

Agreed!

I've been doing a lot behind the scenes with a demo/test vault, primarily to ensure stability of the features, however it should also be able to be used to compare results between models. I'll have to give it some more thought.

In the short-term, maybe we could get some sort of discussion poll going for model usage. Automatically collecting the data could be done, but it would need to be opt-in-only to maintain the privacy-focused principles of the project. If we came up with a few high-impact data points then I think it would be a worthwhile.

I'd also be interested to see if different models provide better/worse results based on the note taking style.

Lastly, on the backend I've been making progress toward customizable "find connections" algorithms (currently just a JavaScript method that returns a score based embedding similarity). Similarly, the "get embed input" method for the text that gets embedded. A more complex algorithm could take into account things like metadata, similarity between centroids, apply "fine-tuning" techniques, etc. And with additional options that take into account a user's note/vault style/preferences, the same model should be able to perform better.

🌴

[fabioscarsi]

@brianpetro, I am so sorry I can't code and contribute. However, I think an opt-in data collection that doesn't collect anything other than usage statistics on the embedding models would be acceptable. The aggregated data should be available to the users to facilitate our choices. The implicit contract to promote opting-in could be that you opt in and have visibility into the aggregated data. In addition, the value of those aggregated data for developing the Smart Connections ecosystem seems enormous. Am I wrong?