.Net: Add Sql Server Support for memory in .NET SDK #2339
Closed
+1,733
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Fix #2312
As discussed in #2265, the objective is to add support of MSSQL Server (and Azure SQL) for vector database of the project.
Description
I created the SQL server support based on what is presented in the following article : https://learn.microsoft.com/en-us/samples/azure-samples/azure-sql-db-openai/azure-sql-db-openai/
The code structure corresponds to that already in place with PostgreSQL or SQLite. A DB client performs actions towards the database, while the MemoryStore uses this client to make the necessary calls.
The DB Client creates 2 basic tables:
In addition, when requesting the creation of a new collection, the client creates a new table
SKEmbedding_{collectionName}
for indexing all vector data.The clustered column store index for vector search is therefore present in the last table.
When deleting collections, the client deletes the corresponding embedding table and deletes the entry in the collections table.
As far as unit and integration tests are concerned, I've reproduced the tests available for PostgreSQL and adapted them to SqlServer.
Contribution Checklist