[Bug] Qdrant MemoryDb seems to recreate collection with every request

[pedershk]

Context / Scenario

Every time I upload a new file with ImportDocumentAsync, the collection is recreated and everything in it is cleared out, making it kind of useless. I've tried creating the collection ahead of time using the C# QdrantClient, but this behavior persist.

I understand that this is the default behavior in Qdrant when using "recreate_collection" - but the memorydb here uses the HTTP REST interface and not the .Net QdrantClient, and I don't know how this maps to SDK methods (it also interferes a bit with Aspire, btw - where the default connection string points to the GRPC endpoint and not the HTTP endpoint for Qdrant).

I'd rewrite the memorydb to use the QdrantClient as a PR, but don't really know enough about Qdrant to do it safely...

What happened?

The collection for a Kernel Memory instance using Qdrant is recreated with every upload.

Importance

a fix would make my life easier

Platform, Language, Versions

Qdrant running in Docker locally using the Aspire Host package extensions for it.

Relevant log output

This is the log from qdrant:

2025-04-23T13:01:24.648361Z  INFO storage::content_manager::toc::collection_meta_ops: Creating collection index-us-nrc-envrep-sections    
2025-04-23T13:01:24.847418Z  INFO actix_web::middleware::logger: 172.17.0.1 "PUT /collections/index-us-nrc-envrep-sections?wait=true HTTP/1.1" 200 48 "-" "-" 0.199216    
2025-04-23T13:01:24.890215Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.001067    
2025-04-23T13:01:24.901144Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000201    
2025-04-23T13:01:24.902510Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000196    
2025-04-23T13:01:24.903543Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 80 "-" "-" 0.000163    
2025-04-23T13:01:24.919122Z  INFO actix_web::middleware::logger: 172.17.0.1 "PUT /collections/index-us-nrc-envrep-sections/points?wait=true HTTP/1.1" 200 81 "-" "-" 0.007004    
2025-04-23T13:01:39.830129Z  INFO storage::content_manager::toc::collection_meta_ops: Creating collection index-us-nrc-envrep-sections    
2025-04-23T13:01:39.830179Z  INFO actix_web::middleware::logger: 172.17.0.1 "PUT /collections/index-us-nrc-envrep-sections?wait=true HTTP/1.1" 409 112 "-" "-" 0.000153    
2025-04-23T13:01:39.832798Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.001327    
2025-04-23T13:01:39.833656Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000210    
2025-04-23T13:01:39.834485Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000239    
2025-04-23T13:01:39.835179Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000115    
2025-04-23T13:01:39.835829Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 80 "-" "-" 0.000130    
2025-04-23T13:01:39.836605Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000210    
2025-04-23T13:01:39.837358Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000134    
2025-04-23T13:01:39.838068Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000133    
2025-04-23T13:01:39.838787Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000180    
2025-04-23T13:01:39.839517Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000134    
2025-04-23T13:01:39.840243Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000134    
2025-04-23T13:01:39.840919Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000143    
2025-04-23T13:01:39.841579Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000136    
2025-04-23T13:01:39.842218Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000131    
2025-04-23T13:01:39.843209Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000166    
2025-04-23T13:01:39.843966Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000174    
2025-04-23T13:01:39.844831Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000336    
2025-04-23T13:01:39.845764Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000201    
2025-04-23T13:01:39.846598Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000203    
2025-04-23T13:01:39.847288Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000136    
2025-04-23T13:01:39.877999Z  INFO actix_web::middleware::logger: 172.17.0.1 "PUT /collections/index-us-nrc-envrep-sections/points?wait=true HTTP/1.1" 200 83 "-" "-" 0.027560    
2025-04-23T13:01:41.121878Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000299    
2025-04-23T13:01:41.123066Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000397    
2025-04-23T13:01:41.123928Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000212    
2025-04-23T13:01:41.124784Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000238    
2025-04-23T13:01:41.125599Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000221    
2025-04-23T13:01:41.126669Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000260    
2025-04-23T13:01:41.127598Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000239    
2025-04-23T13:01:41.128499Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000197    
2025-04-23T13:01:41.129317Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000123    
2025-04-23T13:01:41.130111Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000137    
2025-04-23T13:01:41.130851Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000177    
2025-04-23T13:01:41.131528Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000114    
2025-04-23T13:01:41.132175Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000134    
2025-04-23T13:01:41.132859Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000134    
2025-04-23T13:01:41.140389Z  INFO actix_web::middleware::logger: 172.17.0.1 "PUT /collections/index-us-nrc-envrep-sections/points?wait=true HTTP/1.1" 200 82 "-" "-" 0.004301    
2025-04-23T13:01:49.949547Z  INFO storage::content_manager::toc::collection_meta_ops: Creating collection index-us-nrc-envrep-sections    
2025-04-23T13:01:49.949598Z  INFO actix_web::middleware::logger: 172.17.0.1 "PUT /collections/index-us-nrc-envrep-sections?wait=true HTTP/1.1" 409 112 "-" "-" 0.000221    
2025-04-23T13:01:49.951683Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000599    
2025-04-23T13:01:49.952874Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000183    
2025-04-23T13:01:49.954076Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000178    
2025-04-23T13:01:49.954987Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000169    
2025-04-23T13:01:49.955858Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000152    
2025-04-23T13:01:49.956886Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000394    
2025-04-23T13:01:49.957794Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000223    
2025-04-23T13:01:49.958852Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000235    
2025-04-23T13:01:49.959862Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000206    
2025-04-23T13:01:49.960797Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000138    
2025-04-23T13:01:49.961428Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000126    
2025-04-23T13:01:49.962045Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000146    
2025-04-23T13:01:49.962675Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000161    
2025-04-23T13:01:49.963323Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000141    
2025-04-23T13:01:49.963972Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000152    
2025-04-23T13:01:49.964635Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000143    
2025-04-23T13:01:49.965217Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000113    
2025-04-23T13:01:49.965816Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000129    
2025-04-23T13:01:49.966393Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 81 "-" "-" 0.000132    
2025-04-23T13:01:49.967296Z  INFO actix_web::middleware::logger: 172.17.0.1 "POST /collections/index-us-nrc-envrep-sections/points/scroll HTTP/1.1" 200 80 "-" "-" 0.000142    
2025-04-23T13:01:49.987616Z  INFO actix_web::middleware::logger: 172.17.0.1 "PUT /collections/index-us-nrc-envrep-sections/points?wait=true HTTP/1.1" 200 83 "-" "-" 0.017201

[dluc]

is the memory configured to persist documents?

[pedershk]

Unsure what you're referring to - isnt that what memory is - persistent storage? We've been using KM with AI Search for a while and are looking at supporting Qdrant in addition to AI Search.

Our AI Search extension method for setting up KM looks like this:

    private static IKernelMemory CreateKernelMemoryInstance(
        IServiceProvider serviceProvider,
        ServiceConfigurationOptions serviceConfigurationOptions,
        string blobContainerName,
        string indexName = "")
    {
        var azureCredentialHelper = serviceProvider.GetRequiredService<AzureCredentialHelper>();
        var configuration = serviceProvider.GetRequiredService<IConfiguration>();
        var baseSearchClient = serviceProvider.GetRequiredService<SearchIndexClient>();

        // Get OpenAI connection string
        var openAiConnString = configuration.GetConnectionString("openai-planner") ?? throw new InvalidOperationException("OpenAI connection string not found.");

        // Extract Endpoint and Key
        var openAiEndpoint = openAiConnString.Split(";").FirstOrDefault(x => x.Contains("Endpoint="))?.Split("=")[1]
            ?? throw new ArgumentException("OpenAI endpoint must be provided in the configuration.");
        var openAiKey = openAiConnString.Split(";").FirstOrDefault(x => x.Contains("Key="))?.Split("=")[1]
            ?? throw new ArgumentException("OpenAI key must be provided in the configuration.");

        var openAiEmbeddingConfig = new AzureOpenAIConfig()
        {
            Auth = AzureOpenAIConfig.AuthTypes.APIKey,
            Endpoint = openAiEndpoint,
            APIKey = openAiKey,
            Deployment = serviceConfigurationOptions.OpenAi.EmbeddingModelDeploymentName,
            APIType = AzureOpenAIConfig.APITypes.EmbeddingGeneration
        };

        var openAiChatCompletionConfig = new AzureOpenAIConfig()
        {
            Auth = AzureOpenAIConfig.AuthTypes.APIKey,
            Endpoint = openAiEndpoint,
            APIKey = openAiKey,
            Deployment = serviceConfigurationOptions.OpenAi.Gpt4o_Or_Gpt4128KDeploymentName,
            APIType = AzureOpenAIConfig.APITypes.ChatCompletion
        };

        var azureAiSearchConfig = new AzureAISearchConfig()
        {
            Endpoint = baseSearchClient.Endpoint.AbsoluteUri,
            Auth = AzureAISearchConfig.AuthTypes.ManualTokenCredential
        };

        azureAiSearchConfig.SetCredential(azureCredentialHelper.GetAzureCredential());

        // Blob Connection String
        var blobConnection = configuration.GetConnectionString("blob-docing") ?? throw new ArgumentException("Blob Connection String must be provided");

        // Extract Blob account name
        var blobAccountName = blobConnection.Split(".")[0].Split("//")[1];

        var azureBlobsConfig = new AzureBlobsConfig()
        {
            Account = blobAccountName,
            ConnectionString = blobConnection,
            Container = blobContainerName,
            Auth = AzureBlobsConfig.AuthTypes.ManualTokenCredential
        };

        azureBlobsConfig.SetCredential(azureCredentialHelper.GetAzureCredential());

        var textPartitioningOptions = new TextPartitioningOptions
        {
            MaxTokensPerParagraph = PartitionSize,
            OverlappingTokens = 0
        };

        // Get or create KernelMemoryConfig
        var kernelMemoryConfig = serviceProvider.GetService<KernelMemoryConfig>() ?? configuration.GetSection("KernelMemory").Get<KernelMemoryConfig>() ?? new KernelMemoryConfig();

        if (kernelMemoryConfig.DataIngestion.MemoryDbUpsertBatchSize < 20)
        {
            kernelMemoryConfig.DataIngestion.MemoryDbUpsertBatchSize = 20;
        }

        if (!string.IsNullOrEmpty(indexName))
        {
            kernelMemoryConfig.DefaultIndexName = indexName;
        }

        var kernelMemoryBuilder = new KernelMemoryBuilder();

        kernelMemoryBuilder.Services.AddSingleton(kernelMemoryConfig);

        kernelMemoryBuilder
            .WithAzureOpenAITextEmbeddingGeneration(openAiEmbeddingConfig)
            .WithAzureOpenAITextGeneration(openAiChatCompletionConfig)
            .WithCustomTextPartitioningOptions(textPartitioningOptions)
            .WithAzureBlobsDocumentStorage(azureBlobsConfig)
            .WithAzureAISearchMemoryDb(azureAiSearchConfig)
            ;

        // Add Logging
        kernelMemoryBuilder.Services.AddLogging(l =>
        {
            l.AddConsole().SetMinimumLevel(LogLevel.Information);
            l.AddConfiguration(configuration);
        });

        var kernelMemory = kernelMemoryBuilder.Build();

        return kernelMemory;
    }

And this is the Qdrant version:

    private static IKernelMemory CreateKernelMemoryInstance(
        IServiceProvider serviceProvider,
        ServiceConfigurationOptions serviceConfigurationOptions,
        string blobContainerName,
        string indexName = "")
    {
        var azureCredentialHelper = serviceProvider.GetRequiredService<AzureCredentialHelper>();
        var configuration = serviceProvider.GetRequiredService<IConfiguration>();
        //var baseSearchClient = serviceProvider.GetRequiredService<SearchIndexClient>();

        // Get OpenAI connection string
        var openAiConnString = configuration.GetConnectionString("openai-planner") ?? throw new InvalidOperationException("OpenAI connection string not found.");

        // Extract Endpoint and Key
        var openAiEndpoint = openAiConnString.Split(";").FirstOrDefault(x => x.Contains("Endpoint="))?.Split("=")[1]
            ?? throw new ArgumentException("OpenAI endpoint must be provided in the configuration.");
        var openAiKey = openAiConnString.Split(";").FirstOrDefault(x => x.Contains("Key="))?.Split("=")[1]
            ?? throw new ArgumentException("OpenAI key must be provided in the configuration.");

        var openAiEmbeddingConfig = new AzureOpenAIConfig()
        {
            Auth = AzureOpenAIConfig.AuthTypes.APIKey,
            Endpoint = openAiEndpoint,
            APIKey = openAiKey,
            Deployment = serviceConfigurationOptions.OpenAi.EmbeddingModelDeploymentName,
            APIType = AzureOpenAIConfig.APITypes.EmbeddingGeneration
        };

        var openAiChatCompletionConfig = new AzureOpenAIConfig()
        {
            Auth = AzureOpenAIConfig.AuthTypes.APIKey,
            Endpoint = openAiEndpoint,
            APIKey = openAiKey,
            Deployment = serviceConfigurationOptions.OpenAi.Gpt4o_Or_Gpt4128KDeploymentName,
            APIType = AzureOpenAIConfig.APITypes.ChatCompletion
        };

        var qdrantConnectionString = configuration.GetConnectionString("qdrant") ?? throw new ArgumentException("Qdrant connection string must be provided");

        // Parse the connection string to extract the Qdrant URL and API key
        // Connection string format: "Endpoint=http://localhost:6334;Key=123456!@#$%"

        var qdrantUrl = qdrantConnectionString.Split(";").FirstOrDefault(x => x.Contains("Endpoint="))?.Split("=")[1]
            ?? throw new ArgumentException("Qdrant endpoint must be provided in the configuration.");

        // If the URL points to localhost and a port number - this is the grpc port number. Increase the port number by one so we get 
        // the http port number.

        if (qdrantUrl.Contains("localhost") && qdrantUrl.Contains(":"))
        {
            var portIndex = qdrantUrl.LastIndexOf(":", StringComparison.Ordinal);
            var portNumber = int.Parse(qdrantUrl.Substring(portIndex + 1));
            qdrantUrl = qdrantUrl.Substring(0, portIndex + 1) + (portNumber + 1);
        }

        var qdrantApiKey = qdrantConnectionString.Split(";").FirstOrDefault(x => x.Contains("Key="))?.Split("=")[1]
            ?? throw new ArgumentException("Qdrant API key must be provided in the configuration.");

        var qdrantConfig = new QdrantConfig()
        {
            Endpoint = qdrantUrl,
            APIKey = qdrantApiKey,
        };

        // Blob Connection String
        var blobConnection = configuration.GetConnectionString("blob-docing") ?? throw new ArgumentException("Blob Connection String must be provided");

        // Extract Blob account name
        var blobAccountName = blobConnection.Split(".")[0].Split("//")[1];

        var azureBlobsConfig = new AzureBlobsConfig()
        {
            Account = blobAccountName,
            ConnectionString = blobConnection,
            Container = blobContainerName,
            Auth = AzureBlobsConfig.AuthTypes.ManualTokenCredential
        };

        azureBlobsConfig.SetCredential(azureCredentialHelper.GetAzureCredential());

        var textPartitioningOptions = new TextPartitioningOptions
        {
            MaxTokensPerParagraph = PartitionSize,
            OverlappingTokens = 0
        };

        // Get or create KernelMemoryConfig
        var kernelMemoryConfig = serviceProvider.GetService<KernelMemoryConfig>() ?? configuration.GetSection("KernelMemory").Get<KernelMemoryConfig>() ?? new KernelMemoryConfig();

        if (kernelMemoryConfig.DataIngestion.MemoryDbUpsertBatchSize < 20)
        {
            kernelMemoryConfig.DataIngestion.MemoryDbUpsertBatchSize = 20;
        }

        if (!string.IsNullOrEmpty(indexName))
        {
            kernelMemoryConfig.DefaultIndexName = indexName;
        }

        
        var kernelMemoryBuilder = new KernelMemoryBuilder();

        kernelMemoryBuilder.Services.AddSingleton(kernelMemoryConfig);

        kernelMemoryBuilder
            .WithAzureOpenAITextEmbeddingGeneration(openAiEmbeddingConfig)
            .WithAzureOpenAITextGeneration(openAiChatCompletionConfig)
            .WithCustomTextPartitioningOptions(textPartitioningOptions)
            .WithAzureBlobsDocumentStorage(azureBlobsConfig)
            .WithQdrantMemoryDb(qdrantConfig)
            ;

        // Add Logging
        kernelMemoryBuilder.Services.AddLogging(l =>
        {
            l.AddConsole().SetMinimumLevel(LogLevel.Information);
            l.AddConfiguration(configuration);
        });

        var kernelMemory = kernelMemoryBuilder.Build();
        
        return kernelMemory;
    }

As you can see, very little difference except for the MemorDb provider being swapped out.

This is the method that stores content - working today with AI Search - an index is created on AI search if it doesn't exist, and then content is additively stored.

With Qdrant, each time we import a document, the collection is re-created and the last piece of content imported is the only one available.

public async Task StoreContentAsync(string documentLibraryName, string indexName, Stream fileStream,
    string fileName, string? documentUrl, string? userId = null, Dictionary<string, string>? additionalTags = null)
{
    var memory = await GetKernelMemoryForDocumentLibrary(documentLibraryName);

    if (memory == null)
    {
        _logger.LogError("Kernel Memory service not found for Document Library {DocumentLibraryName}", documentLibraryName);
        throw new Exception("Kernel Memory service not found for Document Library " + documentLibraryName);
    }

    // URL encode the documentUrl if it is not null
    if (documentUrl != null)
    {
        documentUrl = WebUtility.UrlEncode(documentUrl);
    }

    // Decode the fileName
    fileName = WebUtility.UrlDecode(fileName);

    // Sanitize the fileName - replace spaces with underscores, pluses with underscores, tildes with underscores, and slashes with underscores
    fileName = fileName.Replace(" ", "_").Replace("+", "_").Replace("~", "_").Replace("/", "_");

    var documentRequest = new DocumentUploadRequest()
    {
        DocumentId = fileName,
        Files = [new DocumentUploadRequest.UploadedFile(fileName, fileStream)],
        Index = indexName

    };

    var isDocumentLibraryDocument = documentLibraryName.StartsWith("Additional-").ToString().ToLowerInvariant();
    var tags = new Dictionary<string, string>
    {
        {"DocumentProcessName", documentLibraryName},
        {"IsDocumentLibraryDocument", isDocumentLibraryDocument},
        {"OriginalDocumentUrl", documentUrl ?? string.Empty},
        {"UploadedByUserOid", userId ?? string.Empty}
    };

    foreach (var (key, value) in tags)
    {
        documentRequest.Tags.Add(key, value);
    }

    if (additionalTags != null)
    {
        // Check the keys in the additionalParameters dictionary. If any of them look like URLs (if they start with https: or http:, URL encode the value
        foreach (var (key, value) in additionalTags)
        {
            if (key.StartsWith("http:") || key.StartsWith("https:"))
            {
                additionalTags[key] = WebUtility.UrlEncode(value);
            }
        }


        foreach (var (key, value) in additionalTags)
        {
            documentRequest.Tags.Add(key, value);
        }
    }

    await memory.ImportDocumentAsync(documentRequest);
}

[pedershk]

Oh, and here's how Qdrant is set up in the Aspire AppHost, you'll notice it's persistent and with a data volume, if that's what you were referring to.

qdrant = builder.AddQdrant("qdrant")
    .WithExternalHttpEndpoints()
    .WithDataVolume("pvico-qdrant-vol")
    .WithLifetime(ContainerLifetime.Persistent);

[russcam]

I'd rewrite the memorydb to use the QdrantClient as a PR, but don't really know enough about Qdrant to do it safely...

I've opened #1068 to discuss.