Repository navigation
Milvus: move the provider to Milvus.Client.V2 - #93
N-E-W-T-O-N wants to merge 1 commit into
Conversation
Port the Microsoft.Extensions.VectorData Milvus provider from the legacy Milvus.Client to Milvus.Client.V2 3.0.0 (MilvusClientV2 and its request/response API). - Collections take a MilvusClientV2; the host-based registrations build a ConnectConfig from the host, port, database and API key or username/password. - Collection creation sends the schema, the vector and BM25 indexes in one CreateCollection call, then loads the collection and waits for it with LoadTimeout. - Each index is named after its field: Milvus.Client.V2 3.0.0 gives every unnamed index the same default name, and Milvus rejects the second one. - Upserts send rows (UpsertReq.RowsData); queries and searches read the returned columns back into rows. - Hybrid search uses AnnSearchReq with text input for BM25, ranked with RrfReranker(60). - gRPC failures now reach the error handler as MilvusException, with the gRPC status kept. Milvus.Client.V2 cannot take a caller-owned GrpcChannel: a client always owns its channel, and tests route calls through ConnectConfig.ChannelOptions instead.
|
@dotnet-policy-service agree |
Laurianti
left a comment
There was a problem hiding this comment.
Thanks for moving the provider to Milvus.Client.V2. I ran the PR at b2bc2a3 against Milvus standalone v2.6.25 and have three comments, the first one blocking.
A search with no results throws. The PR removes SearchAllowingEmptyResultsAsync because milvus-io/milvus-sdk-csharp#198 fixes the SDK. That fix was merged on 11 November, but Milvus.Client.V2 3.0.0 was released on 23 October and does not contain it. With 3.0.0, a search or hybrid search that matches nothing throws NotSupportedException: None not supported from DqlConversions.FromGrpcFieldData.
- Conformance tests on this PR: 19 of 256 fail, all with this error. On
main(74935f4), same server: 256 of 256 pass. - Smallest case: upsert one record, delete it, search. It throws instead of returning an empty list.
The inline suggestions keep the workaround, adapted to the V2 types, until an SDK release includes the fix. With them applied: Release build clean, unit tests 127 of 127, conformance tests 256 of 256. Removing the workaround can then be a follow-up PR that also bumps the package.
The other two comments are inline.
| Limit = top, | ||
| Parameters = parameters, | ||
| }; | ||
| var results = await RunOperationAsync("Search", () => _clientWrapper.Client.SearchAsync(request, cancellationToken)).ConfigureAwait(false); |
There was a problem hiding this comment.
| var results = await RunOperationAsync("Search", () => _clientWrapper.Client.SearchAsync(request, cancellationToken)).ConfigureAwait(false); | |
| var results = await RunOperationAsync("Search", () => SearchAllowingEmptyResultsAsync( | |
| () => _clientWrapper.Client.SearchAsync(request, cancellationToken), parameters.OutputFields)).ConfigureAwait(false); |
| Limit = limit, | ||
| Parameters = parameters, | ||
| }; | ||
| var results = await RunOperationAsync("HybridSearch", () => _clientWrapper.Client.HybridSearchAsync(request, cancellationToken)).ConfigureAwait(false); |
There was a problem hiding this comment.
| var results = await RunOperationAsync("HybridSearch", () => _clientWrapper.Client.HybridSearchAsync(request, cancellationToken)).ConfigureAwait(false); | |
| var results = await RunOperationAsync("HybridSearch", () => SearchAllowingEmptyResultsAsync( | |
| () => _clientWrapper.Client.HybridSearchAsync(request, cancellationToken), parameters.OutputFields)).ConfigureAwait(false); |
| // Retry without output fields to verify that this really is an empty search; | ||
| // do not hide the SDK error if the retry finds any records. | ||
| private static async Task<SearchResults> SearchAllowingEmptyResultsAsync(Func<Task<SearchResults>> search, IList<string> outputFields) | ||
| private static bool IsEmpty(SearchResp results) => (results.Ids.LongIds?.Count ?? results.Ids.StringIds?.Count ?? 0) == 0; |
There was a problem hiding this comment.
The workaround, on the V2 types. It retries without output fields only for this exact SDK error, and rethrows if the retry finds any records, so it cannot hide a different failure.
| private static bool IsEmpty(SearchResp results) => (results.Ids.LongIds?.Count ?? results.Ids.StringIds?.Count ?? 0) == 0; | |
| // TODO: Remove this workaround once a Milvus.Client.V2 release includes the fix: | |
| // https://github.com/milvus-io/milvus-sdk-csharp/pull/198 | |
| // Milvus.Client.V2 3.0.0 cannot decode the output fields of an empty response. | |
| // Retry without output fields to verify that this really is an empty search; | |
| // do not hide the SDK error if the retry finds any records. | |
| private static async Task<SearchResp> SearchAllowingEmptyResultsAsync(Func<Task<SearchResp>> search, IList<string> outputFields) | |
| { | |
| try | |
| { | |
| return await search().ConfigureAwait(false); | |
| } | |
| catch (NotSupportedException exception) when (exception.Message == "None not supported") | |
| { | |
| outputFields.Clear(); | |
| var retry = await search().ConfigureAwait(false); | |
| if (!IsEmpty(retry)) | |
| { | |
| throw; | |
| } | |
| return retry; | |
| } | |
| } | |
| private static bool IsEmpty(SearchResp results) => (results.Ids.LongIds?.Count ?? results.Ids.StringIds?.Count ?? 0) == 0; |
| var keyName = _model.KeyProperty.StorageName; | ||
| for (var i = 0; i < results.Scores.Count; i++) | ||
| { | ||
| outputFields.Clear(); | ||
| var retry = await search().ConfigureAwait(false); | ||
| if (!IsEmpty(retry)) | ||
| { | ||
| throw; | ||
| } | ||
| return retry; | ||
| var row = i < rows.Count ? rows[i] : new Dictionary<string, object?>(StringComparer.Ordinal); |
There was a problem hiding this comment.
If the response has fewer rows than scores, the missing row becomes an empty dictionary, and the caller gets a record with the right key and default values everywhere else, with no error. The key is always among the output fields, so a healthy response has one row per hit; a mismatch is better reported than filled in. The conformance tests still pass 256 of 256 with this change.
| var keyName = _model.KeyProperty.StorageName; | |
| for (var i = 0; i < results.Scores.Count; i++) | |
| { | |
| outputFields.Clear(); | |
| var retry = await search().ConfigureAwait(false); | |
| if (!IsEmpty(retry)) | |
| { | |
| throw; | |
| } | |
| return retry; | |
| var row = i < rows.Count ? rows[i] : new Dictionary<string, object?>(StringComparer.Ordinal); | |
| var keyName = _model.KeyProperty.StorageName; | |
| if (rows.Count != results.Scores.Count) | |
| { | |
| throw new InvalidOperationException($"Milvus returned {rows.Count} rows for {results.Scores.Count} search hits."); | |
| } | |
| for (var i = 0; i < results.Scores.Count; i++) | |
| { | |
| var row = rows[i]; |
There was a problem hiding this comment.
A unit test for this suggestion, as a new file MEVD/test/Milvus.UnitTests/MilvusSearchResponseTests.cs. It answers the Search call with a hand-built response: two hits and a single row. It fails without the suggestion ("No exception was thrown") and passes with it; with all the suggestions of this review applied, the unit tests pass 129 of 129.
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.VectorData;
using Xunit;
namespace CommunityToolkit.VectorData.Milvus.UnitTests;
public sealed class MilvusSearchResponseTests
{
[Fact]
public async Task SearchThrowsWhenTheResponseHasFewerRowsThanHits()
{
// Two hits, but the output fields hold a single row.
using var handler = new SearchHandler(ids: ["a", "b"], rowIds: ["a"]);
using var client = TestClients.FromHandler(handler);
using var collection = new MilvusCollection<string, KeyOnlyRecord>(client, "records", ownsClient: false);
await Assert.ThrowsAsync<InvalidOperationException>(
async () => await collection.SearchAsync(new ReadOnlyMemory<float>([1f, 0f, 0f]), 2).ToListAsync());
}
[Fact]
public async Task SearchReturnsOneRecordPerHit()
{
using var handler = new SearchHandler(ids: ["a", "b"], rowIds: ["a", "b"]);
using var client = TestClients.FromHandler(handler);
using var collection = new MilvusCollection<string, KeyOnlyRecord>(client, "records", ownsClient: false);
var results = await collection.SearchAsync(new ReadOnlyMemory<float>([1f, 0f, 0f]), 2).ToListAsync();
Assert.Equal(["a", "b"], results.Select(r => r.Record.Id));
}
public sealed class KeyOnlyRecord
{
[VectorStoreKey]
public string Id { get; set; } = string.Empty;
[VectorStoreVector(3)]
public ReadOnlyMemory<float> Vector { get; set; }
}
// Answers Search with a SearchResults message built by hand, and every other gRPC call with an empty status.
private sealed class SearchHandler(string[] ids, string[] rowIds) : HttpMessageHandler
{
protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
var method = request.RequestUri!.AbsolutePath.Split('/').Last();
var message = method == "Search" ? SearchResults() : [10, 0, 16, 1];
var frame = new byte[5 + message.Length];
frame[1] = (byte)(message.Length >> 24);
frame[2] = (byte)(message.Length >> 16);
frame[3] = (byte)(message.Length >> 8);
frame[4] = (byte)message.Length;
message.CopyTo(frame, 5);
var response = new HttpResponseMessage(HttpStatusCode.OK) { Version = new Version(2, 0), Content = new ByteArrayContent(frame) };
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/grpc");
response.TrailingHeaders.Add("grpc-status", "0");
return Task.FromResult(response);
}
// milvus.proto.milvus.SearchResults { status = 1; results = 2 } with
// milvus.proto.schema.SearchResultData { num_queries = 1; top_k = 2; fields_data = 3; scores = 4; ids = 5; topks = 6 }.
private byte[] SearchResults()
{
var keyColumn = Concat(
Field(1, Varint(21)), // type: VarChar
Field(2, Bytes(Encoding.UTF8.GetBytes("Id"))), // field_name
Field(3, Bytes(Field(6, Bytes(Strings(rowIds)))))); // scalars.string_data
var scores = Concat(ids.Select((_, i) => BitConverter.GetBytes(1f - (0.1f * i))).ToArray());
var data = Concat(
Field(1, Varint(1)),
Field(2, Varint((ulong)ids.Length)),
Field(3, Bytes(keyColumn)),
Field(4, Bytes(scores)),
Field(5, Bytes(Field(2, Bytes(Strings(ids))))), // ids.str_id
Field(6, Bytes(Varint((ulong)ids.Length).Item2)));
return Concat(Field(1, Bytes([])), Field(2, Bytes(data)));
}
private static byte[] Strings(IEnumerable<string> values)
=> Concat(values.Select(v => Field(1, Bytes(Encoding.UTF8.GetBytes(v)))).ToArray());
private static byte[] Field(int number, (int WireType, byte[] Payload) value)
=> Concat(Varint((ulong)((number << 3) | value.WireType)).Item2, value.Payload);
private static (int, byte[]) Varint(ulong value)
{
var bytes = new List<byte>();
do
{
var b = (byte)(value & 0x7F);
value >>= 7;
bytes.Add(value == 0 ? b : (byte)(b | 0x80));
}
while (value != 0);
return (0, bytes.ToArray());
}
private static (int, byte[]) Bytes(byte[] value) => (2, Concat(Varint((ulong)value.Length).Item2, value));
private static byte[] Concat(params byte[][] parts) => parts.SelectMany(p => p).ToArray();
}
}| - For NativeAOT, use dynamic dictionary collections with explicit definitions: the typed collection APIs use reflection. Publishing with trimming or NativeAOT shows the warnings `IL2104` and `IL3053` of the dependency `Milvus.Client.V2`. | ||
| - On .NET Framework, `Milvus.Client.V2` calls Milvus over WinHTTP (`WinHttpHandler`, which `Grpc.Net.Client` brings in). This needs: | ||
| - A TLS endpoint: `https: true` (the default) on the host-based registrations, or an `https://` URI in the `ConnectConfig` of a `MilvusClientV2`. A plaintext endpoint fails with `Response protocol downgraded to HTTP/1.1`. | ||
| - A server certificate that Windows trusts. To validate it another way, set `ConnectConfig.ChannelOptions` to `GrpcChannelOptions` with a `WinHttpHandler` and pass that client to the provider. | ||
| - Windows 11 or later. Windows Server 2019 and 2022 support gRPC over WinHTTP only partially. See [Microsoft's gRPC requirements for .NET Framework](https://learn.microsoft.com/aspnet/core/grpc/netstandard#net-framework). | ||
|
|
||
| Verified on Windows 11 with .NET Framework 4.7.2 against Milvus 2.6.25 over TLS. | ||
| Verified on Windows 11 with .NET Framework 4.7.2 against Milvus 2.6.25 over TLS with the V1 `Milvus.Client`; not yet re-verified with `Milvus.Client.V2`. |
There was a problem hiding this comment.
I ran both checks on this branch with the suggestions of this review applied, on Windows 11 against Milvus 2.6.25. The suggestion below updates the two lines.
NativeAOT and trimming: search fails. A minimal app with a dynamic collection, published for win-x64 with PublishAot or PublishTrimmed (.NET SDK 10.0.401): create, upsert and get work, but SearchAsync throws InvalidOperationException: Reflection-based serialization has been disabled for this application from MilvusClientV2.Combine(SearchParameters). The same app on main (74935f4), published with PublishAot, works. With JsonSerializerIsReflectionEnabledByDefault set to true in the app, search works again, both trimmed and under NativeAOT. The warnings IL2104 and IL3053 are correct; trimming alone shows only IL2104.
.NET Framework: works. A .NET Framework 4.7.2 app over TLS with a self-signed certificate, validated through ConnectConfig.ChannelOptions with a WinHttpHandler: create, upsert, get, search and delete work. A plaintext endpoint fails with Response protocol downgraded to HTTP/1.1, as written. Not checked: a certificate from the Windows trust store, and Windows Server.
| - For NativeAOT, use dynamic dictionary collections with explicit definitions: the typed collection APIs use reflection. Publishing with trimming or NativeAOT shows the warnings `IL2104` and `IL3053` of the dependency `Milvus.Client.V2`. | |
| - On .NET Framework, `Milvus.Client.V2` calls Milvus over WinHTTP (`WinHttpHandler`, which `Grpc.Net.Client` brings in). This needs: | |
| - A TLS endpoint: `https: true` (the default) on the host-based registrations, or an `https://` URI in the `ConnectConfig` of a `MilvusClientV2`. A plaintext endpoint fails with `Response protocol downgraded to HTTP/1.1`. | |
| - A server certificate that Windows trusts. To validate it another way, set `ConnectConfig.ChannelOptions` to `GrpcChannelOptions` with a `WinHttpHandler` and pass that client to the provider. | |
| - Windows 11 or later. Windows Server 2019 and 2022 support gRPC over WinHTTP only partially. See [Microsoft's gRPC requirements for .NET Framework](https://learn.microsoft.com/aspnet/core/grpc/netstandard#net-framework). | |
| Verified on Windows 11 with .NET Framework 4.7.2 against Milvus 2.6.25 over TLS. | |
| Verified on Windows 11 with .NET Framework 4.7.2 against Milvus 2.6.25 over TLS with the V1 `Milvus.Client`; not yet re-verified with `Milvus.Client.V2`. | |
| - For NativeAOT, use dynamic dictionary collections with explicit definitions: the typed collection APIs use reflection. Publishing with trimming or NativeAOT shows the warnings `IL2104` and `IL3053` of the dependency `Milvus.Client.V2`. `Milvus.Client.V2` 3.0.0 serializes the search parameters with reflection-based JSON, which trimmed and NativeAOT apps disable by default: set `<JsonSerializerIsReflectionEnabledByDefault>true</JsonSerializerIsReflectionEnabledByDefault>` in the app project, otherwise `SearchAsync` throws `InvalidOperationException`. | |
| - On .NET Framework, `Milvus.Client.V2` calls Milvus over WinHTTP (`WinHttpHandler`, which `Grpc.Net.Client` brings in). This needs: | |
| - A TLS endpoint: `https: true` (the default) on the host-based registrations, or an `https://` URI in the `ConnectConfig` of a `MilvusClientV2`. A plaintext endpoint fails with `Response protocol downgraded to HTTP/1.1`. | |
| - A server certificate that Windows trusts. To validate it another way, set `ConnectConfig.ChannelOptions` to `GrpcChannelOptions` with a `WinHttpHandler` and pass that client to the provider. | |
| - Windows 11 or later. Windows Server 2019 and 2022 support gRPC over WinHTTP only partially. See [Microsoft's gRPC requirements for .NET Framework](https://learn.microsoft.com/aspnet/core/grpc/netstandard#net-framework). | |
| Verified on Windows 11 with .NET Framework 4.7.2 against Milvus 2.6.25 over TLS with `Milvus.Client.V2`, the certificate validated through `ConnectConfig.ChannelOptions`. |
|
A note for the maintainers, apart from the review. This PR changes more than a dependency: the public API moves from With
My view: this should not be merged until a |
I agree 100%, big thanks for flagging those issues @Laurianti ! |
|
Thanks! For the NativeAOT one I opened milvus-io/milvus-sdk-csharp#204. The empty-search fix (milvus-io/milvus-sdk-csharp#198) is already merged there and waits for a release. |
Summary
This moves the Milvus provider (
CommunityToolkit.VectorData.Milvus, added in #77) from the legacyMilvus.ClienttoMilvus.Client.V23.0.0. The provider's behaviour and its MEVD surface stay the same. The SDK client type it takes changes fromMilvusClienttoMilvusClientV2.Related:
Why Milvus.Client.V2
V1 is in maintenance mode. The milvus-sdk-csharp README marks
Milvus.Clientas legacy and recommendsMilvus.Client.V2. Since the V2 release, new features and fixes land in V2. For example, the fix for decoding empty search results (fix: handle empty result output field placeholders milvus-io/milvus-sdk-csharp#198) is V2-only. Staying on V1 would leave the provider on a client that no longer grows.A client built for long-lived use. #139 lists the V1 problems V2 was written to fix:
RetryConfig); V1 makes a single attempt.MilvusErrorCode, andMilvusExceptioncarries the gRPC status).One configuration object. V1 has nine constructor overloads. V2 takes a single
ConnectConfigwith URI, credentials, database, timeouts, keep-alive, retry, logging and gRPC channel options. Users who bring their own client can now set retry and timeouts there.Request/response API that matches the other official SDKs. V2 mirrors the Java/C++/Rust SDKs (
CreateCollectionReq,SearchReq,HybridSearchReq, …). It also drops V1's stateful per-collection handles, which the provider no longer needs to hold.Room to grow. Several things this provider can't do today depend on V2:
OrderByforGetAsync(filter, …), from feat: fill MilvusClientV2 parity gaps (externalField, typed DQL params, resumable + element-filter iterators) milvus-io/milvus-sdk-csharp#190. It is merged on the SDK's main branch but not in a NuGet release yet.Timestamptzdate/time fields.Each would be its own follow-up PR once the provider is on V2.
Changes
MilvusVectorStore,MilvusCollection<TKey, TRecord>andMilvusDynamicCollectiontake aMilvusClientV2.GetService(typeof(MilvusClientV2))returns it.ConnectConfigfrom the host, port,https, database and the API key or username/password. Their signatures are unchanged.EnsureCollectionExistsAsynccreates the collection with its schema, BM25 functions and indexes in oneCreateCollectionAsynccall. It then loads the collection and waits up toLoadTimeout.Milvus.Client.V23.0.0 gives every unnamed index the same default name,_default_idx, and Milvus then rejects the second index of a collection: "at most one distinct index is allowed per field". Without this, collections with several vectors, or a vector plus full-text search, can't be created. I'll also propose the same default upstream in milvus-sdk-csharp.UpsertReq.RowsData, and the SDK maps nulls using the collection schema. Queries and searches read the returned columns back into rows.AnnSearchReqand one BM25AnnSearchReqwith text input, ranked withRrfReranker(60).Breaking changes
The package is still
1.0.0-preview, but callers who pass their own SDK client need to update:new MilvusClient(...)becomesnew MilvusClientV2(new ConnectConfig { Uri = ..., ApiKey = ..., Database = ... }). Host-based registrations need no change.GrpcChannel: a client always owns its channel. A custom handler (for exampleWinHttpHandleron .NET Framework) goes throughConnectConfig.ChannelOptionsinstead.VectorStoreExceptionfor a gRPC failure is now aMilvusExceptionthat carries the gRPC status, not anRpcException.Testing
Milvus.UnitTests: 127 passed. The fake gRPC handler now also answers V2'sGetLoadStatepoll.Milvus.ConformanceTestson .NET 10 against a Milvus 2.6 server: 256 passed. That run used the same provider code with newer test packages (Testcontainers 4.16, Microsoft.NET.Test.Sdk 18.12).MEVD.slnfsolution builds with 0 warnings and 0 errors.