Postgres Create(...)` fix on identity columns,

docs/RELEASE_NOTES.md

@@ -1,3 +1,8 @@
+### 1.5.29 - 02.11.2026
+- Fix: parameters of a sub-query nested in another sub-query are bound (IN, EXISTS, UNION and the like)
+- Fix: groupBy keys with option or value-option columns, and with x.IsSome / x.IsNone in a tuple key
+- PostgreSQL: identity columns count as generated, so the typed ``Create(...)`` no longer takes them. Breaking for code that passed a value to a BY DEFAULT identity column: use Create() and set it, or Create(data). A saved ContextSchemaPath cache keeps the old signature until regenerated.
+
 ### 1.5.28 - 17.10.2026
 - Fix for IndexOutOfRangeException when invoking a stored procedure, #878
 - Code cleanups with fsharp-refactor tool

docs/content/core/composable.fsx

@@ -170,8 +170,9 @@ query { for (i,n,e) in qry1 do
 ### Nested Select Where Queries
 
 You can create a query like `SELECT * FROM xs WHERE xs.x IN (SELECT y FROM ys))`
-with either LINQ Contains or custom operators: in `|=|` and not-in `|<>|`
-This is done by not saying `|> Seq.toArray` to the first query:
+in the purist way with the F# operators `x |=| qry` (NOT IN: `x |<>| qry`), or in the pragmatist way with LINQ's
+`.Contains x` (NOT IN: `not (ys.Contains x)`), which also runs in the mock context, so the query can be
+unit-tested. Both give the same SQL. The example takes the pragmatist way:
 
 *)
 

docs/content/core/crud.fsx

@@ -34,10 +34,24 @@ let now = DateTime.Now
 
 (**
 Create() has various overloads to make inserting records simple.
+
+Prefer the typed \`\`Create(...)\`\`: its name lists the required columns (not nullable, not generated, sorted by
+name), so adding a required column to the table breaks the build where rows are created instead of failing at
+runtime. Set the optional columns afterwards. Call it with the backticked name, so two arguments of the same type
+can't swap places unnoticed when the column list changes.
+*)
+
+let emp = ctx.Main.Employees.``Create(FirstName, LastName)``("Don", "Syme")
+
+(**
+Optional columns are set on the returned row, e.g. `emp.Title <- "Founder"` (or `ValueSome "Founder"` with
+`UseOptionTypes = NullableColumnType.VALUE_OPTION`). A key the application generates, like a UUID, is set the same
+way: `row.Id <- Guid.CreateVersion7()`.
 *)
 
 (**
-Create a new row
+The untyped `Create()` with property sets is for when you build rows dynamically. It compiles even when a required
+column is left out:
 *)
 
 let row = orders.Create()
@@ -61,14 +75,19 @@ ctx.SubmitUpdates()
 (**
 After updating, your item (row) will have the Id property.
 
+To update a row without reading it first, create an entity with its key, set the columns, and mark only those
+columns as modified:
 
-
-You can also create with the longer \`\`Create(...)\`\`(parameters)-method like this:
+```fsharp
+let update = ctx.Main.Orders.Create()
+update.OrderId <- orderId
+update.ShipCity <- "Leeds"
+update._State <- Common.EntityState.Modified [ "ShipCity" ]
+ctx.SubmitUpdates()
+```
 
 *)
 
-let emp = ctx.Main.Employees.``Create(FirstName, LastName)``("Don", "Syme")
-
 (**
 
 Delete the row

docs/content/core/general.fsx

@@ -144,6 +144,10 @@ a relevant query using the entity's primary key.
 
 ## Basic Querying
 The SQL provider supports LINQ queries using F#'s *query expression* syntax.
+
+The examples here select whole entities to keep them short. In application code, select only the columns you use,
+into a tuple, an anonymous record or your domain type, and filter, sort and aggregate in the query: see
+[recommended query style](querying.html#Recommended-query-style).
 *)
 
 let customersQuery =

docs/content/core/querying.fsx

@@ -54,10 +54,73 @@ let example =
         sortBy order.ShipPostalCode
         skip 3
         take 4
-        select order
+        select (order.OrderId, order.ShipPostalCode, order.Freight)
     }
 
-let test = example |> Seq.toArray |> Array.map(fun i -> i.ColumnValues |> Map.ofSeq)
+let test = example |> Seq.toArray
+
+//If you would need the full-width order as map, you could select it and then do |> Array.map(fun i -> i.ColumnValues |> Map.ofSeq)
+
+(**
+### Recommended query style
+
+The query is translated to SQL, so the database does the filtering, sorting, joining, grouping and counting. Write
+the business rule in the query, and keep what comes back small:
+
+- **Select what you use, typed.** Project to a tuple, an anonymous record `{| ... |}` or a domain record, not the
+  whole entity (`select order`). Use a tuple or an anonymous record when the result is used on the spot; when it
+  leaves the function (returned from an API, stored, shared) build your real domain type in the `select`. Whole
+  entities are for rows you are about to modify. Don't pass `ColumnValues` maps around as application data.
+- **Filter and sort in the query, not after it.** `where (x.Removed.IsNone)` and `sortBy x.Created` in the query
+  instead of `Array.filter` / `Array.sortBy` on the fetched rows. Counts, minimums, maximums and existence are
+  queries too: `Seq.lengthAsync`, `groupBy ... into g; select (g.Key, g.Max(...))`, or `Seq.tryHeadAsync` on a
+  query that selects one column.
+- **"IN" over a sub-query.** When the ids come from the database, don't fetch them first: leave the inner query
+  unexecuted (no `Seq.toArray`) and use it directly, so one SQL statement runs (see "Nested Select Where Queries" in
+  [composable queries](composable.html)). There are two ways to write the IN, and they give the same SQL:
+
+  - The purist way, F# only: the operators `|=|` (IN) and `|<>|` (NOT IN).
+  - The pragmatist way: LINQ's `.Contains` extension method (`open System.Linq`), and `not (xs.Contains x)` for NOT
+    IN. It is familiar to anyone who knows LINQ, and it runs in the [mock data context](unittest.html), so the query
+    can be unit-tested; the operators don't run there.
+
+*)
+
+let activeCustomers = query { for c in ctx.Main.Customers do where (c.Country = "UK"); select c.CustomerId }
+
+// The purist way, no LINQ reference
+let xs1 =
+    query { for o in ctx.Main.Orders do
+            where (o.CustomerId |=| activeCustomers)
+            select (o.OrderId, o.Freight) }
+
+// The pragmatist way, easeier to read, easier to unit-test
+open System.Linq
+let xs2 =
+    query { for o in ctx.Main.Orders do
+            where (activeCustomers.Contains o.CustomerId)
+            select (o.OrderId, o.Freight) }
+(**
+
+  F#'s own `Seq.contains` is not translated to SQL.
+- **One query for every case of a filter.** Compute flags outside the query and test them inside it, so the same
+  query can serve "all" and "one":
+
+*)
+// query parameters:
+let allCountries = false // or true
+let onlyCountry = "Finland" // or ""
+
+query { for c in ctx.Main.Customers do
+        where (allCountries || c.Country = onlyCountry)
+        select c.CustomerId }
+(**
+
+- **No N+1.** Don't run a query per row of another query. Join, or read the children of all the parents in one
+  query (`parentIds.Contains child.ParentId`, with `parentIds` a sub-query) and group them in memory.
+- **Pass the data context in.** Functions that take the context as a parameter can be tested with the mock context.
+
+*)
 
 let item =
     query {
@@ -416,8 +479,8 @@ let bergs = ctx.Main.Customers.Individuals.BERGS
 
 You can find some custom operators `using FSharp.Data.Sql`:
 
-* `|=|` (In set)
-* `|<>|` (Not in set)
+* `|=|` (In set: the purist way; LINQ's `xs.Contains x` is the pragmatist way, gives the same SQL and also runs in the mock context)
+* `|<>|` (Not in set; with LINQ, `not (xs.Contains x)`)
 * `=%` (Like)
 * `<>%` (Not like)
 * `!!` (Left join: the joined entity is populated with default values when there is no match)

docs/content/core/unittest.fsx

@@ -151,6 +151,28 @@ There are some limitations with the SQLProvider mock a DB-context:
 - You can call database-table `.Create` methods to create new instances (it doesn't connect the database). You can call update entity columns `x.Col <- Some "hi"`, but it doesn't really do anything.
 - You cannot call stored procedures.
 - Names are database names, and they are case-sensitive. If you miss a table, in your mock, there will be a clear error. If you mistyped the anonymous record column name, you will probably just get a zero-result or ValueNone.Value-error or some other unwanted behaviour.
+- Every table the code under test reads must be in the map, with at least one row: the mock learns a table's columns from its rows. Give a table that should match nothing a row that belongs to something else.
+
+Test the paths through your parametrised queries this way: one mock, and a call per value of each flag
+(`all || x.Id = one`, `not openOnly || ...`), checking which rows come back.
+
+### Where the mock lives
+
+Keep the mock in the test project: the production code holds no mock database. Writes need nothing special, since
+each one makes a new context inside its own transaction. For reads, give the code the mock in one of two ways:
+
+- **Pass the context in.** Read functions take the read-only context as a parameter; production passes the shared
+  one and the test passes the mock.
+- **Swap the shared holder.** When read functions take the shared read-only context from a module-level holder (see
+  [CRUD](crud.html) on sharing it and rebuilding it after a connection failure), put an empty marker interface in
+  that module (`type ContextHolderMarker = interface end`). The test finds the module through
+  `typeof<ContextHolderMarker>.DeclaringType`, replaces the holder with a mock by reflection, and puts the original
+  back when it finishes.
+
+A test project that names the provided context type directly (`sql.readDataContext`) can get type mismatch errors
+such as FS0193, because a provided type named from another assembly is a different type. Name it through a type
+abbreviation in the assembly that declares the provider (`type ReadDataContext = sql.readDataContext`) and use that
+in the test.
 
 If you are running an off-line solution like SSDT or ContextSchemaPath, you should be able to run also these unit-tests with your CI.
 

docs/content/index.fsx

@@ -125,9 +125,10 @@ let mattisOrderDetails =
             // nullable columns can be represented as option types; the following generates IS NOT NULL
             where o.ShipCountry.IsSome
             // standard operators will work as expected; the following shows the like operator and IN operator
+            // ...or you can use "open System.Linq" and do .Contains if that feels more intuitive
             where (c.ContactName =% ("Matti%") && c.CompanyName |=| [|"Squirrelcomapny";"DaveCompant"|] )
             sortBy o.ShipName
-            // arbitrarily complex projections are supported
+            // select only what you use: arbitrarily complex projections are supported
             select (c.ContactName,o.ShipAddress,o.ShipCountry,prod.ProductName,prod.UnitPrice) }
     |> Seq.toArray
 

src/SQLProvider.Common/SqlRuntime.Common.fs

@@ -1015,6 +1015,15 @@ module CommonTasks =
                             // manually and in that case scope_identity would bring back 0 "" or whatever
         | false, _ -> ()
 
+    /// The select-list entry of a group-key column. In a key of several columns, a computed one (a CASE for x.IsSome or
+    /// x.IsNone) is aliased key<position>: left unaliased, two of them come back under one result name (PostgreSQL and
+    /// SQL Server do not name them by their text), and the key tuple cannot be rebuilt. Every other key column keeps
+    /// `plain`, the provider's usual entry.
+    let groupKeyColumn (keys:(alias*SqlColumnType) list) (index:int) (column:SqlColumnType) (notation:string) (plain:string) =
+        match column with
+        | CanonicalOperation _ when keys.Length > 1 -> sprintf "%s as key%i" notation index
+        | _ -> plain
+
     let parseHaving fieldNotation (keys:(alias*SqlColumnType) list) (conditionList : Condition list) =
         if keys.Length <> 1 then
             failwithf "Unsupported having. Expected 1 key column, found: %i" keys.Length

src/SQLProvider.Common/SqlRuntime.Linq.fs

@@ -44,7 +44,34 @@ module internal QueryImplementation =
         abstract GetAsyncEnumerator : unit -> System.Threading.Tasks.Task<IEnumerator<'T>>
     type IAsyncEnumerable =
         abstract EvaluateQuery : unit -> System.Threading.Tasks.Task<unit>
-
+
+    let private memberwiseClone = typeof<obj>.GetMethod("MemberwiseClone", BindingFlags.Instance ||| BindingFlags.NonPublic)
+
+    /// Copy of a command parameter under a new name. The source parameter is not modified.
+    let private copyParameterWithName (name:string) (p:IDbDataParameter) : IDbDataParameter =
+        let copy =
+            match p with
+            | :? ICloneable as c -> c.Clone() :?> IDbDataParameter
+            | _ ->
+                // Field copy, not property copy: getters such as Size report defaults for unset values,
+                // and setting them explicitly changes the binding (e.g. Size = 0 truncates a SqliteParameter value).
+                memberwiseClone.Invoke(p, [||]) :?> IDbDataParameter
+        copy.ParameterName <- name
+        copy
+
+    /// Renames the parameters of a sub-query so they cannot collide with the parameters of the enclosing query.
+    /// Parameter objects stored in the sub-query (including those of its own nested sub-queries) are shared by
+    /// every query that embeds it, so renamed copies are returned instead of mutating them.
+    let internal renameNestedParameters (nestedPrefix:string) (query:string) (parameters:IDbDataParameter seq) =
+        let renamed =
+            parameters
+            |> Seq.map(fun p -> copyParameterWithName (p.ParameterName.Replace("@param", nestedPrefix)) p)
+            |> Seq.toArray
+        let subquery =
+            let paramfixed = query.Replace("@param", nestedPrefix)
+            if paramfixed.EndsWith ";" then paramfixed.Substring(0, paramfixed.Length-1) else paramfixed
+        subquery, renamed
+
     /// Try to detect if IQueryable is a database query.
     /// It can be e.g. direct SQLProvider IWithSqlService (great)
     /// or annoyingly a source inside a wrapper like WhereSelectEnumerableIterator, where it needs to be updated
@@ -649,17 +676,8 @@ module internal QueryImplementation =
                         let ``nested param names`` = $"@param{abs(query.GetHashCode())}{nestCount}nested"
                         nestCount <- nestCount + 1
 
-                        let modified = 
-                            parameters
-                            |> Seq.filter(fun p -> not(p.ParameterName.StartsWith ``nested param names``))
-                            |> Seq.map(fun p ->
-                                p.ParameterName <- p.ParameterName.Replace("@param", ``nested param names``)
-                                p
-                            ) |> Seq.toArray
-                        let subquery = 
-                            let paramfixed = query.Replace("@param", ``nested param names``)
-                            if paramfixed.EndsWith ";" then paramfixed.Substring(0, paramfixed.Length-1) else paramfixed
-
+                        let subquery, modified = renameNestedParameters ``nested param names`` query parameters
+
                         Some(ti,key,op,Some (box (subquery, modified)))
                     | SqlExistsClause(meth,op,src,qual)
                     | SqlNotExistsClause(meth,op,src,qual) ->
@@ -731,17 +749,8 @@ module internal QueryImplementation =
                         let ``nested param names`` = $"@param{abs(query.GetHashCode())}{nestCount}nested"
                         nestCount <- nestCount + 1
 
-                        let modified = 
-                            parameters 
-                            |> Seq.filter(fun p -> not(p.ParameterName.StartsWith ``nested param names``))
-                            |> Seq.map(fun p ->
-                                p.ParameterName <- p.ParameterName.Replace("@param", ``nested param names``)
-                                p
-                            ) |> Seq.toArray
-                        let subquery = 
-                            let paramfixed = query.Replace("@param", ``nested param names``)
-                            if paramfixed.EndsWith ";" then paramfixed.Substring(0, paramfixed.Length-1) else paramfixed
-
+                        let subquery, modified = renameNestedParameters ``nested param names`` query parameters
+
                         Some("",KeyColumn(""),op,Some (box (subquery, modified)))
                     | SimpleCondition ((ti,key,op,c) as x) -> 
                         match c with
@@ -847,10 +856,25 @@ module internal QueryImplementation =
                         | "", _ -> ""
                         | _ -> resolveTuplePropertyName ti source.TupleIndex
 
-                let keycols = 
+                let keycols =
                     match exp with
                     | SqlColumnGet(sourceTi,sourceKey,_) -> [getAlias sourceTi, sourceKey]
-                    | TupleSqlColumnsGet itms -> itms |> List.map(fun (ti,key,typ) -> getAlias ti, key)
+                    | OptionalFSharpOptionValue(NewExpr(cons, args))
+                            when (cons.DeclaringType.Name.StartsWith("Tuple") || cons.DeclaringType.Name.StartsWith("AnonymousObject")) ->
+                        // Every tuple item becomes one key column, in order: the key tuple is rebuilt from them.
+                        let keyItem = function
+                            | SqlColumnGet(ti,key,_) -> Some (getAlias ti, key)
+                            | SimpleCondition (ti,key,op,None) ->
+                                let al = getAlias ti
+                                let cond = And ([al, key, op, None], None)
+                                Some (al, CanonicalOperation(CaseSqlPlain(cond, 1, 0), KeyColumn "Key"))
+                            | _ -> None
+                        let items = args |> List.map(fun a -> a, keyItem a)
+                        if items |> List.forall (snd >> Option.isNone) then []
+                        else
+                            items |> List.map(function
+                                | _, Some col -> col
+                                | item, None -> raise (NotSupportedException $"groupBy: the key item '{item}' is not supported. A group key item must be a column or a column test without a value such as x.IsSome or x.IsNone."))
                     | SimpleCondition ((ti,key,op,None) as x) -> // Because you can do in F# but not in SQL: GroupBy (x.IsSome)
                         let al = getAlias sourceTi
                         let cond = And ([ti, key, op,None], None)
@@ -1397,16 +1421,7 @@ module internal QueryImplementation =
 
                         let ``nested param names`` = $"@param{abs(query.GetHashCode())}nested"
 
-                        let modified = 
-                            parameters
-                            |> Seq.filter(fun p -> not(p.ParameterName.StartsWith ``nested param names``))
-                            |> Seq.map(fun p ->
-                                p.ParameterName <- p.ParameterName.Replace("@param", ``nested param names``)
-                                p
-                            ) |> Seq.toArray
-                        let subquery = 
-                            let paramfixed = query.Replace("@param", ``nested param names``)
-                            if paramfixed.EndsWith ";" then paramfixed.Substring(0, paramfixed.Length-1) else paramfixed
+                        let subquery, modified = renameNestedParameters ``nested param names`` query parameters
 
                         //let ty = typedefof<SqlQueryable<_>>.MakeGenericType(meth.GetGenericArguments().[0])
                         let utyp =