Skip to content

CSHARP-6017: Support LINQ LeftJoin for joins and includes#2017

Open
damieng wants to merge 5 commits into
mongodb:mainfrom
damieng:csharp6017
Open

CSHARP-6017: Support LINQ LeftJoin for joins and includes#2017
damieng wants to merge 5 commits into
mongodb:mainfrom
damieng:csharp6017

Conversation

@damieng
Copy link
Copy Markdown
Contributor

@damieng damieng commented May 29, 2026

Adds support for left outer join semantics in the LINQ3 provider, translating to a $project$lookup$unwind$project aggregation pipeline.

Motivation

  • We need left-join semantics for include and join in our EF Core Provider
  • .NET 10 adds Queryable.LeftJoin to the BCL which we'll want to translate
  • MongoQueryable.LeftJoin is also added so users on earlier runtimes can use the same API

Public API

// IMongoCollection overload (convenience)
orders.AsQueryable()
    .LeftJoin(customers, o => o.CustomerId, c => c.Id, (o, c) => new { o, c })

// IQueryable overload (also handles Queryable.LeftJoin on .NET 10+)
orders.AsQueryable()
    .LeftJoin(customers.AsQueryable(), o => o.CustomerId, c => c.Id, (o, c) => new { o, c })

// Typed result (navigation-property style)
orders.AsQueryable()
    .LeftJoin(customers, o => o.CustomerId, c => c.Id,
        (o, c) => new LeftJoinResult<Order, Customer> { Outer = o, Inner = c })

LeftJoinResult<TOuter, TInner> is an immutable struct (matching the LookupResult convention) whose Inner is null when no matching document exists.

Translation

{ $project: { _outer: "$$ROOT", _id: 0 } }
{ $lookup:  { from: "customers", localField: "_outer.CustomerId", foreignField: "_id", as: "_inner" } }
{ $unwind:  { path: "$_inner", preserveNullAndEmptyArrays: true } }
{ $project: { ... result selector ... } }

preserveNullAndEmptyArrays: true on the $unwind is the single line that distinguishes a left join from an inner join. Outer documents with no matching inner survive with _inner: null.

Notes

  • Chaining — multiple LeftJoin calls compose naturally; each re-wraps the current pipeline output in _outer so subsequent localField paths resolve correctly.
  • Fan-out — multiple inner matches produce multiple result rows (same as SQL left join / $unwind semantics).
  • Collision detection — throws ExpressionNotSupportedException if the outer document type has a BSON element already named _outer or _inner.
  • Server version — uses the simple localField/foreignField $lookup form, which requires MongoDB 3.2+. The driver already targets 3.6+, so no additional version guard is needed.

@damieng damieng added the feature Adds new user-facing functionality. label May 29, 2026
@damieng damieng marked this pull request as ready for review May 29, 2026 17:08
@damieng damieng requested a review from a team as a code owner May 29, 2026 17:08
@damieng damieng requested a review from adelinowona May 29, 2026 17:08
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread src/MongoDB.Driver/Linq/MongoQueryable.cs Outdated
Comment thread src/MongoDB.Driver/Linq/LeftJoinResult.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread src/MongoDB.Driver/Linq/LeftJoinResult.cs Outdated
public TOuter Outer { get; init; }

/// <summary>
/// The inner document (null when no matching inner document exists).
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The doc comment says Inner is "null when no matching inner document exists," but TInner is unconstrained. For a value-type inner (e.g. LeftJoinResult<Order, int>) a no-match row deserializes the missing _inner to default(TInner) (0), not null — so "no match" and a real zero become indistinguishable. Either qualify the comment or steer TInner toward reference/nullable types.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oleks says we shouldn't be introducing NRTs at this point so I guess we'll handle this when we do that.

Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment on lines +85 to +86
? AstStage.Unwind("_inner", preserveNullAndEmptyArrays: true)
: AstStage.Unwind("_inner");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: this ternary can collapse — AstStage.Unwind already defaults preserveNullAndEmptyArrays to null, so AstStage.Unwind("_inner", preserveNullAndEmptyArrays: isLeftJoin ? true : (bool?)null) does the same thing without duplicating the "_inner" literal.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copilot AI review requested due to automatic review settings June 4, 2026 13:27
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds LINQ3 support for left outer join semantics via a new LeftJoin API and corresponding pipeline translation (wrapping the outer document, performing $lookup, and $unwind with preserveNullAndEmptyArrays: true). This extends join/include scenarios (including upcoming .NET Queryable.LeftJoin) and introduces a LeftJoinResult<TOuter, TInner> helper result type.

Changes:

  • Introduces MongoQueryable.LeftJoin overloads and LeftJoinResult<TOuter, TInner>.
  • Extends LINQ3 pipeline translation to recognize and translate LeftJoin, including chained joins.
  • Adds integration/smoke tests covering translation shape, chaining, reserved-name collisions, and round-tripping.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
tests/SmokeTests/MongoDB.Driver.SmokeTests.Sdk/LeftJoinTests.cs Net10 smoke test validating basic left-join behavior against a live server.
tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs LINQ3 integration tests for translation stages, chaining, collision checks, and serialization round-trips.
src/MongoDB.Driver/Linq/MongoQueryable.cs Adds public LeftJoin extension methods for IMongoCollection and IQueryable inner sources.
src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToPipelineTranslators/JoinMethodToPipelineTranslator.cs Implements $lookup + $unwind(preserveNullAndEmptyArrays:true) translation and handles filtered inner queryables.
src/MongoDB.Driver/Linq/Linq3Implementation/Translators/ExpressionToPipelineTranslators/ExpressionToPipelineTranslator.cs Routes LeftJoin method calls to the join translator.
src/MongoDB.Driver/Linq/Linq3Implementation/SerializerFinders/SerializerFinderVisitMethodCall.cs Adds serializer deduction support for LeftJoin.
src/MongoDB.Driver/Linq/Linq3Implementation/Reflection/QueryableMethod.cs Adds reflection handle for BCL Queryable.LeftJoin (when available).
src/MongoDB.Driver/Linq/Linq3Implementation/Reflection/MongoQueryableMethod.cs Adds reflection handle for MongoQueryable.LeftJoin.
src/MongoDB.Driver/Linq/LeftJoinResult.cs New LeftJoinResult<TOuter, TInner> result struct with nullable Inner.
CSHARP-6017-review-findings.md Adds a review-findings document (currently out of sync with the implementation in this PR).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +71 to +97
string innerCollectionName;
IBsonSerializer innerSerializer;
AstPipeline innerFilterPipeline = null;

if (innerExpression is ConstantExpression)
{
(innerCollectionName, innerSerializer) = innerExpression.GetCollectionInfoFromQueryable(containerExpression: expression);
}
else
{
var rootInnerExpression = GetRootQueryableExpression(innerExpression);
(innerCollectionName, innerSerializer) = rootInnerExpression.GetCollectionInfoFromQueryable(containerExpression: expression);
var innerTranslation = ExpressionToPipelineTranslator.Translate(context, innerExpression);
innerSerializer = innerTranslation.OutputSerializer;
innerFilterPipeline = innerTranslation.Ast.Stages.Count > 0 ? innerTranslation.Ast : null;
}

var localField = outerKeySelectorLambda.TranslateToDottedFieldName(context, wrappedOuterSerializer);
var foreignField = innerKeySelectorLambda.TranslateToDottedFieldName(context, innerSerializer);

var lookupStage = AstStage.Lookup(
from: innerCollectionName,
localField,
foreignField,
@as: "_inner");
var lookupStage = innerFilterPipeline != null
? AstStage.Lookup(innerCollectionName, localField, foreignField, [], innerFilterPipeline, "_inner")
: AstStage.Lookup(from: innerCollectionName, localField, foreignField, @as: "_inner");

var unwindStage = AstStage.Unwind("_inner");
var unwindStage = isLeftJoin
? AstStage.Unwind("_inner", preserveNullAndEmptyArrays: true)
: AstStage.Unwind("_inner");
Comment thread src/MongoDB.Driver/Linq/MongoQueryable.cs Outdated
Comment thread src/MongoDB.Driver/Linq/MongoQueryable.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread CSHARP-6017-review-findings.md Outdated
Comment thread src/MongoDB.Driver/Linq/LeftJoinResult.cs Outdated
Comment thread src/MongoDB.Driver/Linq/LeftJoinResult.cs Outdated
Comment thread CSHARP-6017-review-findings.md Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated
Comment thread tests/MongoDB.Driver.Tests/Linq/Linq3Implementation/Jira/CSharpLeftJoinTests.cs Outdated

namespace MongoDB.Driver.Tests.Linq.Linq3Implementation.SerializerFinders;

public class LeftJoinTests
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding this tests. Looks really good. The only thing - we might want to merge these test cases into existing EnumerableTests, where other tests for Quaryable lives.


namespace MongoDB.Driver.Tests.Linq.Integration;

public class CSharpLeftJoinTests : LinqIntegrationTest<CSharpLeftJoinTests.ClassFixture>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for moving the file here. Please remove CSharp from the name, and name it QueryableLeftJoingTests instead.

new OrderDetail { Id = 100, OrderId = 1, ProductId = "P1" },
new OrderDetail { Id = 101, OrderId = 2, ProductId = "P2" }]);
#if NET10_0_OR_GREATER
BlogsCollection = CreateCollection<Blog>("blogs");
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we put some data into the collection?

throw new ExpressionNotSupportedException(expression);
}

private static Expression GetRootQueryableExpression(Expression expression)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check if TranslationContext.GetUltimateSource should be used here instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature Adds new user-facing functionality.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants