Reduce expression collection and LightExpression construction costs

README.md

@@ -40,6 +40,8 @@ Moreover the compiled delegate may be slower than the manually created delegate
 _TL;DR;_
 > Expression.Compile creates a DynamicMethod and associates it with an anonymous assembly to run it in a sand-boxed environment. This makes it safe for a dynamic method to be emitted and executed by partially trusted code but adds some run-time overhead.
 
+FEC keeps owned DynamicMethods for ordinary lambdas. Methods containing exception regions use anonymous hosting to match System's non-Exception throw and catch policy. The APIs that skip collection also use anonymous hosting, because their exception regions are unknown before emission. Anonymous hosting adds eager JIT work when creating the delegate. `CompileFastToIL` inherits the host policy of the caller's method.
+
 See also [a deep dive to Delegate internals](https://mattwarren.org/2017/01/25/How-do-.NET-delegates-work/#different-types-of-delegates).
 
 
@@ -263,17 +265,34 @@ var x = f(new B());
 The idea is to provide the fast compilation for the supported expression types
 and fallback to the system `Expression.Compile()` for the not supported types:
 
+Invoking a runtime `System.Linq.Expressions.Expression<TDelegate>` value compiles that value
+with its `Compile()` method before invoking the resulting delegate, as the system compiler does.
+This runtime operation is distinct from compilation fallback: with
+`CompileFast(ifFastFailedReturnNull: true)`, FEC still compiles the surrounding lambda directly.
+Errors from compiling or executing the runtime expression propagate to its caller.
+Ref-property arguments retain FEC's normal setter copyback after successful invocation.
+The system compiler can lose this copyback when another argument contains an exception
+block; FEC preserves the property mutation instead of reproducing that behavior.
+
+Dynamic expressions use ordinary DLR call sites: each emitted occurrence has its own site,
+reuses binding rules, and reads the current mutable `Target` for every invocation.
+Runtime binder rule compilation is intrinsic DLR behavior; the outer lambda still compiles directly with FEC.
+
+Quoted lambdas and `RuntimeVariables` share live typed cells with compiled outer and sibling lambdas.
+Mutations remain visible after the creating method returns; independent activations keep separate storage.
+Capture-free quoted trees retain their original identity. The closure arrays and direct IL emitter remain in use.
+
 ### What's not supported yet
 
 **FEC does not support yet:** 
 
-- `Quote`
-- `Dynamic`
-- `RuntimeVariables`
+- Captures of by-ref parameters, pointer types, or byref-like values that cannot be stored in a heap cell
+- Reused physical method-tail labels reached with pending operand values or from protected exception regions (strict null / `NotSupported_MethodReturn_Context`)
 - `DebugInfo`
-- `MemberInit` with the `MemberMemberBinding` and the `ListMemberBinding` binding types
 - `NewArrayInit` multi-dimensional array initializer is not supported yet
 
+Nested `MemberInit` member and list bindings are supported. Very deep expression trees still have a known recursive traversal limit; see the [runtime compatibility results](test/FastExpressionCompiler.DotNetRuntimeTests/results/fixes/README.md).
+
 To find what nodes are not supported in your expression you may use the technic described below in the [Diagnostics](#diagnostics) section. 
 
 The compilation is done by traversing the expression nodes and emitting the IL. 

src/FastExpressionCompiler.LightExpression/Expression.cs

@@ -1248,21 +1248,22 @@ public static ElementInit ElementInit(MethodInfo addMethod, params Expression[]
     public static ElementInit ElementInit(MethodInfo addMethod, IEnumerable<Expression> arguments) =>
         ElementInit(addMethod, arguments.AsReadOnlyList());
 
+    [RequiresUnreferencedCode(Trimming.Message)]
+    // The outer tree still consists of LightExpression nodes. A parameter or constant
+    // may hold a System Expression<TDelegate> as runtime data; Invoke compiles that
+    // value and invokes its delegate. This does not mix System nodes into the tree.
+    private static Type GetInvocationReturnType(Expression expression) =>
+        expression is LambdaExpression directLambda ? directLambda.ReturnType :
+        expression is UnaryExpression quote && quote.NodeType == ExpressionType.Quote && quote.Operand is LambdaExpression lambda
+            ? lambda.ReturnType
+            : (expression.Type.FindExpressionCompileMethod()?.ReturnType ?? expression.Type).FindDelegateInvokeMethod().ReturnType;
+
     public static InvocationExpression Invoke(LambdaExpression expression) =>
         new NotNullExpressionInvocationExpression(expression);
 
     public static InvocationExpression Invoke(LambdaExpression expression, Expression a0) =>
         new OneArgumentInvocationExpression(expression, a0);
 
-    [MethodImpl((MethodImplOptions)256)]
-    [RequiresUnreferencedCode(Trimming.Message)]
-    private static Type GetInvocationReturnType(Expression expression) =>
-        expression is LambdaExpression lambda
-            ? lambda.ReturnType
-            : expression is UnaryExpression unExpr && unExpr.NodeType == ExpressionType.Quote && unExpr.Operand is LambdaExpression quotedLambda
-                ? quotedLambda.ReturnType
-                : expression.Type.FindDelegateInvokeMethod().ReturnType;
-
     [RequiresUnreferencedCode(Trimming.Message)]
     public static InvocationExpression Invoke(Expression expression, Expression a0) =>
         expression is LambdaExpression
@@ -1475,12 +1476,13 @@ public static BinaryExpression Modulo(Expression left, Expression right, MethodI
         GetArithmeticBinary(ExpressionType.Modulo, left, right, method);
 
     /// <summary>Creates a BinaryExpression that represents a bitwise right-shift operation.</summary>
+    [RequiresUnreferencedCode(Trimming.Message)]
     public static BinaryExpression RightShift(Expression left, Expression right) =>
-        new LeftTypedBinaryExpression(ExpressionType.RightShift, left, right);
+        GetShiftBinary(ExpressionType.RightShift, left, right);
 
     /// <summary>Creates a BinaryExpression that represents a bitwise right-shift operation.</summary>
     [RequiresUnreferencedCode(Trimming.Message)]
-    public static BinaryExpression RightShift(Expression left, Expression right, MethodInfo method) => GetLeftTypedBinary(ExpressionType.RightShift, left, right, method);
+    public static BinaryExpression RightShift(Expression left, Expression right, MethodInfo method) => GetShiftBinary(ExpressionType.RightShift, left, right, method);
 
     /// <summary>Creates a BinaryExpression that represents an arithmetic subtraction operation that does not have overflow checking.</summary>
     [RequiresUnreferencedCode(Trimming.Message)]
@@ -1552,9 +1554,9 @@ public static BinaryExpression And(Expression left, Expression right) =>
 
     /// <summary>Creates a BinaryExpression that represents a conditional AND operation that evaluates the second operand only if the first operand evaluates to true.</summary>
     public static BinaryExpression AndAlso(Expression left, Expression right) =>
-        left.Type == typeof(bool?)
-            ? new LeftTypedBinaryExpression(ExpressionType.AndAlso, left, right)
-            : new LogicalBinaryExpression(ExpressionType.AndAlso, left, right);
+        left.Type == typeof(bool)
+            ? new LogicalBinaryExpression(ExpressionType.AndAlso, left, right)
+            : new LeftTypedBinaryExpression(ExpressionType.AndAlso, left, right);
 
     /// <summary>Creates a BinaryExpression that represents a conditional AND operation that evaluates the second operand only if the first operand evaluates to true.</summary>
     [RequiresUnreferencedCode(Trimming.Message)]
@@ -1578,9 +1580,9 @@ public static BinaryExpression GetLogicalBinary(ExpressionType nodeType, Express
 
     /// <summary>Creates a BinaryExpression that represents a conditional OR operation that evaluates the second operand only if the first operand evaluates to false.</summary>
     public static BinaryExpression OrElse(Expression left, Expression right) =>
-        left.Type == typeof(bool?)
-            ? new LeftTypedBinaryExpression(ExpressionType.OrElse, left, right)
-            : new LogicalBinaryExpression(ExpressionType.OrElse, left, right);
+        left.Type == typeof(bool)
+            ? new LogicalBinaryExpression(ExpressionType.OrElse, left, right)
+            : new LeftTypedBinaryExpression(ExpressionType.OrElse, left, right);
 
     /// <summary>Creates a BinaryExpression that represents a conditional OR operation that evaluates the second operand only if the first operand evaluates to false.</summary>
     [RequiresUnreferencedCode(Trimming.Message)]
@@ -2909,6 +2911,13 @@ internal static Expression ConvertToLightExpression(SysExpr sysExpr, ref SmallLi
                         vars[i] = (ParameterExpression)sysVars[i].ToLightExpression(ref exprsConverted);
                     return new RuntimeVariablesExpression(vars);
                 }
+            case ExpressionType.TypeIs:
+            case ExpressionType.TypeEqual:
+                var typeTest = (System.Linq.Expressions.TypeBinaryExpression)sysExpr;
+                var tested = typeTest.Expression.ToLightExpression(ref exprsConverted);
+                return nodeType == ExpressionType.TypeIs
+                    ? Expression.TypeIs(tested, typeTest.TypeOperand)
+                    : Expression.TypeEqual(tested, typeTest.TypeOperand);
             case ExpressionType.DebugInfo:
                 {
                     var die = (System.Linq.Expressions.DebugInfoExpression)sysExpr;
@@ -3035,12 +3044,25 @@ internal static PropertyInfo FindProperty(this Type type, string propertyName)
     [RequiresUnreferencedCode(Trimming.Message)]
     internal static FieldInfo FindField(this Type type, string fieldName)
     {
-        var fields = type.GetTypeInfo().DeclaredFields.AsArray();
-        for (var i = 0; i < fields.Length; i++)
-            if (fields[i].Name == fieldName)
-                return fields[i];
+        var info = type.GetTypeInfo();
+        if (fieldName == null)
+            return null;
+        try
+        {
+            var field = info.GetDeclaredField(fieldName);
+            if (field != null)
+                return field;
+        }
+        catch (AmbiguousMatchException)
+        {
+            // CLR metadata can contain same-name fields with different types. Preserve
+            // the existing first-declared-match behavior for such non-C# types.
+            foreach (var field in info.DeclaredFields)
+                if (field.Name == fieldName)
+                    return field;
+        }
 
-        return type.GetTypeInfo().BaseType?.FindField(fieldName);
+        return info.BaseType?.FindField(fieldName);
     }
 
     internal const BindingFlags InstanceMethods = BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic;
@@ -3515,7 +3537,8 @@ public bool IsLifted
         [RequiresUnreferencedCode(Trimming.Message)]
         get =>
             NodeType != ExpressionType.Coalesce & NodeType != ExpressionType.Assign &&
-            Left.Type.IsNullable() &&
+            (Left.Type.IsNullable() ||
+                (NodeType == ExpressionType.LeftShift || NodeType == ExpressionType.RightShift) && Right.Type.IsNullable()) &&
             (Method == null || !Method.GetParameters()[0].ParameterType.GetNonRef().IsEquivalentTo(Left.Type));
     }
 
@@ -5956,18 +5979,19 @@ internal TypedReturnManyParametersExpression(Expression body, IReadOnlyList<Para
         : base(body, parameters) => ReturnType = returnType;
 }
 
-// todo: @feature is not supported yet
 public sealed class DynamicExpression : Expression
 {
     public override ExpressionType NodeType => ExpressionType.Dynamic;
-    public override Type Type => typeof(object);
+    public override Type Type { get; }
     public Type DelegateType { get; }
     public CallSiteBinder Binder { get; }
     public IReadOnlyList<Expression> Arguments { get; }
 
+    [RequiresUnreferencedCode(Trimming.Message)]
     public DynamicExpression(Type delegateType, CallSiteBinder binder, IReadOnlyList<Expression> arguments)
     {
         DelegateType = delegateType;
+        Type = delegateType.FindDelegateInvokeMethod().ReturnType;
         Binder = binder;
         Arguments = arguments;
     }