Skip to content

Commit

Permalink
Update docs about thread safety
Browse files Browse the repository at this point in the history
  • Loading branch information
jnyrup committed Jul 26, 2023
1 parent 54ad0a6 commit cd28678
Show file tree
Hide file tree
Showing 7 changed files with 143 additions and 75 deletions.
12 changes: 12 additions & 0 deletions Src/FluentAssertions/AssertionOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ public static EquivalencyAssertionOptions<T> CloneDefaults<T>()
/// <summary>
/// Allows configuring the defaults used during a structural equivalency assertion.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
/// <param name="defaultsConfigurer">
/// An action that is used to configure the defaults.
/// </param>
Expand All @@ -49,10 +53,18 @@ public static EquivalencyAssertionOptions<T> CloneDefaults<T>()
/// Represents a mutable plan consisting of steps that are executed while asserting a (collection of) object(s)
/// is structurally equivalent to another (collection of) object(s).
/// </summary>
/// <remarks>
/// Members on this property are not thread-safe and should not be invoked from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public static EquivalencyPlan EquivalencyPlan { get; }

/// <summary>
/// Gets the default formatting options used by the formatters in Fluent Assertions.
/// </summary>
/// <remarks>
/// Members on this property should not be invoked from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public static FormattingOptions FormattingOptions { get; } = new();
}
31 changes: 31 additions & 0 deletions Src/FluentAssertions/EquivalencyPlan.cs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ IEnumerator IEnumerable.GetEnumerator()
/// Adds a new <see cref="IEquivalencyStep"/> after any of the built-in steps, with the exception of the final
/// <see cref="SimpleEqualityEquivalencyStep"/>.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked on <see cref="AssertionOptions.EquivalencyPlan"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public void Add<TStep>()
where TStep : IEquivalencyStep, new()
{
Expand All @@ -47,6 +51,10 @@ public void Add<TStep>()
/// <summary>
/// Adds a new <see cref="IEquivalencyStep"/> right after the specified <typeparamref name="TPredecessor"/>.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked on <see cref="AssertionOptions.EquivalencyPlan"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public void AddAfter<TPredecessor, TStep>()
where TStep : IEquivalencyStep, new()
{
Expand All @@ -65,6 +73,10 @@ public void Add<TStep>()
/// <summary>
/// Inserts a new <see cref="IEquivalencyStep"/> before any of the built-in steps.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked on <see cref="AssertionOptions.EquivalencyPlan"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public void Insert<TStep>()
where TStep : IEquivalencyStep, new()
{
Expand All @@ -74,6 +86,10 @@ public void Insert<TStep>()
/// <summary>
/// Inserts a new <see cref="IEquivalencyStep"/> just before the <typeparamref name="TSuccessor"/>.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked on <see cref="AssertionOptions.EquivalencyPlan"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public void InsertBefore<TSuccessor, TStep>()
where TStep : IEquivalencyStep, new()
{
Expand All @@ -92,6 +108,10 @@ public void Insert<TStep>()
/// <summary>
/// Removes all instances of the specified <typeparamref name="TStep"/> from the current step.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked on <see cref="AssertionOptions.EquivalencyPlan"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public void Remove<TStep>()
where TStep : IEquivalencyStep
{
Expand All @@ -101,11 +121,22 @@ public void Remove<TStep>()
/// <summary>
/// Removes each and every built-in <see cref="IEquivalencyStep"/>.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked on <see cref="AssertionOptions.EquivalencyPlan"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public void Clear()
{
steps.Clear();
}

/// <summary>
/// Removes all custom <see cref="IEquivalencyStep"/>s.
/// </summary>
/// <remarks>
/// This method should not be invoked on <see cref="AssertionOptions.EquivalencyPlan"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public void Reset()
{
steps = GetDefaultSteps();
Expand Down
8 changes: 8 additions & 0 deletions Src/FluentAssertions/Formatting/Formatter.cs
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,10 @@ private static void Format(object value, FormattedObjectGraph output, Formatting
/// <summary>
/// Removes a custom formatter that was previously added though <see cref="AddFormatter"/>.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public static void RemoveFormatter(IValueFormatter formatter)
{
CustomFormatters.Remove(formatter);
Expand All @@ -170,6 +174,10 @@ public static void RemoveFormatter(IValueFormatter formatter)
/// <summary>
/// Ensures a custom formatter is included in the chain, just before the default formatter is executed.
/// </summary>
/// <remarks>
/// This method is not thread-safe and should not be invoked from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public static void AddFormatter(IValueFormatter formatter)
{
if (!CustomFormatters.Contains(formatter))
Expand Down
14 changes: 14 additions & 0 deletions Src/FluentAssertions/Formatting/FormattingOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,19 @@ public class FormattingOptions
/// <summary>
/// Indicates whether the formatter should use line breaks when the <see cref="IValueFormatter"/> supports it.
/// </summary>
/// <remarks>
/// This value should not changed on <see cref="AssertionOptions.FormattingOptions"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
public bool UseLineBreaks { get; set; }

/// <summary>
/// Determines the depth until which the library should try to render an object graph.
/// </summary>
/// <remarks>
/// This value should not changed on <see cref="AssertionOptions.FormattingOptions"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </remarks>
/// <value>
/// A depth of 1 will only the display the members of the root object.
/// </value>
Expand All @@ -19,7 +27,13 @@ public class FormattingOptions
/// Sets the maximum number of lines of the failure message.
/// </summary>
/// <remarks>
/// <para>
/// Because of technical reasons, the actual output may be one or two lines longer.
/// </para>
/// <para>
/// This value should not changed on <see cref="AssertionOptions.FormattingOptions"/> from within a unit test.
/// See the <see href="https://fluentassertions.com/extensibility/#thread-safety">docs</see> on how to safely use it.
/// </para>
/// </remarks>
public int MaxLines { get; set; } = 100;

Expand Down
2 changes: 2 additions & 0 deletions docs/_data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ sidebar:
url: /extensibility/#rendering-objects-with-beauty
- title: Customizing Equivalency Assertions
url: /extensibility/#to-be-or-not-to-be-a-value-type
- title: Thread Safety
url: /extensibility/#thread-safety

- title: Tips & Tricks
url: /tips
Expand Down
77 changes: 76 additions & 1 deletion docs/_pages/extensibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,7 @@ Next to the actual value that needs rendering, this method accepts a couple of p
* `context.UseLineBreaks` denotes that the value should be prefixed by a newline. It is used by some assertion code to force displaying the various elements of the failure message on a separate line.
* `formatChild` is used when rendering a complex object that would involve multiple, potentially recursive, nested calls through `Formatter`.

This is what an implementation for the DirectoryInfo would look like.
This is what an implementation for the `DirectoryInfo` would look like.

```csharp
public class DirectoryInfoValueFormatter : IValueFormatter
Expand Down Expand Up @@ -253,3 +253,78 @@ Notice the override of `ToString`. The output of that is included in the message
Another interface, `IMemberMatchingRule`, is used to map a member of the subject to the member of the expectation object with which it should be compared with. It's not something you likely need to implement, but if you do, checkout the built-in implementations `MustMatchByNameRule` and `TryMatchByNameRule`. It receives a `IMember` of the subject's property, the expectation to which you need to map a property, the dotted path to it and the configuration object uses everywhere.

The final interface, the `IOrderingRule`, is used to determine whether FA should be strict about the order of items in collections. The `ByteArrayOrderingRule` is the one used by default, will ensure that FA isn't strict about the order, unless it involves a `byte[]`. The reason behind that is when ordering is treated as irrelevant, FA needs to compare every item in the one collection with every item in the other collection. Each of these comparisons might involve a recursive and nested comparison on the object graph represented by the item. This proved to cause a performance issue with large byte arrays. So I figured that byte arrays are generally used for raw data where ordering is important.

## Thread Safety

The classes `AssertionOptions` and `Formatter` control the global configuration by having static state, so one must be careful when they are mutated.
They are both designed to be configured from a single setup point in your test project and not from within individual unit tests.
Not following this could change the outcome of tests depending on the order they are run in or throw unexpected exceptions when run parallel.

In order to ensure they are configured exactly once, a test framework specific solution might be required depending on the version of .NET you are using.

### .NET 5+

.NET 5 introduced the [`ModuleInitializerAttribute`](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.moduleinitializerattribute) which can be used to setup the defaults _exactly_ once before any tests are run.
```csharp
internal static class Initializer
{
[ModuleInitializer]
public static void SetDefaults()
{
AssertionOptions.AssertEquivalencyUsing(
options => { <configure here> });
}
}
```

### MSTest

MSTest provides the `AssemblyInitializeAttribute` to annotate that a method in a `TestClass` should be run once per assembly.

```csharp
[TestClass]
public static class TestInitializer
{
[AssemblyInitialize]
public static void SetDefaults(TestContext context)
{
AssertionOptions.AssertEquivalencyUsing(
options => { <configure here> });
}
}
```

### xUnit.net

Create a custom [xUnit.net test framework](https://xunit.net/docs/running-tests-in-parallel#runners-and-test-frameworks) where you configure equivalency assertions.
This class can be shared between multiple test projects using assembly references.

```csharp
namespace MyNamespace
{
using Xunit.Abstractions;
using Xunit.Sdk;

public class MyFramework: XunitTestFramework
{
public MyFramework(IMessageSink messageSink)
: base(messageSink)
{
AssertionOptions.AssertEquivalencyUsing(
options => { <configure here> });
}
}
}
```

Add the assembly level attribute so that xUnit.net picks up your custom test framework. This is required for *every* test assembly that should use your custom test framework.

```csharp
[assembly: Xunit.TestFramework("MyNamespace.MyFramework", "MyAssembly.Facts")]
```

Note:

* The `nameof` operator cannot be used to reference the `MyFramework` class. If your global configuration doesn't work, ensure there is no typo in the assembly level attribute declaration and that the assembly containing the `MyFramework` class is referenced by the test assembly and gets copied to the output folder.
* Because you have to add the assembly level attribute per assembly you can define different `AssertionOptions` per test assembly if required.
74 changes: 0 additions & 74 deletions docs/_pages/tips.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,77 +39,3 @@ If you see something missing, please consider submitting a pull request.
{% include assertion-comparison.html header1="MSTest" header2="Fluent Assertions" idPrefix="mstest-" caption="CollectionAssert" examples=site.data.mstest-migration.collectionAssert %}
{% include assertion-comparison.html header1="MSTest" header2="Fluent Assertions" idPrefix="mstest-" caption="StringAssert" examples=site.data.mstest-migration.stringAssert %}
{% include assertion-comparison.html header1="MSTest" header2="Fluent Assertions" idPrefix="mstest-" caption="Exceptions" examples=site.data.mstest-migration.exceptions %}

## Using global AssertionOptions

The `AssertionOptions` class allows you to globally configure how `Should().BeEquivalentTo()` works, see also [Object graph comparison](objectgraphs.md).
Setting up the global configuration multiple times can lead to multi-threading issues when tests are run in parallel.

In order to ensure the global `AssertionOptions` are configured exactly once, a test framework specific solution might be required depending on the version of .NET you are using.

### .NET 5+

.NET 5 introduced the [`ModuleInitializerAttribute`](https://docs.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.moduleinitializerattribute) which can be used to setup the defaults _exactly_ once before any tests are run.

```csharp
internal static class Initializer
{
[ModuleInitializer]
public static void SetDefaults()
{
AssertionOptions.AssertEquivalencyUsing(
options => { <configure here> });
}
}
```

### MSTest

MSTest provides the `AssemblyInitializeAttribute` to annotate that a method in a `TestClass` should be run once per assembly.

```csharp
[TestClass]
public static class TestInitializer
{
[AssemblyInitialize]
public static void SetDefaults(TestContext context)
{
AssertionOptions.AssertEquivalencyUsing(
options => { <configure here> });
}
}
```

### xUnit.net

Create a custom [xUnit.net test framework](https://xunit.net/docs/running-tests-in-parallel#runners-and-test-frameworks) where you configure equivalency assertions.
This class can be shared between multiple test projects using assembly references.

```csharp
namespace MyNamespace
{
using Xunit.Abstractions;
using Xunit.Sdk;

public class MyFramework: XunitTestFramework
{
public MyFramework(IMessageSink messageSink)
: base(messageSink)
{
AssertionOptions.AssertEquivalencyUsing(
options => { <configure here> });
}
}
}
```

Add the assembly level attribute so that xUnit.net picks up your custom test framework. This is required for *every* test assembly that should use your custom test framework.

```csharp
[assembly: Xunit.TestFramework("MyNamespace.MyFramework", "MyAssembly.Facts")]
```

Note:

* The `nameof` operator cannot be used to reference the `MyFramework` class. If your global configuration doesn't work, ensure there is no typo in the assembly level attribute declaration and that the assembly containing the `MyFramework` class is referenced by the test assembly and gets copied to the output folder.
* Because you have to add the assembly level attribute per assembly you can define different `AssertionOptions` per test assembly if required.

0 comments on commit cd28678

Please sign in to comment.