Improve failure message for string assertions when checking for equality by vbreuss · Pull Request #2307 · fluentassertions/fluentassertions · GitHub
Skip to content

Improve failure message for string assertions when checking for equality#2307

Merged
dennisdoomen merged 15 commits into
fluentassertions:developfrom
vbreuss:topic/improve-string-messages
Oct 4, 2023
Merged

Improve failure message for string assertions when checking for equality#2307
dennisdoomen merged 15 commits into
fluentassertions:developfrom
vbreuss:topic/improve-string-messages

Conversation

@vbreuss

@vbreuss vbreuss commented Sep 18, 2023

Copy link
Copy Markdown
Contributor

When asserting that two strings are equal, and one of the strings is longer than 8 characters or contains a NewLine, instead of simply stating at which index the first difference occurs, this difference is made visible using arrows, e.g.

var subject = "this is a long text that differs in between";
var expected = "this was too short";
subject.Should().Be(expected, "because we use arrows now");

throws with message

Expected subject to be the same string because we use arrows now, but they differ at index 5:
        ↓ (actual)
  "this is a long text…"
  "this was too short"
        ↑ (expected)."

For multi-line strings, instead of only providing information about the index, the failure message also contains the line and column information, e.g. …but they differ on line 5 and column 1 (index 93):

For short single-line strings the behaviour remains unchanged!

This closes #2050.

IMPORTANT

  • If the PR touches the public API, the changes have been approved in a separate issue with the "api-approved" label.
  • The code complies with the Coding Guidelines for C#.
  • The changes are covered by unit tests which follow the Arrange-Act-Assert syntax and the naming conventions such as is used in these tests.
  • If the PR adds a feature or fixes a bug, please update the release notes with a functional description that explains what the change means to consumers of this library, which are published on the website.
  • If the PR changes the public API the changes needs to be included by running AcceptApiChanges.ps1 or AcceptApiChanges.sh.
  • If the PR affects the documentation, please include your changes in this pull request so the documentation will appear on the website.
    • Please also run ./build.sh --target spellcheck or .\build.ps1 --target spellcheck before pushing and check the good outcome

@github-actions

github-actions Bot commented Sep 18, 2023

Copy link
Copy Markdown

@vbreuss vbreuss marked this pull request as ready for review September 27, 2023 11:53
@vbreuss vbreuss changed the title Draft: Improve failure message for string assertions when checking for equality Improve failure message for string assertions when checking for equality Sep 27, 2023

@dennisdoomen dennisdoomen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a pretty impressive improvement. I had some quite some issues understanding the algorithm, so I've gone through a pretty thorough review.

Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Common/StringExtensions.cs Outdated
Comment thread Src/FluentAssertions/Primitives/StringEqualityStrategy.cs Outdated
Comment thread Src/FluentAssertions/Primitives/StringEqualityStrategy.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
@vbreuss

vbreuss commented Oct 2, 2023

Copy link
Copy Markdown
Contributor Author

That's a pretty impressive improvement. I had some quite some issues understanding the algorithm, so I've gone through a pretty thorough review.

@dennisdoomen. Thanks for the extensive review. It's really appreciated!

I implemented your requested changes and answered your questions. I hope I didn't overlook anything :-)
I also added this change to the releases.md

@jnyrup jnyrup left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really great!

I haven't gone through all the relational operators to check if they are correct.
I'll try to run mutation tests on StringEqualityStrategy.cs sometime this week to help verifying this. If you want to try mutation tests before I get to it, there are some pointers in #1068 and jnyrup#6

Comment thread Src/FluentAssertions/Primitives/StringEqualityStrategy.cs Outdated
Comment thread Src/FluentAssertions/Primitives/StringEqualityStrategy.cs Outdated
Comment thread Src/FluentAssertions/Primitives/StringEqualityStrategy.cs Outdated
Comment thread Tests/FluentAssertions.Specs/FluentAssertions.Specs.csproj Outdated
@ITaluone

ITaluone commented Oct 3, 2023

Copy link
Copy Markdown
Contributor

I'll try to run mutation tests on StringEqualityStrategy.cs sometime this week to help verifying this.

Shall we add that to our pipeline?

@jnyrup

jnyrup commented Oct 3, 2023

Copy link
Copy Markdown
Member

I'll try to run mutation tests on StringEqualityStrategy.cs sometime this week to help verifying this.

Shall we add that to our pipeline?

That's my plan.
It takes quite a while to do a full run, so my current idea is to only consider the changed files in PRs and then do a scheduled full run e.g. once a week.

@vbreuss

vbreuss commented Oct 3, 2023

Copy link
Copy Markdown
Contributor Author

I'll try to run mutation tests on StringEqualityStrategy.cs sometime this week to help verifying this. If you want to try mutation tests before I get to it, there are some pointers in #1068 and jnyrup#6

I also use Stryker in my project and just ran it against fluentassertions

Mutation Testing Summary

File Score Killed Survived Timeout No Coverage Ignored Compile Errors Total Detected Total Undetected Total Mutants
Primitives\StringEqualityStrategy.cs 97,12% 101 3 0 0 21 8 101 3 133

I will add tests for the 3 survived mutants together with the changes for your review :-)

@vbreuss

vbreuss commented Oct 3, 2023

Copy link
Copy Markdown
Contributor Author

@jnyrup
I implemented your review comments and added the missing tests for the surviving mutants.

Mutation Testing Summary

File Score Killed Survived Timeout No Coverage Ignored Compile Errors Total Detected Total Undetected Total Mutants
Primitives\StringEqualityStrategy.cs 100,00% 103 0 0 0 20 8 103 0 131

@dennisdoomen dennisdoomen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I said before, great work. The new comments and improved naming really helps (even though it is still quite an algorithm).

Since we've tried to get rid of words like when and should in test names and focus more on the fact that such a test covers, I've suggested some improvements. Up to you if you want to include them now. Otherwise, I'll do it at a later point.

Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Comment thread Tests/FluentAssertions.Specs/Primitives/StringAssertionSpecs.Be.cs Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve failure message for StringAssertions.Be

6 participants