Fix test hooks for `CommandLineParameterParser` by daxian-dbw · Pull Request #13459 · PowerShell/PowerShell · GitHub
Skip to content

Fix test hooks for CommandLineParameterParser#13459

Merged
rjmholt merged 3 commits into
PowerShell:masterfrom
daxian-dbw:hostbreak
Aug 21, 2020
Merged

Fix test hooks for CommandLineParameterParser#13459
rjmholt merged 3 commits into
PowerShell:masterfrom
daxian-dbw:hostbreak

Conversation

@daxian-dbw

@daxian-dbw daxian-dbw commented Aug 16, 2020

Copy link
Copy Markdown
Member

PR Summary

xUnit tests fail when running with Debug configuration after #11482. The failed tests are those related to WindowsStyle_With_Right_Value because ConsoleControl.SetConsoleMode actually gets called during testing but there is no console window, so the assert fails.

This PR moves the call to ConsoleControl.SetConsoleMode out of the argument parser.

PR Context

PR Checklist

@daxian-dbw daxian-dbw requested a review from anmenaga as a code owner August 16, 2020 22:15
@ghost ghost assigned TravisEz13 Aug 16, 2020
@daxian-dbw daxian-dbw requested review from iSazonov and rjmholt August 16, 2020 22:15
@daxian-dbw daxian-dbw assigned rjmholt and unassigned TravisEz13 Aug 16, 2020

@iSazonov iSazonov left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention was to decouple completely the parser from other code - parser should only parse. I think it is better to move the ConsoleControl.SetConsoleMode() out to ConsoleHost and SettingsFile too.
Also I believe it is better to turn on a test hook only for test where it is used.

@daxian-dbw

Copy link
Copy Markdown
Member Author

@iSazonov

Copy link
Copy Markdown
Collaborator

ConsoleHost is owner of the parser (s_cpp), it defines how use results of command line parsing so I believe all logic should be in ConsoleHost, not in the parser.
It would be surprising if the PowerShell parser started compiling or interpreting or making WMI queries.

@daxian-dbw

daxian-dbw commented Aug 17, 2020

Copy link
Copy Markdown
Member Author

I don't think it really matters to make CommandLineParser a pure argument parser that doesn't do anything else, and I can imagine that future changes to it for additional flags may likely add simple side effect operations to it. I don't see that's a problem, as long as the new flags are testable with test-hook properties.

@iSazonov

Copy link
Copy Markdown
Collaborator

I don't see that's a problem

Oh, I spent many time to debug new tests because the parser has side effects. If you catch the issue again it confirms that the design is not so good and the worst thing is that this again provokes the addition of workarounds not only in tests but also in the main code. This is what I tried to avoid in my PR. Sorry for long discussion.

@daxian-dbw

daxian-dbw commented Aug 18, 2020

Copy link
Copy Markdown
Member Author

I spent many time to debug new tests because the parser has side effects.

It reveals the fact that the code was not written in a testable way, and hence I added the constructor for testing purpose and the _isForTesting field.

Again, I personally don't care that much to make the argument parser a pure parser without side effect, and it's likely we cannot hold that true in future changes, unless you can catch and review all subsequent changes to the argument parser :).

For now, I care more about fixing the test failure in debug build, so I will just go with your approach for this PR :)

@rjmholt

rjmholt commented Aug 18, 2020

Copy link
Copy Markdown
Collaborator

It reveals the fact that the code was not written in a testable way, and hence I added the constructor for testing purpose and the _isForTesting field.

Again, I personally don't care that much to make the argument parser a pure parser without side effect, and it's likely we cannot hold that true in future changes, unless you can catch and review all subsequent changes to the argument parser :).

To express a personal preference, I think:

  • Testability and purity are closely related, and both also help code quality and robustness
  • Using flags to change side-effects into testable state means testing code paths and production code paths aren't the same
  • My ideal would be for the argument parser to process the string[] args input and spit out an object that we could either test or send to the consuming ConsoleHost component. Similar to this .NET library or the prevailing Rust library. (System.CommandLine operates at a different level, so doesn't really compare here)

With that said, I know we ultimately want to minimise code churn and maximise performance, so an "ideal" design may not be the right move here

@iSazonov

Copy link
Copy Markdown
Collaborator

I know we ultimately want to minimise code churn and maximise performance

As side value, after decoupling the parser we can easy measure its performances with dotnet test.

As side thought, we do xUnit tests based exclusively on SMA and we could test decoupling (independent from SMA) code in separate xUnit tests to check the algorithms used.

@iSazonov

iSazonov commented Aug 19, 2020

Copy link
Copy Markdown
Collaborator

The Windows test fails

[-] Pre-Requisistes link for 'WMF 4.0' is reachable: https://www.microsoft.com/download/details.aspx?id=40855

The link does not work https://www.microsoft.com/download/details.aspx?id=40855

I think it is important for Windows Installer.

/cc @TravisEz13 @adityapatwardhan @SteveL-MSFT

@rjmholt

rjmholt commented Aug 19, 2020

Copy link
Copy Markdown
Collaborator

The Windows test fails

We're working on this. Unclear why it's failing

@daxian-dbw

Copy link
Copy Markdown
Member Author

That test was disabled by #13479. The failure was tracked by #13478

@iSazonov iSazonov added the CL-CodeCleanup Indicates that a PR should be marked as a Code Cleanup change in the Change Log label Aug 19, 2020
@iSazonov iSazonov added this to the 7.1.0-preview.7 milestone Aug 19, 2020
@iSazonov iSazonov removed this from the 7.1.0-preview.7 milestone Aug 21, 2020
@daxian-dbw

Copy link
Copy Markdown
Member Author

@rjmholt I think this one is ready to merge.

@rjmholt rjmholt merged commit 6615fa3 into PowerShell:master Aug 21, 2020
@daxian-dbw daxian-dbw deleted the hostbreak branch August 21, 2020 19:12
@iSazonov iSazonov added this to the 7.1.0-preview.7 milestone Aug 23, 2020
@ghost

ghost commented Sep 8, 2020

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CL-CodeCleanup Indicates that a PR should be marked as a Code Cleanup change in the Change Log

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants