There should not be an extra requirement on the application just because it uses /SUBSYSTEM:WINDOWS. The important difference here is process lifetime: from an interactive Windows session, PowerShell can return immediately after launching a GUI-subsystem executable. That matches the immediate prompt you observed. The > redirection then does not keep the same foreground pipeline alive the way application.exe | Out-File file.log does.

For a GUI-subsystem application, make both redirection and waiting explicit:

$process = Start-Process -FilePath '.\application.exe' `
    -RedirectStandardOutput '.\file.log' `
    -RedirectStandardError '.\error.log' `
    -PassThru

$process.WaitForExit()

Or, equivalently for this case:

Start-Process -FilePath '.\application.exe' `
    -RedirectStandardOutput '.\file.log' `
    -RedirectStandardError '.\error.log' `
    -Wait

Start-Process is the PowerShell API intended for controlling native-process execution and has explicit -RedirectStandardOutput, -RedirectStandardError, and -Wait parameters: https://learn.microsoft.com/powershell/scripting/learn/shell/running-commands#running-native-commands

Since the same executable and writes work under cmd.exe, and piping to Out-File also works in PowerShell, your minimal repro is good evidence of a PowerShell redirection/lifetime edge case rather than a missing CRT step in the application. I would file it as a PowerShell issue and include the PowerShell version, the small application source, and whether Start-Process -Wait -RedirectStandardOutput succeeds.

If you use GetFileType on the handle from GetStdHandle(STD_OUTPUT_HANDLE) in your windows app you should see that with file redirection under CMD.EXE you get a FILE_TYPE_DISK, but under both PowerShell.exe and pwsh.exe you get FILE_TYPE_PIPE.

So under CMD.EXE it opened the file and gave the handle to the child process, where as PowerShell says here is a pipe, write your output and I will make sure it goes to the right place.

It looks like the problem is that PowerShell is not waiting for end-of-file from reading the pipe before closing the file. But from PowerShell's point of view its unit of work was to launch the GUI program, not launch and wait for it to finish.

That said, there are a few facts on the ground

1. The fact that GUI apps are spawned without waiting for them to finish goes back to 16-bit Windows when it was merely a DOS extender. There was no CreateProcess API in Win16, it hooked into the DOS int 0x21 exec call to launch windows programs. Because of the nature of a non-preemptive system a DOS program could not wait for a Windows program to complete.
2. OS/2 and POSIX do not exhibit this, neither care if a program is a GUI program or not. OS/2 CMD.EXE would wait for a PM program to finish.
3. There is a whole lot on shenanigans going on to maintain this where a program has to look at the PE header to determine if it should call WaitForSingleObject/GetExitCodeProcess or not.
4. PowerShell is not bash or cmd.exe, its primary communications mechanism between pipeline components is streams of objects, not streams of bytes. The job of PowerShell is to be PowerShell, not bash or cmd.exe. There are a number of occasions where PowerShell has tried to pretend to be bash and dug a deeper hole for itself, eg with the aliases and the tilde.
5. PowerShell doesn't even pretend to implement "<" file redirection, it's not its job.

I would like to stand back a little, and first ask if pipes and GUI programs should be supported at all. It is an edge case. Then I would suggest looking at all the uses cases for pipes and programs and see how they should work. Eg

cmdlet | winapp | cmdlet
cmdlet | winapp | consoleapp | cmdlet
cmdlet | winapp > file
cmdlet | winapp | consoleapp > file
cmdlet | winapp | cmdlet > file
cmdlet | ( winapp ) > file
consoleapp | winapp > file

etc, if you get the drift.

How do the pipes work and what is the lifespan of each process? An observation I would make is the boundaries between components should be based on end-of-stream/end-of-file of the connecting pipes not the life-span of process at each stage. The only process you need to wait for is the one at the end of the pipeline, if it is not a GUI program and not redirected to another file.

At the moment I would suggest that closing the file on a redirect should depend on reading until end-of-file from the source. If that effectively means waiting until the GUI process exits then so be it. The mechanism to determine that should be when ReadFile on the pipe returns zero length or error, not by WaitForSingleObject on the process. When the process ends there can still be unread bytes in the pipe.

I feel like I'm probably replying to two robots but I'll reiterate the key point just in case there's a real human reading this.

application.exe | Out-File file.log works exactly the same for both /SUBSYSTEM:CONSOLE and /SUBSYSTEM:WINDOWS binaries, it's just application > file.log that shows a difference.

If the problem were simply just that PowerShell always closed pipes as soon as it launched GUI apps or that the problem was intractable and nothing could ever be done about it, piping to Out-File would be just as broken. As PowerShell is already giving a FILE_TYPE_PIPE when > is used, anything that relies on it being a FILE_TYPE_DISK is already broken, and wouldn't get any new symptoms from > being implemented by feeding the pipe to an implicit instance of the mechanism behind Out-File. The code already exists. The only novel bit would potentially by spinning it off into its own thread or subprocess so that control can be returned to the shell immediately.

I had a little play, and noticed a few things

1. pwsh on Linux will wait for the output to be written when you run ./t0.sh > t0.txt even though the initial process had completed
2. Doing nmake test will wait for the output from the SUBSYSTEM:WINDOWS applications
3. Nothing at all is written under pwsh.exe with SUBSYSTEM:WINDOWS, I thought the first write might appear with and then it give up when t3 entered the message loop

The apps are

1. t0.sh - shell script which forks a process to continue the write
2. t1.c - a console app
3. t2.c - a windows app written in linear form with no windows API
4. t3.c - a windows app with a message dispatcher loop

In summary, nmake was a surprise, and different pwsh behaviour between Linux and Windows