I understand you are upset but calling the developers names is probably not helping. I agree that either the documentation should note this issue or we should actually try to fix the bug. Having buffered file objects randomly lose data is not a friendly behavior.

Your documentation change is not quite correct, I think. The problem is not that __del__ isn't called, it is that the ordering of the __del__ calls can be unexpected. For this bug, it happens if your buffered file object is part of a garbage reference cycle. The ordering of __del__ calls is essentially random at that point. It can happen that the buffer object gets cleaned up after the file object has been cleaned (closed). In that case, the buffer cannot be written because the file is already closed.

AFAIK, there is no way to fix that in the Python GC logic. We would need to re-structure the _io classes to prevent it (merge the buffer object with the file descriptor object or use a weakref).

I understand you are upset but calling the developers names is probably not helping.

I know, I'm sorry. I'm just tired of this "You should have expected this surprising behavior that is not documented!" victim-blaming. Multiple people told me that the old documentation was correct.

I agree that either the documentation should note this issue or we should actually try to fix the bug. Having buffered file objects randomly lose data is not a friendly behavior.

Thank you for saying that.

Your documentation change is not quite correct, I think. The problem is not that __del__ isn't called, it is that the ordering of the __del__ calls can be unexpected. For this bug, it happens if your buffered file object is part of a garbage reference cycle. The ordering of __del__ calls is essentially random at that point. It can happen that the buffer object gets cleaned up after the file object has been cleaned (closed). In that case, the buffer cannot be written because the file is already closed.

AFAIK, there is no way to fix that in the Python GC logic. We would need to re-structure the _io classes to prevent it (merge the buffer object with the file descriptor object or use a weakref).

Fixing this bug would not prevent data loss, because the docs says that python can exit without calling __del__. The other question is whether

f = open("path")
f.write("data")
del(f)
exit(0)

is guaranteed to write to the disk.

LGTM. But wait a week for another core dev review.

When will "another core dev review" happen?

When will "another core dev review" happen?

No one knows. I thought if someone find this and want to review. But it was not happened in two weeks.

Thanks @Volker-Weissmann for the PR, and @methane for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8, 3.9.
🐍🍒⛏🤖

GH-23527 is a backport of this pull request to the 3.9 branch.

GH-23528 is a backport of this pull request to the 3.8 branch.

Thanks @Volker-Weissmann!

I agree than relying on destructors / garbage collectors to flush data is a bug magnet, so the doc is way better that way: explicit is better.

Next time you find an inconsistency in the doc, don't hesitate to open such good PRs, they help everybody, and it's appreciated (just skip the part you know, but let's no longer speak about it).