Internal CLR error. (0x80131506) in threads.cpp · Issue #110837 · dotnet/runtime · GitHub
Skip to content

Internal CLR error. (0x80131506) in threads.cpp #110837

Closed
#111257
Closed
Internal CLR error. (0x80131506) in threads.cpp#110837
#111257
Assignees
davidwrighton
Labels
area-VM-coreclrin-prThere is an active PR which will close this issue when it is mergedregression-from-last-release

Description

@majocha
majocha
opened on Dec 19, 2024

Description

We're observing random crashes when running xUnit tests in the CI targeting net9.0:
The active test run was aborted. Reason: Test host process crashed : Fatal error. Internal CLR error. (0x80131506)

Reproduction Steps

I have no way yet to reproduce this reliably.

Minidump:
879847-testhost.exe-28560-1732638273.dmp

FILE_IN_CAB:  879847-testhost.exe-28560-1732638273.dmp

NTGLOBALFLAG:  0

EXCEPTION_RECORD:  (.exr -1)
ExceptionAddress: 0000000000000000
   ExceptionCode: 80000003 (Break instruction exception)
  ExceptionFlags: 00000000
NumberParameters: 0

FAULTING_THREAD:  000072e8

PROCESS_NAME:  testhost.dll

ERROR_CODE: (NTSTATUS) 0x80000003 - {EXCEPTION}  Breakpoint  A breakpoint has been reached.

EXCEPTION_CODE_STR:  80000003

STACK_TEXT:  
000000d6`e8f7df78 00007ffa`4a98464c     : aaaaaaaa`aaaaaaaa aaaaaaaa`aaaaaaaa 00000000`00000000 00000255`e9091050 : ntdll!NtWaitForMultipleObjects+0x14
000000d6`e8f7df80 00007ffa`104cd0f0     : aaaaaaaa`aaaaaaaa 00000000`00000000 aaaaaaaa`aaaaaaaa 00000255`e9091120 : KERNELBASE!WaitForMultipleObjectsEx+0xec
000000d6`e8f7e270 00007ffa`104cce05     : 00000000`00000000 000000d6`e8f7e331 00000255`e9091120 00007ffa`108591b0 : coreclr!Thread::DoAppropriateAptStateWait+0x5c
000000d6`e8f7e2b0 00007ffa`104ccc44     : 000000d6`e8f7e490 aaaaaaaa`aaaaaaaa 00000255`e9091120 aaaaaaaa`aaaaaaaa : coreclr!Thread::DoAppropriateWaitWorker+0x171
000000d6`e8f7e380 00007ffa`104ca37d     : 00007ffa`108590c0 aaaaaaaa`aaaaaaaa 00000000`00000000 aaaaaaaa`aaaaaaaa : coreclr!Thread::DoAppropriateWait+0xb0
000000d6`e8f7e420 00007ffa`104ca4ff     : 000000d6`e8f7e6a0 aaaaaaaa`aaaaaaaa aaaaaaaa`aaaaaaaa 000000d6`e8f7e6a0 : coreclr!SyncBlock::Wait+0x1c9
000000d6`e8f7e560 00007ffa`0f902fc5     : 00000000`00000000 aaaaaaaa`aaaaaaaa 00000255`e9091050 aaaaaaaa`aaaaaaaa : coreclr!Monitor_Wait+0x7f
000000d6`e8f7e5d0 00007ffa`0f90fa4f     : 00000255`f2c09f10 aaaaaaaa`aaaaaaaa aaaaaaaa`aaaaaaaa aaaaaaaa`aaaaaaaa : System_Private_CoreLib+0x3a2fc5
000000d6`e8f7e6a0 00007ffa`0f928ff6     : 00000255`f2c09ee8 aaaaaaaa`aaaaaaaa 00000000`00000000 00007ff9`b095fda0 : System_Private_CoreLib+0x3afa4f
000000d6`e8f7e740 00007ffa`0f928dee     : 00000255`f2c09e60 aaaaaaaa`aaaaaaaa 00000000`00000000 00000255`e9091050 : System_Private_CoreLib+0x3c8ff6
000000d6`e8f7e7c0 00007ffa`0f928a2e     : 00000255`f343ee98 aaaaaaaa`aaaaaaaa 00000000`00000000 00000255`e9091050 : System_Private_CoreLib+0x3c8dee
000000d6`e8f7e840 00007ffa`0f9288c2     : 00000255`f343ee98 aaaaaaaa`aaaaaaaa 00000000`00000000 00000255`e9091050 : System_Private_CoreLib+0x3c8a2e
000000d6`e8f7e870 00007ff9`b098b4c7     : 00000255`f343ee98 00000000`00000000 aaaaaaaa`aaaaaaaa 00000255`e9091050 : System_Private_CoreLib+0x3c88c2
000000d6`e8f7e8a0 00000255`f343ee98     : 00000000`00000000 aaaaaaaa`aaaaaaaa 00000255`e9091050 000000d6`e8f7e8a0 : 0x00007ff9`b098b4c7
000000d6`e8f7e8a8 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00000255`f343ee98


STACK_COMMAND:  ~0s; .ecxr ; kb

FAULTING_SOURCE_LINE:  D:\a\_work\1\s\src\coreclr\vm\threads.cpp

FAULTING_SOURCE_FILE:  D:\a\_work\1\s\src\coreclr\vm\threads.cpp

FAULTING_SOURCE_LINE_NUMBER:  3184

FAULTING_SOURCE_SRV_COMMAND:  https://raw.githubusercontent.com/dotnet/runtime/9d5a6a9aa463d6d10b0b0ba6d5982cc82f363dc3/src/coreclr/vm/threads.cpp

FAULTING_SOURCE_CODE:  
No source found for 'D:\a\_work\1\s\src\coreclr\vm\threads.cpp'


SYMBOL_NAME:  coreclr!Thread::DoAppropriateAptStateWait+5c

MODULE_NAME: coreclr

IMAGE_NAME:  coreclr.dll

FAILURE_BUCKET_ID:  BREAKPOINT_80000003_coreclr.dll!Thread::DoAppropriateAptStateWait

OS_VERSION:  10.0.20348.1

BUILDLAB_STR:  fe_release

OSPLATFORM_TYPE:  x64

OSNAME:  Windows 10

IMAGE_VERSION:  9.0.24.52809

FAILURE_ID_HASH:  {e2358c87-f452-7ecb-c918-08173d45a949}

Followup:     MachineOwner
---------

Expected behavior

No crash.

Actual behavior

testhost crashes with Internal CLR error. (0x80131506)

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

This started after enabling test parallelism, so there is much more pressure on the GC and a lot of threads in use during the test run.

Activity

  1. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Dec 19, 2024
  2. dotnet-policy-service
    added
    untriagedNew issue has not been triaged by the area owner
    on Dec 19, 2024

  3. EgorBo commented on Dec 19, 2024

    @EgorBo
    EgorBo
    on Dec 19, 2024 · Hidden as resolved

  4. jkotas commented on Dec 19, 2024

    @jkotas
    jkotas
    on Dec 19, 2024
    Member

    I have opened 879847-testhost.exe-28560-1732638273.dmp from https://github.com/user-attachments/files/17924725/dumps.zip . I see it crashing with null reference exception at this stacktrace:

    coreclr!LoaderAllocator::SetHandleValue+0xb2:
00007ffa`1060ead2 488b4210        mov     rax,qword ptr [rdx+10h] ds:00000000`00000010=????????????????
0:029> k
 # Child-SP          RetAddr               Call Site
00 000000d6`ebbff9d0 00007ffa`1060dc2e     coreclr!LoaderAllocator::SetHandleValue+0xb2 [D:\a\_work\1\s\src\coreclr\vm\loaderallocator.cpp @ 991] 
01 000000d6`ebbffa40 00007ffa`104ed30a     coreclr!LoaderAllocator::FreeHandle+0x1a [D:\a\_work\1\s\src\coreclr\vm\loaderallocator.cpp @ 883] 
02 000000d6`ebbffa70 00007ffa`104ecec2     coreclr!FreeLoaderAllocatorHandlesForTLSData+0xce [D:\a\_work\1\s\src\coreclr\vm\threadstatics.cpp @ 394] 
03 000000d6`ebbffae0 00007ffa`104ec785     coreclr!Thread::OnThreadTerminate+0xa6 [D:\a\_work\1\s\src\coreclr\vm\threads.cpp @ 2838] 
04 000000d6`ebbffb60 00007ffa`104c5225     coreclr!DestroyThread+0x55 [D:\a\_work\1\s\src\coreclr\vm\threads.cpp @ 862] 
05 000000d6`ebbffb90 00007ffa`4c6f4cb0     coreclr!ThreadNative::KickOffThread+0xc5 [D:\a\_work\1\s\src\coreclr\vm\comsynchronizable.cpp @ 239] 
06 000000d6`ebbffbf0 00007ffa`4d41edcb     kernel32!BaseThreadInitThunk+0x10 [clientcore\base\win32\client\thread.c @ 75] 
07 000000d6`ebbffc20 00000000`00000000     ntdll!RtlUserThreadStart+0x2b [minkernel\ntdll\rtlstrt.c @ 1152]

    This looks like a regression from the statics simplification. cc @davidwrighton

  5. jkotas
    added
    area-VM-coreclr
    and removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Dec 19, 2024

  6. dotnet-policy-service commented on Dec 19, 2024

    @dotnet-policy-service
    dotnet-policy-serviceBot
    on Dec 19, 2024 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @mangod9
    See info in area-owners.md if you want to be subscribed.

  7. jkotas commented on Dec 19, 2024

    @jkotas
    jkotas
    on Dec 19, 2024 · Hidden as resolved

  8. EgorBo commented on Dec 19, 2024

    @EgorBo
    EgorBo
    on Dec 19, 2024 · Hidden as resolved

  9. janvorli commented on Dec 19, 2024

    @janvorli
    janvorli
    on Dec 19, 2024 · Hidden as resolved

  10. majocha commented on Dec 19, 2024

    @majocha
    majocha
    on Dec 19, 2024 · Hidden as resolved
    Author

  11. majocha commented on Dec 19, 2024

    @majocha
    majocha
    on Dec 19, 2024 · Hidden as resolved
    Author
  12. jkotas
    assigned
    davidwrighton
    on Dec 20, 2024

  13. psfinaki commented on Dec 20, 2024

    @psfinaki
    psfinaki
    on Dec 20, 2024

    I think we also have a more recent one if it helps / applies :)
    899623-testhost.exe-37660-1734609522.dmp

  14. jkotas
    added
    regression-from-last-release
    on Jan 7, 2025
  15. dotnet-policy-service
    added
    in-prThere is an active PR which will close this issue when it is merged
    on Jan 9, 2025
  16. davidwrighton
    closed this as completedin #111257on Jan 10, 2025
  17. dotnet-policy-service
    removed
    untriagedNew issue has not been triaged by the area owner
    on Jan 10, 2025
  18. github-actions
    locked and limited conversation to collaborators on Feb 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-VM-coreclrin-prThere is an active PR which will close this issue when it is mergedregression-from-last-release

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions