Kotlin extractor: fix K2 regressions without expected churn by andersfugmann · Pull Request #22076 · github/codeql · GitHub
Skip to content

Kotlin extractor: fix K2 regressions without expected churn#22076

Draft
andersfugmann wants to merge 8 commits into
mainfrom
andersfugmann/fix_kotlin1_tests
Draft

Kotlin extractor: fix K2 regressions without expected churn#22076
andersfugmann wants to merge 8 commits into
mainfrom
andersfugmann/fix_kotlin1_tests

Conversation

@andersfugmann

Copy link
Copy Markdown
Contributor

This draft PR groups the Kotlin extractor fixes needed to stabilise migrated Kotlin1/Kotlin2 test behaviour under Kotlin 2.4.

Included fixes:

  • disambiguate Java binary overload probing to avoid boxed/primitive key collisions in DB-CHECK
  • keep synthetic locations when external file-class binary paths are not concrete
  • scope Object-method redeclaration recovery to avoid spurious equals(Object) extraction

Targeted verification was run one test at a time on the affected suites.

andersfugmann and others added 8 commits June 29, 2026 16:21
Why this is needed:
- The dev wrapper persisted the selected version in .kotlinc_version, but only installed binaries when the selected version changed.
- After a clean working directory (which can remove .kotlinc_installed), the version file can still point at an already-selected compiler, causing forward execution to fail because the binary directory no longer exists.

What this changes:
- Make install() idempotent by returning early when install dir already exists.
- Call install() unconditionally from main() so the selected version is always materialised before forwarding.
- Keep explicit reinstall behaviour on version switches by removing the old install directory when selection changes.

This is an independent reliability fix and not tied to Kotlin 1.x test routing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Why this is needed:
- Under K2, top-level declarations from external binaries are attached directly
  to IrExternalPackageFragment rather than to an IrClass file-class parent.
- That bypassed the normal class-source location path, so some external file-class
  entities ended up without stable binary file locations.
- Missing/unstable locations caused drift in tests that depend on external file
  class member resolution and location facts.

What this changes:
- Resolve binary paths from IrMemberWithContainerSource (JvmPackagePartSource)
  via a dedicated getContainerSourceBinaryPath helper.
- In KotlinUsesExtractor, when extracting top-level external declarations,
  attach file-class location from container-source binary path when available.
- Track external file classes whose locations were emitted to avoid duplicate
  hasLocation facts.

This targets the K2 external file-class location gap (for example file_classes and
external-property-overloads parity).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Why this is needed:
- With Kotlin 2.0 analysis, some local-variable locations resolve to a wider
  declaration span than before.
- The previous extractor logic used provider-based ranges that can cover type,
  annotations, and modifiers, which shifts expected variable location facts.
- This caused parity drift in tests that expect the location to point at the
  variable name token itself.

What this changes:
- Cache current source text per file during extraction.
- Derive variable-name offsets by scanning the declaration slice and locating
  the declared identifier token.
- Emit local-variable declaration/expr locations from that identifier span,
  with fallback to the previous provider when source offsets are unavailable.

This restores stable name-anchored variable locations under Kotlin 2.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Why this is needed:
- Under K2, binary Java symbols are represented differently from K1:
  JavaSourceElement metadata is often absent and sources are exposed through
  VirtualFileBasedSourceElement.
- Without recovery logic, callable matching can miss declared Java methods,
  callable labels can drift (primitive vs boxed reference types), and external
  Java declaration stubs can gain wildcard noise when Java signatures are not
  available.
- These differences produced Kotlin 2.0 parity drift in tests that rely on
  stable Java/Kotlin cross-extractor callable identity.

What this changes:
- Add K2-aware Java binary inspection helpers (ASM-based fallback) to detect
  declared methods and parameter/return reference-vs-primitive shape when
  JavaSourceElement metadata is unavailable.
- Recover Java callables more reliably in KotlinUsesExtractor, including a
  binary-class fallback path.
- Normalise callable labels and call result typing to boxed Java classes when
  K2 enhanced reference types appear as Kotlin primitives.
- Accept K2's `Any` form for Object.equals(Object) and keep binary declaration
  checks stable.
- Suppress default wildcard insertion for external Java declaration stubs when
  no Java callable metadata is available, preventing synthetic wildcard drift.

This commit restores Java interop parity for Kotlin 2.0 extraction paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Why this is needed:
- The extractor compatibility fixes now preserve the information these Kotlin1-era
  tests were protecting, even when compiled with Kotlin 2.4 and
  `-language-version 2.0`.
- Keeping mixed legacy language-version wiring in individual tests is no longer
  necessary and obscures the intended steady-state execution mode.

What this changes:
- Update all affected Kotlin1 compatibility integration tests to run with
  `-language-version 2.0` directly.
- Keep the expected extraction signal aligned for extractor information output.
- Remove the obsolete CODEOWNERS entry for the retired `java/ql/test-kotlin1/`
  path.

This consolidates the language-version transition into a single test rollup
commit, as requested.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Why this is needed:
- library-tests/exprs/DB-CHECK was failing with INVALID_KEY and INVALID_KEY_SET in params for kotlin.jvm.internal.Intrinsics.areEqual.
- The Java binary probing code matched methods by name plus arity and used the first match, which is ambiguous when both primitive and boxed overloads exist.
- Under that ambiguity, callable labels could be boxed while extracted params remained primitive (or vice versa), creating conflicting rows for the same key.

What changed:
- For both parameter and return-type probing, gather all matching overloads and compute classifier-vs-primitive from the full candidate set.
- Return a concrete answer only when all matches agree; return null when matches disagree.
- Apply the same unambiguous matching rule in both K1 metadata and K2 ASM fallback paths.

Effect:
- The boxing fallback now activates only when the Java binary evidence is deterministic, preventing callable-label collisions and restoring DB integrity in the affected Kotlin2 dataset check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Why this is needed:
- library-tests/multiple_files/method_accesses.ql regressed because receiver class locations for external file-class members became concrete file paths.
- For stdlib-style unresolved container-source paths, forcing a concrete location changed stable output from synthetic unknown location to external path-based locations.

What changed:
- Added shouldUseConcreteExternalFileClassLocation to distinguish reliable concrete paths from unresolved placeholders.
- In external package-fragment parent handling, only write an external file-class location when the normalized path is concrete and stable.
- If no reliable path is available, keep prior synthetic behaviour by not forcing a concrete location.

Effect:
- Restores stable receiver-location output for method_accesses while preserving concrete locations when we have trustworthy binary-path information.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Why this is needed:
- library-tests/java-kotlin-collection-type-generic-methods/test.ql regressed with extra equals(Object) rows on generic collection/map/list declaration variants.
- At the same time, java-interface-redeclares-tostring must still recover Object-method redeclarations for Java binary interfaces under K2.

What changed:
- In K2 ASM probing, treat classes with kotlin.Metadata as non-Java binaries for javaBinaryDeclaresMethod, so Java-redeclaration recovery does not fire on Kotlin binary classes.
- Keep equals(Object) K2 Any/Any? compatibility handling, but constrain the workaround to non-generic parent classes and skip it when a concrete sibling declaration already exists.
- Preserve the existing toString/hashCode redeclaration recovery path for affected Java binaries.

Effect:
- Removes the spurious equals(Object) rows in java-kotlin-collection-type-generic-methods while retaining expected Object-method extraction in java-interface-redeclares-tostring.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@andersfugmann andersfugmann force-pushed the andersfugmann/fix_kotlin1_tests branch from a3a28c7 to 831e87b Compare June 29, 2026 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant