pnnx: preserve LayerNorm scale when bias is disabled by ktz03 · Pull Request #7128 · Tencent/ncnn · GitHub
Skip to content

pnnx: preserve LayerNorm scale when bias is disabled - #7128

Open
ktz03 wants to merge 1 commit into
Tencent:masterfrom
ktz03:fix/pnnx-layernorm-optional-bias
Open

ktz03 wants to merge 1 commit into
Tencent:masterfrom
ktz03:fix/pnnx-layernorm-optional-bias

Conversation

@ktz03

@ktz03 ktz03 commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Preserve the learned weight when a TorchScript nn.LayerNorm omits bias. Set the affine flag from the weight attribute and synthesize a zero bias with the same shape and type, keeping the existing PNNX/native representation and ordinary affine/non-affine behavior.

Closes #7127.

Validation: the complete existing PNNX and native LayerNorm programs pass with added one-dimensional and multidimensional missing-bias regressions; both related functional LayerNorm programs pass. On the baseline, the original normal outputs pass before each first added case fails. A separate 9-model public matrix changes from 6 normal / 3 wrong to all correct in both PNNX and native paths, with all 24 normal param/bin artifacts unchanged. An independent 8-model matrix also passes after the fix, with normal artifacts and outputs unchanged; these matrices overlap and are not added together.

Tested on Windows CPU, Python 3.14 and Torch 2.11 with newly built converter and current CPU Python binding, using cached dependencies. The tests set bias=None after construction to avoid requiring the newer constructor keyword on older Torch versions; the public matrix uses bias=False. Native cross-channel normalization remains an existing limitation. No full suite, other OS, GPU, PT2, ONNX frontend or performance claim.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pnnx drops learned LayerNorm weight when bias=False

1 participant