Fix GEGLU docstring: Sigmoid -> GELU by aymuos15 · Pull Request #8696 · Project-MONAI/MONAI · GitHub
Skip to content

Fix GEGLU docstring: Sigmoid -> GELU#8696

Merged
ericspod merged 4 commits into
Project-MONAI:devfrom
aymuos15:fix/geglu-docstring
Mar 1, 2026
Merged

Fix GEGLU docstring: Sigmoid -> GELU#8696
ericspod merged 4 commits into
Project-MONAI:devfrom
aymuos15:fix/geglu-docstring

Conversation

@aymuos15

Copy link
Copy Markdown
Contributor

Summary

  • Fixed GEGLU docstring which incorrectly stated the activation function was Sigmoid
  • The code correctly uses GELU, as specified in the original GEGLU paper

Details

  • GLU uses Sigmoid: GLU(x) = σ(xW) ⊗ xV
  • GEGLU uses GELU: GEGLU(x) = GELU(xW) ⊗ xV

Reference: https://arxiv.org/abs/2002.05202

The docstring incorrectly stated GEGLU uses Sigmoid, but the code
correctly uses GELU. Per the original paper (Shazeer, 2020):
- GLU uses Sigmoid: GLU(x) = σ(xW) ⊗ xV
- GEGLU uses GELU: GEGLU(x) = GELU(xW) ⊗ xV

Reference: https://arxiv.org/abs/2002.05202
Signed-off-by: Soumya Snigdha Kundu <soumya_snigdha.kundu@kcl.ac.uk>
@coderabbitai

coderabbitai Bot commented Jan 13, 2026

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
monai/networks/blocks/activation.py (1)

167-184: LGTM! Docstring now matches implementation.

The fix correctly aligns the documentation with the actual code (nn.functional.gelu(gate) on line 184).

Optional: For consistency with other activation classes in this file, consider adding an Examples section.

📝 Optional: Add Examples section
     Shape:
         - Input: :math:`(N, *, 2 * D)`
         - Output: :math:`(N, *, D)`, where `*` means, any number of additional dimensions
+
+
+    Examples::
+
+        >>> import torch
+        >>> m = GEGLU()
+        >>> input = torch.randn(2, 8)  # last dim must be even
+        >>> output = m(input)
     """
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Cache: Disabled due to data retention organization setting

Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting

📥 Commits

Reviewing files that changed from the base of the PR and between 57fdd59 and 51126f9.

📒 Files selected for processing (1)
  • monai/networks/blocks/activation.py
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py

⚙️ CodeRabbit configuration file

Review the Python code for quality and correctness. Ensure variable names adhere to PEP8 style guides, are sensible and informative in regards to their function, though permitting simple names for loop and comprehension variables. Ensure routine names are meaningful in regards to their function and use verbs, adjectives, and nouns in a semantically appropriate way. Docstrings should be present for all definition which describe each variable, return value, and raised exception in the appropriate section of the Google-style of docstrings. Examine code for logical error or inconsistencies, and suggest what may be changed to addressed these. Suggest any enhancements for code improving efficiency, maintainability, comprehensibility, and correctness. Ensure new or modified definitions will be covered by existing or new unit tests.

Files:

  • monai/networks/blocks/activation.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
  • GitHub Check: min-dep-py3 (3.12)
  • GitHub Check: min-dep-pytorch (2.6.0)
  • GitHub Check: quick-py3 (ubuntu-latest)
  • GitHub Check: min-dep-pytorch (2.8.0)
  • GitHub Check: quick-py3 (windows-latest)
  • GitHub Check: min-dep-pytorch (2.7.1)
  • GitHub Check: min-dep-os (macOS-latest)
  • GitHub Check: quick-py3 (macOS-latest)
  • GitHub Check: min-dep-py3 (3.11)
  • GitHub Check: min-dep-py3 (3.10)
  • GitHub Check: build-docs
  • GitHub Check: min-dep-os (ubuntu-latest)
  • GitHub Check: min-dep-py3 (3.9)
  • GitHub Check: packaging
  • GitHub Check: min-dep-os (windows-latest)
  • GitHub Check: min-dep-pytorch (2.5.1)
  • GitHub Check: flake8-py3 (mypy)
  • GitHub Check: flake8-py3 (codeformat)
  • GitHub Check: flake8-py3 (pytype)

@aymuos15

Copy link
Copy Markdown
Contributor Author

"Optional: For consistency with other activation classes in this file, consider adding an Examples section." -- Happy to add this as well if required.

@ericspod

Copy link
Copy Markdown
Member

Add example showing usage of GEGLU to align docstring style with other activation classes. No functional changes.

Signed-off-by: Soumya Snigdha Kundu <soumya_snigdha.kundu@kcl.ac.uk>
@aymuos15 aymuos15 force-pushed the fix/geglu-docstring branch from 805f33f to f5a63ce Compare January 14, 2026 10:23
@ericspod ericspod enabled auto-merge (squash) March 1, 2026 01:00
@ericspod ericspod merged commit 1e3d29b into Project-MONAI:dev Mar 1, 2026
26 checks passed
@aymuos15 aymuos15 deleted the fix/geglu-docstring branch March 3, 2026 15:48
Rusheel86 pushed a commit to Rusheel86/MONAI that referenced this pull request May 25, 2026
## Summary
- Fixed GEGLU docstring which incorrectly stated the activation function
was Sigmoid
- The code correctly uses GELU, as specified in the original GEGLU paper

## Details
- GLU uses Sigmoid: GLU(x) = σ(xW) ⊗ xV
- GEGLU uses GELU: GEGLU(x) = GELU(xW) ⊗ xV

Reference: https://arxiv.org/abs/2002.05202

---------

Signed-off-by: Soumya Snigdha Kundu <soumya_snigdha.kundu@kcl.ac.uk>
Co-authored-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants