Linked issue

Fixes #6685

Type of change

• New feature or public API

Motivation

ERNIE-4.5 (Ernie4_5ForCausalLM) is Baidu's family of dense decoder-only LLMs,
released under Apache 2.0.
The dense checkpoints follow a standard Llama-style architecture
with GQA and RoPE.

It uses GPT-J style interleaved rotary pairing and store rope_theta at the top level of config.json. The implementation also includes a forward-compatible fallback for checkpoints that may use the newer nested rope_parameters dict style.

MAX had no native support for the Ernie4_5ForCausalLM architecture class.
This PR adds an implementation so baidu/ERNIE-4.5-0.3B-PT (and sibling
checkpoints) can be served directly via max serve without any custom flags
after registration.

What changed

Added max/python/max/pipelines/architectures/ernie4_5/ — a new
architecture package for the Ernie4_5ForCausalLM model family.

New files:

Architecture highlights:

• 0.36B dense backbone: 18 layers, hidden_size=1024, intermediate_size=3072,
  GQA (16 Q / 2 KV heads, group=8), head_dim=128, vocab_size=103424
• GPT-J style interleaved RoPE (interleaved=True in rope_split_store_ragged),
  distinct from the standard half-rotation used by most Llama-family models
• rope_theta (500000.0) read from the top-level config field; the implementation also handles the nested rope_parameters dict style used by some newer checkpoint variants
• Standard causal attention (MHAMaskVariant.CAUSAL_MASK)
• Tied word embeddings by default (tie_word_embeddings=true), with a weight
  adapter edge case to remap a standalone lm_head.weight if present

Testing

Verified on baidu/ERNIE-4.5-0.3B-PT with GPU serving:

max serve \
  --model-path baidu/ERNIE-4.5-0.3B-PT \
  --custom-architectures architectures/ernie4_5

Smoke test — model generates correctly:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"baidu/ERNIE-4.5-0.3B-PT",
       "messages":[{"role":"user","content":"What is the capital of france?"}],
       "max_tokens":32,"temperature":0}'

Output:

The capital of France is Paris.

GSM8K accuracy vs vLLM reference:

Checklist

• The linked issue above has been reviewed by a maintainer and is agreed-upon,
  or this is a trivial fix that does not need prior approval
• PR is small and focused — single new architecture, no changes to existing
  architectures
• I ran ./bazelw run format to format my changes
• I added or updated tests to cover my changes
• If AI tools assisted with this contribution, I have included an Assisted-by: trailer in my commit message or this PR description

Assisted-by: AI