What changes were proposed in this pull request?

ExecutorKubernetesCredentialsFeatureStep now treats an explicitly empty
spark.kubernetes.authenticate.executor.serviceAccountName as unset. The optional config is filtered with .filter(_.nonEmpty) before the fallback runs, so an empty executor account falls back to the driver account, as documented. With no driver account configured either, the executor pod spec is left untouched instead of receiving "".

The config description in docs/running-on-kubernetes.md is updated to say "not set or is empty". The change is confined to the executor config read so it stays independent of #58343 (SPARK-58910), which reworks the pod-template check in the same step.

Why are the changes needed?

The executor service account config is createOptional, so setting it to an empty string yields Some(""). executorServiceAccount.orElse(driverServiceAccount) then never falls back, and buildPodWithServiceAccount writes an empty serviceAccount / serviceAccountName into the executor pod. That contradicts the documented behavior ("if this parameter is not setup, the fallback logic will use the driver's service account") and produces a pod that names no usable account. An empty value is easy to hit from templating or --conf key= in submission scripts.

Does this PR introduce any user-facing change?

Yes, a bug fix. Before, --conf spark.kubernetes.authenticate.executor.serviceAccountName= with a driver account configured wrote serviceAccountName: "" to executor pods. Now such executors use the driver's service account, matching the behavior when the config is absent. Non-empty
values behave exactly as before.

How was this patch tested?

Two new tests in ExecutorKubernetesCredentialsFeatureStepSuite:

• empty executor account plus a driver account resolves to the driver account;
• empty executor account with no driver account leaves serviceAccount / serviceAccountName
  null.

build/sbt -Pkubernetes "kubernetes/testOnly org.apache.spark.deploy.k8s.features.ExecutorKubernetesCredentialsFeatureStepSuite"

All 5 tests pass (3 existing, 2 new).

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Fable 5.1)