{{ message }}
Repository navigation
Conversation
…erver
Add a bin/run-note.sh CLI that executes a single note headlessly -- no
Jetty/REST/WebSocket -- by assembling the interpreter runtime directly,
in the spirit of papermill for Jupyter. Supports parameter injection via
-p (${var} placeholders) and saving results to a separate note via -o.
A failed paragraph exits non-zero so CI/batch callers detect the failure.
On exit the process tears down its interpreter processes, RemoteScheduler
pools, the event server, and ExecutorFactory pools, then System.exit()s so
the JVM does not hang on the runtime's non-daemon threads.
HwangRock
force-pushed
the
ZEPPELIN-5745-run-note-headless
branch
from
September 5, 2026 13:18
a2a6180 to
1806fe2
Compare
… tree NotebookRunnerIntegrationTest counted RemoteInterpreterServer JVMs machine-wide via `jps -l`, so any concurrent zeppelin-server test that spawned its own interpreter made the orphan assertion see non-zero survivors (expected:0 but was:8 in the core-modules CI job). Poll the descendant pids of the run-note.sh process while it is alive and assert none of them survive the parent exit, scoping the check to this test's own process tree.
Member
Contributor
Author
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


What is this PR for?
Today a note can only be run through a live server — you start the web server
(Jetty/REST/WebSocket) and drive it from the browser, or call the REST API of an
already-running instance. That ties every execution to a long-lived server
process, which is awkward for cron jobs, CI checks, and batch pipelines.
This PR adds a small CLI,
bin/run-note.sh, that runs a single note headlessly— no web server at all — in the spirit of papermill
for Jupyter. It assembles just the interpreter runtime, executes the note, saves
the result, and exits.
Usage
-i— logical path of the note to run (e.g./Demo, as shown in the note list). Required.-o— save the executed note to a new path instead of overwriting the input.-p— inject a parameter, substituted into${key}/${key=default}placeholders in the note. Repeatable.Example:
How it works
NotebookRunnerbuilds the notebook + interpreter runtime with manual dependencyinjection (the same recipe the interpreter tests already use), so no Jetty/HK2 is
involved. The interpreter event server is still started — interpreters run as
separate processes and need it to call results back — but the web layer is not.
Paragraph output streams to stdout as the note runs, and the executed note is
saved through the normal
NotebookRepo.Behavior worth calling out
FINISHEDstate (error, abort, orskipped after an earlier failure) fails the run with a non-zero exit, so CI/batch
callers can detect failures; a clean run exits 0.
assumption. On exit the process stops the interpreter processes, each
RemoteScheduler's own thread pool, the event server, andExecutorFactory'spools, then calls
System.exit()as a backstop so a stray non-daemon threadcannot hang the JVM.
> filecaptures onlythe results); the "saved / finished" summary goes to stderr.
Known limitation
Notes that drive other paragraphs programmatically via
z.run()/z.runNote()are not supported yet — that path executes paragraphs asynchronously and can
race the run's completion check. Straight top-to-bottom notes (the common batch
case) are unaffected. This is noted in
HeadlessProcessListenerand can be afollow-up.
What type of PR is it?
Feature
What is the Jira issue?
How should this be tested?
Automated — 19 unit/integration tests under
notebook/cli/, including two thatlaunch
bin/run-note.shin a real separate JVM and assert the process exitswithin a timeout (guarding against the JVM-hang class of bug) with the correct exit
code (0 on success, 1 on a failed paragraph):
Manual:
runs a note end to end, prints its output, saves the result, and returns to the
shell prompt on its own.
Screenshots (if appropriate)
2026-09-05.10.10.53.mov
Questions: