{{ message }}
Repository navigation
Add per-run resource limits, priority, and resource usage - #1499
Draft
Carmine Paolino (crmne) wants to merge 5 commits into
Draft
Carmine Paolino (crmne) wants to merge 5 commits into
Carmine Paolino (crmne) wants to merge 5 commits into
Conversation
MXC confines what a workload can read, write, and reach, but not how much CPU and memory it can consume, and its result does not say how much it used (microsoft#1498). Add an optional `resources` section to the one-shot development contract: `cpuTimeMs`, `memoryMb`, `maxProcesses`, and `priority`. Supplying it, even empty, also asks the backend to report the workload's resource usage in the output metadata, so existing requests keep their current output. Details - Add `Resources` and `ResourcePriority` to the 1.1.0-alpha contract and adapt them into `CommonRequestIR` and `ContainerPolicy.resources`. Published contracts keep rejecting the field. - Add `ResourceUsage` and `ResourceLimit` to `SandboxOutputMetadata` (`ExecutionMetadata` in the SDK). - Add `ResourcePolicySupport`, a per-backend declaration checked by `validate_common`, so every backend rejects a limit it cannot enforce before it creates a sandbox. No backend enforces anything yet; the backends opt in with their implementations. - Skip `resources` when serializing an unset policy so existing policy hashes do not change. - Regenerate the development schema and TypeScript wire types, and document the section in docs/schema.md. Tests - Contract, adapter, normalization, version-boundary, and per-backend validation tests.
Bubblewrap runs accept `resources` but enforce nothing yet. Apply the limits as rlimits and a nice value on `bwrap`, which every process in the sandbox inherits, and report the workload's usage. Details - Add `mxc_unix_resources`, shared by the Unix backends that launch the workload as a direct child: it applies `RLIMIT_CPU` (rounded up to whole seconds, soft equal to hard), `RLIMIT_DATA`, and a nice value between fork and exec, and peeks at the exited child's usage with `waitid(WNOWAIT)` before `std::process::Child` reaps it. - Limit memory with `RLIMIT_DATA`, not `RLIMIT_AS`: a 2 GB address-space limit crashes `soffice --convert-to pdf`, while a 2 GB data limit lets it convert and still stops a script allocating 3 GB. - Start the workload as PID 1 (`--as-pid-1`) when `resources` is supplied. bwrap's own PID 1 reaper reports the exit through an eventfd and is never waited for, so the workload's usage would not reach MXC. - Report `limitExceeded: cpuTime` when the workload was killed by SIGXCPU, or by SIGKILL after using at least the limit. - Reject `maxProcesses`: a per-workload count needs a cgroup, and RLIMIT_NPROC counts every process of the user. Tests - Unit tests for the rlimit translation and limit detection, and a live child that inherits the limits and is peeked before the reap. - Streaming Bubblewrap tests for usage, descendants, CPU time, memory, priority, and the maxProcesses rejection.
Seatbelt runs accept `resources` but enforce nothing yet. Apply the CPU time limit and priority to the process MXC launches, which every process in the sandbox inherits, and report the workload's usage. Details - Set `RLIMIT_CPU` and a nice value between fork and exec, before `sandbox_init`, through the shared `mxc_unix_resources`. - macOS answers `RLIMIT_CPU` with a single SIGXCPU, which a program can catch or ignore, and never escalates. Add a watchdog that kills any member of the workload's session (or, with inherited stdio, its process group) whose own CPU time reached the limit, and report `limitExceeded: cpuTime` when it fires. Inherited stdio with a CPU limit gets its own process group, as a timeout already does. - Read usage with `waitid(WNOWAIT)` and `proc_pid_rusage` on the exited process before it is reaped. macOS reports only the process's own peak footprint, so `peakMemoryBytes` is omitted. - Reject `memoryMb` (macOS rejects RLIMIT_DATA, RLIMIT_AS, and RLIMIT_RSS with EINVAL), `maxProcesses`, and `resources` with the legacy `launchMethod: "open"`, whose workload is not MXC's child. Tests - Streaming Seatbelt tests for usage with descendants, CPU time (including a workload that ignores SIGXCPU), priority, and the rejected limits; unit tests for the open-launch rejection. Run on macOS 27 (arm64). - A Bubblewrap test that a workload ignoring SIGXCPU is still stopped.
ProcessContainer runs accept `resources` but enforce nothing yet. Both the BaseContainer and AppContainer tiers already assign the suspended workload to a job object before resuming it, so set the limits there, where they cover every descendant. Details - `UiJobObject::apply_resource_limits` sets the per-process user-mode time limit, the per-process committed-memory limit, the active process limit, and the priority class, keeping KILL_ON_JOB_CLOSE, and associates an I/O completion port for the job's limit notifications. Both runners call it before assignment and fail closed when it fails. - `UiJobObject::resource_usage` reports the job's total user and kernel time, the peak committed memory of its largest process, and the first limit notification (END_OF_PROCESS_TIME, PROCESS_MEMORY_LIMIT, or ACTIVE_PROCESS_LIMIT). A run MXC ended by timeout or kill reports no limit. - The usage is published in the output metadata after teardown, next to any captureDenials output. Tests - Job object tests for every limit, the priority class, and the notifications, run on Windows 11 (x64). - Ignored ProcessContainer streaming tests for usage, CPU time, and the process limit, run on an elevated host.
`ExecutionMetadata.resource_usage` is public, so its `ResourceUsage` and `ResourceLimit` types belong in the versioned `mxc_sdk::v1` namespace rather than only in the hidden runtime models. Details - Re-export `ResourceUsage` and `ResourceLimit` from `mxc_sdk::v1`. - Document both types and the new `ExecutionMetadata` field in the Rust V1 API reference, including that the typed V1 request (exact 1.0.0) cannot carry `resources` yet. - Use the public paths in the streaming tests.
Carmine Paolino (crmne)
requested review from
a team
and
a balanced review from Copilot
October 14, 2026 18:24
Carmine Paolino (crmne)
added a commit
to crmne/mxc
that referenced
this pull request
Oct 14, 2026
The limits are now submitted to Microsoft as microsoft/mxc#1499, so the notes point readers at the pull request as well as the issue.
Collaborator
|
Hi Carmine Paolino (@crmne) thanks for creating the Feature request and the PR! That said, since this would affect MXC policy, the team will need to discuss this in more detail. In the future, since this is a feature request it would be better if we first communicate in the issue where it was created so that we can confirm the design, implementation and think about any future consequences of adding the feature. We'll get back to you on this issue once we've discussed. Thanks again! |
Collaborator
Branden Bonaby (bbonaby)
marked this pull request as draft
October 21, 2026 22:46
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

📖 Description
MXC confines what a workload can read, write, and reach, but not how much it can consume, and its result does not say how much it used. This PR adds an optional
resourcessection to the development contract (1.1.0-alpha), enforces it on Bubblewrap, Seatbelt, and ProcessContainer, and reports the workload's usage in the output metadata.{ "version": "1.1.0-alpha", "process": { "commandLine": "python3 build.py" }, "resources": { "cpuTimeMs": 120000, "memoryMb": 2048, "maxProcesses": 64, "priority": "low" } }Supplying
resources, even as{}, addsresourceUsageto the output metadata (ExecutionMetadata.resource_usagein the Rust SDK):{ "userCpuTimeUs": 3410000, "systemCpuTimeUs": 12000, "peakMemoryBytes": 391553024, "limitExceeded": "cpuTime" }Requests without
resourcesare unchanged: no new arguments, no new metadata, and the same policy hash.Design
Contract.
resourcesis a top-level, recursively closed object in the one-shot development contract, following the existing flow:mxc_contract::dev::Resources→wire::ResourcesinCommonRequestIR→ContainerPolicy.resources: Option<ResourcePolicy>. Published contracts keep rejecting it (version-boundary tests), state-aware requests do not carry it, and the typed V1 SDK (exact1.0.0) does not expose it yet; it is reachable through "MXC request JSON". The schema and TypeScript wire types are regenerated.cpuTimeMsmemoryMbmaxProcessesprioritynormal(default),low, oridleLimits are per process on every backend. That is what rlimits can express, and the Windows job object uses its per-process limits to match.
Fail closed.
ResourcePolicySupportdeclares, per backend, which fields it enforces, andvalidate_commonrejects any other field before a sandbox is created (FailurePhase::Rejected). Backends that enforce none reject anyresourcessection, so usage is never silently missing either.cpuTimeMsRLIMIT_CPURLIMIT_CPU+ watchdogJOB_OBJECT_LIMIT_PROCESS_TIMEmemoryMbRLIMIT_DATAJOB_OBJECT_LIMIT_PROCESS_MEMORYmaxProcessesJOB_OBJECT_LIMIT_ACTIVE_PROCESSpriorityBELOW_NORMAL/IDLEclassnormalBubblewrap. A new
mxc_unix_resourcescore module (shared by the two Unix backends, likemxc_pty) setsRLIMIT_CPU(rounded up to whole seconds, soft equal to hard),RLIMIT_DATA, and the nice value onbwrapbetween fork and exec, so every process in the sandbox inherits them.RLIMIT_DATA, notRLIMIT_AS. An address-space limit also counts reservations that are never committed. Measured on Linux 7.2 with LibreOffice 26.8:soffice --headless --convert-to pdfon a small pptx and docx crashes (exit 139) under a 2 GBRLIMIT_AS, and is flaky even at 8 GB (thread creation fails), but converts under a 2 GB and even a 256 MBRLIMIT_DATA, which still makes a Python script allocating 3 GB fail withMemoryError. The same holds through MXC itself.waitid(P_PID, WEXITED | WNOHANG | WNOWAIT)and its rusage argument beforestd::process::Childreaps the child, soChildkeeps owning the reap and its pid-reuse guarantees.ChildAccountingwraps every reap path for this.bwrap's parent. A run withresourcestherefore adds--as-pid-1. The namespace still dies with the workload. The trade-off, documented in the backend guide: the workload reaps its own orphans and, as PID 1, ignores signals from inside the sandbox it has no handler for.limitExceeded: cpuTimeis reported when the workload was killed bySIGXCPU, or bySIGKILLafter using the limit (within 100 ms: the kernel's sampled CPU clock and the rusage that reaches the parent differ by a few milliseconds). A memory limit only makes allocations fail, and how a program reacts is its own business, so Unix backends never guessmemory.maxProcessesis rejected: a per-workload count needs a cgroup (pids.max), andRLIMIT_NPROCcounts every process of the user. MXC has no cgroup plumbing today, so cgroup v2 support (memory.max,pids.max,cpu.stat) is left as a follow-up.Seatbelt. The same rlimit and nice code runs before
sandbox_init. Two macOS specifics, both measured on macOS 27 (arm64):setrlimitrejectsRLIMIT_DATA,RLIMIT_AS, andRLIMIT_RSSwithEINVAL, somemoryMbis rejected.RLIMIT_CPUwith a singleSIGXCPUand never escalates: a program that ignores it runs on indefinitely. A watchdog thread checks the workload's session (or, with inherited stdio, its process group, which a CPU limit now creates as a timeout already does) every 100 ms withproc_pid_rusageand kills any member whose own CPU time reached the limit, through the same audit-token signalling as the session sweep. Like the timeout's tree kill, it does not reach a process that leaves the session.proc_pid_rusageon the exited, unreaped process (user, system, and waited-for children's times, converted from Mach units). macOS reports only the process's own peak footprint, sopeakMemoryBytesis omitted.resourceswith the legacylaunchMethod: "open"is rejected: the workload runs inside Terminal.app, not as MXC's child.ProcessContainer. Both tiers already assign the suspended workload to a job object before resuming it.
UiJobObject::apply_resource_limitsadds the limits and priority class there (keepingKILL_ON_JOB_CLOSE) before assignment, failing closed, and associates an I/O completion port.UiJobObject::resource_usagereportsTotalUserTime/TotalKernelTime,PeakProcessMemoryUsed, and the firstEND_OF_PROCESS_TIME,PROCESS_MEMORY_LIMIT, orACTIVE_PROCESS_LIMITnotification. The usage is merged into the output metadata after teardown, next to any captureDenials output. Windows counts the CPU limit in user mode only and checks it every few seconds.Results.
SandboxOutputMetadatagainsresource_usage: Option<ResourceUsage>;ResourceUsageandResourceLimitare re-exported frommxc_sdk::v1and documented in the Rust V1 API reference. A run MXC ended by timeout or kill reports usage but nolimitExceeded.Compatibility
ExecutionMetadata, which has public fields, breaks code that constructs it with a struct literal. Onlywxc's tests did in this repository.resourcesis skipped when unset).Follow-ups
resourcesandresourceUsage, and the typed V1 Rust request, once the field is promoted to a published contract.resourceUsagefrom the executors (wxc-exec,lxc-exec,mxc-exec-mac) the way captureDenials prints its pointer, and resource usage in ETW telemetry.resourceson state-aware exec requests.🔗 References
Resolves #1498.
🔍 Validation
cargo fmt --all -- --check;cargo clippy -p mxc-sdk -p lxc -p mxc_schema_gen --all-targets -- -D warnings;cargo test -p mxc-sdk --lib(2,041 pass; 3 fail identically on unmodifiedmainon this host:sdk_v1_conformance::invalid_documents_are_rejected,mxc_pty::tests::native_bridge_cancellation_does_not_inject_eof, and, under load only,provider_monitor::tests::a_monitor_over_an_already_reaped_sandbox_still_settles); the exact-contract, version-boundary, andstreaming_bubblewrapintegration tests;check-contract-codegen.js,check-schema-versions.js,validate-configs.js.cargo fmt --check,cargo clippy -p mxc-sdk --all-targets -- -D warnings, themxc_unix_resources, validator, and Seatbelt unit tests, and thestreamingintegration tests (29 pass).cargo fmt --check,cargo clippy -p mxc-sdk --all-targets -- -D warnings, the job object and validator unit tests (49 pass), and the ignored ProcessContainer streaming tests with--ignoredon an elevated host (6 pass).Buildworkflow on the fork, all 23 jobs green (lint, build and test on Linux, macOS, and Windows x64 and arm64, versioning, SDK unit and integration tests): https://github.com/crmne/microsoft-mxc/actions/runs/38609552228mxcRuby gem, which builds on this branch, on all three platforms.✅ Checklist
Cargo.lock, thedependency-feed-checkcheck passes (see pull request builds)📋 Issue Type
Microsoft Reviewers: Open in CodeFlow