Repository navigation
keep rejected lengthSizeMinusOne out of AVCDecoderConfigurationRecord - #3625
sahvx655-wq wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
1 open issue
What changed in this PR
This PR fixes a state-corruption bug in AVC sequence header parsing where a rejected lengthSizeMinusOne == 2 value could still overwrite the previously valid AVCDecoderConfigurationRecord, later causing a fatal CHECK in AVCNaluIterator during muxing.
Changes:
- Update
AVCDecoderConfigurationRecord::Createto validatelengthSizeMinusOnebefore assigning it to the record. - Add a regression unit test ensuring a failed
Create()call does not clobber an already-valid record.
Review effort: Lite
| // Reject before storing: AVCNaluIterator CHECKs the stored value, and | ||
| // TsWriter keeps using its record after a failed Create(). | ||
| if ((buf[4] & 0x03) == 2) { | ||
| return butil::Status(EINVAL, "lengthSizeMinusOne should never be 2"); |
There was a problem hiding this comment.
That message is the existing upstream wording, the patch only moves the check ahead of the assignment. I'd rather leave the text alone here so the diff stays limited to the stale-state fix; the comment block directly above already spells out that 0, 1 and 3 are the valid values.
wwbmmm
left a comment
There was a problem hiding this comment.
[replied by brpc-oncall robot] The reordering in Create() correctly stops a rejected lengthSizeMinusOne from clobbering a valid record, but the stated 'rejected header leaves the record untouched' invariant does not hold for the other failure paths of Create().
| // record that had been parsed successfully before. TsWriter keeps muxing with | ||
| // its record after a rejected header, and AVCNaluIterator then trips | ||
| // CHECK_NE(_length_size_minus1, 2u) on the next video frame. A rejected | ||
| // header must leave the record untouched. |
There was a problem hiding this comment.
[replied by brpc-oncall robot] This comment claims a rejected header leaves the record untouched, but Create() still assigns avc_profile/avc_level and clears sps_list (and later pps_list) before several other error returns (bad SPS length, ParseSPS failure, missing PPS data, ...). A sequence header that passes the lengthSizeMinusOne check but fails in SPS/PPS parsing therefore still corrupts a previously valid record, and TsWriter::Write keeps muxing with it because _has_avc_seq_header remains true, producing TS output with stale/partial SPS/PPS. Consider parsing into a local AVCDecoderConfigurationRecord (e.g. let ParseSPS fill a target record) and committing to *this only on success, so the invariant holds for every failure path; otherwise narrow this comment to length_size_minus1 only.
There was a problem hiding this comment.
Right, the reorder only covered lengthSizeMinusOne. Going back through Create() I count avc_profile and avc_level being overwritten up front, sps_list cleared and partially refilled before the SPS-length and ParseSPS checks, and pps_list cleared before the PPS checks, so a header that clears the lengthSizeMinusOne check but is truncated after the SPS still leaves TsWriter muxing with the new profile and an empty PPS list, which it then writes into the TS output.
Took the first option: Create() now copies the record, runs the existing body (moved verbatim into a private Parse()) on the copy, and assigns it back only on success, so the reorder is gone and the record is untouched on every error return. Starting from a copy rather than a fresh record keeps width/height behaving as before when a header carries no SPS. The test now also feeds a header with lengthSizeMinusOne=1 that stops before numPPS; against the previous revision it fails on avc_profile (66 vs 77), and with this change it passes along with the other 31 brpc_rtmp_unittest cases.
There was a problem hiding this comment.
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Copilot review overview
3 open issues
Review effort: Lite
| // Parse into a copy and commit only on success, so that a rejected | ||
| // header leaves *this untouched. TsWriter keeps muxing with its record | ||
| // after a failed Create(), and AVCNaluIterator CHECKs length_size_minus1. | ||
| AVCDecoderConfigurationRecord tmp(*this); | ||
| butil::Status st = tmp.Parse(data, len); | ||
| if (st.ok()) { | ||
| *this = std::move(tmp); | ||
| } | ||
| return st; |



What problem does this PR solve?
Issue Number: resolve
Problem Summary:
AVCDecoderConfigurationRecord::CreatestoreslengthSizeMinusOnefrom byte 4 of an AVC sequence header into the record and only then rejects the reserved value 2. The rejected header still overwriteslength_size_minus1of a record that was parsed successfully before.TsWriter::Writekeeps_has_avc_seq_headerset when a later sequence header is rejected and goes on muxing frames with the same record, so the next video frame constructs anAVCNaluIteratorwithlength_size_minus1 == 2and failsCHECK_NE(_length_size_minus1, 2u)innext_as_ibmf.An RTMP publisher reaches this with three messages: a valid sequence header, a copy of it with bits 0-1 of byte 4 set to
10, then any video frame. Every frame after that logs at FATAL level with a stack trace, and with-crash_on_fatal_logthe process aborts. A small program feeding that sequence intoTsWriterends in:What is changed and the side effects?
Changed:
Create()now parses into a copy of the record and assigns it back only on success, so a rejected header leaves the record as it was on every failure path (the reservedlengthSizeMinusOne, a bad SPS, truncated PPS data, ...), and theCHECKinAVCNaluIteratorkeeps guarding a record invariant instead of wire input. The parsing body moved unchanged into a privateParse().RtmpTest.avc_seq_header_rejected_length_size_keeps_recordcovers both alengthSizeMinusOne=2header and one truncated after the SPS; it fails on master and passes with the fix, and theTsWriterprogram above completes and reports the rejected header through the returned status. All 32 tests inbrpc_rtmp_unittestpass locally.Side effects:
Performance effects: none.
Breaking backward compatibility: none. Headers with
lengthSizeMinusOne == 2were already rejected; only the state left behind changes.Check List: