Repository navigation
fix(data_handling): stop coercing unparseable telemetry to 0.0 in floatify_input (#161) - #199
philphauler wants to merge 2 commits into
Conversation
floatify_input returned 0.0 for any value that failed both float() and
convert_str_to_timestamp(). Downstream code then reads that 0.0 as a valid
telemetry reading, corrupting frames built from bad data. Return the dash
sentinel ('-', already the no-value marker OnAIR uses in frames) so invalid
data is explicit instead of masquerading as a zero measurement.
- parser_util.floatify_input: append '-' instead of 0.0 on parse failure
- update floatify unit tests (2 explicit + multi-typed) to expect '-'
sylvesterkaczmarek
left a comment
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
floatify_input() now returns the string sentinel "-" for invalid telemetry, but the Kalman plugin stores every returned value and later feeds those frames into NumPy/simdkalman numeric arrays. After an invalid sample this can raise a conversion/type error instead of producing a fault result. Use a numeric missing-value representation that the consumers handle, or make the Kalman path explicitly skip/reject invalid samples, and add an end-to-end invalid-telemetry regression.
…inel kalman_plugin.Plugin.update() called floatify_input directly and appended every value into self.frames[i]; _generate_residuals then puts frames into float NumPy arrays for simdkalman, so the '-' sentinel from OnAIR#161 raised ValueError. Skip the whole frame instead of appending a partial one, since _generate_residuals index-aligns each attribute's window by position and a partial frame would desynchronize those windows. - Plugin.update: return early when floatify_input's output contains '-' - test_kalman_plugin: add mocked-frame and real-floatify_input coverage for the skip path
sylvesterkaczmarek
left a comment
sylvesterkaczmarek
left a comment
There was a problem hiding this comment.
Rechecked current 93714c3. The issue I raised is resolved: invalid telemetry still remains explicit as the existing dash sentinel, but the Kalman path now rejects the entire frame before any per-attribute window is mutated, so the numeric arrays never receive the string and the attribute histories stay aligned. The two new regressions cover both a forced sentinel frame and the real floatify_input path. No remaining blocker from my review.
|
Nice one, let's go ⚡️ Imma hit u up on linked in as well 🙏 |
|
Sounds good, thanks. Happy to connect there as well. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #199 +/- ##
==========================================
Coverage 100.00% 100.00%
==========================================
Files 26 31 +5
Lines 970 1171 +201
Branches 136 169 +33
==========================================
+ Hits 970 1171 +201 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary