Skip nokogiri-based sanitization for strings that don't contain angle… · fastruby/allsearch_api@541c9d6 · GitHub
Skip to content

Commit 541c9d6

Browse files
committed
Skip nokogiri-based sanitization for strings that don't contain angle brackets
It can be expensive to run the full sanitization routine on long strings, so let's skip it if they don't need HTML-parser based sanitization, using the heuristic of checking for an opening angle bracket. Also, use ruby's squeeze method to avoid an expensive regular expression. Also, avoid the potential for XSS if the source data has a malicious tag that is encoded with HTML entities (e.g. `&lt;script&gt;` instead of `<script>`. According to the attached microbenchmark: before: ``` Warming up -------------------------------------- sanitizer 583.000 i/100ms Calculating ------------------------------------- sanitizer 6.308k (± 4.5%) i/s (158.54 μs/i) - 31.482k in 5.001876s ``` after: ``` Warming up -------------------------------------- sanitizer 31.808k i/100ms Calculating ------------------------------------- sanitizer 315.987k (± 3.1%) i/s (3.16 μs/i) - 1.590M in 5.038977s ```
1 parent 2279324 commit 541c9d6

3 files changed

Lines changed: 62 additions & 4 deletions

File tree

‎app/models/sanitizer.rb‎

Lines changed: 25 additions & 4 deletions

‎benchmark/microbenchmarks.rb‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,3 +25,20 @@
2525
library_staff_repo.delete
2626
end
2727
end
28+
29+
Benchmark.ips do |b|
30+
long_string = <<~END_LONG_STRING
31+
•Bayesian networks (BNs) offer an alternative approach to risk estimation.•A BN of mercury risks to panthers replicated traditional probabilistic estimates.•BNs facilitate
32+
quantification of uncertainty and causal inference.\nTraditionally hazard quotients (HQs) have been computed for ecological risk assessment, often without quantifying the
33+
underlying uncertainties in the risk estimate. We demonstrate a Bayesian network approach to quantitatively assess uncertainties in HQs using a retrospective case study of
34+
dietary mercury (Hg) risks to Florida panthers (Puma concolor coryi). The Bayesian network was parameterized, using exposure data from a previous Monte Carlo-based assessment
35+
of Hg risks (Barron et al., 2004. ECOTOX 13=>223), as a representative example of the uncertainty and complexity in HQ calculations. Mercury HQs and risks to Florida panthers
36+
determined from a Bayesian network analysis were nearly identical to those determined using the prior Monte Carlo probabilistic assessment and demonstrated the ability of the
37+
Bayesian network to replicate conventional HQ-based approaches. Sensitivity analysis of the Bayesian network showed greatest influence on risk estimates from daily ingested
38+
dose by panthers and mercury levels in prey, and less influence from toxicity reference values. Diagnostic inference was used in a high-risk scenario to demonstrate the
39+
capabilities of Bayesian networks for examining probable causes for observed effects. Application of Bayesian networks in the computation of HQs provides a transparent and
40+
quantitative analysis of uncertainty in risks."
41+
END_LONG_STRING
42+
sanitizer = Sanitizer.new
43+
b.report('sanitizer') { sanitizer.sanitize long_string }
44+
end

‎spec/models/sanitizer_spec.rb‎

Lines changed: 20 additions & 0 deletions

0 commit comments

Comments
 (0)