{{ message }}
feat: add unindexed-scan query counter#4282
Draft
imilinovic wants to merge 10 commits into
Draft
Conversation
Contributor
Author
18e18cc to
ddf5c6a
Compare
Review-driven refinements to the query no-index-lookup counter: - Count per-query, not per-operator: a query with N qualifying ScanAll+Filter scans now bumps memgraph_query_no_index_lookup_total by 1. Makes rate(no_index)/rate(queries) a stable alert ratio. The per-operator count is still produced in PlanHintsResult for future structured use. - Unit-test the counter: query_hint_provider asserts no_index_lookup_count in every case (Cartesian -> 2, ScanAllByLabel+property -> 0). Was untested before. - Rename PlanHintsProvider::hints() -> take_hints() (returns by value) to drop the misleading std::move on an lvalue-ref accessor. - Reword the counter Help string for per-query semantics; document the intentional ScanAllByLabel non-increment and the OpenMetrics/SHOW-only scope (no deprecated JSON-global aggregation). - e2e: UNION asserts +1 (per-query); add a PROFILE no-op assertion.
The exposed metric is per-query (counts once when a plan has any qualifying ScanAll+Filter), so the internal per-operator count was never used as a magnitude. Replace PlanHintsResult::no_index_lookup_count (size_t) with has_no_index_lookup (bool): expresses the actual contract, removes a field that had to stay in sync with hints_, and makes per-operator exposure impossible to reintroduce by accident. Branch-firing coverage is retained via the existing hint-message assertions.
"no_index_lookup" was ambiguous (a lookup that found no index, or the absence of a lookup?) and didn't say a scan happened. Rename to the clearer, convention- aligned (memgraph_*_queries_total) name before release: - metric: memgraph_query_no_index_lookup_total -> memgraph_unindexed_scan_queries_total - SHOW METRICS INFO row: QueryNoIndexLookup -> UnindexedScanQueries - family/handle: query_no_index_lookup* -> unindexed_scan_queries* - PlanHintsResult field: has_no_index_lookup -> has_unindexed_scan Pure rename; behavior and tests unchanged.
…ning The plan-hint provider classified scans with an exact `GetTypeInfo() == ScanAll::kType` check. Parallel-execution plans rewrite `Filter -> Scan*` into `Filter -> ScanChunk -> ParallelMerge -> ScanParallel*`, so the filter sits on a `ScanChunk` (its own kType), which fell through silently and never incremented `memgraph_unindexed_scan_queries_total`. Add `EffectiveScanType` to resolve a `ScanChunk` down to the underlying `ScanParallel*` and map it back to the equivalent sequential type (`ScanParallel` -> `ScanAll`, `ScanParallelByLabel` -> `ScanAllByLabel`), so parallel plans count identically to sequential ones. Index-backed parallel scans stay unflagged. Review hardening: - reuse the guarded `dynamic_cast<ScanAll*>` instead of casting twice - make `take_hints()`/`has_unindexed_scan()` private + `friend ProvidePlanHints` (+ `[[nodiscard]]`) so result accessors can't be read mid-traversal - extract `EmitPlanHints` for the three duplicated hint loops; keep the counter increment only on the regular-execution path, with explicit comments on the EXPLAIN/PROFILE no-op and the before-AccessorCompliance ordering - `Increment(1.0)` -> `Increment()` - document the property-only-filter non-count case and the JSON-endpoint omission Tests: - unit: parallel unindexed scan counts, parallel label scan does not, property-only sequential scan does not - e2e: `USING PARALLEL EXECUTION` unindexed scan counts (+1) and is silenced by a matching index
Apply consolidated review feedback (no behavior change to the metric): - Fix misleading comment: ScanParallelByEdge is a full edge scan, not index-backed; state the out-of-scope reason explicitly. - Reword counter help text to "Planned queries" to match counting before AccessorCompliance. - Mark ProvidePlanHints [[nodiscard]]. - Move HintIndexUsage and EffectiveScanType out-of-line into the .cpp. - Align metric section labels with the emitted "Query" category. - Add regression tests: property-only parallel scan and ScanChunkByEdge parallel pass-through; make WrapInParallelChunk a fixture helper. - Condense over-long comments.
- assert ScanChunk/ParallelMerge chain invariants instead of dead null guards - add unit test covering the ScanParallelByLabelProperties (indexed) branch - spell out trigger and exclusion conditions in the counter help text - trim verbose comments to one line; fix the misleading edge-scan note
Replace the no-op DMG_ASSERTs in EffectiveScanType (compiled out under NDEBUG) with always-on guards that fall back to the scan's own type when the parallel-rewrite chain has an unexpected shape, so a malformed chain can no longer misclassify or null-deref in release builds. Also tighten review-flagged comments: drop the chain description that was duplicated from the function header, correct the EmitPlanHints contract comment (it never touches the counter), and collapse the metric-handle comment to one line.
Reword the metric Help and the EffectiveScanType / has_unindexed_scan comments so they cover parallel-execution plans (not just the sequential ScanAll), correct the parallel-rewrite chain description, and fix a dropped word. Comment/string only; no behavior change.
ddf5c6a to
ce59097
Compare
- Make the EXPLAIN/PROFILE no-count rule structural: EmitPlanHints now takes a DatabaseMetricHandles* and does the increment itself, so every call site must explicitly opt in (Cypher passes handles, EXPLAIN/PROFILE pass nullptr). - Add PlanHintsProvider::AddUnindexedScanHint so a missing-index hint and the has_unindexed_scan_ flag are set in lockstep; suboptimal-index branch stays plain push_back. Fix the stale "sets accordingly" doc comment. - Make PlanHintsProvider single-use (delete move ctor) and move the friend declaration next to the private accessors it gates. - e2e: track a running expected count instead of baseline+N chaining; drop internal task reference from the docstring.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Adds a per-database metric
memgraph_unindexed_scan_queries_total{database,uuid}that counts query executions whose plan does aScanAll+Filteran index could have served.What it does
Surfaces, as an aggregable metric, how often queries run a full scan + filter when an index could have served the scan — previously only emitted as a per-query planner hint notification. Exposed via OpenMetrics and
SHOW METRICS INFO(rowUnindexedScanQueries).How
The plan-hint provider already detects this case at plan time; it now also reports a
has_unindexed_scanbool throughPlanHintsResult. The interpreter increments the counter once per qualifying query at the regular-execution site only (EXPLAIN/PROFILE do not count). Counting is per-query: a query with several qualifying scans bumps the counter by 1, sorate(unindexed)/rate(queries)is a stable ratio. Wired through the existing per-databaseDatabaseMetricHandlesmachinery (Colin's per-tenant pattern).Why
Operators had no fleet-level signal for missing-index workloads — only a per-query notification. This gives a canary ("is this happening / is the rate growing?") across tenants. It is intentionally a per-query occurrence signal, not a cost signal (see follow-ups for slow-query enrichment / scan amplification). The legacy JSON metrics endpoint is being deprecated, so the counter is OpenMetrics +
SHOW METRICS INFOonly.