bug: `COPY INTO … ON_ERROR = ABORT_N` skips bad rows (fewer than N) but reports `ERRORS_SEEN = 0` and no `FIRST_ERROR` · Issue #20782 · databendlabs/databend · GitHub
Skip to content

bug: COPY INTO … ON_ERROR = ABORT_N skips bad rows (fewer than N) but reports ERRORS_SEEN = 0 and no FIRST_ERROR #20782

Closed

Description

@hanke580

Search before asking

  • I had searched in the issues and found no similar issues.

Version

main e9b49fd (source build, v1.2.953-nightly-e9b49fde); also v1.2.952-nightly-8862a6285b and v1.2.925-patch-15.

What's Wrong?

With ON_ERROR = ABORT_N, rows that fail to parse are skipped as long as fewer than N errors are seen. That part is
intended. But the result of COPY INTO then reports the file with ERRORS_SEEN = 0 and FIRST_ERROR = NULL, even
though rows were dropped (the docs define ERRORS_SEEN as "Number of error rows in the source file"). With RETURN_FAILED_ONLY = TRUE the file is not listed at all. The same file loaded with
ON_ERROR = CONTINUE correctly reports ERRORS_SEEN = 2 and the first error. Load pipelines that check ERRORS_SEEN /
FIRST_ERROR (or use RETURN_FAILED_ONLY) therefore see a clean load, the file is marked as loaded, and nothing
records that rows are missing. NDJSON behaves the same.

Cause: on_error in src/query/storages/stage/src/read/error_handler.rs. In the OnErrorMode::Continue branch it
calls file_status.add_error(e.error, line). In the OnErrorMode::AbortNum(n) branch, when the error is tolerated, it
only increments on_error_count and returns Ok(()) without recording the error in file_status. Calling
file_status.add_error(...) there as well would make ERRORS_SEEN/FIRST_ERROR (and RETURN_FAILED_ONLY) correct.

How to Reproduce?

CREATE STAGE IF NOT EXISTS s115;
COPY INTO @s115/a.csv FROM (SELECT number::STRING AS id, 'v' AS s FROM numbers(10)
                            UNION ALL SELECT 'bad1', 'x' UNION ALL SELECT 'bad2', 'y')
     FILE_FORMAT = (TYPE = CSV) SINGLE = TRUE USE_RAW_PATH = TRUE;      -- 12 rows, 2 of them not INT

CREATE OR REPLACE TABLE t_abort (id INT, s STRING);
COPY INTO t_abort FROM @s115 FILES = ('a.csv') FILE_FORMAT = (TYPE = CSV) ON_ERROR = ABORT_3;
-- a.csv | 10 | 0 | NULL | NULL          <-- rows_loaded 10, errors_seen 0, first_error NULL
SELECT count(*) FROM t_abort;            -- 10

CREATE OR REPLACE TABLE t_continue (id INT, s STRING);
COPY INTO t_continue FROM @s115 FILES = ('a.csv') FILE_FORMAT = (TYPE = CSV) ON_ERROR = CONTINUE;
-- a.csv | 10 | 2 | Invalid value 'bad1' for column 0 (id Int32 NULL): invalid text for number | 11

CREATE OR REPLACE TABLE t_abort2 (id INT, s STRING);
COPY INTO t_abort2 FROM @s115 FILES = ('a.csv') FILE_FORMAT = (TYPE = CSV) ON_ERROR = ABORT_3 RETURN_FAILED_ONLY = TRUE;
-- (empty result)

Expected: ABORT_3 reports errors_seen = 2 and the first error (as CONTINUE does), and RETURN_FAILED_ONLY lists
the file.

Single node, default settings, HTTP handler.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Activity

  1. added a commit that references this issue on Oct 28, 2026
    420947c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

C-bugCategory: something isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions