Comparing main...git-systems/pooled-copies · github/go-pipe · GitHub
Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: github/go-pipe
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: github/go-pipe
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: git-systems/pooled-copies
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 7 commits
  • 2 files changed
  • 3 contributors

Commits on Apr 7, 2026

  1. Added a sync.Pool for 32KB copy buffers

    Replace the cureent io.Copy with the io.CopyBuffer with pooled buffer
    migue committed Apr 7, 2026
    Configuration menu
    Copy the full SHA
    010dca5 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2026

  1. Add test for pool buffer bypass via *os.File WriterTo

    On Go 1.26+, *os.File implements WriterTo, which causes
    io.CopyBuffer to bypass the provided pool buffer entirely.
    Instead, File.WriteTo → genericWriteTo → io.Copy allocates
    a fresh 32KB buffer on every call.
    
    This test detects the bypass by counting allocations during
    ioCopier copy operations with an *os.File source (which is
    the common case when the last pipeline stage is a commandStage).
    
    The test currently FAILS, demonstrating the problem.
    
    Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
    znull and Copilot committed Apr 8, 2026
    Configuration menu
    Copy the full SHA
    22c71ee View commit details
    Browse the repository at this point in the history
  2. Fix pool buffer bypass caused by *os.File WriterTo

    On Go 1.26+, *os.File implements WriterTo. When ioCopier's reader is an
    *os.File (the common case for commandStage), io.CopyBuffer detects
    WriterTo and calls File.WriteTo instead of using the provided pool
    buffer. File.WriteTo's sendfile path fails (the dest is not a network
    connection), so it falls back to genericWriteTo → io.Copy, which
    allocates a fresh 32KB buffer on every call — defeating the sync.Pool
    entirely.
    
    Fix: wrap the reader in readerOnly{} to strip all interfaces except
    Read, forcing io.CopyBuffer to use the pool buffer.
    
    Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
    znull and Copilot committed Apr 8, 2026
    Configuration menu
    Copy the full SHA
    c5839cb View commit details
    Browse the repository at this point in the history
  3. Add test for ReaderFrom hidden by nopWriteCloser

    When using WithStdout, the writer is wrapped in nopWriteCloser,
    which hides the ReaderFrom interface of the underlying writer.
    This prevents io.CopyBuffer from dispatching to ReadFrom for
    potential zero-copy paths (e.g., when the destination is a
    network connection or has a custom ReadFrom implementation).
    
    This test currently FAILS, demonstrating the problem.
    
    Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
    znull and Copilot committed Apr 8, 2026
    Configuration menu
    Copy the full SHA
    afe1947 View commit details
    Browse the repository at this point in the history
  4. Unwrap nopWriteCloser to expose ReaderFrom

    When the pipeline's stdout is set via WithStdout, the writer is
    wrapped in nopWriteCloser to add a no-op Close method. This
    wrapper hides the ReaderFrom interface of the underlying writer,
    preventing io.CopyBuffer from dispatching to it.
    
    Fix: unwrap nopWriteCloser in ioCopier and call ReadFrom directly
    when available. This enables zero-copy when the destination has a
    meaningful ReadFrom (e.g., network connections, custom writers).
    For the pipe-to-pipe *os.File case, File.ReadFrom's zero-copy
    paths don't yet support pipe sources, so a follow-up commit adds
    direct splice(2) for that case.
    
    Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
    znull and Copilot committed Apr 8, 2026
    Configuration menu
    Copy the full SHA
    6e7976a View commit details
    Browse the repository at this point in the history
  5. Fix lint errors in ioCopier tests

    Check error returns from pw.Write, c.Start, c.Wait. Remove
    redundant embedded field selectors (w.Buffer.String → w.String).
    
    Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
    znull and Copilot committed Apr 8, 2026
    Configuration menu
    Copy the full SHA
    60bd6c3 View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2026

  1. Merge pull request #50 from github/znull/sendfile

    Enable sendfile for network destinations; fix pool bypass on Go 1.26+
    znull authored Apr 9, 2026
    Configuration menu
    Copy the full SHA
    3f226f1 View commit details
    Browse the repository at this point in the history
Loading