I'm surprised that we write the midx to a git_buf and then write the git_buf to disk. I expected us to not use an intermediate git_buf. How big is a midx file going to be?

I'm surprised that we write the midx to a git_buf and then write the git_buf to disk. I expected us to not use an intermediate git_buf. How big is a midx file going to be?

as large as all the .pack files in a repo combined (my copy of linux' is 1G).

IIRC I did that for simplicity, but i can refactor things so that git_midx_writer_dump and git_midx_writer_commit both call the same function that creates the midx and call a write_cb(const char *buf, size_t size, void *payload) to avoid that extra in-memory copy.

IIRC I did that for simplicity, but i can refactor things so that git_midx_writer_dump and git_midx_writer_commit both call the same function that creates the midx and call a write_cb(const char *buf, size_t size, void *payload) to avoid that extra in-memory copy.

I think that would be nice if you could - I think that midx is going to be used in large repositories, so avoiding that malloc of a contiguous hunk of memory to store the data would be a win.

IIRC I did that for simplicity, but i can refactor things so that git_midx_writer_dump and git_midx_writer_commit both call the same function that creates the midx and call a write_cb(const char *buf, size_t size, void *payload) to avoid that extra in-memory copy.

I think that would be nice if you could - I think that midx is going to be used in large repositories, so avoiding that malloc of a contiguous hunk of memory to store the data would be a win.

refactored and rebased!