Calculate content-md5 on S3 only when required. by dwsteele · Pull Request #2592 · pgbackrest/pgbackrest · GitHub
Skip to content

Calculate content-md5 on S3 only when required. - #2592

Merged
dwsteele merged 8 commits into
pgbackrest:integrationfrom
dwsteele:dev-s3-md5
Apr 9, 2025
Merged

dwsteele merged 8 commits into
pgbackrest:integrationfrom
dwsteele:dev-s3-md5

Conversation

@dwsteele

@dwsteele dwsteele commented Apr 1, 2025

Copy link
Copy Markdown
Member

The content-md5 header was generated for all requests with content but it is only required for batch delete requests. It is not clear why this header is required when x-amz-content-sha256 is also provided or why it is required only for this request but the documentation is clear on the matter. However, the content for these requests is relatively small compared to uploading files so omitting content-md5 where possible will save some CPU cycles.

Current AWS S3 and recent Minio don't complain if this header is missing but since it is still required by older versions of Minio and it is specified in the documentation for batch delete it is makes sense to keep it.

@dwsteele dwsteele added this to the 2.55 milestone Apr 1, 2025
@dwsteele dwsteele self-assigned this Apr 1, 2025

@pgguru pgguru left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name of this flag seems odd to me; perhaps something indicating an action like computeMd5, or more of an attribute about the param likerequiresMd5.

@dwsteele

dwsteele commented Apr 7, 2025

Copy link
Copy Markdown
Member Author

@pgguru pgguru left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fine with the flag as-is, I'm content with the content as it were.</groan>

@dwsteele
dwsteele merged commit 20bfd14 into pgbackrest:integration Apr 9, 2025
@dwsteele
dwsteele deleted the dev-s3-md5 branch April 9, 2025 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants