{{ message }}
Repository navigation
fix(storage): keep the real format for files sniffed as their container - #13825
Merged
Merged
Conversation
Content sniffing identifies a zip container from the entries at the head of the archive, so an APK whose archive starts with res/ is reported as application/zip while the same app built with AndroidManifest.xml near the front is reported as application/vnd.android.package-archive. The sniffed value is persisted as the file's mimeType and echoed back as Content-Type on download, next to a Content-Disposition filename ending in .apk. Responses also carry X-Content-Type-Options: nosniff, so a client that trusts the declared type over the filename saves the download as a .zip. Fall back to the filename extension when, and only when, sniffing already returned a generic container type. The fallback can only ever resolve to another binary type, so a file can never be promoted from its extension to something a browser would render and the stored-XSS property of the view endpoint is unchanged.
Contributor
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
✨ Benchmark resultsComparing
Per-scenario breakdown & investigation detailsMetrics below reflect the current branch (after). Δ P95 compares against the base.
Top API waits (after)
|
The S3 gateway resolves an object's content type itself, so the Storage API regression does not reach it. Sign a PutObject with SigV4 (the access key is the project ID, the secret an API key) and read the object back: an .apk declared as a plain zip downloads as application/vnd.android.package-archive, while the same bytes under a .zip key keep application/zip. The e2e client only passed a string body through for text/plain, so any other declared content type turned the body into a query string. Send a string body verbatim under any content type instead.
The test hand-rolled AWS Signature V4 — credential scope, signing key, canonical headers and signature — which restated the gateway's own verification rather than exercising observable S3 behaviour. Drive the PutObject and the mime-type read through Utopia\Storage\Device\S3, which already signs AWS4-HMAC-SHA256 and accepts a full URL as its host, so the test only sees what any S3 client sees. The e2e client no longer needs to pass a raw string body through, so its body handling goes back to what it was.
The S3 case only read the type back through the gateway's own HEAD, so it never covered what a client actually receives. Upload through the S3 device, then download each object through the Storage API and assert the Content-Type header.
…r types Sniffing reports the container a file is packed in, not the format inside it. A zip-based JAR, WAR, KMZ, 3MF, USDZ, XPI or MSIX comes back as a plain zip, a TrueType font as the generic SFNT container it shares with OpenType, and an AAR as an APK because both start with AndroidManifest.xml. The upload path and the S3 gateway each carried their own list, and the two had already drifted apart on which types count as ambiguous. Both now read app/config/storage/formats.php. No type the table maps to is in storage-mimes, so the view endpoint still serves every one of them as text/plain and nothing becomes renderable.
HarshMN2345
changed the title
The fixture was a 635-byte zip crafted to sniff as a plain archive, not an APK any build tool would produce, and the property it turned on — that its first entry sorts before AndroidManifest.xml — was invisible in review. Building the same bytes in the test would only move the fabrication, so both cases go. Real coverage needs an archive from an actual Android build, which is too large to commit.
Gradle writes JARs without the 0xCAFE extra field the JDK jar tool adds, so libmagic reports them as application/zip. The fixture is the unmodified output of a one-class 'java' plugin build.
Reuses public/fonts/Poppins-Bold.ttf, which sniffs as font/sfnt, instead of committing a new JAR fixture.
A generic or empty Content-Type returned the mapped extension type before the body was inspected, so gzip bytes named file.jar were stored as a JAR. The extension now only resolves an ambiguous declared or sniffed type, matching the Storage API.
Member
Author
TorstenDittmann
approved these changes
Sep 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Content sniffing recognises the container a file is packed in, not the format packed inside it. An APK is a zip, so one whose archive starts with
res/is reported asapplication/zip, while the same app built withAndroidManifest.xmlnear the front is reported asapplication/vnd.android.package-archive. The sniffed value is persisted as the file'smimeTypeand echoed back asContent-Typeon download, next to aContent-Dispositionfilename carrying the real extension. Responses also carryX-Content-Type-Options: nosniff, so a client that trusts the declared type over the filename saves the download under the wrong one.APK is not alone. Measured with
mime_content_type()— the call Appwrite makes — against libmagic 5.47 in the runtime image:.jarbuilt by Gradle orzipapplication/zipapplication/java-archive.warbuilt byzipapplication/zipapplication/java-archive.aarapplication/vnd.android.package-archiveapplication/java-archive.kmzapplication/zipapplication/vnd.google-earth.kmz.3mfapplication/zipmodel/3mf.usdzapplication/zipmodel/vnd.usdz+zip.xpiapplication/zipapplication/x-xpinstall.msixapplication/zipapplication/msix.ttffont/sfntfont/ttf.otfapplication/vnd.ms-opentypefont/otf.aaris the sharp edge: probing libmagic's zip rules shows it has exactly two container shortcuts, on a first entry namedAndroidManifest.xmlorclasses.dex. An Android library starts withAndroidManifest.xmltoo, so every AAR upload is currently stored as an Android app. Everything else zip-based falls through toapplication/zipunless libmagic has a full-file heuristic for it — OOXML, ODF and EPUB do, and are left alone.The fallback is driven by
app/config/storage/formats.php, alongside the existingmimes.php/inputs.php/outputs.php. The upload path and the S3 gateway previously each carried their own list and had already drifted apart on which types count as ambiguous; both now read the one table.The extension is consulted only when sniffing already returned an ambiguous container type, and every type it can resolve to is binary. None of them appears in
app/config/storage/mimes.php, soGET /viewstill serves all of them astext/plain— nothing becomes renderable, and the stored-XSS property of the view endpoint is unchanged.Test Plan
pint→{"tool":"pint","result":"passed"}on all four changed files.jartool are already detected asapplication/java-archiveand are not affected.plain.zipstaysapplication/zip, DOCX/EPUB/WOFF2 and an already-correct APK are untouched, and.aabis left alone since no registered type exists for it.array_intersect(formats.extensions, storage-mimes)is empty, as is the intersection for the ambiguous list.contentType()run in the container against real bodies: a font sent asapplication/octet-streamis stored asfont/ttf, gzip bytes namedfile.jarstayapplication/gzip, and an SVG named.ttfstaysimage/svg+xml. With no body, the extension resolves only an ambiguous or generic type.public/fonts/Poppins-Bold.ttf, already in the repo, added totestFileViewContentType. It sniffs asfont/sfntand must be stored asfont/ttfand viewed astext/plain. No new fixture.Related PRs and Issues
Two things deliberately left out:
mimeType. A previously uploaded APK or JAR stays wrong until re-uploaded; backfilling needs a migration.metadata['content_type']is still the uncorrected sniff, so on S3-backed storage the bucket object keepsContent-Type: application/zipwhile the Appwrite row is right. Download and view read the row, so the user-visible bug is fixed, but anything reading the bucket directly still sees the container type.Text formats have the same shape of problem (
.md,.yaml,.sqland.tomlall flatten totext/plain; a minified.jssniffs astext/csv) but needtext/plainandtext/csvin the ambiguous set, which is a far wider blast radius than container types. Left for a separate discussion..tsin particular must stay sniff-first — it is both TypeScript and MPEG transport stream, and both resolve correctly today.Checklist