{{ message }}
googlephotos: map EXIF/IPTC/XMP description to Google Photos description field#9499
Draft
davispw wants to merge 3 commits into
Draft
googlephotos: map EXIF/IPTC/XMP description to Google Photos description field#9499davispw wants to merge 3 commits into
davispw wants to merge 3 commits into
Conversation
aa6bbc9 to
b2eb08b
Compare
5 tasks
b2eb08b to
2731cb1
Compare
5 tasks
bea07b0 to
449a3ce
Compare
When uploading media, rclone can now read description metadata from EXIF/IPTC/XMP tags and pass it to the Google Photos API as the media item description, visible in the Google Photos UI. This is controlled by two new options: - read_exif_description: enable the feature (default: false) - exif_description_fields: ordered list of tag names to try (default: Description,Caption-Abstract,ImageDescription,Title,ObjectName) The first non-empty matching tag value is used. The feature uses the github.com/bep/imagemeta library and reads only the first 512 KiB of the upload stream to extract metadata before uploading. Add unit test TestEXIFDescriptionMapping.
…identical byte deduplication
…ption fields This adds a custom HandleXMP parser to extract Dublin Core nested title and description tags (dc:title and dc:description) which are skipped by the default imagemeta parser or are otherwise ignored since they are nested tags and not attributes. This ensures that Lightroom-exported titles and descriptions successfully map to the Google Photos description on upload.
449a3ce to
e83cf5d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
This PR adds a new feature to the
google photosrclone backend: automatic mapping of EXIF/IPTC/XMP image metadata to the Google Photosdescriptionfield on upload.How It Works
Two new configuration options are introduced:
--gphotos-upload-exif-descriptionfalse--gphotos-exif-description-fieldsXMP-dc:Description,IPTC:Caption-Abstract,EXIF:ImageDescription,XMP-dc:Title,IPTC:ObjectName,XMP-photoshop:Headline,IPTC:HeadlineOn each upload, if
--gphotos-upload-exif-descriptionis set, rclone reads the source file's metadata and searches the configured tag list in order. The first non-empty value found is sent as thedescriptionin themediaItems:batchCreateAPI call.Namespace Prefixes and Custom XMP Parser
To distinguish between tags of the same name across different metadata blocks, fields can be prefixed with their namespace (e.g.,
EXIF:,IPTC:,XMP-dc:,XMP-photoshop:).While the
github.com/bep/imagemetalibrary is used to extract raw EXIF, IPTC, and XMP segments, it only parses simple key-value attributes and does not decode nested XML/RDF structures (such as Dublin Core metadata elements). To support these structures (e.g., nested<dc:title>tags generated by Adobe Lightroom Classic), this PR implements a custom XML pull-parser insideextractEXIFDescription.Values are trimmed.
Tag Priority (default order)
The default field list reflects common conventions across modern and legacy metadata editors:
XMP-dc:Description— Modern Dublin Core description (Lightroom / capture software)IPTC:Caption-Abstract— Legacy IPTC caption abstract (journalism standard)EXIF:ImageDescription— Baseline EXIF hardware/camera descriptionXMP-dc:Title— Dublin Core title panelIPTC:ObjectName— Legacy IPTC object name / titleXMP-photoshop:Headline— Modern Photoshop headline tagIPTC:Headline— Legacy IPTC headline tagConfiguration Example
Or on the command line:
Manual Test Results
Important
Step 2 (Hasher Integration) relies on the changes in
fix-equal-hash-modtime-unsupported(PR #9500). The hasher overlay features (in-flight caching on Update and local size fingerprinting) are implemented in PR #9500; testing the metadata-only update without it will result in infinite upload loops under async batch mode or cache misses due to missing remote sizes.Prerequisites
Step 1: Upload and Verify Description Mapping
Expected: file is uploaded successfully. Verify in the Google Photos web UI (https://photos.google.com):
rclone_Metadata_Testalbum and clickimage.jpg.Step 2: Hasher Integration — Detecting Metadata-Only Changes
Using the
hasherbackend overlay, the local BoltDB cache tracks MD5 checksums. Because changing EXIF metadata modifies the file's binary content,hashercorrectly detects the change and triggers a re-upload.Verify in the Google Photos web UI that the description for
image.jpginrclone_Metadata_Testnow shows "Spectacular Mountain Range" ✓Automated Tests
Self-contained unit tests are added to verify the metadata extraction:
TestEXIFDescriptionMapping: Mocks the batch upload flow using an in-memory TIFF file with an EXIFImageDescriptiontag.TestExtractEXIFDescriptionXMP: Mocks the Dublin Core nested XMP tag parsing flow using an in-memory TIFF with a synthesized XMP block.Notes
--gphotos-upload-exif-descriptionis not set (zero performance impact by default).Dependencies (gh-stack)
This PR is independent and has no dependencies.
Was the change discussed in an issue or in the forum before?
As far as I know, this was not discussed previously. This is a new feature to map EXIF/IPTC/XMP description metadata to Google Photos descriptions on upload.
Checklist