Reference Glossary
Metadata harvesting
The automated process of collecting metadata records from one or more remote repositories over a defined protocol — most notably OAI-PMH — without transferring the underlying asset files themselves.
Why it matters in a DAM
Harvesting is how a DAM or aggregator builds a combined catalog across multiple source systems without duplicating storage: a museum consortium's shared search portal, for example, harvests metadata records from each member institution's own repository on a schedule and indexes them centrally, while the actual image files stay hosted where they originated. Getting the harvesting contract wrong — not tracking deletions, not respecting update timestamps — means the aggregated catalog quietly drifts out of sync with the source systems it's supposed to mirror.
A worked example
Common mistake
A source system exposes a harvesting endpoint but doesn't reliably flag deleted records as deleted (rather than simply omitting them from the next response) — harvesters that only look for new or changed records never learn an asset was removed, so the aggregated catalog keeps showing records for assets that no longer exist at the source.
The dominant standard for metadata harvesting is OAI-PMH — the Open Archives Initiative Protocol for Metadata Harvesting — which emerged from the 1999 Santa Fe Convention bringing together the e-print and digital library communities, incubated at Cornell University. The first version was released in 2001, and the current specification, version 2.0, was published on 14 June 2002. It defines six HTTP-based verbs (Identify, ListMetadataFormats, ListSets, ListIdentifiers, ListRecords, GetRecord) that let a harvester request records incrementally, by date range or by collection, from a compliant repository.
OAI-PMH is deliberately a low-barrier protocol — it doesn’t move the actual files, only structured metadata records (commonly Dublin Core, though repositories can offer richer formats too), which is what makes it practical for building union catalogs and aggregator search portals across institutions that would never agree to centralize their actual digital collections.
In an enterprise DAM context, the same underlying pattern — periodically pulling metadata updates from an external system of record rather than owning that data directly — shows up in integrations with PIMs, ERPs, and rights-management databases, even when the specific protocol isn’t OAI-PMH. The design questions are the same regardless of protocol: how does the harvester know what changed since the last run, and how does it learn about deletions, not just additions.
Frequently asked
What does metadata harvesting mean in a DAM context?
The automated process of collecting metadata records from one or more remote repositories over a defined protocol — most notably OAI-PMH — without transferring the underlying asset files themselves.
What kind of DAM use case relies on harvesting?
A museum consortium's shared search portal is a typical example — it harvests metadata records from each member institution's own repository on a schedule and indexes them centrally, while the actual image files stay hosted where they originated.
What goes wrong when a harvesting contract is set up poorly?
Not tracking deletions or not respecting update timestamps means the aggregated catalog quietly drifts out of sync with the source systems it's supposed to mirror.
Why do deleted records cause particular problems for harvesters?
A source system may expose a harvesting endpoint but not reliably flag deleted records as deleted, instead simply omitting them from the next response. Harvesters that only look for new or changed records never learn an asset was removed, so the aggregated catalog keeps showing records for assets that no longer exist at the source.
Does metadata harvesting move the actual asset files?
No — only the metadata records are collected and indexed centrally; the underlying files remain hosted at their original source system throughout.
What protocol is commonly associated with metadata harvesting?
OAI-PMH is named as the most notable protocol used for this kind of scheduled, automated metadata collection between repositories.
Sources
- OAI-PMH emerged from the 1999 Santa Fe Convention, with the first protocol version released in 2001 and version 2.0 published 14 June 2002 checked 2026-08-07 — Open Archives Initiative, OAI-PMH v2.0