The dominant standard for metadata harvesting is OAI-PMH \u2014 the Open Archives Initiative Protocol for Metadata Harvesting \u2014 which emerged from the 1999 Santa Fe Convention bringing together the e-print and digital library communities, incubated at Cornell University. The first version was released in 2001, and the current specification, version 2.0, was published on 14 June 2002. It defines six HTTP-based verbs (Identify, ListMetadataFormats, ListSets, ListIdentifiers, ListRecords, GetRecord) that let a harvester request records incrementally, by date range or by collection, from a compliant repository.<\/p>\n
OAI-PMH is deliberately a low-barrier protocol \u2014 it doesn’t move the actual files, only structured metadata records (commonly Dublin Core, though repositories can offer richer formats too), which is what makes it practical for building union catalogs and aggregator search portals across institutions that would never agree to centralize their actual digital collections.<\/p>\n
In an enterprise DAM context, the same underlying pattern \u2014 periodically pulling metadata updates from an external system of record rather than owning that data directly \u2014 shows up in integrations with PIMs, ERPs, and rights-management databases, even when the specific protocol isn’t OAI-PMH. The design questions are the same regardless of protocol: how does the harvester know what changed since the last run, and how does it learn about deletions, not just additions.<\/p>","protected":false},"excerpt":{"rendered":"
The automated process of collecting metadata records from one or more remote repositories over a defined protocol \u2014 most notably OAI-PMH \u2014 without transferring the underlying asset files themselves.<\/p>\n","protected":false},"author":0,"featured_media":0,"template":"","meta":{"footnotes":"","faq":[{"question":"What does metadata harvesting mean in a DAM context?","answer":"In a DAM, metadata harvesting is the automated, scheduled collection of metadata records from one or more external repositories \u2014 a partner system, contributing archive, or federated source \u2014 over a defined protocol such as OAI-PMH. Rather than staff re-keying descriptions asset by asset, the DAM pulls updates on a recurring interval and indexes them centrally, while the underlying asset files themselves stay hosted at their original source system."},{"question":"What kind of DAM use case relies on harvesting?","answer":"A museum consortium's shared search portal is a typical example \u2014 it harvests metadata records from each member institution's own repository on a schedule and indexes them centrally, while the actual image files stay hosted where they originated."},{"question":"What goes wrong when a harvesting contract is set up poorly?","answer":"A poorly defined harvesting contract creates several failure modes: field schemas between source and destination don't line up, so titles, rights, or creator data land in the wrong properties or get dropped; records harvested more than once without a stable identifier produce duplicates; and mandatory fields go unmapped, leaving required metadata blank. Combined with unhandled deletions and ignored update timestamps, the aggregated catalog quietly drifts out of sync with the source systems it's meant to mirror."},{"question":"Why do deleted records cause particular problems for harvesters?","answer":"A source system may expose a harvesting endpoint but not reliably flag deleted records as deleted, instead simply omitting them from the next response. Harvesters that only look for new or changed records never learn an asset was removed, so the aggregated catalog keeps showing records for assets that no longer exist at the source."},{"question":"Does metadata harvesting move the actual asset files?","answer":"No. Harvesting moves only metadata records \u2014 titles, descriptions, rights statements, identifiers, and similar structured fields \u2014 not the binary asset files they describe. The files themselves stay stored at their original source system; the harvesting DAM builds a central index that points back to those sources. This keeps storage costs down and avoids duplicating large media, though it also means broken links appear if a source system later removes or relocates a file."},{"question":"What protocol is commonly associated with metadata harvesting?","answer":"OAI-PMH \u2014 the Open Archives Initiative Protocol for Metadata Harvesting \u2014 is the dominant standard, tracing back to the 1999 Santa Fe Convention. It defines six HTTP-based verbs (Identify, ListSets, ListMetadataFormats, ListIdentifiers, ListRecords, and GetRecord) that let a harvester request records incrementally by date range or collection, typically returning Dublin Core metadata. Deliberately low-barrier, it transfers only structured metadata, never the underlying files, making it well suited to consortium and aggregator catalogs."}],"checked_date":"2026-08-11","sources":[{"statement":"OAI-PMH emerged from the 1999 Santa Fe Convention, with the first protocol version released in 2001 and version 2.0 published 14 June 2002","source_name":"Open Archives Initiative, OAI-PMH v2.0","url":"https:\/\/www.openarchives.org\/OAI\/openarchivesprotocol.html","checked":"2026-08-07"}],"kicker":"","fact_checker":0,"reading_time":0,"revisions":[],"seo_title":"Metadata harvesting: collecting records via OAI-PMH","seo_description":"","noindex":false,"related":[2481,2642,2434,2575,2479,2437],"definition":"The automated process of collecting metadata records from one or more remote repositories over a defined protocol \u2014 most notably OAI-PMH \u2014 without transferring the underlying asset files themselves.","why":"Harvesting is how a DAM or aggregator builds a combined catalog across multiple source systems without duplicating storage: a museum consortium's shared search portal, for example, harvests metadata records from each member institution's own repository on a schedule and indexes them centrally, while the actual image files stay hosted where they originated. Getting the harvesting contract wrong \u2014 not tracking deletions, not respecting update timestamps \u2014 means the aggregated catalog quietly drifts out of sync with the source systems it's supposed to mirror.","example_rows":[{"field":"Identify","values":"Returns repository info (name, admin contact, earliest record date)"},{"field":"ListRecords","values":"Bulk-returns metadata records, optionally filtered by date range or set"},{"field":"GetRecord","values":"Returns a single record by identifier"},{"field":"ListSets","values":"Returns the repository's defined collections\/sets"}],"mistake":"A source system exposes a harvesting endpoint but doesn't reliably flag deleted records as deleted (rather than simply omitting them from the next response) \u2014 harvesters that only look for new or changed records never learn an asset was removed, so the aggregated catalog keeps showing records for assets that no longer exist at the source.","deep_link":""},"silo":[24],"class_list":["post-2451","glossary","type-glossary","status-publish","hentry","silo-glossary"],"_links":{"self":[{"href":"https:\/\/picajet.com\/articles\/wp-json\/wp\/v2\/glossary\/2451","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/picajet.com\/articles\/wp-json\/wp\/v2\/glossary"}],"about":[{"href":"https:\/\/picajet.com\/articles\/wp-json\/wp\/v2\/types\/glossary"}],"version-history":[{"count":3,"href":"https:\/\/picajet.com\/articles\/wp-json\/wp\/v2\/glossary\/2451\/revisions"}],"predecessor-version":[{"id":3631,"href":"https:\/\/picajet.com\/articles\/wp-json\/wp\/v2\/glossary\/2451\/revisions\/3631"}],"wp:attachment":[{"href":"https:\/\/picajet.com\/articles\/wp-json\/wp\/v2\/media?parent=2451"}],"wp:term":[{"taxonomy":"silo","embeddable":true,"href":"https:\/\/picajet.com\/articles\/wp-json\/wp\/v2\/silo?post=2451"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}