Melbourne's War on Duplicate Images in Public Archives: How It Stacks Up Against Amsterdam, Toronto and Seoul

Melbourne's State Library of Victoria confirmed earlier this year that its digital collections team had identified tens of thousands of duplicate image files sitting across its online catalogues — redundant scans, re-uploaded heritage photographs and repeated digitisation of the same physical objects — consuming server capacity and degrading search results for researchers. The library, on Swanston Street in the CBD, has been working since late 2025 to deploy automated deduplication software across its holdings, a project that archivists say is still months from completion.

The problem is not unique to Melbourne, but how cities handle it varies enormously — and those differences are becoming harder to ignore as public institutions pour money into digital infrastructure. Across Amsterdam's Rijksmuseum, Toronto Public Library and Seoul's National Folk Museum, similar duplicate-image crises have been addressed with sharply different tools, timelines and levels of investment. Melbourne's response, stitched together across multiple agencies with no single coordinating body, looks fragmented by comparison.

What Melbourne Is — and Isn't — Doing

The City of Melbourne's own digital asset register, which covers images held by council for planning, heritage and public communications purposes, runs on a separate system to the State Library. The Victorian Public Record Office, based in North Melbourne, operates a third platform. None of the three systems currently talk to each other in real time. Researchers at RMIT University's Digital Humanities Lab in the CBD have been studying this siloed structure since 2024 and have raised concerns about duplicated public expenditure as well as the practical headaches it creates for archival researchers trying to trace, say, images of Flinders Lane in the 1950s through multiple repositories.

Amsterdam took a different route. The Rijksmuseum's Rijksstudio platform, which opened its deduplication pipeline to public scrutiny in 2022, uses perceptual hashing — a technique that detects visually identical or near-identical images even when file names and metadata differ. By 2024 the museum reported removing more than 40,000 duplicate entries from its public-facing catalogue. Toronto Public Library completed a similar consolidation in 2023, merging its digital image holdings into a single federated search layer after a two-year project that cost approximately CAD $1.4 million. Seoul's National Folk Museum went further, partnering with Naver — South Korea's dominant tech platform — to apply AI-assisted duplicate detection across 280,000 items.

Melbourne has no equivalent cross-institutional project on the books. The State Library's current work is internal and institution-specific. A spokesperson for the Victorian Department of Creative Industries did not respond to questions by deadline about whether a coordinated statewide approach was being considered.

Why the Gaps Matter for Researchers and Ratepayers

Duplicate images are not merely a tidy-housekeeping issue. When the same photograph appears under different catalogue numbers, automated rights-clearance tools can misfire, researchers waste time, and storage costs compound. Cloud storage pricing, while falling, still runs at roughly AUD $0.02 per gigabyte per month on standard tiers — meaning even a moderately sized archive carrying 500,000 unnecessary duplicate files at an average of 5MB each can accumulate tens of thousands of dollars in annual waste.

The Museum of Chinese Australian History on Cohen Place in the CBD and the Immigration Museum on Flinders Street have both undertaken internal deduplication reviews in the past 18 months, according to publicly available annual reports from both institutions. Neither project was coordinated with the other, or with the State Library.

For Melbourne to close the gap on cities like Amsterdam or Toronto, archivists and digital infrastructure specialists broadly agree the first practical step is a shared metadata standard — a common language for describing images across Victorian public institutions. Creative Victoria, which funds arts and cultural organisations across the state, is the most logical body to anchor such a standard, given its existing relationships with major collecting institutions. Without a lead agency willing to convene the relevant parties and commit funding, the deduplication work now underway at Swanston Street risks being repeated, expensively and independently, at every institution on the block.