Melbourne's Duplicate Image Problem: How the City Stacks Up Against London, Amsterdam and Seoul

Melbourne's State Library on Swanston Street is sitting on a digital archive of roughly 1.4 million images, and by the institution's own internal estimates, a significant share contain duplicate or near-duplicate files accumulated over two decades of digitisation drives. The library is now mid-way through a deduplication audit — a tedious, technically demanding process that peer institutions in London and Amsterdam largely completed by 2023.

The issue has sharpened in 2026 for a simple reason: generative AI tools have made it faster than ever to flood shared cultural repositories with visually identical or near-identical content. What was once a housekeeping problem for archivists is now a data-integrity crisis touching everything from school curriculum resources to planning department records at the City of Melbourne.

What Melbourne Is Actually Doing

The State Library's audit, which began in the second half of 2025, uses open-source perceptual hashing software to flag images with a similarity score above a set threshold — the same basic approach adopted by the British Library in London when it overhauled its digital collections ahead of its 2023–24 financial year. The difference is scale and speed. The British Library reportedly processed its backlog across an 18-month dedicated project with a ring-fenced budget. Melbourne's equivalent work is being folded into existing staff workflows across two Swanston Street-based teams, according to the library's published 2025–26 operational plan.

The Australian Centre for the Moving Image (ACMI) in Federation Square has taken a different route. ACMI overhauled its collections management system in late 2024, moving to a cloud-hosted platform that flags duplicate ingestion at the point of upload rather than retrospectively. That architecture mirrors what Amsterdam's Rijksmuseum implemented in 2022, when the Dutch institution made its deduplication logic part of the intake pipeline rather than a cleanup afterthought. ACMI's approach is considered by collections professionals to be the more sustainable model, though it requires consistent metadata standards that older Melbourne institutions have historically struggled to maintain.

The City of Melbourne's planning and permits division holds a separate, less-discussed problem. Property inspection photographs and heritage documentation images have been ingested across at least three different internal systems since 2005. A council spokesperson confirmed in June 2026 that a review of the imaging workflow was underway, though no completion date has been set publicly.

The Global Comparison Is Not Flattering

Seoul's National Museum of Korea completed a full deduplication of its 290,000-item digital image collection in 2024, publishing a methodology paper that has since been cited by archivists in New Zealand and Canada. The museum's team found that approximately 11 percent of ingested files were exact or near-exact duplicates — a figure that collections managers here consider broadly consistent with what Melbourne institutions are likely to find once their own audits conclude.

Amsterdam's municipal archive, the Stadsarchief, went further: it released its deduplication scripts as open-source tools in March 2025 under a Creative Commons licence, meaning any institution globally can adapt them at no cost. So far, no Melbourne institution has publicly announced adopting the Stadsarchief toolkit, though archivists in the Victorian Collections network — a program run by Museums Victoria that supports more than 400 community museums statewide — have discussed it in internal forums.

The practical stakes are higher than they might appear. Duplicate images degrade search results, inflate storage costs, and create licensing headaches when the same photograph has been catalogued under two different rights statuses. Cloud storage is not free: AWS S3 standard pricing as of mid-2026 sits at roughly USD $0.023 per gigabyte per month, and a poorly managed archive of 500,000 high-resolution duplicates can represent tens of thousands of dollars in unnecessary annual expenditure.

For Melbourne's institutions, the clearest path forward runs through the Victorian Government's Digital Infrastructure for Cultural Organisations (DICO) grant program, which closes its next funding round on 31 August 2026. Archivists who have reviewed the grant criteria say deduplication and collections-management infrastructure projects are explicitly eligible. Whether the State Library, regional galleries, or community museum networks submit coordinated applications — rather than competing individually — will go some way toward determining how quickly Melbourne closes the gap on cities that moved earlier and faster.