Melbourne's Duplicate Image Problem: How the City Stacks Up Against London, Amsterdam and Toronto

Melbourne's major public galleries and local councils are confronting a sprawling duplicate image crisis across their digital collections, with the City of Melbourne estimating tens of thousands of redundant files sitting across its civic asset libraries — a problem that counterparts in London, Amsterdam and Toronto have been grappling with for the better part of a decade.

The issue matters now because Victoria's cultural sector is midway through a government-backed digitisation push. The State Library of Victoria, which holds more than two million photographs in its Pictures Collection, accelerated its digital cataloguing program after pandemic-era closures exposed just how poorly structured many of its holdings were. When collections go online in bulk, duplicate entries follow — the same image indexed under different metadata, uploaded by different staff, or ingested from multiple heritage databases simultaneously.

What Melbourne is doing — and where it's falling short

The State Library's Swanston Street facility began deploying perceptual hashing software in late 2024 to flag near-identical image pairs in its digital repository. The technology — which assigns a fingerprint to each image and compares it algorithmically rather than pixel-by-pixel — had already been trialled at the National Gallery of Victoria on St Kilda Road, where curators reported significant reductions in duplicated catalogue entries across its Australian art holdings after a six-month pilot.

The City of Melbourne's own Creative Archives unit, which manages digital assets for public art installations from Docklands to Fitzroy, has been slower off the mark. Sources familiar with the program — speaking generally, not on behalf of the council — say the unit has leaned heavily on manual review processes that other city governments moved away from years ago. Amsterdam's Rijksmuseum completed an automated deduplication overhaul of its 700,000-image Rijksstudio platform in 2022, cutting storage overhead significantly and improving search result accuracy for the three million users who access the platform annually, according to figures the museum published on its website. Melbourne's comparable public-facing collections remain fragmented across at least four separate platforms.

Toronto's experience is instructive. The Toronto Public Library and the Art Gallery of Ontario ran a joint deduplication project in 2023 that standardised metadata schemas across both institutions before running automated tools. The sequencing mattered: institutions that tried to automate first and clean up metadata later found they were generating new duplicates faster than the software could remove old ones. Melbourne's NGV has reportedly taken note of that lesson, though the gallery has not made a public statement on its current methodology.

The cost of doing nothing

Digital storage is not free. Cloud storage costs for cultural institutions running large unoptimised image libraries typically run into hundreds of thousands of dollars annually at enterprise scale, according to publicly available pricing from major providers including Amazon Web Services and Microsoft Azure. For smaller inner-city councils — Yarra, Moreland, Port Phillip — the burden tends to land on already stretched IT budgets rather than dedicated digital asset management lines.

London's Victoria and Albert Museum published a case study in 2023 showing that after a two-year deduplication project it had reduced its active image file count by roughly 30 percent, freeing storage and, more importantly, improving the accuracy of its public-facing search tools. The V&A project cost approximately £400,000 across the two years, a figure cited in the museum's annual report. Melbourne institutions have not published comparable cost data publicly.

The practical upshot for Melbourne is sequencing and coordination. The State Library, NGV and City of Melbourne's digital teams are each running separate processes on separate timelines with no shared deduplication standard. A coordinated approach — similar to the joint framework that Toronto's institutions adopted — would likely produce better results than three parallel programs producing incompatible outputs.

The Victorian Government's Creative State 2025–2028 strategy, released by Creative Victoria, nominates digital access as a priority without specifying technical standards for image asset management. That gap is where the duplicate problem tends to grow. Institutions watching this space say the next twelve months, as several major digitisation grants reach their completion milestones, will test whether Melbourne's piecemeal approach has actually worked — or whether the archive bloat simply moves to a new platform.