How Melbourne's Digital Archive Problem Became a Crisis: The Long Road to Duplicate Image Replacement

Melbourne's major public institutions are sitting on millions of duplicate digital images — the accumulated wreckage of more than two decades of overlapping digitisation drives, platform migrations and emergency scanning projects that were never properly reconciled. The State Library of Victoria, along with City of Melbourne-managed cultural venues, is now working through a structured duplicate image replacement program designed to consolidate those fractured collections into coherent, searchable archives. The work is unglamorous. It is also long overdue.

The timing matters. Victoria's government is three years into a broader digital transformation agenda that touches everything from court records to public health data. Cultural heritage repositories have largely been left on the margins of that push, but a renewed focus on the Creative State strategy — the state government's arts and culture funding framework — has forced the question of archival integrity back onto agency agendas. If institutions cannot guarantee that the image a researcher retrieves from a catalogue is the highest-quality, correctly attributed version available, the collection's usefulness collapses.

How the Duplicates Accumulated

The problem has roots in the late 1990s, when galleries, libraries and municipal councils each launched independent digitisation projects with different software standards, different file-naming conventions and no shared metadata protocol. The City of Melbourne's own records management history reflects this patchwork: the Queen Victoria Market precinct's photographic holdings, for instance, were scanned separately by at least three different contractors over a 15-year period as part of heritage assessments tied to planning applications along Elizabeth Street and in the Hoddle Grid. Each scan produced a new file. None of the contractors were required to check whether the image already existed in a central repository.

The Australian Communications and Media Authority's 2019 report on digital preservation standards flagged exactly this pattern across state and territory archives nationally, noting that redundant file proliferation drives up storage costs and degrades discoverability. For institutions operating on tight budgets — the State Library of Victoria's annual operating allocation has faced repeated flat-funding cycles in successive Victorian state budgets — storage inefficiency translates directly into reduced capacity to acquire new material.

The NGV's digital collections team, operating out of its St Kilda Road campus, encountered the same structural issue when it migrated to a new digital asset management system in 2022. Staff discovered thousands of image records where the same physical work had been photographed multiple times under different catalogue numbers, with no automated flag to link the records. The migration project ran significantly over its original timeline as a result.

What Replacement Actually Involves

Duplicate image replacement is not simply deleting the lower-resolution file. Archivists must verify that the replacement image carries the correct provenance data, that any prior citation or permalink pointing to the old file is redirected, and that rights and licensing metadata transfers cleanly. At the Public Record Office Victoria, based in North Melbourne, staff working on the Victorian Collections platform — a aggregator used by more than 300 community museums and historical societies across the state — have been developing a semi-automated deduplication workflow since mid-2024. That workflow uses perceptual hashing, a technique that identifies visually near-identical images even when file names differ, to flag candidates for human review.

The practical scale of the task is significant. Victorian Collections alone hosts more than 1.3 million object records, a figure published by the program's administrators. Even a conservative estimate of five per cent duplication across image assets produces tens of thousands of individual records requiring manual adjudication.

For institutions and community groups navigating this process, the most immediate practical advice from archivists is to audit file-naming conventions before any new digitisation project begins, and to register new image assets against a shared identifier — such as the Europeana-compatible Dublin Core standard — from the moment of capture. Retrofitting metadata is far more expensive than building it in at the start. The Hellenic Museum on Lonsdale Street and the Immigration Museum on Flinders Street have both moved toward this approach in recent years as part of broader collection management reviews. The lesson from Melbourne's experience is that the cost of not doing it accumulates quietly — until suddenly it doesn't.