18 September 2007 : Backup

The theme of this Forum was Backup, with a specific focus on issues surrounding data retention policies and retrieval of old data. Discussions centred on the scenario below:

Scenario: Data Retrieval 18 September 2017

The year is 2017 and Jules is head of IT at your practice.

Jules has just had a flash from one of the Management team in your office. A Director has just been advised by one of the legal team that your practice has a serious problem concerning a building component from a drawing issue on Project Auckland, dated 10 September 2007. The Director requires all the data making up the issue to be provided to her by the end of the week, and emphasises that the retrieved data should satisfy legal tests of veracity.

Questions

  1. How will Jules know where the material from 2007 is to be found?
  2. What registers do you currently provide that Jules could refer to, to help him track down the backed-up material?
  3. What registers are likely to be available in 2017 to help Jules find the reference?
  4. Is it possible that these data will not have been backed up by your practice?
  5. How would Jules learn that a weekly snapshot backup on 14 September 2007 had failed?
  6. Will the backup medium be readable by Jules?
  7. Will the data in the backup be readable by contemporary programmes? If not, how would Jules read the data?
  8. How will Jules try to prove the veracity of the data he retrieves?
  9. How long does it currently take your office to retrieve data from a week, two years and 10 years ago?
  10. What are the legal data retention periods for the projects in your office?

Notes from the Forum

The following summary lists the main points raised in the discussion:

Live Storage versus Tape Archive
Most practices are using a combination of live storage (that is, keeping all data live on disk) and tape archiving. Operating System level snapshots, such as those provided by NetApp, VSS or LVM, allow incremental snapshots to be taken in moments. Typically snapshots of all live data are taken at least weekly to LTO standard tape, and stored offsite.
One company runs incremental backups to disk and does synchronisation between sites for Disaster Recovery. The incremental data and server are hosted offsite. In one sense this seems an elegant and inexpensive solution. However, it provides only a single point of failure for data that may have been excised from the live data set. Despite this weakness, it is an interesting approach.
The Point in Time dilemma
Most of those attending noted that incremental or "fine-grained" backups were typically required only for a four- to six-week window period, which covered files that might have been inadvertently corrupted or deleted. After this, snapshots are needed only to recover large data sets, missing files or project milestone information releases. For those using tape backups, avoiding the retention of "fine-grained" backup data into the future has the advantage of saving money.However, the participants were generally concerned that this would mean having too few snapshots in the event of backup or media failures, which would expose their firms to serious difficulties in situations like the Jules scenario.
The provision of project milestone backups offers an attractive solution. As projects mature, backup retrievals that pre-date the "fine-grained" window backup tend to be required on a project milestone basis, such as "Stage D", from an office worker's point of view. This approach could be seen as more user-friendly than global system snapshot backups, but it is not clear how such backups could be retrieved.
Auditing
Few attendees audited their backups.
Records and Policies
Records and Policies are important, because they will enable people to retrieve data easily from backup files in the future. Records should not only reveal the contents of the backed-up material, but also indicate the physical location of the backup media, details of medium format and so on.
Backup Retention Periods
The attendees typically intended to keep backups for 15 years.
Attendees:
  • David Appel, Fielden Clegg Bradley
  • Hannah Heathcote, Hopkins Architects
  • Jochen Glemser, David Chipperfield Architects
  • Jeremy Mitchell and Lee Howard, Allies and Morrison
  • Rory Campbell-Lange, Tim Whiteley and Mark Adams, Campbell-Lange Workshop

Links

NetApp snapshot overview.
Windows Volume Snapshot Service/VSS.
LVM snapshot example.