Deepfaked Evidence?

By: Tristan Jenkinson

Further warnings that audio and video evidence are now subject to manipulation.

Misuse of Machine Learning

An article in Legal Cheek today (which can be read here) discusses a case where deepfake technology was allegedly used to generate an audio recording of an individual issuing threats to another party over the phone.

This is not the first occasion highlighting the use of deepfake technology for illicit means – last year deepfake audio was apparently used in a phonecall equivalent of a Business Email Compromise fraud – you can read more here.

I have worked in digital forensics for some time and have investigated and reported on many files which have been fabricated or manipulated in some way. Typically these files have been emails, photographs, PDF files or Microsoft Office files. With the use of deepfake technology, parties, lawyers and judges need to be aware that multimedia files they may once have accepted at face value could also have been manipulated.

Evidence of Deepfakes

As with other types of files, audio and video files will often leave traces that they have been fabricated (i.e. fraudulently created) or manipulated (i.e. a legitimate file then fraudulently altered). The exact traces will depend on the type of data and the software that was used to generate it.

When investigating such matters, it is key that the file that the party claims is real is provided in its native format, preferably in a forensically sound manner (i.e. ensuring that the file is correctly preserved). Details should also be sought regarding the files alleged provenance, covering what device, software etc. was apparently used to take the recording.

One of the most typical approaches is to look at metadata relating to the date that the file was created. It is easily overlooked but could determine that the file was not created on the date than an alleged phone call or discussion took place.

Another approach includes looking at the details of the file and checking that they are consistent with the provenance information. For example, what audio format is used, what are the specific settings (for example the sampling rate and bit depth of a recording) as well as embedded metadata that should be present as it is created automatically by the specific software. Alternatively looking for embedded information that should not be present in the file based on the provenance information. Such approaches can demonstrate that the recording could not have been made by a specific piece of software or by a particular device.

Analysis of the waveform of the file (a visual representation of the audio content) can also identify inconsistencies that may be easier to see than hear. This approach can be used to identify many audio anomalies, including those which may indicate splicing of audio – a low-tech approach to fake audio files prior to the development of the machine learning tools which are used to develop current deepfakes.

Deepfake Detectors

There are several automated software approaches to deepfake detection, both for audio and video. For video software analysis can identify small inconsistencies such as differences in colour or illumination. For audio, there are systems such as Resemblyzer which is an open source audio deepfake detector written by Resemble, an AI voice cloning software company. Dessa (who developed RealTalk and demonstrated it by deepfaking podcast presenter Joe Rogen) have also provided open source code which can be used to build deepfake detection systems.

Catching a Deepfake

Back to the Legal Cheek article – Byron James, the family lawyer whose client was alleged to have made the deepfaked threat, has written an article providing more information on how he was able to identify the recording as a fake. The article is due to be published in the March edition of International Family Law Journal. It is sure to make interesting reading!