Abstract
Digital open source information (DOSI) has emerged as a significant source of evidence for the International Criminal Court (‘the Court’) and other jurisdictions applying international criminal law. However, its use in litigation raises questions about who can be considered an expert in DOSI and what methods and safeguards they should adopt. This article examines how the Court can receive DOSI via expert evidence while maintaining rigorous fact-finding standards. It addresses challenges that DOSI introduces to the paradigm of expert testimony, including the lack of an overarching formalized system of DOSI accreditation, the typically group-based nature of DOSI investigations and the scope for misinterpretations and biases to result in erroneous conclusions. It proposes a novel six-factor test for the Court’s identification of sufficiently qualified DOSI specialists as expert witnesses. At the same time, it highlights that DOSI specialists should utilize transparent, accessible and replicable methodologies, with quality control feedback loops, peer review processes and bias controls. The aim of the article is to facilitate the use of DOSI evidence to resolve atrocity crimes cases, while also maintaining rigorous fact-finding standards and conforming with due process and fair trial rights.
1. Introduction
Let there be no mistake. As science continues to change the social world, great transformations of factual inquiry lie ahead for all justice systems. These transformations could turn out to be as momentous as those that occurred in the twilight of the Middle Ages, when magical forms of proof retreated before the prototypes of our present evidentiary technology.1
The use of digital open source information (DOSI)2 raises fundamental questions about the normative framework governing expert witnesses before the International Criminal Court (‘ICC’ or ‘the Court’). In particular, the increasing prevalence of DOSI in ICC proceedings demands scrutiny of the definition, role and methodology of the expert witness.3 Unlike regular witnesses,4 expert witnesses are permitted to testify about matters that they did not experience in person.5 This powerful role can have a major impact on the outcome of a trial, particularly in international cases where traditional crime scene evidence may be difficult to obtain.6 However, DOSI itself fits awkwardly with the approach of international tribunals towards expert evidence because (i) it challenges the binary distinction between experts and lay witnesses; (ii) its group-based ontology is incongruous with the traditional single expert paradigm; and (iii) its dynamic nature must be reconciled with the need for transparency and replicability.7
To redress the potential dissonance between DOSI and expert evidence, this article explores pathways for the Court to facilitate the receipt of DOSI whilst maintaining rigorous fact-finding standards. It addresses how DOSI specialists can mitigate the risks of their work being considered unreliable and/or inadmissible. Significantly, it proposes a novel six-factor test for the identification of DOSI specialists as expert witnesses.
The central thesis of this article is that a firm distinction between experts and lay witnesses should be maintained as part of the Court’s normative architecture. This will serve the interests of reliable fact-finding and will strengthen the legitimacy of the ICC’s determinations. At the same time, it is posited that the ICC (and other institutions applying international criminal law) along with the DOSI community, can make adjustments to their working methodologies to assist the investigation and adjudication of atrocity crimes. International courts can accommodate DOSI by establishing clear, objectively justified, accessible and non-biased parameters for the recognition of DOSI specialists as expert witnesses. For their part, DOSI specialists should utilize transparent, accessible and replicable methodologies, with quality control feedback loops, peer review processes and bias controls.
By examining the use of DOSI under the expert evidence framework, this article enters novel territory. It differs from previous assessments of expert evidence under international law, which have tended to focus on the more ‘traditional’ evidentiary materials presented before international courts such as blast analysis and DNA identification.8 Similarly, it differs from studies on DOSI, which have tended to focus on DOSI’s probative value in abstracto, without assessing how it will be submitted into evidence before courts such as the ICC. Indeed, several leading publications on DOSI’s role for atrocity crimes omit any mention of expert evidence altogether.9 Those which address expert evidence contain only limited analysis of the applicability of the framework governing expert testimony to DOSI.10 Even much-cited manuals on DOSI, such as the Berkeley Protocol on Digital Evidence, lack any detailed assessment of expert evidence in this respect.11
Conversely, this piece sets out a thorough assessment of the normative framework governing DOSI evidence and of the challenges it presents for the ICC itself and for the DOSI community. Significantly, the present article makes a unique contribution to scholarship on international criminal justice by proposing a framework to determine who may qualify as a DOSI expert before the ICC. The framework consists of six criteria, which draw together guidance from jurisprudence, academic commentary and experience.12
The formulation of this framework for DOSI expert evidence provides an important resource for the Court’s treatment of DOSI materials. Because DOSI is typically collected remotely by people who may have played no role in its creation,13 opinion evidence will be needed to explain and verify it, normally through an expert witness.14 Using the normative framework governing expert evidence will help insulate against the misinterpretation or mishandling of DOSI.15 It will also help to ensure that the increasingly important process of16 gathering and presentation of DOSI materials is conducted lawfully and without violating procedural rules or human rights protections, or otherwise harming third parties.17
The need to examine how DOSI can be submitted via expert evidence is also important for other institutions applying international criminal law. In recent years, documentation efforts by civil society have augmented and the use of digital materials in court proceedings has correspondingly increased.18 Beyond the ICC, DOSI is used by international organizations such as UN fact-finding missions,19 human rights organizations, investigative journalists20 and domestic courts in atrocity crimes cases conducted under universal jurisdiction.21 Ensuring that the normative framework governing expert evidence can accommodate DOSI is essential for the wider international criminal justice endeavour.22
2. The Growing Need to Address DOSI in the Context of Expert Evidence
DOSI captures a broad array of materials including video, photos, websites, satellite imagery, drone footage, machine logs, financial transactions and government records.23 The generation, gathering and verification24 of DOSI involve huge volumes of material and are highly dynamic phenomena.25 Each minute, in excess of 350,000 tweets are posted on Twitter (now ‘X’), and over 500 hours of video are uploaded to YouTube.26 Much of this material is contained in formats never tested before international courts. Despite the ground-breaking substantive and procedural advances made by the 1990s ad hoc tribunals, they relied far more on analogue evidence than digital evidence.27 Consequently, the ICC must forge its own approach to DOSI with little in the way of authoritative precedent to look to for procedural guidance.
To date, there are very few, if any, instances where the ICC has relied on DOSI for a critical finding.28 Whereas DOSI played a central role in the Al-Werfalli proceedings, the case did not progress to trial due to his reported death while at large.29 In Lubanga, the Trial and Appeals Chambers relied on videos, but they do not appear to have been DOSI in the present sense,30 and their technical aspects were not subject to detailed discussion.31 The Bemba Trial Chamber’s admission and reliance on media recordings of Radio France Internationale Journal Afrique32 was overturned by the Appeals Chamber, which described the evidence as ‘weak, often consisting of media reports including anonymous hearsay’.33 In the contempt case of Bemba et al., the Prosecution tendered Facebook photographs to demonstrate the relationship between the parties to the bribery scheme, but the Trial Chamber did not explicitly address the social media posts in its judgment.34
The ICC has had occasion to address DOSI-related issues, including in the context of expert evidence, particularly in the Mali cases. However, in Al Mahdi, the expert evidence was not challenged, as the parties stipulated that they would not offer evidence or submissions inconsistent with the plea agreement.35 In Al Hassan, a video analysis expert was instructed by the Prosecution to use digital evidence and platforms such as Google Earth to geolocate monuments in the Timbuktu area in Mali, but the judgment had not been issued at the date of writing this article.36 Consequently, the Court has not had to grapple with the full extent of litigation challenges that DOSI may present.
It is imperative that the Court adapt its procedures to accommodate the technical challenges which surround DOSI when presented as evidence. Current situations before the Court, such as those in Libya, Ukraine and Palestine, promise to contain significant volumes of DOSI materials. For example, Libyan Marshal Khalifa Haftar, who is reportedly being monitored by the ICC,37 could see DOSI used in a putative case against him for multiple purposes, including to demonstrate the executions committed by his lieutenant now-deceased Mahmoud al-Werfalli,38 to show his own statements reportedly directing that no quarter should be given to prisoners,39 and to prove the existence of armed conflict. If so, experts will inevitably be sought to verify the authenticity and reliability of the DOSI materials, including through the geolocation, chronolocation, reverse-image searching and meta-data analyses discussed herein. In the context of Ukraine, aerial and satellite type DOSI has already been heavily relied on for civil society investigations,40 online statements from the Kremlin have been cited by the Independent International Commission of Inquiry on Ukraine,41 and posts in Telegram channels and other platforms have conveyed inflammatory statements by influential figures,42 which could provide a basis to charge incitement to genocide and persecution. Issues of accessibility and fairness arising from the use of such materials have already been raised,43 which will likely lead to the engagement of DOSI specialists as experts. Similarly, the current conflict between Hamas and Israel has seen a torrent of DOSI materials being disseminated, including footage of the 7 October 2023 attack on the Nova music festival, interviews with detained Hamas attackers and missile and rocket strikes by both parties. Any cases that emerge from these clashes will involve a large proportion of DOSI type materials, which in many cases will have been widely viewed, shared, and in some cases altered online, again necessitating the involvement of DOSI experts to analyse and verify their contents and provenance. Procedural battles over the parameters of expert testimony, including who can qualify as an expert, will arise in connection with any ensuing legal proceedings.
The six-factor framework proposed in this article for determining DOSI experts44 can assist in those situations and others featuring heavy DOSI components. Whereas formal adoption of this framework is a matter for the Assembly of States Parties or the Court (depending which instrument they are introduced into),45 the criteria can nonetheless be used by chambers and by organizations as a guide in their current work on cases potentially emerging from these situations, particularly regarding specialized work to gather, analyse and verify DOSI materials.
Under the court’s current procedures, the lack of an established normative framework specifically governing the admission of DOSI is concerning. The potential misuse of DOSI continues to grow in magnitude and sophistication. Modern technology can already produce fakes that are indistinguishable from real footage to the naked eye.46 In 2022, a deepfake video of Ukrainian President Volodymyr Zelensky calling his troops to surrender to Russian forces was circulated on social media.47 Other images published by international media purported to show Buddhist monks burning Rohingya victims, but actually depicted the cremation of victims of China’s 2010 earthquake.48 The risks of misplaced reliance on DOSI can be exacerbated when the technology is new and visually impressive, such as digital reconstruction technology.49
Detecting fake or manipulated DOSI materials will increasingly require specialized analysis.50 This includes comprehensive metadata checks, and reverse image searches, as well as more sophisticated tools and techniques.51 Analyses of DOSI material are best conducted by persons with specialized skills and knowledge in this domain, who can testify in relation to its provenance, reliability and authenticity.52 Those persons would usually be led as expert witnesses, particularly if they did not originally discover the DOSI materials themselves but instead were involved in verifying and interpreting it. In this light, it is apposite to examine whether the ICC’s procedural framework governing expert evidence is suitable for application to DOSI.
3. The ICC’s Minimalist Procedural Framework Concerning Expert Evidence and DOSI
A. ICC Normative Prescriptions of Particular Importance to Expert Evidence
In keeping with the historical precedents of ‘relaxed’ evidentiary regimes in war crimes proceedings,53 the ICC’s procedural framework is flexible and open.54 This minimalist and flexible approach is particularly notable with respect to expert evidence, which contrasts with the more detailed prescriptions on expert evidence applied in some domestic systems.55
The gateway question is who can qualify as an expert witness.56 In Ntaganda, the Court relied on ICTY jurisprudence in describing an expert as ‘a person who, by virtue of some specialized knowledge, skill or training can assist the Chamber in understanding or determining an issue of a technical nature that is in dispute’.57 Whether or not a person will be considered to have sufficient expertise on a particular matter depends on the person’s education, experience in the relevant field, publications and additional background relating to the subject on which they would testify.58 Despite this broad framing, the range of persons recognized as experts at the ICC and other international institutions has typically focused on traditional categories of scientific assessment. The ICC’s List of Experts application form refers to the following categories of expertise: ballistics; finance (financial investigations/freezing assets); forensic medicine; graphology; psychology; and reparations, as well as situation-specific expertise on history; judicial systems; military science; policing; politics and geopolitics; and linguistics.59 There is no overt mention of DOSI or one of its sub-categories.60 Whilst persons in non-enumerated fields, such as DOSI, may apply to be included on the List of Experts, they ‘must explain how their expertise is relevant to ICC proceedings’.61
The jurisprudence of international courts has established parameters for the delivery of expert evidence. Fundamentally, expert witnesses are permitted to provide their opinions on matters exceeding their direct sensory experience.62 They can comment on evidence, addressing factors beyond those immediately apparent on its face (apparent features can be conveyed by lay ‘summary’ witnesses).63 Importantly, the expert’s evidence should lie outside of the typical lay person’s knowledge.64 This justifies the expert being able to provide their opinions as evidence and reflects the ‘special epistemic nature’ of expert witnesses in litigation.65 In light of this special role, ICC jurisprudence has established that experts must provide their evidence with the ‘utmost neutrality and objectivity’.66
Substantively, a specialist’s expertise must cover the subject-matter of the evidence to be adduced.67 Moreover, in theory, experts should not opine on ultimate issues of fact or law which are contested and fall into the province of the Chamber to determine.68 Expert evidence which would usurp the functions of the Chamber include, for example, opinions as to an accused’s guilt or innocence, or whether the contextual, material or mental elements of the crimes charged are satisfied.69 At the International Criminal Tribunal for Rwanda (ICTR), this prohibition resulted in the exclusion of Dr Bernard Lugan’s views on the ‘planning of genocide, the legitimacy of civil defence, and evidence regarding the conduct of the accused’.70 However, the rule is subject to a range of interpretations. In other instances, experts have been allowed to testify on matters essentially amounting to elements of offences.71
Procedurally, experts typically provide a report, setting out their methodology and findings, which is disclosed to the other party well in advance of the expert testifying. The content of the report and proposed testimony must fall within the witness’s expertise and the evidence must potentially be of assistance to the Chamber.72 ICC judges can proprio motu order experts to provide a report and to testify before them.73 They can also order jointly commissioned expert reports74 and can order experts to testify at the same time75 In this way, Chambers can control the mode of expert instruction, the manner in which their evidence is to be presented and the time limits for the preparation and notification of their reports.76 While the ICC approach was intended to expand the judicial role in the instruction of experts,77 in practice the parties have instructed experts in many of the cases heard to date as discussed herein.
B. The Treatment of DOSI Under the ICC’s Normative Framework
Although there is little direct prescriptive regulation of DOSI in the ICC’s framework, its jurisprudence has set out broad guidance of potentially relevance, such as requiring parties to indicate the date and location from which open source materials were obtained.78 There is also judicial and academic support for DOSI being verified for its provenance, meta-data, potential tampering,79 misattribution and authorship80 and authenticity.81 Establishing those (counter-)indicia of reliability will also be important for the submission/admission and weight of DOSI.
DOSI will frequently contain hearsay. The ICC has expressed considerable reluctance to admit open source materials based on anonymous hearsay,82 such as from NGOs, the United Nations and media sources.83 The scepticism reflects concerns about the conditions in which hearsay is typically received — with no oath, no ability to cross-examine on issues of recollection, perception, sincerity and meaning, along with no ability to observe the statement maker’s demeanour.84 Relatedly, the best evidence rule, which has been applied by international tribunals,85 provides that ‘the Trial Chamber will rely on the best evidence available in the circumstances’.86 For DOSI, where there are frequently multiple copies of an item or post, sometimes amplified with adjustments and extraneous material added, the Court will be assisted in its deliberations if parties seek to submit the most authoritative ‘original’ version of the evidence.
Other than those broad and non-DOSI specific forms of guidance, to date, the ICC has not adopted any specific legal rules (in the formal sense under Article 21) regarding the collection, preservation, admissibility and verification of DOSI.87 In the absence of any detailed legal regulation of DOSI, expert evidence will be particularly important for its authentication and interpretation.
An important underlying question concerns the taxonomical categorization of DOSI. Some commentators classify DOSI as documentary evidence,88 others as real evidence89 or even as a mix of the two.90 At the same time expert views on DOSI may be seen as testimonial evidence.91 This disparate terminology highlights the need for conceptual clarity regarding DOSI.
Typical DOSI, such as videos or photos, involve editorial choices, at minimum in the sense of determining where to point the camera and when to start and end the images or video that are shown.92 Whilst such items could be introduced as either documentary evidence or real evidence, the former is more likely than the latter, as they will be tendered to prove the veracity of their contents rather than simply their existence and physical features.93
The evidentiary categorization of DOSI is particularly important when it comes to the purpose of expert evidence. If the images or video are real evidence, which ‘speak for themselves’,94 then the expert’s role (if any) would logically be restricted to questions of verification and authenticity. However, if the DOSI is documentary evidence, then expert testimony will be required to explain and interpret the evidence, including its contents, for the fact-finders. This is inherently a case-by-case analysis. But it is clear that the more forensically relevant the DOSI contents, the more likely the parties will contest the veracity of the contents or the way in which they have been edited. Resolving these issues will involve technical tests, such as ‘geolocation, chronolocation, checks for internal consistency within a video and source analysis, checks for consistency across multiple items purporting to depict the same event, and other, ad-hoc, methods’.95 These would likely exceed the normal bounds of lay persons’ experience.96 In this way, tendering DOSI will often require the accompanying expert opinion, as ‘matters of personal opinion or expertise beyond the remit of a factual witness must be excluded’.97
The preceding survey shows that the ICC’s rules and jurisprudence sit uneasily with DOSI materials in litigation, revealing gaps and inconsistencies even at the taxonomical level, which will only grow as the potential for manipulation of digital materials increases. Addressing the challenges posed by DOSI to the ICC’s regime governing expert evidence is imperative in light of the wide accessibility and growing prevalence of digital materials in international crimes litigation.98
4. Challenges to the Established Paradigm Governing Expert Evidence
A. DOSI Tests the Binary Between Experts and Lay Witnesses
The accessibility and democratizing nature of DOSI presents challenges to the existing binary between expert and lay witness.99 In relation to a novel form of expertise (acoustic analysis), Lord Justice Bingham observed in the English case of R v. Robb:
the risk that if, in a criminal case, the Crown are permitted to call an expert witness of some but tenuous qualifications the burden of proof may imperceptibly shift and a burden be cast on the defendant to rebut a case which should never have been before the jury at all … a defendant cannot fairly be asked to meet evidence of opinion given by a quack, a charlatan or an enthusiastic amateur … .100
Lord Bingham’s reference to enthusiastic amateurs has particular resonance for DOSI-based evidence. Easy access to the tools and underlying material mean that no qualification is required to begin conducting DOSI investigations. Leading DOSI specialist Eliot Higgins has argued that ‘OSI analysis can be performed by anyone — it is easy to learn how to do’.101 He himself is a ‘self-taught college dropout’.102 Consequently, participants range from renowned experts to undergraduate students with no prior experience or particular technical expertise.103 In this respect, the inclusive and democratizing characteristics of DOSI fit awkwardly with expert status, which has traditionally been based on a level of specialized skill or knowledge, beyond the usual remit of a lay person or judge.104 As a growing proportion of society becomes digitally literate and computer savvy, it will be increasingly difficult to maintain that basic forms of digital verification constitute a specialized form of knowledge exceeding the normal capabilities of fact-finders.105
In determining which type of specialized knowledge will be accepted as expert evidence for DOSI materials, it is instructive to review examples of specialists on audio-visual type materials who have been recognized as experts before the ICC. These experts typically utilized an array of technical analyses with a statistical backing to their findings and/or had certification or membership of widely recognized industry bodies.
For example, in the ICC case of Al Hassan, a voice authentication expert compared items under examination (purportedly containing clips of the accused speaking) against reference items from his appearances in court.106 Whereas experts’ CVs are usually provided,107 this expert’s CV was not disclosed publicly. But he testified that he had 10 years of experience in the area and attended expert group sessions of the European Network of Forensic Science Institutes on voice recognition.108 He noted that his analyses incorporate highly technical software, including automatic voice comparison using an algorithm-based system, which accounted for the acoustic context and inter-variability between moments of speaking, and generated likelihood ratios of a match between the items under examination and the reference items.109 Given its ostensibly scientific and statistically grounded nature, this type of assessment is analogous to traditional expert evidence, such as DNA match analysis.
Another example of specialized skills considered sufficient to qualify as an expert is those of ICC Forensic Officer Amy Hak, who was called as an expert on forensic video analysis in Al-Hassan.110 In addition to setting out her professional roles related to video analysis, her CV included reference to her previous experience of being an expert witness in multiple jurisdictions, such as Canada, details of the numerous trainings she had undertaken and her certification as a forensic video analyst.111
Aside from the high degree of technical skill involving in these experts’ assessments, their certification and/or membership of professional bodies indicates that possession of industry qualifications will increase the likelihood of being recognized as an expert. This is also conveyed by the Court’s expert application form, which asks whether the applicant is registered with a professional body in their home jurisdiction and carries professional insurance.112
In this respect, it is notable that there is no overarching qualification or certification in DOSI that has formal recognition at the national or international level on a par with status as a medical doctor or forensic pathologist, for example.113 Some leading international justice related organizations offer professional training in DOSI,114 and more academic courses have been developed in this field in recent years.115 Many private entities offer certification in DOSI in the context of cybersecurity.116 However, those courses have not received the formal recognition from State structures equivalent established regulatory bodies such as national bar councils for lawyers or medical boards for doctors. The lack of formal qualification system overarching DOSI will make it difficult for judges to determine who can be considered an expert witness regarding DOSI.
In lieu of a system of formal qualifications, one option is to lower the threshold of expertise required and eschew any insistence on certification. This could see DOSI investigators and enthusiasts liberally accepted as experts. In some domestic jurisdictions, the scope of acceptable expert evidence has been extended to encompass areas beyond established scientific disciplines, including ‘fingerprints, handwriting and accident reconstruction … the market value of land, ships, pictures or rights … the quality of commodities, or on the literary, artistic, scientific or other merit of works alleged to be obscene’.117 Allowing DOSI specialists into the expert club would follow in the same evolutionary spirit.
However, broadening the parameters may lower the quality of the analyses and may exacerbate claims of selectivity in determining who is an expert. Questions of subjectivity have previously been raised, with the process described as ‘essentially a subjective one, with normative judgements being formed on the basis of an expert’s education, experience, and other pedigrees, and often, the quantum of his or her white hair’.118 For example, before the ICTR, ‘proposed expert witness Jean Rubaduka was not qualified as an expert in Rwandan constitutional law, although he was a member of the Rwandan constitutional court and conseil d’etat, and had taught law at the National University of Rwanda’.119 Further, ‘[a]n academic teaching on Rwanda at the School of African and Oriental Studies (SOAS), London, was not qualified as an expert’ on the basis that ‘he lacked a PhD in the subject, he was working part-time as an optometrist, and had never conducted field research in Rwanda, and the journals in which he had been published lacked pedigree’.120 At the same time, other chambers have qualified persons as experts simply based on their deep knowledge of the subject, without the types of formal qualifications traditionally sought for this categorization.121
To avoid fuelling arguments about disparate treatment and biased selectivity, it will be important for the Court to establish objectively justifiable criteria for its qualification of experts. The move to objective criteria reflects a broader underlying shift away from expert status being accorded to a small number of typically privileged individuals based on relatively opaque assessments, towards a more accessible set of criteria which may be met by any person who can demonstrate sufficiently specialized skills and/or knowledge. Objective criteria also help avoid any ‘adverse distinction founded on grounds such as gender, age, race, colour, language, religion or belief, political or other opinion, national, ethnic or social origin, wealth, birth or other status’.122 To these ends, six criteria that could be used to discern DOSI experts are set out further below.123
B. DOSI as a Group-based Form of Information Gathering
The expert witness is often envisioned as an enlightened individual on the stand educating the judges or jury about a complex technical issue that the expert has years of experience working on.124 The language of ICC jurisprudence demonstrates this stereotype,125 referring to ‘a single, impartial and suitably qualified expert’.126 Even the ICC’s application form for organizations to be listed as experts, which theoretically permits group-based expert activity to be recognized before the Court, explicitly requests the ‘name of the person designated to represent the expert organization’.127
DOSI challenges that individually focused paradigm. It is frequently gathered by multi-disciplinary groups of persons of varying expertise and levels of experience, working collaboratively.128 DOSI may be collected and verified by multiple parties across various platforms.129 This collaborative, group-based activity places a significant burden on the lead investigator to ensure coherence and overall standards and procedures applied by the group. Where there is collaboration between civil society groups,130 or between State entities and individual volunteers (e.g. the identification of 6 January Capitol rioters),131 or between international organizations and individual volunteers (e.g. Europol’s Trace an Object campaign),132 the risk of inconsistent approaches is acute.
The fact that DOSI is typically group-generated is not an absolute bar to its admission qua expert evidence.133 The immediate question that would arise is who from the collaborative team should serve as the expert witness or witnesses.134 In analogous situations, a single team member has testified as an expert and others provided factual testimony.135 Nonetheless, caution must be taken; if the expert has only joined the project ex post facto, an opposing party will have strong grounds to object to their testimony, arguing that it cannot comprehensively explore other team members’ potential errors and biases.
Howsoever chosen, the designated ‘expert’ (or experts) should be instructed by the litigant to personally conduct or oversee the analyses. If additional help is necessary, they should be told to explicitly set out the identity and contribution of any other person and include those other persons’ CV(s).136 During cross-examination, the expert may be asked about additional persons who have assisted with their experiments or analysis and should note the name of any such persons in their expert report.137 Under UK law, for example, experts are required to state in their reports not only their own qualifications and the instructions received but also ‘[i]nformation relating to who has carried out measurements, examinations, tests etc and the methodology used, and whether or not such measurements etc were carried out under the expert’s supervision’.138 Under no circumstances should the expert should try to pass off the work of others as their own expert evidence without acknowledgement.139 The DOSI team also should be aware that once testimony begins, the designated member(s) will not be able to communicate with them (barring exceptional permission being granted by the Court). In a recent domestic case, the court explicitly criticized an expert who, after being sworn in, called several people outside of court to discuss issues he was being questioned on.140
The composition of a DOSI team will be particularly contestable during trial proceedings if the lead investigator, or any other member of the team, is affiliated with one of the parties to the litigation. Such affiliation is not an absolute bar; in the ICC case of Al-Hassan, the report of Amy Hak, then a Prosecution Forensic Officer, was submitted during proceedings.141 Further, a number of prosecution analysts have testified in proceedings before the ICTY.142 However, some judges have considered this inappropriate, such as when Philip Coo’s military report was rejected in the Milutinović et al. case due to his position as a member of the Prosecution personnel.143
The single-expert model would clash with the democratizing trend for collaborative DOSI research, noted above. It is unlikely that one single person will be a sufficient repository of knowledge to be able to testify comprehensively about an entire DOSI investigation.144 For example, investigators of human rights violations against Rohingya have used both satellite imagery and social media posts to confirm the burning and pillaging of villages by the Myanmar military.145 An expert on satellite imagery may be reluctant to also testify as an expert on social media posts. Adhering to the single-expert model may undermine the efficiency of a DOSI investigation, by funnelling all information through one person, and thereby creating analytical and decision-making bottlenecks. For these reasons, international courts must be receptive to having more than one member of an DOSI team testify regarding their particular specialized function and findings, or to having a single member testify but with an appreciation that the expert will not have performed each function underlying the team’s findings and may have instead served in a reviewer role. Doctrinally, this means the expert’s evidence will be very much opinion in nature rather than a relaying of their direct experiences.146 This shift towards opinion heightens the need for rigorous standards of review and transparency to be adopted within DOSI teams, as set out herein, and to be vigilantly examined by the Court in hearing and assessing the expert evidence.
In sum, the collaborative nature of DOSI projects presents opportunities for international courts to benefit from highly probative materials, but demands adjustments to the traditional single-expert approach. While this shift may appear seismic to courts habituated to the single-expert model, it reflects a deeper evolution in judicial proceedings (and society more broadly) away from high-status individuals being the exclusive bearers of enlightened knowledge and towards teams of specialized personnel carrying out highly technical projects collaboratively.
C. DOSI Imports Heightened Risks of Error and Conscious or Unconscious Bias
The use of DOSI in legal proceedings results in elevated risks because audio-visual materials can be highly persuasive but may have been gathered on selective bases, whether knowingly or unknowingly. These risks are exacerbated by the team-based nature of DOSI investigations, in which vulnerabilities are potentially multiplied. Moreover, presenting DOSI as evidence involves a shift from a crucible model of justice (focusing as narrowly as possible on contested issues and bringing the witnesses before the judges to test their credibility),147 to a more flexible approach with diverse and multiple points of truth-testing (some of which may occur outside of any framework designed to ensure reliability).148 Because of this, ensuring that transparent, accessible and defensible methodologies are adopted is particularly important for DOSI investigations.149 In particular, DOSI teams should put in place review processes, whereby senior team members check the team’s work to ensure its quality and consistency. In addition to adhering to the Do No Harm principle,150 a looped process should be adopted, whereby information regarding approaches, standards, and relevant findings is reviewed and relayed with feedback to team members.151 Such rigorous methodologies are critical to reduce the risk of errors and biases remaining undetected and unchallenged.152
Surveying the delivery of expert evidence in international criminal justice provides guidance as to the elements of a rigorous methodology. Experts should receive clear instructions from the calling party (or the Court), which should be explained along with their methodology in their report.153 In Al-Hassan, the video analysis expert was provided with images to geolocate and reference material to compare it with. However, the prosecution’s instructions also allowed the expert to rely on ‘any other means’ in case of ‘trouble’ conducting the geolocation.154 The instructions lacked guidance as to whether these ‘other means’ included searching for images or materials and, if so, whether those searches should be preserved and documented. That type of omission is potentially prejudicial given that materials can disappear from the internet,155 and given that the defence may have an interest in attempting to replicate the searches to check for any disputable issues that arise.
Regarding transparency and accessibility, a party should ensure that ‘sources used in support of any expert witness statement must be clearly indicated and easily accessible to the other party upon request’, so that other party can challenge the probative value of such evidence.156 The failure to do so may impact the weight the judges are willing to place on the report,157 especially if the expert relies on hearsay to reach their views.158 In Bemba et al., screenshots from Facebook used by the Prosecutor to link two individuals were challenged by the defence ‘on the basis that the ownership of the Facebook account could not be forensically verified and that there was no metadata attached to the screenshots’.159
Best practices for DOSI investigations already require the maintenance of a trail detailing the collection of information and the chain of custody.160 This helps to ensure transparency, accessibility and potentially replicability.161 Technical means, such as website capture tools,162 can help automate the process. However, the trail may only be partial and replication may not be always possible, as there are thousands of automated functions that occur when conducting any online search. Owing to search algorithms, two people searching the same terms may well get differing results. Consequently, whereas experts should seek to adhere to objectively verifiable approaches,163 the traditional scientific method of replicability fits awkwardly in the context of DOSI.
This incongruity re-emphasises the need for clear guidelines to be adopted, or at least common understandings among the judiciary, the parties, and the broader DOSI community to be reached regarding the admissibility and weight of expert evidence regarding DOSI.164 The difficulty in replicating searches may eventually be overcome as technology advances. But, in the interim, it will require the judiciary to ensure that DOSI experts are vigorously examined regarding their methods, while at the same time appreciating the algorithmic penumbra of uncertainty and individuality accompanying DOSI analyses.
Identifying the origin and creator of DOSI materials, where possible, will be important for assessing their weight and admissibility.165 Evidence based on anonymous sources has become a pariah category at the ICC,166 with Chambers holding that open source news articles can be admitted only if the background and qualifications of the journalists or of their sources are sufficiently established regarding their objectivity and professionalism.167 Underlying the rebuke of anonymous hearsay are the concerns that the opposing party’s ability to challenge the evidence will be undermined,168 and that the information may originate from a misleading source.169
Wherever possible, DOSI teams should triangulate evidentiary materials, in order to demonstrate the probative value of their expert evidence.170 This should include checks against reference materials taken from as close as possible to the time of the item being tested, such as the appearance of a person (or sound of their voice)171 or building,172 or other item, as these can change over time. Any alterations to the materials being examined, for instance to enhance their visibility or audio properties (denoising), should be explicitly acknowledged.173
At the same time, teams should avoid attempting to step into the judges’ shoes to determine the ultimate issues.174 Terminologically, judicial phrasing like ‘beyond reasonable doubt’ should be avoided. As an alternative, the voice comparison expert in Al-Hassan used terms like ‘strongly supports’ and ‘slightly supports’ to explain his findings.175
DOSI teams should also be aware of the wider context of potential human rights protections. The potential to infringe on privacy is a critical matter.176 Experts should ensure that depicted persons have consented to the dissemination of their image or that there is a legitimate forensic purpose to show them. Measures such as image and voice distortion can mitigate incursions into privacy. If DOSI experts present additional materials they themselves have obtained in order to triangulate the materials that they are verifying, they should be again careful to avoid unjustifiably violating privacy rights and the dignity of other witnesses, victims and third parties.
At the normative level, legal regulations or a similar binding instrument on expert evidence and DOSI, with status under Article 21 of the Rome Statute (such as amendments to the Rules of Procedure or Evidence, Regulations of the Court, or otherwise qualifying as a source under Article 21), would provide the highest level of consistency of practice regarding this type of evidence.177 The formalization of practice into the prescriptive law would signal the maturation of DOSI into a recognized area of forensically relevant specialization. It would also reflect the growing importance of DOSI for the resolution of atrocity crimes cases. Pending the adoption of such regulatory provisions, the DOSI community will remain unanchored when predicting who may qualify as an expert in this domain, who should be called as a rebuttal expert, and how the probative value of DOSI materials will be assessed.178 In this context, the criteria in the next section are set out as a guide to assist the identification of DOSI experts.
5. Criteria for Classification as a DOSI Expert Witness
The growing prevalence and forensic significance of DOSI demands standards that are clear, accessible, objectively justifiable, and non-biased for its presentation to international courts such as the ICC.179 It is important that the Court maintain a rigorous approach to admission as an expert, particularly given DOSI’s easy accessibility and potential for misinterpretation and manipulation. With these objectives in mind, a list of factors designed to assist in determining whether a person has the specialized skill or knowledge in the DOSI field to give expert evidence is set out below. The list is based on relevant jurisprudence and national laws, including in the US180 and UK,181 where considerable litigation has addressed digital evidence.
While the factors are listed in the order they would typically be considered when determining expertise, it is not intended to be an inflexible order. Moreover, there is not necessarily a hierarchy of significance among the factors. However, some factors are particularly significant, such as the first one, as discussed below. It is also not intended that all the factors are cumulative. Instead, they are a set of considerations to be taken into account in their totality in order to guide the chamber’s evaluation (and parties, and DOSI community more broadly) regarding who may qualify as a DOSI expert.
A. Demonstrable Experience and Competence
The first, and most important criterion for DOSI witnesses is demonstrable experience and competence in applying the relevant expertise. This would typically be shown through a portfolio (or record) of past or ongoing projects, along with the outputs or outcomes of those projects, ideally submitted to reputable institutions. Experience and competence can augment (and potentially substitute for) formal qualifications, as long as the high quality and coherence of the work is evident.182 Extensive experience in technical testing can assist; a factor in recognizing the audio voice comparison expert in Al-Hassan as an expert was his 10 years working on this field.183
B. Prior Recognition as an Expert Witness
Second is any prior recognition as an expert witness (whether before an international or national judicial body) on a substantially similar area of expertise. Such recognition will serve as an indicator of the proposed expert’s specialized skill and/or knowledge. This is consistent with the current approach on the application form to be listed as an expert.184 However, a proposing party should disclose any information indicating that the putative expert’s status was undermined in prior proceedings, for example on cross-examination.185
C. Academic and Professional Qualifications
Third is academic and professional qualifications (or formal courses undertaken).186 This will signal a record of meeting established benchmarks and also that the expert has benefitted from instruction by experienced persons used to dealing with technical and systemic questions. Here, the ICC expert application form prominently refers to formal education and requests a copy of the applicant’s certification with the regulatory or professional body with which they are registered. However, the lack of overarching DOSI professional or regulatory bodies (though there are some bodies in sub-areas related to DOSI) potentially conflicts with the tenor of this requirement,187 prompting calls to generate a form of DOSI certification or qualification with State-level recognition, as discussed herein.
D. The Demonstrable Knowledge or Skill Exceeds that Normally Available to the Fact-finder
Fourth, possessing knowledge or skill that is demonstrably beyond that normally available to the fact-finder will be an important factor in determining expert status.188 Given the highly accessible nature of DOSI, it will be incumbent on the proposing party to show why the specialist’s evidence exceeds that available to a well-informed lay person. There is a vast range between the technical capacity of participants in DOSI investigations, from inexperienced newcomers to highly seasoned and technically proficient professionals, as can be seen by the work of DOSI experts in the MH17 litigation.189 Maintaining a strict insistence on this criterion is important because (i) the judges themselves should possess or obtain readily accessible information or knowledge; (ii) allowing expert evidence from persons without knowledge or skill exceeding the norm would beg the question of why all other witnesses are prevented from providing opinion evidence; and (iii) the risk of misinterpretations and errors is generally higher if a person has not developed a specialized level of knowledge or skill, as they may not be aware of important contextual factors for interpretation, methodological pitfalls and past instances of erroneous views being reached in the field of study.
E. Commitment to Ethical Duties
Fifth, it is important to ensure that experts understand and observe their ethical duties, including integrity, objectivity and neutrality.190 In relation to the gun-for-hire approach that has developed in some spheres regarding expert testimony, courts have often lamented that ‘… the unsatisfactory, as well as dangerous, character of this kind of evidence is well known; [e]xperts are nowadays often the mere paid advocates or partisans who employ and pay them, as much as the attorneys who conduct suit’.191 Ideally, the proposed expert should have completed ethics training — ensuring that they are aware of the impact of their work on victim communities, and their awareness of essential precepts such as the Do No Harm principle, the principle of informed consent and appropriate frameworks for questioning witnesses.192
F. Integration of Peer Review into Working Methodology
Finally, as a sixth factor, a level of peer review should feature in the proposed expert’s methodology.193 Although this is less a personal characteristic than a way of conducting the task, it is an important indicium of commitment to generating accurate and defensible results. This would not only reduce the risks of errors in assessment creeping in, but would also foster a more scientific approach, and generation of standards, within the DOSI community. That could in turn contribute to the establishment of a broader type of formal certification or qualification in this area.
G. Overarching Considerations
The fourth, fifth and sixth proposed factors, in particular, are directed to the risks of Lord Bingham’s ‘quacks, charlatans, or enthusiastic amateurs’. If a party is permitted to call an expert witness who lacked sufficient rigour in their methodology, or whose methodology cannot be tested, the burden will be placed on the opposing party to unfairly counteract evidence that should not be laid against it. For criminal defendants, serious human rights issues could arise from such curtailment of their ability to challenge the evidence against them, implicating Article 67 of the ICC Statute. For the experts themselves, they may face suits for negligence based on their testimony194 and so have a professional and fiscal motivation to avoid exceeding the ambit of their expertise and opining on matters on which they are not truly expert.
In expanding the range of persons considered to be experts and in formulating criteria which should be taken into account, the Court may face accusations of arbitrariness and potential biases. The ICC must conduct its procedures without any discrimination on prohibited grounds.195 Insisting on a high level of technical skill, knowledge or specialization does not violate any of those grounds. However, restricting those who can qualify as experts to persons from specific institutions, such as elite colleges, or with access exclusive to certain qualifications, will risk running afoul of these important protections and undermine the increased accessibility of DOSI materials to populations, including victim groups around the world.196
Courts have long been wary of experts being tainted by bias due to the instructions or fees that they receive from a party to proceedings.197 The potential for improper personal connection is reflected in the ICC’s application forms to be listed as an expert (or expert organization), which ask about connections with any person participating in the Court’s proceedings or with a staff member of the Court.198 The Regulations of the ICC’s Office of the Prosecutor confirm that it must implement bias controls but provide no detail as to how this should be achieved, and provide no guidance as to the instruction of experts to this end.199 That is a significant lacuna, as experts in areas relevant to DOSI will be questioned about biases, including cognitive biases.200
It is important that the Court considers how to address potential biases in relation to DOSI,201 including conscious political or social biases, to unconscious prejudices, to algorithmic biases that are extremely hard to detect.202 One means of controlling biases is to encourage the parties to jointly instruct expert witnesses, as the Court attempted in Lubanga, so that ‘the single expert will not be in any sense influenced, however unconsciously, by the viewpoint of only one party’ and therefore ‘he or she will be particularly able to present a balanced view of the issues’.203 However, that control mechanism only addresses potential external influences from parties to litigation. It does not directly counter the possibility of internal biases, particularly of an unconscious form, from affecting the reliability of the evidence.204
To insulate against those challenges, the ICC should ensure that its approach to extending the definition of an expert on DOSI (or applying the existing definition in a more extensive way) is both (i) formulated in a transparent and objective way; and (ii) based on ongoing interaction with the DOSI sub-community (so as to keep pace with the rapidly developing technology and also the advancing misuse of this technology to manipulate potential evidence).205 Through meeting with the DOSI community to exchange information regarding qualifications and industry-standards, the ICC can refine and integrate criteria, such as those proposed above, into its jurisprudence, and thereby provide heightened consistency and objectivity into its approach to determining DOSI expert status, while mitigating the risk of biases affecting its approach. This will be particularly important in relation to platforms and media with new features and functionality, such as the incorporation of AI, in order to make sure that the Court has the substantive understanding and lexicon to conduct the necessary assessments and to articulate their reasons regarding the admission and reliance on DOSI materials.
6. Conclusion
When the ICC commenced operations in 2002, the digital age was in its infancy. In the two decades since, the growing ubiquity of digital recording devices and the ease of transferring files has led to an exponential increase in the availability of videos, images and other DOSI for forensic proceedings.206 Whereas the ad hoc tribunals for the former Yugoslavia and Rwanda were essentially pre-DOSI international courts, the first twenty years of the ICC’s operations can be seen as the digital dawn, with DOSI being used sporadically and largely collaterally to traditional forms of evidence. Reflecting the collateral and incidental role of DOSI in that era, the judicial approach to DOSI at the ICC has been ad hoc, variable and largely reliant on simply transposing the broadly framed Rules of Procedure and Evidence to this new type of evidence. Two decades on, DOSI is being submitted to the Court in greater quantity and with greater forensic relevance than ever. Current conflicts and crises, such as those in Libya, Ukraine and Palestine demonstrate that DOSI will inevitably play a core role in judicial determinations of the outcome of trials.
As DOSI becomes more prevalent in international criminal proceedings, so too will the risks of its being misinterpreted. D’Alessandra and Sutherland observe that ‘in a “post-truth” world, the camera often lies’.207 To counteract this risk, opposing parties will contest the presentation of DOSI in court, as well as its reliability, and use by the judges in reaching their verdicts. Technical challenges typically call for expert assessment and evidence. However, the ICC framework on DOSI expert evidence is embryonic. Key challenges arise from the nature of DOSI, including its straddling of expert and lay person domains, the group-based nature of the teams that assess it, and the potentially misleading aspects of DOSI and biases that may impact its interpretation.208
Judges should maintain a firm distinction regarding the category of expert witnesses and ensure a high standard is adhered to for this privileged class of witness in international proceedings.209 In particular, judges should develop criteria that are clear, objectively justifiable, accessible, non-biased, but also sufficiently flexible to adapt to emerging forms of relevant expertise. The six criteria proposed above provide a novel combination to recognize and regulate the delivery of DOSI expert evidence. Nonetheless, the judiciary should not passively defer to DOSI experts, but should undertake training on DOSI.210 In this way, judges’ technological literacy could be augmented, in line with calls from the Independent Expert Review.211 At the same time, as the judges’ competence in digital materials grows, the bar for the qualification as an expert may also rise commensurately, as a function of the requirement that the expert’s specialized skill lies outside the knowledge of the bench.212
For the DOSI community, the generation of a set of principles would assist in having their expertise recognized for formal qualification as experts. But for specific DOSI specialists, ensuring that their teams adopt clear and defensible methodology, a transparent approach, peer-review, as well as an internal review process will assist their findings to be accepted by the judges in international criminal litigation.
DOSI challenges the binary between experts and regular fact witnesses. In response to that challenge, the Court should ensure rigorous standards of fact-finding and technical assessment on the part of experts, while also shifting to a more experience and skills-based approach to determining expertise, in lieu of traditional models based largely on reputation and formal qualification from elite institutions. On the part of the DOSI community, efforts should be made to adopt overarching qualifications, with appropriate attention given to technical skills, robust and transparent methodology, and ethical considerations. This challenge is a fruitful one, which portends deeper benefits for the Court, the parties to litigation and the broader epistemic community interested in the establishment of truth and responsibility for the gravest crimes plaguing the world.
Acknowledgements
The authors thank Nema Milaninia for insightful inputs. The views herein are those of the authors alone and do not necessarily reflect those of any other person or organization. All websites cited were last visited on 27 October 2023.
© The Author(s) (2023). Published by Oxford University Press.
This post was originally published on this site be sure to check out more of their content.
