A critical metric in Technology Assisted Review (TAR) is recall, which is the percentage of relevant documents actually found from the collection. One of the most compelling reasons for using TAR is the promise that a review team can achieve a desired level of recall (say 75% of the relevant documents) after reviewing only a small portion of the total document population (say 5%). The savings come from not having to review the remaining 95% of the documents. The argument is that the remaining documents (the “discard pile”) include so few that are relevant (against so many irrelevant documents) that further review is not economically justified. Continue reading
Given the increasing prevalence of technology assisted review in e-discovery, it seems hard to believe that it was just 19 months ago that TAR received its first judicial endorsement. That endorsement came, of course, from U.S. Magistrate Judge Andrew J. Peck in his landmark ruling in Moore v. Publicis Groupe, 287 F.R.D. 182 (S.D.N.Y. 2012), adopted sub nom. Moore v. Publicis Groupe SA, No. 11 Civ. 1279 (ALC)(AJP), 2012 WL 1446534 (S.D.N.Y. Apr. 26, 2012), in which he stated, “This judicial opinion now recognizes that computer-assisted review is an acceptable way to search for relevant ESI in appropriate cases.”
Other courts have since followed suit, and now there is another to add to the list: the U.S. Tax Court. Continue reading
In a recent memorandum, a U.S. Department of Justice attorney questioned the effectiveness of using technology assisted review with non-English documents. While the DOJ “would be open to discussion” about using TAR in such cases, it is not ready to adopt it as a standard procedure, the memo said.
In an article published Sept. 1 in The National Law Journal, Catalyst founder and CEO John Tredennick responds to that DOJ memo. In the article, Yes, Predictive Coding Works in Non-Western Languages, Tredennick explains that TAR, when done properly, can be just as effective for non-English as it is for English documents. This is true even for the so-called “CJK languages” — Asian languages including Chinese, Japanese and Korean.
Read the full article at the NLJ website: Yes, Predictive Coding Works in Non-Western Languages.
I am sad to report that Browning Marean passed away last Friday. He will be sorely missed by his partners at DLA Piper, his clients and his many friends and colleagues. I am proud to say that I have been friends with Browning for many years and count myself in his fan club. We go back to the early days of Catalyst and before that even. The time was too short.
Browning served on the Catalyst Advisory Board for the past two years and was always quick to help whenever I asked. You couldn’t ask for a better sounding board or friend.
Many have already posted their thoughts and regrets about the loss of Browning, including our friend Craig Ball, who as usual made the case as eloquently as possible (Browning Marean 1942-2014). Thanks to Chris Dale as well for his comments, Goodbye Old Friend: Farewell to Browning Marean, and photo gallery. And to Ralph Losey: Browning Marean: The Life and Death of a Great Lawyer. And Tom O’Connor: Browning Marean: A Remembrance.
Browning and I go back to the early days, before there was an “E” in front of discovery. He told me once that he got his start on the speaking circuit after hearing one of my talks. It inspired him to see a lawyer up there talking about litigation technology, he said. Having watched Browning leave me in the dirt with his speaking prowess, I was both honored and pleased to have played a small part in getting him going.
I had the privilege of being with Browning on the dais, at conferences and in quiet evening meals from Hong Kong to London and many places in between. Had I realized time was short, there are so many things I would have wanted to say. Alas, that seldom happens and it didn’t here. He wrote me a few weeks ago to say he expected to be back on his feet in September. How I wish that were still true.
Browning: You touched a lot of people over your too few years and made the world a better place. We carry on in your honor.
Rest in peace old friend.
Last month, two of the leading experts on e-discovery, Maura R. Grossman and Gordon V. Cormack, presented a peer-reviewed study on continuous active learning to the annual conference of the Special Interest Group on Information Retrieval, a part of the Association for Computing Machinery (ACM), “Evaluation of Machine-Learning Protocols for Technology-Assisted Review in Electronic Discovery.”
In the study, they compared three TAR protocols, testing them across eight different cases. Two of the three protocols, Simple Passive Learning (SPL) and Simple Active Learning (SAL), are typically associated with early approaches to predictive coding, which we call TAR 1.0. The third, continuous active learning (CAL), is a central part of a newer approach to predictive coding, which we call TAR 2.0. Continue reading
Common belief is that technology assisted review is useful only when making productions. In fact, it is also highly effective for reviewing productions from an opposing party. This is especially true when imminent depositions create an urgent need to identify hot documents.
A recent multi-district medical device litigation dramatizes this. The opposing party’s production was a “data dump” containing garbled OCR and little metadata. As a result, keyword searching was virtually useless. But by using TAR, the attorneys were able to highlight hot documents and prepare for the depositions with time to spare. Continue reading
Maura Grossman and Gordon Cormack just released another blockbuster article, “Comments on ‘The Implications of Rule 26(g) on the Use of Technology-Assisted Review,’” 7 Federal Courts Law Review 286 (2014). The article was in part a response to an earlier article in the same journal by Karl Schieneman and Thomas Gricks, in which they asserted that Rule 26(g) imposes “unique obligations” on parties using TAR for document productions and suggested using techniques we associate with TAR 1.0 including: Continue reading
Two years ago, it was big news in the world of e-discovery when U.S. Magistrate Judge Andrew J. Peck issued the first judicial opinion expressly approving the use of predictive coding. As other judges followed suit, issuing their own opinions endorsing or approving predictive coding, the trend led law firm Gibson Dunn, in its annual e-discovery update, to declare 2012 “the year of predictive coding.”
The trend towards judicial acceptance of predictive coding and other forms of technology assisted review (TAR) has continued, to the point where it is now newsworthy when a judge declines to order TAR. Continue reading
This past weekend I received an advance copy of a new research paper prepared by Gordon Cormack and Maura Grossman, “Evaluation of Machine-Learning Protocols for Technology-Assisted Review in Electronic Discovery.” They have posted an author’s copy here.
The study attempted to answer one of the more important questions surrounding TAR methodology: Continue reading
The purpose of the article was to report on several successful uses of technology-assisted review. While that was interesting, my attention was drawn to another aspect of the report. Three of the case studies provided data shedding further light on that persistent e-discovery mystery: “How many documents in a gigabyte?” Continue reading