FBI Laboratory Publishes Results of Their Major Handwriting Analysis Study

As a licensed private investigator in Texas many moons ago, I studied Graphology and examined documents for various crime cases and court proceedings. Today, I retain an interest in this forensic activity and it’s a fun hobby and exercise at parties and get togethers with friends.

Recently I learned of a five year study performed by researchers evaluating many examiners, most of them government employees, where they undertook 100 handwriting comparisons using digital images of such writing produced by 230 people.

Of the 100 tasks:

🔹44 were comparison of documents handwritten by the same person

🔹56 were comparison of documents written by two individuals.

🔹Unknown to the participants, a tenth of the comparison sets were repeats of sets they had already seen—a way to test how consistent each participant was over time.

The FBI’s Laboratory Division, in conjunction with Noblis, Inc., recently published their scientific research paper in the Proceedings of the National Academy of Sciences about the accuracy and reliability of forensic handwriting comparison.

The paper, “Accuracy and Reliability of Forensic Handwriting Comparisons,” summarizes the results of the five-year project.

The FBI Laboratory undertook this research to provide estimates of error rates—how often document examiners make correct writership decisions—as well as how often an examiner reaches the same conclusion when seeing the same documents again, and how often other examiners reach the same conclusions.

This study was the largest of its kind, involving examiners from U.S. and international crime laboratories and private practice. Collectively, these examiners made more than 7,000 document comparisons and provided information with which to correlate results to levels of education and experience, along with other metadata.

Examiners in the FBI study expressed their conclusions in the form of five ratings: definitive that the same writer had or had not written the compared samples, probable that the same writer had or had not written them, or no conclusion.

Features of interest included letter spacing, how letters connect, and the drop or rise of “legs” below or above a letter, such as the tail of a small letter “g” or the upsweep of a small letter “d.”

Overall, in 3.1 percent of cases, examiners incorrectly concluded that the same writer had composed the comparison samples. Different writers who were twins tripped the examiners up more often, leading to a false-positive rate of 8.7 percent. The false-negative rate of samples that were incorrectly attributed to two different writers was even lower, at 1.1 percent.

The study is part of a portfolio of research projects conducted by the FBI Laboratory to evaluate the accuracy, repeatability, and reproducibility of pattern evidence examiner decisions.

It was modeled after a highly acclaimed 2011 FBI Laboratory study about the accuracy and reliability of fingerprint examiner decisions, which is widely regarded within the forensic community as a gold standard in pattern evidence study design. That research project formed the basic design for this study and resulted in more than 15 scientific publications to date.