An Empirical Evaluation of Structured Argumentation Using the Toulmin Argument Formalism

Structured argumentation tools are software-based cognitive aids intended to help information analysts more rigorously develop and communicate the reasoning behind their conclusions. Some of these tools employ Toulmin's argument formalism, but there has been no controlled research demonstrating the formalism's effectiveness in supporting argument evaluation or communication. An experiment was conducted to address this need by assessing whether the use of the Toulmin formalism positively impacted participants' ability to evaluate and communicate the arguments presented in two articles, each approximately 2,000 words in length. The results were mixed, with the formalism having a positive impact for only one of the two articles. In general, participants found it difficult to generate Toulmin structures, and their structures varied greatly even though they started with the same content. Consequently, one should be cautious of the claimed value of structured argumentation tools employing the Toulmin formalism without future empirical research demonstrating its value.