{"id":1435,"date":"2022-01-01T02:06:00","date_gmt":"2022-01-01T02:06:00","guid":{"rendered":"https:\/\/sappan-project.eu\/?p=1435"},"modified":"2021-12-31T02:26:13","modified_gmt":"2021-12-31T02:26:13","slug":"challenges-in-visualization-for-ai","status":"publish","type":"post","link":"https:\/\/sappan-project.eu\/?p=1435","title":{"rendered":"Challenges in Visualization for AI"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"1435\" class=\"elementor elementor-1435\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-6037d5a elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6037d5a\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-5ad5a1a\" data-id=\"5ad5a1a\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-e395350 elementor-widget elementor-widget-heading\" data-id=\"e395350\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<div class=\"elementor-heading-title elementor-size-default\">By Franziska Becker (University of Stuttgart, Institute for Visualization and Interactive Systems)<\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-063c4a1 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"063c4a1\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-f6aaf81\" data-id=\"f6aaf81\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-d531852 elementor-widget elementor-widget-heading\" data-id=\"d531852\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h5 class=\"elementor-heading-title elementor-size-default\">Artificial intelligence (AI) is one of the buzzwords that defined many conversations in the last 5-10 years. Especially in regards to technology, \u201cCan we use AI to improve our product?\u201d is not an uncommon question. With these conversations come issues concerning interpretability and explainability of AI models. Visualization can offer one way of approaching these topics, but also introduces new challenges, like effects of and on cognitive biases.<\/h5>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-ceeb3cd elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"ceeb3cd\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-7c093ef\" data-id=\"7c093ef\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-4652db4 elementor-widget elementor-widget-text-editor\" data-id=\"4652db4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p style=\"text-align:justify;line-height:normal\"><span style=\"font-size:12.0pt;mso-bidi-font-size:11.0pt;font-family:&quot;Arial&quot;,sans-serif;\nmso-ansi-language:EN-IE\" lang=\"EN-IE\">AI harnesses the power of machine learning to perform\ntasks more efficiently, more accurately or on a bigger scale than people are\ncapable of doing. In chess, AI outperforms masters in terms of speed and skill.\nEven a supposedly simple task such as online search includes AI, since it can\ndeal with the massive amounts of data that exist on the web. AI models can\nexhibit different degrees of interpretability, depending on the architecture\nand data employed. However, in general, more interpretability comes with lower\naccuracy: the interpretability-accuracy trade-off.<\/span><\/p>\n\n\n\n\n\n<p><style>@font-face\n\t{font-family:SimSun;\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-alt:\u5b8b\u4f53;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:roman;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:swiss;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-469750017 -1073732485 9 0 511 0;}@font-face\n\t{font-family:\"\\@SimSun\";\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal\n\t{mso-style-unhide:no;\n\tmso-style-qformat:yes;\n\tmso-style-parent:\"\";\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoChpDefault\n\t{mso-style-type:export-only;\n\tmso-default-props:yes;\n\tfont-size:11.0pt;\n\tmso-ansi-font-size:11.0pt;\n\tmso-bidi-font-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ascii-font-family:Calibri;\n\tmso-ascii-theme-font:minor-latin;\n\tmso-fareast-font-family:Calibri;\n\tmso-fareast-theme-font:minor-latin;\n\tmso-hansi-font-family:Calibri;\n\tmso-hansi-theme-font:minor-latin;\n\tmso-bidi-font-family:Arial;\n\tmso-bidi-theme-font:minor-bidi;\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:8.0pt;\n\tline-height:107%;}div.WordSection1\n\t{page:WordSection1;}<\/style><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-440d291 elementor-widget elementor-widget-image\" data-id=\"440d291\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"768\" height=\"575\" src=\"https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.01.26-768x575.png\" class=\"attachment-medium_large size-medium_large wp-image-1436\" alt=\"\" srcset=\"https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.01.26-768x575.png 768w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.01.26-400x300.png 400w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.01.26-698x523.png 698w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.01.26-1536x1150.png 1536w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.01.26.png 1704w\" sizes=\"(max-width: 768px) 100vw, 768px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 1: The interpretability-accuracy trade-off showing that models\u2019 accuracy decreases as their interpretability increases, figure taken from Duttaroy [1].<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2a5859c elementor-widget elementor-widget-text-editor\" data-id=\"2a5859c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/p><p class=\"MsoNormal\" style=\"text-align:justify;line-height:normal\"><span style=\"font-size:12.0pt;mso-bidi-font-size:11.0pt;font-family:&quot;Arial&quot;,sans-serif;\nmso-ansi-language:EN-IE\" lang=\"EN-IE\">This means that with an increasing desire to integrate\nhigh-performance AI in existing systems, interpretability of these models also\ngains in importance. Visualizations for AI interpretability aim to meet a\nmultitude of goals. They may provide support for model debugging, help users\ncompare and choose between different models or give some kind of explanation\nfor a specific model output. Visualizations can give a detailed and interactive\nperformance analysis, show patterns in model behaviour (see <\/span><span style=\"font-size:12.0pt;font-family:&quot;Arial&quot;,sans-serif\" lang=\"EN-GB\">Figure <span style=\"mso-no-proof:yes\">2<\/span><\/span><span style=\"font-size:12.0pt;\nfont-family:&quot;Arial&quot;,sans-serif;mso-ansi-language:EN-IE\" lang=\"EN-IE\"><\/span><span style=\"font-size:12.0pt;font-family:&quot;Arial&quot;,sans-serif;mso-ansi-language:EN-IE\" lang=\"EN-IE\">)<\/span><span style=\"font-size:12.0pt;mso-bidi-font-size:11.0pt;font-family:&quot;Arial&quot;,sans-serif;\nmso-ansi-language:EN-IE\" lang=\"EN-IE\"> or display outputs from XAI methods like feature\nvisualization or saliency maps.<\/span><\/p>\n\n\n\n\n\n<p><style>@font-face\n\t{font-family:SimSun;\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-alt:\u5b8b\u4f53;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:roman;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:swiss;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-469750017 -1073732485 9 0 511 0;}@font-face\n\t{font-family:\"\\@SimSun\";\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal\n\t{mso-style-unhide:no;\n\tmso-style-qformat:yes;\n\tmso-style-parent:\"\";\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoChpDefault\n\t{mso-style-type:export-only;\n\tmso-default-props:yes;\n\tfont-size:11.0pt;\n\tmso-ansi-font-size:11.0pt;\n\tmso-bidi-font-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ascii-font-family:Calibri;\n\tmso-ascii-theme-font:minor-latin;\n\tmso-fareast-font-family:Calibri;\n\tmso-fareast-theme-font:minor-latin;\n\tmso-hansi-font-family:Calibri;\n\tmso-hansi-theme-font:minor-latin;\n\tmso-bidi-font-family:Arial;\n\tmso-bidi-theme-font:minor-bidi;\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:8.0pt;\n\tline-height:107%;}div.WordSection1\n\t{page:WordSection1;}<\/style><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9737fa5 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9737fa5\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-78e5c3b\" data-id=\"78e5c3b\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-71115c8 elementor-widget elementor-widget-image\" data-id=\"71115c8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1764\" height=\"1376\" src=\"https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.04.05.png\" class=\"attachment-2048x2048 size-2048x2048 wp-image-1437\" alt=\"\" srcset=\"https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.04.05.png 1764w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.04.05-400x312.png 400w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.04.05-698x544.png 698w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.04.05-768x599.png 768w, https:\/\/sappan-project.eu\/wp-content\/uploads\/2021\/12\/Screenshot-2021-12-31-at-03.04.05-1536x1198.png 1536w\" sizes=\"(max-width: 1764px) 100vw, 1764px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Figure 2: Example visualizations from a SAPPAN prototype for a DGA classifier showing a 2D projection of activations (left) that are clustered using HDBSCAN, and a decision tree (right) that gives a local explanation for these clusters.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b21ca17 elementor-widget elementor-widget-text-editor\" data-id=\"b21ca17\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/p><p class=\"MsoNormal\" style=\"text-align:justify;line-height:normal\"><span style=\"font-size:12.0pt;mso-bidi-font-size:11.0pt;font-family:&quot;Arial&quot;,sans-serif;\nmso-ansi-language:EN-IE\" lang=\"EN-IE\">From the visualization point of view, we need not only\nconsider perceptual mechanisms and rules for good visual encoding that answer\nour questions, but also how our presentation (including order, emphasis, etc.) and\nchoice of <i style=\"mso-bidi-font-style:normal\">what <\/i>to visualize affects\nthe viewer\u2019s decision-making process. Research from cognitive psychology (e.g.\nin Caverni\u2019s book <span style=\"mso-ansi-language:EN-US;mso-no-proof:yes\" lang=\"EN-US\">[2]<\/span>) has shown that\npeople often employ an ever-growing number of cognitive biases. These biases\ncan be characterized as a deviation from the \u2018regular\u2019 or \u2018rational\u2019 judgement\nprocess, though they do not necessarily have to lead to bad judgements. One\nexample for a widely known cognitive bias is <i style=\"mso-bidi-font-style:\nnormal\">anchoring<\/i>, which describes the (undue) influence an initial anchor\nhas on a final judgement. Nourani et al. <span style=\"mso-no-proof:\n yes\">[3]<\/span>\nhave recently shown that users of a system can exhibit such behaviour when\nasked to judge model outputs. If participants started with cases where the\nmodel had obvious weaknesses, they were much more likely to distrust the model,\neven in cases where the model generally performed well. This can be seen as an\nexample reducing automation bias (trusting automated systems too much) but\nincreasing anchoring bias. Participants significantly underestimated model\naccuracy when starting with the model weaknesses, but had generally higher task\naccuracy, so they made less mistakes by relying on the model too much.<\/span><\/p>\n\n\n\n\n\n<p><style>@font-face\n\t{font-family:SimSun;\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-alt:\u5b8b\u4f53;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:roman;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:swiss;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-469750017 -1073732485 9 0 511 0;}@font-face\n\t{font-family:\"\\@SimSun\";\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal\n\t{mso-style-unhide:no;\n\tmso-style-qformat:yes;\n\tmso-style-parent:\"\";\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoChpDefault\n\t{mso-style-type:export-only;\n\tmso-default-props:yes;\n\tfont-size:11.0pt;\n\tmso-ansi-font-size:11.0pt;\n\tmso-bidi-font-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ascii-font-family:Calibri;\n\tmso-ascii-theme-font:minor-latin;\n\tmso-fareast-font-family:Calibri;\n\tmso-fareast-theme-font:minor-latin;\n\tmso-hansi-font-family:Calibri;\n\tmso-hansi-theme-font:minor-latin;\n\tmso-bidi-font-family:Arial;\n\tmso-bidi-theme-font:minor-bidi;\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:8.0pt;\n\tline-height:107%;}div.WordSection1\n\t{page:WordSection1;}<\/style><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-090a6c9 elementor-widget elementor-widget-text-editor\" data-id=\"090a6c9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/p><p class=\"MsoNormal\" style=\"text-align:justify;line-height:normal\"><span style=\"font-size:12.0pt;mso-bidi-font-size:11.0pt;font-family:&quot;Arial&quot;,sans-serif;\nmso-ansi-language:EN-IE\" lang=\"EN-IE\">Wang et al. <span style=\"mso-no-proof:\n yes\">[4]<\/span>\nsuggest that anchoring bias in can be mitigated by showing input attributions\nfor multiple outcomes or providing counterfactual explanations. Interestingly,\nwhether participants were also given an explanation for model outputs did not\nhave a significant effect on task accuracy in Nourani\u2019s study <span style=\"mso-no-proof:yes\">[3]<\/span>. Whether this is an\nindicator that the chosen type of explanation does not fit the given task well\nor that other factors were at fault is an opportunity for further research. In\nSAPPAN, we are currently conducting a study to see how differences in expertise\naffect appropriate trust and decision accuracy when using our visualization for\nDGA (domain generation algorithm) classifiers.<\/span><\/p>\n\n\n\n\n\n<p><style>@font-face\n\t{font-family:SimSun;\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-alt:\u5b8b\u4f53;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:roman;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:swiss;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-469750017 -1073732485 9 0 511 0;}@font-face\n\t{font-family:\"\\@SimSun\";\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal\n\t{mso-style-unhide:no;\n\tmso-style-qformat:yes;\n\tmso-style-parent:\"\";\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoChpDefault\n\t{mso-style-type:export-only;\n\tmso-default-props:yes;\n\tfont-size:11.0pt;\n\tmso-ansi-font-size:11.0pt;\n\tmso-bidi-font-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ascii-font-family:Calibri;\n\tmso-ascii-theme-font:minor-latin;\n\tmso-fareast-font-family:Calibri;\n\tmso-fareast-theme-font:minor-latin;\n\tmso-hansi-font-family:Calibri;\n\tmso-hansi-theme-font:minor-latin;\n\tmso-bidi-font-family:Arial;\n\tmso-bidi-theme-font:minor-bidi;\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:8.0pt;\n\tline-height:107%;}div.WordSection1\n\t{page:WordSection1;}<\/style><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8bf398c elementor-widget elementor-widget-text-editor\" data-id=\"8bf398c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/p><p class=\"MsoNormal\" style=\"text-align:justify;line-height:normal\"><span style=\"font-size:12.0pt;mso-bidi-font-size:11.0pt;font-family:&quot;Arial&quot;,sans-serif;\nmso-ansi-language:EN-IE\" lang=\"EN-IE\">AI will undoubtedly play an integral part in our future.\nWhile interpretability is not essential in all areas, if we want to adopt AI\ntechniques more widely and for critical sectors, it is <i style=\"mso-bidi-font-style:\nnormal\">people<\/i> that need to understand its capabilities and limitations.\nConsequently, we must consider what visualizations <i style=\"mso-bidi-font-style:\nnormal\">ought<\/i> to do and how different designs can achieve their goals for\nspecific users. Which biases affect us most when we have to make decisions\nbased on machine outputs and how can systems mitigate these biases? To that\nend, it is also necessary to further improve our methods of extracting users\u2019\nmental models so that we can study the interactions between design and the\ndecision-making process.<\/span><\/p>\n\n\n\n\n\n<p><style>@font-face\n\t{font-family:SimSun;\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-alt:\u5b8b\u4f53;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:roman;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:swiss;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-469750017 -1073732485 9 0 511 0;}@font-face\n\t{font-family:\"\\@SimSun\";\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal\n\t{mso-style-unhide:no;\n\tmso-style-qformat:yes;\n\tmso-style-parent:\"\";\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoChpDefault\n\t{mso-style-type:export-only;\n\tmso-default-props:yes;\n\tfont-size:11.0pt;\n\tmso-ansi-font-size:11.0pt;\n\tmso-bidi-font-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ascii-font-family:Calibri;\n\tmso-ascii-theme-font:minor-latin;\n\tmso-fareast-font-family:Calibri;\n\tmso-fareast-theme-font:minor-latin;\n\tmso-hansi-font-family:Calibri;\n\tmso-hansi-theme-font:minor-latin;\n\tmso-bidi-font-family:Arial;\n\tmso-bidi-theme-font:minor-bidi;\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:8.0pt;\n\tline-height:107%;}div.WordSection1\n\t{page:WordSection1;}<\/style><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e190ed9 elementor-widget elementor-widget-heading\" data-id=\"e190ed9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h5 class=\"elementor-heading-title elementor-size-default\">References<\/h5>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b3f22de elementor-widget elementor-widget-text-editor\" data-id=\"b3f22de\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/p><table class=\"MsoNormalTable\" style=\"width:100.0%;mso-cellspacing:1.5pt;mso-yfti-tbllook:1184\" width=\"100%\" cellpadding=\"0\" border=\"0\">\n <tbody><tr style=\"mso-yfti-irow:0;mso-yfti-firstrow:yes\">\n  <td style=\"width:1.0%;padding:.75pt .75pt .75pt .75pt\" width=\"1%\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:DE;mso-no-proof:yes\" lang=\"DE\">[1] <\/span><\/p>\n  <\/td>\n  <td style=\"padding:.75pt .75pt .75pt .75pt\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:EN-US;mso-no-proof:yes\" lang=\"EN-US\">A. Duttaroy, \u201e3 X\u2019s of Explainable\n  AI,\u201c 2021. [Online]. Available:\n  https:\/\/www.lntinfotech.com\/wp-content\/uploads\/2021\/01\/3xExplainable-AI.pdf.\n  [Access: 14 December 2021].<\/span><\/p>\n  <\/td>\n <\/tr>\n <tr style=\"mso-yfti-irow:1\">\n  <td style=\"width:1.0%;padding:.75pt .75pt .75pt .75pt\" width=\"1%\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:DE;mso-no-proof:yes\" lang=\"DE\">[2] <\/span><\/p>\n  <\/td>\n  <td style=\"padding:.75pt .75pt .75pt .75pt\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:EN-US;mso-no-proof:yes\" lang=\"EN-US\">J.-P. Caverni, J.-M. Fabre und M.\n  Gonzalez, Cognitive biases, Elsevier, 1990. <\/span><\/p>\n  <\/td>\n <\/tr>\n <tr style=\"mso-yfti-irow:2\">\n  <td style=\"width:1.0%;padding:.75pt .75pt .75pt .75pt\" width=\"1%\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:DE;mso-no-proof:yes\" lang=\"DE\">[3] <\/span><\/p>\n  <\/td>\n  <td style=\"padding:.75pt .75pt .75pt .75pt\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:EN-US;mso-no-proof:yes\" lang=\"EN-US\">M. Nourani et al. \u201eInvestigating\n  the Importance of First Impressions and Explainable AI with Interactive Video\n  Analysis\u201c in <i>CHI EA &#8217;20: Extended Abstracts of the 2020 CHI Conference on\n  Human Factors in Computing Systems.<\/i> 2020. <\/span><\/p>\n  <\/td>\n <\/tr>\n <tr style=\"mso-yfti-irow:3;mso-yfti-lastrow:yes\">\n  <td style=\"width:1.0%;padding:.75pt .75pt .75pt .75pt\" width=\"1%\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:EN-US;mso-no-proof:yes\" lang=\"EN-US\">[4] <\/span><\/p>\n  <\/td>\n  <td style=\"padding:.75pt .75pt .75pt .75pt\" valign=\"top\">\n  <p class=\"MsoBibliography\"><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:DE;mso-no-proof:yes\" lang=\"DE\">D. Wang, Q. Yang, A. Abdul und B. Y.\n  Lim. <\/span><span style=\"font-family:&quot;Arial&quot;,sans-serif;\n  mso-ansi-language:EN-US;mso-no-proof:yes\" lang=\"EN-US\">\u201eDesigning Theory-Driven User-Centric\n  Explainable AI\u201c in <i>Proceedings of the 2019 CHI conference on human factors\n  in computing systems<\/i>. 2019. <\/span><\/p>\n  <\/td>\n <\/tr>\n<\/tbody><\/table>\n\n\n\n\n\n<p><style>@font-face\n\t{font-family:SimSun;\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-alt:\u5b8b\u4f53;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:roman;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:swiss;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-469750017 -1073732485 9 0 511 0;}@font-face\n\t{font-family:\"\\@SimSun\";\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal\n\t{mso-style-unhide:no;\n\tmso-style-qformat:yes;\n\tmso-style-parent:\"\";\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}p.MsoBibliography, li.MsoBibliography, div.MsoBibliography\n\t{mso-style-priority:37;\n\tmso-style-next:Normal;\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoChpDefault\n\t{mso-style-type:export-only;\n\tmso-default-props:yes;\n\tfont-size:11.0pt;\n\tmso-ansi-font-size:11.0pt;\n\tmso-bidi-font-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ascii-font-family:Calibri;\n\tmso-ascii-theme-font:minor-latin;\n\tmso-fareast-font-family:Calibri;\n\tmso-fareast-theme-font:minor-latin;\n\tmso-hansi-font-family:Calibri;\n\tmso-hansi-theme-font:minor-latin;\n\tmso-bidi-font-family:Arial;\n\tmso-bidi-theme-font:minor-bidi;\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:8.0pt;\n\tline-height:107%;}div.WordSection1\n\t{page:WordSection1;}<\/style><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8b7665b elementor-widget elementor-widget-text-editor\" data-id=\"8b7665b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p>\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n<\/p><p class=\"MsoNormal\" style=\"text-align:justify;line-height:normal\"><span style=\"font-family:&quot;Times New Roman&quot;,serif;mso-ansi-language:EN-IE\" lang=\"EN-IE\">About\nthe author(s): Franziska Becker studied cognitive science and computer science\nat the University of Osnabr\u00fcck and is currently a researcher at the\nVisualization Institute (VIS) at the University of Stuttgart. Her work concerns\nvisualization for AI and the human factors involved in designing such\nvisualization systems. <\/span><span style=\"mso-ansi-language:EN-IE\" lang=\"EN-IE\"><\/span><\/p>\n\n\n\n\n\n<p><style>@font-face\n\t{font-family:SimSun;\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-alt:\u5b8b\u4f53;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}@font-face\n\t{font-family:\"Cambria Math\";\n\tpanose-1:2 4 5 3 5 4 6 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:roman;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-536870145 1107305727 0 0 415 0;}@font-face\n\t{font-family:Calibri;\n\tpanose-1:2 15 5 2 2 2 4 3 2 4;\n\tmso-font-charset:0;\n\tmso-generic-font-family:swiss;\n\tmso-font-pitch:variable;\n\tmso-font-signature:-469750017 -1073732485 9 0 511 0;}@font-face\n\t{font-family:\"\\@SimSun\";\n\tpanose-1:2 1 6 0 3 1 1 1 1 1;\n\tmso-font-charset:134;\n\tmso-generic-font-family:auto;\n\tmso-font-pitch:variable;\n\tmso-font-signature:3 680460288 22 0 262145 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal\n\t{mso-style-unhide:no;\n\tmso-style-qformat:yes;\n\tmso-style-parent:\"\";\n\tmargin-top:0cm;\n\tmargin-right:0cm;\n\tmargin-bottom:10.0pt;\n\tmargin-left:0cm;\n\tline-height:115%;\n\tmso-pagination:widow-orphan;\n\tfont-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-fareast-font-family:SimSun;\n\tmso-bidi-font-family:\"Times New Roman\";\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoChpDefault\n\t{mso-style-type:export-only;\n\tmso-default-props:yes;\n\tfont-size:11.0pt;\n\tmso-ansi-font-size:11.0pt;\n\tmso-bidi-font-size:11.0pt;\n\tfont-family:\"Calibri\",sans-serif;\n\tmso-ascii-font-family:Calibri;\n\tmso-ascii-theme-font:minor-latin;\n\tmso-fareast-font-family:Calibri;\n\tmso-fareast-theme-font:minor-latin;\n\tmso-hansi-font-family:Calibri;\n\tmso-hansi-theme-font:minor-latin;\n\tmso-bidi-font-family:Arial;\n\tmso-bidi-theme-font:minor-bidi;\n\tmso-ansi-language:EN-GB;\n\tmso-fareast-language:EN-US;}.MsoPapDefault\n\t{mso-style-type:export-only;\n\tmargin-bottom:8.0pt;\n\tline-height:107%;}div.WordSection1\n\t{page:WordSection1;}<\/style><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>By Franziska Becker (University of Stuttgart, Institute for Visualization and Interactive Systems) Artificial intelligence (AI) is one of the buzzwords that defined many conversations in the last 5-10 years. Especially in regards to technology, \u201cCan we use AI to improve our product?\u201d is not an uncommon question. With these conversations come issues concerning interpretability and&hellip;<\/p>\n","protected":false},"author":7,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"footnotes":""},"categories":[17],"tags":[],"coauthors":[10],"class_list":["post-1435","post","type-post","status-publish","format-standard","hentry","category-blog-post"],"_links":{"self":[{"href":"https:\/\/sappan-project.eu\/index.php?rest_route=\/wp\/v2\/posts\/1435","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sappan-project.eu\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sappan-project.eu\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sappan-project.eu\/index.php?rest_route=\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/sappan-project.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1435"}],"version-history":[{"count":4,"href":"https:\/\/sappan-project.eu\/index.php?rest_route=\/wp\/v2\/posts\/1435\/revisions"}],"predecessor-version":[{"id":1450,"href":"https:\/\/sappan-project.eu\/index.php?rest_route=\/wp\/v2\/posts\/1435\/revisions\/1450"}],"wp:attachment":[{"href":"https:\/\/sappan-project.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1435"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sappan-project.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1435"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sappan-project.eu\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1435"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/sappan-project.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcoauthors&post=1435"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}