{"id":873,"date":"2023-10-18T22:06:16","date_gmt":"2023-10-18T20:06:16","guid":{"rendered":"https:\/\/www.olivertacke.de\/labs\/?p=873"},"modified":"2023-10-19T16:54:46","modified_gmt":"2023-10-19T14:54:46","slug":"using-scientific-research-to-improve-h5p-creating-discrete-option-multiple-choice","status":"publish","type":"post","link":"https:\/\/www.olivertacke.de\/labs\/2023\/10\/18\/using-scientific-research-to-improve-h5p-creating-discrete-option-multiple-choice\/","title":{"rendered":"Using scientific research to improve H5P: Creating &#8220;Discrete Option Multiple Choice&#8221;"},"content":{"rendered":"<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-157\" class=\"h5p-iframe\" data-content-id=\"157\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Using scientific research to improve H5P: Creating \u201cDiscrete Option Multiple Choice\u201d\"><\/iframe><\/div>\n<p><em>Please note: This is not a blog post intended to be self explanatory, but it merely is a place where I dumped some material that is supposed to accompany a talk. So if you think that something is missing here, then you are probably correct <\/em>\ud83d\ude42<\/p>\n<hr \/>\n<h2>You will know this: A traditional multiple choice question created with H5P (TDMC)<\/h2>\n<p><iframe loading=\"lazy\" title=\"Multiple Choice\" src=\"https:\/\/h5p.org\/h5p\/embed\/712\" width=\"1090\" height=\"580\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><script src=\"https:\/\/h5p.org\/sites\/all\/modules\/h5p\/library\/js\/h5p-resizer.js\" charset=\"UTF-8\"><\/script><\/p>\n<hr \/>\n<h2>The new kid in town: Discrete Option Multiple Choice (DOMC)<\/h2>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-156\" class=\"h5p-iframe\" data-content-id=\"156\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Multiple Choice as DOMC\"><\/iframe><\/div>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-149\" class=\"h5p-iframe\" data-content-id=\"149\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Discrete Option Multiple Choice Demo\"><\/iframe><\/div>\n<p>See details at <a href=\"https:\/\/www.olivertacke.de\/labs\/2023\/04\/28\/a-new-h5p-content-type-discrete-option-multiple-choice\/\" target=\"_blank\" rel=\"noopener\">https:\/\/www.olivertacke.de\/labs\/2023\/04\/28\/a-new-h5p-content-type-discrete-option-multiple-choice\/<\/a><\/p>\n<hr \/>\n<h2>Test Wiseness: Comparison of TDMC and DOMC<\/h2>\n<p>Have a look at the following questions. You will probably find that the traditional multiple choice questions can be solved quite well with a little bit of &#8220;test wiseness&#8221; even if you don&#8217;t know anything about the subject. What do you think about this issue when looking at the variants using the discrete option multiple choice format?<\/p>\n<h3>Traditional Multiple Choice<\/h3>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-153\" class=\"h5p-iframe\" data-content-id=\"153\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Test Wiseness Test 3 (TDMC)\"><\/iframe><\/div>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-155\" class=\"h5p-iframe\" data-content-id=\"155\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Test Wiseness Test 2 (TDMC)\"><\/iframe><\/div>\n<h3>Discrete Option Multiple Choice<\/h3>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-152\" class=\"h5p-iframe\" data-content-id=\"152\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Test Wiseness Test 3 (DOMC)\"><\/iframe><\/div>\n<div class=\"h5p-iframe-wrapper\"><iframe id=\"h5p-iframe-154\" class=\"h5p-iframe\" data-content-id=\"154\" style=\"height:1px\" src=\"about:blank\" frameBorder=\"0\" scrolling=\"no\" title=\"Test Wiseness Test 2 (DOMC)\"><\/iframe><\/div>\n<hr \/>\n<h2>Some research results<\/h2>\n<h3>Foster &amp; Miller (2009)<\/h3>\n<p><a href=\"https:\/\/www.psychologie-aktuell.com\/fileadmin\/download\/PschologyScience\/4-2009\/psq_4_2009_355-369.pdf\" target=\"_blank\" rel=\"noopener\">Foster, D., &amp; Miller Jr, H. L. (2009). A new format for multiple-choice testing: Discrete-Option Multiple-Choice. Results from early studies. Psychological Test and Assessment Modeling, 51(4), 355.<\/a><\/p>\n<blockquote><p>The effects were seen in the increased difficulty and discriminative ability of items, reduction of the time needed to complete the assessment, and the improved security of the assessment.<br \/>\nOne other effect is worth noting and may deserve a research program all its own. Because it reduces the effects of test-taking skills as well as attempts at testing fraud, the DOMC format improves the fairness of the assessment.<\/p><\/blockquote>\n<h3>Kingston, Tiemann, Miller &amp; Foster (2012)<\/h3>\n<p><a href=\"https:\/\/www.researchgate.net\/profile\/Kingston-Neal\/publication\/265750861_An_analysis_of_the_discrete-option_multiple-_choice_item_type\/links\/54d36a1a0cf2b0c6146d802c\/An-analysis-of-the-discrete-option-multiple-choice-item-type.pdf\" target=\"_blank\" rel=\"noopener\">Kingston, N. M., Tiemann, G. C., Miller Jr, H. L., &amp; Foster, D. (2012). An analysis of the discrete-option multiple-choice item type. Psychological Test and Assessment Modeling, 54(1), 3.<\/a><\/p>\n<blockquote><p>Results showed that, across all forms, MC items were consistently easier than DOMC items. Mean scores from DOMC item sets were also lower than for MC sets, which could possibly be attributed to a reduced impact of testwiseness in responding to DOMC items or alternatively to the fact that examinees cannot revisit options once they make a choice to select or not select an option.<\/p>\n<p>Based on the results of this study, there appear to be no psychometric reasons for excluding DOMC items from testing programs.<\/p>\n<h3>Willing (2013)<\/h3>\n<\/blockquote>\n<p><a href=\"https:\/\/docserv.uni-duesseldorf.de\/servlets\/DerivateServlet\/Derivate-29719\/Dissertation%20Sonja%20Willing.pdf\" target=\"_blank\" rel=\"noopener\">Willing, S. (2013). Discrete-option multiple-choice: Evaluating the psychometric properties of a new method of knowledge assessment (Doctoral dissertation).<\/a><\/p>\n<blockquote><p>Taken together, the psychometric properties of DOMC testing did not surpass but were able to match those of the format hitherto considered to be the most valid for an objective assessment of knowledge. In view of some of its unique new features, the sequential answering format therefore seems to offer a promising alternative to the traditional MC format.<\/p>\n<p>Bolt et al. (2020)<\/p><\/blockquote>\n<p><a href=\"https:\/\/journals.sagepub.com\/doi\/full\/10.1177\/0146621619835499\" target=\"_blank\" rel=\"noopener\">Bolt, D. M., Kim, N., Wollack, J., Pan, Y., Eckerly, C., &amp; Sowles, J. (2020). A psychometric model for discrete-option multiple-choice items. Applied Psychological Measurement, 44(1), 33-48.<\/a><\/p>\n<blockquote><p>To the extent that incorrect responses on DOMC items can occur in either of two ways (i.e., not selecting a keyed response OR selecting a distractor), the correctness of a DOMC item score can be viewed as the outcome of conjunctively interacting processes, namely, successes in ruling out distractors AND successes in selecting keyed responses.<\/p><\/blockquote>\n<h3>Eckerly, Smith &amp; Sowles (2018)<\/h3>\n<p><a href=\"https:\/\/files.eric.ed.gov\/fulltext\/EJ1196127.pdf\" target=\"_blank\" rel=\"noopener\">Eckerly, C., Smith, R., &amp; Sowles, J. (2018). Fairness Concerns of Discrete Option Multiple Choice Items. Practical Assessment, Research &amp; Evaluation, 23(16).<\/a><\/p>\n<blockquote><p>We have shown that item difficulty and discrimination varied substantially for the DOMC items in this dataset, depending on key position, leading test takers to see forms of varying difficulty and reliability.<\/p>\n<p>We recommend that testing programs not use DOMC items until a methodology is developed to address the fairness and measurement model fit issues addressed in this paper. As shown in this study, without doing so can introduce significant fairness issues with respect to varying item difficulty and discrimination.<\/p><\/blockquote>\n<hr \/>\n<h2>H5P would not be H5P without some more options \ud83d\ude42<\/h2>\n<p>Let&#8217;s have a look what you can change!<\/p>\n<p>For instance, ask the student to state how confident she\/he is in her\/his answer in order to improve self-reflection, self-assessment, and the ability to recognize one\u2019s own strengths and weaknesses (metacognitive monitoring) &#8211; again backed up by scientific research (Barenberg, J., &amp; Dutke, S. (2022). <a href=\"https:\/\/link.springer.com\/article\/10.1007\/s42010-021-00138-3\" target=\"_blank\" rel=\"noopener\">Testen als evidenzbasierte Lernmethode: Empirische und theoretische Gr\u00fcnde f\u00fcr eine Anwendung im Unterricht<\/a>. Unterrichtswissenschaft, 50(1), 17-36.)<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Please note: This is not a blog post intended to be self explanatory, but it merely is a place where I dumped some material that is supposed to accompany a talk. So if you think that something is missing here, then you are probably correct \ud83d\ude42 You will know this: A traditional multiple choice question &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/www.olivertacke.de\/labs\/2023\/10\/18\/using-scientific-research-to-improve-h5p-creating-discrete-option-multiple-choice\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Using scientific research to improve H5P: Creating &#8220;Discrete Option Multiple Choice&#8221;&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-873","post","type-post","status-publish","format-standard","hentry","category-h5p"],"_links":{"self":[{"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/posts\/873","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/comments?post=873"}],"version-history":[{"count":0,"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/posts\/873\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/media?parent=873"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/categories?post=873"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.olivertacke.de\/labs\/wp-json\/wp\/v2\/tags?post=873"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}