Files
org_backend/output/20250331202447-ise_week_7.html
2025-12-28 20:41:18 +00:00

506 lines
20 KiB
HTML
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2025-12-28 Sun 20:37 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>ise_week_7</title>
<meta name="generator" content="Org Mode" />
<style type="text/css">
#content { max-width: 60em; margin: auto; }
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #e6e6e6;
border-radius: 3px;
background-color: #f2f2f2;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: auto;
}
pre.src:before {
display: none;
position: absolute;
top: -8px;
right: 12px;
padding: 3px;
color: #555;
background-color: #f2f2f299;
}
pre.src:hover:before { display: inline; margin-top: 14px;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-authinfo::before { content: 'Authinfo'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.equation-container {
display: table;
text-align: center;
width: 100%;
}
.equation {
vertical-align: middle;
}
.equation-label {
display: table-cell;
text-align: right;
vertical-align: middle;
}
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { }
</style>
<link rel="stylesheet" href="/assets/styles/style.css" />
<link rel="stylesheet" href="/assets/styles/bigger-picture.min.css" />
<link rel="stylesheet" href="/assets/styles/media.css" />
<script src="/assets/scripts/script.js" defer></script>
<script src="/assets/scripts/search.js" defer></script>
<script src="/assets/scripts/bigger-picture.min.js" defer></script>
<script src="/assets/scripts/svg-pan-zoom.min.js" defer></script>
<script src="/assets/scripts/gallery-init.js" defer></script>
</head>
<body>
<div id="preamble" class="status">
<div class="banner-header">
<div class="banner-left">
<a href="/20241210233721-brain_moc.html"> <img src="/assets/gr.png" alt="Site Logo" class="banner-logo" /> </a>
<div id="updated">Updated: 2025-12-28 Sun 19:48</div>
</div>
<div class="banner-search">
<input id="search-box"
type="search"
placeholder="Search notes…"
aria-label="Search notes"
autocomplete="off" />
<div id="search-results"></div>
</div>
<button id="close-all">Close All</button>
</div>
</div>
<div id="stack-root">
<div class="stack-track">
<article class="stack-pane pane-root" data-url="/20250331202447-ise_week_7.html">
<div id="content" class="content">
<div class="title-section">
<div class="title-controls">
<button class="pane-fullscreen" aria-label="Fullscreen">F</button>
<button class="pane-edit" aria-label="Edit pane">E</button>
<button class="pane-close" aria-label="Close pane">×</button>
</div>
<h1 class="title">ise<sub>week</sub><sub>7</sub></h1>
<div class="title-metadata">
<span class="metadata-item">
<span class="metadata-label">planted:</span>
<span class="metadata-value">2025-03-31</span>
</span>
<span class="metadata-item">
<span class="metadata-label">last tended to:</span>
<span class="metadata-value">2025-12-28</span>
</span>
</div>
</div>
<div id="outline-container-orgf4f7db0" class="outline-2">
<h2 id="orgf4f7db0"><a href="#orgf4f7db0"><span class="done DONE">DONE</span> Statistical Test Selection and Use (pages 21-67)</a></h2>
</div>
<div id="outline-container-org1c8d9c0" class="outline-2">
<h2 id="org1c8d9c0"><a href="#org1c8d9c0">7.1 Comparing Algorithms in Intelligent Software Engineering</a></h2>
<div class="outline-text-2" id="text-org1c8d9c0">
</div>
<div id="outline-container-org6f86e3e" class="outline-3">
<h3 id="org6f86e3e"><a href="#org6f86e3e">1. Motivation</a></h3>
<div class="outline-text-3" id="text-org6f86e3e">
<ul class="org-ul">
<li>Algorithms and configurations vary in performance.</li>
<li>No universally best algorithm: performance depends on the specific problem (&ldquo;No Free Lunch&rdquo; theorem).</li>
<li>To determine which algorithm/configuration is suitable, <b><b>comparison is essential</b></b>.</li>
<li>However, comparison is challenging due to the <b><b>stochastic nature</b></b> of computational intelligence algorithms.</li>
</ul>
</div>
</div>
<div id="outline-container-org0072836" class="outline-3">
<h3 id="org0072836"><a href="#org0072836">2. Stochastic Behaviour in Algorithms</a></h3>
<div class="outline-text-3" id="text-org0072836">
<ul class="org-ul">
<li>Sources of randomness:
<ul class="org-ul">
<li>In the algorithm (e.g., random initial population, stochastic gradient descent, mutation/crossover probabilities).</li>
<li>In data sampling.</li>
</ul></li>
<li>Result: Running the same algorithm multiple times on the same problem yields <b><b>different results</b></b>.</li>
<li>Therefore, comparisons must account for this randomness.</li>
</ul>
</div>
</div>
<div id="outline-container-org2578b69" class="outline-3">
<h3 id="org2578b69"><a href="#org2578b69">3. Handling Stochastic Behaviour</a></h3>
<div class="outline-text-3" id="text-org2578b69">
<ul class="org-ul">
<li>To compare algorithms meaningfully:
<ul class="org-ul">
<li>Run each algorithm <b><b>multiple times</b></b> (e.g., 30+ runs) using <b><b>different random seeds</b></b>.</li>
<li>This helps capture typical performance and reduce reliance on single-run outliers.</li>
</ul></li>
</ul>
</div>
</div>
<div id="outline-container-orge1c8339" class="outline-3">
<h3 id="orge1c8339"><a href="#orge1c8339">4. Methods for Comparison</a></h3>
<div class="outline-text-3" id="text-orge1c8339">
</div>
<div id="outline-container-org72d5d0f" class="outline-4">
<h4 id="org72d5d0f"><a href="#org72d5d0f">a. Mean (Average)</a></h4>
<div class="outline-text-4" id="text-org72d5d0f">
<ul class="org-ul">
<li>Simple and common.</li>
<li>Problems:
<ul class="org-ul">
<li>Sensitive to <b><b>outliers</b></b>.</li>
<li>Does not represent <b><b>variability</b></b> in results.</li>
</ul></li>
</ul>
</div>
</div>
<div id="outline-container-orgdf083a9" class="outline-4">
<h4 id="orgdf083a9"><a href="#orgdf083a9">b. Mean + Standard Deviation</a></h4>
<div class="outline-text-4" id="text-orgdf083a9">
<ul class="org-ul">
<li>Adds information about variation.</li>
<li>Still affected by outliers.</li>
<li>Hard to tell whether differences are statistically significant.</li>
</ul>
</div>
</div>
<div id="outline-container-org0abd1f5" class="outline-4">
<h4 id="org0abd1f5"><a href="#org0abd1f5">c. Median</a></h4>
<div class="outline-text-4" id="text-org0abd1f5">
<ul class="org-ul">
<li>More robust to outliers.</li>
<li>Example:
<ul class="org-ul">
<li>Sorted list: 0.000001, 0.6, 0.62, 0.65, 0.7, 0.75, 0.8, 0.8, 0.81</li>
<li>Median = 0.7</li>
</ul></li>
<li>Problem: Ignores <b><b>variation</b></b> in data.</li>
</ul>
</div>
</div>
<div id="outline-container-orgbad24f4" class="outline-4">
<h4 id="orgbad24f4"><a href="#orgbad24f4">d. Median + Quartiles</a></h4>
<div class="outline-text-4" id="text-orgbad24f4">
<ul class="org-ul">
<li>1st and 3rd quartiles provide information about data spread.</li>
<li>Still doesn&rsquo;t guarantee ability to distinguish between groups.</li>
</ul>
</div>
</div>
<div id="outline-container-orgacd9171" class="outline-4">
<h4 id="orgacd9171"><a href="#orgacd9171">e. Statistical Hypothesis Testing</a></h4>
<div class="outline-text-4" id="text-orgacd9171">
<ul class="org-ul">
<li>Scientific method to determine if observed differences are <b><b>statistically significant</b></b>.</li>
<li>Necessary for robust and credible comparison of algorithms.</li>
</ul>
</div>
</div>
</div>
<div id="outline-container-orgc4650f0" class="outline-3">
<h3 id="orgc4650f0"><a href="#orgc4650f0">5. Statistical Hypothesis Testing: Process</a></h3>
<div class="outline-text-3" id="text-orgc4650f0">
<ol class="org-ol">
<li>Define what to compare (e.g., accuracy or fitness).</li>
<li>Ensure fair comparison:
<ul class="org-ul">
<li>Equal number of evaluations or explain why not.</li>
<li>Example: Adjust generations to equate computational budget across algorithms.</li>
</ul></li>
<li>Formulate hypotheses:
<ul class="org-ul">
<li><b><b>Null hypothesis (H₀)</b></b>: No difference between the two groups.</li>
<li><b><b>Alternative hypothesis (H₁)</b></b>: A statistically significant difference exists.</li>
</ul></li>
<li>Select an appropriate test based on data distribution.</li>
</ol>
</div>
</div>
<div id="outline-container-org4eb637a" class="outline-3">
<h3 id="org4eb637a"><a href="#org4eb637a">6. Choosing the Test</a></h3>
<div class="outline-text-3" id="text-org4eb637a">
</div>
<div id="outline-container-org80e038a" class="outline-4">
<h4 id="org80e038a"><a href="#org80e038a">a. Normality Assumption</a></h4>
<div class="outline-text-4" id="text-org80e038a">
<ul class="org-ul">
<li>Many statistical tests assume a <b><b>normal distribution</b></b> of values.</li>
<li>Visual inspection or tests (e.g., Shapiro-Wilk) can check this.</li>
</ul>
</div>
</div>
<div id="outline-container-org9158ad5" class="outline-4">
<h4 id="org9158ad5"><a href="#org9158ad5">b. Parametric vs Non-parametric Tests</a></h4>
<div class="outline-text-4" id="text-org9158ad5">
<ul class="org-ul">
<li>Parametric tests (e.g., t-test):
<ul class="org-ul">
<li>More powerful.</li>
<li>Require assumptions (e.g., normality, homogeneity of variance).</li>
</ul></li>
<li>Non-parametric tests (e.g., Wilcoxon, Mann-Whitney):
<ul class="org-ul">
<li>Safer for non-normal data.</li>
<li>Widely used in stochastic algorithm comparisons.</li>
</ul></li>
</ul>
</div>
</div>
<div id="outline-container-org17604cb" class="outline-4">
<h4 id="org17604cb"><a href="#org17604cb">c. Paired vs Unpaired Tests</a></h4>
<div class="outline-text-4" id="text-org17604cb">
<ul class="org-ul">
<li><b><b>Paired</b></b>: Use when comparing results from same initial conditions.</li>
<li><b><b>Unpaired</b></b>: Use when runs are completely independent.</li>
</ul>
</div>
</div>
</div>
<div id="outline-container-org97b2589" class="outline-3">
<h3 id="org97b2589"><a href="#org97b2589">7. Test Outputs</a></h3>
<div class="outline-text-3" id="text-org97b2589">
<ul class="org-ul">
<li>Test produces a <b><b>statistic</b></b> and a <b><b>p-value</b></b>.
<ul class="org-ul">
<li>If <b><b>p ≤ 0.05</b></b>, reject H₀: significant difference exists.</li>
<li>If <b><b>p &gt; 0.05</b></b>, do not reject H₀: no significant difference found.</li>
</ul></li>
<li>Significance level is usually set to <b><b>0.05</b></b>, corresponding to 95% confidence.</li>
<li>Lower significance (e.g., 0.01) may be used in critical applications.</li>
</ul>
</div>
</div>
<div id="outline-container-org9186d09" class="outline-3">
<h3 id="org9186d09"><a href="#org9186d09">8. Interpreting P-Values</a></h3>
<div class="outline-text-3" id="text-org9186d09">
<ul class="org-ul">
<li>High p-value → Observed difference likely due to chance → <b><b>Do not reject H₀</b></b>.</li>
<li>Low p-value → Observed difference unlikely due to chance → <b><b>Reject H₀</b></b>.</li>
</ul>
</div>
</div>
<div id="outline-container-org2a33cb7" class="outline-3">
<h3 id="org2a33cb7"><a href="#org2a33cb7">9. Test Examples (in R)</a></h3>
<div class="outline-text-3" id="text-org2a33cb7">
<ul class="org-ul">
<li>Two-tailed Wilcoxon Rank-Sum Test (unpaired).</li>
<li>Two-tailed Wilcoxon Signed-Rank Test (paired).</li>
</ul>
</div>
</div>
<div id="outline-container-org8176556" class="outline-3">
<h3 id="org8176556"><a href="#org8176556">10. Multiple Comparisons Problem</a></h3>
<div class="outline-text-3" id="text-org8176556">
<ul class="org-ul">
<li>Comparing many algorithms or configurations increases the risk of <b><b>Type I errors</b></b> (false positives).</li>
<li>Correction methods:
<ul class="org-ul">
<li>Adjust the significance threshold (e.g., Bonferroni correction).</li>
<li>Downside: Conservative → <b><b>reduced power</b></b> (risk of missing real differences).</li>
</ul></li>
</ul>
</div>
</div>
<div id="outline-container-orgb3365b9" class="outline-3">
<h3 id="orgb3365b9"><a href="#orgb3365b9">11. Tests for N Groups</a></h3>
<div class="outline-text-3" id="text-orgb3365b9">
<ul class="org-ul">
<li>Stronger than multiple pairwise tests with correction.</li>
<li>Common tests:
<ul class="org-ul">
<li><b><b>Kruskal-Wallis Test</b></b>: for unpaired comparisons across groups.</li>
<li><b><b>Friedman Test</b></b>: for paired comparisons across groups.</li>
</ul></li>
</ul>
</div>
<div id="outline-container-org09587aa" class="outline-4">
<h4 id="org09587aa"><a href="#org09587aa">Post-hoc Analysis</a></h4>
<div class="outline-text-4" id="text-org09587aa">
<ul class="org-ul">
<li>Needed when the global test finds significant differences but doesn&rsquo;t specify <b><b>which pairs</b></b> differ.
<ul class="org-ul">
<li>Kruskal-Wallis → Dunn post-hoc test.</li>
<li>Friedman → Nemenyi post-hoc test.</li>
</ul></li>
</ul>
<p class="backlinks-section" id="backlinks">
Backlinks
</p>
<ul class="org-ul backlinks-list">
<li><a href="20250329114733-ise.html#ID-c69e4c4d-2fb4-4cf1-a835-a235cf6db8e9">ise</a></li>
</ul>
</div>
</div>
</div>
</div>
</div>
</div></article></div></div>
<div id="postamble" class="status">
<footer>
<div class="copyright-container">
<div class="copyright">
Copyright &copy; 2022-2025 Zaine Qayyum. All rights reserved unless otherwise noted.</div></div>
<div class="generated">
Created with <a href="https://www.gnu.org/software/emacs/">Emacs</a> 30.2 (<a href="https://orgmode.org">Org</a> mode 9.7.11) on <a href="https://www.archlinux.org/">Arch</a> <a href="https://www.gnu.org">GNU</a>/<a href="https://www.kernel.org/">Linux</a>
</div>
</footer>
</div>
</body>
</html>