Files
org_roam/output/20250331201944-ise_week_5.html
2025-12-28 20:55:38 +00:00

497 lines
20 KiB
HTML
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2025-12-28 Sun 20:54 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>ise_week_5</title>
<meta name="generator" content="Org Mode" />
<style type="text/css">
#content { max-width: 60em; margin: auto; }
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #e6e6e6;
border-radius: 3px;
background-color: #f2f2f2;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: auto;
}
pre.src:before {
display: none;
position: absolute;
top: -8px;
right: 12px;
padding: 3px;
color: #555;
background-color: #f2f2f299;
}
pre.src:hover:before { display: inline; margin-top: 14px;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-authinfo::before { content: 'Authinfo'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.equation-container {
display: table;
text-align: center;
width: 100%;
}
.equation {
vertical-align: middle;
}
.equation-label {
display: table-cell;
text-align: right;
vertical-align: middle;
}
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { }
</style>
<link rel="stylesheet" href="/assets/styles/style.css" />
<link rel="stylesheet" href="/assets/styles/bigger-picture.min.css" />
<link rel="stylesheet" href="/assets/styles/media.css" />
<script src="/assets/scripts/script.js" defer></script>
<script src="/assets/scripts/search.js" defer></script>
<script src="/assets/scripts/bigger-picture.min.js" defer></script>
<script src="/assets/scripts/svg-pan-zoom.min.js" defer></script>
<script src="/assets/scripts/gallery-init.js" defer></script>
<script>
window.MathJax = {
tex: {
ams: {
multlineWidth: '85%'
},
tags: 'ams',
tagSide: 'right',
tagIndent: '.8em'
},
chtml: {
scale: 1.0,
displayAlign: 'center',
displayIndent: '0em'
},
svg: {
scale: 1.0,
displayAlign: 'center',
displayIndent: '0em'
},
output: {
font: 'mathjax-modern',
displayOverflow: 'overflow'
}
};
</script>
<script
id="MathJax-script"
async
src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js">
</script>
</head>
<body>
<div id="preamble" class="status">
<div class="banner-header">
<div class="banner-left">
<a href="/20241210233721-brain_moc.html"> <img src="/assets/gr.png" alt="Site Logo" class="banner-logo" /> </a>
<div id="updated">Updated: 2025-12-28 Sun 19:48</div>
</div>
<div class="banner-search">
<input id="search-box"
type="search"
placeholder="Search notes…"
aria-label="Search notes"
autocomplete="off" />
<div id="search-results"></div>
</div>
<button id="close-all">Close All</button>
</div>
</div>
<div id="stack-root">
<div class="stack-track">
<article class="stack-pane pane-root" data-url="/20250331201944-ise_week_5.html">
<div id="content" class="content">
<div class="title-section">
<div class="title-controls">
<button class="pane-fullscreen" aria-label="Fullscreen">F</button>
<button class="pane-edit" aria-label="Edit pane">E</button>
<button class="pane-close" aria-label="Close pane">×</button>
</div>
<h1 class="title">ise<sub>week</sub><sub>5</sub></h1>
<div class="title-metadata">
<span class="metadata-item">
<span class="metadata-label">planted:</span>
<span class="metadata-value">2025-03-31</span>
</span>
<span class="metadata-item">
<span class="metadata-label">last tended to:</span>
<span class="metadata-value">2025-12-28</span>
</span>
</div>
</div>
<div id="outline-container-org958f870" class="outline-2">
<h2 id="org958f870"><a href="#org958f870"><span class="todo TODO">TODO</span> Model free Tuning (for ORM) (pages 9-17)</a></h2>
</div>
<div id="outline-container-org3b61b9b" class="outline-2">
<h2 id="org3b61b9b"><a href="#org3b61b9b"><span class="todo TODO">TODO</span> Model free Tuning (BestConfig) (pages 18-23)</a></h2>
</div>
<div id="outline-container-org3046aac" class="outline-2">
<h2 id="org3046aac"><a href="#org3046aac">5.1 Model-Free Tuning for ORM Systems - Intelligent Software Engineering</a></h2>
<div class="outline-text-2" id="text-org3046aac">
</div>
<div id="outline-container-org9196a2e" class="outline-3">
<h3 id="org9196a2e"><a href="#org9196a2e">1. Introduction and Background</a></h3>
<div class="outline-text-3" id="text-org9196a2e">
<ul class="org-ul">
<li>This approach was proposed by Singh et al. (2016).</li>
<li>The goal is to optimize Object-Relational Mapping (ORM) systems without relying on models.</li>
<li>Uses <b><b>NSGA-II</b></b>, a multi-objective evolutionary algorithm, to handle <b><b>multiple performance concerns</b></b>.</li>
</ul>
<p>
Reference: Singh, Ravjot et al. <b>&ldquo;Optimizing the performance-related configurations of object-relational mapping frameworks using a multi-objective genetic algorithm.&rdquo;</b> ACM/SPEC ICPE 2016.
</p>
</div>
</div>
<div id="outline-container-org7ad873e" class="outline-3">
<h3 id="org7ad873e"><a href="#org7ad873e">2. Architecture and Setup</a></h3>
<div class="outline-text-3" id="text-org7ad873e">
<ul class="org-ul">
<li>Focuses exclusively on <b><b>binary configuration options</b></b>.</li>
<li>Example of a configuration: `{0011}` a binary vector where each bit represents a configuration toggle (on/off).</li>
<li>Evaluates performance using three objective metrics:
<ul class="org-ul">
<li><b><b>Execution time</b></b></li>
<li><b><b>CPU load</b></b></li>
<li><b><b>Memory consumption</b></b></li>
</ul></li>
<li>Other components follow the standard NSGA-II flow: selection, crossover, mutation, and fitness evaluation.</li>
</ul>
</div>
</div>
<div id="outline-container-org4229c33" class="outline-3">
<h3 id="org4229c33"><a href="#org4229c33">3. Stopping Criteria</a></h3>
<div class="outline-text-3" id="text-org4229c33">
<p>
Two specific stopping rules are proposed for determining when to terminate the evolutionary process:
</p>
</div>
<div id="outline-container-org93f0342" class="outline-4">
<h4 id="org93f0342"><a href="#org93f0342">a. Setting 1: t-test-based Stopping</a></h4>
<div class="outline-text-4" id="text-org93f0342">
<ul class="org-ul">
<li>Conducts statistical <b><b>t-tests</b></b> to compare changes in objective values between generations.</li>
<li>For each pair of consecutive generations \( g_i \) and \( g_j \):
<ul class="org-ul">
<li>Run a t-test on ∆CPU and ∆MEM between all configurations in both generations.</li>
</ul></li>
<li>If <b><b>all p-values &gt; 0.05</b></b> for <b><b>two consecutive generations</b></b>, it indicates <b><b>no statistically significant improvement</b></b>, and the algorithm is stopped.</li>
</ul>
</div>
</div>
<div id="outline-container-orgc722cfe" class="outline-4">
<h4 id="orgc722cfe"><a href="#orgc722cfe">b. Setting 2: Mutual Dominance Rate (MDR)</a></h4>
<div class="outline-text-4" id="text-orgc722cfe">
<ul class="org-ul">
<li>Measures how much progress is made by comparing the current and previous generation.</li>
<li>Let:
<ul class="org-ul">
<li>Set A = configurations from the previous generation</li>
<li>Set B = configurations from the current generation</li>
</ul></li>
<li>Define \( dom(A,B) \) as the number of configurations in A that are <b><b>dominated</b></b> by any configuration in B.</li>
</ul>
<p>
Interpretation:
</p>
<ul class="org-ul">
<li><b><b>MDR = 0</b></b>: No progress — performance plateau</li>
<li><b><b>MDR &lt; 0</b></b>: Regression — performance is deteriorating</li>
<li><b><b>MDR &gt; 0</b></b>: Improvement — current generation is better than the last</li>
</ul>
</div>
</div>
</div>
<div id="outline-container-orgdabead0" class="outline-3">
<h3 id="orgdabead0"><a href="#orgdabead0">4. Termination Condition</a></h3>
<div class="outline-text-3" id="text-orgdabead0">
<ul class="org-ul">
<li>The tuning process should stop if <b><b>any</b></b> of the defined stopping conditions (t-test or MDR) are met.</li>
</ul>
</div>
</div>
<div id="outline-container-org75b7607" class="outline-3">
<h3 id="org75b7607"><a href="#org75b7607">5. Experimental Results</a></h3>
<div class="outline-text-3" id="text-org75b7607">
<ul class="org-ul">
<li>NSGA-II consistently found configurations that ranked within the <b><b>top 25%</b></b> of all possible configurations across tested applications.</li>
<li>Results were obtained by combining different <b><b>aggregation functions</b></b> and <b><b>stopping rules</b></b>, demonstrating strong generalization and effectiveness.</li>
</ul>
</div>
</div>
</div>
<div id="outline-container-org5e9c0bc" class="outline-2">
<h2 id="org5e9c0bc"><a href="#org5e9c0bc">5.2 Model-Free Tuning with BestConfig - Intelligent Software Engineering</a></h2>
<div class="outline-text-2" id="text-org5e9c0bc">
</div>
<div id="outline-container-org200b5fc" class="outline-3">
<h3 id="org200b5fc"><a href="#org200b5fc">1. Introduction and Background</a></h3>
<div class="outline-text-3" id="text-org200b5fc">
<ul class="org-ul">
<li>BestConfig is a model-free configuration tuning system proposed by <b><b>Zhu et al. (2017)</b></b>.</li>
<li>It focuses on tuning for <b><b>a single performance objective</b></b> (e.g., throughput, latency).</li>
<li>Utilizes <b><b>local search techniques</b></b> rather than global evolutionary approaches.</li>
<li>Employs <b><b>label encoding</b></b> for parameters (e.g., {0, 23, 100}).</li>
<li>Key strategy: aggressively explore <b><b>promising regions</b></b> of the configuration space.</li>
</ul>
<p>
Reference: Zhu, Yuqing et al. <b>&ldquo;BestConfig: tapping the performance potential of systems via automatic configuration tuning.&rdquo;</b> SoCC 2017.
</p>
</div>
</div>
<div id="outline-container-org92fae6e" class="outline-3">
<h3 id="org92fae6e"><a href="#org92fae6e">2. Architecture Overview</a></h3>
<div class="outline-text-3" id="text-org92fae6e">
<ul class="org-ul">
<li>BestConfig is designed to intelligently search a high-dimensional configuration space.</li>
<li>Architecture relies on two core components:
<ul class="org-ul">
<li>DDS (Divide &amp; Diverge Sampling)</li>
<li>RBS (Recursive Bound &amp; Search)</li>
</ul></li>
</ul>
</div>
</div>
<div id="outline-container-orgd75c60f" class="outline-3">
<h3 id="orgd75c60f"><a href="#orgd75c60f">3. DDS: Divide &amp; Diverge Sampling</a></h3>
<div class="outline-text-3" id="text-orgd75c60f">
<ul class="org-ul">
<li>Purpose: Ensures <b><b>coverage</b></b> of the entire configuration space by dividing it into <b><b>subspaces</b></b>.</li>
<li>Process:
<ol class="org-ol">
<li>Each configuration parameter&rsquo;s range is divided into <b><b>k intervals</b></b>.</li>
<li>These intervals are combined across all parameters, forming \( k^n \) subspaces.</li>
<li><b><b>One random sample</b></b> is taken from each subspace.</li>
</ol></li>
<li>Advantages:
<ul class="org-ul">
<li>Avoids bias in sampling (common in uniform random search).</li>
<li>More likely to sample from all areas of the space.</li>
<li>Especially useful in <b><b>high-dimensional spaces</b></b>.</li>
</ul></li>
</ul>
</div>
</div>
<div id="outline-container-org0edda77" class="outline-3">
<h3 id="org0edda77"><a href="#org0edda77">4. RBS: Recursive Bound &amp; Search</a></h3>
<div class="outline-text-3" id="text-org0edda77">
<ul class="org-ul">
<li>Purpose: Locally refines and improves the best-known configuration.</li>
<li>Steps:
<ol class="org-ol">
<li>Identify the best-performing configuration \( C_0 \) from the initial samples.</li>
<li>Define <b><b>bounds</b></b> for each parameter based on neighboring values around \( C_0 \).</li>
<li>Sample new points within this bounded space to find a better configuration \( C_1 \).</li>
<li>Repeat the bounding and sampling process <b><b>recursively</b></b> until no improvement is found.</li>
</ol></li>
<li>Bound Definition:
<ul class="org-ul">
<li>For each parameter value in \( C_0 \), the closest lower and higher values in the dataset are chosen as bounds.</li>
</ul></li>
<li>Termination Conditions:
<ul class="org-ul">
<li>If no better configuration is found in a recursive round, the search <b><b>restarts from a broader space</b></b>.</li>
<li>The entire tuning process <b><b>stops</b></b> only when a <b><b>predefined resource budget</b></b> (e.g., time, evaluations) is exhausted.</li>
</ul></li>
</ul>
</div>
</div>
<div id="outline-container-org2f2d133" class="outline-3">
<h3 id="org2f2d133"><a href="#org2f2d133">5. Results and Observations</a></h3>
<div class="outline-text-3" id="text-org2f2d133">
<ul class="org-ul">
<li>BestConfig consistently finds configurations <b><b>significantly better than the systems default settings</b></b>.</li>
<li>Achieves these improvements within a <b><b>reasonable time frame</b></b>, making it practical for real-world use.</li>
</ul>
<p class="backlinks-section" id="backlinks">
Backlinks
</p>
<ul class="org-ul backlinks-list">
<li><a href="20250329114733-ise.html#ID-c69e4c4d-2fb4-4cf1-a835-a235cf6db8e9">ise</a></li>
</ul>
</div>
</div>
</div>
</div>
</div></article></div></div>
<div id="postamble" class="status">
<footer>
<div class="copyright-container">
<div class="copyright">
Copyright &copy; 2022-2025 Zaine Qayyum. All rights reserved unless otherwise noted.</div></div>
<div class="generated">
Created with <a href="https://www.gnu.org/software/emacs/">Emacs</a> 30.2 (<a href="https://orgmode.org">Org</a> mode 9.7.11) on <a href="https://www.archlinux.org/">Arch</a> <a href="https://www.gnu.org">GNU</a>/<a href="https://www.kernel.org/">Linux</a>
</div>
</footer>
</div>
</body>
</html>