org_roam/Uni/20250329142725-ise_week_3.org

ISE Week 3

• Classic Code Complexity Metrics (pages 8-27)
• How to Mine Bugs for Learning? (pages 37-43)
• Cross Project Prediction (HDP) (pages 50-60)
• 3.1 Intelligent Software Engineering: Classic Metrics
  • Software Defect Prediction
  • Classic Code Metrics
    • 1. McCabe Cyclomatic Complexity
      • Why it Matters
      • Simple Definition
      • What is a “Simple Condition”?
      • Compound Conditions
      • Use Case
    • 2. Halstead Complexity Measures
      • Definitions:
      • Why Use Halstead?
    • 3. Lines of Code (LOC)
      • Usefulness:
• 3.2 Intelligent Software Engineering: Within-Project Prediction
  • Just-in-Time (JIT) Defect Prediction
  • Steps in the JIT Defect Prediction Pipeline:
  • Change-wise Prediction Details
    • Change History Extraction
    • Identifying Bug-Introducing Changes
      • Step 1: Search for Bug Fixes
      • Step 2: Use the SZZ Algorithm
  • Example Walkthrough
    • Revision 1:
    • Revision 2:
    • Revision 3:
• 3.3 Intelligent Software Engineering: Cross-Project Prediction (HDP)
  • Heterogeneous Defect Prediction (HDP)
  • HDP Architecture
    • Metric Selection in Source Datasets
    • Matching Source and Target Metrics
      • Key Steps:
      • Example:
  • Methods for Calculating Matching Scores
    • Percentile-Based Method
    • Kolmogorov-Smirnov (KS) Test Method
    • Spearman’s Rank Correlation Coefficient Method
  • Classifier Independence

DONE Classic Code Complexity Metrics (pages 8-27)

DONE How to Mine Bugs for Learning? (pages 37-43)

DONE Cross Project Prediction (HDP) (pages 50-60)

3.1 Intelligent Software Engineering: Classic Metrics

Software Defect Prediction

• The foundation of software defect prediction lies in metric identification.
• This was a key research direction in the 1980s.
• Metrics aim to quantify properties of code to detect potential defects and improve quality.

Classic Code Metrics

1. McCabe Cyclomatic Complexity

• Purpose: Measures the complexity of code based on the number of linearly independent paths in the code’s flow graph.

Why it Matters

• More conditional statements = More possible execution paths = Higher complexity.
• Useful for identifying complex, hard-to-test, and error-prone code.

Simple Definition

• McCabe Complexity = Number of simple conditions + 1

What is a “Simple Condition”?

• A conditional without logical connectors (AND, OR).

• Examples:

  • if (a > b)
  • while (a > b)
  • for (a=b; a > b; b++)
  • do {…} while (a > b)

Compound Conditions

• Count each simple condition inside:

  • if (a > b || a > 2) → 2 simple conditions
  • if (a > b && a > 2) → 2 simple conditions

Use Case

• Helps determine test case count needed for complete branch coverage.

2. Halstead Complexity Measures

• Purpose: Measures complexity based on the operators and operands used in code.

Definitions:

• n1: Number of distinct operators (e.g., !=, !, %, /, *, +, &&, ||)
• n2: Number of distinct operands (e.g., variable names, constants, types like bool, char)
• N1: Total occurrences of operators
• N2: Total occurrences of operands

Why Use Halstead?

• Evaluates:

  • Code length
  • Code vocabulary
  • Effort required to implement or understand the code
  • Potential bugs

3. Lines of Code (LOC)

• LOC: Total number of lines in a program.
• Comment Lines: Lines containing only comments.

Usefulness:

• Simple indicator of:

  • Code size
  • Code density
  • Maintainability and readability

3.2 Intelligent Software Engineering: Within-Project Prediction

Just-in-Time (JIT) Defect Prediction

• Based on the classic work by Kim et al. (2008).
• Focuses on predicting defects at the commit/change level rather than file or module level.

Steps in the JIT Defect Prediction Pipeline:

1. File-level changes are extracted from a project's revision history.
2. Bug fix changes are identified using keywords in SCM (Source Code Management) change log messages.
3. Bug-introducing and clean changes are identified by tracing backwards from the bug fix commits.
4. A classification model (e.g., SVM) is trained on these labeled examples.
5. Once trained, the classifier can predict if new code changes are likely to be buggy or clean.

Change-wise Prediction Details

Change History Extraction

• Collected information includes:

  • Change log
  • Author
  • Change date
  • Source code
  • Change delta
  • Change metadata

Identifying Bug-Introducing Changes

Step 1: Search for Bug Fixes

• Use keywords (e.g., “fix”, “bug”, “patch”) to find bug-fixing commits.

Step 2: Use the SZZ Algorithm

• Determine what was changed in bug fixes.
• Produces a list of regions ("hunks") showing differences between two revisions.
• Deleted or modified code in each hunk is treated as the location of a bug.
• Traces origin of this code to find the earlier bug-introducing changes.

Example Walkthrough

Revision 1:

• Initial creation of a function `bar`.
• Introduces a bug: `if (report == null)` (should be `!=`).
• SCM annotate shows all lines as modified in revision 1 by "kim".

Revision 2:

• Two changes:

  • Function `bar` renamed to `foo`.
  • Argument changed from `report` to `report.str` in `println`.
• Annotate output shows lines 1 and 4 were last modified by "ejw" in revision 2.

Revision 3:

• Bug fix applied: changes `==` to `!=` on line 3.
• SZZ algorithm compares revisions 3 and 2, identifying line 3 as modified.
• Traces line 3’s origin back to revision 1 — identifying the bug-introducing change.

3.3 Intelligent Software Engineering: Cross-Project Prediction (HDP)

Heterogeneous Defect Prediction (HDP)

• Based on the work by Nam and Kim (2015).
• Motivation: Metrics used for defect prediction often differ across projects.
• Goal: Address the metric mismatching problem across projects (heterogeneous settings).
• Classifier agnostic — can be used with any machine learning model.

HDP Architecture

Metric Selection in Source Datasets

• Uses well-known feature selection methods:

  • Gain ratio
  • Chi-square
  • Relief-F
  • Significance attribute evaluation
• Empirical testing used to choose the best approach.
• Top 15% metrics per source project are selected.
• Metric mismatching arises because each project may prioritize different metrics.

Matching Source and Target Metrics

Key Steps:

1. Pair all metrics from source and target projects.
2. Remove poorly matched metrics based on a cutoff threshold for matching scores.

3. Apply maximum weighted bipartite matching to select the best group of matched metric pairs:

   • Goal: Maximize sum of matching scores.
   • Ensure no duplicated metrics are selected.

Example:

• 2 source metrics: X1, X2
• 2 target metrics: Y1, Y2
• Matching pairs: (X1,Y1), (X1,Y2), (X2,Y1), (X2,Y2)

After applying a cutoff threshold of 0.30:

• Group 1: (X1,Y1) and (X2,Y2) with total score 1.3 (=0.8+0.5)
• Group 2: (X2,Y1) with score 0.4 (Stands alone (can't be paired with any other remaining pair without duplication)).
• Group 1 is chosen as the matched metric set.

Methods for Calculating Matching Scores

Percentile-Based Method

• Compares 9 percentiles (10th, 20th, …, 90th) between source and target metric values.

• Uses the formula: Pij(n) = 1 - |spij(n) - bpij(n)| / bpij(n)

  • spij(n): smaller percentile value
  • bpij(n): bigger percentile value
• Matching score is 1 when all percentiles are identical.

Kolmogorov-Smirnov (KS) Test Method

• Non-parametric two-sample test.
• Useful when distributions are unknown or have unequal variances.
• Computes a p-value to indicate the similarity.
• Matching score derived from the p-value.

Spearman’s Rank Correlation Coefficient Method

• Measures correlation between two sets of values.
• If dataset sizes differ, randomly sample the larger set to match sizes.

Classifier Independence

• HDP approach can be paired with any machine learning algorithm (e.g., SVM, RF, etc.)