:PROPERTIES:
:ID:       06b2a012-4e8a-4a8a-9494-de7ed9fbe1d3
:END:
#+title: ise_week_1
#+filetags: :uni:notes:


* Bug report classification 

Perform bug report classification with AI following the steps
below:

 - Text filtering
 - Stemming
 - Indexing
 - Train machine learning models, i.e., Decision Tree, Naive Bayes, and Logistic
   Regression
 - Prediction (after data pre-processing)

** Text filtering:
 - Punctuation removal
 - Specific transformations occur:
   the myUser has no meaning, its just a word. You need to split it into two meaningful words.
 - Stop words should not be removed, as it changes the whole meaning of a bug report.
 - Unlike other NLP tasks, removing stop words may distort the meaning of bug reports.

** Stemming:
 - Stemming reduces different forms of a word to its common base by sequential application of stemming rules. For example: light caresses colours becomes Light caress colour

** Indexing:

 - Indexing: we need to convert the text into numeric representation, there are two common ways of doing this: TF and IDF
 - TF: total words / specific word . for example: "bug" appears 50 times in a 150 terms. 
 - IDF: We penalise certain words based on their frequency.
 - bug report classifications here often does not use IDF but TF, because some frequent terms should not be penalized

 - For each report, we will have a vector of indexing for words in the reports according to a dictionary
 - Example: A bug report = {“this”, “bugs”, “failure”, “interesting”, …}
            Vector       = <0.123, 0.34, 0.1, ….>

** DT
 - What is decision tree: a tree structure that split the data depending on different values.
 - Each node is a test on the attribute.
 - Each branch represents the outcome.

** NB
 - What is Naive Bayes? Classifier based on the conditional probability given by the Bayes theorem.
 - Calculating the probability of each class, and classify the given features into the one with higher probability.

** LR
 - What is logistic regression: a linear model for classification
 - Fitting the logistic growth and the sigmoid midpoint
 - Output a probability, but can use a cutoff point to decide class.

** Results: 
Choose the top 20/50 words when theyre used as a feature