How to create a model

Important steps:

1. Find dataset

2. Data pre-processing

3. Data splitting (testing and training data)

4. Model training

5. Model evaluation

6. Deployment

Data-pre processing

Cleaning

• Remove bad data

• Remove outliers

Scaling and normalisation

• Data should be between 0 to 1 or whatever range you choose, dont combine 100000000 and a 0.1 cols without normalising

Featurisation

• Only include features that are needed

• Feature reduction: Merge columns? (example is PCA)
  • PCA - it will reduce the number or cols down to a number you want

  

Data encoding

• Labels should be converted into numbers

Data splitting

Simple approach:

• Split the data into two

K-fold:

• Split the data into k equal subsets

• Perform k rounds of training, where one set is used for training and the other for testing

• Take the avg performance of all the runs

• I think it allows you to train and test on the same data