Machine Learning Process (in 6 steps)
The following was summarised from the link below:
1. Formulating a Question
What do you want to find out?
How can you achieve said goal?
YOU MUST narrow down your question to one measurable variable.
Examples: wait times, value of a stock...
2. Finding and Understanding the Data
Most time is spend getting relevant data, and getting it into the correct format, for predictive analysis.
You will then need to analyse the data to determine which model / approach is best.
This could be done by:
- Getting the Mean and Median, to realise the datas distribution
- Getting the percentiles
- Finding correlations between data and possible relationships
- Using graphs to visualise data (box plot to spot outliers), (histograms to show the structure), (scatter plot to show clustering and relationships)
/Untitled.png)
The image above shows:
Avg wait time of: 6.5 min
You can clearly see that there are two groups a short 4 min group, and a longer ~11min group.
You could use a supervised learning model to say, wether the food is going to be a 'long' or 'short' wait time, apposed to actually predicting the result.
3. Cleaning the Data and Feature Engineering
Note: a feature is a group of data
Cleaning data, is the process of removing things that may effect the result, such as outliers.
Feature Engineering, is where you choose the important columns to look at (the important data), and preparing the data for the model.
To prepare the data we might:
- Normalise it (putting different variables on the same scale i.e. num of rooms 0 to 1, price 0 to 1)
- Removing unnecessary columns
- Adding new columns (known as Augmenting)
Repeat steps above to get better results.
4. Choosing a Model
There is alot of models each with there own pros and cons.
Obviously when:
Regression algorithm - when dealing with continuous outputs.
Classification algorithm - when dealing with discrete outputs.
5. Tuning and Evaluating
Are you looking for Accuracy?
Are you looking for Precision? (how repeatable with the same results it is)
You can adjust stuff to make it better at what you are after.
6. Using the Model and Presenting Results
Once you have the model as good as you like it to be, you can use it on the data set you care about.
You can then show off your new found knowledge often by analysing the model.
And example being a feature importance graph.