How to make your neural network work well
Ranging from things like hyperparameter tuning ,to how to set up your data, to how to make sure your optimization algorithm runs quickly
chapter one
Training, Validation & Test dataset
the work flow is:
- keep on training algorithms on your training sets
- use hold-out cross validation set to see which of many different models performs best on your validation set
- evaluate final model on test set
First trend: the way to split your dataset
- previous era of mechine learning
- train:test = 7:3
- train:val:test = 6:2:2
- big data era
- don't need a whole 20% of your data for val
- the val just needs to be big enough for you to evaluate, say, two different algorithm choices which one is doing better
- so, probably, if total = 1milion, 1,000,000
- train = 980,000
- val = 10,000
- tets = 10,000
So, if you have a relatively small dataset, these traditional ratios might be okay. But if you have a much larger data set, it's also fine to set your val and test sets to be much smaller than your 20%
second trend: train on mismatched train and test distributions
suggestion: make sure val and test come from same distribution
Learning video address：https://www.bilibili.com/video/BV12E411a7Xn?p=47&vd_source=cce8c915a5f402d01897a495ccc35141