딥러닝 학습 관련(epoch, loss)

 

-----

metric 함수. 얘도 함수였구나.

아무튼 metric은 격하게 오락가락 하고

[링크 : https://modulabs.co.kr/blog/loss-versus-accuracy]

 

loss 가 점진적 감소 -> 잘 학습

loss 급격히 감소 후 유지 -> 정상

loss 일정 시점 이후 증가 -> over fitting

[링크 : https://m.blog.naver.com/datapreprep/223748771141]

 

근데.. underfitting은 먼지 감이 안오네.

​   When to change the number of epochs (training cycles) Selecting the appropriate number of epochs is a balance between underfitting and overfitting. Underfitting: One of the most straightforward indicators of underfitting is if the model performs poorly on the training data. This can be observed in Edge Impulse Studio through metrics such as accuracy, or loss, depending on the type of problem (classification or regression). If these metrics indicate poor performance, it suggests that the model has not learned the patterns of the data well. In that case, increasing the number of epochs can improve your model performance. Please note that other solutions exist such as increasing your neural network architecture complexity, changing the preprocessing technique or reducing regularization. Overfitting: Detecting overfitting involves recognizing when the model has learned too much from the training data, including its noise and outliers, to the detriment of its performance on new, unseen data. Overfitting is characterized by the model performing exceptionally well on the training data but poorly on the validation or test data. Evaluating overfitting can be achieved by comparing the performance of the model between the training set and the validation set during training. When the performance on the validation set starts to degrade, it might indicate that the model is beginning to overfit the training data. In that case, decreasing the number of epochs can improve your model performance. As with underfitting, other solutions exist to reduce overfitting such as increasing the number of training data, adding regularization techniques to add penalties on large weights, adding dropout layers, simplifying the model architecture and even using early stopping.

[링크 : https://docs.edgeimpulse.com/knowledge/concepts/machine-learning/neural-networks/epochs]

 

overfitting 중인 loss 값 이라는데, 

 

학습셋이 너무 적을 경우 overfitting이 빠르게 발생하는 것으로 보인다.

그나저나 저 130 miliion parameter는 ssd300 에서 어떻게 산출된걸까?

What you are experiencing is called overfitting and it happens because of your very small dataset. All the model cares about is performance on the training dataset, so given the opportunity, it will simply attempt to memorize it. This is what happens in you case, you feed a model which contains over 130 Million parameters less than 319 images. So regarding your questions: The loss function shows a clear case of overfitting. On general, it is okay to use a trained model, especially when you only have a small dataset, but in your case, the dataset is too small for any deep-learning model. When I say small dataset, I mean 10k images, not several hundreds. You should not train for longer time, once the validation loss stops improving, it is a clear sign to stop. There is even a training technique named "early stopping" which is designed to stop training once the validation loss stops to drop. You have to understand that currently, your dataset of 300 images, is irrelevant to the world of deep-learning. So if you still want to use it for object detection, you need to revert to more classic computer-vision techniques like using HOG or SIFT features, or even manually engineering the features for your special case.

[링크 : https://datascience.stackexchange.com/questions/46456/issues-with-training-ssd-on-own-dataset]

 

The SSD300 has 35.6 million parameters, while the SSDlite320 has about a tenth of them, i.e., 3.4 million.

[링크 : https://www.mdpi.com/2673-6470/3/3/12]