Combine datasets
Hankyu Kim
I merged two datasets: a custom dataset with 2,418 images and the COCO128 dataset with 128 images.
After training the model, I evaluated its performance using the trained weight.pt for object detection.
I observed two issues: incorrect detections and failure to recognize objects altogether.
The problem stemmed from the imbalance between the datasets; such a significant disparity can weaken the classification standards.
In classification models, a biased dataset can lead to overfitting, which negatively impacts the model's performance on larger datasets.
If you're facing difficulties in dataset collection, consider setting the training data ratio to 9:1 or 8:2 to mitigate this issue. ๐
Subscribe to my newsletter
Read articles from Hankyu Kim directly inside your inbox. Subscribe to the newsletter, and don't miss out.
Written by
Hankyu Kim
Hankyu Kim
๐ ๐ฉ๐ข๐ท๐ฆ ๐ข ๐ฌ๐ฆ๐ฆ๐ฏ ๐ช๐ฏ๐ต๐ฆ๐ณ๐ฆ๐ด๐ต ๐ข๐ฏ๐ฅ ๐ฑ๐ข๐ด๐ด๐ช๐ฐ๐ฏ ๐ง๐ฐ๐ณ ๐ข๐ถ๐ต๐ฐ๐ฏ๐ฐ๐ฎ๐ฐ๐ถ๐ด ๐ฅ๐ณ๐ช๐ท๐ช๐ฏ๐จ ๐ต๐ฆ๐ค๐ฉ๐ฏ๐ฐ๐ญ๐ฐ๐จ๐บ. ๐๐ถ๐ต๐ฐ๐ฏ๐ฐ๐ฎ๐ฐ๐ถ๐ด ๐ค๐ข๐ณ๐ด ๐ฐ๐ง๐ง๐ฆ๐ณ ๐ญ๐ช๐ฎ๐ช๐ต๐ญ๐ฆ๐ด๐ด ๐ด๐ค๐ฆ๐ฏ๐ข๐ณ๐ช๐ฐ๐ด ๐ง๐ฐ๐ณ ๐ค๐ฐ๐ฏ๐ท๐ฆ๐ฏ๐ช๐ฆ๐ฏ๐ค๐ฆ ๐ข๐ฏ๐ฅ ๐ฒ๐ถ๐ข๐ญ๐ช๐ต๐บ-๐ฐ๐ง-๐ญ๐ช๐ง๐ฆ ๐ช๐ฎ๐ฑ๐ณ๐ฐ๐ท๐ฆ๐ฎ๐ฆ๐ฏ๐ต๐ด. ๐๐ฉ๐ฆ๐บ ๐ค๐ข๐ฏ ๐ฑ๐ณ๐ฐ๐ท๐ช๐ฅ๐ฆ ๐ช๐ฏ๐ฅ๐ฆ๐ฑ๐ฆ๐ฏ๐ฅ๐ฆ๐ฏ๐ค๐ฆ ๐ต๐ฐ ๐ต๐ฉ๐ฆ ๐ฆ๐ญ๐ฅ๐ฆ๐ณ๐ญ๐บ ๐ข๐ฏ๐ฅ ๐ฑ๐ฉ๐บ๐ด๐ช๐ค๐ข๐ญ๐ญ๐บ ๐ฅ๐ช๐ด๐ข๐ฃ๐ญ๐ฆ๐ฅ ๐ช๐ฏ๐ฅ๐ช๐ท๐ช๐ฅ๐ถ๐ข๐ญ๐ด. ๐๐ฐ๐ณ ๐ช๐ฏ๐ด๐ต๐ข๐ฏ๐ค๐ฆ, ๐ช๐ง ๐บ๐ฐ๐ถ๐ณ ๐ค๐ฉ๐ช๐ญ๐ฅ๐ณ๐ฆ๐ฏ ๐ง๐ฐ๐ณ๐จ๐ฆ๐ต ๐ต๐ฉ๐ฆ๐ช๐ณ ๐ฃ๐ข๐ต๐ฉ๐ช๐ฏ๐จ ๐ด๐ถ๐ช๐ต๐ด ๐ข๐ฏ๐ฅ ๐ต๐ฐ๐ฐ๐ต๐ฉ๐ฃ๐ณ๐ถ๐ด๐ฉ๐ฆ๐ด ๐ข๐ต ๐ด๐ถ๐ฎ๐ฎ๐ฆ๐ณ ๐ค๐ข๐ฎ๐ฑ, ๐ต๐ฉ๐ฆ ๐ค๐ข๐ณ ๐ค๐ฐ๐ถ๐ญ๐ฅ ๐ฃ๐ณ๐ช๐ฏ๐จ ๐ต๐ฉ๐ฆ๐ฎ ๐ต๐ฉ๐ฆ ๐ฎ๐ช๐ด๐ด๐ช๐ฏ๐จ ๐ช๐ต๐ฆ๐ฎ๐ด. ๐ ๐ฐ๐ถ ๐ค๐ข๐ฏ ๐ฆ๐ท๐ฆ๐ฏ ๐ด๐ฆ๐ฏ๐ฅ ๐บ๐ฐ๐ถ๐ณ ๐ฅ๐ฐ๐จ ๐ต๐ฐ ๐ข ๐ท๐ฆ๐ต๐ฆ๐ณ๐ช๐ฏ๐ข๐ณ๐บ ๐ข๐ฑ๐ฑ๐ฐ๐ช๐ฏ๐ต๐ฎ๐ฆ๐ฏ๐ต ๐ถ๐ด๐ช๐ฏ๐จ ๐ข๐ถ๐ต๐ฐ๐ฏ๐ฐ๐ฎ๐ฐ๐ถ๐ด ๐ค๐ข๐ณ๐ด. ๐ ๐ข๐ฎ ๐ฑ๐ข๐ด๐ด๐ช๐ฐ๐ฏ๐ข๐ต๐ฆ ๐ข๐ฃ๐ฐ๐ถ๐ต ๐ค๐ฐ๐ฏ๐ต๐ณ๐ช๐ฃ๐ถ๐ต๐ช๐ฏ๐จ ๐ต๐ฐ ๐ต๐ฉ๐ฆ ๐ณ๐ฆ๐ด๐ฆ๐ข๐ณ๐ค๐ฉ ๐ข๐ฏ๐ฅ ๐ฅ๐ฆ๐ท๐ฆ๐ญ๐ฐ๐ฑ๐ฎ๐ฆ๐ฏ๐ต ๐ฐ๐ง ๐ข๐ถ๐ต๐ฐ๐ฏ๐ฐ๐ฎ๐ฐ๐ถ๐ด ๐ฅ๐ณ๐ช๐ท๐ช๐ฏ๐จ ๐ต๐ฆ๐ค๐ฉ๐ฏ๐ฐ๐ญ๐ฐ๐จ๐บ, ๐ฌ๐ฆ๐ฆ๐ฑ๐ช๐ฏ๐จ ๐ช๐ฏ ๐ฎ๐ช๐ฏ๐ฅ ๐ต๐ฉ๐ฆ ๐ฑ๐ฐ๐ด๐ด๐ช๐ฃ๐ช๐ญ๐ช๐ต๐ช๐ฆ๐ด ๐ช๐ต ๐ฉ๐ฐ๐ญ๐ฅ๐ด. ๐๐ช๐ต๐ฉ ๐ฎ๐บ ๐ด๐ฌ๐ช๐ญ๐ญ๐ด ๐ข๐ฏ๐ฅ ๐ฌ๐ฏ๐ฐ๐ธ๐ญ๐ฆ๐ฅ๐จ๐ฆ, ๐ ๐ข๐ช๐ฎ ๐ต๐ฐ ๐ค๐ฐ๐ฏ๐ต๐ณ๐ช๐ฃ๐ถ๐ต๐ฆ ๐ต๐ฐ ๐ข ๐ด๐ถ๐ด๐ต๐ข๐ช๐ฏ๐ข๐ฃ๐ญ๐ฆ ๐ง๐ถ๐ต๐ถ๐ณ๐ฆ ๐ต๐ณ๐ข๐ฏ๐ด๐ฑ๐ฐ๐ณ๐ต๐ข๐ต๐ช๐ฐ๐ฏ ๐ด๐บ๐ด๐ต๐ฆ๐ฎ. ๐ ๐ข๐ฎ ๐ค๐ถ๐ณ๐ณ๐ฆ๐ฏ๐ต๐ญ๐บ ๐ธ๐ฐ๐ณ๐ฌ๐ช๐ฏ๐จ ๐ช๐ฏ ๐ข๐ถ๐ต๐ฐ๐ฏ๐ฐ๐ฎ๐ฐ๐ถ๐ด ๐ฅ๐ณ๐ช๐ท๐ช๐ฏ๐จ ๐ด๐ช๐ฎ๐ถ๐ญ๐ข๐ต๐ฐ๐ณ ๐ค๐ฐ๐ฎ๐ฑ๐ข๐ฏ๐บ ๐ด๐ถ๐น๐จ๐ฐ. ๐๐ญ๐ฆ๐ข๐ด๐ฆ ๐ง๐ฆ๐ฆ๐ญ ๐ง๐ณ๐ฆ๐ฆ ๐ต๐ฐ ๐ฎ๐ฆ๐ด๐ด๐ข๐จ๐ฆ ๐ฎ๐ฆ. ๐'๐ฎ ๐ญ๐ฐ๐ฐ๐ฌ๐ช๐ฏ๐จ ๐ง๐ฐ๐ณ๐ธ๐ข๐ณ๐ฅ ๐ต๐ฐ ๐ด๐ฉ๐ข๐ณ๐ฆ ๐ข๐ฏ๐ฅ ๐ค๐ฐ๐ฏ๐ฏ๐ฆ๐ค๐ต. ๐๐ฉ๐ข๐ฏ๐ฌ ๐บ๐ฐ๐ถ! ๐๐ข๐ฏ๐ฌ๐บ๐ถ ๐๐ช๐ฎ