The thing I have is randomly sampling days from the year however this runs into flaws of the day just not being optimal. Other research papers in related work has two years one for training and another for testing. I only have one year and asked the professor for another year of data and he said it isn’t available with no explanation behind it. I tried searching online for the data since it’s supposedly public data but can’t find it. It’s kind of hard to get an accurate display of evaluation when you don’t have a good test environment. Instead of complaining I have to figure out something. This is in the works in trying to get published to a smaller journal. Not sure if any of you all had this so curious how you would handle such situations?
At least an idea
- Sample months instead of just time samples and create a pseudo environment and run it through there. Picking diverse set that has similar yearly trends.
I don’t have much experience so open ears when you all had similar limitations and how you all overcame it.
When I hear: not a lot of data, i immediately think: overfitting danger.
If it’s a Reinforcement Learning algorithm, then maybe pretraining with synthetic data that’s similar to the real one, so that you already have some rough Q values. And then k-fold crossvalidation. Train on a subset and test on another, and then rotate through.
It has about 18 thousand samples.
Sounds like a decently sized dataset.