How many fold cross validation

Active Oldest Votes. Chad Bernier Chad Bernier 1 1 silver badge 10 10 bronze badges. K fold cross validation is exploited to solve problems where Training data is limited. Sign up or log in Sign up using Google. Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown. The Overflow Blog. Does ES6 make JavaScript frameworks obsolete? Podcast Do polyglots have an edge when it comes to mastering programming Variance due to limited sample size usually dominates over model uncertainty.

While it is true that this is only part of the total variance, at least in the situations I encounter in my work, this uncertainty is often so large that even a rough guesstimate is enough to make clear that conclusions are severely limited. And this limitation stays, it won't go away by using 50x 8-folds or 80x 5-folds instead of 40x fold cross validation. Show 7 more comments.

What am I missing? When k is big your are closer to LOO-CV which is very dependent on the particular training set you have at hand: if the number of samples is small it can be not so representative of the true distribution hence the variance. When k is big, k-fold CV can simulate such arbitrary hard samples of the training set.

I highly recommend it. Show 6 more comments. Serge Rogatch Serge Rogatch 3 3 bronze badges. Subhash Rajagopal Subhash Rajagopal 9. So it is not logical to use that value for every condition. Featured on Meta. Now live: A fully responsive profile. Linked 1. OpenML generates train-test splits given the number of folds and repeats, so that different users can evaluate their models with the same splits.

Stratification is applied by default for classification problems unless otherwise specified. The uploaded predictions should be labeled with the fold and repeat number of the test instance, so that the results can be properly evaluated and aggregated. This is because you will want to train on as much data as possible. However, you pay by introducing high variance. It's fine if needed, but best to avoid it. The only downside the the computational cost.

More data in your training set, less bias. However, in doing so you will introduce variance because the testing set is not well representative of your distribution. This will affect the generalization of your model when subject to novel data. Sign up or log in Sign up using Google.

Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown. The Overflow Blog. Does ES6 make JavaScript frameworks obsolete? Podcast Do polyglots have an edge when it comes to mastering programming Featured on Meta.