SimpleDataLoaderMapProvider sample batches without replacement if num_samples is not specified

Summary: Samples batches without replacement if the number of samples is not specified. This makes sure that we always iterate over the whole dataset in each epoch.

Reviewed By: bottler

Differential Revision: D39270786

fbshipit-source-id: 0c983d1f5e0af711463abfb23939bc0d2b5172a0
This commit is contained in:
David Novotny 2022-09-06 05:59:01 -07:00 committed by Facebook GitHub Bot
parent f6d43eaa62
commit 73ea4187de

View File

@ -130,7 +130,14 @@ class SimpleDataLoaderMapProvider(DataLoaderMapProviderBase):
num_samples = self.batch_size * num_batches
else:
num_samples = None
sampler = RandomSampler(dataset, replacement=True, num_samples=num_samples)
# sample with replacement only if a custom number of samples is specified
sampler = RandomSampler(
dataset,
replacement=num_samples is not None,
num_samples=num_samples,
)
batch_sampler = BatchSampler(sampler, self.batch_size, drop_last=True)
return DataLoader(
dataset,