def load_data( *, data_dir, batch_size, image_size, class_cond=False, deterministic=False, random_crop=False, random_flip=False, ): mage.save_img(image_path, images) if not data_dir: raise ValueError("unspecified data directory") all_files = _list_image_files_recursively(data_dir) # 递归的去找到所有图片文件 classes = None if class_cond: # Assume classes are the first part of the filename, # before an underscore. class_names = [bf.basename(path).split("_")[0] for path in all_files] sorted_classes = {x: i for i, x in enumerate(sorted(set(class_names)))} # class id 将每一类class做成一个整型变量 classes = [sorted_classes[x] for x in class_names] dataset = ImageDataset( image_size, all_files, classes=classes, shard=MPI.COMM_WORLD.Get_rank(), num_shards=MPI.COMM_WORLD.Get_size(), random_crop=random_crop, random_flip=random_flip, ) if deterministic: loader = DataLoader( dataset, batch_size=batch_size, shuffle=False, num_workers=1, drop_last=True ) else: loader = DataLoader( dataset, batch_size=batch_size, shuffle=True, num_workers=1, drop_last=True ) while True: yield from loader,data = load_data( data_dir="F://diffuse_model_ptcg//picture//test", batch_size=4, image_size=128, class_cond=False, )for batch in data:load_data中batch是无尽的吗
时间: 2023-06-27 22:01:31 浏览: 106
data_batch_5_data_batch5_cifar10_data_batch_0_batch_
是的,load_data函数返回的是一个Python生成器对象,使用yield from语句使其不断地产生新的数据批次。因此,使用for循环遍历batch时可以得到无尽的数据批次,直到程序被手动终止或者发生错误。需要注意的是,由于数据集的大小通常是有限的,因此在使用生成器对象时应该谨慎,避免产生过多的数据批次导致内存溢出等问题。
阅读全文