pax_global_header00006660000000000000000000000064146367423150014525gustar00rootroot0000000000000052 comment=873cc8084baac4431c0e8db7e000ebc3ca16444d openi_cloudbrain_example/000077500000000000000000000000001463674231500161205ustar00rootroot00000000000000openi_cloudbrain_example/.openi-aitemplate000066400000000000000000000016011463674231500213540ustar00rootroot00000000000000[ { "Name": "GPU训练任务", "Description": "", "JobType": "TRAIN", "Cluster": "C2Net", "ComputeSource": "GPU", "HasInternet": 2, "AccCardsNum": 1, "AccCardType": "V100", "CpuCores": 8, "MemGiB": 50, "GPUMemGiB": 32, "ShareMemGiB": 0, "Image": { "ImageID": "", "ImageName": "", "ImageUrl": "192.168.204.22:5000/default-workspace/99280a9940ae44ca8f5892134386fddb/image:ubuntu22.04-cuda11.8.0-py310-torch2.1.0-tf2.14.0" }, "PretrainModelList": [ { "ID": "f1711e48-65d6-436c-b34b-9698061979e7", "ModelName": "Torch_MNIST_Example_Model" } ], "DatasetList": [ { "ID": "8d5178b2-eb56-4c0d-b532-be8c685b1e61", "DatasetName": "MnistDataset_torch.zip" } ], "BranchName": "liuzx", "BootFile": "gpu_mnist_example/train.py", "Parameters": [] } ]openi_cloudbrain_example/README.md000066400000000000000000000017621463674231500174050ustar00rootroot00000000000000# 如何在启智平台上进行模型调试和训练 ## 1. sdk使用方法 使用c2net的SDK方式 ``` #导入包 from c2net.context import prepare, upload_output #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取代码路径，数据集路径，预训练模型路径，输出路径 code_path = c2net_context.code_path dataset_path = c2net_context.dataset_path pretrain_model_path = c2net_context.pretrain_model_path output_path = c2net_context.output_path #必须将输出结果保存到c2net_context.output_path，才能回传结果到openi，并且训练任务才能回传，调试任务回传后暂时不支持下载 upload_output() ``` ## 2. 手写数字识别示例 * GPU示例请参考[gpu_mnist_example](./gpu_mnist_example/README.md) * GPGPU示例请参考[gpgpu_mnist_example](./gpgpu_mnist_example/README.md) * NPU示例请参考[npu_mnist_example](./npu_mnist_example/README.md) * GCU示例请参考[gcu_mnist_example](./gcu_mnist_example/README.md) openi_cloudbrain_example/gcu_mnist_example/000077500000000000000000000000001463674231500216235ustar00rootroot00000000000000openi_cloudbrain_example/gcu_mnist_example/README.md000066400000000000000000000117541463674231500231120ustar00rootroot00000000000000# 如何在启智平台上进行模型调试和训练—GCU_手写数字识别示例 ## 一 ，数据集及预训练模型准备 ##### 1，数据集说明： 数据集可从本项目的数据集中引用，[数据集引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/datasets) - MnistDataset_torch数据集是由10类28∗28的灰度图片组成，训练数据集包含60000张图片，测试数据集包含10000张图片。 - MnistDataset_torch.zip数据集压缩包的目录结构如下： > MnistDataset_torch.zip > > ├── test > > └── train > ##### 2，预训练模型说明： Torch_MNIST_Example_Model模型可从本项目的模型目录中引用，[预训练模型引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/modelmanage/show_model) * Torch_MNIST_Example_Model模型的目录结构如下： > Torch_MNIST_Example_Model > ├── mnist_epoch1.pkl > ## 二. 如何在云脑上获取代码路径，数据集路径，预训练模型路径，输出路径 ##### 1，准备数据 ``` from c2net.context import prepare c2net_context = prepare() ``` ##### 2，获取代码路径 ``` code_path = c2net_context.code_path +"/" +"项目名".lower() 在本示例中代码路径为： code_path = c2net_context.code_path + "/" + "Openl_Cloudbrain_Example".lower() ``` ##### 3，获取数据集路径 ``` dataset_path = c2net_context.dataset_path +"/" +"数据集名称" 在本示例中代码路径为： dataset_path = c2net_context.dataset_path + "/" + "MnistDataset_torch" ``` ##### 4，获取预训练模型路径 ``` pretrain_model_path = c2net_context.pretrain_model_path +"/" +"模型名称" 在本示例中预训练模型路径为： pretrain_model_path = c2net_context.pretrain_model_path + "/" + "Torch_MNIST_Example_Model" ``` ##### 5，获取输出路径 ``` output_path = c2net_context.output_path 在本示例中输出路径为： output_path = c2net_context.output_path ``` ##### 6，回传结果 ``` from c2net.context import upload_output upload_output() 在本示例中回传结果为（只有训练任务才能回传结果）： from c2net.context import prepare,upload_output upload_output() ``` ## 三.GCU样例准备 ##### 1，GCU示例代码： - 训练任务单卡示例请参考示例中[train.py](./train.py)的代码注释 ##### 2，创建GCU调试任务 表1创建训练作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择含有python和torch的镜像 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择gcu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_torch.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[GCU] | | 模型 | 模型选择Torch_MNIST_Example_Model | 启动调试任务后，先执行prepare()进行数据准备； 进入对应的代码目录后，可在终端执行python train.py； ##### 3，创建GCU训练任务 表2 创建训练作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择含有python和torch的镜像 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择gpu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_torch.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[GCU] | | 模型 | 模型选择Torch_MNIST_Example_Model | 启动训练任务后，训练结束会在任务的结果下载页提供输出结果下载 ## 四.GCU任务注意事项 ##### 1, 使用超参数的方法： 请在代码中加入 ``` import parser args, unknown = parser.parse_known_args() #可忽略掉 `--ckpt_url`，`--data_url`, `--multi_date_url`等参数无定义导致的报错问题 ``` ## 对于示例代码有任何问题，欢迎在本项目中提issue。 openi_cloudbrain_example/gcu_mnist_example/model.py000066400000000000000000000016561463674231500233050ustar00rootroot00000000000000from torch.nn import Module from torch import nn class Model(Module): def __init__(self): super(Model, self).__init__() self.conv1 = nn.Conv2d(1, 6, 5) self.relu1 = nn.ReLU() self.pool1 = nn.MaxPool2d(2) self.conv2 = nn.Conv2d(6, 16, 5) self.relu2 = nn.ReLU() self.pool2 = nn.MaxPool2d(2) self.fc1 = nn.Linear(256, 120) self.relu3 = nn.ReLU() self.fc2 = nn.Linear(120, 84) self.relu4 = nn.ReLU() self.fc3 = nn.Linear(84, 10) self.relu5 = nn.ReLU() def forward(self, x): y = self.conv1(x) y = self.relu1(y) y = self.pool1(y) y = self.conv2(y) y = self.relu2(y) y = self.pool2(y) y = y.view(y.shape[0], -1) y = self.fc1(y) y = self.relu3(y) y = self.fc2(y) y = self.relu4(y) y = self.fc3(y) y = self.relu5(y) return yopeni_cloudbrain_example/gcu_mnist_example/train.py000066400000000000000000000121171463674231500233140ustar00rootroot00000000000000#!/usr/bin/python #coding=utf-8 ''' If there are Chinese comments in the code，please add at the beginning： #!/usr/bin/python #coding=utf-8 示例选用的数据集是MnistDataset_torch.zip 数据集结构是： MnistDataset_torch.zip ├── test └── train 预训练模型文件夹结构是： Torch_MNIST_Example_Model ├── mnist_epoch1.pkl ''' import torch from model import Model import numpy as np from torchvision.datasets import mnist from torch.nn import CrossEntropyLoss from torch.optim import SGD from torch.utils.data import DataLoader from torchvision.transforms import ToTensor import argparse import os #导入c2net包 from c2net.context import prepare, upload_output import importlib.util def is_torch_dtu_available(): if importlib.util.find_spec("torch_dtu") is None: return False if importlib.util.find_spec("torch_dtu.core") is None: return False return importlib.util.find_spec("torch_dtu.core.dtu_model") is not None # Training settings parser = argparse.ArgumentParser(description='PyTorch MNIST Example') parser.add_argument('--epoch_size', type=int, default=1, help='how much epoch to train') parser.add_argument('--batch_size', type=int, default=256, help='how much batch_size in epoch') if __name__ == '__main__': args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_torch_path = c2net_context.dataset_path+"/"+"MnistDataset_torch" #获取预训练模型路径 Torch_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Torch_MNIST_Example_Model" #获取输出路径 output_path = c2net_context.output_path # load DPU envs-xx.sh DTU_FLAG = True if is_torch_dtu_available(): import torch_dtu import torch_dtu.distributed as dist import torch_dtu.core.dtu_model as dm from torch_dtu.nn.parallel import DistributedDataParallel as torchDDP print('dtu is available: True') device = dm.dtu_device() DTU_FLAG = True else: print('dtu is available: False') device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") DTU_FLAG = False # 参数声明 model = Model().to(device) optimizer = SGD(model.parameters(), lr=1e-1) args, unknown = parser.parse_known_args() #log output batch_size = args.batch_size train_dataset = mnist.MNIST(root=os.path.join(MnistDataset_torch_path, "train"), train=True, transform=ToTensor(),download=False) test_dataset = mnist.MNIST(root=os.path.join(MnistDataset_torch_path, "test"), train=False, transform=ToTensor(),download=False) train_loader = DataLoader(train_dataset, batch_size=batch_size) test_loader = DataLoader(test_dataset, batch_size=batch_size) model = Model().to(device) sgd = SGD(model.parameters(), lr=1e-1) cost = CrossEntropyLoss() epochs = args.epoch_size print('epoch_size is:{}'.format(epochs)) # 如果有保存的模型，则加载模型，并在其基础上继续训练 if os.path.exists(os.path.join(Torch_MNIST_Example_Model_path, "mnist_epoch1.pkl")): checkpoint = torch.load(os.path.join(Torch_MNIST_Example_Model_path, "mnist_epoch1.pkl")) model.load_state_dict(checkpoint['model']) optimizer.load_state_dict(checkpoint['optimizer']) start_epoch = checkpoint['epoch'] print('加载 epoch {} 权重成功！'.format(start_epoch)) else: start_epoch = 0 print('无保存模型，将从头开始训练！') for _epoch in range(start_epoch, epochs): print('the {} epoch_size begin'.format(_epoch + 1)) model.train() for idx, (train_x, train_label) in enumerate(train_loader): train_x = train_x.to(device) train_label = train_label.to(device) label_np = np.zeros((train_label.shape[0], 10)) sgd.zero_grad() predict_y = model(train_x.float()) loss = cost(predict_y, train_label.long()) if idx % 10 == 0: print('idx: {}, loss: {}'.format(idx, loss.sum().item())) loss.backward() if DTU_FLAG: dm.optimizer_step(sgd, barrier=True) else: sgd.step() correct = 0 _sum = 0 model.eval() for idx, (test_x, test_label) in enumerate(test_loader): test_x = test_x test_label = test_label predict_y = model(test_x.to(device).float()).detach() predict_ys = np.argmax(predict_y.cpu(), axis=-1) label_np = test_label.numpy() _ = predict_ys == test_label correct += np.sum(_.numpy(), axis=-1) _sum += _.shape[0] print('accuracy: {:.2f}'.format(correct / _sum)) state = {'model':model.state_dict(), 'optimizer':optimizer.state_dict(), 'epoch':_epoch+1} torch.save(state, '{}/mnist_epoch{}_{:.2f}.pkl'.format(output_path, _epoch+1, correct / _sum)) print(os.listdir('{}'.format(output_path)))openi_cloudbrain_example/gpgpu_mnist_example/000077500000000000000000000000001463674231500221675ustar00rootroot00000000000000openi_cloudbrain_example/gpgpu_mnist_example/README.md000066400000000000000000000121071463674231500234470ustar00rootroot00000000000000# 如何在启智平台上进行模型调试和训练—GPGPU_手写数字识别示例 ## 一 ，数据集及预训练模型准备 ##### 1，数据集说明： 数据集可从本项目的数据集中引用，[数据集引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/datasets) - MnistDataset_torch数据集是由10类28∗28的灰度图片组成，训练数据集包含60000张图片，测试数据集包含10000张图片。 - MnistDataset_torch.zip数据集压缩包的目录结构如下： > MnistDataset_torch.zip > > ├── test > > └── train > ##### 2，预训练模型说明： Torch_MNIST_Example_Model模型可从本项目的模型目录中引用，[预训练模型引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/modelmanage/show_model) * Torch_MNIST_Example_Model模型的目录结构如下： > Torch_MNIST_Example_Model > ├── mnist_epoch1.pkl > ## 二. 如何在云脑上获取代码路径，数据集路径，预训练模型路径，输出路径 ##### 1，准备数据 ``` from c2net.context import prepare c2net_context = prepare() ``` ##### 2，获取代码路径 ``` code_path = c2net_context.code_path +"/" + "项目名".lower() 在本示例中代码路径为： code_path = c2net_context.code_path + "/" + "Openl_Cloudbrain_Example".lower() ``` ##### 3，获取数据集路径 ``` dataset_path = c2net_context.dataset_path +"/" +"数据集名称" 在本示例中代码路径为： dataset_path = c2net_context.dataset_path + "/" + "MnistDataset_torch" ``` ##### 4，获取预训练模型路径 ``` pretrain_model_path = c2net_context.pretrain_model_path +"/" +"模型名称" 在本示例中预训练模型路径为： pretrain_model_path = c2net_context.pretrain_model_path + "/" + "Torch_MNIST_Example_Model" ``` ##### 5，获取输出路径 ``` output_path = c2net_context.output_path 在本示例中输出路径为： output_path = c2net_context.output_path ``` ##### 6，回传结果 ``` from c2net.context import upload_output upload_output() 在本示例中回传结果为（只有训练任务才能回传结果）： from c2net.context import prepare,upload_output upload_output() ``` ## 三.GPGPU样例准备 ##### 1，GPGPU示例代码： - 训练任务示例请参考示例中[train.py](./train.py)的代码注释 - 推理任务示例请参考示例中[inference.py](./inference.py)的代码注释 ##### 2，创建GPGPU调试任务 表1创建调试作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择含有python和torch的镜像 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择gpu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_torch.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[GPGPU] | | 模型 | 模型选择Torch_MNIST_Example_Model | 启动调试任务后，先执行prepare()进行数据准备； 进入对应的代码目录后，可在终端执行python train.py； ##### 3，创建GPGPU训练任务 表2 创建推理作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择含有python和torch的镜像 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择gpgpu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_torch.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[GPGPU] | | 模型 | 模型选择Torch_MNIST_Example_Model | 启动训练任务后，训练结束会在任务的结果下载页提供输出结果下载 ## 四.GPGPU任务注意事项 ##### 1, 使用超参数的方法： 请在代码中加入 ``` import parser args, unknown = parser.parse_known_args() #可忽略掉 `--ckpt_url`，`--data_url`, `--multi_date_url`等参数无定义导致的报错问题 ``` ## 对于示例代码有任何问题，欢迎在本项目中提issue。 openi_cloudbrain_example/gpgpu_mnist_example/inference.py000066400000000000000000000061011463674231500244750ustar00rootroot00000000000000#!/usr/bin/python #coding=utf-8 ''' If there are Chinese comments in the code，please add at the beginning： #!/usr/bin/python #coding=utf-8 示例选用的数据集是MnistDataset_torch.zip 数据集结构是： MnistDataset_torch.zip ├── test └── train 预训练模型文件夹结构是： Torch_MNIST_Example_Model ├── mnist_epoch1.pkl ''' from model import Model import numpy as np import torch from torchvision.datasets import mnist from torch.nn import CrossEntropyLoss from torch.optim import SGD from torch.utils.data import DataLoader from torchvision.transforms import ToTensor import argparse import os #导入c2net包 from c2net.context import prepare, upload_output # Training settings parser = argparse.ArgumentParser(description='PyTorch MNIST Example') parser.add_argument('--epoch_size', type=int, default=10, help='how much epoch to train') parser.add_argument('--batch_size', type=int, default=256, help='how much batch_size in epoch') # 参数声明 WORKERS = 0 device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") model = Model().to(device) optimizer = SGD(model.parameters(), lr=1e-1) cost = CrossEntropyLoss() # 模型测试 def test(model, test_loader, data_length, output_path): model.eval() test_loss = 0 correct = 0 with torch.no_grad(): for i, data in enumerate(test_loader, 0): x, y = data x = x.to(device) y = y.to(device) y_hat = model(x) test_loss += cost(y_hat, y).item() pred = y_hat.max(1, keepdim=True)[1] correct += pred.eq(y.view_as(pred)).sum().item() test_loss /= (i+1) # 结果写入输出文件夹 filename = 'result.txt' file_path = os.path.join(output_path, filename) with open(file_path, 'w') as file: file.write('Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format( test_loss, correct, data_length, 100. * correct / data_length)) if __name__ == '__main__': args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_torch_path = c2net_context.dataset_path+"/"+"MnistDataset_torch" #获取预训练模型路径 Torch_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Torch_MNIST_Example_Model" #获取输出路径 output_path = c2net_context.output_path print('cuda is available:{}'.format(torch.cuda.is_available())) device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") batch_size = args.batch_size epochs = args.epoch_size test_dataset = mnist.MNIST(root=MnistDataset_torch_path + "/test", train=False, transform=ToTensor(),download=False) test_loader = DataLoader(test_dataset, batch_size=batch_size) model = Model().to(device) checkpoint = torch.load(Torch_MNIST_Example_Model_path + "/mnist_epoch1.pkl") model.load_state_dict(checkpoint['model']) test(model,test_loader,len(test_dataset),output_path) upload_output()openi_cloudbrain_example/gpgpu_mnist_example/model.py000066400000000000000000000016571463674231500236520ustar00rootroot00000000000000from torch.nn import Module from torch import nn class Model(Module): def __init__(self): super(Model, self).__init__() self.conv1 = nn.Conv2d(1, 6, 5) self.relu1 = nn.ReLU() self.pool1 = nn.MaxPool2d(2) self.conv2 = nn.Conv2d(6, 16, 5) self.relu2 = nn.ReLU() self.pool2 = nn.MaxPool2d(2) self.fc1 = nn.Linear(256, 120) self.relu3 = nn.ReLU() self.fc2 = nn.Linear(120, 84) self.relu4 = nn.ReLU() self.fc3 = nn.Linear(84, 10) self.relu5 = nn.ReLU() def forward(self, x): y = self.conv1(x) y = self.relu1(y) y = self.pool1(y) y = self.conv2(y) y = self.relu2(y) y = self.pool2(y) y = y.view(y.shape[0], -1) y = self.fc1(y) y = self.relu3(y) y = self.fc2(y) y = self.relu4(y) y = self.fc3(y) y = self.relu5(y) return y openi_cloudbrain_example/gpgpu_mnist_example/train.py000066400000000000000000000103271463674231500236610ustar00rootroot00000000000000#!/usr/bin/python #coding=utf-8 ''' If there are Chinese comments in the code，please add at the beginning： #!/usr/bin/python #coding=utf-8 数据集结构是： MnistDataset_torch.zip ├── test └── train 预训练模型文件夹结构是： Torch_MNIST_Example_Model ├── mnist_epoch1.pkl ''' from model import Model import numpy as np import torch from torchvision.datasets import mnist from torch.nn import CrossEntropyLoss from torch.optim import SGD from torch.utils.data import DataLoader from torchvision.transforms import ToTensor import argparse import os #导入c2net包 from c2net.context import prepare # Training settings parser = argparse.ArgumentParser(description='PyTorch MNIST Example') parser.add_argument('--epoch_size', type=int, default=10, help='how much epoch to train') parser.add_argument('--batch_size', type=int, default=256, help='how much batch_size in epoch') # 参数声明 WORKERS = 0 # dataloder线程数 device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") model = Model().to(device) optimizer = SGD(model.parameters(), lr=1e-1) cost = CrossEntropyLoss() # 模型训练 def train(model, train_loader, epoch): model.train() train_loss = 0 for i, data in enumerate(train_loader, 0): x, y = data x = x.to(device) y = y.to(device) optimizer.zero_grad() y_hat = model(x) loss = cost(y_hat, y) loss.backward() optimizer.step() train_loss += loss loss_mean = train_loss / (i+1) print('Train Epoch: {}\t Loss: {:.6f}'.format(epoch, loss_mean.item())) # 模型测试 def test(model, test_loader, test_data): model.eval() test_loss = 0 correct = 0 with torch.no_grad(): for i, data in enumerate(test_loader, 0): x, y = data x = x.to(device) y = y.to(device) optimizer.zero_grad() y_hat = model(x) test_loss += cost(y_hat, y).item() pred = y_hat.max(1, keepdim=True)[1] correct += pred.eq(y.view_as(pred)).sum().item() test_loss /= (i+1) print('Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format( test_loss, correct, len(test_data), 100. * correct / len(test_data))) if __name__ == '__main__': args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_torch_path = c2net_context.dataset_path+"/"+"MnistDataset_torch" #获取预训练模型路径 Torch_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Torch_MNIST_Example_Model" #log output print('cuda is available:{}'.format(torch.cuda.is_available())) device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") batch_size = args.batch_size epochs = args.epoch_size train_dataset = mnist.MNIST(root=os.path.join(MnistDataset_torch_path, "train"), train=True, transform=ToTensor(),download=False) test_dataset = mnist.MNIST(root=os.path.join(MnistDataset_torch_path, "test"), train=False, transform=ToTensor(),download=False) train_loader = DataLoader(train_dataset, batch_size=batch_size) test_loader = DataLoader(test_dataset, batch_size=batch_size) #如果有保存的模型，则加载模型，并在其基础上继续训练 if os.path.exists(os.path.join(Torch_MNIST_Example_Model_path, "mnist_epoch1.pkl")): checkpoint = torch.load(os.path.join(Torch_MNIST_Example_Model_path, "mnist_epoch1.pkl")) model.load_state_dict(checkpoint['model']) optimizer.load_state_dict(checkpoint['optimizer']) start_epoch = checkpoint['epoch'] print('加载 epoch {} 权重成功！'.format(start_epoch)) else: start_epoch = 0 print('无保存模型，将从头开始训练！') for epoch in range(start_epoch+1, epochs+1): train(model, train_loader, epoch) test(model, test_loader, test_dataset) # 将模型保存到c2net_context.output_path state = {'model':model.state_dict(), 'optimizer':optimizer.state_dict(), 'epoch':epoch} torch.save(state, '{}/mnist_epoch{}.pkl'.format(c2net_context.output_path, epoch)) openi_cloudbrain_example/gpu_mnist_example/000077500000000000000000000000001463674231500216405ustar00rootroot00000000000000openi_cloudbrain_example/gpu_mnist_example/README.md000066400000000000000000000120751463674231500231240ustar00rootroot00000000000000# 如何在启智平台上进行模型调试和训练—GPU_手写数字识别示例 ## 一 ，数据集及预训练模型准备 ##### 1，数据集说明： 数据集可从本项目的数据集中引用，[数据集引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/datasets) - MnistDataset_torch数据集是由10类28∗28的灰度图片组成，训练数据集包含60000张图片，测试数据集包含10000张图片。 - MnistDataset_torch.zip数据集压缩包的目录结构如下： > MnistDataset_torch.zip > > ├── test > > └── train > ##### 2，预训练模型说明： Torch_MNIST_Example_Model模型可从本项目的模型目录中引用，[预训练模型引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/modelmanage/show_model) * Torch_MNIST_Example_Model模型的目录结构如下： > Torch_MNIST_Example_Model > ├── mnist_epoch1.pkl > ## 二. 如何在云脑上获取代码路径，数据集路径，预训练模型路径，输出路径 ##### 1，准备数据 ``` from c2net.context import prepare c2net_context = prepare() ``` ##### 2，获取代码路径 ``` code_path = c2net_context.code_path +"/" +"项目名".lower() 在本示例中代码路径为： code_path = c2net_context.code_path + "/" + "Openl_Cloudbrain_Example".lower() ``` ##### 3，获取数据集路径 ``` dataset_path = c2net_context.dataset_path +"/" +"数据集名称" 在本示例中代码路径为： dataset_path = c2net_context.dataset_path + "/" + "MnistDataset_torch" ``` ##### 4，获取预训练模型路径 ``` pretrain_model_path = c2net_context.pretrain_model_path +"/" +"模型名称" 在本示例中预训练模型路径为： pretrain_model_path = c2net_context.pretrain_model_path + "/" + "Torch_MNIST_Example_Model" ``` ##### 5，获取输出路径 ``` output_path = c2net_context.output_path 在本示例中输出路径为： output_path = c2net_context.output_path ``` ##### 6，回传结果 ``` from c2net.context import upload_output upload_output() 在本示例中回传结果为（只有训练任务才能回传结果）： from c2net.context import prepare,upload_output upload_output() ``` ## 三.GPU样例准备 ##### 1，gpu示例代码： - 训练任务单卡示例请参考示例中[train.py](./train.py)的代码注释 - 推理任务示例请参考示例中[inference.py](./inference.py)的代码注释 ##### 2，创建GPU调试任务 表1创建训练作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择含有python和torch的镜像 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择gpu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_torch.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[GPU] | | 模型 | 模型选择Torch_MNIST_Example_Model | 启动调试任务后，先执行prepare()进行数据准备； 进入对应的代码目录后，可在终端执行python train.py； ##### 3，创建GPU训练任务 表2 创建训练作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择含有python和torch的镜像 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择gpu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_torch.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[GPU] | | 模型 | 模型选择Torch_MNIST_Example_Model | 启动训练任务后，训练结束会在任务的结果下载页提供输出结果下载 ## 四.GPU任务注意事项 ##### 1, 使用超参数的方法 请在代码中加入 ``` import parser args, unknown = parser.parse_known_args() #可忽略掉 `--ckpt_url`，`--data_url`, `--multi_date_url`等参数无定义导致的报错问题 ``` ## 对于示例代码有任何问题，欢迎在本项目中提issue。 openi_cloudbrain_example/gpu_mnist_example/inference.py000066400000000000000000000060221463674231500241500ustar00rootroot00000000000000#!/usr/bin/python #coding=utf-8 ''' If there are Chinese comments in the code，please add at the beginning： #!/usr/bin/python #coding=utf-8 示例选用的数据集是MnistDataset_torch.zip 数据集结构是： MnistDataset_torch.zip ├── test └── train 预训练模型文件夹结构是： Torch_MNIST_Example_Model ├── mnist_epoch1.pkl ''' from model import Model import numpy as np import torch from torchvision.datasets import mnist from torch.nn import CrossEntropyLoss from torch.optim import SGD from torch.utils.data import DataLoader from torchvision.transforms import ToTensor import argparse import os #导入c2net包 from c2net.context import prepare # Training settings parser = argparse.ArgumentParser(description='PyTorch MNIST Example') parser.add_argument('--epoch_size', type=int, default=10, help='how much epoch to train') parser.add_argument('--batch_size', type=int, default=256, help='how much batch_size in epoch') # 参数声明 WORKERS = 0 device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") model = Model().to(device) optimizer = SGD(model.parameters(), lr=1e-1) cost = CrossEntropyLoss() # 模型测试 def test(model, test_loader, data_length): model.eval() test_loss = 0 correct = 0 with torch.no_grad(): for i, data in enumerate(test_loader, 0): x, y = data x = x.to(device) y = y.to(device) y_hat = model(x) test_loss += cost(y_hat, y).item() pred = y_hat.max(1, keepdim=True)[1] correct += pred.eq(y.view_as(pred)).sum().item() test_loss /= (i+1) # 结果写入输出文件夹 print('accuracy: {:.2f}'.format(correct / data_length)) filename = 'result.txt' file_path = os.path.join('/tmp/output', filename) with open(file_path, 'w') as file: file.write('Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format( test_loss, correct, data_length, 100. * correct / data_length)) if __name__ == '__main__': args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_torch_path = c2net_context.dataset_path+"/"+"MnistDataset_torch" #获取预训练模型路径 Torch_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Torch_MNIST_Example_Model" #log output print('cuda is available:{}'.format(torch.cuda.is_available())) device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") batch_size = args.batch_size epochs = args.epoch_size test_dataset = mnist.MNIST(root=MnistDataset_torch_path + "/test", train=False, transform=ToTensor(),download=False) test_loader = DataLoader(test_dataset, batch_size=batch_size) model = Model().to(device) checkpoint = torch.load(Torch_MNIST_Example_Model_path + "/mnist_epoch1.pkl") model.load_state_dict(checkpoint['model']) test(model,test_loader,len(test_dataset))openi_cloudbrain_example/gpu_mnist_example/model.py000066400000000000000000000016571463674231500233230ustar00rootroot00000000000000from torch.nn import Module from torch import nn class Model(Module): def __init__(self): super(Model, self).__init__() self.conv1 = nn.Conv2d(1, 6, 5) self.relu1 = nn.ReLU() self.pool1 = nn.MaxPool2d(2) self.conv2 = nn.Conv2d(6, 16, 5) self.relu2 = nn.ReLU() self.pool2 = nn.MaxPool2d(2) self.fc1 = nn.Linear(256, 120) self.relu3 = nn.ReLU() self.fc2 = nn.Linear(120, 84) self.relu4 = nn.ReLU() self.fc3 = nn.Linear(84, 10) self.relu5 = nn.ReLU() def forward(self, x): y = self.conv1(x) y = self.relu1(y) y = self.pool1(y) y = self.conv2(y) y = self.relu2(y) y = self.pool2(y) y = y.view(y.shape[0], -1) y = self.fc1(y) y = self.relu3(y) y = self.fc2(y) y = self.relu4(y) y = self.fc3(y) y = self.relu5(y) return y openi_cloudbrain_example/gpu_mnist_example/train.py000066400000000000000000000103271463674231500233320ustar00rootroot00000000000000#!/usr/bin/python #coding=utf-8 ''' If there are Chinese comments in the code，please add at the beginning： #!/usr/bin/python #coding=utf-8 数据集结构是： MnistDataset_torch.zip ├── test └── train 预训练模型文件夹结构是： Torch_MNIST_Example_Model ├── mnist_epoch1.pkl ''' from model import Model import numpy as np import torch from torchvision.datasets import mnist from torch.nn import CrossEntropyLoss from torch.optim import SGD from torch.utils.data import DataLoader from torchvision.transforms import ToTensor import argparse import os #导入c2net包 from c2net.context import prepare # Training settings parser = argparse.ArgumentParser(description='PyTorch MNIST Example') parser.add_argument('--epoch_size', type=int, default=10, help='how much epoch to train') parser.add_argument('--batch_size', type=int, default=256, help='how much batch_size in epoch') # 参数声明 WORKERS = 0 # dataloder线程数 device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") model = Model().to(device) optimizer = SGD(model.parameters(), lr=1e-1) cost = CrossEntropyLoss() # 模型训练 def train(model, train_loader, epoch): model.train() train_loss = 0 for i, data in enumerate(train_loader, 0): x, y = data x = x.to(device) y = y.to(device) optimizer.zero_grad() y_hat = model(x) loss = cost(y_hat, y) loss.backward() optimizer.step() train_loss += loss loss_mean = train_loss / (i+1) print('Train Epoch: {}\t Loss: {:.6f}'.format(epoch, loss_mean.item())) # 模型测试 def test(model, test_loader, test_data): model.eval() test_loss = 0 correct = 0 with torch.no_grad(): for i, data in enumerate(test_loader, 0): x, y = data x = x.to(device) y = y.to(device) optimizer.zero_grad() y_hat = model(x) test_loss += cost(y_hat, y).item() pred = y_hat.max(1, keepdim=True)[1] correct += pred.eq(y.view_as(pred)).sum().item() test_loss /= (i+1) print('Test set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format( test_loss, correct, len(test_data), 100. * correct / len(test_data))) if __name__ == '__main__': args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_torch_path = c2net_context.dataset_path+"/"+"MnistDataset_torch" #获取预训练模型路径 Torch_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Torch_MNIST_Example_Model" #log output print('cuda is available:{}'.format(torch.cuda.is_available())) device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") batch_size = args.batch_size epochs = args.epoch_size train_dataset = mnist.MNIST(root=os.path.join(MnistDataset_torch_path, "train"), train=True, transform=ToTensor(),download=False) test_dataset = mnist.MNIST(root=os.path.join(MnistDataset_torch_path, "test"), train=False, transform=ToTensor(),download=False) train_loader = DataLoader(train_dataset, batch_size=batch_size) test_loader = DataLoader(test_dataset, batch_size=batch_size) #如果有保存的模型，则加载模型，并在其基础上继续训练 if os.path.exists(os.path.join(Torch_MNIST_Example_Model_path, "mnist_epoch1.pkl")): checkpoint = torch.load(os.path.join(Torch_MNIST_Example_Model_path, "mnist_epoch1.pkl")) model.load_state_dict(checkpoint['model']) optimizer.load_state_dict(checkpoint['optimizer']) start_epoch = checkpoint['epoch'] print('加载 epoch {} 权重成功！'.format(start_epoch)) else: start_epoch = 0 print('无保存模型，将从头开始训练！') for epoch in range(start_epoch+1, epochs+1): train(model, train_loader, epoch) test(model, test_loader, test_dataset) # 将模型保存到c2net_context.output_path state = {'model':model.state_dict(), 'optimizer':optimizer.state_dict(), 'epoch':epoch} torch.save(state, '{}/mnist_epoch{}.pkl'.format(c2net_context.output_path, epoch)) openi_cloudbrain_example/npu_mnist_example/000077500000000000000000000000001463674231500216475ustar00rootroot00000000000000openi_cloudbrain_example/npu_mnist_example/README.md000066400000000000000000000157251463674231500231400ustar00rootroot00000000000000# 如何在启智平台上进行模型调试和训练—NPU_手写数字识别示例 ## 一 ，数据集及预训练模型准备 ##### 1，数据集说明： 数据集可从本项目的数据集中引用，[数据集引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/datasets) - MnistDataset_mindspore数据集是由10类28∗28的灰度图片组成，训练数据集包含60000张图片，测试数据集包含10000张图片。 - MnistDataset_mindspore.zip数据集压缩包的目录结构如下： > MnistDataset_mindspore.zip > ├── test > │ ├── t10k-images-idx3-ubyte > │ └── t10k-labels-idx1-ubyte > └── train > ├── train-images-idx3-ubyte > └── train-labels-idx1-ubyte > ##### 2，预训练模型说明： Mindspore_MNIST_Example_Model模型可从本项目的模型目录中引用，[预训练模型引用](https://openi.pcl.ac.cn/OpenIOSSG/OpenI_Cloudbrain_Example/modelmanage/show_model) * Mindspore_MNIST_Example_Model模型的目录结构如下： > Mindspore_MNIST_Example_Model > ├── checkpoint_lenet-1_1875.ckpt > ## 二. 如何在云脑上获取代码路径，数据集路径，预训练模型路径，输出路径 ##### 1，准备数据 ``` from c2net.context import prepare c2net_context = prepare() ``` ##### 2，获取代码路径 ``` code_path = c2net_context.code_path +"/" + "项目名".lower() 在本示例中代码路径为： code_path = c2net_context.code_path + "/" + "Openl_Cloudbrain_Example".lower() ``` ##### 3，获取数据集路径 ``` dataset_path = c2net_context.dataset_path +"/" +"数据集名称" 在本示例中代码路径为： dataset_path = c2net_context.dataset_path + "/" + "MnistDataset_mindspore" ``` ##### 4，获取预训练模型路径 ``` pretrain_model_path = c2net_context.pretrain_model_path +"/" +"模型名称" 在本示例中预训练模型路径为： pretrain_model_path = c2net_context.pretrain_model_path + "/" + "Mindspore_MNIST_Example_Model" ``` ##### 5，获取输出路径 ``` output_path = c2net_context.output_path 在本示例中输出路径为： output_path = c2net_context.output_path ``` ##### 6，回传结果 ``` from c2net.context import upload_output upload_output() 在本示例中回传结果为（只有训练任务才能回传结果）： from c2net.context import prepare,upload_output upload_output() ``` ## 三.NPU样例准备 ##### 1，npu示例代码： - 训练任务单卡示例请参考示例中[train.py](./train.py)的代码注释 - 训练任务多卡示例请参考示例中[train_multi_card.py](./train_multi_card.py)的代码注释 - 训练任务在每个epoch结束后就上传文件，可参考[train_epoch_upload.py](./train_epoch_upload.py)的代码注释 - 推理任务示例请参考示例中[inference.py](./inference.py)的代码注释 - 启智集群训练任务已经将ImageNet-1k数据集挂载到训练镜像，具体使用方法请参考示例中[read_imagenet.py](./read_imagenet.py)的代码注释 - 继续训练示例参考示例中[train_continue.py](./train_continue.py)的代码注释 ##### 2，创建NPU调试任务 表1创建训练作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择mindspore_1.10.1 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择npu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_mindspore.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[Ascend: 1 * Ascend 910 CPU：24 核 256GiB]，表示单机单卡 | | 模型 | 模型可选择Mindspore_MNIST_Example_Model | 启动调试任务后，先执行prepare()进行数据准备； 进入对应的代码目录后，可在终端执行python train.py； ##### 3，创建NPU训练任务 表2 创建训练作业界面参数说明 | 参数名称 | 说明 | | -------- | ----------------------------------------------------------------------------------- | | 代码分支 | 选择仓库代码中要使用的代码分支，默认可选择master分支 | | 镜像 | 镜像选择mindspore_1.10.1 | | 启动文件 | 启动文件选择代码目录下的启动脚本，在本示例中选择npu_mnist_example/train.py | | 数据集 | 数据集选择MnistDataset_mindspore.zip | | 运行参数 | 选择增加运行参数可以向脚本中其他参数传值，如epoch_size,需要在代码里定义增加的超参数 | | 资源规格 | 规格选择[Ascend: 1 * Ascend 910 CPU：24 核 256GiB]，表示单机单卡 | | 模型 | 模型选择Mindspore_MNIST_Example_Model | 启动训练任务后，训练结束会在任务的结果下载页提供输出结果下载 ## 四.NPU任务注意事项 ##### 1, 使用超参数的方法： 请在代码中加入 ``` import parser args, unknown = parser.parse_known_args() #可忽略掉 `--ckpt_url`，`--data_url`, `--multi_date_url`等参数无定义导致的报错问题 ``` ##### 2, 多卡训练任务如何只让数据集只拷贝一次 使用缓存文件和local_rank%8作为判断，若第0卡拷贝完成，则其它卡不拷贝，并在第0卡拷贝数据完成后才执行之后的代码，具体可参考示例中[train_multi_card.py](./train_multi_card.py)的代码注释 ``` if local_rank%8==0: 这里省略下载数据的代码... f = open("/cache/download_input.txt", 'w') f.close() try: if os.path.exists("/cache/download_input.txt"): print("download_input succeed") except Exception as e: print("download_input failed") while not os.path.exists("/cache/download_input.txt"): time.sleep(1) ``` ##### 3, 公共库[c2net](https://openi.pcl.ac.cn/OpenIOSSG/c2net-pypi)高级用法： 主要使用的方法有以下几个： ``` from c2net.context import prepare, upload_output, moxing_helper prepare() 准备数据集，模型，输出路径 upload_output() 将训练镜像的输出结果拷贝回启智平台 moxing_helper.obs_copy_file(string,string) 通过mox拷贝文件 moxing_helper.obs_copy_folder(string,string) 通过mox拷贝文件夹 ``` ## 对于示例代码有任何问题，欢迎在本项目中提issue。 openi_cloudbrain_example/npu_mnist_example/config.py000066400000000000000000000021131463674231500234630ustar00rootroot00000000000000# Copyright 2020 Huawei Technologies Co., Ltd # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ============================================================================ """ network config setting, will be used in train.py """ import os os.system('pip install easydict') from easydict import EasyDict as edict mnist_cfg = edict({ 'num_classes': 10, 'lr': 0.01, 'momentum': 0.9, 'epoch_size': 10, 'batch_size': 32, 'buffer_size': 1000, 'image_height': 32, 'image_width': 32, 'save_checkpoint_steps': 1875, 'keep_checkpoint_max': 150, 'air_name': "lenet", }) openi_cloudbrain_example/npu_mnist_example/dataset.py000066400000000000000000000046741463674231500236610ustar00rootroot00000000000000# Copyright 2020 Huawei Technologies Co., Ltd # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ============================================================================ """ Produce the dataset """ import mindspore.dataset as ds import mindspore.dataset.vision.c_transforms as CV import mindspore.dataset.transforms.c_transforms as C from mindspore.dataset.vision import Inter from mindspore.common import dtype as mstype def create_dataset(data_path, batch_size=32, repeat_size=1, num_parallel_workers=1): """ create dataset for train or test """ # define dataset mnist_ds = ds.MnistDataset(data_path) resize_height, resize_width = 32, 32 rescale = 1.0 / 255.0 shift = 0.0 rescale_nml = 1 / 0.3081 shift_nml = -1 * 0.1307 / 0.3081 # define map operations resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR) # Bilinear mode rescale_nml_op = CV.Rescale(rescale_nml, shift_nml) rescale_op = CV.Rescale(rescale, shift) hwc2chw_op = CV.HWC2CHW() type_cast_op = C.TypeCast(mstype.int32) # apply map operations on images mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns="label", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=resize_op, input_columns="image", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=rescale_op, input_columns="image", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns="image", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns="image", num_parallel_workers=num_parallel_workers) # apply DatasetOps buffer_size = 10000 mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size) # 10000 as in LeNet train script mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True) mnist_ds = mnist_ds.repeat(repeat_size) return mnist_ds openi_cloudbrain_example/npu_mnist_example/dataset_distributed.py000066400000000000000000000045401463674231500262530ustar00rootroot00000000000000 """ Produce the dataset： 与单机不同的是，在数据集接口需要传入num_shards和shard_id参数，分别对应卡的数量和逻辑序号，建议通过HCCL接口获取： get_rank：获取当前设备在集群中的ID。 get_group_size：获取集群数量。 """ import mindspore.dataset as ds import mindspore.dataset.vision.c_transforms as CV import mindspore.dataset.transforms.c_transforms as C from mindspore.dataset.vision import Inter from mindspore.common import dtype as mstype from mindspore.communication.management import get_rank, get_group_size def create_dataset_parallel(data_path, batch_size=32, repeat_size=1, num_parallel_workers=1, shard_id=0, num_shards=8): """ create dataset for train or test """ resize_height, resize_width = 32, 32 rescale = 1.0 / 255.0 shift = 0.0 rescale_nml = 1 / 0.3081 shift_nml = -1 * 0.1307 / 0.3081 # get shard_id and num_shards.Get the ID of the current device in the cluster And Get the number of clusters. shard_id = get_rank() num_shards = get_group_size() # define dataset mnist_ds = ds.MnistDataset(data_path, num_shards=num_shards, shard_id=shard_id) # define map operations resize_op = CV.Resize((resize_height, resize_width), interpolation=Inter.LINEAR) # Bilinear mode rescale_nml_op = CV.Rescale(rescale_nml, shift_nml) rescale_op = CV.Rescale(rescale, shift) hwc2chw_op = CV.HWC2CHW() type_cast_op = C.TypeCast(mstype.int32) # apply map operations on images mnist_ds = mnist_ds.map(operations=type_cast_op, input_columns="label", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=resize_op, input_columns="image", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=rescale_op, input_columns="image", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=rescale_nml_op, input_columns="image", num_parallel_workers=num_parallel_workers) mnist_ds = mnist_ds.map(operations=hwc2chw_op, input_columns="image", num_parallel_workers=num_parallel_workers) # apply DatasetOps buffer_size = 10000 mnist_ds = mnist_ds.shuffle(buffer_size=buffer_size) # 10000 as in LeNet train script mnist_ds = mnist_ds.batch(batch_size, drop_remainder=True) mnist_ds = mnist_ds.repeat(repeat_size) return mnist_ds openi_cloudbrain_example/npu_mnist_example/inference.py000066400000000000000000000067611463674231500241710ustar00rootroot00000000000000 """ 示例选用的数据集是MnistDataset_mindspore.zip 数据集结构是： MnistDataset_mindspore.zip ├── test │ ├── t10k-images-idx3-ubyte │ └── t10k-labels-idx1-ubyte └── train ├── train-images-idx3-ubyte └── train-labels-idx1-ubyte 使用注意事项： 1、在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 2、用户需要调用c2net的python sdk包 """ import time import os import argparse from config import mnist_cfg as cfg from dataset import create_dataset from lenet import LeNet5 import mindspore.nn as nn import numpy as np from mindspore import context from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor from mindspore import load_checkpoint, load_param_into_net from mindspore.train import Model from mindspore import Tensor #导入c2net包 from c2net.context import prepare, upload_output parser = argparse.ArgumentParser(description='MindSpore Lenet Example') parser.add_argument( '--device_target', type=str, default="Ascend", choices=['Ascend', 'CPU'], help='device where the code will be implemented (default: Ascend),if to use the CPU on the Qizhi platform:device_target=CPU') parser.add_argument('--epoch_size', type=int, default=5, help='Training epochs.') if __name__ == "__main__": ###请在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_mindspore_path = c2net_context.dataset_path+"/"+"MnistDataset_mindspore" #获取预训练模型路径 Mindspore_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Mindspore_MNIST_Example_Model" #获取输出路径 save_path = c2net_context.output_path context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target) network = LeNet5(cfg.num_classes) net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") repeat_size = cfg.epoch_size net_opt = nn.Momentum(network.trainable_params(), cfg.lr, cfg.momentum) #model = Model(network, net_loss, net_opt, metrics={"Accuracy"}) model = Model(network, net_loss, net_opt) print("============== Starting Testing ==============") load_param_into_net(network, load_checkpoint(os.path.join(Mindspore_MNIST_Example_Model_path, "checkpoint_lenet-1_1875.ckpt"))) ds_test = create_dataset(os.path.join(MnistDataset_mindspore_path, "test"), batch_size=1).create_dict_iterator() data = next(ds_test) images = data["image"].asnumpy() labels = data["label"].asnumpy() print('Tensor:', Tensor(data['image'])) output = model.predict(Tensor(data['image'])) predicted = np.argmax(output.asnumpy(), axis=1) pred = np.argmax(output.asnumpy(), axis=1) print('predicted:', predicted) print('pred:', pred) print(f'Predicted: "{predicted[0]}", Actual: "{labels[0]}"') filename = 'result.txt' file_path = os.path.join(save_path, filename) with open(file_path, 'a+') as file: file.write(" {}: {:.2f} \n".format("Predicted", predicted[0])) ###上传训练结果到启智平台,注意必须将要输出的模型存储在c2net_context.output_path upload_output()openi_cloudbrain_example/npu_mnist_example/lenet.py000066400000000000000000000040271463674231500233330ustar00rootroot00000000000000# Copyright 2020 Huawei Technologies Co., Ltd # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ============================================================================ """LeNet.""" import mindspore.nn as nn from mindspore.common.initializer import Normal class LeNet5(nn.Cell): """ Lenet network Args: num_class (int): Number of classes. Default: 10. num_channel (int): Number of channels. Default: 1. Returns: Tensor, output tensor Examples: >>> LeNet(num_class=10) """ def __init__(self, num_class=10, num_channel=1, include_top=True): super(LeNet5, self).__init__() self.conv1 = nn.Conv2d(num_channel, 6, 5, pad_mode='valid') self.conv2 = nn.Conv2d(6, 16, 5, pad_mode='valid') self.relu = nn.ReLU() self.max_pool2d = nn.MaxPool2d(kernel_size=2, stride=2) self.include_top = include_top if self.include_top: self.flatten = nn.Flatten() self.fc1 = nn.Dense(16 * 5 * 5, 120, weight_init=Normal(0.02)) self.fc2 = nn.Dense(120, 84, weight_init=Normal(0.02)) self.fc3 = nn.Dense(84, num_class, weight_init=Normal(0.02)) def construct(self, x): x = self.conv1(x) x = self.relu(x) x = self.max_pool2d(x) x = self.conv2(x) x = self.relu(x) x = self.max_pool2d(x) if not self.include_top: return x x = self.flatten(x) x = self.relu(self.fc1(x)) x = self.relu(self.fc2(x)) x = self.fc3(x) return x openi_cloudbrain_example/npu_mnist_example/read_imagenet.py000066400000000000000000000050361463674231500250110ustar00rootroot00000000000000''' imagnet-1k 数据集已通过磁盘挂载的方式挂载在训练镜像中， 用户可参考下列示例代码读取直接使用。 挂载路径为 . └── cache/ ├── ascend ├── outputs ├── user-job-dir └── sfs/ └── data/ └── imagenet/ ├── train/ │ └── n01440764/ │ └── n01440764_11063.JPEG └── val/ └── n01440764/ └── ILSVRC2012_val_00011993.JPEG mindspore.dataset.ImageFolderDataset - 读取imagenet-1k数据，同一文件夹下的数据为同一类class。 mindspore.dataset.vision.c_transforms - 数据加载和预处理。 mindspore.dataset.ImageFolderDataset - map：给定一组数据增强列表，按顺序将数据增强作用在数据集对象上。 - batch：将数据集中连续 batch_size 条数据合并为一个批处理数据。 - to_json：将数据处理管道序列化为JSON字符串，如果提供了文件名，则转储到文件中。 ''' import os import argparse import moxing as mox import mindspore as ms from mindspore.dataset import ImageFolderDataset import mindspore.dataset.vision.c_transforms as transforms from c2net.context import upload_output parser = argparse.ArgumentParser(description='Read big dataset ImageNet Example') if __name__ == "__main__": args, unknown = parser.parse_known_args() #注意只有训练任务可用 data_path = '/cache/sfs/data/imagenet/' modelart_output = '/cache/output' if not os.path.exists(modelart_output): os.makedirs(modelart_output) mean = [0.485 * 255, 0.456 * 255, 0.406 * 255] std = [0.229 * 255, 0.224 * 255, 0.225 * 255] dataset_train = ImageFolderDataset(os.path.join(data_path, "train"), shuffle=True) trans_train = [ transforms.RandomCropDecodeResize(size=224, scale=(0.08, 1.0), ratio=(0.75, 1.333)), transforms.RandomHorizontalFlip(prob=0.5), transforms.Normalize(mean=mean, std=std), transforms.HWC2CHW() ] dataset_train = dataset_train.map(operations=trans_train, input_columns=["image"]) dataset_train = dataset_train.batch(batch_size=16, drop_remainder=True) data_info = dataset_train.to_json(filename= modelart_output + '/data_info.json') print(data_info) upload_output()openi_cloudbrain_example/npu_mnist_example/train.py000066400000000000000000000101321463674231500233330ustar00rootroot00000000000000 """ 示例选用的数据集是MnistDataset_mindspore.zip 数据集结构是： MnistDataset_mindspore.zip ├── test │ ├── t10k-images-idx3-ubyte │ └── t10k-labels-idx1-ubyte └── train ├── train-images-idx3-ubyte └── train-labels-idx1-ubyte 模型文件夹结构是： Mindspore_MNIST_Example_Model ├── checkpoint_lenet-1_1875.ckpt 使用注意事项： 1、在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 2、用户需要调用c2net的python sdk包 """ import os import argparse from config import mnist_cfg as cfg from dataset import create_dataset from lenet import LeNet5 import mindspore.nn as nn from mindspore import context from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor from mindspore import load_checkpoint, load_param_into_net from mindspore.train import Model import time #导入c2net包 from c2net.context import prepare, upload_output parser = argparse.ArgumentParser(description='MindSpore Lenet Example') parser.add_argument( '--device_target', type=str, default="Ascend", choices=['Ascend', 'CPU'], help='device where the code will be implemented (default: Ascend),if to use the CPU on the Qizhi platform:device_target=CPU') parser.add_argument('--epoch_size', type=int, default=5, help='Training epochs.') if __name__ == "__main__": ###请在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_mindspore_path = c2net_context.dataset_path+"/"+"MnistDataset_mindspore" #获取预训练模型路径 Mindspore_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Mindspore_MNIST_Example_Model" #获取输出路径 output_path = c2net_context.output_path context.set_context(mode=context.GRAPH_MODE,device_target=args.device_target) #使用数据集的方式 ds_train = create_dataset(os.path.join(MnistDataset_mindspore_path, "train"), cfg.batch_size) network = LeNet5(cfg.num_classes) net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") net_opt = nn.Momentum(network.trainable_params(), cfg.lr, cfg.momentum) time_cb = TimeMonitor(data_size=ds_train.get_dataset_size()) # load_param_into_net(network, load_checkpoint(os.path.join(Mindspore_MNIST_Example_Model_path, "checkpoint_lenet-1_1875.ckpt"))) if os.path.exists(os.path.join(Mindspore_MNIST_Example_Model_path, "checkpoint_lenet-1_1875.ckpt")): load_param_into_net(network, load_checkpoint(os.path.join(Mindspore_MNIST_Example_Model_path, "checkpoint_lenet-1_1875.ckpt"))) if args.device_target != "Ascend": model = Model(network, net_loss, net_opt, metrics={"accuracy"}) else: model = Model(network, net_loss, net_opt, metrics={"accuracy"}, amp_level="O2") config_ck = CheckpointConfig( save_checkpoint_steps=cfg.save_checkpoint_steps, keep_checkpoint_max=cfg.keep_checkpoint_max) #将模型保存到c2net_context.output_path outputDirectory = output_path + "/" ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", directory=outputDirectory, config=config_ck) print("============== Starting Training ==============") epoch_size = cfg['epoch_size'] if (args.epoch_size): epoch_size = args.epoch_size print('epoch_size is: ', epoch_size) model.train(epoch_size, ds_train,callbacks=[time_cb, ckpoint_cb,LossMonitor()]) ###上传训练结果到启智平台,注意必须将要输出的模型存储在c2net_context.output_path upload_output()openi_cloudbrain_example/npu_mnist_example/train_continue.py000066400000000000000000000130441463674231500252440ustar00rootroot00000000000000##################################################################################################### # 继续训练功能：修改训练任务时，若勾选复用上次结果，则可在新训练任务的输出路径中读取到上次结果 # # 示例用法 # - 增加两个训练参数 # 'ckpt_save_name' 此次任务的输出文件名，用于保存此次训练的模型文件名称（不带后缀） # 'ckpt_load_name' 上一次任务的输出文件名，用于加载上一次输出的模型文件名称（不带后缀），首次训练默认为空，则不读取任何文件 # - 训练代码中判断 'ckpt_load_name' 是否为空，若不为空，则为继续训练任务 ##################################################################################################### import os import argparse from config import mnist_cfg as cfg from dataset import create_dataset from lenet import LeNet5 import mindspore.nn as nn from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor from mindspore import load_checkpoint, load_param_into_net from mindspore.train import Model from mindspore.nn.metrics import Accuracy from mindspore.communication.management import get_rank #导入c2net包 from c2net.context import prepare from c2net.context.moxing_helper import obs_copy_file, obs_copy_folder parser = argparse.ArgumentParser(description='MindSpore Lenet Example') parser.add_argument('--train_url', help='output folder to save/load', default= '') parser.add_argument( '--device_target', type=str, default="Ascend", choices=['Ascend', 'CPU'], help='device where the code will be implemented (default: Ascend),if to use the CPU on the Qizhi platform:device_target=CPU') parser.add_argument('--epoch_size', type=int, default=5, help='Training epochs.') ### continue task parameters parser.add_argument('--ckpt_load_name', help='model name to save/load', default= '') parser.add_argument('--ckpt_save_name', help='model name to save/load', default= 'checkpoint') if __name__ == "__main__": ###请在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_mindspore_path = c2net_context.dataset_path+"/"+"MnistDataset_mindspore" data_dir = '/cache/data' base_path = '/cache/output' try: if not os.path.exists(data_dir): os.makedirs(data_dir) if not os.path.exists(base_path): os.makedirs(base_path) except Exception as e: print("path already exists") device_num = int(os.getenv('RANK_SIZE')) if device_num == 1: ds_train = create_dataset(os.path.join(MnistDataset_mindspore_path, "train"), cfg.batch_size) if ds_train.get_dataset_size() == 0: raise ValueError("Please check dataset size > 0 and batch_size <= dataset size") network = LeNet5(cfg.num_classes) net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") net_opt = nn.Momentum(network.trainable_params(), cfg.lr, cfg.momentum) time_cb = TimeMonitor(data_size=ds_train.get_dataset_size()) ### 继续训练模型加载 if args.ckpt_load_name: obs_copy_folder(args.train_url, base_path) load_path = "{}/{}.ckpt".format(base_path,args.ckpt_load_name) param_dict = load_checkpoint(load_path) load_param_into_net(network, param_dict) print("Successfully load ckpt file:{}, saved_net_work:{}".format(load_path,param_dict)) ### 保存已有模型名避免重复回传结果 outputFiles = os.listdir(base_path) if args.device_target != "Ascend": model = Model(network, net_loss, net_opt, metrics={"accuracy": Accuracy()}) else: model = Model(network, net_loss, net_opt, metrics={"accuracy": Accuracy()}, amp_level="O2") config_ck = CheckpointConfig( save_checkpoint_steps=cfg.save_checkpoint_steps, keep_checkpoint_max=cfg.keep_checkpoint_max) #Note that this method saves the model file on each card. You need to specify the save path on each card. # In this example, get_rank() is added to distinguish different paths. if device_num == 1: save_path = base_path + "/" if device_num > 1: save_path = base_path + "/" + str(get_rank()) + "/" ckpoint_cb = ModelCheckpoint(prefix=args.ckpt_save_name, directory=save_path, config=config_ck) print("============== Starting Training ==============") epoch_size = cfg['epoch_size'] if (args.epoch_size): epoch_size = args.epoch_size print('epoch_size is: ', epoch_size) model.train(epoch_size, ds_train, callbacks=[time_cb, ckpoint_cb, LossMonitor()]) ### 将训练容器中的新输出模型 回传到启智社区 outputFilesNew = os.listdir(base_path) new_models = [i for i in outputFilesNew if i not in outputFiles] for n in new_models: ckpt_url = base_path + "/" + n obs_ckpt_url = args.train_url + "/" + n obs_copy_file(ckpt_url, obs_ckpt_url)openi_cloudbrain_example/npu_mnist_example/train_epoch_upload.py000066400000000000000000000105571463674231500260700ustar00rootroot00000000000000 """ 示例选用的数据集是MnistDataset_mindspore.zip 数据集结构是： MnistDataset_mindspore.zip ├── test │ ├── t10k-images-idx3-ubyte │ └── t10k-labels-idx1-ubyte └── train ├── train-images-idx3-ubyte └── train-labels-idx1-ubyte 模型文件夹结构是： Mindspore_MNIST_Example_Model ├── checkpoint_lenet-1_1875.ckpt 使用注意事项： 1、在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 2、用户需要调用c2net的python sdk包 """ import os import argparse from config import mnist_cfg as cfg from dataset import create_dataset from lenet import LeNet5 import mindspore.nn as nn from mindspore import context from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor from mindspore import load_checkpoint, load_param_into_net from mindspore.train import Model from mindspore.train.callback import Callback #导入c2net包 from c2net.context import prepare, upload_output class EnvToOpenIEpochEnd(Callback): """ upload output to openi when epoch end """ def epoch_end(self,run_context): upload_output() parser = argparse.ArgumentParser(description='MindSpore Lenet Example') parser.add_argument( '--device_target', type=str, default="Ascend", choices=['Ascend', 'CPU'], help='device where the code will be implemented (default: Ascend),if to use the CPU on the Qizhi platform:device_target=CPU') parser.add_argument('--epoch_size', type=int, default=5, help='Training epochs.') if __name__ == "__main__": ###请在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 args, unknown = parser.parse_known_args() #初始化导入数据集和预训练模型到容器内 c2net_context = prepare() #获取数据集路径 MnistDataset_mindspore_path = c2net_context.dataset_path+"/"+"MnistDataset_mindspore" #获取预训练模型路径 Mindspore_MNIST_Example_Model_path = c2net_context.pretrain_model_path+"/"+"Mindspore_MNIST_Example_Model" #获取输出路径 output_path = c2net_context.output_path context.set_context(mode=context.GRAPH_MODE,device_target=args.device_target) #使用数据集的方式 ds_train = create_dataset(os.path.join(MnistDataset_mindspore_path, "train"), cfg.batch_size) network = LeNet5(cfg.num_classes) net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") net_opt = nn.Momentum(network.trainable_params(), cfg.lr, cfg.momentum) time_cb = TimeMonitor(data_size=ds_train.get_dataset_size()) load_param_into_net(network, load_checkpoint(os.path.join(Mindspore_MNIST_Example_Model_path, "checkpoint_lenet-1_1875.ckpt"))) if args.device_target != "Ascend": model = Model(network, net_loss, net_opt, metrics={"accuracy"}) else: model = Model(network, net_loss, net_opt, metrics={"accuracy"}, amp_level="O2") config_ck = CheckpointConfig( save_checkpoint_steps=cfg.save_checkpoint_steps, keep_checkpoint_max=cfg.keep_checkpoint_max) #将模型保存到c2net_context.output_path outputDirectory = output_path + "/" ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", directory=outputDirectory, config=config_ck) print("============== Starting Training ==============") epoch_size = cfg['epoch_size'] if (args.epoch_size): epoch_size = args.epoch_size print('epoch_size is: ', epoch_size) # set callback functions callback =[time_cb,LossMonitor()] local_rank=int(os.getenv('RANK_ID')) #非必选，每个epoch结束后，都手动上传训练结果到启智平台，注意这样使用会占用很多内存，只有在部分特殊需要手动上传的任务才需要使用 uploadOutput = EnvToOpenIEpochEnd() callback.append(uploadOutput) # for data parallel, only save checkpoint on rank 0 if local_rank==0 : callback.append(ckpoint_cb) model.train(epoch_size,ds_train,callbacks=callback) openi_cloudbrain_example/npu_mnist_example/train_multi_card.py000066400000000000000000000112111463674231500255350ustar00rootroot00000000000000 """ 示例选用的数据集是MnistDataset_mindspore.zip 数据集结构是： MnistDataset_mindspore.zip ├── test │ ├── t10k-images-idx3-ubyte │ └── t10k-labels-idx1-ubyte └── train ├── train-images-idx3-ubyte └── train-labels-idx1-ubyte 使用注意事项： 1、在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 2、用户需要调用c2net的python sdk包 """ import os import argparse from config import mnist_cfg as cfg from dataset_distributed import create_dataset_parallel from lenet import LeNet5 import mindspore.nn as nn from mindspore import context from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor from mindspore.train import Model from mindspore.context import ParallelMode from mindspore.communication.management import init, get_rank import time from c2net.context import prepare, upload_output parser = argparse.ArgumentParser(description='MindSpore Lenet Example') parser.add_argument( '--device_target', type=str, default="Ascend", choices=['Ascend', 'CPU'], help='device where the code will be implemented (default: Ascend),if to use the CPU on the Qizhi platform:device_target=CPU') parser.add_argument('--epoch_size', type=int, default=5, help='Training epochs.') if __name__ == "__main__": ###请在代码中加入args, unknown = parser.parse_known_args()，可忽略掉--ckpt_url参数报错等参数问题 args, unknown = parser.parse_known_args() device_num = int(os.getenv('RANK_SIZE')) #使用多卡时 # set device_id and init for multi-card training context.set_context(mode=context.GRAPH_MODE, device_target=args.device_target, device_id=int(os.getenv('ASCEND_DEVICE_ID'))) context.reset_auto_parallel_context() context.set_auto_parallel_context(device_num = device_num, parallel_mode=ParallelMode.DATA_PARALLEL, gradients_mean=True, parameter_broadcast=True) init() #Copying obs data does not need to be executed multiple times, just let the 0th card copy the data local_rank=int(os.getenv('RANK_ID')) #初始化导入数据集和预训练模型到容器内,并行任务先让0卡拷贝数据,并用一个缓存文件标记0卡已prepare完成 if local_rank == 0: c2net_context = prepare() f = open("/cache/prepare_completed.txt", 'w') f.close() try: if os.path.exists("/cache/prepare_completed.txt"): print("prepare completed!") except Exception as e: print("prepare failed") while not os.path.exists("/cache/prepare_completed.txt"): time.sleep(1) c2net_context = prepare() #获取数据集路径 MnistDataset_mindspore_path = c2net_context.dataset_path+"/"+"MnistDataset_mindspore" output_path = c2net_context.output_path ds_train = create_dataset_parallel(os.path.join(MnistDataset_mindspore_path, "train"), cfg.batch_size) network = LeNet5(cfg.num_classes) net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction="mean") net_opt = nn.Momentum(network.trainable_params(), cfg.lr, cfg.momentum) time_cb = TimeMonitor(data_size=ds_train.get_dataset_size()) if args.device_target != "Ascend": model = Model(network, net_loss, net_opt, metrics={"accuracy"}) else: model = Model(network, net_loss, net_opt, metrics={"accuracy"}, amp_level="O2") config_ck = CheckpointConfig( save_checkpoint_steps=cfg.save_checkpoint_steps, keep_checkpoint_max=cfg.keep_checkpoint_max) #Note that this method saves the model file on each card. You need to specify the save path on each card. # In this example, get_rank() is added to distinguish different paths. outputDirectory = output_path + "/" + str(get_rank()) + "/" ckpoint_cb = ModelCheckpoint(prefix="checkpoint_lenet", directory=outputDirectory, config=config_ck) print("============== Starting Training ==============") epoch_size = cfg['epoch_size'] if (args.epoch_size): epoch_size = args.epoch_size print('epoch_size is: ', epoch_size) model.train(epoch_size, ds_train,callbacks=[time_cb, ckpoint_cb,LossMonitor()]) ###上传训练结果到启智平台,注意必须将要输出的模型存储在c2net_context.output_path upload_output()