[MISC] Distill-DSM Model #630
Conversation
github-actions
bot
added
the
DEPENDENCY
|
Conversion of the model from ONNX to OpenVINO IR fails with the following error.
What could be the possible reasons for this? |
Are there any additional details available? Could you, please, provide the full log from the model conversion command? |
Here is the full log.
The input shape is the same as the one I had used to train the .pth model and also to convert it into ONNX. |
|
You can try two options:
|
|
Also, you can try to move return statement in the model |
You are correct. The model got converted without errors but is unable to infer. |
druzhkov-paul
added
the
MISC
|
Hi, @Rakshith2597! Can you please check if I try to reproduce model conversion correctly? net = U_Net(1, 2, conv_type='conv_2d', tsm=True, learn=True)
net.eval()
dummy_inp = torch.randn([1, 1, 128, 160, 160])
torch.onnx.export(net, dummy_inp, "model.onnx", opset_version=11)with I've found that there is a place with
If that's expected, please apply this patch to make model OpenVINO compatible: @@ -107,7 +107,11 @@ class learnTSM(nn.Module):
shape = T, C, H, W = tensor.shape
split_size = self.split_size
- shift_tensor, main_tensor = tensor.split([split_size*2, C - 2 * split_size], dim=1)
+ if split_size * 2 == tensor.shape[1]:
+ shift_tensor, main_tensor = tensor, None
+ else:
+ shift_tensor, main_tensor = tensor.split([split_size*2, C - 2 * split_size], dim=1)
+
# pre_tensor, post_tensor = shift_tensor.split([split_size, split_size], dim=1)
pre_tensor = shift_tensor
post_tensor = shift_tensor
@@ -115,7 +119,8 @@ class learnTSM(nn.Module):
main_conv_tensor = self.main_conv(shift_tensor).view(T//tsm_length, tsm_length, split_size, H, W)
pre_tensor = self.pre_conv(pre_tensor).view(T//tsm_length, tsm_length, split_size//2, H, W)
post_tensor = self.post_conv(post_tensor).view(T//tsm_length, tsm_length, split_size//2, H, W)
- main_tensor = main_tensor.view(T//tsm_length, tsm_length, C - 2*split_size, H, W)
+ if main_tensor is not None:
+ main_tensor = main_tensor.view(T//tsm_length, tsm_length, C - 2*split_size, H, W)
if self.version == 'zero':
pre_tensor = F.pad(pre_tensor, (0, 0, 0, 0, 0, 0, 1, 0))[:, :-1, ...] # NOQA
@@ -126,7 +131,10 @@ class learnTSM(nn.Module):
post_conv_tensor = torch.cat((post_conv_tensor[:, 1: , ...], # NOQA
post_conv_tensor[:, :1 , ...]), dim=1) # NOQA
# print(pre_tensor.shape, post_tensor.shape, main_conv_tensor.shape, main_tensor.shape, shape)
- return torch.cat((pre_tensor, post_tensor, main_conv_tensor, main_tensor), dim=2).view(shape)
+ if main_tensor is not None:
+ return torch.cat((pre_tensor, post_tensor, main_conv_tensor, main_tensor), dim=2).view(shape)
+ else:
+ return torch.cat((pre_tensor, post_tensor, main_conv_tensor), dim=2).view(shape)Tested accuracy (with OpenVINO 2021.4): net = U_Net(1, 2, conv_type='conv_2d', tsm=True, learn=True)
net.eval()
dummy_inp = torch.randn([1, 1, 128, 160, 160])
torch.onnx.export(net, dummy_inp, "model.onnx", opset_version=11,
input_names=["input"], output_names=["output"])
inp = torch.randn([1, 1, 128, 160, 160])
ref = net(inp)
from openvino.inference_engine import IECore
ie = IECore()
net = ie.load_network("model.onnx", "CPU")
out = net.infer({"input": inp})["output"]
print(ref.shape)
print(out.shape)
print(np.max(np.abs(ref.detach().numpy() - out)))max diff: |
|
Can one of the admins verify this patch? |
|
@goodsong81 can your team take a look at this? |
|
Please resolve the merge conflicts then mark this PR as 'ready for review'. |
Submitting training module for Distill DSM: A computationally efficient method for segmentation of medical imaging volumes.
Paper: MIDL 2021
Dataset used for this code repo: Medical decathalon
This is part of the project MIRIAD: Many Incarnations of Screening of Radiology for High Throughput Disease Screening via Multiple Instance Reinforcement Learning with Adversarial Deep Neural Networks, sponsored by INTEL TECHNOLOGY INDIA PVT. LTD.
Principal Investigators:
Dr Debdoot Sheet (PI), Dr Nirmalya Ghosh (Co-PI)
Department of Electrical Engineering
Indian Institute of Technology Kharagpur
Dr Ramanathan Sethuraman (Co-PI)
Intel Technology India Pvt. Ltd.