SR-ITM--融合超分辨率和逆色调映射（三）-FlyAI

SR-ITM--融合超分辨率和逆色调映射（三）

深度学习视觉计算

最后更新 2020-08-06 11:43 阅读 4133

最后更新 2020-08-06 11:43

阅读 4133

深度学习视觉计算

前面两期以综述的形式讲解了超分辨率和逆色调技术的经典论文，这次主要讲解这篇融合了超分辨率和逆色调映射的技术ICCV2019的oral论文，算法框架如下：

![](https://dataset.flyai.com/aaceb41cc50a0e9b19659c62583693f.png)
该网络将输入图像通过Guided filter分解为Detail layer和Base layer
Detail layer:restoring high frequency details(SR)
Base layer:recovering local contrast(ITM)

Base layer即为低频：低频做逆色调映射
Detail layer高频：高频做超分辨率
![](https://dataset.flyai.com/bb3dcd6b2c40f3e2f71c2b7b7fb5872.png)
网络的核心思想主要是这种图像分解做不同任务，最后通过融合，得到最终的效果。网络细节部分主要是一些模块的叠加，主要为如下几大模块：

（1）ResBlock

class BaseBlock(nn.Module):
    
		def __init__(self, channels):
			super(BaseBlock, self).__init__()
			self.conv1 = convBlock_c(channels=channels)
			self.conv2 = convBlock_c(channels=channels)
			self.relu = reluBlock()

def forward(self, x):

out = self.relu(x)
			out = self.conv1(out)

out = self.relu(out)
			out = self.conv2(out)

return out

class ResBlock(nn.Module):
    
		def __init__(self, channels):
			super(ResBlock, self).__init__()
			self.baseblock = BaseBlock(channels=channels)

def forward(self, x):
			identity = x

out = self.baseblock(x)
			out += identity

return out

（2）ResModBlock

class ResModBlock(nn.Module):
    
		def __init__(self, channels):
			super(ResModBlock, self).__init__()
			self.baseblock = BaseBlock(channels=channels)
			self.conv_smf_1 = convBlock_c(channels=channels)
			self.conv_smf_2 = convBlock_c(channels=channels)
			self.relu = reluBlock()

def forward(self, x, SMF):
			identity = x

out = self.baseblock(x)
			out_shared = self.conv_smf_1(SMF)
			out_shared = self.relu(out_shared)
			out_shared = self.conv_smf_2(out_shared)

# multiply elementwisely
			out *= out_shared
			out += identity

return out
			
（3）ResSkipModBlock

class ResSkipModBlock(nn.Module):
    
		def __init__(self, channels):
			super(ResSkipModBlock, self).__init__()
			self.skipblock = SkipBlock(channels=channels)
			self.conv_smf_1 = convBlock_c(channels=channels)
			self.conv_smf_2 = convBlock_c(channels=channels)
			self.relu = reluBlock()

def forward(self, x, RB_out_feature, SMF):
			identity = x

out = self.skipblock(x, RB_out_feature)

out_shared = self.conv_smf_1(SMF)
			out_shared = self.relu(out_shared)
			out_shared = self.conv_smf_2(out_shared)

out *= out_shared
			out += identity

return out

（4）ResSkipBlock

class SkipBlock(nn.Module):
    
		def __init__(self, channels):
			super(SkipBlock, self).__init__()
			self.conv1 = convBlock_c(channels=channels)
			self.conv2 = convBlock_c(channels=channels)
			self.relu = reluBlock()
			self.dr = convBlock_dr(out_channels=channels)

def forward(self, x, block_out_feature):

in_features = torch.cat((x, block_out_feature), dim=1)   #按列拼接
			out = self.relu(in_features)
			out = self.dr(out)
			out = self.conv1(out)

out = self.relu(out)
			out = self.conv2(out)

return out

class ResSkipBlock(nn.Module):
    
		def __init__(self, channels):
			super(ResSkipBlock, self).__init__()
			self.skipblock = SkipBlock(channels=channels)

def forward(self, x, RMB_out_feature):
			identity = x

out = self.skipblock(x, RMB_out_feature)
			out += identity

return out

最后通过模块的连接，最终融合在一起，形成整个网络的框架。

实验结果
![](https://dataset.flyai.com/7997f7f8b5d3f7df92cafeb80fd2014.png)
通过数据测试，可以看出很好的恢复效果：

（1）测试1
![](https://dataset.flyai.com/32a280c0b896e330ff212bdfd5d8bd8.png)
![](https://dataset.flyai.com/46e3fd35dcaf974758126ee87f21733.png)
![](https://dataset.flyai.com/77e0722ea19abe35e10daefe49e7360.png)

（2）测试2
![](https://dataset.flyai.com/27e6ae9fe89e9c3dbec7dd3a96c56c0.png)
![](https://dataset.flyai.com/218e5d42fbd7f451f04bbb67f488dd2.png)
![](https://dataset.flyai.com/a03d325451d02f32e7d0e8add4e6e2a.png)

（3）测试3
![](https://dataset.flyai.com/3b0ed0a8b48b584e810c57155efa799.png)
![](https://dataset.flyai.com/82d65509652ab81fb8326f735f3f5a5.png)
![](https://dataset.flyai.com/1e1711c86e30199c075977ce383f65f.png)

本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可，转载请附上原文出处链接和本声明。
本文链接地址：https://www.flyai.com/article/632

讨论

500字

表情

发送

删除确认

是否删除该条评论？

取消删除