交流群:462197261站长百科站长论坛热门标签收藏本站北冥有鱼 互联网前沿资源第一站 助力全行业互联网+
点击这里给我发消息
  • 当前位置:
  • 解决Keras TensorFlow 混编中 trainable=False设置无效问题

    北冥有鱼 教程大全 2020-06-28 ,,,

    这是最近碰到一个问题,先描述下问题:

    首先我有一个训练好的模型(例如vgg16),我要对这个模型进行一些改变,例如添加一层全连接层,用于种种原因,我只能用TensorFlow来进行模型优化,tf的优化器,默认情况下对所有tf.trainable_variables()进行权值更新,问题就出在这,明明将vgg16的模型设置为trainable=False,但是tf的优化器仍然对vgg16做权值更新

    以上就是问题描述,经过谷歌百度等等,终于找到了解决办法,下面我们一点一点的来复原整个问题。

    trainable=False 无效

    首先,我们导入训练好的模型vgg16,对其设置成trainable=False

    from keras.applications import VGG16
    import tensorflow as tf
    from keras import layers
    # 导入模型
    base_mode = VGG16(include_top=False)
    # 查看可训练的变量
    tf.trainable_variables()
    [<tf.Variable 'block1_conv1/kernel:0' shape=(3, 3, 3, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv1/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block1_conv2/kernel:0' shape=(3, 3, 64, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv2/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block2_conv1/kernel:0' shape=(3, 3, 64, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv1/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block2_conv2/kernel:0' shape=(3, 3, 128, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv2/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block3_conv1/kernel:0' shape=(3, 3, 128, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv2/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv2/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv3/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv3/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block4_conv1/kernel:0' shape=(3, 3, 256, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv2/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv2/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv3/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv3/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv2/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv2/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv3/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv3/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block1_conv1_1/kernel:0' shape=(3, 3, 3, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv1_1/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block1_conv2_1/kernel:0' shape=(3, 3, 64, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv2_1/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block2_conv1_1/kernel:0' shape=(3, 3, 64, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv1_1/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block2_conv2_1/kernel:0' shape=(3, 3, 128, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv2_1/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block3_conv1_1/kernel:0' shape=(3, 3, 128, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv1_1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv2_1/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv2_1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv3_1/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv3_1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block4_conv1_1/kernel:0' shape=(3, 3, 256, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv1_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv2_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv2_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv3_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv3_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv1_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv1_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv2_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv2_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv3_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv3_1/bias:0' shape=(512,) dtype=float32_ref>]
    # 设置 trainable=False
    # base_mode.trainable = False似乎也是可以的
    for layer in base_mode.layers:
      layer.trainable = False

    设置好trainable=False后,再次查看可训练的变量,发现并没有变化,也就是说设置无效

    # 再次查看可训练的变量
    tf.trainable_variables()

    [<tf.Variable 'block1_conv1/kernel:0' shape=(3, 3, 3, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv1/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block1_conv2/kernel:0' shape=(3, 3, 64, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv2/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block2_conv1/kernel:0' shape=(3, 3, 64, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv1/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block2_conv2/kernel:0' shape=(3, 3, 128, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv2/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block3_conv1/kernel:0' shape=(3, 3, 128, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv2/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv2/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv3/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv3/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block4_conv1/kernel:0' shape=(3, 3, 256, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv2/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv2/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv3/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv3/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv2/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv2/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv3/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv3/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block1_conv1_1/kernel:0' shape=(3, 3, 3, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv1_1/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block1_conv2_1/kernel:0' shape=(3, 3, 64, 64) dtype=float32_ref>,
     <tf.Variable 'block1_conv2_1/bias:0' shape=(64,) dtype=float32_ref>,
     <tf.Variable 'block2_conv1_1/kernel:0' shape=(3, 3, 64, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv1_1/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block2_conv2_1/kernel:0' shape=(3, 3, 128, 128) dtype=float32_ref>,
     <tf.Variable 'block2_conv2_1/bias:0' shape=(128,) dtype=float32_ref>,
     <tf.Variable 'block3_conv1_1/kernel:0' shape=(3, 3, 128, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv1_1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv2_1/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv2_1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block3_conv3_1/kernel:0' shape=(3, 3, 256, 256) dtype=float32_ref>,
     <tf.Variable 'block3_conv3_1/bias:0' shape=(256,) dtype=float32_ref>,
     <tf.Variable 'block4_conv1_1/kernel:0' shape=(3, 3, 256, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv1_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv2_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv2_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block4_conv3_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block4_conv3_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv1_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv1_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv2_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv2_1/bias:0' shape=(512,) dtype=float32_ref>,
     <tf.Variable 'block5_conv3_1/kernel:0' shape=(3, 3, 512, 512) dtype=float32_ref>,
     <tf.Variable 'block5_conv3_1/bias:0' shape=(512,) dtype=float32_ref>]

    解决的办法

    解决的办法就是在导入模型的时候建立一个variable_scope,将需要训练的变量放在另一个variable_scope,然后通过tf.get_collection获取需要训练的变量,最后通过tf的优化器中var_list指定需要训练的变量

    from keras import models
    with tf.variable_scope('base_model'):
      base_model = VGG16(include_top=False, input_shape=(224,224,3))
    with tf.variable_scope('xxx'):
      model = models.Sequential()
      model.add(base_model)
      model.add(layers.Flatten())
      model.add(layers.Dense(10))
    
    
    # 获取需要训练的变量
    trainable_var = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, 'xxx')
    trainable_var

    [<tf.Variable 'xxx_2/dense_1/kernel:0' shape=(25088, 10) dtype=float32_ref>,
    <tf.Variable 'xxx_2/dense_1/bias:0' shape=(10,) dtype=float32_ref>]

    # 定义tf优化器进行训练,这里假设有一个loss
    loss = model.output / 2; # 随便定义的,方便演示
    train_step = tf.train.AdamOptimizer().minimize(loss, var_list=trainable_var)

    总结

    在keras与TensorFlow混编中,keras中设置trainable=False对于TensorFlow而言并不起作用

    解决的办法就是通过variable_scope对变量进行区分,在通过tf.get_collection来获取需要训练的变量,最后通过tf优化器中var_list指定训练

    以上这篇解决Keras TensorFlow 混编中 trainable=False设置无效问题就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持北冥有鱼。


    广而告之:
    热门推荐:
    教你如何优雅的实现垂直居中(推荐)

    前言 css水平垂直居中的方法有很多,本文给大家介绍的方法非常的全面详细,分享出来供大家参考学习,下面来一起看看详细的介绍吧。 "44年前我们就把人类送上月球了,但现在我们仍然无法在 CSS 中实现垂直居中。" --Jams Anderson 在网页设计中,每当涉及到居中时,···

    利用百度热度锁定长尾关键词技巧分享

    如何锁定关键词,今天我来讲一下一个很简单的小方法来锁定用户的搜索关键词,从而制定长尾关键词达到软文高曝光量!  关键词和长尾关键词有什么区别?  关键词:就是通过百度客户想了解的产品名,服务或者产品公司。比如:减肥。这就是关键词,如果你想推···

    Vue2.x中利用@font-size引入字体图标报错的解决方法

    利用 vue-cli 搭建的项目平台 利用stylus写的css样式 有 css-loader 依赖包x 下图是 webpack.base.conf.js 关于字体文件的配置 有人这里会有重复的字体文件的配置,删除一项即可 出现的问题:引入字体图标出现问题 1.报错 将字体引入的相对路径改成绝对路径 相对路径 绝对路···

    安装MySQL后include目录下没有找到libmysql.lib

    最近在研究C++连接数据库的问题。 安装了MySQL后,在其安装目录下的include文件夹并没有找到libmysql.lib. 这个经过研究,发现是由于在安装MySQL的时候, 没有勾选 develop component这一选项 悲剧的是哥使用的是WampServer集成安装mysql, 安装过程并没有具体配置选择。 最后···

    EasyUI为Numberbox添加blur事件的方法

    EasyUI为Numberbox添加blur事件的方法: 为名为'txtCouponRateParam'的Numberbox添加blur事件 $("#txtCouponRateParam").numberbox({ precision:4, }); $("#txtCouponRateParam").next("span").children().first().blur(function(){ //方法体 }); 以上所述是小编···

    js调用百度地图及调用百度地图的搜索功能

    js调用百度地图的方法 代码如下: <!DOCTYPE html> <html> <head> <meta name="viewport" content="initial-scale=1.0, user-scalable=no" /> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <title>Hello, ···

    JavaScript实现SHA-1加密算法的方法

    本文实例讲述了JavaScript实现SHA-1加密算法的方法。分享给大家供大家参考。具体实现方法如下: 调用方法:hex_sha1即可。 复制代码 代码如下:/*  *  * A JavaScript implementation of the Secure Hash Algorithm, SHA-1, as defined  * in FIPS PUB 180-1 &nb···

    JavaScript  event对象整理及详细介绍

    Event属性和方法: 1. type:事件的类型,如onlick中的click; 2. srcElement/target:事件源,就是发生事件的元素; 3. button:声明被按下的鼠标键,整数,1代表左键,2代表右键,4代表中键,如果按下多个键,酒把这些值加起来,所以3就代表左右键同时按下;(firefox中 0代···

    node.js中Socket.IO的进阶使用技巧

    在上一篇博文Socket.IO中,我简要介绍了Socket.IO的基本使用方法并创建了一个简单的聊天室DEMO。本篇在入门篇的基础上,继续探讨Socket.IO的进阶用法。本篇将从配置、房间、事件等方面入手,介绍一些Socket.IO中实用的API和注意事项。 1. 配置 Socket.IO提供了4个配置的API:io···

    jquery插件bootstrapValidator数据验证详解

    因为项目需要数据验证,看bootstrapValidator 还不错,就上手一直,完美兼容,话不多说。 bootstrap:能够增加兼容性的强大框架. 需要引用css: bootstrap.min.css bootstrapValidator.min.css js: jquery-1.10.2.min.js bootstrap.min.js bootstrapValidator.min.js (下载···