-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathatom.xml
487 lines (262 loc) · 198 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>Hexo</title>
<link href="/JunFu.github.io/atom.xml" rel="self"/>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/"/>
<updated>2019-03-22T04:39:23.787Z</updated>
<id>https://github.com/SunshineJunFu/JunFu.github.io/</id>
<author>
<name>Jun Fu</name>
</author>
<generator uri="http://hexo.io/">Hexo</generator>
<entry>
<title>paper1</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/03/22/paper1/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/03/22/paper1/</id>
<published>2019-03-22T04:38:25.000Z</published>
<updated>2019-03-22T04:39:23.787Z</updated>
<content type="html"><![CDATA[<h1 id="Spatial-Temporal-Graph-Convolutional-Networks-for-Skeleton-Based-Action-Recognition"><a href="#Spatial-Temporal-Graph-Convolutional-Networks-for-Skeleton-Based-Action-Recognition" class="headerlink" title="Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition"></a>Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition</h1><h2 id="Abstract"><a href="#Abstract" class="headerlink" title="Abstract"></a>Abstract</h2><p>人体骨骼点的变化对人体动作的识别提供了重要的信息。传统骨骼建模的方法通常依赖于手工或者遍历的方法，导致表达能力受限以及很难泛化。本文提出了一种对动态骨骼建模的新方法，该方法叫做时空域图卷积网络。该方法通过自动地从数据中学习时空域的模式，避免了前面方法的局限性。该方法不仅得到了更好的表达能力，而且具有更好的泛化性。在Kinetics和NTU-RGBD两个大数据集上，该方法相比于主流方法，在性能方面实现了大幅度的提升。</p><h2 id="Current-Research-State"><a href="#Current-Research-State" class="headerlink" title="Current Research State"></a>Current Research State</h2><h2 id="Pipeline-OVerview"><a href="#Pipeline-OVerview" class="headerlink" title="Pipeline OVerview"></a>Pipeline OVerview</h2><h2 id="Skeleton-Graph-Construction"><a href="#Skeleton-Graph-Construction" class="headerlink" title="Skeleton Graph Construction"></a>Skeleton Graph Construction</h2><p>T帧骨骼序列，每帧包含N个连接点，其可以构成一个无向时空图G = (V, E)，图中的节点集为 $V={v_{ti}|t=1,…,T, i=1,…,N}$, 每个节点的特征向量用$F(v_{ti})$来表示，$F(v_{ti})$ 表示第t帧第i个节点的特征向量，其由该点的坐标向量以及坐标向量的置信度组成。</p><p>时空域图的构成分成两部分：</p><ol><li>帧内(intra): 由人体的结构来确定节点间是否相连</li><li>帧间(inter): 时序上，相同的节点相连</li></ol><p>时空域图上的边：</p><ol><li>帧内边， 记为 $E_{S} = {v_{ti}v_{tj}|(i,j) \in H}$, H代表自然连接的人体节点。</li><li>帧间边， 记为 $E_{F} = {v_{ti}v_{(t+1)i}$， 代表了它随时间变化的轨迹。</li></ol><h2 id="Spatial-Graph-Convoluional-Neural-Network"><a href="#Spatial-Graph-Convoluional-Neural-Network" class="headerlink" title="Spatial Graph Convoluional Neural Network"></a>Spatial Graph Convoluional Neural Network</h2><p>先介绍下二维图像的卷积。给定卷积核大小为 $K x K$, 操作对象为 宽为W, 高为H, 通道数为c的特征图。对于空间上的某点x， 其卷积后的值可以通过下式计算得到：</p><p>$ f_{out} = \sum_{h=1}^{K}\sum_{w=1}^{K} f_in(p(x,h,w)) \dot w(h,w) $</p><p>其中，p(x,h,w)为采样函数：$$Z^{2} x Z^{2} \rightarrow Z^{2} $$ 枚举了点x的邻域，在图像里，也可以简化为$$p(x,h,w)= x + p^{‘}(h,w) $$, $$f_in(p(x,h,w)) \in R^{1x1xc} w(h,w) \in R^{1x1xc} $$, $$w(h,w)$$与位置无关。</p><p>接着，基于二维图像卷积的认识，我们推导在图上的卷积。从上面可以发现，卷积的核心部分在于采样函数p，权重函数w以及特征图$$f_in$$的定义。</p><p>1： 对于t时刻的空域图$$V_{t}$$, 特征图为$$f_in^{t}: V_{t} \rightarrow R^{Nxc}$$, 是N节点的c维特征集合。<br>2： 采样函数p(x,h,w), 在图像中，采样函数p(h,w)定义为x的邻域， 数学上可以表达为: $$D_{8}(x) \leq k $$, $$ k $$为街区距离。因此，类似地，我们定义采样函数为节点的邻域，例如，对于节点$$v_{ti}$$, 采样函数表示为$$B(v_{ti} = {v_{tj}|d(v_{tj}, v_{ti}) \leq D}$$, D代表$$v_{ti}$$与$$v_{tj}$$的路径长度。论文中给出的D，取值为1。<br>3: 权重函数w， 相比于图像而言，图的邻域没有明确空间的顺序。解决该问题的一种方式是对邻域进行标号。本文也采用的该思路，不同于对每一个邻域节点进行标号，文中将$$B(v_{ti})$$分成固定的$$K$$个子集，也就是有一个映射函数 $$l_{ti}: B(v_{ti}) \rightarrow {0,…,K-1} $$。 那么基于此，权重函数$$w(v_{ti},v_{tj})：B(v_{ti}) \rightarrow R^{c} $$，c是单个节点特征的维度，这个可以通过预先定义一个cxK维的张量，然后通过子集序号来索引，即：<br>$$w(v_{ti},v_{tj})=w^{‘}(l_{ti}(v_{tj}))$$.</p><p>总结而言，图卷积仍然是在邻域内进行加权(特征图和权重内积），改变了采样函数p，权重函数w以及特征图$$f_in$$的定义方式。</p><h2 id="Spatial-Graph-Convolution"><a href="#Spatial-Graph-Convolution" class="headerlink" title="Spatial Graph Convolution"></a>Spatial Graph Convolution</h2><p>通过重定义采样函数p，权重函数w以及特征图$$f_in$$，则可以推出下式：</p><p>$f_{out}(v_{ti}= \sum\limits_{v_{tj} \in B(v_{ti})} \frac{1}{Z_{ti}(v_{tj})}f_{in}(p(v_{ti},v_{tj}))w(v_{ti},v_{tj})$</p><p>其中，$$Z_{ti}(v_{tj})} = |{v_{tk}|l_{ti}(v_{tk})= l_{ti}(v_{tj})|$$, 代表了对应子集的基数，这个是用于平衡不同自己对输出的贡献。最终的出下式：</p><p>$f_{out}(v_{ti}= \sum\limits_{v_{tj} \in B(v_{ti})} \frac{1}{Z_{ti}(v_{tj})}f_{in}(v_{tj})w(v_{ti},v_{tj})$。</p><p>值得注意的是， 如果把图像当作一个规则的2D网格，上式类似于标准的2D卷积。以3x3的卷积操作为例，我们有9个pixels，邻域被分成9个子集，每个子集含有1个像素点。</p><h2 id="Spatial-Temporal-Modeling"><a href="#Spatial-Temporal-Modeling" class="headerlink" title="Spatial Temporal Modeling"></a>Spatial Temporal Modeling</h2><p>了解完空域的图卷积，那么接下来将它扩展至时域图卷积。</p><ol><li>定义邻域： $$ B(v_{ti}) = {v_{qj}|d(v_{tj},v_{ti}) \leq K, |q-t| \leq \lfloor \Gamma/2 \rfloor}, 其中$$\Gamma$$控制者考虑时域的范围，可以理解为时域的核大小。</li><li>定义权重函数：与空域类似，仍然采用标号的方式（类似于embedding），由于时域上顺序是确定的，因此，对于节点$v_{ti}$, 其$l_{ST}$用下式确定：<br>$l_{ST}(v_{qj}) = l_{ti}(v_{tj}) + (q-t+ \lfloor \Gamma/2 \rfloor) x K $<br>其中$l_{ti}(v_{tj})$是t时刻， i节点的空域标号。<br>理解： 参照图1， 就是时域的邻域为前后 $$\lfloor \Gamma/2 \rfloor)$$ 帧， 每帧考虑K个节点(图1很好的解释了该公式)。<br>Ps:之所以如此编号，是为了确定邻接矩阵的index</li></ol><h2 id="Partition-Strategies"><a href="#Partition-Strategies" class="headerlink" title="Partition Strategies"></a>Partition Strategies</h2><h3 id="单一labeling"><a href="#单一labeling" class="headerlink" title="单一labeling"></a>单一labeling</h3><p>邻域内所有节点有着相同的贡献。</p><p>缺点： 丢失局部信息。</p><h3 id="距离划分"><a href="#距离划分" class="headerlink" title="距离划分"></a>距离划分</h3><p>根据距离中心节点的距离进行划分</p><h3 id="空域配置划分"><a href="#空域配置划分" class="headerlink" title="空域配置划分"></a>空域配置划分</h3><p>自定义一些划分方式</p><h2 id="可学习的边重要性"><a href="#可学习的边重要性" class="headerlink" title="可学习的边重要性"></a>可学习的边重要性</h2><p>尽管当人在执行动作时，骨骼点一起运动，但是一个骨骼点可能出现在多个身体部位。 但是，每个节点应该在不同的部分有不同的重要性。因此，我们在每一时空图卷积加了一个可学习的掩模（mask）。</p><h2 id="实现ST-GCN"><a href="#实现ST-GCN" class="headerlink" title="实现ST-GCN"></a>实现ST-GCN</h2><p>输入Tensor: shape ( C, V, T)</p><p>C : 单个节点上的feature维度<br>V : 空域节点<br>T ： 时间帧数</p><p>该文章实现思路可以分为两步走：</p><ol><li>对每个时间点t，进行空域图卷积，此时邻接矩阵是已知的</li><li>然后对不同时刻的相同节点进行2-D卷积，核大小为 $1x \Gamma$, 即进行时域聚合，此时，邻接矩阵是用卷积核来替代的。</li></ol><p>理解： </p><ol><li>第一步，相当于把空域的邻接节点的信息考虑了进来</li><li>第二步，相当于把 时域上的相同节点的邻接节点考虑了进来</li></ol><p>实现过程中，论文中采用了批归一化，主要类似于kipf的归一化邻接矩阵，起到防止梯度爆炸的作用。</p>]]></content>
<summary type="html">
<h1 id="Spatial-Temporal-Graph-Convolutional-Networks-for-Skeleton-Based-Action-Recognition"><a href="#Spatial-Temporal-Graph-Convolutional-
</summary>
<category term="Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/Spatial-Temporal-Graph-Convolutional-Networks-for-Skeleton-Based-Action-Recognition/"/>
</entry>
<entry>
<title>tensorboard_install</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/27/tensorboard-install/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/27/tensorboard-install/</id>
<published>2019-02-27T03:19:31.000Z</published>
<updated>2019-02-27T03:29:04.924Z</updated>
<content type="html"><![CDATA[<h1 id="安装Tensorflow-gpu"><a href="#安装Tensorflow-gpu" class="headerlink" title="安装Tensorflow-gpu"></a>安装Tensorflow-gpu</h1><ul><li><p>查看cuda版本: <code>cat /usr/local/cuda/version.txt</code></p></li><li><p>查看cudnn版本: <code>cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2</code></p></li><li><p>在<a href="https://www.tensorflow.org/install/source#linux" target="_blank" rel="noopener">该网站</a>上，根据cuda和cudnn版本选择合适的tensorflow-gpu版本</p></li><li><p>在<a href="https://pypi.tuna.tsinghua.edu.cn/simple/tensorflow-gpu/" target="_blank" rel="noopener">该网站</a>上下载对应版本的whl文件</p></li><li><p>执行<code>pip install xxx.whl</code>, 即可</p></li><li><p>验证安装成功：</p><ul><li><p>import tensorflow a tf</p></li><li><p>tf.Session() </p></li><li><p>上述语句无报错，则表示安装无误</p></li></ul></li></ul>]]></content>
<summary type="html">
<h1 id="安装Tensorflow-gpu"><a href="#安装Tensorflow-gpu" class="headerlink" title="安装Tensorflow-gpu"></a>安装Tensorflow-gpu</h1><ul>
<li><p>查看cud
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>tensorflowtutorial11</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/20/tensorflowtutorial11/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/20/tensorflowtutorial11/</id>
<published>2019-02-20T13:01:22.000Z</published>
<updated>2019-02-20T13:48:08.444Z</updated>
<content type="html"><![CDATA[<h1 id="tf-estimator-RunConfig"><a href="#tf-estimator-RunConfig" class="headerlink" title="tf.estimator.RunConfig()"></a>tf.estimator.RunConfig()</h1><p>函数：</p><p>replace(**kwargs)</p><p>返回新的RunConfig</p><p>以下可以被修改：</p><ul><li>model_dir,</li><li>tf_random_seed,</li><li>save_summary_steps,</li><li>save_checkpoints_steps,</li><li>save_checkpoints_secs,</li><li>session_config,</li><li>keep_checkpoint_max,</li><li>keep_checkpoint_every_n_hours,</li><li>log_step_count_steps,</li><li>train_distribute,</li><li>device_fn,</li><li>protocol.</li><li>eval_distribute,</li><li>experimental_distribute, </li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">config = tf.estimator.RunConfig().replace(</span><br><span class="line"> session_config=tf.ConfigProto(device_count={<span class="string">'CPU'</span>:self._num_threads}),</span><br><span class="line"> log_step_count_steps=<span class="number">20</span>)</span><br></pre></td></tr></table></figure><p>tf.ConfigProto()</p><p>配置seseion参数</p><p>…</p><h1 id="tf-estimator-Estimator"><a href="#tf-estimator-Estimator" class="headerlink" title="tf.estimator.Estimator"></a>tf.estimator.Estimator</h1><p>创建Estimator</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">__init__(</span><br><span class="line"> model_fn,</span><br><span class="line"> model_dir=<span class="keyword">None</span>,</span><br><span class="line"> config=<span class="keyword">None</span>,</span><br><span class="line"> params=<span class="keyword">None</span>,</span><br><span class="line"> warm_start_from=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li><p>model_fn: 模型函数</p><p> (features, labels, mode, params, config) or (features, labels, mode, params)</p><ul><li>features: This is the first item returned from the input_fn passed to train, evaluate, and predict. This should be a single tf.Tensor or dict of same.</li><li>labels: This is the second item returned from the input_fn passed to train, evaluate, and predict. </li><li>mode: Optional. Specifies if this training, evaluation or prediction. See tf.estimator.ModeKeys</li><li>params: Optional dict of hyperparameters. Will receive what is passed to Estimator in params parameter. This allows to configure Estimators from hyper parameter tuning</li><li><p>config: Optional estimator.RunConfig object. Will receive what is passed to Estimator as its config parameter, or a default value. Allows setting up things in your model_fn based on configuration such as num_ps_replicas, or model_dir</p><p>return: tf.estimator.EstimatorSpec()</p></li></ul></li><li><p>model_dir: 保存模型的路径</p></li><li><p>config： estimator.RunConfig configuration object.</p></li><li><p>params： dict of hyper parameters that will be passed into model_fn. Keys are names of parameters, values are basic python types.</p></li></ul><p>Returns:</p><p>an Estimator instance</p><h1 id="tf-estimator-EstimatorSpec"><a href="#tf-estimator-EstimatorSpec" class="headerlink" title="tf.estimator.EstimatorSpec()"></a>tf.estimator.EstimatorSpec()</h1><p><strong>new</strong>(<br> cls,<br> mode,<br> predictions=None,<br> loss=None,<br> train_op=None,<br> eval_metric_ops=None,<br> export_outputs=None,<br> training_chief_hooks=None,<br> training_hooks=None,<br> scaffold=None,<br> evaluation_hooks=None,<br> prediction_hooks=None<br>)</p><p>Depending on the value of mode, different arguments are required. Namely</p><ul><li>For mode == ModeKeys.TRAIN: required fields are loss and train_op.</li><li>For mode == ModeKeys.EVAL: required field is loss.</li><li>For mode == ModeKeys.PREDICT: required fields are predictions.</li></ul><h1 id="tf-estimator-TrainSpec"><a href="#tf-estimator-TrainSpec" class="headerlink" title="tf.estimator.TrainSpec"></a>tf.estimator.TrainSpec</h1><p>创建训练的TrainSpec实例</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">@staticmethod</span></span><br><span class="line">__new__(</span><br><span class="line"> cls,</span><br><span class="line"> input_fn,</span><br><span class="line"> max_steps=<span class="keyword">None</span>,</span><br><span class="line"> hooks=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li>input_fn: 提供minibatches的输入数据的函数。函数的返回值结构如下：<ul><li>a tf.data.Dataset 对象</li><li>(features, labels): features 是tensor或者dict{name： tensor}</li></ul></li><li>max_steps: int, 训练的最大步数，未设置，则永远训练</li><li>hooks： 钩子函数</li></ul><p>Returns：</p><p>TrainSpec实例</p><h1 id="tf-estimator-EvalSpec"><a href="#tf-estimator-EvalSpec" class="headerlink" title="tf.estimator.EvalSpec"></a>tf.estimator.EvalSpec</h1><p>创建验证的EvalSpec实例</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">@staticmethod</span></span><br><span class="line">__new__(</span><br><span class="line"> cls,</span><br><span class="line"> input_fn,</span><br><span class="line"> steps=<span class="number">100</span>,</span><br><span class="line"> name=<span class="keyword">None</span>,</span><br><span class="line"> hooks=<span class="keyword">None</span>,</span><br><span class="line"> exporters=<span class="keyword">None</span>,</span><br><span class="line"> start_delay_secs=<span class="number">120</span>,</span><br><span class="line"> throttle_secs=<span class="number">600</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li>input_fn: 提供minibatches的输入数据的函数。函数的返回值结构如下：<ul><li>a tf.data.Dataset 对象</li><li>(features, labels): features 是tensor或者dict{name： tensor}</li></ul></li><li>steps: None,则在整个验证集验证</li></ul><p>Returns：</p><p>EvalSpec实例</p><h1 id="tf-estimator-train-and-evaluate"><a href="#tf-estimator-train-and-evaluate" class="headerlink" title="tf.estimator.train_and_evaluate"></a>tf.estimator.train_and_evaluate</h1><p>函数： Train and evaluate the estimator </p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.estimator.train_and_evaluate(</span><br><span class="line"> estimator,</span><br><span class="line"> train_spec,</span><br><span class="line"> eval_spec</span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li><p>estimator: estimator实例</p></li><li><p>train_spec： train_spec实例</p></li><li><p>eval_spec： eval_spec实例</p></li></ul><h1 id="tf-nn-embedding-lookup"><a href="#tf-nn-embedding-lookup" class="headerlink" title="tf.nn.embedding_lookup"></a>tf.nn.embedding_lookup</h1>]]></content>
<summary type="html">
<h1 id="tf-estimator-RunConfig"><a href="#tf-estimator-RunConfig" class="headerlink" title="tf.estimator.RunConfig()"></a>tf.estimator.RunCo
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>linuxweek2</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/17/linuxweek2/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/17/linuxweek2/</id>
<published>2019-02-17T07:22:46.000Z</published>
<updated>2019-02-17T08:16:09.724Z</updated>
<content type="html"><![CDATA[<h1 id="正则表达式"><a href="#正则表达式" class="headerlink" title="正则表达式"></a>正则表达式</h1><p>功能：描述一个字符串模式，用于字符串匹配操作和替换操作，针对文本内容</p><table><thead><tr><th>字符类别</th><th>描述</th></tr></thead><tbody><tr><td>元字符</td><td>.，*，[，\，^，$</td></tr><tr><td>其他字符</td><td>与自身匹配</td></tr></tbody></table><table><thead><tr><th>元字符</th><th>描述</th></tr></thead><tbody><tr><td>.</td><td>代表匹配任意单字符</td></tr><tr><td>*</td><td>表示该符号前一个字符出现0次或者任意多次</td></tr><tr><td>[</td><td>表示集合，在集合中\，*，.代表其本身</td></tr><tr><td>\</td><td>转移字符，将特殊字符转义为本身</td></tr><tr><td>-</td><td>[a-z]两个字符之间表示区间，否则代表本身</td></tr><tr><td>^</td><td>集合中时表示取反，在正则表达式行首表示匹配行首的字符串</td></tr><tr><td>$</td><td>在正则表达式行尾表示匹配行尾的字符串，其他位置代表本身</td></tr></tbody></table><h1 id="PCRE"><a href="#PCRE" class="headerlink" title="PCRE"></a>PCRE</h1><p>对BRE进行了改进：</p><ul><li>()，表示分组</li><li>| 表示逻辑运算 (pink | green)</li><li><ul><li>表示1次或多次</li></ul></li><li>？ 表示出现0次或1次</li><li>{m,n}, 出现m至n次</li><li>命名的预定义集合： [[:xdigit:]] 16进制数字，\d $\rightarrow$ 数字，\D $\rightarrow$ 非数字</li></ul><h1 id="文本编辑工具"><a href="#文本编辑工具" class="headerlink" title="文本编辑工具"></a>文本编辑工具</h1><h2 id="grep"><a href="#grep" class="headerlink" title="grep"></a>grep</h2><p>行筛选</p><p>用法： grep [option] 模式 文件名列表</p><p>选项： </p><ul><li>-n 显示时每行前面显示行号</li><li>-v 显示所有不包含模式的行</li><li>-i 字母比较时忽略字母的大小写</li></ul><h2 id="sed"><a href="#sed" class="headerlink" title="sed"></a>sed</h2><p>流编辑 </p><p>用法： </p><ul><li><p>单个命令: sed ‘cmd’ 文件列表 </p></li><li><p>多个命令： sed -e ‘cmd’ -e ‘cmd1’ 文件列表 </p></li><li><p>执行文件里的命令： sed -f 命令文件 文件列表 </p></li><li><p>模式描述增加( 和 )， 不影响匹配， 用于分组， 与之配套用的是 \0, \1, \2…表示第几个括号括起来的内容</p></li><li><p>替换： ‘s/xxx/xx/g’</p></li></ul><h2 id="awk"><a href="#awk" class="headerlink" title="awk"></a>awk</h2><p>复杂筛选和加工，朱行扫描进行文本处理的一门语言</p><p>用法:</p><ul><li><p>awk ‘程序’ 文件名列表</p></li><li><p>awk -f 程序文件名 文件名列表</p></li><li><p>程序： 条件 {动作}， awk自动对每行文本执行条件判断，满足则执行动作， 多段程序使用空格或分号隔开</p></li></ul><h3 id="变量"><a href="#变量" class="headerlink" title="变量"></a>变量</h3><ul><li><p>NR， 行号</p></li><li><p>每行以空格为域分隔符，$1 代表第1个空格间的内容， $0代表整行内容</p></li></ul><h3 id="条件"><a href="#条件" class="headerlink" title="条件"></a>条件</h3><ul><li><p>关系运算： <，>, ==， <=, >=, !=</p></li><li><p>逻辑运算： || ， &&， !</p></li><li><p>正则表达式的模式匹配： /regexpr/ 包含该模式的行，执行动作</p></li><li><p>不指定任何条件，对对所有文本行执行动作</p></li><li><p>BEGIN 开始处理所有文本行之前执行动作，END 处理完所有文本行之后执行动作</p></li></ul><h3 id="动作"><a href="#动作" class="headerlink" title="动作"></a>动作</h3><ul><li><p>加减乘除</p></li><li><p>print 变量1，变量2</p></li><li><p>print(“格式串”，变量1，变量2)</p></li><li><p>流程控制： if 条件判断，for 循环控制</p></li></ul>]]></content>
<summary type="html">
<h1 id="正则表达式"><a href="#正则表达式" class="headerlink" title="正则表达式"></a>正则表达式</h1><p>功能：描述一个字符串模式，用于字符串匹配操作和替换操作，针对文本内容</p>
<table>
<thead>
<tr
</summary>
<category term="linux" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/linux/"/>
</entry>
<entry>
<title>linuxweek5</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/15/linuxweek5/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/15/linuxweek5/</id>
<published>2019-02-15T13:39:14.000Z</published>
<updated>2019-02-17T07:18:54.654Z</updated>
<content type="html"><![CDATA[<h1 id="week5"><a href="#week5" class="headerlink" title="week5"></a>week5</h1><h2 id="文件系统"><a href="#文件系统" class="headerlink" title="文件系统"></a>文件系统</h2><table><thead><tr><th>类别</th><th>描述</th></tr></thead><tbody><tr><td>根文件系统</td><td>整个文件系统的基础，不能脱卸(unmount)</td></tr><tr><td>子文件系统</td><td>包含硬盘，软盘，…，以根根文件系统中某一目录的身份出现</td></tr></tbody></table><p>注： 根文件系统和子文件系统都有其自己独立的文件存储结构，甚至文件系统的格式也不同。</p><h3 id="创建与删除"><a href="#创建与删除" class="headerlink" title="创建与删除"></a>创建与删除</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">1. 在块设备文件 /dev/sdb上创建文件系统</span><br><span class="line"></span><br><span class="line">mkfs /dev/sdb</span><br><span class="line"></span><br><span class="line">2. 安装子文件系统mount，/mnt为预先创建的空目录名</span><br><span class="line"></span><br><span class="line">mount /dev/sdb /mnt</span><br><span class="line"></span><br><span class="line">3. 列出当前已安装的所有的子文件系统</span><br><span class="line"></span><br><span class="line">mount </span><br><span class="line"></span><br><span class="line">4. 卸载某个文件系统</span><br><span class="line"></span><br><span class="line">unmount /dev/sdb</span><br><span class="line"></span><br><span class="line">5. 显示文件系统空间</span><br><span class="line"></span><br><span class="line">df -h</span><br></pre></td></tr></table></figure><h3 id="文件系统的结构"><a href="#文件系统的结构" class="headerlink" title="文件系统的结构"></a>文件系统的结构</h3><figure class="image-bubble"> <div class="img-lightbox"> <div class="overlay"></div> <img src="1.png" alt="hh" title> </div> <div class="image-caption">hh</div> </figure><table><thead><tr><th>组成</th><th>描述</th></tr></thead><tbody><tr><td>引导块</td><td>用于启动系统，仅根文件系统的引导块有效</td></tr><tr><td>专用块</td><td>存放文件系统的管理信息，例如文件系统的大小，i节点区的大小</td></tr><tr><td>i节点区</td><td>存放指向文件存储区数据块的一些索引指针，即逻辑地址与物理地址的映射，文件类型，不包含文件名</td></tr><tr><td>文件存储区</td><td>存放文件数据，包含目录表</td></tr></tbody></table><p>i节点区</p><ul><li>i节点区由若干块构成，在 mkfs 命令 创建 文件系统时 确定</li><li>每块 可容 若干 个i节点，每个 i节点的大小是固定（比如 64kB)</li><li>i节点从0开始编号，根据编号可以索引到磁盘块</li><li>每个文件度对应1个i节点</li><li>i 节点包含的信息：<ul><li>指向文件存储区数据块的一些索引 指向文件存储区数据块的一些索引指针</li><li>文件类型，属组，权限，link数，大小，时戳，不包含文件名<br>注：</li></ul></li></ul><ol><li>目录表包含：文件名和i节点号</li><li>ls命令列出的目录大小为目录表文件本身的长度</li><li>目录表和i节点两级结构主要是为了提高目录检索的效率</li></ol><h3 id="软，硬连接"><a href="#软，硬连接" class="headerlink" title="软，硬连接"></a>软，硬连接</h3><p>硬连接：目录表中的“文件名-i节点号”映射关系，可以在同一目录或者不同目录中的两个文件项有相同的i节点号。</p><p>硬连接数目：同一i节点被目录项引用次数</p><p>创建硬连接： ln chapt0 intro</p><p>注：</p><ul><li>chapt0与intro同时存在时，地位平等</li><li>删除chapt0文件，则intro仍存在但link数减1</li><li>硬连接，只限于同一文件系统的普通文件</li></ul><p>符号连接：又叫做软连接</p><ul><li>文件仅包含一个路径名</li></ul><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">ln -s users_on sym.link </span><br><span class="line">ls -l sym.link</span><br></pre></td></tr></table></figure><p>符号连接中的相对路径：</p><ul><li>若符号连接文件包含绝对路径名，引用绝对路径名</li><li>若包含相对路径名，是相对符号连接文件的位置，而非调用进程的当前工作目录</li></ul><p>异同点：</p><ul><li>软连接在算法实现，可以在目录和不同的文件系统中使用</li></ul><h2 id="文件的权限"><a href="#文件的权限" class="headerlink" title="文件的权限"></a>文件的权限</h2><p>普通文件的权限：</p><ul><li><p>读、写、可执行，</p></li><li><p>不可写文件也可能被删除</p></li></ul><p>可执行文件：</p><ul><li><p>程序文件： 可执行文件</p></li><li><p>脚本文件： 文本文件， 第一行#！执行解释器，脚本运行时，是由解释程序创建一个进程</p></li></ul><h2 id="目录的权限"><a href="#目录的权限" class="headerlink" title="目录的权限"></a>目录的权限</h2><ul><li><p>读权限， 是说能否访问目录表文件</p></li><li><p>写权限，是说能否写目录表文件</p><ul><li>修改文件会修改i节点，不需要修改目录文件</li><li>目录无写权限不是指目录下所有文件禁止写</li><li>创建文件和删除文件，文件改名会修改目录文件</li><li>无写权限，目录表文件不许写</li></ul></li><li>执行权限： 意味着分析路径名过程中可检索该目录</li></ul><h2 id="修改权限"><a href="#修改权限" class="headerlink" title="修改权限"></a>修改权限</h2><p>chmod [ugoa][+-=][rwxst] 文件名表</p><p>u 文件主权限<br>g 同组用户权限<br>o 其他用户<br>a 所有</p><p>chmod 644 文件列表</p><p>注： 只允许文件主和超级 用户修改文件权限</p><p>unmask： 控制文件/目录的初始权限</p><p>unmask 打印当前值</p><p>unmask 022 将mask值设置为8进制的022，000 010 010 取消新文件和新目录的其他用户的w权限</p>]]></content>
<summary type="html">
<h1 id="week5"><a href="#week5" class="headerlink" title="week5"></a>week5</h1><h2 id="文件系统"><a href="#文件系统" class="headerlink" title="文件系统"
</summary>
<category term="linux" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/linux/"/>
</entry>
<entry>
<title>linuxweek8</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/14/linuxweek8/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/14/linuxweek8/</id>
<published>2019-02-14T15:06:53.000Z</published>
<updated>2019-02-17T07:18:54.657Z</updated>
<content type="html"><![CDATA[<h1 id="shell-之变量"><a href="#shell-之变量" class="headerlink" title="shell 之变量"></a>shell 之变量</h1><h2 id="内部变量之位置参数"><a href="#内部变量之位置参数" class="headerlink" title="内部变量之位置参数"></a>内部变量之位置参数</h2><table><thead><tr><th>变量</th><th>描述</th></tr></thead><tbody><tr><td>$0</td><td>脚本文件本身的名字</td></tr><tr><td>$1,$2,…,$n</td><td>n号命令行参数</td></tr><tr><td>$#</td><td>命令行参数的个数</td></tr><tr><td>“$*”</td><td>等同于”$1 $2 $3…”</td></tr><tr><td>“$@”</td><td>等同于”$1”,”$2”,”$3”,…</td></tr></tbody></table><p>内部的shift命令： 位置参数的移位操作，$#的值减一，旧的$2变为$1,…，也可以shift n</p><h2 id="自定义变量"><a href="#自定义变量" class="headerlink" title="自定义变量"></a>自定义变量</h2><ul><li><p>存储的内容：字符串，执行的过程中可以被修改。</p></li><li><p>变量名： 字母开头，其余字符可以是数字，字母，下划线</p></li><li><p>创建变量： name=1.12 等号两侧不能有空格</p></li><li><p>引用变量： $name 或者 ${name}</p></li></ul><p>当变量未定义时，变量值为空</p><table><thead><tr><th>shell内部一些开关</th><th>描述</th></tr></thead><tbody><tr><td>set -u</td><td>当引用一个未定义的变量时，产生一个错误</td></tr><tr><td>set +u</td><td>当引用一个未定义的变量时，认为是一个空串</td></tr><tr><td>set -x</td><td>执行命令时，打印执行的命令及参数，前面冠以+号</td></tr><tr><td>set +x</td><td>取消上述设置</td></tr></tbody></table><h2 id="环境变量和局部变量"><a href="#环境变量和局部变量" class="headerlink" title="环境变量和局部变量"></a>环境变量和局部变量</h2><ul><li><p>创建的变量，默认为局部变量</p></li><li><p>局部变量$\rightarrow$环境变量，使用export语句 export proto</p></li><li><p>局部变量： shell启动的子进程继承环境变量，不继承局部变量</p></li><li><p>子进程对环境变量的修改，不影响父进程的同名变量</p></li></ul><p>几个环境变量</p><ul><li>HOME： 用户主目录的路径名</li><li>PATH： 命令查找路径</li><li>TERM： 终端类型</li></ul><p>另外： set命令列出当前所有变量及其值以及函数的定义， env累出环境变量及其值。</p><h2 id="元字符"><a href="#元字符" class="headerlink" title="元字符"></a>元字符</h2><table><thead><tr><th>元字符</th><th>描述</th></tr></thead><tbody><tr><td>空格，制表符</td><td>命令行参数的分隔符</td></tr><tr><td>回车</td><td>执行键入的命令</td></tr><tr><td>>,<,</td><td></td><td>重定向与管道</td></tr><tr><td>；</td><td>用于一行内输入多个命令（;;）</td></tr><tr><td>&</td><td>后台运行(&&)</td></tr><tr><td>$</td><td>应用shell变量</td></tr><tr><td>`</td><td>反向单引号，用于命令替换</td></tr><tr><td>* [] ?</td><td>文件通配符</td></tr><tr><td>\</td><td>去向后继字符的特殊作用，转义</td></tr><tr><td>()</td><td>用于定义shell函数或在子shell中执行一组命令</td></tr></tbody></table><p>注： ()><|;&除本身意义外还可以起到分隔符的作用</p><h2 id="转义"><a href="#转义" class="headerlink" title="转义"></a>转义</h2><h3 id="双引号-“”"><a href="#双引号-“”" class="headerlink" title="双引号 “”"></a>双引号 “”</h3><ul><li>除$和`外特殊字符的特殊含义被取消，即仍可以使用命令替换和变量替换</li><li>需要的转义： \”, \$, `, \</li></ul><h3 id="单引号"><a href="#单引号" class="headerlink" title="单引号"></a>单引号</h3><ul><li><p>对所括起的任何字符，不作任何特殊解释</p></li><li><p>转义符代表斜杠本身，不许中间插入单引号</p></li></ul><h3 id="反撇号"><a href="#反撇号" class="headerlink" title="反撇号"></a>反撇号</h3><ul><li>\ 代表反斜线吱声</li><li>`代表反撇号本身</li></ul><h3 id="转义符后跟非特殊字符"><a href="#转义符后跟非特殊字符" class="headerlink" title="转义符后跟非特殊字符"></a>转义符后跟非特殊字符</h3><ul><li><p>引号内，系统处理时尽量维持字面含义</p></li><li><p>引号外，属于未定义的情况，最好不跟非特殊字符</p></li></ul><h2 id="shell-替换"><a href="#shell-替换" class="headerlink" title="shell 替换"></a>shell 替换</h2><ul><li><p>命令替换 name=<code>cmd</code> 或者name = $(cmd) ， 代表着name=执行cmd的标准输出，</p></li><li><p>变量替换 $name ${name}</p></li><li><p>文件名替换 ls *.c </p></li></ul><h2 id="辅助的命令"><a href="#辅助的命令" class="headerlink" title="辅助的命令"></a>辅助的命令</h2><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>read name</td><td>从标准输入读入一行内容赋值给变量</td></tr><tr><td>echo arg1 arg2 arg3</td><td>打印命令行参数，没两个间用1空格分开，最后打印换行符</td></tr><tr><td>echo -e arg1 arg2 arg3</td><td>除上面功能外，会打印转义符</td></tr><tr><td>printf ‘\033[01;33mConnect to %s Network\n’ $proto</td><td>打印</td></tr></tbody></table><p>注： </p><table><thead><tr><th>echo转义</th><th>描述</th></tr></thead><tbody><tr><td>\c</td><td>打印完毕，不换行</td></tr><tr><td>\b</td><td>退格</td></tr><tr><td>\n</td><td>换行</td></tr><tr><td>\r</td><td>回车</td></tr><tr><td>\t</td><td>水平制表</td></tr><tr><td>\</td><td>反斜线</td></tr><tr><td>\nnn</td><td>八进制描述的字符ASCII码</td></tr></tbody></table>]]></content>
<summary type="html">
<h1 id="shell-之变量"><a href="#shell-之变量" class="headerlink" title="shell 之变量"></a>shell 之变量</h1><h2 id="内部变量之位置参数"><a href="#内部变量之位置参数" class
</summary>
<category term="linux" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/linux/"/>
</entry>
<entry>
<title>linuxweek7</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/14/linuxweek7/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/14/linuxweek7/</id>
<published>2019-02-14T06:54:38.000Z</published>
<updated>2019-02-17T07:18:54.656Z</updated>
<content type="html"><![CDATA[<h1 id="shell"><a href="#shell" class="headerlink" title="shell"></a>shell</h1><h2 id="种类"><a href="#种类" class="headerlink" title="种类"></a>种类</h2><table><thead><tr><th>种类</th><th>描述</th></tr></thead><tbody><tr><td>B-shell</td><td>/bin/sh Stephen R.Bourne 于贝尔实验室开发</td></tr><tr><td>C-shell</td><td>/bin/csh, Bill Joy</td></tr><tr><td>K-shell</td><td>/bin/csh, korn shell, David Korn</td></tr><tr><td>Bash</td><td>/bin/bash, Bourne Again shell</td></tr></tbody></table><h2 id="功能"><a href="#功能" class="headerlink" title="功能"></a>功能</h2><ul><li><p>shell是命令解释器</p></li><li><p>文件名替换，命令替换，变量替换</p></li><li><p>历史替换，别名替换</p></li><li><p>流程控制的内部命令(内部命令和外部命令)</p></li><li><p>主要用途：批处理，执行效率比算法语言低</p></li></ul><p><code>学习bash的目的</code>: </p><ul><li><p>交互方式下：熟习shell的替换机制、转义机制，掌握循环等流程控制，<br>可以编写复合命令</p></li><li><p>非交互方式：编写shell脚本程序，把一系列的操作，编纂成一个脚本文<br>件，批量处理</p></li></ul><h2 id="具体功能"><a href="#具体功能" class="headerlink" title="具体功能"></a>具体功能</h2><ol><li>重定向与管道</li><li>方便交互使用的功能：历史替换与别名替换</li><li>shell变量</li><li>shell的变量替换，命令替换，文件名替换</li><li>元字符，如：单引号，双引号</li><li>流程控制</li><li>子程序</li></ol><h3 id="启动交互式shell"><a href="#启动交互式shell" class="headerlink" title="启动交互式shell"></a>启动交互式shell</h3><p>注册shell 和 交互式shell的区别？</p><p>启动方法</p><ul><li><p>注册shell： 注册shell</p></li><li><p>键入bash 命令： 交互式shell</p></li><li><p>脚本解释器</p></li></ul><p>自动执行一批命令(用户偏好)：</p><ul><li><p>注册shell时：自动执行用户主目录下的.bash_profile文件中的命令</p></li><li><p>bash作为注册shell退出时： 自动执行$HOME/.bash_logout</p></li><li><p>bash作为注册shell启动时： 自动执行$HOME/.bashrc</p></li></ul><p>自动执行一批命令(系统级)：</p><ul><li><p>当bashbash bash作为注册shell被启动时:自动执行 /etc/profile 文件中命令</p></li><li><p>当bash 作为交互式shell启动时： 自动执行 /etc/bash.bashrc</p></li><li><p>当bash作为注册shell退出时：自动执行/etc/bash.bash.bash_logout</p></li></ul><h2 id="历史与别名"><a href="#历史与别名" class="headerlink" title="历史与别名"></a>历史与别名</h2><h3 id="历史表"><a href="#历史表" class="headerlink" title="历史表"></a>历史表</h3><p>查看历史表</p><p>history</p><p>设置历史表的大小：</p><p>修改变量HISTSIZE</p><p>历史替换：</p><ul><li><p>人机交互时，直接使用上下箭头</p></li><li><p>！！ 引用上一条命令</p></li><li><p>！str 以str开头的最近用过的命令</p></li></ul><h3 id="别名"><a href="#别名" class="headerlink" title="别名"></a>别名</h3><p>创建别名：alias h=”alias”， 注意等号两端无空格</p><p>查看别名表： alias</p><p>取消别名： unalias h</p><h3 id="Tab键补全"><a href="#Tab键补全" class="headerlink" title="Tab键补全"></a>Tab键补全</h3><ul><li><p>每行的首个单词： 全搜索$PATH下的命令</p></li><li><p>行中的其他单词： 当前目录下的文件名</p></li></ul><h2 id="重定向"><a href="#重定向" class="headerlink" title="重定向"></a>重定向</h2><h3 id="输入"><a href="#输入" class="headerlink" title="输入"></a>输入</h3><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td><filename</td><td>从filename中获取stdin</td></tr><tr><td><<word</td><td>从shell脚本文件中获取数据直到再次遇到定界符word ？？？？</td></tr><tr><td><<<word</td><td>从命令行获取信息作为标准输入</td></tr></tbody></table><h3 id="输出"><a href="#输出" class="headerlink" title="输出"></a>输出</h3><table><thead><tr><th>fd</th><th>描述</th></tr></thead><tbody><tr><td>0</td><td>标准输入， stdin</td></tr><tr><td>1</td><td>标准输出，stdout</td></tr><tr><td>2</td><td>标准错误输出， stderr</td></tr></tbody></table><table><thead><tr><th></th><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>>filename</td><td>将stdout重定向到文件 filename,文件已存在则先清空（ 覆盖方式</td></tr><tr><td>>>filename</td><td>将stdout重定向 追加 到文件 filename尾</td></tr><tr><td>2> filename</td><td>将文件句柄2重定向到文件 filename</td></tr><tr><td>2>&1</td><td>将文件句柄 2重定向到文件描述符 1</td></tr><tr><td>cmd1</td><td>cmd2</td><td>将cmd1的标准输出作为cmd2的标准输入</td></tr></tbody></table>]]></content>
<summary type="html">
<h1 id="shell"><a href="#shell" class="headerlink" title="shell"></a>shell</h1><h2 id="种类"><a href="#种类" class="headerlink" title="种类"></a>种
</summary>
<category term="linux" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/linux/"/>
</entry>
<entry>
<title>tensorflow-tutorial-11</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/11/tensorflow-tutorial-11/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/11/tensorflow-tutorial-11/</id>
<published>2019-02-11T02:53:52.000Z</published>
<updated>2019-02-17T07:18:54.696Z</updated>
<content type="html"><![CDATA[<h1 id="导入数据"><a href="#导入数据" class="headerlink" title="导入数据"></a>导入数据</h1><h2 id="tf-data-Dataset"><a href="#tf-data-Dataset" class="headerlink" title="tf.data.Dataset"></a>tf.data.Dataset</h2><h3 id="属性"><a href="#属性" class="headerlink" title="属性"></a>属性</h3><table><thead><tr><th>属性</th><th>描述</th></tr></thead><tbody><tr><td>output_classes</td><td>返回数据集中每个元素的每个组成部分的类名</td></tr><tr><td>output_shapes</td><td>返回数据集中每个元素的每个组成部分的形状</td></tr><tr><td>output_types</td><td>返回数据集中每个元素的每个组成部分的数据类型</td></tr></tbody></table><h3 id="方法"><a href="#方法" class="headerlink" title="方法"></a>方法</h3><h4 id="创建数据集"><a href="#创建数据集" class="headerlink" title="创建数据集"></a>创建数据集</h4><table><thead><tr><th>方法</th><th>描述</th></tr></thead><tbody><tr><td>from_tensor_slices(tensors)</td><td>建立给定tensors的数据集，多个元素，该数据集嵌入在图中，适用于小数据集（小于1G),更大的数据集<a href="https://www.tensorflow.org/guide/datasets#consuming_numpy_arrays" target="_blank" rel="noopener">操作</a></td></tr><tr><td>from_tensors(tensors)</td><td>同上，但针对单个元素</td></tr></tbody></table><h4 id="基于数据集上的变换"><a href="#基于数据集上的变换" class="headerlink" title="基于数据集上的变换"></a>基于数据集上的变换</h4><table><thead><tr><th>方法</th><th>描述</th></tr></thead><tbody><tr><td>map(map_func,num_parallel_calls=None)</td><td>对数据集中的每个元素执行map_func操作</td></tr><tr><td>shuffle(buffer_size,seed=None,reshuffle_each_iteration=None)</td><td>随机抽取元素</td></tr><tr><td>batch(batch_size,drop_remainder=False)</td><td>设置batch的大小</td></tr><tr><td>repeat(count=None)</td><td>将数据集重复几次，默认为无限循环</td></tr><tr><td>zip(datasets)</td><td>datasets 元组，将多个数据集合并</td></tr></tbody></table><h4 id="创建迭代器"><a href="#创建迭代器" class="headerlink" title="创建迭代器"></a>创建迭代器</h4><table><thead><tr><th>方法</th><th>描述</th></tr></thead><tbody><tr><td>make_initializable_iterator(shared_name=None)</td><td>创建一个未初始化的iterator</td></tr><tr><td>make_one_shot_iterator()</td><td>创建一个已经自动初始化的iterator， 每次只返回单个元素</td></tr></tbody></table><h2 id="tf-data-Iterator"><a href="#tf-data-Iterator" class="headerlink" title="tf.data.Iterator"></a>tf.data.Iterator</h2><h3 id="属性-1"><a href="#属性-1" class="headerlink" title="属性"></a>属性</h3><table><thead><tr><th>属性</th><th>描述</th></tr></thead><tbody><tr><td>initializer</td><td>返回初始化迭代器的操作</td></tr><tr><td>output_classes</td><td>返回数据集中每个元素的每个组成部分的类名</td></tr><tr><td>output_shapes</td><td>返回数据集中每个元素的每个组成部分的形状</td></tr><tr><td>output_types</td><td>返回数据集中每个元素的每个组成部分的数据类型</td></tr></tbody></table><h3 id="方法-1"><a href="#方法-1" class="headerlink" title="方法"></a>方法</h3><h4 id="创建迭代器-1"><a href="#创建迭代器-1" class="headerlink" title="创建迭代器"></a>创建迭代器</h4><table><thead><tr><th>方法</th><th>描述</th></tr></thead><tbody><tr><td>from_string_handle(string_handle,output_types)</td><td>基于给定的句柄创建一个未初始化的迭代器</td></tr><tr><td>from_structure(output_types)</td><td>基于给定的结构创建一个未初始化的迭代器</td></tr></tbody></table><p>注： from_structure可以不同的数据集进行复用（reusable，Iterator.make_initializer(dataset)重新初始化，from_string_handle “feedable” iterator 。</p><p>区别：</p><h4 id="常用方法"><a href="#常用方法" class="headerlink" title="常用方法"></a>常用方法</h4><table><thead><tr><th>方法</th><th>描述</th></tr></thead><tbody><tr><td>get_next(name=None)</td><td>返回下一个数据集中的元素</td></tr><tr><td>make_initializer(dataset,name=None)</td><td>返回通过指定的dataset初始化iterator的操作</td></tr><tr><td>string_handle(name=None)</td><td>返回代表迭代器的string</td></tr></tbody></table><p>注：make_initializer 与 from_structure 连用， string_handle与from_string_handle连用。get_next抵达数据集最后一个元素会有<code>tf.errors.OutOfRangeError错误</code>。</p><h1 id="References"><a href="#References" class="headerlink" title="References"></a>References</h1><ol><li><a href="https://tensorflow.google.cn/api_docs/python/tf/data/Dataset#shuffle" target="_blank" rel="noopener">tf.data.Dataset</a></li><li><a href="https://www.tensorflow.org/api_docs/python/tf/data/Iterator" target="_blank" rel="noopener">tf.data.Iterator</a></li></ol>]]></content>
<summary type="html">
<h1 id="导入数据"><a href="#导入数据" class="headerlink" title="导入数据"></a>导入数据</h1><h2 id="tf-data-Dataset"><a href="#tf-data-Dataset" class="header
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>linuxweek4</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/07/linuxweek4/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/07/linuxweek4/</id>
<published>2019-02-07T06:52:24.000Z</published>
<updated>2019-02-17T07:18:54.653Z</updated>
<content type="html"><![CDATA[<h1 id="文件管理"><a href="#文件管理" class="headerlink" title="文件管理"></a>文件管理</h1><h2 id="列出当前文件"><a href="#列出当前文件" class="headerlink" title="列出当前文件"></a>列出当前文件</h2><p><code>ls [optional] [file/dir]</code></p><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>ls</td><td>列出当前目录下所有文件和目录</td></tr><tr><td>ls file</td><td>列出文件项</td></tr><tr><td>ls dir</td><td>列出目录下的所有文件项</td></tr></tbody></table><table><thead><tr><th>选项</th><th>描述</th></tr></thead><tbody><tr><td>-F</td><td>列出的如果是目录则在名字后以斜线/表示，可执行文件*，符号连接文件@，普通文件无标记</td></tr><tr><td>-l</td><td>以长格式显示信息</td></tr><tr><td>-a</td><td>列出文件名首字符为圆点的文件，默认情况下不列出</td></tr><tr><td>-A</td><td>与-a类似，不列出.和..</td></tr><tr><td>-s</td><td>列出文件占用的磁盘空间</td></tr><tr><td>-i</td><td>列出文件的i节点号</td></tr><tr><td>-h</td><td>以便于人阅读的方式打印数值</td></tr><tr><td>-d</td><td>当ls参数是目录时，列出目录自身信息</td></tr></tbody></table><h3 id="长格式列表"><a href="#长格式列表" class="headerlink" title="长格式列表"></a>长格式列表</h3><p><code>input</code>: ls -l arg<br><code>output</code>: -rwxr-x–x 1 liang stud 519 Jul 5 15:02 arg</p><ul><li>第一列：文件属性: 文件类型+文件的访问权限(rwx)（用户，与用户同组的用户，与该用户不同组的用户）</li></ul><table><thead><tr><th>符号</th><th>文件类型</th></tr></thead><tbody><tr><td>-</td><td>普通文件</td></tr><tr><td>d</td><td>目录</td></tr><tr><td>l</td><td>符号连接文件</td></tr><tr><td>b</td><td>块设备文件</td></tr><tr><td>c</td><td>字符设备文件</td></tr><tr><td>p</td><td>命名管道文件</td></tr></tbody></table><ul><li><p>第二列: 文件link数，涉及到此文件的目录项数</p></li><li><p>第3,4列： 用户名，该用户所在的组</p></li><li><p>第5列 </p></li></ul><table><thead><tr><th>文件类别</th><th>描述</th></tr></thead><tbody><tr><td>普通文件</td><td>列出文件大小</td></tr><tr><td>目录</td><td>列出目录表大小</td></tr><tr><td>符号连接文件</td><td>列出符号连接文件自身的长度</td></tr><tr><td>字符设备和块设备</td><td>列出主次设备号</td></tr><tr><td>管道文件</td><td>列出管道的数据长度</td></tr></tbody></table><ul><li><p>第6列： 文件最后一次被修改的日期和时间</p></li><li><p>第7列： 文件名，对于符号连接文件，附带列出符号连接文件的内容</p></li></ul><h2 id="文件的复制与删除"><a href="#文件的复制与删除" class="headerlink" title="文件的复制与删除"></a>文件的复制与删除</h2><h3 id="拷贝文件"><a href="#拷贝文件" class="headerlink" title="拷贝文件"></a>拷贝文件</h3><p>cp file1 file2 或者 cp file1 file2… filen dir</p><ul><li>第二种格式，dir必须存在并且是目录</li><li>第一种格式，file2不存在，则创建；file2存在且是文件，则覆盖，若是目录则按格式2处理。</li></ul><h3 id="移动文件"><a href="#移动文件" class="headerlink" title="移动文件"></a>移动文件</h3><p>mv file1 file2 or mv file1 file2…filen dir or mv dir1 dir2</p><ul><li>使用mv命令可以将文件和目录改名</li><li>可以将文件和从子目录从一个目录移动到另一个目录</li></ul><h3 id="删除文件"><a href="#删除文件" class="headerlink" title="删除文件"></a>删除文件</h3><p>rm file1 file2…filen</p><table><thead><tr><th>选项</th><th>描述</th></tr></thead><tbody><tr><td>-r</td><td>recursive 递归地删除实参表中的目录</td></tr><tr><td>-i</td><td>inform 没删除一个文件需要操作员确认</td></tr><tr><td>-f</td><td>force 强迫删除，只读文件也被删除并且无提示</td></tr></tbody></table><h3 id="文件打包命令"><a href="#文件打包命令" class="headerlink" title="文件打包命令"></a>文件打包命令</h3><p>tar ctxv [f device] file-list</p><table><thead><tr><th>选项</th><th>描述</th></tr></thead><tbody><tr><td>-c</td><td>create 创建新磁带，以前创建的会被覆盖</td></tr><tr><td>-t</td><td>table 磁带的文件名列表。当不指定文件名时，将列出所有文件</td></tr><tr><td>-x</td><td>eXtract 从磁带中抽取指定的文件，当不指定文件名时，抽取所有文件</td><td>-v</td><td>verbose 每处理一个文件，就打印出文件的文件名，并在该文件名前冠以功能字母</td></tr><tr><td>f</td><td>File指定设备文件名</td></tr><tr><td>z</td><td>gzip 采用gzip压缩格式</td></tr><tr><td>j</td><td>采用bzip2算法</td></tr></tbody></table><p>e.g. tar cvf my.tar *.[ch] makefile</p><p>e.g. tar czvf my.tar.gz *[ch] makefile</p><p>e.g. tar cjvf my.tar.bz2 *[ch] makefile</p><p>其他压缩/解压命令：</p><ul><li>gzip/gunzip 执行速度快</li><li>bzip2/bunzip2 占用较多的cpu时间</li></ul><h2 id="目录操作"><a href="#目录操作" class="headerlink" title="目录操作"></a>目录操作</h2><h3 id="打印-改变当前目录"><a href="#打印-改变当前目录" class="headerlink" title="打印/改变当前目录"></a>打印/改变当前目录</h3><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>pwd</td><td>print work directory 打印当前工作目录</td></tr><tr><td>cd dir</td><td>切换至dir</td></tr><tr><td>cd</td><td>windows 打印当前目录，linux回到用户主目录</td></tr><tr><td>cd ..</td><td>回到上层目录</td></tr></tbody></table><h3 id="创建-删除目录"><a href="#创建-删除目录" class="headerlink" title="创建/删除目录"></a>创建/删除目录</h3><p>mkdir sun/work.d</p><ul><li>-p 自动创建路径不存在的目录 mkdir -p database/2019/09/04/log</li></ul><p>rmdir sum/work.d</p><ul><li>要求被删除的目录除.与..外无其他文件或目录</li><li>-r 递归的删除目录及其子目录？</li></ul><h3 id="复制目录"><a href="#复制目录" class="headerlink" title="复制目录"></a>复制目录</h3><p>cp -r dir1 dir2</p><ul><li>dir2 不存在，新建目录dir2，把dir1下内容拷贝至dir2下</li><li>dir2已存在，在dir2目录下新建dir1，把dir1内容拷贝纸dir2/dir1下</li><li>-v 实时显示正在复制的文件的名字</li><li>-u 增量拷贝，便于备份目录， 根据文件的时间戳，不拷贝相同的或者过时的版本文件，以提高拷贝速度。</li></ul><h3 id="touch"><a href="#touch" class="headerlink" title="touch"></a>touch</h3><p>将文件最后一次修改时间设置为当前时间，但不改变文件。 touch *.[ch]</p><h3 id="目录遍历-find"><a href="#目录遍历-find" class="headerlink" title="目录遍历 find"></a>目录遍历 find</h3><ul><li>功能： 从指定的查找范围开始，递归地查找子目录，凡满足条件的文件或者目录，执行规定的动作。</li><li>动作</li></ul><table><thead><tr><th>动作</th><th>描述</th></tr></thead><tbody><tr><td>-print</td><td>打印查找的文件的路径名</td></tr><tr><td>-exec</td><td>对查找的目标执行某一命令 在-exec及随后的分号之间的内容作为一条命令，{}代表遍历道德目标文件的路径名</td></tr><tr><td>-ok</td><td>只是对查找到符合条件的目标执行前需要管理人员确认</td></tr><tr><td>xxx</td><td>自定义命令</td></tr></tbody></table><ul><li>条件</li></ul><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>-name wildcard</td><td>文件名与wildcare匹配，必须引号，文件名指的是路径名的<code>最后一部分</code>，对通配符的解释由find完成</td></tr><tr><td>-regex pattern</td><td><code>整个路径名</code>与正则表达式pattern匹配</td></tr><tr><td>-type f/d/l/c/b/p</td><td>文件类型</td></tr><tr><td>-size +， ， - N</td><td>文件大小，+ 大于，- 小于， 单位b,k,M</td></tr><tr><td>-mtime +, ,- ndays</td><td>文件最近修改时间</td></tr><tr><td>-newer file</td><td>文件修改时间比file的修改时间还晚</td></tr></tbody></table><ul><li><p>() -o ! 表示多条件的“与”，“或”，”非”</p></li><li><p>示例： find verl.d ver2.d -name ‘*.c’ -print</p><ul><li>范围： 当前目录的子目录ver1.d和ver2.d</li><li>条件： 与名字<em>.c匹配。注：</em>.c应当用引号括起来</li><li>动作： 把查找的文件的路径打印出来</li></ul></li></ul>]]></content>
<summary type="html">
<h1 id="文件管理"><a href="#文件管理" class="headerlink" title="文件管理"></a>文件管理</h1><h2 id="列出当前文件"><a href="#列出当前文件" class="headerlink" title="列出当前文
</summary>
<category term="linux" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/linux/"/>
</entry>
<entry>
<title>linux之vi编辑器</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/06/linuxweek3/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/06/linuxweek3/</id>
<published>2019-02-06T14:57:11.000Z</published>
<updated>2019-02-17T07:18:54.650Z</updated>
<content type="html"><![CDATA[<p>vi编辑器有两种状态：</p><ul><li>命令状态：键盘输入解释为命令<ul><li>vi 一启动就进入命令行模式</li><li>以冒号可以引入行编辑的命令和查找命令</li><li>编辑命令i(insert, 当前字符前)，a(append，当前字符后)，可以从命令状态转到文本状态 </li></ul></li><li>文本状态<ul><li>键盘输入解释为输入的文本</li><li>可以输入多行</li><li>有回显</li><li>按Esc键，返回到命令状态</li></ul></li></ul><p><img src="1.png"></p><h1 id="常见命令"><a href="#常见命令" class="headerlink" title="常见命令"></a>常见命令</h1><h2 id="光标移动"><a href="#光标移动" class="headerlink" title="光标移动"></a>光标移动</h2><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>h</td><td>左移1列</td></tr><tr><td>j</td><td>下移1行</td></tr><tr><td>k</td><td>上移1行</td></tr><tr><td>l</td><td>右移1列</td></tr><tr><td>数值n+h，j,k,l</td><td>对应操作执行n次</td></tr><tr><td>ctrl+b</td><td>向后翻页</td></tr><tr><td>ctrl+f</td><td>向前翻页</td></tr><tr><td>^</td><td>将光标移至行首</td></tr><tr><td>$</td><td>将光标移至行尾</td></tr><tr><td>w</td><td>右移1个单词</td></tr><tr><td>b</td><td>左移1个单词</td></tr><tr><td>:476</td><td>移动至476行</td></tr><tr><td>:1</td><td>移动首行</td></tr><tr><td>:$</td><td>移动至文件末尾</td></tr></tbody></table><h2 id="编辑"><a href="#编辑" class="headerlink" title="编辑"></a>编辑</h2><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>x</td><td>删除当前字符</td></tr><tr><td>dd</td><td>删除行</td></tr><tr><td>u</td><td>取消上一次编辑</td></tr><tr><td>.</td><td>重复上一次编辑操作</td></tr><tr><td>:n1,n2d</td><td>删除第n1-n2行</td></tr><tr><td>:1,.d</td><td>删除首行至当前行</td></tr><tr><td>:.,$d</td><td>删除当前行至文件尾</td></tr><tr><td>:10,50y</td><td>拷贝10-50行至剪贴板（yank)</td></tr><tr><td>p</td><td>粘贴剪贴板信息</td></tr><tr><td>ra</td><td>将当前光标处字符替换为a</td></tr></tbody></table><h2 id="保存"><a href="#保存" class="headerlink" title="保存"></a>保存</h2><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>:wq</td><td>写入并保存</td></tr><tr><td>:w</td><td>写入不退出</td></tr><tr><td>:q!</td><td>不存盘退出</td></tr></tbody></table><h2 id="块操作"><a href="#块操作" class="headerlink" title="块操作"></a>块操作</h2><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>:5,10co56</td><td>复制第5-10行至56行之下</td></tr><tr><td>:8,34m78</td><td>移动8-34行至第78行之下</td></tr><tr><td>J</td><td>当前行和下面行合并至当前行</td></tr><tr><td>ctrl + l</td><td>刷新屏幕显示</td></tr><tr><td>ctrl+g</td><td>状态显示</td></tr></tbody></table><h2 id="查找与替换"><a href="#查找与替换" class="headerlink" title="查找与替换"></a>查找与替换</h2><table><thead><tr><th>命令</th><th>描述</th></tr></thead><tbody><tr><td>/pattern</td><td>查找匹配该模式的项</td></tr><tr><td>n</td><td>向下查找下一个</td></tr><tr><td>N</td><td>向上查找下一个</td></tr><tr><td>:n1,n2s/pattern/string/g</td><td>替换成string</td></tr></tbody></table><h1 id="几个问题"><a href="#几个问题" class="headerlink" title="几个问题"></a>几个问题</h1><h2 id="死机"><a href="#死机" class="headerlink" title="死机"></a>死机</h2><ul><li>现象：vi 编辑结束后执行存盘操作，果导致屏幕 卡死，输入任何信息都不在有显示</li><li>原因： 编辑结束后按下 Ctrl-S，因为 WindowsWindows 编辑器一般设置 Ctrl-S热键的动作为Save ，但Linux 却进入流量控制状态。</li><li>解决方法： 按下Ctrl-Q键后流量控制解除</li></ul><h2 id="意外中止问题"><a href="#意外中止问题" class="headerlink" title="意外中止问题"></a>意外中止问题</h2><ul><li><p>现象： vi编辑结束后存盘，程序意外中止成果丢失文件内容未发生变化 </p></li><li><p>原因： vi存盘命令Shift-ZZ，误操作为Ctrl-ZZ，而Ctrl-Z按键导致当前运行进程被挂起 （suspend ），暂停运行（但进程尚在，处于stopped状态）</p></li><li><p>解决方式： 调用bash的作业管理机制，恢复运行被stopped的进程</p><ul><li><p>jobs 列表当前被Stopped的进程有哪些</p></li><li><p>fg %1 将1号作业恢复到前台运行</p></li><li><p>%1 将1号作业恢复到前台运行</p></li></ul></li></ul><h2 id="屏幕显示乱码问题"><a href="#屏幕显示乱码问题" class="headerlink" title="屏幕显示乱码问题"></a>屏幕显示乱码问题</h2><ul><li>现象： cat /bin/bash 或head -n 1 /bin/bash 导致屏幕乱码，键盘输入也只能看到画表用的框框符</li><li>原因：/bin/bash /bin/bash /bin/bash文件不是本，这些二进制内容逐字节送到终端，凑巧一个序列被理解 终端，凑巧一个序列被理解 终端，凑巧一个序列被理解 为一个转义序列，执行动作：修改了终端字符集</li><li>解决方式：断开连接，重新连接，注意不要把非文本信息在终端输出</li></ul><h2 id="文本文件格式问题"><a href="#文本文件格式问题" class="headerlink" title="文本文件格式问题"></a>文本文件格式问题</h2><ul><li>现象： Linux下的文本件在 Windows用notepad打开，所有内容粘在一行内</li><li>原因： Linux和Windows的文本文件的存储格式不同， windows行尾为回车和换行\r\n，linux为\n</li><li>解决方式： 在linux完成格式转化后再用windows处理<ul><li>dos2unix/unix2dos, todos/frodos</li></ul></li></ul><h2 id="中文编码配置问题"><a href="#中文编码配置问题" class="headerlink" title="中文编码配置问题"></a>中文编码配置问题</h2><ul><li>现象： 从windows迁来的文件，只有在显示中文字符时是乱码。Linux本身的正常显示</li><li>原因： 中文GBK与UTF8不兼容<ul><li>7比特 ASCII码，字节高位为0,后面7位是英文ASCII码</li><li>gbk , 两个字节表示一个汉字，字节的高位为1以区分ASCII码</li></ul></li><li><p>解决方式： </p><ul><li><p>检查系统设置： export LANG=en_US.UTF-8</p></li><li><p>iconv:中文字符编码的转化： iconv -f gbk -t utf8</p></li></ul></li></ul>]]></content>
<summary type="html">
<p>vi编辑器有两种状态：</p>
<ul>
<li>命令状态：键盘输入解释为命令<ul>
<li>vi 一启动就进入命令行模式</li>
<li>以冒号可以引入行编辑的命令和查找命令</li>
<li>编辑命令i(insert, 当前字符前)，a(append，当前字符后)，
</summary>
<category term="linux" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/linux/"/>
</entry>
<entry>
<title>tensorflow_tutorial_10</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-10/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-10/</id>
<published>2019-02-02T17:03:10.000Z</published>
<updated>2019-02-17T07:18:54.694Z</updated>
<content type="html"><![CDATA[<h1 id="梯度下降"><a href="#梯度下降" class="headerlink" title="梯度下降"></a>梯度下降</h1><p>损失函数$J(\theta, x)$是一个关于网络参数$\theta$和样本$x$二元函数，它是一个<a href="https://baike.so.com/doc/1434644-1516473.html" target="_blank" rel="noopener">凸函数</a>，表示了用网络参数$\theta$来预测与真实结果的偏差大小。神经网络的目标是使得预测结果与真实结果尽可能一致，这句话可以等价于：找到一组网络参数$\theta$使得loss function最小。求解这一组网络参数$\theta$的方法之一是梯度下降法：</p><p>$$\theta=\theta-\alpha\frac{\partial J(\theta, x)}{\partial\theta}$$</p><p><img src="1.png"></p><p>其中，$\alpha$, 术语叫做学习率，也就是说每一步参数$\theta$更新步长的大小，$\frac{\partial J(\theta, x)}{\partial\theta}$为$J(\theta, x)$对参数$\theta$的偏导，反映了最快变化量。另一方面，由于采用的下降法，所以是负号。</p><p>直观的理解：以图2为例：</p><ul><li><p>对于P点，它的$\frac{\partial J(\theta, x)}{\partial\theta}< 0$, 因为使得损失越小，所以$\theta$应该增大，对应于$-\frac{\partial J(\theta, x)}{\partial\theta}>0$。</p></li><li><p>对于Q点，它的$\frac{\partial J(\theta, x)}{\partial\theta}>0$, 因为使得损失越小，所以$\theta$应该减小，对应于$-\frac{\partial J(\theta, x)}{\partial\theta}<0$。</p></li></ul><p><img src="2.png"></p><p>关于学习率$\alpha$,</p><ul><li><p>设置过大，如图3中的红线部分，$\alpha$ 比较大 \Rightarrow$ 每一步走的比较多 \Rightarrow$ 优点：容易走出极小值点 缺点：易出现震荡，可能无法收敛至最小值点。</p></li><li><p>设置过小，如图3中的蓝线部分，$\alpha$ 比较小 \Rightarrow$ 每一步走的比较小 \Rightarrow$ 优点：比较细粒度 缺点：比较难走出极小值点。</p></li></ul><p>总结来说，学习率$\alpha$一般起初可设置的比较大，随着训练，逐步减小。</p><p><img src="3.png"></p><h1 id="从链式法则看-batch-normalization"><a href="#从链式法则看-batch-normalization" class="headerlink" title="从链式法则看 batch normalization"></a>从链式法则看 batch normalization</h1><p><img src="4.png"></p><p>以图4为例，该网络由3个子系统$h_1, h_2, h_3$组成, 假设$h_1, h_2, h_3$均为无激活函数全连接系统。</p><p>其框架为：x $\rightarrow$ <code>$h_1$</code> $\rightarrow$ <code>$h_2$</code> $\rightarrow$ <code>$h_3$</code> $\rightarrow$ <code>$\hat{y}$</code>.</p><p>由图4可以写出以下公式：</p><p>$$y_1=w_1 \cdot x+b$$</p><p>$$y_2=w_2 \cdot y_1+b$$</p><p>$$y_3=w_3 \cdot y_2+b$$</p><p>$$\hat{y} = y_3$$</p><p>采用$L_2$损失函数为：</p><p>$$L = \frac{1}{2}(\hat{y}-y)^2$$</p><p>每层的$w$参数的偏导是：<br>\begin{align}<br>\frac{\partial L}{\partial w_1} &= \frac{\partial L}{ \partial \hat{y}} \cdot \frac{\partial \hat{y}}{ \partial y_3} \cdot \frac{\partial y_3}{ \partial y_2}\cdot \frac{\partial y_2}{ \partial y_1}\cdot \frac{\partial y_1}{ \partial w_1} \\<br>&= (\hat{y}-y) \cdot 1 \cdot w_3 \cdot w_2 \cdot x<br>\end{align}</p><p>\begin{align}<br>\frac{\partial L}{ \partial w_2} &= \frac{\partial L}{ \partial \hat{y}} \cdot \frac{\partial \hat{y}}{ \partial y_3} \cdot \frac{\partial y_3}{ \partial y_2}\cdot \frac{\partial y_2}{ \partial w_2} \\<br>&= (\hat{y}-y) \cdot 1 \cdot w_3 \cdot y_1<br>\end{align}</p><p>\begin{align}<br>\frac{\partial L}{ \partial w_2} &= \frac{\partial L}{ \partial \hat{y}} \cdot \frac{\partial \hat{y}}{ \partial y_3} \cdot \frac{\partial y_3}{ \partial y_3}\\<br>&= (\hat{y}-y) \cdot 1 \cdot y_2<br>\end{align}</p><p>如果对于有n个子系统的神经网络(多层) 从1计数，每层的参数的偏导为：</p><p>$$\frac{\partial L}{ \partial w_i} = \prod_{k=n}^{i+1} w_k \cdot y_{i-1} $$</p><p>从该表达式可以发现参数的偏导与$\prod_{k=n}^{i+1} w_k$ 以及前一层的输出$y_{i-1}$有关。</p><p>梯度弥散：</p><ul><li><p>梯度过小： 假设每个子系统为一个缩放器，即w<1。 那么，当网络比较深的时候，对于比较深的层来说，其$\prod_{k=n}^{i+1} w_k$不会太小，但是$y_{i-1}$会由于输入经过多个缩放器，变得非常小，最终梯度也变得非常小。对于浅层网络，$y_{i-1}$缩放的程度不是很大，但是$\prod_{k=n}^{i+1} w_k$非常小，最终梯度也变得非常小。</p></li><li><p>梯度过大： 假设每个子系统为一个放大器，即w>1。 那么，当网络比较深的时候，对于比较深的层来说，其$\prod_{k=n}^{i+1} w_k$不会太大，但是$y_{i-1}$会由于输入经过多个放大器，变得非常大，最终梯度也变得非常大。对于浅层网络，$\prod_{k=n}^{i+1} w_k$放大的程度不是很大，但是$\prod_{k=n}^{i+1} w_k$非常大，最终梯度也变得非常大。</p></li></ul><p>如何解决梯度弥散：</p><p>从上面可以知道梯度的影响是$\prod_{k=n}^{i+1} w_k$和$\prod_{k=n}^{i+1} w_k$。</p><p>所以解决方式从两方面考虑：</p><ul><li><p>w的初始化： 取值尽可能的适中</p></li><li><p>对每层的输出做归一化处理(batch normalization)，去掉直流信号，仍然保留信息。另外， 归一化处理(batch normalization) 对深层求梯度是有帮助的，但对于浅层求梯度求梯度起到的作用比较小。</p></li></ul>]]></content>
<summary type="html">
<h1 id="梯度下降"><a href="#梯度下降" class="headerlink" title="梯度下降"></a>梯度下降</h1><p>损失函数$J(\theta, x)$是一个关于网络参数$\theta$和样本$x$二元函数，它是一个<a href="htt
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>tensorflow_tutorial_9</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-9/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-9/</id>
<published>2019-02-02T17:02:40.000Z</published>
<updated>2019-02-17T07:18:54.707Z</updated>
<content type="html"><![CDATA[<h1 id="batch-Normalization"><a href="#batch-Normalization" class="headerlink" title="batch Normalization"></a>batch Normalization</h1><h2 id="定义"><a href="#定义" class="headerlink" title="定义"></a>定义</h2><p>将均值为$\mu$, 标准差$\sigma$的数据集合$x$，转化为均值为$\beta$, 标准差$\gamma$的数据集合$x^{’}$：</p><p>$$x^{’}=\gamma\frac{(x-\mu)}{\sigma}+\beta$$</p><h2 id="作用"><a href="#作用" class="headerlink" title="作用"></a>作用</h2><p>解决梯度弥散的问题</p><p><a href="https://blog.csdn.net/alxe_made/article/details/80834305" target="_blank" rel="noopener">padding的方式</a></p><h2 id="实现步骤"><a href="#实现步骤" class="headerlink" title="实现步骤"></a>实现步骤</h2><ul><li>计算数据集合$x$的均值为$\mu$, 标准差$\sigma$</li><li>套用变化公式</li></ul><h2 id="问题："><a href="#问题：" class="headerlink" title="问题："></a>问题：</h2><ul><li>为什么采用滑动平均的方式来求解均值和方差？</li></ul><p>先说结论：并不是测试时的mean,var的计算方式与训练时不同，而是测试时的mean,var在训练完成整个网络中就全部固定了。由于在优化网络的时候，我们一般采用的是batch梯度下降。所以在训练过程中，只能计算当前batch样本上的mean和var。但是我们做的normalization是对于整个输入样本空间，因此需要对每个batch的mean, var做指数加权平均来将batch上的mean和var近似成整个样本空间上的mean和var.而在测试Inference过程中，一般不必要也不合适去计算测试时的batch的mean和var，比如测试仅对单样本输入进行测试时，这时去计算单样本输入的mean和var是完全没有意义的。因此会直接拿训练过程中对整个样本空间估算的mean和var直接来用。此时对于inference来说，BN就是一个线性变换。</p><ul><li>起始值为什么是 0 1？</li></ul><p>说白了就是，怎么去估计数据的真实分布？</p><p><a href="https://www.cnblogs.com/34fj/p/8805979.html" target="_blank" rel="noopener">https://www.cnblogs.com/34fj/p/8805979.html</a></p><h2 id="Tensorflow-实现"><a href="#Tensorflow-实现" class="headerlink" title="Tensorflow 实现"></a>Tensorflow 实现</h2><p>Note: when training, the moving_mean and moving_variance need to be updated. By default the update ops are placed in tf.GraphKeys.UPDATE_OPS, so they need to be added as a dependency to the train_op. Also, be sure to add any batch_normalization ops before getting the update_ops collection. Otherwise, update_ops will be empty, and training/inference will not work properly. For example:</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">x_norm = tf.layers.batch_normalization(x, training=training)</span><br><span class="line"></span><br><span class="line"><span class="comment"># ...</span></span><br><span class="line"></span><br><span class="line">update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)</span><br><span class="line"><span class="keyword">with</span> tf.control_dependencies(update_ops):</span><br><span class="line"> train_op = optimizer.minimize(loss)</span><br></pre></td></tr></table></figure><h3 id="Preliminary"><a href="#Preliminary" class="headerlink" title="Preliminary"></a>Preliminary</h3><h4 id="计算均值和方差"><a href="#计算均值和方差" class="headerlink" title="计算均值和方差"></a>计算均值和方差</h4><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">tf.nn.moments(</span><br><span class="line"> x,</span><br><span class="line"> axes,</span><br><span class="line"> shift=<span class="keyword">None</span>,</span><br><span class="line"> name=<span class="keyword">None</span>,</span><br><span class="line"> keep_dims=<span class="keyword">False</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li>x: 张量</li><li>axes： 列表。求均值和方差的方向</li><li>name： 操作的名字</li><li>keep_dims： true，均值和方差的shape与输入一致</li></ul><p>Returns：</p><ul><li>mean，variance</li></ul><p>When using these moments for batch normalization (see tf.nn.batch_normalization):</p><ul><li><p>for so-called “global normalization”, used with convolutional filters with shape [batch, height, width, depth], pass axes=[0, 1, 2].即，对同一个channel的数据进行计算均值和方差。</p></li><li><p>for simple batch normalization pass axes=[0] (batch only).</p></li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.nn.batch_normalization(</span><br><span class="line"> x,</span><br><span class="line"> mean,</span><br><span class="line"> variance,</span><br><span class="line"> offset,</span><br><span class="line"> scale,</span><br><span class="line"> variance_epsilon,</span><br><span class="line"> name=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li>x : 张量</li><li>mean： 均值</li><li>variance： 方差</li><li>offset: \beta, default None</li><li>scale: \gamma, default None </li><li>variance_epsilon: A small float number to avoid dividing by 0</li><li>name: 操作的名字</li></ul><p>性质： </p><p>如果不提供scale，offset，则新的张量的均值和方差分别为0,1。</p><p>mean, variance, offset and scale are all expected to be of one of two shapes:</p><ul><li>In all generality, they can have the same number of dimensions as the input x, with identical sizes as x for the dimensions that are not normalized over (the ‘depth’ dimension(s)), and dimension 1 for the others which are being normalized over. mean and variance in this case would typically be the outputs of tf.nn.moments(…, keep_dims=True) during training, or running averages thereof during inference.</li><li>In the common case where the ‘depth’ dimension is the last dimension in the input tensor x, they may be one dimensional tensors of the same size as the ‘depth’ dimension. This is the case for example for the common [batch, depth] layout of fully-connected layers, and [batch, height, width, depth] for convolutions. mean and variance in this case would typically be the outputs of tf.nn.moments(…, keep_dims=False) during training, or running averages thereof during inference.<br>Args:</li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.assign(</span><br><span class="line"> ref,</span><br><span class="line"> value,</span><br><span class="line"> validate_shape=<span class="keyword">None</span>,</span><br><span class="line"> use_locking=<span class="keyword">None</span>,</span><br><span class="line"> name=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li>ref: A mutable Tensor. Should be from a Variable node. May be uninitialized.</li><li>value: A Tensor. Must have the same type as ref. The value to be assigned to the variable.</li><li>validate_shape: An optional bool. Defaults to True. If true, the operation will validate that the shape of ‘value’ matches the shape of the Tensor being assigned to. If false, ‘ref’ will take on the shape of ‘value’.<br>use_locking: An optional bool. Defaults to True. If True, the assignment will be protected by a lock; otherwise the behavior is undefined, but may exhibit less contention.</li><li>name: A name for the operation (optional).<br>Returns:</li><li>A Tensor that will hold the new value of ‘ref’ after the assignment has completed.</li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.cond(</span><br><span class="line"> pred,</span><br><span class="line"> true_fn=<span class="keyword">None</span>,</span><br><span class="line"> false_fn=<span class="keyword">None</span>,</span><br><span class="line"> strict=<span class="keyword">False</span>,</span><br><span class="line"> name=<span class="keyword">None</span>,</span><br><span class="line"> fn1=<span class="keyword">None</span>,</span><br><span class="line"> fn2=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li>pred: A scalar determining whether to return the result of true_fn or false_fn.</li><li>true_fn: The callable to be performed if pred is true.</li><li>false_fn: The callable to be performed if pred is false.</li><li>strict: A boolean that enables/disables ‘strict’ mode; see above.</li><li>name: Optional name prefix for the returned tensors.<br>Returns:</li><li>Tensors returned by the call to either true_fn or false_fn. If the callables return a singleton list, the element is extracted from the list.</li></ul><h4 id="BN在神经网络进行training和testing"><a href="#BN在神经网络进行training和testing" class="headerlink" title="BN在神经网络进行training和testing"></a>BN在神经网络进行training和testing</h4><h4 id="Example"><a href="#Example" class="headerlink" title="Example"></a>Example</h4><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br><span class="line">254</span><br><span class="line">255</span><br><span class="line">256</span><br><span class="line">257</span><br><span class="line">258</span><br><span class="line">259</span><br><span class="line">260</span><br><span class="line">261</span><br><span class="line">262</span><br><span class="line">263</span><br><span class="line">264</span><br><span class="line">265</span><br><span class="line">266</span><br><span class="line">267</span><br><span class="line">268</span><br><span class="line">269</span><br><span class="line">270</span><br><span class="line">271</span><br><span class="line">272</span><br><span class="line">273</span><br><span class="line">274</span><br><span class="line">275</span><br><span class="line">276</span><br><span class="line">277</span><br><span class="line">278</span><br><span class="line">279</span><br><span class="line">280</span><br><span class="line">281</span><br><span class="line">282</span><br><span class="line">283</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">"""</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">大多数情况下，您将能够使用高级功能，但有时您可能想要在较低的级别工作。例如，如果您想要实现一个新特性—一些新的内容，那么TensorFlow还没有包括它的高级实现，</span></span><br><span class="line"><span class="string">比如LSTM中的批处理规范化——那么您可能需要知道一些事情。</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">这个版本的网络的几乎所有函数都使用tf.nn包进行编写，并且使用tf.nn.batch_normalization函数进行标准化操作</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">'fully_connected'函数的实现比使用tf.layers包进行编写的要复杂得多。然而，如果你浏览了Batch_Normalization_Lesson笔记本，事情看起来应该很熟悉。</span></span><br><span class="line"><span class="string">为了增加批量标准化，我们做了如下工作:</span></span><br><span class="line"><span class="string">Added the is_training parameter to the function signature so we can pass that information to the batch normalization layer.</span></span><br><span class="line"><span class="string">1.在函数声明中添加'is_training'参数，以确保可以向Batch Normalization层中传递信息</span></span><br><span class="line"><span class="string">2.去除函数中bias偏置属性和激活函数</span></span><br><span class="line"><span class="string">3.添加gamma, beta, pop_mean, and pop_variance等变量</span></span><br><span class="line"><span class="string">4.使用tf.cond函数来解决训练和预测时的使用方法的差异</span></span><br><span class="line"><span class="string">5.训练时，我们使用tf.nn.moments函数来计算批数据的均值和方差，然后在迭代过程中更新均值和方差的分布，并且使用tf.nn.batch_normalization做标准化</span></span><br><span class="line"><span class="string"> 注意：一定要使用with tf.control_dependencies...语句结构块来强迫Tensorflow先更新均值和方差的分布，再使用执行批标准化操作</span></span><br><span class="line"><span class="string">6.在前向传播推导时(特指只进行预测，而不对训练参数进行更新时)，我们使用tf.nn.batch_normalization批标准化时其中的均值和方差分布来自于训练时我们</span></span><br><span class="line"><span class="string"> 使用滑动平均算法估计的值。</span></span><br><span class="line"><span class="string">7.将标准化后的值通过RelU激活函数求得输出</span></span><br><span class="line"><span class="string">8.不懂请参见https://github.com/udacity/deep-learning/blob/master/batch-norm/Batch_Normalization_Lesson.ipynb</span></span><br><span class="line"><span class="string"> 中关于使用tf.nn.batch_normalization实现'fully_connected'函数的操作</span></span><br><span class="line"><span class="string">"""</span></span><br><span class="line"><span class="keyword">import</span> tensorflow <span class="keyword">as</span> tf</span><br><span class="line"><span class="keyword">from</span> tensorflow.examples.tutorials.mnist <span class="keyword">import</span> input_data</span><br><span class="line"></span><br><span class="line">mnist = input_data.read_data_sets(<span class="string">"MNIST_data/"</span>, one_hot=<span class="keyword">True</span>, reshape=<span class="keyword">False</span>)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">fully_connected</span><span class="params">(prev_layer, num_units, is_training)</span>:</span></span><br><span class="line"> <span class="string">"""</span></span><br><span class="line"><span class="string"> num_units参数传递该层神经元的数量，根据prev_layer参数传入值作为该层输入创建全连接神经网络。</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string"> :param prev_layer: Tensor</span></span><br><span class="line"><span class="string"> 该层神经元输入</span></span><br><span class="line"><span class="string"> :param num_units: int</span></span><br><span class="line"><span class="string"> 该层神经元结点个数</span></span><br><span class="line"><span class="string"> :param is_training: bool or Tensor</span></span><br><span class="line"><span class="string"> 表示该网络当前是否正在训练，告知Batch Normalization层是否应该更新或者使用均值或方差的分布信息</span></span><br><span class="line"><span class="string"> :returns Tensor</span></span><br><span class="line"><span class="string"> 一个新的全连接神经网络层</span></span><br><span class="line"><span class="string"> """</span></span><br><span class="line"></span><br><span class="line"> layer = tf.layers.dense(prev_layer, num_units, use_bias=<span class="keyword">False</span>, activation=<span class="keyword">None</span>)</span><br><span class="line"></span><br><span class="line"> gamma = tf.Variable(tf.ones([num_units]))</span><br><span class="line"> beta = tf.Variable(tf.zeros([num_units]))</span><br><span class="line"></span><br><span class="line"> pop_mean = tf.Variable(tf.zeros([num_units]), trainable=<span class="keyword">False</span>)</span><br><span class="line"> pop_variance = tf.Variable(tf.ones([num_units]), trainable=<span class="keyword">False</span>)</span><br><span class="line"></span><br><span class="line"> epsilon = <span class="number">1e-3</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">batch_norm_training</span><span class="params">()</span>:</span></span><br><span class="line"> batch_mean, batch_variance = tf.nn.moments(layer, [<span class="number">0</span>])</span><br><span class="line"></span><br><span class="line"> decay = <span class="number">0.99</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 为什么其实是0均值, 1方差#</span></span><br><span class="line"> train_mean = tf.assign(pop_mean, pop_mean*decay + batch_mean*(<span class="number">1</span> - decay))</span><br><span class="line"> train_variance = tf.assign(pop_variance, pop_variance*decay + batch_variance*(<span class="number">1</span> - decay))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">with</span> tf.control_dependencies([train_mean, train_variance]):</span><br><span class="line"> <span class="keyword">return</span> tf.nn.batch_normalization(layer, batch_mean, batch_variance, beta, gamma, epsilon)</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">batch_norm_inference</span><span class="params">()</span>:</span></span><br><span class="line"> <span class="keyword">return</span> tf.nn.batch_normalization(layer, pop_mean, pop_variance, beta, gamma, epsilon)</span><br><span class="line"></span><br><span class="line"> batch_normalized_output = tf.cond(is_training, batch_norm_training, batch_norm_inference)</span><br><span class="line"> <span class="keyword">return</span> tf.nn.relu(batch_normalized_output)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="string">"""</span></span><br><span class="line"><span class="string">我们对conv_layer卷积层的改变和我们对fully_connected全连接层的改变几乎差不多。</span></span><br><span class="line"><span class="string">然而也有很大的区别，卷积层有多个特征图并且每个特征图在输入图层上共享权重</span></span><br><span class="line"><span class="string">所以我们需要确保应该针对每个特征图而不是卷积层上的每个节点进行Batch Normalization操作</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">为了实现这一点，我们做了与fully_connected相同的事情，有两个例外:</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">1.将gamma、beta、pop_mean和pop_方差的大小设置为feature map(输出通道)的数量，而不是输出节点的数量。</span></span><br><span class="line"><span class="string">2.我们改变传递给tf.nn的参数。时刻确保它计算正确维度的均值和方差。</span></span><br><span class="line"><span class="string">"""</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">conv_layer</span><span class="params">(prev_layer, layer_depth, is_training)</span>:</span></span><br><span class="line"> <span class="string">"""</span></span><br><span class="line"><span class="string"> 使用给定的参数作为输入创建卷积层</span></span><br><span class="line"><span class="string"> :param prev_layer: Tensor</span></span><br><span class="line"><span class="string"> 传入该层神经元作为输入</span></span><br><span class="line"><span class="string"> :param layer_depth: int</span></span><br><span class="line"><span class="string"> 我们将根据网络中图层的深度设置特征图的步长和数量。</span></span><br><span class="line"><span class="string"> 这不是实践CNN的好方法，但它可以帮助我们用很少的代码创建这个示例。</span></span><br><span class="line"><span class="string"> :param is_training: bool or Tensor</span></span><br><span class="line"><span class="string"> 表示该网络当前是否正在训练，告知Batch Normalization层是否应该更新或者使用均值或方差的分布信息</span></span><br><span class="line"><span class="string"> :returns Tensor</span></span><br><span class="line"><span class="string"> 一个新的卷积层</span></span><br><span class="line"><span class="string"> """</span></span><br><span class="line"> strides = <span class="number">2</span> <span class="keyword">if</span> layer_depth%<span class="number">3</span> == <span class="number">0</span> <span class="keyword">else</span> <span class="number">1</span></span><br><span class="line"></span><br><span class="line"> in_channels = prev_layer.get_shape().as_list()[<span class="number">3</span>]</span><br><span class="line"> out_channels = layer_depth*<span class="number">4</span></span><br><span class="line"></span><br><span class="line"> weights = tf.Variable(</span><br><span class="line"> tf.truncated_normal([<span class="number">3</span>, <span class="number">3</span>, in_channels, out_channels], stddev=<span class="number">0.05</span>))</span><br><span class="line"></span><br><span class="line"> layer = tf.nn.conv2d(prev_layer, weights, strides=[<span class="number">1</span>, strides, strides, <span class="number">1</span>], padding=<span class="string">'SAME'</span>)</span><br><span class="line"></span><br><span class="line"> gamma = tf.Variable(tf.ones([out_channels]))</span><br><span class="line"> beta = tf.Variable(tf.zeros([out_channels]))</span><br><span class="line"></span><br><span class="line"> pop_mean = tf.Variable(tf.zeros([out_channels]), trainable=<span class="keyword">False</span>)</span><br><span class="line"> pop_variance = tf.Variable(tf.ones([out_channels]), trainable=<span class="keyword">False</span>)</span><br><span class="line"></span><br><span class="line"> epsilon = <span class="number">1e-3</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">batch_norm_training</span><span class="params">()</span>:</span></span><br><span class="line"> <span class="comment"># 一定要使用正确的维度确保计算的是每个特征图上的平均值和方差而不是整个网络节点上的统计分布值</span></span><br><span class="line"> batch_mean, batch_variance = tf.nn.moments(layer, [<span class="number">0</span>, <span class="number">1</span>, <span class="number">2</span>], keep_dims=<span class="keyword">False</span>)</span><br><span class="line"></span><br><span class="line"> decay = <span class="number">0.99</span></span><br><span class="line"> train_mean = tf.assign(pop_mean, pop_mean*decay + batch_mean*(<span class="number">1</span> - decay))</span><br><span class="line"> train_variance = tf.assign(pop_variance, pop_variance*decay + batch_variance*(<span class="number">1</span> - decay))</span><br><span class="line"></span><br><span class="line"> <span class="keyword">with</span> tf.control_dependencies([train_mean, train_variance]):</span><br><span class="line"> <span class="keyword">return</span> tf.nn.batch_normalization(layer, batch_mean, batch_variance, beta, gamma, epsilon)</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">batch_norm_inference</span><span class="params">()</span>:</span></span><br><span class="line"> <span class="keyword">return</span> tf.nn.batch_normalization(layer, pop_mean, pop_variance, beta, gamma, epsilon)</span><br><span class="line"></span><br><span class="line"> batch_normalized_output = tf.cond(is_training, batch_norm_training, batch_norm_inference)</span><br><span class="line"> <span class="keyword">return</span> tf.nn.relu(batch_normalized_output)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="string">"""</span></span><br><span class="line"><span class="string">为了修改训练函数，我们需要做以下工作:</span></span><br><span class="line"><span class="string">1.Added is_training, a placeholder to store a boolean value indicating whether or not the network is training.</span></span><br><span class="line"><span class="string">添加is_training，一个用于存储布尔值的占位符，该值指示网络是否正在训练</span></span><br><span class="line"><span class="string">2.Each time we call run on the session, we added to feed_dict the appropriate value for is_training.</span></span><br><span class="line"><span class="string">每次调用sess.run函数时，我们都添加到feed_dict中is_training的适当值用以表示当前是正在训练还是预测</span></span><br><span class="line"><span class="string">3.We did not need to add the with tf.control_dependencies... statement that we added in the network that used tf.layers.batch_normalization</span></span><br><span class="line"><span class="string">because we handled updating the population statistics ourselves in conv_layer and fully_connected.</span></span><br><span class="line"><span class="string">我们不需要将train_opt训练函数放进with tf.control_dependencies... 的函数结构体中,这是只有在使用tf.layers.batch_normalization才做的更新均值和方差的操作</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">"""</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">train</span><span class="params">(num_batches, batch_size, learning_rate)</span>:</span></span><br><span class="line"> <span class="comment"># Build placeholders for the input samples and labels</span></span><br><span class="line"> <span class="comment"># 创建输入样本和标签的占位符</span></span><br><span class="line"> inputs = tf.placeholder(tf.float32, [<span class="keyword">None</span>, <span class="number">28</span>, <span class="number">28</span>, <span class="number">1</span>])</span><br><span class="line"> labels = tf.placeholder(tf.float32, [<span class="keyword">None</span>, <span class="number">10</span>])</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Add placeholder to indicate whether or not we're training the model</span></span><br><span class="line"> <span class="comment"># 创建占位符表明当前是否正在训练模型</span></span><br><span class="line"> is_training = tf.placeholder(tf.bool)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Feed the inputs into a series of 20 convolutional layers</span></span><br><span class="line"> <span class="comment"># 把输入数据填充到一系列20个卷积层的神经网络中</span></span><br><span class="line"> layer = inputs</span><br><span class="line"> <span class="keyword">for</span> layer_i <span class="keyword">in</span> range(<span class="number">1</span>, <span class="number">20</span>):</span><br><span class="line"> layer = conv_layer(layer, layer_i, is_training)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Flatten the output from the convolutional layers</span></span><br><span class="line"> <span class="comment"># 将卷积层输出扁平化处理</span></span><br><span class="line"> orig_shape = layer.get_shape().as_list()</span><br><span class="line"> layer = tf.reshape(layer, shape=[<span class="number">-1</span>, orig_shape[<span class="number">1</span>]*orig_shape[<span class="number">2</span>]*orig_shape[<span class="number">3</span>]])</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Add one fully connected layer</span></span><br><span class="line"> <span class="comment"># 添加一个具有100个神经元的全连接层</span></span><br><span class="line"> layer = fully_connected(layer, <span class="number">100</span>, is_training)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Create the output layer with 1 node for each</span></span><br><span class="line"> <span class="comment"># 为每一个类别添加一个输出节点</span></span><br><span class="line"> logits = tf.layers.dense(layer, <span class="number">10</span>)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Define loss and training operations</span></span><br><span class="line"> <span class="comment"># 定义loss 函数和训练操作</span></span><br><span class="line"> model_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=logits, labels=labels))</span><br><span class="line"> train_opt = tf.train.AdamOptimizer(learning_rate).minimize(model_loss)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Create operations to test accuracy</span></span><br><span class="line"> <span class="comment"># 创建计算准确度的操作</span></span><br><span class="line"> correct_prediction = tf.equal(tf.argmax(logits, <span class="number">1</span>), tf.argmax(labels, <span class="number">1</span>))</span><br><span class="line"> accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Train and test the network</span></span><br><span class="line"> <span class="comment"># 训练并测试网络模型</span></span><br><span class="line"> <span class="keyword">with</span> tf.Session() <span class="keyword">as</span> sess:</span><br><span class="line"> sess.run(tf.global_variables_initializer())</span><br><span class="line"> <span class="keyword">for</span> batch_i <span class="keyword">in</span> range(num_batches):</span><br><span class="line"> batch_xs, batch_ys = mnist.train.next_batch(batch_size)</span><br><span class="line"></span><br><span class="line"> <span class="comment"># train this batch</span></span><br><span class="line"> <span class="comment"># 训练样本批次</span></span><br><span class="line"> sess.run(train_opt, {inputs: batch_xs, labels: batch_ys, is_training: <span class="keyword">True</span>})</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Periodically check the validation or training loss and accuracy</span></span><br><span class="line"> <span class="comment"># 定期检查训练或验证集上的loss和精确度</span></span><br><span class="line"> <span class="keyword">if</span> batch_i%<span class="number">100</span> == <span class="number">0</span>:</span><br><span class="line"> loss, acc = sess.run([model_loss, accuracy], {inputs: mnist.validation.images,</span><br><span class="line"> labels: mnist.validation.labels,</span><br><span class="line"> is_training: <span class="keyword">False</span>})</span><br><span class="line"> print(</span><br><span class="line"> <span class="string">'Batch: {:>2}: Validation loss: {:>3.5f}, Validation accuracy: {:>3.5f}'</span>.format(batch_i, loss, acc))</span><br><span class="line"> <span class="keyword">elif</span> batch_i%<span class="number">25</span> == <span class="number">0</span>:</span><br><span class="line"> loss, acc = sess.run([model_loss, accuracy], {inputs: batch_xs, labels: batch_ys, is_training: <span class="keyword">False</span>})</span><br><span class="line"> print(<span class="string">'Batch: {:>2}: Training loss: {:>3.5f}, Training accuracy: {:>3.5f}'</span>.format(batch_i, loss, acc))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># At the end, score the final accuracy for both the validation and test sets</span></span><br><span class="line"> <span class="comment"># 最后在验证集和测试集上对模型准确率进行评分</span></span><br><span class="line"> acc = sess.run(accuracy, {inputs: mnist.validation.images,</span><br><span class="line"> labels: mnist.validation.labels,</span><br><span class="line"> is_training: <span class="keyword">False</span>})</span><br><span class="line"> print(<span class="string">'Final validation accuracy: {:>3.5f}'</span>.format(acc))</span><br><span class="line"> acc = sess.run(accuracy, {inputs: mnist.test.images,</span><br><span class="line"> labels: mnist.test.labels,</span><br><span class="line"> is_training: <span class="keyword">False</span>})</span><br><span class="line"> print(<span class="string">'Final test accuracy: {:>3.5f}'</span>.format(acc))</span><br><span class="line"></span><br><span class="line"> <span class="comment"># Score the first 100 test images individually, just to make sure batch normalization really worked</span></span><br><span class="line"> <span class="comment"># 对100个独立的测试图片进行评分,对比验证Batch Normalization的效果</span></span><br><span class="line"> correct = <span class="number">0</span></span><br><span class="line"> <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">100</span>):</span><br><span class="line"> correct += sess.run(accuracy, feed_dict={inputs: [mnist.test.images[i]],</span><br><span class="line"> labels: [mnist.test.labels[i]],</span><br><span class="line"> is_training: <span class="keyword">False</span>})</span><br><span class="line"></span><br><span class="line"> print(<span class="string">"Accuracy on 100 samples:"</span>, correct/<span class="number">100</span>)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">num_batches = <span class="number">800</span> <span class="comment"># 迭代次数</span></span><br><span class="line">batch_size = <span class="number">64</span> <span class="comment"># 批处理数量</span></span><br><span class="line">learning_rate = <span class="number">0.002</span> <span class="comment"># 学习率</span></span><br><span class="line"></span><br><span class="line">tf.reset_default_graph()</span><br><span class="line"><span class="keyword">with</span> tf.Graph().as_default():</span><br><span class="line"> train(num_batches, batch_size, learning_rate)</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"><span class="string">"""</span></span><br><span class="line"><span class="string">再一次，批量标准化的模型很快达到了很高的精度。</span></span><br><span class="line"><span class="string">但是在我们的运行中，注意到它似乎并没有学习到前250个批次的任何东西，然后精度开始上升。</span></span><br><span class="line"><span class="string">这只是显示——即使是批处理标准化，给您的网络一些时间来学习是很重要的。</span></span><br><span class="line"><span class="string"></span></span><br><span class="line"><span class="string">PS:再100个单个数据的预测上达到了较高的精度，而这才是BN算法真正关注的！！</span></span><br><span class="line"><span class="string">"""</span></span><br><span class="line"><span class="comment"># Extracting MNIST_data/train-images-idx3-ubyte.gz</span></span><br><span class="line"><span class="comment"># Extracting MNIST_data/train-labels-idx1-ubyte.gz</span></span><br><span class="line"><span class="comment"># Extracting MNIST_data/t10k-images-idx3-ubyte.gz</span></span><br><span class="line"><span class="comment"># Extracting MNIST_data/t10k-labels-idx1-ubyte.gz</span></span><br><span class="line"><span class="comment"># 2018-03-18 19:35:28.568404: I D:\Build\tensorflow\tensorflow-r1.4\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX</span></span><br><span class="line"><span class="comment"># Batch: 0: Validation loss: 0.69113, Validation accuracy: 0.10020</span></span><br><span class="line"><span class="comment"># Batch: 25: Training loss: 0.57341, Training accuracy: 0.07812</span></span><br><span class="line"><span class="comment"># Batch: 50: Training loss: 0.45526, Training accuracy: 0.04688</span></span><br><span class="line"><span class="comment"># Batch: 75: Training loss: 0.37936, Training accuracy: 0.12500</span></span><br><span class="line"><span class="comment"># Batch: 100: Validation loss: 0.34601, Validation accuracy: 0.10700</span></span><br><span class="line"><span class="comment"># Batch: 125: Training loss: 0.34113, Training accuracy: 0.12500</span></span><br><span class="line"><span class="comment"># Batch: 150: Training loss: 0.33075, Training accuracy: 0.12500</span></span><br><span class="line"><span class="comment"># Batch: 175: Training loss: 0.34333, Training accuracy: 0.15625</span></span><br><span class="line"><span class="comment"># Batch: 200: Validation loss: 0.37085, Validation accuracy: 0.09860</span></span><br><span class="line"><span class="comment"># Batch: 225: Training loss: 0.40175, Training accuracy: 0.09375</span></span><br><span class="line"><span class="comment"># Batch: 250: Training loss: 0.48562, Training accuracy: 0.06250</span></span><br><span class="line"><span class="comment"># Batch: 275: Training loss: 0.67897, Training accuracy: 0.09375</span></span><br><span class="line"><span class="comment"># Batch: 300: Validation loss: 0.48383, Validation accuracy: 0.09880</span></span><br><span class="line"><span class="comment"># Batch: 325: Training loss: 0.43822, Training accuracy: 0.14062</span></span><br><span class="line"><span class="comment"># Batch: 350: Training loss: 0.43227, Training accuracy: 0.18750</span></span><br><span class="line"><span class="comment"># Batch: 375: Training loss: 0.39464, Training accuracy: 0.37500</span></span><br><span class="line"><span class="comment"># Batch: 400: Validation loss: 0.50557, Validation accuracy: 0.25940</span></span><br><span class="line"><span class="comment"># Batch: 425: Training loss: 0.32337, Training accuracy: 0.59375</span></span><br><span class="line"><span class="comment"># Batch: 450: Training loss: 0.14016, Training accuracy: 0.75000</span></span><br><span class="line"><span class="comment"># Batch: 475: Training loss: 0.11652, Training accuracy: 0.78125</span></span><br><span class="line"><span class="comment"># Batch: 500: Validation loss: 0.06241, Validation accuracy: 0.91280</span></span><br><span class="line"><span class="comment"># Batch: 525: Training loss: 0.01880, Training accuracy: 0.96875</span></span><br><span class="line"><span class="comment"># Batch: 550: Training loss: 0.03640, Training accuracy: 0.93750</span></span><br><span class="line"><span class="comment"># Batch: 575: Training loss: 0.07202, Training accuracy: 0.90625</span></span><br><span class="line"><span class="comment"># Batch: 600: Validation loss: 0.03984, Validation accuracy: 0.93960</span></span><br><span class="line"><span class="comment"># Batch: 625: Training loss: 0.00692, Training accuracy: 0.98438</span></span><br><span class="line"><span class="comment"># Batch: 650: Training loss: 0.01251, Training accuracy: 0.96875</span></span><br><span class="line"><span class="comment"># Batch: 675: Training loss: 0.01823, Training accuracy: 0.96875</span></span><br><span class="line"><span class="comment"># Batch: 700: Validation loss: 0.03951, Validation accuracy: 0.94080</span></span><br><span class="line"><span class="comment"># Batch: 725: Training loss: 0.02886, Training accuracy: 0.95312</span></span><br><span class="line"><span class="comment"># Batch: 750: Training loss: 0.06396, Training accuracy: 0.87500</span></span><br><span class="line"><span class="comment"># Batch: 775: Training loss: 0.02013, Training accuracy: 0.98438</span></span><br><span class="line"><span class="comment"># Final validation accuracy: 0.95820</span></span><br><span class="line"><span class="comment"># Final test accuracy: 0.95780</span></span><br><span class="line"><span class="comment"># Accuracy on 100 samples: 0.98</span></span><br></pre></td></tr></table></figure><h1 id="Reference"><a href="#Reference" class="headerlink" title="Reference"></a>Reference</h1><p><a href="https://github.com/SunshineJunFu/deep-learning/blob/master/batch-norm/Batch_Normalization_Solutions.ipynb">https://github.com/SunshineJunFu/deep-learning/blob/master/batch-norm/Batch_Normalization_Solutions.ipynb</a></p><p><a href="https://www.cnblogs.com/zhengmingli/p/8031690.html" target="_blank" rel="noopener">https://www.cnblogs.com/zhengmingli/p/8031690.html</a></p><p><a href="https://www.tensorflow.org/api_docs/python/tf/nn/batch_normalization?hl=zh-cn" target="_blank" rel="noopener">https://www.tensorflow.org/api_docs/python/tf/nn/batch_normalization?hl=zh-cn</a></p><p><a href="https://www.jianshu.com/p/615113382fac" target="_blank" rel="noopener">https://www.jianshu.com/p/615113382fac</a></p><p><a href="http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html" target="_blank" rel="noopener">http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html</a></p><p><a href="https://www.zhihu.com/question/38102762" target="_blank" rel="noopener">https://www.zhihu.com/question/38102762</a></p><p><a href="http://www.cnblogs.com/cloud-ken/p/9314769.html" target="_blank" rel="noopener">http://www.cnblogs.com/cloud-ken/p/9314769.html</a></p><p><a href="https://www.cnblogs.com/wuliytTaotao/p/9479958.html" target="_blank" rel="noopener">https://www.cnblogs.com/wuliytTaotao/p/9479958.html</a></p><p><a href="https://www.zhihu.com/question/66873757/answer/405455697" target="_blank" rel="noopener">https://www.zhihu.com/question/66873757/answer/405455697</a></p>]]></content>
<summary type="html">
<h1 id="batch-Normalization"><a href="#batch-Normalization" class="headerlink" title="batch Normalization"></a>batch Normalization</h1><h2 i
</summary>
</entry>
<entry>
<title>tensorflow_tutorial_8</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-8/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-8/</id>
<published>2019-02-02T17:00:57.000Z</published>
<updated>2019-02-17T07:18:54.706Z</updated>
<content type="html"><![CDATA[<h1 id="如何使用高低阶的API搭建网络"><a href="#如何使用高低阶的API搭建网络" class="headerlink" title="如何使用高低阶的API搭建网络"></a>如何使用高低阶的API搭建网络</h1><h2 id="全连接"><a href="#全连接" class="headerlink" title="全连接"></a>全连接</h2><p>结构：<code>flatten</code>$\rightarrow$ <code>xw+b</code>$\rightarrow$ <code>norm layer</code>(optional)$\rightarrow$<code>activation function</code> $\rightarrow$<code>dropout</code>(optional)</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">add_fc_layer</span><span class="params">(inputs, in_dim, out_dim, scope_name, activation_function=None)</span>:</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">with</span> tf.variable_scope(scope_name):</span><br><span class="line"></span><br><span class="line"> w = tf.get_variable(shape=[in_dim, out_dim], initializer=tf.contrib.layers.xavier_initializer(), name=<span class="string">'weights'</span>)</span><br><span class="line"></span><br><span class="line"> b = tf.get_variable(shape=[<span class="number">1</span>, out_dim], initializer=tf.contrib.layers.xavier_initializer(), name=<span class="string">'biases'</span>)</span><br><span class="line"></span><br><span class="line"> wx_plus_b = tf.matmul(inputs, w) + b <span class="comment"># broadcast</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> activation_function:</span><br><span class="line"></span><br><span class="line"> wx_plus_b = activation_function(wx_plus_b)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> wx_plus_b</span><br></pre></td></tr></table></figure><h2 id="卷积"><a href="#卷积" class="headerlink" title="卷积"></a>卷积</h2><p>结构： <code>convolution</code> $\rightarrow$ <code>norm layer</code>(optional) $\rightarrow$<code>activation function</code> $\rightarrow$<code>pooling</code>(optional) </p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">add_conv2d_layer</span><span class="params">(inputs, in_channel, out_channel, scope_name, activation_function, filter_h, filter_w,stride)</span>:</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">with</span> tf.variable_scope(scope_name):</span><br><span class="line"></span><br><span class="line">fliter_mask = tf.get_variable(shape=[filter_h, filter_w, in_channel, out_channel], dtype=inputs.dtype, initializer=tf.contrib.layers.xavier_initializer(),name=<span class="string">'filter'</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 'same' 输入输出的宽高一致， 'valid', out_width = xxxx #</span></span><br><span class="line">conv_inputs = tf.nn.conv2d(inputs, fliter_mask, [<span class="number">1</span>,stride,stride,<span class="number">1</span>], <span class="string">'SAME'</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># batch normalization #</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">if</span> activation_function:</span><br><span class="line"></span><br><span class="line">conv_inputs = activation_function(conv_inputs)</span><br><span class="line"></span><br><span class="line"><span class="comment"># pooling #</span></span><br><span class="line"></span><br><span class="line">conv_inputs = tf.nn.max_pool(conv_inputs,[<span class="number">1</span>,<span class="number">2</span>,<span class="number">2</span>,<span class="number">1</span>],[<span class="number">1</span>,<span class="number">2</span>,<span class="number">2</span>,<span class="number">1</span>],<span class="string">'SAME'</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">return</span> conv_inputs</span><br></pre></td></tr></table></figure><h2 id="反卷积"><a href="#反卷积" class="headerlink" title="反卷积"></a>反卷积</h2><p>结构： <code>inverse-convolution</code> $\rightarrow$ <code>norm layer</code>(optional) $\rightarrow$<code>activation function</code> </p>]]></content>
<summary type="html">
<h1 id="如何使用高低阶的API搭建网络"><a href="#如何使用高低阶的API搭建网络" class="headerlink" title="如何使用高低阶的API搭建网络"></a>如何使用高低阶的API搭建网络</h1><h2 id="全连接"><a href=
</summary>
</entry>
<entry>
<title>tensorflow_tutorial_7</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-7/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/03/tensorflow-tutorial-7/</id>
<published>2019-02-02T16:59:00.000Z</published>
<updated>2019-02-23T02:07:05.738Z</updated>
<content type="html"><![CDATA[<h1 id="分类任务里的衡量指标"><a href="#分类任务里的衡量指标" class="headerlink" title="分类任务里的衡量指标"></a>分类任务里的衡量指标</h1><h2 id="二分类"><a href="#二分类" class="headerlink" title="二分类"></a>二分类</h2><h3 id="混淆矩阵"><a href="#混淆矩阵" class="headerlink" title="混淆矩阵"></a>混淆矩阵</h3><table><thead><tr><th style="text-align:left"></th><th style="text-align:center">predicted 1</th><th style="text-align:center">predicted 0</th></tr></thead><tbody><tr><td style="text-align:left">label 1</td><td style="text-align:center">TP</td><td style="text-align:center">FN</td></tr><tr><td style="text-align:left">label 0</td><td style="text-align:center">FP</td><td style="text-align:center">TN</td></tr></tbody></table><ul><li>TP：label为正例，预测为正例</li><li>FP：label为负例，预测为正例</li><li>TN：label为正例，预测为正例</li><li>FN：label为正例，预测为负例</li></ul><p>精度(Accuracy):<br>\begin{align}<br>Accuracy &=\frac{预测正确样本个数}{总样本数} \\<br>&=\frac{TP+TN}{TP+FP+TN+FN}<br>\end{align}</p><p>查全率(Recall): 预测正确的正例个数与总的正例的比值。</p><p>$$Recall=\frac{TP}{TP+FN}$$</p><p>查准率(Precision)：预测正确的正例个数与预测为正例个数的比值。</p><p>$$Precision=\frac{TP}{TP+FP}$$</p><p>性质: Recall 和 Precision 是一对矛盾的指标</p><p>证明： </p><p>假定<code>Recall</code> $\uparrow$, 由于$TP+FP=数据集内的正样本数=常数$ $\Rightarrow$ <code>TP</code>$\uparrow$, 由于<code>TP</code>$\uparrow$,为了提高<code>TP</code>，会同时增加<code>FP</code>且<code>FP+TP</code>的增量大于<code>TP</code>的增量(直觉上，为了提高<code>TP</code>，分类器会偏向于将分类为正类，这样也增加了<code>FP</code>)，所以<code>Precision</code>$\downarrow$。</p><p>假定<code>Recall</code> $\downarrow$, 由于$TP+FP=数据集内的正样本数=常数$ $\Rightarrow$ <code>TP</code>$\downarrow$, 由于<code>TP</code>$\downarrow$,为了减小<code>TP</code>，会同时减小<code>FP</code>且<code>FP+TP</code>的减小量大于<code>TP</code>的减小量(直觉上，为了减小<code>TP</code>，分类器会偏向于将分类为负类，这样也减小了<code>FP</code>)，所以<code>Precision</code>$\uparrow$。</p><p>所以：</p><ul><li><code>Recall</code> $\uparrow$ $\Rightarrow$ <code>TP</code>$\uparrow$，<code>FP+TP</code>$\uparrow$ $\Rightarrow$ <code>Precision</code> $\downarrow$</li><li><code>Recall</code> $\downarrow$ $\Rightarrow$ <code>TP</code>$\downarrow$，<code>FP+TP</code>$\downarrow$ $\Rightarrow$ <code>Precision</code> $\uparrow$</li></ul><p>有的模型是将预测结果与一个阈值进行比较，如果大于该阈值则预测为正例，否则为反例。所以这个阈值非常重要，直接决定了模型的泛化能力。同时根据这个阈值我们可以将样本进行排序，“最可能”是正例的排在最前面，“最不可能”是正例的排在最后面。因此，如果我们想让precision更高，可以将阈值设置在排序靠前的位置，相反如果想让recall更高，可以将阈值设置在排序靠后的位置。</p><p>F1 Measure/Score</p><p>为了更准确反映模型的性能，引入了F1 score, 该指标同时考虑了<code>Recall</code>和<code>Precision</code>， 能让我们表达出对查准率/查全率的不同偏好。</p><p>$$F_{\alpha}=(1+\alpha^{2})\frac{Precision*Recall}{\alpha^{2}*Precision+Recall}$$</p><p>当\alpha > 1 时查全率有更大影响，当\alpha < 1 时查准率有更大影响。</p><p>当$\alpha=1$, 即为F1-score，<code>Recall</code>和<code>Precision</code>的调和平均数。<br>$$F_{1}=(2)\frac{Precision*Recall}{Precision+Recall}$$</p><h2 id="多label二分类"><a href="#多label二分类" class="headerlink" title="多label二分类"></a>多label二分类</h2><h3 id="Macro-f1-score"><a href="#Macro-f1-score" class="headerlink" title="Macro-f1-score"></a>Macro-f1-score</h3><p>(宏观角度)先在每个confusion matrix上计算precision和recall，最后计算平均值。</p><p>$$ Precision_{macro} = \frac{1}{N}\sum_{i}Precision_{i} $$<br>$$ Recall_{macro} = \frac{1}{N}\sum_{i}Recall_{i} $$<br>$$ F1_{macro}=(2)\frac{Precision_{macro}*Recall_{macro}}{Precision_{macro}+Recall_{macro}}$$</p><h3 id="Micro-f1-score"><a href="#Micro-f1-score" class="headerlink" title="Micro-f1-score"></a>Micro-f1-score</h3><p>(微观角度)先将每个confusion matrix中各个对应位置上的元素平均，得到TP、FP、TN、FN的平均值，然后计算precision、recall和F1<br>$$Recall_{micro}=\frac{\overline{TP}}{\overline{TP}+\overline{FN}}$$<br>$$Precision_{micro}=\frac{\overline{TP}}{\overline{TP}+\overline{FP}}$$<br>$$F1_{micro}=(2)\frac{Precision_{micro}*Recall_{micro}}{Precision_{micro}+Recall_{micro}}$$</p><h3 id="ROC和AUC-Area-under-curve"><a href="#ROC和AUC-Area-under-curve" class="headerlink" title="ROC和AUC(Area under curve)"></a>ROC和AUC(Area under curve)</h3><p>每次预测一个样本后计算当下的两个值，最后用这两个值作为坐标画曲线。这两个值分别是真正例率（True Positive Rate）和假正例率（False Positive Rate），定义如下：<br>$$ TPR = \frac{TP}{TP+FN}$$<br>$$ FPR = \frac{FP}{TN+FP}$$</p><p>AUC越大越好，考虑了所有阈值。</p><p>ROC曲线的横坐标为false positive rate（FPR）即负类样本中被判定为正类的比例，也就是传说中的误纳率</p><p>纵坐标为true positive rate（TPR）即正类样本中被判定为正类的样本，1-TPR也就是传说中的误拒率</p><p>接下来我们考虑ROC曲线图中的四个点和一条线。</p><p>第一个点，(0,1)，即左上角的点，在这个点意味着FPR=0，TPR=1，稍微翻译一下就是误纳率为0，误拒率为0，再翻译成人话就是负类样本中被判断为正类的比例为0，说明负类样本都被判断为负类，判断正确，正类样本中被判断为正类的比例为1，说明正类样本都被判断正确，所以这是一个完美的分类器，它将所有的样本都正确分类。</p><p>第二个点，(1,0)，即右下角的点，在这个点意味着FPR=1，TPR=0，类似地分析可以发现这是一个最糟糕的分类器，因为它成功避开了所有的正确分类。把该判断为正类的判断为负类，把该判断为负类的判断为正类</p><p>第三个点，(0,0)，即左下角的点，在这个点意味着FPR=TPR=0，可以发现该分类器预测所有的样本都为负样本（negative），在后面我们可以看到这种情况说明阈值选得过高。</p><p>第四个点（1,1），即右下角的点，分类器实际上预测所有的样本都为正样本，在后面我们可以看到这种情况说明阈值选得过低。</p><p>对角线： </p><p>如果是随机分类，则有$Tp=Fp$，$Tn=Fn$，$\Rightarrow$ $FPR=TPR$</p><p>好的分类器：$Tp>Fp$，$Tn>Fn$,，$\Rightarrow$ $TPR=\frac{TP\uparrow}{TP\uparrow+FN\downarrow}>TPR=\frac{FP\downarrow}{FP\downarrow+TN\uparrow}$，位于对角线右侧</p><p>差的分类器：$Tp<Fp$，$Tn<Fn$,，$\Rightarrow$ $TPR=\frac{TP\downarrow}{TP\downarrow+FN\uparrow}>TPR=\frac{FP\uparrow}{FP\uparrow+TN\downarrow}$，位于对角线右侧</p><h1 id="AUC-0"><a href="#AUC-0" class="headerlink" title="AUC = 0"></a>AUC = 0</h1><p>说明样本中没有正例</p><h1 id="如何画ROC曲线"><a href="#如何画ROC曲线" class="headerlink" title="如何画ROC曲线"></a>如何画ROC曲线</h1><p>由于每次从分类模型中只能得到一个用于判定分类结果的分数，要将这个分数与一个阈值进行比较，判定这个样本属于哪个类，所以我们可以更改阈值，得到不同的分类结果，也就是不同的TPR和FPR</p><p>之前说到当我们将threshold设置为1和0时，分别可以得到ROC曲线上的(0,0)和(1,1)两个点</p><p>将这些(FPR,TPR)对连接起来，就得到了ROC曲线。当threshold取值越多，ROC曲线越平滑。</p><p>既然有了ACC为什么要有ROC呢(既生瑜何生亮呢)</p><p>我们知道，我们常用ACC准确率来判断分类器分类结果的好坏，既然有了ACC为什么还需要ROC呢，很重要的一个因素是实际的样本数据集中经常会出现数据偏斜的情况，要么负类样本数大于正类样本数，要么正类样本数大于负类样本数。</p><p>比如说我负类样本数有9,000个，正类样本数有100个，如果阈值选得过高，正类样本都判断为负类，同样负类样本都判定为负类，那么准确率90%，看起来还不错，但是如果考虑ROC中的TPR和FPR的话就会知道，此时的TPR=0，FPR=0，也就是误纳率是0，但是误拒率是100%，是左下角的点，并不是很好的一个点，而原来的ACC就不具有代表性</p><p>既然有了ROC为什么要有AUC呢(既生瑜何生亮呢)</p><p>使用AUC值作为评价标准是因为很多时候ROC曲线并不能清晰的说明哪个分类器的效果更好，而相对于AUC是个数值而言，对应AUC更大的分类器效果更好，数值更好判断一些。</p><h1 id="References"><a href="#References" class="headerlink" title="References"></a>References</h1><p><a href="https://zhuanlan.zhihu.com/p/39957290" target="_blank" rel="noopener">https://zhuanlan.zhihu.com/p/39957290</a></p><p><a href="http://www.cnblogs.com/zle1992/p/6689136.html" target="_blank" rel="noopener">http://www.cnblogs.com/zle1992/p/6689136.html</a></p><p><a href="https://www.zhihu.com/question/39840928" target="_blank" rel="noopener">https://www.zhihu.com/question/39840928</a></p><p><a href="http://www.cnblogs.com/keedor/p/4463988.html" target="_blank" rel="noopener">http://www.cnblogs.com/keedor/p/4463988.html</a></p><p><a href="https://www.cnblogs.com/keedor/category/669162.html" target="_blank" rel="noopener">https://www.cnblogs.com/keedor/category/669162.html</a></p>]]></content>
<summary type="html">
<h1 id="分类任务里的衡量指标"><a href="#分类任务里的衡量指标" class="headerlink" title="分类任务里的衡量指标"></a>分类任务里的衡量指标</h1><h2 id="二分类"><a href="#二分类" class="header
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>tensorflow_tutorial_6</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/02/02/tensorflow-tutorial-6/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/02/02/tensorflow-tutorial-6/</id>
<published>2019-02-02T14:43:55.000Z</published>
<updated>2019-02-17T07:18:54.702Z</updated>
<content type="html"><![CDATA[<h1 id="机器学习中的Loss"><a href="#机器学习中的Loss" class="headerlink" title="机器学习中的Loss"></a>机器学习中的Loss</h1><h2 id="回归任务"><a href="#回归任务" class="headerlink" title="回归任务"></a>回归任务</h2><h3 id="MAE，L1-Loss"><a href="#MAE，L1-Loss" class="headerlink" title="MAE，L1 Loss"></a>MAE，L1 Loss</h3><p>$ L_{MAE} = \frac{1}{N}\sum_i^{N-1}|pred_i - label_i|$</p><h3 id="MSE，L2-Loss"><a href="#MSE，L2-Loss" class="headerlink" title="MSE，L2 Loss"></a>MSE，L2 Loss</h3><p>$ L_{MSE} = \frac{1}{N}\sum_i^{N-1}(pred_i - label_i)^{2}$</p><h2 id="分类任务"><a href="#分类任务" class="headerlink" title="分类任务"></a>分类任务</h2><h3 id="CrossEntropy-Loss"><a href="#CrossEntropy-Loss" class="headerlink" title="CrossEntropy Loss"></a>CrossEntropy Loss</h3><p>$$ loss(x,class)= -log(\frac{exp(x[class])}{\sum_{j}exp(x[j])}) = -x[class]+log(\sum_{j}exp(x[j])) $$</p><h1 id="推导CrossEntropy-Loss"><a href="#推导CrossEntropy-Loss" class="headerlink" title="推导CrossEntropy Loss"></a>推导CrossEntropy Loss</h1><h2 id="one-hot"><a href="#one-hot" class="headerlink" title="one-hot"></a>one-hot</h2><p>定义：用N个二进制位来表示N个状态，对于任意一状态，其N个二进制位，只有1位非零。例如， 0，1,2 分别会表示为001,010,100。</p><p>为什么用one-hot？</p><figure class="image-bubble"> <div class="img-lightbox"> <div class="overlay"></div> <img src="1.jpeg" alt="Iris 数据集" title> </div> <div class="image-caption">Iris 数据集</div> </figure><p>以Iris数据集为例，数据集中共有3种花：Versicolor,Setosa,Virginica。如果直接按顺序给这3种花编号，并将编号作为每种花的label，会存在以下问题：</p><ul><li>欧式距离问题：如果将Versicolor,Setosa,Virginica的label采用编号的方式则是：1,2,3。则$L_{Ver,Setosa}=1$,$L_{Ver,Vir}=2$,$L_{vir,Setosa}=1$,可以发现这三者两两的距离不一致。如果将label编号为001,010,100，则两两之间的距离为$L_{Ver,Setosa}=2$。因而，这样做显得更加合理。</li></ul><h2 id="交叉熵-CrossEntropy"><a href="#交叉熵-CrossEntropy" class="headerlink" title="交叉熵(CrossEntropy)"></a>交叉熵(CrossEntropy)</h2><p>自信息量(Self-information)</p><p>定义： 变量x发生概率的对数取反，表明该变量x携带的信息量的多少。</p><p>$$ I(x) = - log(p(x))$$</p><p>其中，$ p(x) = Pr(X=x) $ 代表了在信息空间X中x出现的的概率。</p><p>熵</p><p>定义： 表明X的不确定程度，熵越大，不确定程度越大，越多的信息量。</p><p>$$H(X) = E[I(X)]$$</p><p>对于离散的X信息空间:</p><p>$$H(X) = \sum_{i} (-p(x_i)logp(x_i)) $$</p><p>对于连续的X信息空间:</p><p>\begin{align}<br>H(x) &= \int_{-\infty}^{+\infty} (p(x)I(x)dx) \\<br> &= \int_{-\infty}^{+\infty}(-p(x)logp(x)dx)<br>\end{align} </p><p>KL 散度 又叫相对熵：</p><p>定义：D(P||Q)表示当用概率分布Q来拟合真实分布P时，产生的信息损耗，其中P表示真实分布，Q表示P的拟合分布。散度越小，意味着Q越接近于P。 </p><p>Noted that：有人将KL散度称为KL距离，但事实上，KL散度并不满足距离的概念</p><ul><li>KL散度不是对称的</li><li>KL散度不满足三角不等式。</li></ul><p>$$ D(p||q) = E_{~p}[log{p(x)/q(x)}]$$</p><p>对于离散的X信息空间:</p><p>$$D(P||Q)=\sum_{i} p(i)logp(i)/q(i)$$</p><p>对于连续的X信息空间:</p><p>$$D(P||Q)=\int_{-\infty}^{+\infty} p(x)logp(x)/q(x)dx$$</p><p>交叉熵: </p><p>定义：$ CEH(p,q) = Ep[-logq]$</p><p>性质： 交叉熵 = 熵 + KL散度</p><p>证明： </p><p>\begin{align}<br>CEH(p,q) &= E_{~p}[-logq] \\<br>&= E_{~p}[-logq/p * p] \\<br>&= E_{~p}[-logp-logq/p] \\<br>&= E_{~p}[-logp] + E_{~p}[-logq/p] \\<br>&= E_{~p}[-logp] + E_{~p}[logp/q] \\<br>&= H(p) + D(p||q)<br>\end{align}</p><p>由此可以发现，当p为真实分布时，其熵$H(p)$是固定不变的。因此，最小化交叉熵，可以被认为是最小化KL散度。 </p><h2 id="交叉熵的损失公式推导"><a href="#交叉熵的损失公式推导" class="headerlink" title="交叉熵的损失公式推导"></a>交叉熵的损失公式推导</h2><p>\begin{align}<br>L_{CEH(p,q)} &= Dkl(p||q) \\<br>&= \sum_{i}p(i)log(\frac{p(i)}{q(i)})<br>\end{align}</p><p>对于二分类问题：</p><p>\begin{align}<br>L_{CEH(p,q)} &= Dkl(p||q) \\<br>&= -p(0)logq(0) - p(1)log(q(1)) \\<br>&= -p(0)logq(0) - [1-p(0)]log[1-q(0)]<br>\end{align} </p><p>对于n分类：</p><p>神经网络的输出层是n维的向量，x[i]为类别i的取值，那么类别为i的概率为：</p><p>$$q(i)= \frac{exp(x[i])}{\sum_{j}exp(x[j])} $$</p><p>这一过程叫做归一化，学名Softmax。</p><p>根据前面所述的one-hot可知，对于单个样本，真实分布p是只在类别正确处有值且为1，所以</p><p>\begin{align}<br>L_{CEH(p,q)} &= Dkl(p||q) \\<br>&= \sum_{i}p(i)log(\frac{p(i)}{q(i)}) \\<br>&= 0 + 1 * log(\frac{1}{q(i)}) \\<br>&= - log(q(i)) \\<br>&= - log(\frac{exp(x[class])}{\sum_{j}exp(x[j]))} \\<br>&= - x[class]+log(\sum_{j}exp(x[j]))<br>\end{align} </p><h2 id="推广至多label"><a href="#推广至多label" class="headerlink" title="推广至多label"></a>推广至多label</h2>]]></content>
<summary type="html">
<h1 id="机器学习中的Loss"><a href="#机器学习中的Loss" class="headerlink" title="机器学习中的Loss"></a>机器学习中的Loss</h1><h2 id="回归任务"><a href="#回归任务" class="head
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>tensorflow_tutorial_5</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/01/31/tensorflow-tutorial-5/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/01/31/tensorflow-tutorial-5/</id>
<published>2019-01-31T08:25:52.000Z</published>
<updated>2019-02-17T07:18:54.702Z</updated>
<content type="html"><![CDATA[<h1 id="Summary"><a href="#Summary" class="headerlink" title="Summary"></a><a href="https://www.tensorflow.org/api_docs/python/tf/summary?hl=en" target="_blank" rel="noopener">Summary</a></h1><h2 id="class-tf-Summary-FileWriter"><a href="#class-tf-Summary-FileWriter" class="headerlink" title="class tf.Summary.FileWriter"></a>class tf.Summary.FileWriter</h2><h3 id="init"><a href="#init" class="headerlink" title="init"></a><strong>init</strong></h3><p>创建一个writer</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">__init__(</span><br><span class="line"> logdir,</span><br><span class="line"> graph=<span class="keyword">None</span>,</span><br><span class="line"> max_queue=<span class="number">10</span>,</span><br><span class="line"> flush_secs=<span class="number">120</span>,</span><br><span class="line"> graph_def=<span class="keyword">None</span>,</span><br><span class="line"> filename_suffix=<span class="keyword">None</span>,</span><br><span class="line"> session=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><h3 id="add-summary"><a href="#add-summary" class="headerlink" title="add_summary"></a>add_summary</h3><p>将summary string添加至事件文件中</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">add_summary(</span><br><span class="line"> summary,</span><br><span class="line"> global_step=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><h3 id="flush"><a href="#flush" class="headerlink" title="flush"></a>flush</h3><p>将事件文件写到磁盘<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">flush()</span><br></pre></td></tr></table></figure></p><h3 id="close"><a href="#close" class="headerlink" title="close"></a>close</h3><p>将事件文件写到磁盘并关闭FileWriter。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">close()</span><br></pre></td></tr></table></figure><h2 id="visulization-Function"><a href="#visulization-Function" class="headerlink" title="visulization Function"></a>visulization Function</h2><h3 id="scalar"><a href="#scalar" class="headerlink" title="scalar"></a>scalar</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">tf.summary.scalar(</span><br><span class="line"> name,</span><br><span class="line"> tensor,</span><br><span class="line"> collections=<span class="keyword">None</span>,</span><br><span class="line"> family=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li><p>name: 显示的名字</p></li><li><p>family： 同一族</p></li><li><p>Return： 一个<code>operation</code></p></li></ul><h3 id="image"><a href="#image" class="headerlink" title="image"></a>image</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">tf.summary.image(</span><br><span class="line"> name,</span><br><span class="line"> tensor,</span><br><span class="line"> max_outputs=<span class="number">3</span>,</span><br><span class="line"> collections=<span class="keyword">None</span>,</span><br><span class="line"> family=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><ul><li><p>name: 显示的名字</p></li><li><p>tensor： shape [batch_size, height, width, channels]， channels可以是1,3,4，分别对应于Grayscale，RGB，RGBA。</p></li></ul><h3 id="histogram"><a href="#histogram" class="headerlink" title="histogram"></a>histogram</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.summary.histogram(</span><br><span class="line"> name,</span><br><span class="line"> values,</span><br><span class="line"> collections=<span class="keyword">None</span>,</span><br><span class="line"> family=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><ul><li><p>name: 显示的名称</p></li><li><p>values： 处理的数据，可以是任意形状</p></li></ul><h3 id="text"><a href="#text" class="headerlink" title="text"></a>text</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.summary.text(</span><br><span class="line"> name,</span><br><span class="line"> tensor,</span><br><span class="line"> collections=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><ul><li>tensor: a string-type Tensor to summarize.</li></ul><h3 id="audio"><a href="#audio" class="headerlink" title="audio"></a><a href="https://www.tensorflow.org/api_docs/python/tf/summary/audio?hl=en" target="_blank" rel="noopener">audio</a></h3><h2 id="合并可视化操作"><a href="#合并可视化操作" class="headerlink" title="合并可视化操作"></a>合并可视化操作</h2><p>目的: 使得1次forward propagation，获取多个summary</p><h3 id="tf-summary-merge"><a href="#tf-summary-merge" class="headerlink" title="tf.summary.merge"></a>tf.summary.merge</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.summary.merge(</span><br><span class="line"> inputs,</span><br><span class="line"> collections=<span class="keyword">None</span>,</span><br><span class="line"> name=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><ul><li>inputs: A list of string Tensor objects containing serialized Summary protocol buffers.</li></ul><h3 id="tf-summary-merge-all"><a href="#tf-summary-merge-all" class="headerlink" title="tf.summary.merge_all"></a>tf.summary.merge_all</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">tf.summary.merge_all(</span><br><span class="line"> key=tf.GraphKeys.SUMMARIES,</span><br><span class="line"> scope=<span class="keyword">None</span>,</span><br><span class="line"> name=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Merges all summaries collected in the default graph.</p><p>Args:</p><ul><li>key: GraphKey used to collect the summaries. Defaults to GraphKeys.SUMMARIES.</li><li>scope: Optional scope used to filter the summary ops, using re.match</li></ul><p>Return: 返回一个ops</p><h2 id="Conclusion"><a href="#Conclusion" class="headerlink" title="Conclusion"></a>Conclusion</h2><p>可视化主要流程：</p><ul><li><p>预定义Summary objects</p></li><li><p>合并Summary objects</p></li><li><p>启动Session</p></li><li><p>定义tf.summary.FileWriter</p></li><li><p>获取summary string ： sess.run(summary_ops， feed_dict={})</p></li><li><p>添加至事件文件</p></li><li><p>写入磁盘</p></li></ul><p>For example</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">import</span> tensorflow <span class="keyword">as</span> tf</span><br><span class="line"></span><br><span class="line">var = tf.Variable(tf.zeros([], name=<span class="string">'var'</span>))</span><br><span class="line"></span><br><span class="line">scalar_op = tf.Summary.scalar(<span class="string">'var'</span>, var)</span><br><span class="line"></span><br><span class="line">merge_op = tf.Summary.merge_all()</span><br><span class="line"></span><br><span class="line">init_op = tf.global_variables_initializer()</span><br><span class="line"></span><br><span class="line"><span class="keyword">with</span> tf.Session() <span class="keyword">as</span> sess:</span><br><span class="line"></span><br><span class="line">writer = tf.Summary.FileWriter(<span class="string">'./logs'</span>)</span><br><span class="line"></span><br><span class="line">sess.run(init_op)</span><br><span class="line"></span><br><span class="line">summary_string = sess.run(merge_op)</span><br><span class="line"></span><br><span class="line">writer.add_summary(summary_string, global_step=<span class="number">0</span>)</span><br><span class="line"></span><br><span class="line">writer.flush()</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h1 id="Summary"><a href="#Summary" class="headerlink" title="Summary"></a><a href="https://www.tensorflow.org/api_docs/python/tf/summary?hl
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>tensorflow_tutorial_4</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/01/31/tensorflow-tutorial-4/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/01/31/tensorflow-tutorial-4/</id>
<published>2019-01-31T08:25:48.000Z</published>
<updated>2019-02-17T07:18:54.701Z</updated>
<content type="html"><![CDATA[<h1 id="class-saver"><a href="#class-saver" class="headerlink" title="class saver"></a>class <a href="https://www.tensorflow.org/api_docs/python/tf/train/Saver" target="_blank" rel="noopener">saver</a></h1><p>作用：用于保存和恢复网络的参数</p><h2 id="Function"><a href="#Function" class="headerlink" title="Function"></a>Function</h2><h3 id="Init"><a href="#Init" class="headerlink" title="Init"></a>Init</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">__init__(</span><br><span class="line"> var_list=<span class="keyword">None</span>,</span><br><span class="line"> reshape=<span class="keyword">False</span>,</span><br><span class="line"> sharded=<span class="keyword">False</span>,</span><br><span class="line"> max_to_keep=<span class="number">5</span>,</span><br><span class="line"> keep_checkpoint_every_n_hours=<span class="number">10000.0</span>,</span><br><span class="line"> name=<span class="keyword">None</span>,</span><br><span class="line"> restore_sequentially=<span class="keyword">False</span>,</span><br><span class="line"> saver_def=<span class="keyword">None</span>,</span><br><span class="line"> builder=<span class="keyword">None</span>,</span><br><span class="line"> defer_build=<span class="keyword">False</span>,</span><br><span class="line"> allow_empty=<span class="keyword">False</span>,</span><br><span class="line"> write_version=tf.train.SaverDef.V2,</span><br><span class="line"> pad_step_number=<span class="keyword">False</span>,</span><br><span class="line"> save_relative_paths=<span class="keyword">False</span>,</span><br><span class="line"> filename=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:</p><p>var_list: 指定保存的变量。可以使list或者dict</p><p>max_to_keep: 最近可以保存ckeckpoints的最大数目</p><p>keep_checkpoint_every_n_hours： 多久保存一次ckeckpoints<br><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br><span class="line">v1 = tf.Variable(..., name=<span class="string">'v1'</span>)</span><br><span class="line">v2 = tf.Variable(..., name=<span class="string">'v2'</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># Pass the variables as a dict:</span></span><br><span class="line">saver = tf.train.Saver({<span class="string">'v1'</span>: v1, <span class="string">'v2'</span>: v2})</span><br><span class="line"></span><br><span class="line"><span class="comment"># Or pass them as a list.</span></span><br><span class="line">saver = tf.train.Saver([v1, v2])</span><br><span class="line"><span class="comment"># Passing a list is equivalent to passing a dict with the variable op names</span></span><br><span class="line"><span class="comment"># as keys:</span></span><br><span class="line">saver = tf.train.Saver({v.op.name: v <span class="keyword">for</span> v <span class="keyword">in</span> [v1, v2]})</span><br></pre></td></tr></table></figure></p><h2 id="save"><a href="#save" class="headerlink" title="save"></a>save</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">save(</span><br><span class="line"> sess,</span><br><span class="line"> save_path,</span><br><span class="line"> global_step=<span class="keyword">None</span>,</span><br><span class="line"> latest_filename=<span class="keyword">None</span>,</span><br><span class="line"> meta_graph_suffix=<span class="string">'meta'</span>,</span><br><span class="line"> write_meta_graph=<span class="keyword">True</span>,</span><br><span class="line"> write_state=<span class="keyword">True</span>,</span><br><span class="line"> strip_default_attrs=<span class="keyword">False</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>作用：保存网络参数</p><p>Args:<br>sess: 会话句柄<br>save_path：保存路径<br>global_step：步数</p><h2 id="restore"><a href="#restore" class="headerlink" title="restore"></a>restore</h2><p>作用：恢复网络参数</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">restore(</span><br><span class="line"> sess,</span><br><span class="line"> save_path</span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>Args:<br>sess: 会话句柄<br>save_path：保存路径</p><h2 id="Example"><a href="#Example" class="headerlink" title="Example"></a>Example</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># Create a saver.</span></span><br><span class="line">saver = tf.train.Saver(...variables...)</span><br><span class="line"><span class="comment"># Launch the graph and train, saving the model every 1,000 steps.</span></span><br><span class="line">sess = tf.Session()</span><br><span class="line"><span class="keyword">for</span> step <span class="keyword">in</span> xrange(<span class="number">1000000</span>):</span><br><span class="line"> sess.run(..training_op..)</span><br><span class="line"> <span class="keyword">if</span> step % <span class="number">1000</span> == <span class="number">0</span>:</span><br><span class="line"> <span class="comment"># Append the step number to the checkpoint name:</span></span><br><span class="line"> saver.save(sess, <span class="string">'my-model'</span>, global_step=step)</span><br></pre></td></tr></table></figure><h2 id="查看保存文件里的tensor"><a href="#查看保存文件里的tensor" class="headerlink" title="查看保存文件里的tensor"></a><a href="https://www.jianshu.com/p/27d7955da2f6" target="_blank" rel="noopener">查看保存文件里的tensor</a></h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">from</span> tensorflow.python <span class="keyword">import</span> pywrap_tensorflow</span><br><span class="line"></span><br><span class="line"><span class="comment"># 从检查点文件中读取数据</span></span><br><span class="line">reader = pywrap_tensorflow.NewCheckpointReader(checkpoint_path)</span><br><span class="line">var_to_shape_map = reader.get_variable_to_shape_map()</span><br><span class="line"><span class="comment"># 显示变量名及其值</span></span><br><span class="line"><span class="keyword">for</span> key <span class="keyword">in</span> var_to_shape_map:</span><br><span class="line"> print(<span class="string">"tensor_name: "</span>, key)</span><br><span class="line"> print(reader.get_tensor(key))</span><br></pre></td></tr></table></figure><p>保存文件：</p><ul><li>.meta文件保存了当前图结构</li><li>.index文件保存了当前参数名</li><li>.data文件保存了当前参数值</li></ul><p>注：每调用一次save方法会产生新的文件</p><h2 id="To-do"><a href="#To-do" class="headerlink" title="To do"></a>To do</h2><ul><li style="list-style: none"><input type="checkbox"> transfer learning 里如何恢复部分模型参数</li></ul>]]></content>
<summary type="html">
<h1 id="class-saver"><a href="#class-saver" class="headerlink" title="class saver"></a>class <a href="https://www.tensorflow.org/api_docs/py
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>tensorflow_tutorial_2</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/01/31/tensorflow-tutorial-2/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/01/31/tensorflow-tutorial-2/</id>
<published>2019-01-31T03:31:24.000Z</published>
<updated>2019-02-17T07:18:54.697Z</updated>
<content type="html"><![CDATA[<h1 id="Tensors"><a href="#Tensors" class="headerlink" title="Tensors"></a>Tensors</h1><p>注： 必须初始化，才能使用</p><p>eval()</p><p>run()</p><h2 id="Variable"><a href="#Variable" class="headerlink" title="Variable"></a>Variable</h2><p>作用：用于记录graph的状态，即，记录拟合函数$g(X)$的权重。</p><p>创建Variable操作具有以下作用：</p><ul><li>将该Variable作为graph的一个节点并<code>必须</code>指定<code>初始化器</code>或者<code>初值</code></li><li>将该Variable放到键值为tf.GraphKeys.GLOBAL_VARIABLES的collections集合中</li><li>若trainable为true，则将该Variable放到键值为tf.GraphKeys.TRAINABLE_VARIABLES的collections集合</li></ul><p>注：trainable为true表明该变量会进行更新，collections可以认为是一个空间，用于存放变量。</p><p>变量创建方式1：</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">var = tf.Variable(<initial-value>, name=<optional-name>, dtype=tf.float32, trainable=true)</span><br></pre></td></tr></table></figure><p>以此种方式创建变量var， var与initial-value的数据和形状一致，若定义出现两个名字一致的变量，会在后面定义的变量名称加上后缀<code></code>_1,_2,_3,…”。</p><p>tensorflow定义了如下函数可以用来指定初值：</p><table><thead><tr><th style="text-align:center">Function</th><th style="text-align:center">Description</th></tr></thead><tbody><tr><td style="text-align:center">tf.zeros(shape, dtype=tf.float32,name=None)</td><td style="text-align:center">产生全0的tensor</td></tr><tr><td style="text-align:center">tf.ones(shape, dtype=tf.float32,name=None)</td><td style="text-align:center">产生全1的tensor</td></tr><tr><td style="text-align:center">tf.random.normal(shape, mean=0.0, stddev=1.0)</td><td style="text-align:center">产生服从正态分布的tensor</td></tr><tr><td style="text-align:center">tf.random.uniform(shape, minval=0.0, maxval=NOne)</td><td style="text-align:center">产生服从均匀分布的tensor</td></tr><tr><td style="text-align:center">tf.random.truncated_normal(shape, mean=0.0, stddev=1.0)</td><td style="text-align:center">产生服从截断正态分布的tensor</td></tr></tbody></table><p>注：shape参数用来指定产生随机数的维度</p><table><thead><tr><th style="text-align:center">List</th><th style="text-align:center">Description</th></tr></thead><tbody><tr><td style="text-align:center">[]</td><td style="text-align:center">0维，标量</td></tr><tr><td style="text-align:center">[d1]</td><td style="text-align:center">1维</td></tr><tr><td style="text-align:center">[d1,d2]</td><td style="text-align:center">2维</td></tr><tr><td style="text-align:center">[None,d2]</td><td style="text-align:center">2维,第一维度数目可以是任意值</td></tr><tr><td style="text-align:center">[d1,d2,d3]</td><td style="text-align:center">3维</td></tr><tr><td style="text-align:center">[d1,d2,d3,…]</td><td style="text-align:center">n维</td></tr></tbody></table><p>变量创建方式2：</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">var = tf.get_variable(name, shape=<span class="keyword">None</span>, dtype=<span class="keyword">None</span>, initializer=<span class="keyword">None</span>, trainable=true)</span><br></pre></td></tr></table></figure><p>创建名为name的变量，初始化器由initializer指定，形状由shape指定。 在不复用变量的情况下，若定义两个名字一样的变量，则会报错。</p><p>Tensorflow内的<a href="https://www.tensorflow.org/api_docs/python/tf/initializers?hl=zh-cn" target="_blank" rel="noopener">initializer</a>有：</p><table><thead><tr><th style="text-align:center">class</th><th style="text-align:center">constructor</th></tr></thead><tbody><tr><td style="text-align:center">tf.contrib.layers.xavier_initializer</td><td style="text-align:center">~(uniform=True)，uniform=False采用正态分布</td></tr><tr><td style="text-align:center">tf.random_normal_initializer</td><td style="text-align:center">~(mean=0.0, stddev=1.0)</td></tr><tr><td style="text-align:center">tf.random_uniform_initializer</td><td style="text-align:center">~(minval=0, maxval=None)</td></tr><tr><td style="text-align:center">tf.truncated_normal_initializer</td><td style="text-align:center">~(mean=0.0, stddev=1.0)</td></tr><tr><td style="text-align:center">tf.ones_initializer</td><td style="text-align:center">~()</td></tr><tr><td style="text-align:center">tf.zeros_initializer</td><td style="text-align:center">~()</td></tr></tbody></table><p>For Examples:</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">var = tf.Variable(tf.random.normal([<span class="number">1</span>,<span class="number">2</span>,<span class="number">3</span>]),name=<span class="string">"var"</span>)</span><br><span class="line"></span><br><span class="line"><span class="comment"># 等价于 #</span></span><br><span class="line"></span><br><span class="line">var = tf.getVariable(<span class="string">"var"</span>,shape=[<span class="number">1</span>,<span class="number">2</span>,<span class="number">3</span>],initializer=tf.random_normal_initializer())</span><br></pre></td></tr></table></figure><h2 id="Constant"><a href="#Constant" class="headerlink" title="Constant"></a>Constant</h2><p><a href="https://tensorflow.google.cn/api_docs/python/tf/constant" target="_blank" rel="noopener">常量</a>,在图中值不改变，不能进行reshape操作。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">tf.constant(</span><br><span class="line"> value,</span><br><span class="line"> dtype=<span class="keyword">None</span>,</span><br><span class="line"> shape=<span class="keyword">None</span>,</span><br><span class="line"> name=<span class="string">'Const'</span>,</span><br><span class="line"> verify_shape=<span class="keyword">False</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><p>For Examples:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"># Constant 1-D Tensor populated with value list.</span><br><span class="line">tensor = tf.constant([1, 2, 3, 4, 5, 6, 7]) => [1 2 3 4 5 6 7]</span><br><span class="line"></span><br><span class="line"># Constant 2-D tensor populated with scalar value -1.</span><br><span class="line">tensor = tf.constant(-1.0, shape=[2, 3]) => [[-1. -1. -1.]</span><br><span class="line"> [-1. -1. -1.]]</span><br></pre></td></tr></table></figure><h2 id="placeholder"><a href="#placeholder" class="headerlink" title="placeholder"></a>placeholder</h2><p><a href="https://tensorflow.google.cn/api_docs/python/tf/placeholder" target="_blank" rel="noopener">placeholder</a>用于graph的输入,一般指定shape和dtype，name。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">tf.placeholder(</span><br><span class="line"> dtype,</span><br><span class="line"> shape=<span class="keyword">None</span>,</span><br><span class="line"> name=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><h1 id="manager-Tensors"><a href="#manager-Tensors" class="headerlink" title="manager Tensors"></a>manager Tensors</h1><p><img src="collections.png" width="500" hegiht="313" align="center"></p><p>如图所示，在Tensorflow中，采用层次的结构来管理operations：<code>collection</code> $\leftarrow$ <code>variable scope</code> $\leftarrow$ <code>operations</code>。 从该图还可以得到如下信息：</p><ul><li>tensorflow图自定义了多个集合，例如集合1,2。集合1,2分别存放所有的，只用于训练的操作和张量。</li><li>集合通过键值进行区分，例如集合1,2的键值分别为tf.GraphKeys.GLOBAL_VARIABLES，tf.GraphKeys.TRAINABLE_VARIABLES。</li><li>为了避免出现操作和张量同名，定义了次级空间variable scope。在同一scope下的操作和张量具有相同的前缀名。该scope类似于c++里的namespace。</li><li>同一scope可以同时在多个集合中，图中v1，v2都属于集合1,2。</li><li>也可以存放一些没有在variable scope内的张量和操作。</li></ul><h2 id="variable-scope"><a href="#variable-scope" class="headerlink" title="variable scope"></a>variable scope</h2><p>作用：定义一个子空间，存放操作和张量。<br>定义: <a href="https://tensorflow.google.cn/api_docs/python/tf/variable_scope" target="_blank" rel="noopener">tf.variable_scope</a></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">with</span> tf.variable_scope(name, reuse=false):</span><br><span class="line"></span><br><span class="line"><span class="comment"># define your tensors</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># define your operations</span></span><br></pre></td></tr></table></figure><p>注：当reuse=false时，利用tf.get_variable定义tensor时不能出现两个同名张量。如若为True，后定义的张量与前面定义的张量一致。此外，对Variable定义tensor无影响。</p><p>For example<br><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">with</span> tf.variable_scope(<span class="string">"hello"</span>):</span><br><span class="line"></span><br><span class="line">var = tf.get_variable(<span class="string">"var"</span>, shape =[<span class="number">1</span>,<span class="number">2</span>], intializer=tf.zeros_initializer())</span><br><span class="line"></span><br><span class="line">var1 = tf.get_variable("var", shape =[1,2], intializer=tf.zeros_initializer()) ==> error</span><br></pre></td></tr></table></figure></p><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"><span class="keyword">with</span> tf.variable_scope(<span class="string">"hello"</span>, reuse=<span class="keyword">True</span>):</span><br><span class="line"></span><br><span class="line">var = tf.get_variable(<span class="string">"var"</span>, shape =[<span class="number">1</span>,<span class="number">2</span>], intializer=tf.zeros_initializer())</span><br><span class="line"></span><br><span class="line">var1 = tf.get_variable(<span class="string">"var"</span>, shape =[<span class="number">1</span>,<span class="number">2</span>], intializer=tf.zeros_initializer()) </span><br><span class="line"></span><br><span class="line">assert var == var1 ==> True</span><br></pre></td></tr></table></figure><h2 id="collection"><a href="#collection" class="headerlink" title="collection"></a>collection</h2><p>作用：存放操作和张量。</p><p>操作：</p><ul><li>从某个collection<a href="https://www.tensorflow.org/api_docs/python/tf/get_collection?hl=zh-cn" target="_blank" rel="noopener">获取</a>某个scope的variable集合：</li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.get_collection(</span><br><span class="line"> key,</span><br><span class="line"> scope=<span class="keyword">None</span></span><br><span class="line">)</span><br></pre></td></tr></table></figure><ul><li>从某个collection获取所有的variable：</li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.get_collection(key)</span><br></pre></td></tr></table></figure><ul><li>将variable <a href="https://tensorflow.google.cn/api_docs/python/tf/add_to_collection" target="_blank" rel="noopener">添加</a>至自定义集合name</li></ul><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">tf.add_to_collection(</span><br><span class="line"> name,</span><br><span class="line"> value</span><br><span class="line">)</span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h1 id="Tensors"><a href="#Tensors" class="headerlink" title="Tensors"></a>Tensors</h1><p>注： 必须初始化，才能使用</p>
<p>eval()</p>
<p>run()</p>
<h2 i
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>tensorflow_tutorial_1</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/01/30/tensorflow-tutorial-1/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/01/30/tensorflow-tutorial-1/</id>
<published>2019-01-30T07:50:15.000Z</published>
<updated>2019-02-17T07:18:54.659Z</updated>
<content type="html"><![CDATA[<h1 id="Basic-concepts"><a href="#Basic-concepts" class="headerlink" title="Basic concepts"></a>Basic concepts</h1><h2 id="Preface"><a href="#Preface" class="headerlink" title="Preface"></a>Preface</h2><p>机器学习中的大部分任务是去学习一个将输入$X$映射到标签$Y$的映射函数$f(X)$: $X \rightarrow Y$，由于在实际中，我们很难通过解析式求得$f(X)$的具体表达形式。因而，我们希望通过数值迭代的方式去求得一个近似函数$g(X;\theta)$并且使得$g(X)$尽可能地逼近$f(X)$: $g(X;\theta) \rightarrow f(X)$。</p><h2 id="Graph"><a href="#Graph" class="headerlink" title="Graph"></a>Graph</h2><figure class="image-bubble"> <div class="img-lightbox"> <div class="overlay"></div> <img src="tensors_flowing.gif" alt="图1 graph的框架" title> </div> <div class="image-caption">图1 graph的框架</div> </figure><p><code>Tensorflow</code>将这一过程表征为<code>graph</code>的形式。graph的组成可以分成以下3部分：</p><ul><li>参数化层： <code>Input</code> $\rightarrow$ <code>Reshape</code> $\rightarrow$ <code>ReLu Layer</code> $\rightarrow$<code>ReLu Layer</code> $\rightarrow$ <code>softmax</code></li><li>逼近量化层： <code>Input</code> $\rightarrow$ <code>Class Labels</code>$\rightarrow$ <code>Cross Entropy</code></li><li>参数更新层： <code>Gradients</code> $\rightarrow$ <code>Logit Layer</code> $\rightarrow$ <code>ReLu Layer</code></li></ul><p>graph由<code>节点</code>和<code>边</code>组成，节点和边分别对应<code>操作(operations)</code>和<code>张量(Tensors)</code>。 操作(operations)包含:</p><ul><li>定义Tensor</li><li>基于Tensor的运算($e.g.$ +-*/)。</li></ul><p>从动态图可知，graph中的Tensors flow(流动)分为两步:</p><ul><li><p>forward propagation: <code>Input</code> $\rightarrow$ <code>Reshape</code> $\rightarrow$ <code>ReLu Layer</code> $\rightarrow$ <code>Logit Layer</code> $\rightarrow$ <code>softmax</code></p></li><li><p>backpropagation: <code>Gradients</code> $\rightarrow$ <code>Logit Layer</code> $\rightarrow$ <code>ReLu Layer</code></p></li></ul><p><code>forward propagation</code> 用于预测输入数据对应的label; <code>backpropagation</code> 用于更新网络参数$\theta$，使得$g(X)$尽可能地逼近$f(X)$。此外，从图中可以看出，为了方便区分operations，定义了作用域(scope), $e.g.$ <code>ReLu Layer</code>, <code>Logit Layer</code>。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 显示定义图：（默认是定义了一张图）</span></span><br><span class="line">g = tf.Graph()</span><br><span class="line"><span class="comment"># 往图里添加operations</span></span><br><span class="line"><span class="keyword">with</span> g.as_default():</span><br><span class="line">c = tf.constant(<span class="number">30.0</span>)</span><br><span class="line"><span class="keyword">assert</span> c.graph == g</span><br><span class="line"><span class="comment"># 返回当前图</span></span><br><span class="line">g = tf.get_default_graph()</span><br><span class="line"><span class="comment"># 查看当前图的operations</span></span><br><span class="line">operations = g.get_operations() <span class="comment">#list</span></span><br></pre></td></tr></table></figure><h2 id="Tensors"><a href="#Tensors" class="headerlink" title="Tensors"></a>Tensors</h2><p>张量是对矢量和矩阵向潜在的更高维度的泛化，TensorFlow 在内部将张量表示为基本数据类型的 n 维数。</p><p>Tensor的属性：</p><ul><li>数据类型(dtype): $e.g.$ <code>tf.float32</code>, <code>tf.int32</code>, <code>tf.string</code></li><li>形状(shape)</li><li>名称(name)</li></ul><p>Tensor的阶的定义</p><table><thead><tr><th style="text-align:center">rank</th><th style="text-align:center">Description</th></tr></thead><tbody><tr><td style="text-align:center">0</td><td style="text-align:center">标量</td></tr><tr><td style="text-align:center">1</td><td style="text-align:center">一维数组</td></tr><tr><td style="text-align:center">2</td><td style="text-align:center">二维数组</td></tr><tr><td style="text-align:center">3</td><td style="text-align:center">三维数组</td></tr><tr><td style="text-align:center">n</td><td style="text-align:center">n维数组</td></tr></tbody></table><p>注:<code>rank = len(tensor.shape)=tf.rank(tensor)</code></p><p>Tensor的分类</p><table><thead><tr><th style="text-align:center">kind</th><th style="text-align:center">Description</th></tr></thead><tbody><tr><td style="text-align:center">tf.Variable</td><td style="text-align:center">用于存储graph的状态</td></tr><tr><td style="text-align:center">tf.placeholder</td><td style="text-align:center">用于graph的输入</td></tr><tr><td style="text-align:center">tf.constant</td><td style="text-align:center">用于存常量</td></tr><tr><td style="text-align:center">tf.SparseTensor</td><td style="text-align:center">用于稀疏张量</td></tr></tbody></table><p>Tensor Auxiliary:</p><ul><li>改变形状，使用<code>tf.reshape(tensor, new_shape)</code></li><li>改变数据类型，使用<code>tf.cast(tensor，new_dtype)</code></li><li>获取Tensor的值，在某个Session下使用<code>tensor.eval()</code></li></ul><h2 id="Session"><a href="#Session" class="headerlink" title="Session"></a>Session</h2><p>会话(Session)和图(graph)相互绑定的，graph只是定义了符号运算，但并未获得实际的物理运行环境，会话(Session)则为graph提供了可实际运行的上下文。因而，在一个会话下，才能进行数值运算。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># create Session</span></span><br><span class="line"></span><br><span class="line">tf.Session(target=<span class="string">''</span>,graph=<span class="keyword">None</span>,config=<span class="keyword">None</span>) <span class="comment">#graph默认指定为当前默认图</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># usage1：</span></span><br><span class="line"></span><br><span class="line">define some operations</span><br><span class="line"></span><br><span class="line">sess = tf.Session()</span><br><span class="line"></span><br><span class="line">sess.run(ops, feed_dict={a: xxx})</span><br><span class="line"></span><br><span class="line">sess.close()</span><br><span class="line"></span><br><span class="line"><span class="comment"># usage2:</span></span><br><span class="line"></span><br><span class="line">define some operations</span><br><span class="line"></span><br><span class="line"><span class="keyword">with</span> tf.Session() <span class="keyword">as</span> sess:</span><br><span class="line"></span><br><span class="line">sess.run(ops, feed_dict={a: xxx}) <span class="comment">#feed_dict{xxx} 对应于该ops所需要的输入值</span></span><br></pre></td></tr></table></figure>]]></content>
<summary type="html">
<h1 id="Basic-concepts"><a href="#Basic-concepts" class="headerlink" title="Basic concepts"></a>Basic concepts</h1><h2 id="Preface"><a href=
</summary>
<category term="tensorflow" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/tensorflow/"/>
</entry>
<entry>
<title>linuxweek1</title>
<link href="https://github.com/SunshineJunFu/JunFu.github.io/2019/01/12/linuxweek1/"/>
<id>https://github.com/SunshineJunFu/JunFu.github.io/2019/01/12/linuxweek1/</id>
<published>2019-01-11T16:25:58.000Z</published>
<updated>2019-02-17T07:18:54.649Z</updated>
<content type="html"><![CDATA[<h1 id="Linux"><a href="#Linux" class="headerlink" title="Linux"></a>Linux</h1><h2 id="基本的Linux命令"><a href="#基本的Linux命令" class="headerlink" title="基本的Linux命令"></a>基本的Linux命令</h2><ul><li>man<ul><li>man name or man section name or man -k regexp</li><li>section: 1 ->命令, 2 -> 系统调用 3 -> 库函数</li></ul></li><li>date<ul><li>读取系统的时间和日期</li><li>定制输出格式： data ”+%Y-%m-%d %H:%M:%S Day %j” or data “+%s”</li><li>ntpdate 0.pool.ntp.org 设置时间（only root)</li><li>ntpdate -q 0.pool.ntp.org 查询时间</li></ul></li><li>cal<ul><li>cal 当前月份日历</li><li>cal year </li><li>cal month year</li></ul></li><li>bc<ul><li>bc 缺省精度为数点后0位</li><li>bc -l 缺省精度为数点后20位</li></ul></li><li>passwd<ul><li>passwd 普通用户修改登陆密码</li><li>passwd 用户名 root 可重置用户的密码</li></ul></li></ul><h2 id="了解系统状态的命令"><a href="#了解系统状态的命令" class="headerlink" title="了解系统状态的命令"></a>了解系统状态的命令</h2><ul><li>who<ul><li>确定谁在系统中</li><li>who 列出已登录系统的用户</li><li>who am i </li><li>whoami</li></ul></li><li>uptime<ul><li>已开机时间， CPU负载， 登陆用户数</li></ul></li><li>top</li><li>free</li><li>vmstat</li><li>ps<ul><li>e 所有进程</li><li>f full格式</li><li>l long格式</li></ul></li></ul><h2 id="文本处理命令"><a href="#文本处理命令" class="headerlink" title="文本处理命令"></a>文本处理命令</h2><ul><li>more/less<ul><li>more filename</li><li>满屏后，空格翻页，换行，向下滚动一行， q 退出， /pattern， /</li><li>不支持回退</li><li>less is more， 可利用上下箭头，支持回退</li></ul></li><li>cat/od<ul><li>文本格式打印 cat filename or cat -n filename 带行号</li><li>od 逐字节打印</li><li>od -c/-t c(字符), -t x1(16进制), -t d1(10进制), -t u1(8进制)</li></ul></li><li>head/tail<ul><li>head -n number filename / tail -n number filename</li><li>head -n -number filename 除末尾number行，均为头</li><li>tail -n +number filename 除首number行，均为尾</li><li>tail -f filename 实时打印文件尾部信息</li></ul></li><li>tee<ul><li>tee filename 将stdin的数据送到stdout，并存入文件filename</li></ul></li><li>wc<ul><li>列出文件一共多行，多少单词，多少字符</li><li>文件大于1，会有个合计</li><li>-l 只输出行</li></ul></li><li>sort<ul><li>sort -n 按数值大小排顺序，不按字符比较规则</li></ul></li><li>tr<ul><li>翻译字符</li><li>tr string1 string2 翻译string1至string2</li></ul></li><li>uniq <ul><li>uniq options inputfile </li><li>-u 只保留没有重复的行（只打印一次）</li><li>-d 只保留重复的行（只打印一次）</li><li>-c 计数同样的行出现的次数</li></ul></li></ul>]]></content>
<summary type="html">
<h1 id="Linux"><a href="#Linux" class="headerlink" title="Linux"></a>Linux</h1><h2 id="基本的Linux命令"><a href="#基本的Linux命令" class="headerlink"
</summary>
<category term="linux" scheme="https://github.com/SunshineJunFu/JunFu.github.io/tags/linux/"/>
</entry>
</feed>