# Tensorflow Ops

Tensorflow is a famous library for creating deep learning models. We can run efficient deep learning algorithms in Google scale. But it’s more GENERAL library than you thought. I have found that in this lessons. So I want to write several useful tips about Tensorflow Ops.

## Visualize you graph on Tensorboard

Tensorboard visualize your graph in fine way easily.

import tensorflow as tf

with tf.Session() as sess:
writer = tf.summary.FileWriter('./graphs', sess.graph)
sess.run()

writer.close()


The graph information is written under ./graphs directory. You can launch your tensorboard.

\$ tensorboard --logdir="./graphs" ## Constant

You can add the name shown in tensorboard.

a = tf.constant([2.0, 3.0], name='a')


Some ops has syntax sugar.

a = tf.constant(2.0, name='a')
b = tf.constant(3.0, name='b')

# Same result
c = a + b


Filled with specific values.

tf.zeros([2, 3], tf.int32) # ==> [[0, 0, 0], [0, 0, 0]]
tf.ones([2, 3], tf.int32)  # ==> [[1, 1, 1], [1, 1, 1]]

t = tf.constant([[1, 2], [3, 4]])
tf.zeros_like(t) # ==> [[0, 0], [0, 0]]
tf.ones_like(t)  # ==> [[1, 1], [1, 1]]

tf.fill([2, 3], 8) # ==> [[8, 8, 8], [8, 8, 8]]


Sequences of constant values.

tf.linspace(10.0, 20.0, 4) # ==> [10., 13.33333302, 16.66666603,20.]

tf.range(1.0, 4.0, 1.2) # ==> [1., 2.20000005, 3.4000001]


Randomly generated constants.

tf.random_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None, name=None)
tf.truncated_normal(shape, mean=0.0, stddev=1.0, dtype=tf.float32, seed=None,
name=None)
tf.random_uniform(shape, minval=0, maxval=None, dtype=tf.float32, seed=None,
name=None)
tf.random_shuffle(value, seed=None, name=None)
tf.random_crop(value, size, seed=None, name=None)
tf.multinomial(logits, num_samples, seed=None, name=None)
tf.random_gamma(shape, alpha, beta=None, dtype=tf.float32, seed=None, name=None)


One thing to be noted here is that constant are stored in the graph definition itself. If you add a huge constant which consumes a lot of memory, it is recommended to be added as Variable. Otherwise loading time of model definition can be longer.

## Data Types

Tensorflow ops can receive Python native types as well.

t_0 = 42
tf.zeros_like(t_0) # ==> 0

t_1 = ["apple", "peach", "banana"]
tf.zeros_like(t_1) # ==> ['', '', '']
tf.ones_like(t_1) # ==> TypeError is thrown

t_2 = [[True, True], [False, False]]
tf.zeros_like(t_2) # ==> All False matrix
tf_ones_like(t_2)  # ==> All True matrix


ones is not defined in string value in Tensorflow. Tensorflow can receive numpy data types as well.

import tensorflow as tf
import numpy as np
tf.int32 == np.int32 # True


But basically, it’s not recommended because the compatibility between TensorFlow and numpy can be broken in future release.

## Variable

Variable is in-memory buffer to hold variables of the model.

v = tf.Variable([1,2], name='v')


Variable is a class, on the other hand constant is a TensorFlow op. So please note Variable’s first latter is upper case. Actually v has several ops internally.

v.initializer
v.value()
v.assign(...)


You have to initialize all variables in advance.

init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)


You can see the actual value of Variable by using eval method.

W = tf.Variable(tf.truncated_normal([700, 10]))
with tf.Session() as sess:
sess.run(W.initializer)
print W.eval()
>> [[-0.76781619 -0.67020458 1.15333688 ..., -0.98434633 -1.25692499
-0.90904623]
[-0.36763489 -0.65037876 -1.52936983 ..., 0.19320194 -0.38379928
0.44387451]
[ 0.12510735 -0.82649058 0.4321366 ..., -0.3816964 0.70466036
1.33211911]
...,
[ 0.9203397 -0.99590844 0.76853162 ..., -0.74290705 0.37568584
0.64072722]
[-0.12753558 0.52571583 1.03265858 ..., 0.59978199 -0.91293705
-0.02646019]
[ 0.19076447 -0.62968266 -1.97970271 ..., -1.48389161 0.68170643
1.46369624]]


When you initialize the Variable with another Variable, please use initialized_value.

W = tf.Variable(tf.truncated_normal([10, 10]))
U = tf.Variable(W.initialized_value())


It ensures U is initialized with the value of W.

## Session

Usually you have to specify the session to be used explicitly.

with tf.Session() as sess:
sess.run(a + b)


You can set default session by using InteractiveSession.

sess = tf.InteractiveSession()

a = tf.constant(1, name='a')
b = tf.constant(2, name='b')
c = a + b

print(c.eval()) # Use InteractiveSession
sess.close()


## Placeholder

Placeholder enables us to pass some values to graph at runtime.

a = tf.placeholder(tf.float32, shape=)
b = tf.constant([5,5,5], name='b', dtype=tf.float32)

c = a + b

with tf.Session() as sess:
sess.run(c, {a: [1,2,3]}) # [6,7,8]


If you specify None, placeholder can accept any shape in the field.

x = tf.placeholder(tf.float32, shape=[None, 3])
W = tf.Variable(tf.truncated_normal([3, 2]), name='W')
b = tf.Variable(tf.truncated_normal(), name='b')

y = tf.matmul(x, W) + b
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(y, {x: [[0.1, 0.2, 0.3], [0.3, 0.3, 0.3], [0.1, 0.1, 0.1]]})

#	array([[ 1.08204758,  0.64428711],
#	       [ 0.99868155,  0.86420214],
#	       [ 0.76671058,  0.50330997]], dtype=float32)


But placeholder is just a way to indicate the actual value must be provided. So you can provide any value at runtime.

a = tf.constant(1, name='a')
b = tf.constant(2, name='b')
c = a + b
with tf.Session() as sess:
sess.run(c, {a: 10}) # 12


## Reference

• http://web.stanford.edu/class/cs20si/lectures/slides_02.pdf