The First Cry of Atom Today is the first day of the rest of my life.

Mandelbrot in Tensorflow

I found a interesting example in Tensorflow tutorials. Tensorflow is famous as a deep learning framework. We can write various kind of deep learning algorithms in Tensorflow efficiently. But it is also sufficiently general machine learning and numerical computation library. We can write any kind of computation like K-means. Tensorflow community... Read more

VC dimension and predictive loss

How can we estimate the loss of a machine learning model based on its complexity? We know that complex model can explain various kinds of data set space but it can overfit to the sample space. We want to optimize the loss in whole data space instead of sample space. It means that we need to know the balance between model complexity and the poss... Read more

DB Migration with Flyway

Now database migration should be managed under version control system. Every database schema can be changed through time. If we don’t track this, it must be difficult to rollback or test. We often use ActiveRecord as migration script because it is database agnostic and easy to write. But Ruby script is not the best choice when we are developing... Read more

Simple side effect by forEach in Java 8

As you know, Java 8 provides lambda expression and stream API. forEach is one of the them. List<String> a = ImmutableList.of("a", "b", "c"); a.forEach(s -> { System.out.println(s); }); Having side-effect in lambda expression is not recommended because it can be a hinder against thread-safe. For example we sometimes want to add elem... Read more

Backpropagation Through Time

Neural Networkにおいて最適化の中心を担うのはBackpropagation(誤差逆伝播法)ですが、Recurrent Neural Networkではどのように計算されるのかよくわからなかったので、まとめてみました。 モデル 最もシンプルなRNNを考えます。入力はx(0), x(1),...,x(t)の系列でそれぞれが多次元ベクトルです。 このときこのRNNの出力は下記のように記述されます。とはそれぞれsigmoid関数などの非線形な活性化関数とします。現在の隠れ層の入力は現在の入力と一つ前の時刻の隠れ層の値にそれぞれ重みをかけ、バイアスを加えたものになります。これに活性化関数を通したものが隠れ層の出力になります。 \begin{equation} \bold... Read more