The First Cry of Atom Today is the first day of the rest of my life.

Assemble and creating table in Hive UDF

histogram_numeric is a UDAF which should calculate the distribution of given records. But at the same time it should generate a table that represents one category by one row. In this point we can regard this type of UDF is a combination of UDAF and UDTF. For example the output of histogram_numeric looks like hive> SELECT explode(histogram_nu... Read more

Digdag syntax highlighter in Atom

Digdag was released from Treasure Data. This is a highly scalable distributed workflow engine. It was developed for both analyst and engineers in order to make their daily batch and adhoc jobs more easy. The important part I want to say here is we can define workflow in one file called *.dig. So you can put the file under version control system ... Read more

Professional Hadoop

Wiley gave me another chance to write a book about Big Data technology. This was offered at the almost same time when I started to write Spark book that is introduced previously because they expected me to write something about Hadoop too. So this is a book about Hadoop especially for deep dive into core Hadoop technology and recent version whic... Read more

Conditional mysqldump

mysqldump is a useful tool for migration of database. This tool enables us to move a data into another database through human readable format, SQL. But as default mysqldump dumps all records in all tables in database. I wanted to use mysqldump as a tool to move only specified record to another table. We can assume a situation when a data in prod... Read more

Update the design to use indigo theme by Kopplin

I searched several themes which available for Jekyll from yesterday. Indigo is the best and minimal one which fit my favor. If you want to try this design, please follow the instruction written in README. indigo README It was very easy. Read more