histogram_numeric is a UDAF which should calculate the distribution of given records. But at the same time it should generate a table that represents one category by one row. In this point we can regard this type of UDF is a combination of UDAF and UDTF. For example the output of histogram_numeric looks like hive> SELECT explode(histogram_nu... Read more 24 Jul 2016 - less than 1 minute read
Digdag was released from Treasure Data. This is a highly scalable distributed workflow engine. It was developed for both analyst and engineers in order to make their daily batch and adhoc jobs more easy. The important part I want to say here is we can define workflow in one file called *.dig. So you can put the file under version control system ... Read more 18 Jun 2016 - 1 minute read
Wiley gave me another chance to write a book about Big Data technology. This was offered at the almost same time when I started to write Spark book that is introduced previously because they expected me to write something about Hadoop too. So this is a book about Hadoop especially for deep dive into core Hadoop technology and recent version whic... Read more 09 Jun 2016 - 3 minute read
mysqldump is a useful tool for migration of database. This tool enables us to move a data into another database through human readable format, SQL. But as default mysqldump dumps all records in all tables in database. I wanted to use mysqldump as a tool to move only specified record to another table. We can assume a situation when a data in prod... Read more 07 Jun 2016 - 2 minute read
I searched several themes which available for Jekyll from yesterday. Indigo is the best and minimal one which fit my favor. If you want to try this design, please follow the instruction written in README. indigo README It was very easy. Read more 05 Jun 2016 - less than 1 minute read