The First Cry of Atom Today is the first day of the rest of my life.

Finagle future and ...

Twitter seems to have moved their infrastructure to NodeJS from Scala. Today we moved all of Twitter’s mobile web traffic (that’s like, a lot) to our new web stack – Node.js, Express, React PWA. I’m wondering they will keep supporting finagle in future… Read more

gimg - Terminal Image Viewer

At the beginning of this year, I decided to learn some new programming languages. For now, these are Go and Haskell. Go is obtaining popularity among system programming and development tools like Docker. Haskell seems to be good for learning another programming paradigm, pure functional programming. Anyway the best way to learn a new programmin... Read more

bzip2 and MAPREDUCE-13270

少し前だけれど、HADOOP-13270について書こうと思う。 Hadoop MapReduceフレームワークはInputFormatというインターフェースを通じて様々なフォーマットのファイルを読むことができる。単純なテキストファイルを一行ずつ読んだり、Avroなどのシリアライゼーションフォーマットやgzipなどの圧縮フォーマットもサポートしている。 InputFormatはgetSplitsというメソッドで一つのファイルを複数のSplitという単位に分割することが要求される。 InputSplit[] getSplits(JobConf job, int numSplits) throws IOException; 1つのMapperは1つのSplitを読んで処理を行う... Read more

Keep blogging is Fun

This blog was kept for 4 years. Of course there are some times when I stop writing. But anyway updating blog posts itself are kept for 4 years. I’m now considering keep blogging is difficult and tough work. But I found an practical article for keep blogging. This post describes the motivation and mentality for keep blogging. Want to blog? Read ... Read more

Presto Driver,Split and Pipeline

Collecting workload metrics of distributed system is important task to improve performance and make it scalable. Presto is not exception. Presto is fast distributed SQL engine mainly developed by Facebook. Recently AWS start using Presto as backend of Amazon Athena. We are using Presto in daily analysis too. Just same as other distributed system... Read more