In one of my recent posts, I briefly talked about using Apache Pig, to index an Elasticsearch cluster. In this post, I do a walkthrough of the DevOps configurations and steps I took, along with the code that was required to get it work at the start (barring some issues that I’ll talk about in the next post).

Production Setup

The process starts...

At Pocketmath, we heavily utilize the EC2 container service (ECS) to host a significant portion of our applications. It provides us with an easily scalable, zero-downtime infrastructure. Recently, I upgraded the Elasticsearch to 2.3.5 for our clusters, so I thought it was a good idea just to jot down some of the things I had to do or was already...

From past experience, I found the maintenance and tuning of a Elastisearch cluster to be a little troublesome overtime. So it isn’t surprising to see hosted Elasticsearch services pop up one after another. Ok, to be fair, there are hosted services for nearly everything nowadays, from Kafka to Wordpress lol. There is really no shortage of them. Most...

Recently, I left a comment on one of my colleague’s PR and we had a discussion with him about the use of try vs the lonely operator &. and it led to a number of conclusions personally.

I used to use lots of .try. I’ve also come across codebases littered with it, be it in the presentation layer or in the models. From personal experience, I’ll say...

All of us do TDD or at least some form of automated testing, I hope! If you’re writing tests in Rails, you’re likely to be doing feature tests with Capybara as well.

Some of these slipped my mind while adding feature specs at work at pocketmath and I spent extra time that I shouldn’t have! So I hope this post can be a reminder to myself in future...

My previous deploy on the Rails stack was a little more involved so I chose to just deploy it in the conventional capistrano way after setting the server up.

But with the rise in popularity of Docker recently, I’ve been wanting to deploy something into production with Docker but never found the right app for it until this one.

It was a simple Golang...

Recently, I’ve been working on my squash club, UCSC’s new site. And of course, being slightly short of time, I kinda just fell back on Rails to quickly get something up for the club.

Before Heroku decided to put a 7 USD price on their free tier, it was an easy default for hosting any mini prototypes or projects. Ok I admit, I’ve historically used...

This is part 2 of the 3 part series of a quick intro of the realtime logging system in Wego. In Part 1 of this series, I talked about how we generated Go packages for Protobuf (protocol buffers) in Wego.

The packages allows us to process 2 different modes of communication protocol with just one set of schema defined in proto files:

• json package...

(This is part 1 of the 3 part series where I discuss how we handled one of our new API projects and the usage of Protobuf + Heka + Kafka + BigQuery to view realtime logs.)

Recently, I’ve had the chance to work with Golang in Wego to build our analytics API that will log our visits into flat files + upload them into Kafka for further processing,...

Suppose you have a JSON of this format:

{
  "results": 
    "collection": [
      {
        "name": { first_name: "Ho Man", last_name: "Boa"
        "phone": "123456"
      },
      {
        "name": { first_name: "John", last_name: "Lee"
        "phone": "2345"
      }
    ]
}

Similarities

There are essentially 2 methods to do this, using pre...