Tag Cloud
Concourse CI/CD devops prometheus kubernetes monitoring modbus kepware c-programming IoT golang telegram bot python cli urwid elasticsearch aws ecs apache spark scala AWS EMR hadoop webhooks ssl nginx digital-ocean emr apache pig datapipeline found.io elastic-cloud rails try capybara docker capistrano heka bigquery kafka protobuf vim iterm javascript emberjs git scripting dnsmasq bem frontend meteorjs meteorite heroku

Page 1 of 4
Older Posts

Concourse CI - Lessons
Written by Ho Man Nov 28, 2020 12 minutes read

I run a mid-sized Concourse CI cluster for Tulip, that runs ~3000 fairly resource-intensive builds weekly. I’ve encountered a fair share of stability issues with it, some from lack of experience, some from real issues, but overall, my experience with it has been fairly positive. I can’t speak about Github Actions, or TravisCI and CircleCI but my...

Read more →

Monitoring Stack in Kubernetes, with Prometheus
Written by Ho Man Aug 01, 2019 17 minutes read

For the past year or so, I’ve been working with DevOps in Tulip. It’s a fairly big change in direction but quite frankly, it’s been a refreshing experience!

One of the first projects was to build a monitoring system for a number of different components in our kubernetes cluster: various microservices, main monolith application, our ingress controller...

Read more →

Programming with the Modbus RTU & TCP/IP Protocol
Written by Ho Man Oct 30, 2017 15 minutes read

Today’s post probably has a very different audience- modbus protocol; it’s nowhere near the web projects that I’ve been doing so far but definitely something I’m super interested in. This project mostly works with the modbus protocol, which is an open, communication protocol used for transmitting information over serial lines between hardware devices...

Read more →

Golang Telegram Bot - Migrations, Cronjobs & Refactors
Written by Ho Man Aug 20, 2017 7 minutes read

This post is kind of like a continuation from the previous posts of my Golang Telegram Bot, so if you haven’t seen that yet, it’s probably better to start with those first: part 1 and part 2. I basically wanted my telegram bot to be able to remember dated / timed reminders and send messages to notify me when that time comes (like a calendar). Furthermore...

Read more →

Building a Python CLI Stock Ticker with Urwid
Written by Ho Man Jun 28, 2017 11 minutes read

A bit of context - I do some investing in equities on the side and I’ve always wanted to build a simple stock ticker in the form of a CLI app that runs in my terminal setup. There were a few out there but none that would show just the information I needed, in a minimalistic fashion. And I thought it would be a fun project for me since I don’t have...

Read more →

Upgrading to ElasticSearch 5.2.2 on Amazon ECS
Written by Ho Man Apr 03, 2017 5 minutes read

In one of my previous post, I talked about how I set up Elasticsearch 2.3.5 on ECS. I got a comment in that post that prompted me to update the setup for Elasticsearch 5. It’s been awhile, but better late than never right? So I gave it a go! In this post I’ll like to share what I found in the process.

There were a couple of other configuration changes...

Read more →

Tuning My Apache Spark Data Processing Cluster on Amazon EMR
Written by Ho Man Mar 10, 2017 14 minutes read

Lately, I had the chance to work on some data integration at Pocketmath where I wrote a bunch of Spark scripts in Scala to run some transformations on a data set of about 250GB that will run on a monthly basis. In this post, I talk about some of the problems I encountered, and some considerations while setting up the cluster and also how I improved...

Read more →

How I Deployed My Golang Telegram Bot
Written by Ho Man Jan 21, 2017 13 minutes read

Continuing where I left off in the first part of my Golang Telegram Bot, in this post I go through all the steps I took to get to a one command deployment of my Telegram bot into a Digital Ocean Ubuntu 16.04 instance. A number of components were involved: dockerizing the app, setting up a self-signed SSL cert, get the Nginx to work as a reverse proxy...

Read more →

How I Built a Simple Telegram Bot in Go
Written by Ho Man Dec 25, 2016 20 minutes read

This is the first part of the Golang Telegram Bot series. In this series, I’ll show you, with code samples, how I built a Golang Telegram Bot for my own use. It would listen in and respond in real-time to certain text cues. Finally I’ll also show you how to get a self-signed SSL cert working with Nginx and deploying the application in a Docker...

Read more →

Preventing Duplicates: Amazon EMR Pig to Elasticsearch
Written by Ho Man Dec 24, 2016 7 minutes read

In the previous post, I went through some steps I took to set up the Amazon EMR Hadoop cluster to run Apache Pig scripts for indexing data to Elasticsearch. In today’s series, I walk through some of the problems I encountered when I set the system up and some of the configuration tweaks to both Elasticsearch and the EMR cluster that I feel you should...

Read more →