A Tale of a Suicidal Container

Wed, 21 Feb 2024 12:25:03 +0530

One fine day, I sat down to optimize the size of a Docker image. Like many times before, I opted for distroless images as my base, a choice I had made countless times before without a hitch.

Distroless images, for the uninitiated, are peak minimalism, containing only the essential libraries and binaries required to run the application. Not only do they trim the fat off the image size, but they also mitigate the risk of CVEs lurking within.

Go faster with Go: Golang for ML Serving

Mon, 20 Jun 2022 21:36:00 +0530

So the ask is to do 3 Million Predictions per second with as little resources as possible. Thankfully its one of the simpler model of Recommendation systems, Multi Armed Bandit(MAB). Multi Armed bandit usually involves sampling from distribution like Beta Distribution. That’s where the most time is spent. If we can concurrently do as many sampling as we can, we’ll use the resources well. Maximizing Resource utilization is the key to reducing overall resources needed for the model.

golang on AI Logs

A Tale of a Suicidal Container

Go faster with Go: Golang for ML Serving