Photo by Daniel Cheung on Unsplash

In this demo, we show how to migrate a sample MySQL database to PostgreSQL in k8s using a pgloader job. The overall processes are:

  1. ‘helm install’ a MySQL db and a PostgreSQL db in a k8s cluster (GKE).
  2. Load a simple employee data set to MySQL db.
  3. Create a pgloader job reading MySQL employees db and moving the data to postgres db employees schema.

First, we use helm install MySQL and Postgresql from bitnami. The simple point we want to demo is running pgloader in K8S job, so we won’t discuss different helm charts here. Then we load a sample…

Photo by Alan Hardman on Unsplash

After the Habor/docker troubleshooting, I finally figured out how to make a python web app push to my k8s ingress.

The first part, I created a hello world flask app, built a docker image, tagged the image, then pushed the image to harbor registry.

Once the image is in harbor, I created a deployment and a service to default namespace.

Since this deployment will be my web layer, I want to expose it through ingress with TLS. In my home lab, I setup a DNS name ‘ingress.home.lab’ with IP ‘’. I used cert-manager to issue a server certificate…

Photo by HONG LIN on Unsplash

Recently, I switch the container registry from docker hub to harbor and encountered “x509: certificate signed by unknown issuer error” using Docker Desktop and Harbor private registry. This article is about how I resolved this issue in my Docker desktop on Mac and my home lab k8s containerd.

In my home lab, I deployed a harbor registry to replace the default one (docker hub) and exposed it as a load balancer service. This way I can build docker image on my mac, push the image to harbor, then deploy the app to my k8s environment. …

Photo by Atish Sewmangel on Unsplash

My Terraform/Ansible script doesn’t work anymore after I turned on GCP OS Login. I didn’t know what OS Login means and just turned it on. Then I spent a couple of hours figuring out if it is caused by my custom image (OEL7). It turns out it is not. OS Login is a better authentication (oauth2) for Enterprise customers. In short, GCP OS Login lets you use your own desktop ssh key to log in to all GCE instances you are allowed to access (limited by service account). It is pretty straightforward if you use “gcloud compute ssh” like this.

Photo by James Pond on Unsplash

In an “enterprise” context, it is common to block users from pulling images from public container registries. Harbor is a private registry for k8s providing many security features, such as content signing and vulnerability scanning. For more info on why harbor in your k8s env, visit the harbor website. In this article, I describe how I set up the Harbor and run a vulnerability scan on an example image and, of course, troubleshooting.

  • Installing cert-manager with ca cluster-issuer
  • Installing Harbor with cert-manager (auto issuing TLS certificates) and load balancer service
  • Configuring Harbor
  • Pulling and pushing images to harbor registry
  • Scanning…

Photo by Daniel Cheung on Unsplash

Long story short, I left my 8-year job and moved to a startup company. As such, I lost my company-sponsored GCP account as my lab. So I picked up my home lab equipment and made my first baremetal K8S cluster at home. This is what I have learned so far.

Before the home lab project, I used git, Github, ansible, terraform, visual code, and kubeadm to quickly bring up a cluster and automate an environment to experiment with microservice. The learning path is bumpy, but I think I picked the right tools to make my learning less frustrating. I want…

Photo by Will Porada on Unsplash

This article is the last part of pg_hba.conf explained. Note that pg_hba.conf is only for authentication. Most auth-methods make sure the client and the postmaster’s data exchange in this period secured, for example, ldap with tls, krb, pam_sss, scram-sha-256. In other words, the password is secure in transition.
What about the data in transit encryption? Can someone turn on the network sniffer and get all the query resultsets (network packets) I sent to the postmaster? Yes, it is possible. That the topic I want to explore; TLS/SSL. Let’s turn on TLS on pg-master. First, you need a server certificate from…

Photo by Daniel Cheung on Unsplash

In this part, I explain the pam authentication in pg_hba.conf. PAM stands for “pluggable authentication modules.” PAM supports four types of services, auth, account, password, and session, but Postgresql pam only supports two services; auth and account. In the last part, we installed ipa-client on pg-master. ipa-client should setup sssd/krb/ldap/pki on pg-master already. After installing PostgreSQL, you should have a default pam configuration in /etc/pam.d/postgresql.

[root@master1 pam.d]# cat postgresql
auth include password-auth
account include password-auth

Using pam in PostgreSQL is as easy as making pg_hba.conf like the following and reloading the configuration.

host all  all  pam pamservice=postgresql


Photo by Alphacolor on Unsplash

In part1, we understand the basic rules of pg_hba.conf. Let’s review the entry I put in the pg_hba.conf in part1. It was:

host      all     all      scram-sha-256

Translation: All clients (users) connecting from subnet try to access ALL databases will use scram-sha-256 password. From a dba perspective, this entry is still too open. I would suggest narrow it down to something like the following:

host  dvdrental remote_user1    scram-sha-256

As you can see, the CIDR is narrow to one single database (dvdrental) from single IP. This is more rigid but well…you start to think: Oh dear, maintaning this pg_hba.conf…

Photo by Daniel Cheung on Unsplash

This article attempts to demystify how to configure pg_hba.conf and integrate “enterprise systems” for different use cases.

The GCE env I demo contains three VMS, ipa-server, pg-master, and pg-client (you can git clone and deploy the same environment from my GitHub repo FreeIPA is like “MS Active Directory.” FreeIPA integrates a Directory Server( 389), MIT Kerberos, NTP, DNS, and DogTag (PKI).

In Postgresql, hba stands for “host-based authentication.” pg_hba.conf contains a set of rules. The first field is the connection type. In the beginning, you need to know two basic types, local and host. “local” means local domain socket…

Yuwei Sung

A data nerd started from data center field engineer to cloud database reliability engineer.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store