Some Expectations at Work

Since I'm about to lead a team, so thought it'd be helpful to have these things written down. They might change over time but hopefully not too much. I'd also want to be transparent and clear about what I'd expect from my team.…

Don't be afraid of the command line

We are so used to graphical user interfaces that we forget computers started off without it. As a software engineer, there are many reasons why using the command line can help with development.…

Learning Vim in 30 days

If you'd like to learn vim, I'll be sharing what I found helpful while picking it up. Vim seems ancient with all the fancy new IDEs like Visual Studio Code. However, I've found vim to greatly improve my productivity. Within a month, I started to use vim full time and…

Maintainable SQL in Data Warehousing

Since SQL isn't a programming language, and say you don't have the option of using a programming language to generate SQL, how do you go about making it maintainable? If I had the opportunity to build a new data warehouse from scratch, I'd probably tell myself these things to prevent…

Continuous Integration and Deployment for Data Pipelines

Although continuous integration and deployment (CI/CD) are not new concepts, new tools such as Docker, Kubernetes, and Jenkins have allowed CI/CD to be more easily implemented in recent years. However, there are interesting challenges when applying CI/CD to data engineering.…