CI/CD for databases: bad idea

Here a brief story of Continuous Integration / Continuous Delivery.

If you were developing a software between 1995 and 2007 there was no such thing as CI/CD. the software delivery was based on the CD that users (still called customers at that time) were buying on a shelve or could find inside a newspaper or any other non-digital, real World stuff.

Everything was going fine and it didn’t really matter if you were using the old version of a software, you didn’t really notice that. Companies were shipping their software abroad in CDs and life was going on “business as usual”.

2007

Then 2007 arrived and 2 major events took place.

I remember it vividly because I was living in London and both events made a big shift in my life and also in your:

— Facebook goes ballistic: I was living in Brixton Lane, London. My Lithuanian flatmate, Jay, told me there was a website called Facebook that allowed me to get in contact with all the friends I had through all my life. And that was true! As soon as I created my account I could become “friend” of people that I met many years ago in different parts of Europe. And I could message them. And see what they were doing. And it was free!

The Facebook page was very basic and clean: people loved it. It was so easy to understand that when the layout changed (now we call it UI or UX) every now and then people were creating groups called “Please-Facebook-Revert-To-The-Old-Layout” or whatever to ask to Facebook to change the layout.

— iPhone came out: The very much announced release of the iPhone took place and UK was the first country in Europe to see the first iPhone release. As far as I can remember there were no Apple Store so people could not “queue” as they did few years later but you could buy one from your ISP (Vodafone, O2, etc..).

In December 2007, it was a Friday night, around 9PM I was on a bus in Oxford Street. The bus was packed with young people craving for party. Suddenly an Asian girl receives a call and with nonchalance she takes off from her fur coat…. an iPhone 1… and she reply at the phone call as it was normal. As everything was fine… as if the World didn’t have changed. The bus was mute. There were people finger pointing the girl. We all saw it: the phone had no keyboard… and there was a big screen that was taking the size of the whole phone. It was like living in the future. It was like a scene from Total Recall (the old one, with Schwarzenegger) or Minority Report, something that belongs to science fiction movies… except it was the reality.

A monitor without cathode-ray tube… that is unbelievable

Automatic Updates

Personally I didn’t like Steve Jobs. Apart of being rude he was parking his car on the handicap slot and died trying to cure pancreatic cancer with boiled courgette.

That doesn’t make him a “smart dude” to my eyes but I have to admit that he changed the World as we know it.

The whole idea of Automatic Updates was already present in Windows XP but you still had to click and allow for the installation of new components. Plus, you don’t have to forget that with Windows XP you were the guinea pig. But with the iPhone it was different. If your app wasn’t good enough or didn’t had HD resolution images you were out of the “iTunes Club”.

As results the software presents on iTunes (now called Apps) were clean and well shaped.

Very soon Google created Android. Nokia died. Samsung and LG replaced the market gap… what a 2008.

The Apps on iTunes and Google Play were updating seamlessly, you didn’t really noticed that: CI/CD was born.

CI/CD everywhere?

I could not find an image but I remember that back in 2010 applications like Facebook, Google+ or whatever, they had this phrase on Google Play that said “The application is updated approximately every 10 days”. It translate into “you didn’t have to care any more about updates”.

…Thank you Lord!

It was a massive shift. But this was a mobile behaviour.

What happened in the next decade is that mobile application and mobile in general took so much attention that even desktop application started to behave the same way. That’s why your VS Code updates periodically… while the good old Visual Studio has version 2017, 2019, 2022.

And while you can update your VS Code seamlessly you are scared to update your Visual Studio. Before clicking on the .exe file you ask yourself:

— What libraries are not going to be compatible?

— Is .NET going to be compatible?

— How many minutes/hours will the update take?

— What about Visual Studio C++ Build tools? That always need to be installed with the right version of Visual Studio and I have to manually add the PATH in the Environment Variables

— ADO.NET… oh ADO.NET… that would never be compatible…

— etc…

That’s why many times the Visual Studio upgrade is manged by your Sysadmin. Otherwise people brake their own machine.

So why all of a sudden we all want to have CI/CD in place to the back-end and to the database?

FOMO

Because you are experiencing Fear Of Missing Out.

While I understand the need of a continuous release for mobile application and front-end, things get a bit nasty when you deal with back-end and simply crazy when we talk about databases in the back end.

Yes, because thanks to the social networks we now experience FOMO and we have exported the idea of CI/CD, which was basically a need for mobile, first to the desktop World and then to the software. Like everything is in need of CI/CD all the time.

But mobile application they rely on SQLite. The mobile application is just the “face” of the software. Is not the back-end. Do you want to know what’s going on behind the mobile app? This:

The database is not the the core of the application itself and the app is mainly querying a remote database somewhere: this is called the back-end and this has to change as less as possible.

…or change it every couple of weeks if you want but keep in mind that you will do that at cost of time, money and resources.

You look to the right and to the left and you see that every software company around you is doing CI/CD… therefore you experience FOMO, you feel insecure because you are not Agile, and you run buying Liquibase licenses like they were antibiotics for your ill son.

But the reality is that people are just lying.

Very few companies are setting up CI/CD on their back-end databases. And even less are successful doing it. You don’t believe me? Listen to Jesus Christ at minute 16:06 and you will see that he could “ count CI/CD pipeline for databases on 3 fingers of his hands…”:

And I’m not surprised. He also add that in the only successful CI/CD pipeline there were 2–3 people only dedicate to the CI/CD pipeline setup.

Do you see how expensive is going to be a CI/CD for databases?

Conclusion:

If you are experiencing FOMO because you don’t have a CI/CD for databases in your company that is mainly caused by two factors:

— Your Product Owner or Leader is lacking in technical knowledge and he/she doesn’t have the courage to say out loud: this is BS and it’s a terrible idea!

— You don’t have a DBA and developers are treating the database as it was code.

You are wasting money on Liquibase license because Continuous Integration “is a development practice that requires developers to integrate code into a shared repository several times a day.” And I bet you don’t:

— Commit changes as frequently as “sever times a day”

— Merge feature branches to master as frequently as “sever times a day”

So no, you are not doing Continuous Integration.

CI is for OOP languages like Java, Ruby, Scala, Python, Go, etc… , not for structuring database architecture!

And you are not doing Continuous Deployment neither because you are not releasing a new version of your back-end database weekly and not even monthly. Maybe every quarter of the year.

…. but I use it for deployment of 10, 100, 1000 databases on Test, Dev, etc…

You can just instruct people to download a vanilla version of your latest database and use that.

They can find it on an internal FTP and they can deploy it with all sort of script and job without need of setting up a CI/CD pipeline.

So why softwares like Liquibase exists?

For the same reason as scarfs for new-born exists: because there are people willing to sell and people willing to buy. That doesn’t mean that is a good idea.

Every years there are new-borns strangled with scarfs but shops keeps selling them even if doctors discourage the use of scarfs for new-borns.

Maybe there might be 4 or 5 families in Antarctica that really need a scarf for their baby but that is a very peculiar and unique situation.

No, you probably don’t live in Antartica.

No, you probably don’t need a CI/CD pipeline for your database architecture.