Welcome to random devlog that no one read by Meorung, this time talking about the why and how of setting up CI/CD for our projects, with no experience beforehand!
Why do I need CI/CD
Cause I am lazy
From the beginning, we (mostly me) have decided to set up and managed our own server, in this case Lightsail from AWS. This gives us the freedom of doing whatever the heck we want on the server but also makes it a bit more complicated to deploy anything since there’s no simple one-click option to deploy an app or a website. This is not that big of a problem for us at the beginning since we are starting out with a simple static HTML website and all we need to do is to copy those static files to the server. However, these “simple” things present us with a problem.
Not everyone knows what a server is, how a website works
I am working in a team of 3 students: me, a Senior, and a Junior, and unfortunately, I am the only one on the team who has any experience with web development. This makes a seemingly “simple” task of copying a file to the server difficult! Why you may ask? Here’s the list of questions:
- What is a server? (is it scary?)
- How do you connect to it?
- Why is there no user interface?
- Where do I put my file?
- …
Here’s an example of how thing can go wrong:
One of my teammates, not knowing about the power of index file (cause he/she only double click on a .html file instead of using a server) think that you can only link directly to a file, and therefore change my login index.html file to a more meaning full meaningfullName.html render haft of the website useless cause you can’t log in anymore!
That’s why instead of forcing everyone to learn using Linux or about using a server, I can auto-deploy whatever they push to Git, and telling them that the magic of “when you put a file in this folder, it will appear on the website” (of course I tried to explain!)
I am messy, and there’s no guidelines
Uploading a file to the server meaning that I can do whatever I want, and if there are no rules, it will soon become a mess!
Imagine making a new index.html page, what do you do with the old index.html page, do you delete them? What if your new file is not working and you need the old file back? How about moving them to a different folder? What folder will you put it at, If there’s already an index.html there, then what is the name of your “new-old” file? How long will you keep it?
In the past, I have manually upload static HTML pages to my personal website (hphucs.me), and I had faced all these problems. having to deal with oldIndex.html, oldIndex1.html was not fun, especially when your new website crash and you want to go back to your “old” index file.
I want to learn new things
…that’s it, I want to learn new things.
Process of deploying AWS CodePipline
At this point, I had no experience with CI/CD and I have limited experience with AWS, so combining these 2 together, I had no idea what was happening
Starting
Having no idea what to do, I did the only thing that anyone can do: Google search. And fortunately, there’s one good article made by Amazon about connecting AWS Codedeploy with Lightsail, which is the exact thing that I’m wanting to do.
While this article is perfect for what I need and did help me setting the very initial Codedeploy on our Lightsail server, one thing that I find it lacking is the explanation of why a certain step is needed or certain information is used. I find myself frequently coming back to this blog post because I don’t fully understand the process, I have to make a lot of guests about what I need to do once I start customizing the process to feed our website.
Real problem #? – Gitlab
After successfully deploying my Django API on my personal Github using the previous blog post from Amazon, I run into a first problem: our main repository is on a self-host GitLab instance, and there’s no easy out of the box way to connect to a GitLab instance. Combining this with my little understanding of AWS Codepipeline, it took me a decent amount of time to make it work
Fortunately, there is plenty of resource talk about this online, and with some experimenting I had a working system: When a push meets a certain condition on our repository, a GitLab action will run, pushing our code to an S3 bucket on AWS. This bucket change will then trigger a pipeline, deploying our code to the main server at /var/www/html/. This is nowhere near a good solution, and I can definitely skip the AWS step altogether and deploy directly to our server from GitLab. But in the name of consistency and “convention”, I decided to keep the aws part.
This turned out to be a good option as later on, I have a talk with my supervisor (who is also my professor), and we ended up moving our repository to Github for easier integration, and the only thing that I need to update for my pipeline is the codesource!
Another problem – Pricing
Making a website for a non-profit organization means that we need to be considerate when it comes to resources and money. Non-profit organizations have limited money to spend and therefore before spinning up any service, i usually check their prices to see if it is reasonable to use it.
Codedeploy pricing is stated as “free” if we use it with EC2 instance aka with any AWS VPS. However, Lightsail for some reason, even a part of AWS, does not count as an AWS EC2 instance, and because of this, we have to pay for every deployment that we run. This ended up making me having to do further configurations to make sure that no unnecessary deployment was run and wasting our budget!
Conclude
This is obviously not a full list of what I did or all the problems that I ran into. Regardless of all the trouble that I have to overcome, setting these up was a fun experience!
Rating: 8/10 may want to do it again.