Today marked a significant milestone in my cloud journey. I moved from manually building Docker images on my laptop to having a fully automated CI/CD pipeline that builds, tests, and deploys container images to AWS. This is the kind of automation that separates hobby projects from production-grade systems.
What I Needed to Learn
Before today, I understood containers conceptually and could build Docker images locally. But I had never set up a real deployment pipeline. I needed to learn how GitHub Actions works, how to securely store and use AWS credentials, how to configure Amazon ECR, and how to implement security controls that prevent bad code from reaching production.
The authentication piece seemed particularly daunting. GitHub Actions needed to authenticate with my AWS account, access my private container registry, and push images. All of this had to happen securely without exposing credentials. I also needed to understand the difference between building images for testing versus building them for deployment.
Setting Up GitHub Actions
I created my first GitHub Actions workflow file in the .github/workflows directory of my repository. The workflow is defined in YAML and describes exactly what should happen when code is pushed. It checks out the latest code, authenticates with AWS, logs into ECR, builds the Docker image, and pushes it to the registry.
The workflow uses actions from the GitHub marketplace, which are pre-built components that handle complex tasks. The aws-actions/configure-aws-credentials action manages AWS authentication, and aws-actions/amazon-ecr-login handles the ECR-specific login process. I didn't have to figure out the low-level AWS API calls myself.
What impressed me most was seeing it work for the first time. I pushed code to GitHub, switched to the Actions tab, and watched the workflow execute in real time. Each step showed green checkmarks as it progressed. Within two minutes, my new Docker image appeared in ECR, built entirely from code without any manual intervention.
Implementing Security Controls
With automation comes responsibility. At an enterprise level, you cannot let any code push and deploy directly to production. They need security and for that they need controls.
The GitHub Actions workflow can distinguish between pull requests and main branch commits. On PRs, it can build the Docker image to verify everything works while not pushing to ECR. I practiced this and it gave me fast feedback about whether my changes are valid. Only when code merges to main does the workflow push to ECR, ensuring my container registry only contains images from approved code.
I stored AWS access keys as GitHub secrets, which are encrypted and never exposed in logs. The IAM user I created for GitHub Actions has limited permissions, following the principle of least privilege. If those credentials were somehow compromised, they can only push to ECR and nothing else.
Configuring Amazon ECR
Setting up ECR required more than just creating a repository. I had to create an IAM user specifically for automation, generate access keys, authenticate with ECR, and configure everything to work together.
ECR uses a two-step authentication process. First, you authenticate with AWS using IAM credentials. Then you use that authenticated session to get a temporary Docker login token for ECR. This token expires after 12 hours, so the workflow has to generate a fresh one on every run.
I configured the same credentials locally using the AWS CLI so I could test the process manually before automating it. Understanding the authentication flow can make debugging much easier when things don't work the first time.
The repository itself is straightforward. I gave it a name, chose private visibility, and left most settings at their defaults. The interesting part was seeing how the workflow tags images. I experimented with different types of tagging: one labeled "latest" for easy reference and another with a focus on precise version tracking using Git commit SHA. With my app in such a premature state I can probably just focus on using a simple one like latest, but it's great to know that there are multiple tagging capabilities.
Adding Automated Testing
One of the key benefits of CI/CD is catching problems early. I configured the workflow to run on every pull request, building the Docker image to verify it works. This seems simple but catches many issues automatically.
If I add a Python package to my code without updating requirements.txt, the Docker build fails immediately. If there's a syntax error in the Dockerfile, I find out right in the PR before merging. If I accidentally break an import, the build catches it. All of this happens automatically within minutes of pushing code.
The workflow provides immediate feedback directly in the GitHub interface. Green checkmarks mean the code is ready. Red indicators mean something needs fixing. I can see the full logs of what went wrong without having to reproduce the issue locally.
This is the continuous integration part of CI/CD. Every integration of new code into the main branch is automatically verified. As my project grows, I can add more sophisticated tests like unit tests, integration tests, and security scans, all running automatically.
How My AWS Certification Helped
Having the AWS Solutions Architect Associate certification made this entire process significantly smoother. I already understood what ECR and IAM were and how they fit into the AWS ecosystem. The certification covered IAM best practices extensively, which made it obvious why I needed a dedicated user for automation rather than using root credentials.
I understood the security model behind AWS credentials, why access keys should be rotated, and how to scope IAM policies to minimum necessary permissions. This theoretical knowledge translated directly into practical decisions about how to set up my pipeline securely.
The certification also taught me about AWS service limits, regions, and cost optimization. I knew to check which region offered the best pricing, how ECR storage costs work, and when to clean up old images to save money. These might seem like small details, but they add up in real projects.
What I Accomplished
I now have a complete CI/CD pipeline from code to container registry. When I push code to GitHub, it automatically builds a Docker image and pushes it to ECR within minutes. The process is secured with proper IAM policies and GitHub secrets. Branch protection ensures only tested code reaches production.
My current costs are minimal. ECR charges $0.10 per GB per month for storage, and my Docker image is under 200MB. I'm paying pennies. GitHub Actions is free for public repositories and includes generous free minutes for private repositories.
Moving Toward My End Goal
This pipeline is a critical piece of the larger puzzle. My end goal is deploying AI workloads to AWS, which requires reliable container deployment. I now have half of that equation solved: reliable container building and storage.
The next phase is using these containers. I need to deploy them to ECS using Terraform, which means learning infrastructure as code. Once I can define my infrastructure in code and deploy containers automatically, I'll have a complete system that takes me from a code commit to running containers in the cloud.
Every step builds on the previous one. The momentum is building, and I can see the path forward clearly. Tomorrow I'll tackle infrastructure as code with Terraform, defining the AWS resources needed to actually run these containers. The pieces are coming together and I'm genuinely excited about what I'm building.
No comments:
Post a Comment