Streamlining GitOps with Atlantis and Terraform

11 min readJul 13, 2023

Introduction

GitOps has emerged as a powerful approach to manage infrastructure and application deployments using version control systems like Git. By leveraging GitOps, organizations can ensure infrastructure-as-code (IaC) changes are tracked, auditable, and easily reproducible. Terraform, a widely adopted IaC tool, can be seamlessly integrated with GitOps principles using Atlantis — a self-hosted, Git-based workflow automation tool. In this blog, we will explore the concept of GitOps, discuss the benefits it offers, and delve into how Atlantis can be utilized to enable GitOps philosophy with Terraform.

Understanding GitOps

What is GitOps?

GitOps is a software development and operations approach that extends the principles of version control systems, particularly Git, to infrastructure provisioning and application deployment processes. It emphasizes using Git as the single source of truth for both code and infrastructure configurations. With GitOps, organizations can maintain a declarative specification of their infrastructure and applications in a Git repository, enabling a centralized and auditable system for managing and tracking changes.

Core Principles of GitOps

Declarative Infrastructure as Code (IaC): GitOps promotes the use of infrastructure provisioning tools like Terraform or Kubernetes manifests to define infrastructure configurations in a declarative manner. These configurations are stored as code in Git repositories, making them version-controlled and easily auditable.
Git as the Single Source of Truth: In GitOps, the Git repository becomes the authoritative source for all infrastructure and application configurations. Any changes to the infrastructure are made through Git commits and pull requests, ensuring that the Git repository always reflects the desired state of the infrastructure.
Continuous Delivery and Automation: GitOps emphasizes automating the deployment process using continuous delivery practices. Whenever changes are pushed to the Git repository, a GitOps tool, automatically detects the updates and applies them to the target environment, ensuring consistency and repeatability.
Pull-Based Synchronization: GitOps follows a pull-based synchronization model, where the target environment continuously pulls and applies changes from the Git repository. This approach ensures that the infrastructure and applications are always in sync with the desired state defined in the Git repository.

Benefits of GitOps

Auditable and Version-Controlled Infrastructure: By storing infrastructure configurations as code in Git, organizations gain a complete audit trail of all changes made to the infrastructure. It becomes easier to track who made the changes, when they were made, and why. Additionally, the ability to revert changes to previous versions provides a safety net for rolling back in case of issues.
Consistency and Reproducibility: GitOps enables infrastructure and application deployments to be easily replicated across environments. With Git as the single source of truth, teams can ensure consistent configurations across different stages, such as development, testing, and production. This reproducibility reduces the risk of deployment errors caused by manual configuration drift.
Collaboration and Visibility: GitOps fosters collaboration among developers, operations teams, and other stakeholders. By leveraging pull requests and code reviews, teams can discuss and validate infrastructure changes before they are applied. This promotes better communication, knowledge sharing, and alignment across teams.
Continuous Compliance and Security: GitOps allows organizations to enforce compliance and security measures by applying them as code. Infrastructure configurations can be reviewed for security best practices, and any non-compliant changes can be caught early in the development process. This helps organizations adhere to regulatory requirements and maintain a secure infrastructure.
Scalability and Efficiency: GitOps automates the deployment process, reducing the manual effort required for infrastructure provisioning and updates. As a result, organizations can scale their infrastructure more efficiently and respond to changes and updates faster, ultimately improving the overall efficiency of the development and operations workflows.

Integrating GitOps with Terraform

Why GitOps for Terraform?

GitOps is a natural fit for Terraform, as it allows organizations to manage their infrastructure as code (IaC) in a version-controlled and auditable manner. By adopting GitOps principles with Terraform, organizations can reap several benefits:

Infrastructure as Code (IaC) in Git
Declarative and Auditable Infrastructure Changes
Collaboration and Review Processes
Automated and Consistent Deployments

Let’s try to understand the difference between the traditional approach and the gitops approach with a diagram:

Traditional Approach

GitOps Approach

Enabling GitOps using Atlantis

Atlantis is a self-hosted, Git-based workflow automation tool that integrates seamlessly with Terraform to enable GitOps practices. Here’s how Atlantis facilitates GitOps with Terraform:

Infrastructure Repository Configuration: Atlantis is configured to monitor specific Git repositories containing Terraform code. It connects to the Git repository and listens for events such as pull requests or pushes.
Pull Request-Based Workflow: Developers initiate infrastructure changes by creating pull requests with their Terraform code. Atlantis automatically detects these pull requests and performs a series of actions based on predefined configuration, such as running Terraform plan to preview the changes.
Code Review and Collaboration: Atlantis integrates with code review tools like GitHub or Bitbucket, allowing team members to review and provide feedback on the Terraform code changes. Discussions and approvals can take place within the pull request, ensuring that changes are thoroughly reviewed before being applied.
Automated Validation and Deployment: Once the code changes are approved, Atlantis executes Terraform apply to deploy the infrastructure changes. Atlantis can be configured to enforce additional validation steps, such as running tests, security scans, or policy checks, before allowing the deployment.
Notifications and Reporting: Atlantis provides notifications and feedback on the status of infrastructure changes, keeping stakeholders informed about the progress of deployments. It can send notifications to team members, Slack channels, or other communication platforms, ensuring visibility and transparency throughout the process.

By leveraging Atlantis in combination with Terraform, organizations can establish a robust GitOps workflow for their infrastructure provisioning. This integration streamlines collaboration, improves code quality through code reviews, and automates the deployment process, leading to more reliable and scalable infrastructure deployments.

In conclusion, integrating GitOps with Terraform using tools like Atlantis empowers organizations to manage infrastructure as code efficiently. By adopting a GitOps approach, organizations can leverage the benefits of version control, collaboration, and automation, resulting in more auditable, consistent, and scalable infrastructure deployments.

Setting up Atlantis

Preparing the Infrastructure

Before setting up Atlantis, you need to ensure that the necessary infrastructure is in place. Here are the key steps:

Provision a Server: Choose a Linux server or virtual machine where you will host Atlantis. Ensure that the server meets the system requirements and has sufficient resources for handling your Terraform deployments.

Install Dependencies: Install the required dependencies, such as Git, and Terraform, on the server. These dependencies are essential for Atlantis to function properly.

First of all you need to do SSH into your server or VM.
Download the latest release of Atlantis from the official GitHub repository using the wget command:
wget -q https://github.com/runatlantis/atlantis/releases/download/v0.24.2/atlantis_linux_amd64.zip
Replace <version> with the desired Atlantis version and <arch> with the architecture of your server (e.g., amd64, arm64).
Unzip the downloaded file:
unzip atlantis_linux_amd64.zip
Move the extracted atlantis binary to a directory in your system's PATH, for example:
sudo mv atlantis /usr/bin/
Verify the installation by running the following command:
atlantis version atlantis 0.24.2
It should display the version of Atlantis installed.

Creating a webhook secret: Atlantis uses webhook secrets to validate that the webhook it receives from your git host are legitimitate.

You can use a random string generator to create your Webhook secret. It should be >24 characters.

openssl rand -hex 24

Configure Access and Authentication: Set up access controls and authentication mechanisms for Atlantis. Decide how users will authenticate with Atlantis, such as using GitHub, GitLab, or a local authentication method. Configure appropriate access rights to the Git repositories that Atlantis will manage.

Generate a Git credential using which atlantis will clone your git repository. Use the link below for the instructions based on your git host.

Starting the atlantis server: After doing the above steps you are now ready to start your atlantis server. If you want to run this in the foreground, It will start the atlantis server for you.

atlantis server --atlantis-url="https://atlantis.mydomain.com" --gh-user="my-user" --gh-token="xxx_vLXXXXXXXXXXXXXXXXXGNdv" --gh-webhook-secret="XXXXXXXXXXXXX" --repo-allowlist="github.com/MYORG/MYREPO"
Please replace the value of --atlantis-url , --gh-user, --gh-token, --gh-webhook-secret and --repo-allowlist. according to you.

This command run the atlantis server in foreground process.

If you want to run your process in background service as a systemd service, you have to create a systemd service file.
Create a file using the command
sudo vim /etc/systemd/system/atlantis.service
and paste the below content into it.
[Unit] Description=Atlantis Service for Terraform PR Automation After=network.target [Service] ExecStart=/usr/bin/atlantis server --atlantis-url="https://atlantis.mydomain.com" --gh-user="MYUSER" --gh-token="xxx_vLXXGNdv" --gh-webhook-secret="xxxxxxXXXXXX" --repo-allowlist="github.com/MYORG/MYREPO" Restart=always [Install] WantedBy=multi-user.target
Start and Enable the Atlantis Service:
Reload the systemd configuration:
sudo systemctl daemon-reload
Start the Atlantis service:
sudo systemctl start atlantis
Enable the Atlantis service to start on system boot:
sudo systemctl enable atlantis
Check the status of Atlantis service:
sudo systemctl status atlantis.service

Configuring Atlantis with Git Repositories

Once the infrastructure is ready, you can configure Atlantis to work with your Git repositories. Follow these steps:

Configure Git Provider Integration: Atlantis supports popular Git providers like GitHub, GitLab, and Bitbucket. Configure the integration by providing the necessary credentials and access tokens. This integration enables Atlantis to interact with your Git repositories, manage pull requests, and trigger deployments.
Repository-Specific Configuration: Customize Atlantis configuration on a per-repository basis. You can define repository-specific settings such as Terraform version, backend configurations, and allowed workflow actions. These settings allow you to fine-tune Atlantis behavior for each repository based on specific requirements.
Below are the steps mentioned for the different git hosts.

Setting up Webhooks and Automated Workflows

To enable automated workflows with Atlantis, you need to set up webhooks and configure the desired deployment actions. Follow these steps:

Configure Webhooks: Set up webhooks between your Git provider and Atlantis. Webhooks allow the Git provider to notify Atlantis of relevant events, such as pull request creation, updates, or merges. Configure the webhooks to point to the Atlantis server’s endpoint.
Define Workflow Triggers: Determine the triggers that will initiate the Terraform workflow. Typically, Atlantis responds to pull requests or pushes to specific branches. Define the conditions that will trigger Atlantis to perform Terraform actions, such as creating a plan or applying changes.
Define Workflow Actions: Specify the actions Atlantis should take when triggered by a webhook. For example, when a pull request is created, Atlantis can automatically run a Terraform plan to preview the changes. When a pull request is merged, Atlantis can execute Terraform apply to deploy the changes.
Configure Notification and Reporting: Set up notifications to keep stakeholders informed about the status of deployments. Configure the notifications to be sent to relevant team members or channels. This ensures visibility and transparency in the deployment process.
Test and Validate: Test the webhook and automated workflow setup by creating pull requests or pushing changes to the configured branches. Observe how Atlantis reacts to these events and performs the defined Terraform actions. Validate that the deployments are executed correctly and the notifications are sent as expected.

By following these steps, you can set up Atlantis to automate and streamline your Terraform workflows. Atlantis will integrate with your Git repositories, respond to relevant events, execute Terraform actions, and provide notifications and reporting, making your infrastructure provisioning process more efficient and manageable.

Congratulations! You have successfully installed Atlantis for Terraform Pull Request Automation. Atlantis is now running as a service on your Linux instance. Any changes to your GitHub repository will trigger Atlantis to apply the necessary Terraform operations.

GitOps Workflow with Atlantis and Terraform

Submitting Infrastructure Changes via Pull Requests

The GitOps workflow with Atlantis and Terraform starts with submitting infrastructure changes via pull requests. Here’s how it works:

Create a Feature Branch: Developers create a new branch in the Git repository to work on their infrastructure changes. They base their branch on the main branch or any other relevant branch.
Make Infrastructure Changes: Developers make the necessary changes to the Terraform code in their branch, defining the desired state of the infrastructure. They add, modify, or remove resources, configure variables, or update-modules.
Push Changes and Open a Pull Request: After making the infrastructure changes, developers push their branch to the Git repository and open a pull request. The pull request includes a description of the changes and any additional context or documentation.

Automated Validation and Review Processes

Once the pull request is opened, Atlantis triggers automated validation and review processes to ensure the quality and correctness of the infrastructure changes. Here’s what happens:

Webhook Event Detection: Atlantis, integrated with the Git provider, detects the pull request event and receives a webhook notification.
Plan Execution: Atlantis automatically runs a Terraform plan on the infrastructure code changes. This step previews the modifications without applying them, allowing stakeholders to review the proposed changes and identify any potential issues.
Commenting and Collaboration: Atlantis posts the plan output as a comment on the pull request, providing visibility into the expected changes. Team members can review the plan, ask questions, provide feedback, and discuss the proposed modifications within the pull request’s comment thread.
Approval and Merge: Once the infrastructure changes have been reviewed and approved, the pull request can be merged into the main branch. The merge event triggers further actions in the GitOps workflow.

Applying Changes and Deploying Infrastructure

After the pull request is merged, Atlantis applies the infrastructure changes and deploys the updated infrastructure. Here’s the final part of the workflow:

Merge Event Detection: Atlantis detects the merge event in the Git repository and receives the corresponding webhook notification.
Apply Execution: Atlantis automatically runs Terraform apply, which applies the changes defined in the merged branch to the target environment. Terraform ensures that the infrastructure matches the desired state described in the code.
Infrastructure Deployment: Terraform applies the changes to the target environment, provisioning or updating the infrastructure resources accordingly. Atlantis monitors the execution and provides feedback on the progress and status of the deployment.
Notification and Reporting: Atlantis sends notifications to relevant team members or channels, informing them about the success or failure of the infrastructure deployment. These notifications provide visibility into the final state of the infrastructure and any potential issues encountered during the deployment process.

By following this GitOps workflow with Atlantis and Terraform, organizations can achieve a streamlined and collaborative approach to infrastructure provisioning. It combines the power of version-controlled infrastructure code, automated validation, and review processes, and reliable deployment mechanisms to ensure that infrastructure changes are auditable, reproducible, and consistent across environments.

Conclusion

Our exploration of GitOps with Atlantis and Terraform has highlighted their remarkable potential for streamlined infrastructure deployment. This synergy enhances collaboration, security, and traceability of changes, setting the stage for efficient and reliable provisioning. But this is just the beginning.

For deeper insights and optimal infrastructure management, proceed to our next segment: Terraform Best Practices. Uncover invaluable strategies, from module design to state management, and elevate your infrastructure prowess.

Join us in shaping the future of deployment efficiency and uncovering the Terraform landscape’s untapped potential.