Automating Coolify Deployment with GitHub Actions: Overcoming Challenges

October 28, 2024

In this article, I explore the process of managing deployments with Coolify using GitHub Actions, focusing on the challenges encountered, the tools used to address them, and the step-by-step solutions that were implemented. This guide includes reusable scripts, GitHub Actions workflows, and insights that can help streamline similar deployments, particularly for self-hosted setups using Next.js applications.

Overview of Coolify Deployment

For more information about Coolify, please refer to the official Coolify Documentation.

Coolify is a self-hosted, open-source platform that simplifies deploying applications, offering an experience similar to popular cloud services but with full control. In our case, we had a Next.js application that needed frequent and automated deployment, and we were using Coolify as our deployment solution hosted on our own infrastructure. To make things even more streamlined, we enabled IP access restrictions to ensure secure application management.

Coolify automatically created an application for us that managed the deployment, built Docker containers using a Nix-based Dockerfile, and pushed changes effectively. While Coolify provides a great user experience, managing deployments programmatically, particularly using GitHub Actions, presented some unique challenges. These included monitoring deployment statuses, handling health checks, and ensuring that the deployed version always reflected the correct commit.

The goal of this article is to show how these challenges were handled, providing practical code examples and scripts that you can reuse for your own deployment processes.

Challenges in Deployment

Incomplete Deployments

For troubleshooting incomplete deployments, you can consult the Coolify Deployment Troubleshooting Guide.

One common issue faced was that Coolify deployments would sometimes not complete or appear stuck in the "in_progress" state. This made it difficult to determine whether an application had successfully deployed or whether manual intervention was required.

Application Health Monitoring

Another challenge was ensuring that applications transitioned to the "running" state after deployment. It was critical to distinguish between "running" and "running" statuses to decide whether further action was needed, such as a rollback or alert.

Commit Tracking Issues

At times, the latest commit information in the Coolify UI remained set to "HEAD", which indicated that the deployment process was not reflecting the correct Git commit. This problem needed to be addressed to verify that the right code was being deployed.

Tools and Solutions Used

To get a comprehensive understanding of Coolify's capabilities and deployment options, visit the Coolify Documentation Overview.

To manage and solve these deployment issues, a combination of tools and scripts were utilized. Below, I'll explain the tools, scripts, and the GitHub Actions workflow that were put in place to overcome the difficulties.

Using Curl and Shell Scripts

The use of curl and shell scripts (.sh files) played a pivotal role in interacting with Coolify's API to retrieve deployment statuses and other necessary information.

getCoolifyAppStatus.sh

This script was used to fetch the current application status from Coolify's API.

# getCoolifyAppStatus.sh
curl -s -H "Authorization: Bearer YOUR_COOLIFY_TOKEN" \
"https://INSTANCE_URL/api/v1/applications/YOUR_APP_ID"

This script returned the application status, Git commit SHA, and other metadata, which was critical to track the progress of deployments.

checkCoolifyDeployments.sh

This script was used to monitor any ongoing deployments. It checked for active deployments and retrieved their status.

# checkCoolifyDeployments.sh
curl -s -H "Authorization: Bearer YOUR_COOLIFY_TOKEN" \
"https://INSTANCE_URL/api/v1/deployments"

By using these scripts, it was possible to poll Coolify for real-time deployment data and make decisions based on the returned results.

GitHub Secrets Explained

Below is an explanation of the GitHub secrets used in the workflow file:

  • GITHUB_PAT: Personal Access Token for GitHub. Used to authenticate and set deployment statuses on GitHub via API requests.
  • COOLIFY_AUTH_TOKEN: Authorization token for Coolify’s API. Enables access to Coolify’s deployment and application statuses.
  • COOLIFY_BASE_URL: Base URL for Coolify’s API endpoint. Used to construct API requests for retrieving deployment and app data.
  • COOLIFY_APP_ID: The unique application ID in Coolify for the staging environment. Used to fetch status and monitor deployments.

For the full workflow and more details, you can access the code here.

GitHub Actions Workflow

To learn more about integrating Coolify with CI/CD tools like GitHub Actions, see the Coolify CI/CD Integration Guide.

A GitHub Actions workflow was created to automate the process of managing Coolify deployments. This workflow used environment variables and API polling to ensure that deployments were completed correctly.

name: Deployment Notification Workflow

on:
  push:
    branches:
      - staging

jobs:
  notify-deployment:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v2

      - name: Set deployment status to pending
        run: |
          curl -X POST \
          -H "Authorization: token ${{ secrets.GITHUB_PAT }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/statuses/${{ github.sha }} \
          -d '{"state": "pending", "description": "Deployment initiated", "context": "Deployment"}'

      - name: Capture deployment start time
        run: |
          DEPLOY_START_TIME=$(date -u '+%Y-%m-%dT%H:%M:%S')
          echo "DEPLOY_START_TIME=$DEPLOY_START_TIME" >> $GITHUB_ENV

      - name: Monitor Coolify Deployment Status
        id: monitor
        env:
          COOLIFY_AUTH_TOKEN: ${{ secrets.COOLIFY_AUTH_TOKEN }}
          COOLIFY_BASE_URL: ${{ secrets.COOLIFY_BASE_URL }}
          COOLIFY_APP_ID: ${{ secrets.COOLIFY_APP_ID }}
        run: |
          deployment_status="in_progress"
          
          # Step 1: Wait for any deployments to complete
          for i in {1..60}; do
            # Fetch the list of ongoing deployments
            response=$(curl -s -H "Authorization: Bearer $COOLIFY_AUTH_TOKEN" \
              "$COOLIFY_BASE_URL/api/v1/deployments?appId=$COOLIFY_APP_ID&limit=1")

            # If response is empty, break the loop
            if [ "$response" == "[]" ]; then
              break
            else
              sleep 20
            fi
          done

          # Step 2: Confirm application status is running:healthy
          for i in {1..30}; do
            # Fetch the application status
            response=$(curl -s -H "Authorization: Bearer $COOLIFY_AUTH_TOKEN" \
              "$COOLIFY_BASE_URL/api/v1/applications/$COOLIFY_APP_ID")

            # Extract status and updated_at timestamp using jq
            app_status=$(echo "$response" | jq -r '.status')
            update_time=$(echo "$response" | jq -r '.updated_at')
            git_commit_sha=$(echo "$response" | jq -r '.git_commit_sha')

            # Compare timestamps (update_time should be after DEPLOY_START_TIME)
            if [[ "$update_time" > "$DEPLOY_START_TIME" ]]; then
              if [[ "$app_status" == "running:healthy" ]]; then
                deployment_status="success"
                break
              elif [[ "$app_status" == "running:unhealthy" || "$app_status" == "error" || "$app_status" == "failed" ]]; then
                deployment_status="failure"
                break
              fi
            fi
            sleep 20
          done

          # Exit gracefully if still in progress after retries
          if [ "$deployment_status" == "in_progress" ]; then
            deployment_status="failure"
          fi

          # Set an output for the deployment status
          echo "deployment_status=$deployment_status" >> $GITHUB_ENV

      - name: Update deployment status to success
        if: env.deployment_status == 'success'
        run: |
          curl -X POST \
          -H "Authorization: token ${{ secrets.GITHUB_PAT }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/statuses/${{ github.sha }} \
          -d '{"state": "success", "description": "Deployment completed successfully", "context": "Deployment"}'

      - name: Update deployment status to failure
        if: env.deployment_status == 'failure'
        run: |
          curl -X POST \
          -H "Authorization: token ${{ secrets.GITHUB_PAT }}" \
          -H "Accept: application/vnd.github.v3+json" \
          https://api.github.com/repos/${{ github.repository }}/statuses/${{ github.sha }} \
          -d '{"state": "failure", "description": "Deployment failed", "context": "Deployment"}'

Key Insights

  • Automation with GitHub Actions: Automating deployments with GitHub Actions can streamline the process, but it is essential to incorporate proper checks and retry mechanisms to handle unexpected issues.
  • Polling and Monitoring: Polling Coolify's API effectively using scripts was crucial to managing deployment statuses. Tools like curl and jq helped parse and respond to the information from Coolify's API.
  • Reusable Scripts: The .sh scripts (getCoolifyAppStatus.sh and checkCoolifyDeployments.sh) can be reused for future deployments to interact with Coolify more efficiently.

Conclusion

Managing Coolify deployments with GitHub Actions requires thoughtful automation strategies, particularly when it comes to monitoring and handling errors. By using curl commands, shell scripts, and well-defined GitHub Actions workflows, I was able to overcome the challenges and ensure smooth deployments.

This is a draft solution, and it can be further updated to improve efficiency and handle more complex scenarios.

You can reuse the provided scripts and GitHub Actions examples to manage your own Coolify deployments. For more details and access to the complete code, visit the GitHub repository.