DB Deployment gets freeze or failing in step level

General Discussion
1

This category is where most discussions regarding Liquibase will typically take place.

We are using Liquibase for DB deploy and when ever we raise a PR we have ETL deploy workflow which uses Liquibase to deploy db change. Most of the time it will stuck in liquibase tag or liquibase running validation. Here is the sh code we are using

Any help is appreciated. Thanks in Advance

Liquibase version : 4.24.0 Data warehosue: Redshift

set -e

ls -la

if [[ ${ROLLBACK} == "true" ]]; then
  export LIQUIBASE_COMMAND_TAG=$DB_VERSION
fi

echo "DB Deployment execution"
echo "- LIQUIBASE_SEARCH_PATH:                  $LIQUIBASE_SEARCH_PATH"
echo "- LIQUIBASE_COMMAND_URL:                  $LIQUIBASE_COMMAND_URL"
echo "- LIQUIBASE_COMMAND_USERNAME:             $LIQUIBASE_COMMAND_USERNAME"
echo "- LIQUIBASE_COMMAND_CHANGELOG_FILE:       $LIQUIBASE_COMMAND_CHANGELOG_FILE"
echo "- LIQUIBASE_COMMAND_CONTEXTS:             $LIQUIBASE_COMMAND_CONTEXTS"
echo "- LIQUIBASE_LIQUIBASE_SCHEMA_NAME:        $LIQUIBASE_LIQUIBASE_SCHEMA_NAME"
echo "- LIQUIBASE_COMMAND_DEFAULT_SCHEMA_NAME:  $LIQUIBASE_COMMAND_DEFAULT_SCHEMA_NAME"
echo "- DB_VERSION:                             $DB_VERSION"
echo "- DRY_RUN:                                $DRY_RUN"
echo "- ROLLBACK:                               $ROLLBACK"

echo "Running Validation =========================================================="
liquibase validate

echo "Checking Status =========================================================="
liquibase status --verbose

echo "Printing sqls ============================================================"
if [[ ${ROLLBACK} == "true" ]]; then
  liquibase rollback-sql
else
  liquibase update-sql
fi

if [[ ${DRY_RUN} == "true" ]]; then
  echo "Skipping deployment as the pipeline is triggered in dryrun mode =========="
  exit 0;
fi

if [[ ${ROLLBACK} == "true" ]]; then
  echo "Executing rollback =================================="
  liquibase rollback
else
  echo "Executing deployment =================================="
  liquibase update
  echo "Performing tagging ====================================================="  
  liquibase tag --tag=$DB_VERSION
fi

liquibase history ```

Yaml code:

name: etl-scripts-pr-pipeline

on:
pull_request:
branches:
- master
paths:
- ‘src/parameters/’
- 'src/data-schemas/’
- ‘src/glue/’
- 'src/packages/’
- ‘redshift-database/**’

jobs:
create-artifact:
runs-on: “gh-runner-ops-generic”
outputs:
version: ${{ steps.version.outputs.version }}
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set temporary version
  id: version
  run: |
    version=${{ github.event.pull_request.head.sha }}-${{ github.run_id }}
    echo "version=$version" >> "$GITHUB_ENV"
    echo "version=$version" >> "$GITHUB_OUTPUT"
- name: Create artifact
  run: |
    ./.github/scripts/create-package.sh ${{ env.version }}
- name: Upload artifact
  uses: actions/upload-artifact@v4
  with:
    name: ${{ env.version }}
    path: asset/*.zip

perform-deployment:
needs:
- create-artifact
uses: ./.github/workflows/deploy-pipeline.yaml
with:
env: dev
version: ${{ needs.create-artifact.outputs.version }}
secrets: inherit

##################

uses: ./.github/workflows/deploy-pipeline.yaml

name: db-deploy-pipeline

on:
workflow_call:
inputs:
env:
description: Environment to deploy DB scripts
required: true
type: string
version:
description: Version of DB scripts to deploy
type: string
required: true
rollback:
description: Performing rollback? - BE CAUTIOUS
type: boolean
required: false
default: false
dry_run:
description: Execute dry-run only?
type: boolean
required: false
default: true
jobs:
db-deploy:
runs-on: self-hosted
environment: ${{ inputs.env }}
env:
REDSHIFT_ACCOUNT_ROLE: ${{ vars.REDSHIFT_ROLE }}
LIQUIBASE_COMMAND_CHANGELOG_FILE: ${{ vars.LIQUIBASE_COMMAND_CHANGELOG_FILE }}
LIQUIBASE_COMMAND_CONTEXTS: ${{ vars.LIQUIBASE_COMMAND_CONTEXTS }}
LIQUIBASE_COMMAND_DEFAULT_SCHEMA_NAME: ${{ vars.LIQUIBASE_COMMAND_DEFAULT_SCHEMA_NAME }}
LIQUIBASE_COMMAND_URL: ${{ vars.LIQUIBASE_COMMAND_URL }}
LIQUIBASE_LIQUIBASE_SCHEMA_NAME: ${{ vars.LIQUIBASE_LIQUIBASE_SCHEMA_NAME }}
DB_NAME: ${{ vars.REDSHIFT_DB }}
DB_DEPLOY_USER: ${{ vars.REDSHIFT_DB_DEPLOY_USER }}
LIQUIBASE_SEARCH_PATH: “deploy-artifact”
DB_VERSION: ${{ inputs.version }}
DRY_RUN: ${{ inputs.dry_run }}
ROLLBACK: ${{ inputs.rollback }}
container:
image: ghcr.io/liquibase-redshift:latest
permissions:
id-token: write
contents: read
packages: read
actions: read

steps:
  - name: Checkout
    uses: actions/checkout@v4

  - name: Download latest release artifact
    if: ${{ inputs.rollback }} # for rollbacks we have to download current/latest version
    uses: robinraju/release-downloader@v1.10
    with:
      latest: true
      fileName: "db-scripts-*.zip"

  - name: Check version is release or temporary build (hash)
    id: check-ver
    run: |
      if [[ "${{ inputs.version }}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
        echo "Released version is provided"
        echo "RELEASED_VERSION=true" >> "$GITHUB_ENV"
      else
        # in PR builds, artifact contains commithash-runid
        ARTIFACT="${{ inputs.version }}"
        echo "ARTIFACT_RUN_ID="${ARTIFACT##*-}"" >> "$GITHUB_ENV"
      fi
  
  - name: Download temporary build artifact=${{ inputs.version }}
    if: ${{ !env.RELEASED_VERSION && !inputs.rollback }}
    uses: actions/download-artifact@v4
    with:
      name: ${{ inputs.version }}
      run-id: ${{ env.ARTIFACT_RUN_ID }}
      github-token: ${{ secrets.GITHUB_TOKEN }}

  - name: Download release artifact version=${{ inputs.version }}
    if: ${{ env.RELEASED_VERSION && !inputs.rollback }}
    uses: robinraju/release-downloader@v1.10
    with:
      tag: "${{ inputs.version }}"
      fileName: "db-scripts-${{ inputs.version }}.zip"
  
  - name: Extract downloaded content
    run: |
      ls -la
      pwd
      unzip db-scripts-*.zip -d deploy-artifact
      cd deploy-artifact
      ls -la

  - name: Debug
    run: |
      ls -la
      pwd

  - name: Assume the role for this project in the ${{ inputs.env }}
    uses: aws-actions/configure-aws-credentials@v4
    with:
      role-to-assume: ${{ vars. REDSHIFT_DB_MGMENT_ROLE_ARN }}
      role-session-name: "${{ github.run_id }}-${{ github.run_number }}"
      aws-region: eu-central-1
  
  - name: Get cluster creds for db-user
    run: |
      LIQUIBASE_COMMAND_USERNAME="IAM:${{ vars.REDSHIFT_DB_DEPLOY_USER }}"
      echo "LIQUIBASE_COMMAND_USERNAME=$LIQUIBASE_COMMAND_USERNAME" >> "$GITHUB_ENV"
      db_passwd=$(aws redshift get-cluster-credentials \
      --db-user ${{ vars.REDSHIFT_DB_DEPLOY_USER }} --db-name ${{ vars.REDSHIFT_DB }} \
      --cluster-identifier ${{ vars.REDSHIFT_CLUSTER_ID }} --query "DbPassword" --output text)
      echo "LIQUIBASE_COMMAND_PASSWORD=$db_passwd" >> "$GITHUB_ENV"
    shell: bash

  - name: Perform db deployment
    run: |
      ls -la          
      chmod +x .github/scripts/db-deploy.sh
      ./.github/scripts/db-deploy.sh
    shell: bash

update-dashboard:
needs:
- db-deploy
name: Updating dashboard for db-scripts
if: ${{ !inputs.dry_run }}
uses: ./.github/workflows/update-confluence-dashboard.yaml
with:
component: db-scripts
col_name: ${{ inputs.env }}
version: ${{ inputs.version }}
link: “https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}”
secrets: inherit