Automating Jira → Snowflake Data Pipeline Using ScriptRunner

This guide explains Jira -> Snowflake Data Pipeline automation. We are showing how to extract data from multiple Jira projects using ScriptRunner (Groovy), transform it, and securely load it into Snowflake using the Snowflake REST API, fully automated and schedulable.

Dependencies – Environment used for this how-to article (expand):

Core managed services:

  • Jira Cloud tenant with ScriptRunner for Jira Cloud installed and licensed
  • Snowflake account in your target region with Statements REST API enabled

Accounts and roles:

  • Jira technical user with site admin and project admin on target projects
  • Snowflake service user JIRA_LOADER with role JIRA_ROLE
    • Grants: USAGE on warehouse, database, schema; INSERT and SELECT on target tables; CREATE VIEW if you want views

Networking:

  • Outbound HTTPS from ScriptRunner to https://<account>.snowflakecomputing.com
  • TLS 1.2 or higher

Auth and secrets (store in ScriptRunner secure variables):

  • Preferred: RSA key pair for Snowflake user (private key encrypted)
  • Alternative: strong password for basic auth
  • Never hardcode secrets in Groovy

Snowflake objects:

  • Warehouse: JIRA_WH (XSMALL or SMALL, auto suspend 60 seconds)
  • Database: JIRA_DB
  • Schemas: RAW for landings, MART or ANALYTICS for modeled views
  • Table: RAW.JIRA_ISSUES_RAW with a VARIANT column and typed columns for hot fields

ScriptRunner job settings:

  • Type: Scheduled job
  • Cron: 0 1 * * * (01:00 Europe/Berlin) or as needed
  • Pagination for Jira search API with startAt and maxResults=100
  • Backoff and retry for HTTP 429 and 5xx from Snowflake
  • Chunk payloads if the row count is large

Observability:

  • Log run id, JQL scope, page counts, durations, row counts
  • Capture Snowflake statement ID from API responses
  • Alert on non-zero exit or exceptions

Governance:

  • Separate loader and reader roles
  • Keep raw payload 30 to 90 days, then purge or anonymize
  • Avoid ingesting PII unless needed

Diagram

1. Architecture Diagram (Text Representation):

[Jira Cloud Projects] 
    ↓ (JQL + ScriptRunner Groovy)
ScriptRunner Automation Job
    ↓ (JSON Dataset)
HTTPS POST → Snowflake REST API (INSERT… SQL/PUT)
    ↓
Snowflake Table (RAW or STG Schema)
    ↓
Reporting / SLA Dashboard / BI Tools

2. Prerequisites

ComponentWhat You Need
Jira CloudAdmin access, ScriptRunner installed
ScriptRunnerAbility to write Groovy scripts + Scheduled Jobs
Jira API TokenFrom https://id.atlassian.com/manage/api-tokens
SnowflakeWarehouse + Database + Schema + Role with INSERT privileges
Snowflake AuthenticationUser + password or OAuth/JWT (keypair recommended)
Allowed NetworkJira must be allowed to send HTTPS requests to Snowflake

3. Create a Snowflake Target Table

CREATE OR REPLACE TABLE jira_issues_raw (
    issue_key      STRING,
    summary        STRING,
    status         STRING,
    project        STRING,
    issue_type     STRING,
    created_at     TIMESTAMP_TZ,
    updated_at     TIMESTAMP_TZ,
    assignee       STRING,
    reporter       STRING,
    priority       STRING,
    raw_payload    VARIANT
);

Optional: Create a dedicated Snowflake Warehouse + Role for Jira ingestion.

4. ScriptRunner: Complex JQL Query

Example JQL that spans multiple projects:

String jql = """
project in ("Service Desk", "Internal Software") 
AND statusCategory != Done 
ORDER BY updated DESC
"""
def issues = get("/rest/api/3/search")
    .queryString("jql", jql)
    .queryString("maxResults", "1000")
    .asObject(Map)
    .body.issues

5. Convert Jira Issue Data to JSON Payload

import groovy.json.JsonOutput

def rows = issues.collect { it ->
    [
        issue_key : it.key,
        summary   : it.fields.summary,
        status    : it.fields.status.name,
        project   : it.fields.project.key,
        issue_type: it.fields.issuetype.name,
        created_at: it.fields.created,
        updated_at: it.fields.updated,
        assignee  : it.fields.assignee?.displayName,
        reporter  : it.fields.reporter?.displayName,
        priority  : it.fields.priority?.name,
        raw_payload: it // entire issue object
    ]
}
String jsonBody = JsonOutput.toJson(rows)

6. Send Data to Snowflake via REST API

Snowflake provides a /queries/v1/query-request REST endpoint.

import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import java.util.Base64

// Snowflake credentials
def sfAccount   = "xy12345.eu-central-1"
def sfUser      = "JIRA_LOADER"
def sfPassword  = "StrongPassword!"
def sfWarehouse = "JIRA_WH"
def sfDatabase  = "JIRA_DB"
def sfSchema    = "PUBLIC"
def sfRole      = "JIRA_ROLE"

// Build INSERT SQL
def sql = """
INSERT INTO jira_issues_raw (
    issue_key, summary, status, project, issue_type,
    created_at, updated_at, assignee, reporter, priority, raw_payload
)
SELECT
    value:issue_key,
    value:summary,
    value:status,
    value:project,
    value:issue_type,
    TO_TIMESTAMP(value:created_at),
    TO_TIMESTAMP(value:updated_at),
    value:assignee,
    value:reporter,
    value:priority,
    parse_json(value:raw_payload)
FROM TABLE(FLATTEN(input => PARSE_JSON('$jsonBody')));
"""

def insertBody = [
    "sqlText": sql,
    "sequenceId": 1
]

def response = post("https://${sfAccount}.snowflakecomputing.com/api/v2/statements")
    .header("Content-Type", "application/json")
    .basicAuth(sfUser, sfPassword)
    .body(insertBody)
    .asString()

return response.body

7. Schedule as a Recurring Job in ScriptRunner

Jira → Administration → ScriptRunner → Scheduled Jobs → Add Job

SettingValue
Job NameJira → Snowflake Sync
Schedule0 0 * * * (daily at midnight)
Script TypeInline Script
ScriptContains JQL → JSON → Snowflake Insert code

8. Validate in Snowflake

SELECT * FROM jira_issues_raw ORDER BY updated_at DESC LIMIT 10;

9. Enrich with SLA/Reporting Table

CREATE VIEW jira_sla_summary AS
SELECT
    project,
    COUNT(*) AS open_issues,
    COUNT_IF(status = 'Done') AS completed,
    AVG(DATEDIFF('day', created_at, updated_at)) AS avg_resolution_days
FROM jira_issues_raw
GROUP BY project;

10. Explanation Glossary (For End of Article)

TermExplanation
ScriptRunnerJira add-on for automation and custom scripting (Groovy)
GroovyJVM scripting language used by ScriptRunner
JQLJira Query Language — powerful filtering over issues
Snowflake REST APIAllows executing SQL over HTTPS instead of JDBC/ODBC
VariantSnowflake data type for semi-structured JSON data
Statement APISnowflake endpoint: /api/v2/statements
ISO8601 TimestampFormat used by Jira: 2025-01-10T15:22:11.000+0000

BlueGrid.io Content Team

Three people pose together against a plain white background. The woman on the left is smiling with her hand on her hip, while the two men beside her stand closely, one in a hoodie and the other in a plaid shirt.

BlueGrid.io Content Team

BlueGrid.io Team is an editorial collective of engineers, practitioners, and contributors sharing insights across technology, operations, company culture, and the people behind the systems. Content is created through interviews, hands-on experience, internal collaboration, and editorial review, reflecting both how systems are built and how teams work together in real-world environments.

Share this post

Share this link via

Or copy link