This guide explains Jira -> Snowflake Data Pipeline automation. We are showing how to extract data from multiple Jira projects using ScriptRunner (Groovy), transform it, and securely load it into Snowflake using the Snowflake REST API, fully automated and schedulable.
Dependencies – Environment used for this how-to article (expand):
Core managed services:
- Jira Cloud tenant with ScriptRunner for Jira Cloud installed and licensed
- Snowflake account in your target region with Statements REST API enabled
Accounts and roles:
- Jira technical user with site admin and project admin on target projects
- Snowflake service user
JIRA_LOADERwith roleJIRA_ROLE- Grants: USAGE on warehouse, database, schema; INSERT and SELECT on target tables; CREATE VIEW if you want views
Networking:
- Outbound HTTPS from ScriptRunner to
https://<account>.snowflakecomputing.com - TLS 1.2 or higher
Auth and secrets (store in ScriptRunner secure variables):
- Preferred: RSA key pair for Snowflake user (private key encrypted)
- Alternative: strong password for basic auth
- Never hardcode secrets in Groovy
Snowflake objects:
- Warehouse:
JIRA_WH(XSMALL or SMALL, auto suspend 60 seconds) - Database:
JIRA_DB - Schemas:
RAWfor landings,MARTorANALYTICSfor modeled views - Table:
RAW.JIRA_ISSUES_RAWwith aVARIANTcolumn and typed columns for hot fields
ScriptRunner job settings:
- Type: Scheduled job
- Cron:
0 1 * * *(01:00 Europe/Berlin) or as needed - Pagination for Jira search API with
startAtandmaxResults=100 - Backoff and retry for HTTP 429 and 5xx from Snowflake
- Chunk payloads if the row count is large
Observability:
- Log run id, JQL scope, page counts, durations, row counts
- Capture Snowflake statement ID from API responses
- Alert on non-zero exit or exceptions
Governance:
- Separate loader and reader roles
- Keep raw payload 30 to 90 days, then purge or anonymize
- Avoid ingesting PII unless needed
Diagram
1. Architecture Diagram (Text Representation):
[Jira Cloud Projects]
↓ (JQL + ScriptRunner Groovy)
ScriptRunner Automation Job
↓ (JSON Dataset)
HTTPS POST → Snowflake REST API (INSERT… SQL/PUT)
↓
Snowflake Table (RAW or STG Schema)
↓
Reporting / SLA Dashboard / BI Tools2. Prerequisites
| Component | What You Need |
|---|---|
| Jira Cloud | Admin access, ScriptRunner installed |
| ScriptRunner | Ability to write Groovy scripts + Scheduled Jobs |
| Jira API Token | From https://id.atlassian.com/manage/api-tokens |
| Snowflake | Warehouse + Database + Schema + Role with INSERT privileges |
| Snowflake Authentication | User + password or OAuth/JWT (keypair recommended) |
| Allowed Network | Jira must be allowed to send HTTPS requests to Snowflake |
3. Create a Snowflake Target Table
CREATE OR REPLACE TABLE jira_issues_raw (
issue_key STRING,
summary STRING,
status STRING,
project STRING,
issue_type STRING,
created_at TIMESTAMP_TZ,
updated_at TIMESTAMP_TZ,
assignee STRING,
reporter STRING,
priority STRING,
raw_payload VARIANT
);Optional: Create a dedicated Snowflake Warehouse + Role for Jira ingestion.
4. ScriptRunner: Complex JQL Query
Example JQL that spans multiple projects:
String jql = """
project in ("Service Desk", "Internal Software")
AND statusCategory != Done
ORDER BY updated DESC
"""
def issues = get("/rest/api/3/search")
.queryString("jql", jql)
.queryString("maxResults", "1000")
.asObject(Map)
.body.issues5. Convert Jira Issue Data to JSON Payload
import groovy.json.JsonOutput
def rows = issues.collect { it ->
[
issue_key : it.key,
summary : it.fields.summary,
status : it.fields.status.name,
project : it.fields.project.key,
issue_type: it.fields.issuetype.name,
created_at: it.fields.created,
updated_at: it.fields.updated,
assignee : it.fields.assignee?.displayName,
reporter : it.fields.reporter?.displayName,
priority : it.fields.priority?.name,
raw_payload: it // entire issue object
]
}
String jsonBody = JsonOutput.toJson(rows)6. Send Data to Snowflake via REST API
Snowflake provides a /queries/v1/query-request REST endpoint.
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import java.util.Base64
// Snowflake credentials
def sfAccount = "xy12345.eu-central-1"
def sfUser = "JIRA_LOADER"
def sfPassword = "StrongPassword!"
def sfWarehouse = "JIRA_WH"
def sfDatabase = "JIRA_DB"
def sfSchema = "PUBLIC"
def sfRole = "JIRA_ROLE"
// Build INSERT SQL
def sql = """
INSERT INTO jira_issues_raw (
issue_key, summary, status, project, issue_type,
created_at, updated_at, assignee, reporter, priority, raw_payload
)
SELECT
value:issue_key,
value:summary,
value:status,
value:project,
value:issue_type,
TO_TIMESTAMP(value:created_at),
TO_TIMESTAMP(value:updated_at),
value:assignee,
value:reporter,
value:priority,
parse_json(value:raw_payload)
FROM TABLE(FLATTEN(input => PARSE_JSON('$jsonBody')));
"""
def insertBody = [
"sqlText": sql,
"sequenceId": 1
]
def response = post("https://${sfAccount}.snowflakecomputing.com/api/v2/statements")
.header("Content-Type", "application/json")
.basicAuth(sfUser, sfPassword)
.body(insertBody)
.asString()
return response.body7. Schedule as a Recurring Job in ScriptRunner
Jira → Administration → ScriptRunner → Scheduled Jobs → Add Job
| Setting | Value |
|---|---|
| Job Name | Jira → Snowflake Sync |
| Schedule | 0 0 * * * (daily at midnight) |
| Script Type | Inline Script |
| Script | Contains JQL → JSON → Snowflake Insert code |
8. Validate in Snowflake
SELECT * FROM jira_issues_raw ORDER BY updated_at DESC LIMIT 10;9. Enrich with SLA/Reporting Table
CREATE VIEW jira_sla_summary AS
SELECT
project,
COUNT(*) AS open_issues,
COUNT_IF(status = 'Done') AS completed,
AVG(DATEDIFF('day', created_at, updated_at)) AS avg_resolution_days
FROM jira_issues_raw
GROUP BY project;10. Explanation Glossary (For End of Article)
| Term | Explanation |
|---|---|
| ScriptRunner | Jira add-on for automation and custom scripting (Groovy) |
| Groovy | JVM scripting language used by ScriptRunner |
| JQL | Jira Query Language — powerful filtering over issues |
| Snowflake REST API | Allows executing SQL over HTTPS instead of JDBC/ODBC |
| Variant | Snowflake data type for semi-structured JSON data |
| Statement API | Snowflake endpoint: /api/v2/statements |
| ISO8601 Timestamp | Format used by Jira: 2025-01-10T15:22:11.000+0000 |