Run PDI 5.0 Under Ubuntu 14.04 LTS

The problem faced when starting PDI 5.0 either from binary distribution either from source build in Ubuntu is that it breaks down the JVM with dump like:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007fe30873d2a1, pid=4547, tid=140615535015680
#
# JRE version: Java(TM) SE Runtime Environment (8.0_31-b13) (build 1.8.0_31-b13)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.31-b07 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libsoup-2.4.so.1+0x6c2a1]  soup_session_feature_detach+0x11
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

The problem is with the packed SWT library. This can be easily fixed if the swt.jar from working PDI 5.1 or 5.2 distributions is used.

Export JIRA Data Using Kettle REST Client

Kettle version 4.4 provides step which is able to connect remote REST services. As JIRA provides such interface it is fully possible to get data remotely using this step. Anyway some tricks should be applied in order to do that.

Mandatory Input Row

To start REST step communication with remote host an input row is needed. This was hard for me to find out as it is normal for such step to provide data input directly from its communication.

So if you need to input data from remote JIRA you'll have to provide at least one (in this case generated) input row.

Configure JIRA Connection

To connect remote JIRA server you'll need its root URL.

Add the root URL and don't forget to set the REST call statement. In this case calling search statement for project with key 'DOX'.

Set HTTP method to GET and Application type to JSON. Resulting JSON text will be passed to following steps under 'result' field name.

It is also necessary to provide authentication information if your JIRA instance interface is not publicly accessible.

Set your JIRA user name into 'Http Login' field and your password into 'Http Password' field.

Extract Data

As the REST service step provides JSON text input you can use 'Json Input' step provided in Kettle the same way as mentioned in Export JIRA Data Using Pentaho DI

Just provide JSON Path expressions for each field you need to extract. On this picture extracting task key, task summary and task status is shown.

PDI Jira

The idea behind PDI Jira is to be more specialized in using JIRA REST interface for integrating JIRA data using Pentaho DI although it is in too early phase and still needs JSON Input step for parsing result.

Export JIRA Data Using Pentaho DI

JIRA tool is very useful when organizing development process. Although there are lots of information available in the application and lots of plugins which give different views, sometimes there is a need for extracting data from JIRA database. It is normal for the JIRA server to hide its database and forbid direct access to it though. Most of the times there is web interface used for accessing particular JIRA data.

There are lots of tools and development libraries available for accessing this web interface also. What I'm providing is a useful plugin for those who use Pentaho Data Integration. This plugin is a simple step to connect remote JIRA instance, and fetch some data into the transformation using simple JQL (JIRA Query Language) expressions. All the data input can be then processed in different ways into the PDI transformations or jobs.

You can get trial JIRA version here, and you can download absolutely free of charge Pentaho DI from here.

Plugin Installation

PDI Jira Plugin can be downloaded from here.
After downloading the zipped package, you have to just unzip it in the PDI steps folders:

${pdi.home}/plugins/steps

Where ${pdi.home} is the folder where PDI is installed. That is all you need. Then start the PDI Spoon GUI.

PDI Jira Quick Start

Start a new transformation with Spoon. Then add 'Jira Plugin' from the 'Input' category.

After double clicking the Jira Plugin step you'll be able to edit remote Jira instance connection properties:

On connection tab there is also a 'Test Connection' button which can be used for testing connection to remote instance.

On JQL tab you'll be able to edit Jira Query Language statement which will be used to obtain data from remote Jira server.

Data from the Jira instance is provided to the next step in the transformation as JSON string which has field named as specified in 'Output field name'.

For the moment you can use available JSON Input plugin in Pentaho DI to extract required data from the JSON text. JSON Input plugin uses JSON Path which is much like XPath query language. For instance to extract issue key, issue status and summary you can configure following JSON Path statements: