Tuesday, May 17, 2011

Java Program to run Pentaho kettle ETL Job

I got to use Kettle spoon as the ETL tool, and created Jobs and transformations.

and used the Pentaho provided API's to run the Transformations/Jobs from java.

An ETL Job file can be created and get stored in a disk.. the API requires the file path to get passed.


Provided the system has the required Pentaho libraries and the class path.,

The Jars can be downloaded from the Pentaho Site.

First Initialize the Kettle Environment from Java.,

/** static block that initializes the kettle environment */
    static {
        try {
            KettleEnvironment.init(false);
        } catch (KettleException e) {
            smLogger.error("Pentaho.Job.Run.Error.Initialization :" + e.getMessage());
        }
    }


And now pass the Kettle Job File Path to the API.

     /**
     * Executes the Job
     * @param jobPath jobFile
     * @throws ETLRunException
     */
    public void executeJob(String jobPath) throws ETLRunException {

# Initialize the Job Meta.

       try {
            JobMeta jobMeta = new JobMeta(jobPath, null, null);

            // Create the Job Instance
            Job job = new Job(null, jobMeta);
            job.getJobMeta().setInternalKettleVariables(job);

            job.setLogLevel("BASIC");

            job.setName(Thread.currentThread().getName());

            // Start the Job, as it is a Thread itself by Kettle.
            job.start();
            job.waitUntilFinished();

            if (job.getResult() != null && job.getResult().getNrErrors() != 0) {
                smLogger.error("Pentaho.Job.Run.Error.FinishedWithErrors");
                throw new ETLRunException("Pentaho.Job.Run.Error.FinishedWithErrors");
            }

            // Now the job task is finished, mark it as finished.
            job.setFinished(true);

            // Cleanup the parameters used by the job. Post that invoke GC.
            jobMeta.eraseParameters();
            job.eraseParameters();

        } catch (Exception e) {
            smLogger.error("Pentaho.Job.Run.Error.FinishedWithErrors :" + e.getMessage());
             throw new ETLRunException(e);
        }

    }
     

3 comments: