A Spring Batch Job For GitHub Repo Stats

We added a new Spring Batch Job in NixMash Spring v0.5.0 to end the practice of calling the GitHub API in real-time to display the NixMash Spring Repository Stats on the Home Page. Here are the GitHub NixMash Spring Repository Statistics which appear on the home page, previously generated with JSON and Spring RestTemplates.

With the Spring Batch Job online, the numbers are now retrieved from this local MySQL table. The method data is cached with Spring Cache, so double bonus.

The Job

We wrote a Four Part Spring Batch Concepts series if you want to compare the GitHub Job with that series’ DemoJob. Here are the GitHub Job Classes.

The GitHub JobRunner is nearly identical to the Demo JobRunner. The Runner will launch if –job=github is on the Command Line when executing the JAR because we added a @Condition on the class. (See the 4-part series.) We’re setting the Job Launch @Scheduled interval from an External Properties file.

Nothing new about the Job Definition in GitHubJobConfiguration.

The Step Definition on the other hand is quite different from our DemoJob where we used the standard Reader | Processor | Writer configuration. We are retrieving our GitHub data with JSON and there’s no ItemReader Interface for that, so instead of creating a Reader, we perform the entire job in a Spring Batch TASKLET.

We want to write out the MySQL GitHub_Stats Table StatId value to our logfile when the job completes so we add a PromotionListener for the statId key. (Second from the bottom line of code.)

Our JobListener records the statId key value for us, which we’re going to send to a logfile as well as save in a Spring Batch table as the Job’s Exit_Message.

The job logic overwrites any data in a MySQL github_stats table retrieved from the GitHub API for that current Day. One recorded statistic grouping per day, in other words. A custom MySQL View of Spring Batch Job data shows the hourly execution of the job. Notice the GitHub Table StatId value at right and how the value increments on each new day. (Records 88-89.)

And here again is the final result of GitHub statistics for each day.

Source Code Notes for this Post

All source code discussed in this post can be found in my NixMash Spring GitHub repo and viewed online here.