Writing jobs for Kitto

For the past couple of weeks, I've been focused on a new framework, in the Elixir world, called Kitto. Kitto is a rebirth, so to speak, of the popular Dashing framework originally written by Shopify in Ruby. For those that haven't heard of either, Kitto is a framework for building data driven dashboards. Data to feed the widgets of the dashboard is pulled in via Jobs which polls web services, databases or any other source of information on a predefined schedule.

Given that jobs are the driving force for getting data into the dashboard (for now), I thought I'd write a quick post about how to write a simple job. I wanted to go a step further than the silly jobs in the sample dashboard that you get when you install Kitto and integrate with a real service. We use JIRA a lot at Cage Data and that was one of the first jobs I wrote for our internal dashboard.

I'm not going to go over how to create a new Kitto dashboard in this post. Kitto's readme covers everything for installation and creating dashboards. After setting up the Kitto dashboard, you'll need to add HTTPoison and Poison as dependencies to the application.

Getting the Data

Rather than putting our API request logic into the job file, I recommend to abstract as much logic as possible out into a separate module. That way, we can use the same logic across jobs or, if the module grows large enough, we can pull it out into it's own package that we include as a dependency.

To talk to the JIRA API you'll need to store the server, username and password somewhere. How you load your secrets is up to you. For my sample, I'll load them into the Application's environment as :jira_url, :jira_username and :jira_password.

Now that we have our secrets somewhere, let's start by creating our module which will handle communicating with the JIRA API at lib/apis/jira.ex. I want to ultimately display two widgets on my dashboard with the following information: 1) a count of open bugs are there across all projects and 2) a list of blocker and critical issues.

defmodule APIs.Jira do  
  def issues(filter) do
  end

  def count(issues) do
  end

  defp username, do: Application.get_env(:sample_dashboard, :jira_username)
  defp password, do: Application.get_env(:sample_dashboard, :jira_password)
  defp url, do: Application.get_env(:sample_dashbaord, :jira_url)

  defp authentication, do: ["Authorization": "Basic " <> Base.encode64("#{username}:#{password}")]
end  

As you can see from the arguments, count requires a list of issues, so let's build out our issues method first.

def issues(filter) do  
  jql = URI.encode "filter=" <> to_string(filter) <> "+order+by+priority+DESC,updated+ASC"
  url = URI.parse url <> "/rest/api/2/search?maxResults=25&jql=" <> jql
  HTTPoison.get!(url, authentication).body
  |> Poison.decode!
end  

Let's break down what's happening. First off, the value that we pass into the issues method is the numeric ID of a JIRA filter that you'll need to define on your JIRA instance.

JIRA Filter ID

The jql variable sets what we want from JIRAs issue API: the filter we are asking for, ordered by priority descending (highest priority first) and then by when issues were created (oldest first). JIRA's Confluence has a ton more options for configuring our response object, but I won't get into that right now. After we compose our jql query, we build the URL we need to request. I have maxResults=25 simply to have a reasonable limit to how many tickets will be returned. Dashboards tend to be displayed on a TV or something else where you won't have access to the scrollbar.

After we build the URL we are going to request, we send a GET request to JIRA passing our authentication information as a header. From there, we decode the response JSON with Poison and return the result.

It probably would be good to handle any errors such as rate limiting, authentication failure or some other error from JIRA, but I'm not going to build that out here to keep the example short.

The issue count method is easy, since JIRA returns the count of issues as a field in the response JSON. Instead of querying the API again, I'll just expect the developer using my module to pass an existing return object to the count method:

def count(issues) do  
  issues["total"]
end  

One last method I'm going to add to APIs.Jira is to format an issue for the dashboard:

def issue_for_dashboard(issue) do  
  %{label: issue["fields"]["summary"], value: issue["key"]}
end  

We now have a module for interacting with JIRA to get everything we need for our dashboard. To test you can drop into iex and query the JIRA API:

iex> APIs.Jira.issues(10300)  
%{...}
iex> APIs.Jira.issues(10300) |> APIs.Jira.count  
32  

With the module built, the job is easy. We'll create our job at jobs/jira.exs:

use Kitto.Job.DSL

filters = [  
  "high_priority": "10300", # <= All open issues with critical or higher priority
  "bugs": "10100" # <= All open issues that are bugs
]

Our job isn't going to do anything yet. All we did was create a list to save our filter IDs as something easy to understand when a developer is just looking at code and not JIRA. We could write our jobs individually and only broadcast the count of bugs and the list of high priority issues, but I think it makes more sense to just broadcast both the list and count of both bugs and high priority issues:

Enum.each(filters, fn ({name, filter}) ->  
  job name, every: {5, :minutes} do
    issues = APIs.Jira.issues(filter)

    list = %{items: issues["issues"] |> Enum.map(&APIs.Jira.issue_for_dashboard/1)}
    count = %{value: APIs.Jira.count(issues)}

    broadcast! to_string(name) <> "_list", list
    broadcast! to_string(name) <> "_count", count
  end
end))  

By looping through the filters we now have, available to the widgets in our dashboard the following data sources:

  • highprioritylist
  • highprioritycount
  • bugs_list
  • bugs_count

Data source names are based on what you pass to the first argument of broadcast!.

Furthermore, to add more filters in the future, we only need to add them to our filters list.

Showing the Data

Now that our data is streaming to the dashboard, just add the widgets. Create a new dashboard at dashboards/issues.html.eex:

<div class="gridster">  
  <ul></ul>
</div>  

To list out high priority issues, add a li tag to the list like the following:

<li data-row="1" data-col="1" data-sizex="2" data-sizey="2">  
  <div class="widget-jira"
       data-source="high_priority_list"
       data-widget="List"
       data-unordered="true"
       data-title="High Priority Issues"
       data-moreinfo="Blocker and Critical"></div>
</li>  

This creates a widget in the first row, first column that is two units tall and two units wide. The div has a number of options for setting up our widget. data-source sets the source for the data which we will feed into the widget. That's the broadcasted topic from our job, and data-widget is the actual widget that we want to use. You can look in the widgets directory for all of the built-in widgets for Kitto. The other options are specific to the List widget itself.

To add a count of bugs, we'll use the built-in Number widget:

<li data-row="1" data-col="3" data-sizex="1" data-sizey="1">  
  <div class="widget-jira"
       data-source="bugs_count"
       data-widget="Number"
       data-title="Open Bugs"></div>
</li>  

Like before, data-source contains the name given to broadcast! when we broadcast our data and data-widget is the widget we want to use.

Going Further

We now have a dashboard that we can put up on a TV in our office and engineers can monitor open issues by simply looking up from their computers. Everything works pretty well, but there's a few improvements that can be made. We could show the priority of issues in the issue list widget. It would also be nice if we could react to issues coming in. For example, if a new blocker is created in JIRA, maybe we want to change the background of issue list to red. In the next post, I'll walk through how we can create our own widgets with Kitto's React implementation.

The source for the sample dashboard that I built can be found on GitHub, or if you want to see how Cage Data uses Kitto, take a look at our ExDashboard repo.

Dave Long

Read more posts by this author.

Subscribe to Dave Long

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!