Over the last ten weeks, I’ve been part of a small team that’s been building new tools to help us manage user needs better. You may have seen Lisa’s post about the user needs project back in October. I’m going to go into a bit more detail here about how we built it.
GDS has built APIs into the heart of many of our projects, but in this case we decided to start out with entirely separate API and user interface components. At the centre is the Need API - an internal read and write JSON API service which can be used by any other application in our stack. Above it sits Maslow, our front-end tool for browsing, creating and updating user needs.
Building these as separate pieces felt the right decision to us for a few reasons. The user needs project needs strong integration across our other apps and publishing services in order to be useful, so right from the start we wanted to build a service that could be used easily by other teams. We also wanted to acknowledge that Maslow is not the single point of creation for user needs. In some cases, it makes sense for user needs to be created elsewhere. For example, the Department & Policy publishing tools are a better place for user needs about detailed guidance to be created by editors in other government organisations.
Building a good API was critical to what we wanted to do, so by building Maslow as a client we forced ourselves to think hard about its design early on in the development process. We were ‘dogfooding’ our own API, which encouraged us to iterate it whenever we ran into something which wasn’t quite right. It also means that other teams can look at and reuse the API client libraries and implementation code we wrote for Maslow.
Of course, building a separate API has its disadvantages too. The biggest impact for our team was that the initial progress was slower than we’re used to when building a product. There’s also a higher cost to build new features, as you have to duplicate quite a bit of the development effort across each app. Performing end-to-end testing is significantly more complex, as is keeping track of who makes changes to needs. Despite this, the benefits of the API-first approach outweighed the drawbacks for us and we found that, once we’d done a lot of the groundwork, subsequent features and updates became significantly easier and faster to build.
I think we’ve learned a few things from the project. First of all, we found that we set up the same validation constraints in both the Need API and Maslow, as we were reluctant to make requests to the API that we knew in advance would fail. In reality, the duplication here just feels a bit cumbersome and the pattern wouldn’t scale well in a situation with many clients.
When building the Need API client in Maslow, we decided to roll our own ORM-like interface using ActiveModel, directly implementing our own API adapters library. Over time, this resulted in a complex class which was doing things typically abstracted away from a model - for example manually making the distinction in which attributes can be assigned when creating a need instead of loading an existing need. This is illustrated in a simplified example below.
class Need
extend ActiveModel::Naming
attr_reader :id
attr_accessor :role, :goal, :benefit
def initialize(id, exists=false)
need = need_api_client.need(id)
if exists
assign_protected_attributes(need)
end
assign_filtered_attributes(need)
end
end
Libraries like Her or Active Resource are good contenders to abstract away some of this additional behaviour by providing common bindings to any RESTful API, and using one of these libraries would likely simplify a lot of our client code.
Finally, I think this project has made us more aware of the web of dependencies around our services. For example, the Need API already relies on an API in the Departments & Policy publishing tools to keep the list of organisations up-to-date, and Maslow relies on the Content API to show which content which meets a need. We had to carefully consider which requests could fail gracefully, and which requests were critical to what the user asked for. In this case, we made Maslow resilient to a Content API failure, but not a failure in the Need API. Based on our existing libraries, implementing this was straightforward:
class Need
def artefacts
content_api.for_need(@id)
rescue GdsApi::BaseError
[]
end
end
We haven’t taken a public by default approach to the API itself, because we’re capturing and making decisions about organisation user needs, which may or may not be in scope for GOV.UK. However, in the future we’re aiming to open some of the API up to the public, and display more information about each user need on GOV.UK.
If work like this sounds good for you, take a look at Working for GDS - we're usually in search of talented people to come and join the team.
You can follow Jordan on twitter, sign up now for email updates from this blog or subscribe to the feed.
1 comment
Comment by Sue posted on
What is API?? Shouldn't you explain at the beginning that it means application programming interface?