We’re gonna talk about API history and our ways of working and how that has changed a little bit over time. First, a little bit about PEXA. If you guys haven’t heard of us, if you have bought or sold a house in the last few years, chances are, that transaction has gone through PEXA. We’ve spent around 10 years really trying to bring that property exchange world into the digital world. Looking at creating less paperwork, reducing process time and reducing that margin for error. So to put that into perspective, only a few years ago, folks would actually meet at a time and place in a room, swap checks, sign bits of paper, and that’s how you sold. And these folks would go from room to room throughout the day selling properties for people. These days, through the PEXA platform, you can do that at home in the comfort of your COVID couch, no need to go to a salesman room. So that’s fantastic! We have been building this for around 10 years so anyone working in software for that amount of time will probably recognise the architecture or something around about this age, e.g close coupling of server and UI componentry. And of course, we have APIs, but perhaps not in the early days, not how we think of APIs today. So 2014 saw our first real public API come about and this API was really designed with a single customer in mind. It was driven by what the customer’s needs were with a focus on getting those integrators on board and getting our platform rolling and getting that critical mass of customers. 2017 is when we introduced Swagger into our ecosystem so that we became much better engineer friendly documentation. Moving the needle on that developer experience, in 2019, the developer portal came about, and this was really trying to shift that burden of support off of our teams, and back on to the user, therefore moving towards a more of a self service model, exposing our swagger in a portal. You’d find much richer documentation, code samples, code starter packs, and great resources for people to get on board and start the journey. And finally in 2020, that’s when our API first journey really began.
About our ways of working: Often when you start on something like an API first journey, you have teams that are already executing really, really well. So we have these existing cross functional delivery-focused teams, they’re working from web to service to database. I suppose you could call them sort of BAU.
When we got to an API mindset in the early days, what we created was what I like to call the Tiger Team. That Tiger team is distinct from those stream-aligned teams, and, as a blatant “Team Topologies” reference (Team Topologies: Organizing Business and Technology Teams for Fast Flow by Matthew Skelton and Manuel Pais. TN), we wanted to protect those stream-aligned teams from the API work. The dedicated API team is a formation of experts to drive the outcome and they can keep as close to the customer as possible. When we introduce these target teams, with these functionally aligned teams, they’re really there to work independently of that delivery pressure so the stream-aligned teams can go ahead and just do what they do. The dedicated API team trailblaze those APIs and can do that distinctively of delivery pressure, which can be quite compelling. From a short term perspective, that does fantastic and that team can really do a lot of great work. But ultimately, as we get bigger and bigger, and as we have more and more APIs, across many domains, we get this problem of those teams relying on the stream-aligned teams or those domain driven teams, to really figure out that domain, so they can expose those APIs. And we also run into this problem of cognitive load. This is where we really see how these Tiger teams can start to take a bit of a hit. So what I mean by that is that the domain knowledge becomes a bit of a problem for us. So this functional team is targeting needs to be able to expose those APIs, right? So they have to be across everything and there’s a lot of context switching. Anyone in the room who’s ever been asked to look at domain A then switch to domain B, and suddenly there’s a defect on domain C, you know the pain of context switching, and the time it takes to ramp up again… Prioritisation can become an issue again, we’ve got a new API for domain A, we’ve got an enhancement for domain B, and we’ve got a bug in domain C, how do we juggle that as a single team, functional team? We do like to be as cross functional and as build and run as possible so support can start to become an issue as we reach that critical mass of APIs. So we onboard customer support, technical people onto our platform and the technical support that comes with that, and all that lands back on that team. Therefore, as our API ecosystem grows, these functional API centric teams start to break down. So what have we done to move forward with API first? Again, another blatant “Team Topologies” reference, the idea of this enabling team, they’re here to help the streamline teams acquire those missing capabilities (if you guys haven’t had a crack at the Team Topologies book, I highly recommend it, it’s a great way to start framing how you’re thinking about your teams and interactions). So the focus here for us is really around developer experience, documenting guidelines and guardrails. How do we ensure that the teams are on the right path at standardisation pace and above all really empowering and uplifting those teams? And that Learning and Development becomes a really big part of this team’s mandate? So how do we do that? Pairing the teams is a really great start. So we love to get in there and embed with these teams and make sure that we’re doing the right things. However, definitely beware the ivory tower. We don’t want this team to be just handing down standards and guardrails and ways of working without actually doing them themselves. So by that, I mean we need to get in there, show what good looks like, focus on this and focus on production APIs, as well as just the guardrails, guard browsing and guidelines. We also have zero tolerance for debt. When you have an enabling team or a centre of excellence that’s laying down these guardrails for other people to follow, we need to make sure there’s no debt at the very beginning, otherwise, as we federate out and as stream-aligned teams build more and more of this workup, it just accumulates and it’s really hard to rein back in. So on that, it’s important to deliver but we can’t be delivery centric. We do have to take a little bit more time to deliver, just to make sure we get things right. And when we federate out, we do the right thing. And to be honest, one of the lessons we learned is, perhaps we did throw a little bit too much work to the enabling team and we did ask them to deliver probably a little bit more than they should have. That did cause a little bit of tension. As an enabling team, your customers are the other teams. So, again, yes, we’re doing production APIs, but really, how about the developer experience? How do we uplift the Devs? How do we make sure that we’re doing the right thing, so the customers of the enabling team are the other teams? Ultimately, we want to get to a point where that functional team, that target team, can dissolve. By that, I don’t mean firing everyone on the team, I mean pivoting that team so that they’re less delivery centric and they’re looking at those asynchronous APIs that are coming in the future, streaming APIs events, more setting up other teams for success and less just punching out APIs.
As far as the enabling team is concerned, we went on a bit of a road show, and I guess we had three major focus points to try and get us on this API first journey. The first one was really that overall socialisation of API first and its benefits. So this is us presenting what our all hands socialising terminology is, and trying to get that broad messaging out across the business. Outside of that, we split up into probably a two prong sort of attack where we changed the views of product owners and BAs on how to think of APIs as a product, how to analyse APIs governance, and how that shapes up. And then our engineers are so much more hands-on, showcasing the DX developer experience, the tooling in our workshops, blogging, etc.
Important lesson for us, we’re pretty chuffed with the roadshow, we thought we did a pretty good job presenting and shaping that messaging depending on who was speaking to, affiliates versus devs, and so on. But just presenting and going on a roadshow does not equal uptake. And big lesson for us, we are only going to see the uptake once the rubber hits the road, so driving much, much harder down through product and delivery to ensure that this API first stuff was getting into people’s backlogs. Teams will just naturally do what they do, unless you put quite a lot of hard force on them. So next, we’re going to talk a bit about execution.
In general at PEXA, we do try to encourage a culture of documentation and reduce that keyman risk. So just over a year ago, we actually decided to introduce the RFC process, which is “request for comments”. This actually allows you to facilitate agreements on an approach implementation if you think about proposing any large changes, so the likes of Spotify, Facebook, GitHub, use this approach as well. So as you can guess, the first RFC that we actually introduced was API guidelines. One of the best ways to keep focus is to point back to ratify documentation. So once we actually publish the RFC, any changes must go through a change process. And this actually stops a lot of those inane conversations and any ad hoc meetings. The RFC process allows any engineer willing to contribute and have their say skin in the game, so to speak. I actually received feedback quite recently from a BA at PEXA, designing APIs for Salesforce integration and he actually said the guidance was a breeze to follow. So whether it was conventions, statuses, aerostructures, pagination, it was all there. And I would actually adopt a similar approach if I was looking at other interfaces such as events or GraphQL, for example. So it’s all about striking a balance of centralised governance and Federation, so this comes back to the maturity in your organisation so you just have to remain patient. Rome wasn’t built in a day! Next is “Dogfooding”, you guys might have heard about this before. Effectively, it is the practice of using one’s own products or services. We mentioned earlier the enabling team: the need to not only talk to talk but also to walk the walk. So I work quite heavily with this team to define best practices patterns tuning around APIs. They must be using the tools that they’re evangelising. So the team is actually able to iron out any issues with the tooling, pave the way for other delivery teams to get up to speed quite easily and efficiently and follow up with ongoing meetings, workshops, and slack channels to help empower the teams on our journey. This helped drive good behaviours or herding the cats, so to speak, and actually reduce any ambiguity. So as mentioned in the previous slide, we do encourage a culture of documentation at PEXA so the team is actually able to document the required steps along the API lifecycle, relevant blog posts and readmes. One of the lessons learned since the beginning of PEXA is that there used to be a bit of the “not invented here attitude” and usage of the status quo. But, over time, with the help of this enabling team, and by paving the way eating your own dog food, those perceptions changed and we actually noticed higher attraction and buying from the business and engineers. So in relation to the API first lifecycle, you can break it down into three main phases, which is design, implement, and validate. Design is the most crucial initial phase. This is your continuous discovery, ideation and problem solving. It’s about figuring out your domain, and who’s going to be consuming your APIs. So once you’ve decided that, you actually look at designing your API spec, and making sure it adheres to the API guidelines. We actually also have a development portal at PEXA for our external facing APIs. Along with internal stakeholders, we also provide a draft of our API spec to partners. This feedback is actually critical in validating that the API solves the problem it is supposed to solve, as well as help us understand the aspects of our API that needs improvement. By seeking this feedback, before we write any code, we can actually quickly iterate the API design and build a much better API. So it’s that saying of “measure twice, cut once”, so to speak. This approach gains a higher adoption by engaging partners early in the lifecycle. And finally, the design phase and beta testing, this is something that we’re going to look at in the future where we can identify any issues before public release. When we talk about product design, and UI, a lot of companies actually have design systems, component libraries use a lot of tools like Invision and Figma. So why can’t APIs be treated as first class citizens as well?
So then, in relation to the Implement phase, this is the fun part for engineers, so we actually use Open API generator quite heavily. This actually allows us to generate the service stubs off the open API spec. This helps engineers because it reduces a lot of the boilerplate code. They can then just start focusing on the actual business logic itself. We’re also actually currently evaluating API management platforms at the moment. We’re speaking with multiple vendors and once we actually agree on a product, we will be looking at adopting an API ops approach. Finally, when it comes to validate, there’s actually a really cool tool called Spectral from Stoplight. We can actually define our own custom rule set that aligns with the API guidelines. So as part of our CI pipeline, anytime that we push to change the open API spec, which is our source of truth, we can actually validate to make sure that aligns with our API guidelines. And then in relation to shifting testing left, we actually look at reducing that very heavy manual end to end testing. So we try and shift left and focus on integration testing APIs and using test containers.
Just wrapping up, some of the key lessons for us: those tiger and functional teams, they’re a great start and they can really get you going quite quickly and it’s a good sort of safe thing to do. But ultimately, once we get to that critical mass of APIs, we do need to start to embed and evangelise to succeed. That’s where we start to bring in those enabling teams to make sure that we can engage heavily with product and delivery to align and uplift the teams. That’s getting involved in Sprint Planning, making sure that we’re doing the right things, and then trying to change how our POs and PMs work. Living documentation is also really vital because we don’t want to retell things, we want to make things as open as possible for everyone to contribute. Skin in the game is incredibly important to make sure that developers feel that they can own a bit of it, and they can invoke that change. When you’re dealing with developers and dealing with change, that smooth developer experience is a must, you have to pave that road as some devs can be quite picky on how they work. We therefore need to make that DX as smooth as possible.
I suppose the mantra throughout API first journey is that we really do have to go slow to go fast, we have to take that step back and analyse what we want to do, where we want to head with APIs as a product. We’re not just doing one set of APIs anymore, we want to create this suite. So take a step back, go slow, tool out the team and then eventually, you’re going to hit that hockey stick of everyone getting involved. Next steps for us: looking at API first has really given us clarity on our domains (when I talk about API first, we’re really talking about contract first). That’s led us to think harder now on team structure and alignment. When you start talking about APIs as a product and looking at domains, then you start to look at your teams, and how we align correctly through domains that we have the right talent in the teams, the right functional bits in there. And it’s also led us to think hard on events. So similar to API First, we do have events here at PEXA. But taking a step back and looking at those domains, and looking at APIs as a product, makes us think about events as well, how we can do events in a similar way, which has been really interesting. And then of course, there’s that long tail of getting everyone up to speed on API First’s ways of working. We have had some success there, where I think we’ve embedded and evangelised really well, and in fact, in our last BRP session (big room planning session), we had a PO from a squad that we have embedded and evangelised, put their hand up and say, “Hey, we can help other squads”. So now, we’re starting to see that snowball effect of helping each other. We have our enablement team, but now we have other teams that have been enabled that are enabling other teams. That’s really fantastic to see. That means we’re definitely seeing a lot of traction now, which is great. And that just about wraps it up for us.
Q&A Section
Q: How is PEXA empowering the tech writers and contributors to be able to keep pace with the product development?
A: So with the RFC process, we have guardrails in place, when it comes to actually making changes to the RCS, so you just have to go through that public press process, and you have code owners specific to certain overseas as well to get notified. And it means then they can have a conversation and see where it fits within the guidelines, and whether it actually aligns with the business values at PEXA. And so that’s the process that we’ve adopted.
Q: The approach of a product owner is more of a business centric model traditionally, how key and critical has the product owners been in how much they need to understand from a technical viewpoint, or they are really just focused on the business side of things and what they want the API to achieve in terms of business outcomes?
A: We have had to do a bit of work helping our product owners and product managers understand APIs and I think APIs are a true product now, right? We do need our POs to understand what APIs are, how to craft the domain and what we want to expose out of our domain by APIs. I think there is a focus of learning and development for us to make sure that we empower our peers and PMs to understand that.
Q: Has becoming more API-led seen real world improvements in the business processes around conveyancing and the clearancing when it comes to settlements between financial institutions?
A: Absolutely. We’re seeing more and more people wanting that API channel. So making sure we have these beautiful composable APIs that integrators can build upon, is incredibly important. And we’re setting more and more that one foot for API so that businesses can execute that much faster.
Dean Baker, Head of Engineering and Shane Lee, Staff Engineer, are from PEXA and talk about modernising development using an API first based approach and the lessons they have learned within PEXA.
