I'm a big believer that the order you read a set of books can matter more than the books you read. When different ideas and perspectives mix into a savory synapse soup -- good things happen. That was my last 10 days. I've gotten to talk to some amazing people about the history of Swagger/OpenAPI (history matters), why API Specifications matter, and the problems we face building and scaling tools across the wider API Community.

My brain was overloading, so I sat down and did a talk:

Watch the Talk#

According to Gallup, my top strength is Learner. That’s probably the reason I got into software development over two decades ago. Technology changes so quickly, there’s always something new to learn.

Java was the first real programming language I used back then. At the time, it had been around for about three years and was still very immature. But all this time, there’s something that I have always remembered and appreciated about Java: its exceptional API documentation. It was easy to understand, the examples were useful, and it was usually all I needed to solve any problem.

Good API documentation shouldn’t be exceptional, but it is. And it’s important.

Design First is a practice that encourages us to design any API change before writing the code code. The goal is to do the upfront work to produce better API changes. Design First is normally done by modifying an OpenAPI document first and then using that change as guidance on what to develop.

But Design First is a small piece to the larger API development process. There are more practices and patterns we need in order to make design a cornerstone to building good APIs.

When taken alone, Design First seems to imply that the process is linear—first there's the design, then development, then deployment. But design doesn't work this way. Design is something we're always doing, even when writing code.

What then are the practices we need to add into our Design First workflows that makes Design First work?

I’m new around here, and my first post is about something I really love… code that makes life easier!

To begin with Optic's CLI is ridiculously easy to start using within your application, you simply head to the root of your directory and run api init, configure your .optic.yaml to wrap your application start command, and tada you’ve started gathering data about your API endpoints by just using them. Super simple and great right? To look at the full process and implement it yourself, please check out our documentation.

But what happens when you can’t do that? Plenty of instances exist where it’s very complex to work in a team and wrap the command that starts your application. What happens when you use feature flags on that command, or you have to make operating system specific changes, or your API testing is done entirely internally and doesn't hit the network?

Code reviews are an essential part of any high-quality software development process. A thorough code review can prevent bugs and regressions from slipping into production, improve code quality and consistency, and ensure your team knows about important changes before they go live.

While looking at code can help you find flaws in your logic, it's notoriously hard to trace changes through your application to see how they might actually affect your API.

Application Programming Interface (API) testing is more complicated than testing a single-host application. Single-host applications can often be covered by unit and integration tests, but an API functions only if there is a both requester and a respondent. To fully test your API, you have to think about testing the application from both of these perspectives.

Testing the API from the perspective of the requester means testing authentication, data accuracy, data formatting, overall consistency, and relevance of available documentation. The respondent needs unit tests for the components it uses and the data transformations it makes. It also needs to ensure it can both handle properly formulated requests and reject requests that are malformed or maliciously formed. You also need to test the connection to the data layer to ensure that whatever source the API draws from is accessible and understandable. This article will go into more detail about the tools and best methods for effectively testing your API.

I've worked on two different products that provided API design guide tooling. The idea for both was the same—provide a tool that helps companies design consistent APIs and helps communicate good design patterns to all of their developers.

It's common for a company to take inventory of their APIs and see lots of inconsistency. One API uses camel case while another uses snake case. One puts the version in the URL, another doesn't. These companies try to address this by putting together their own set of API guidelines and adopting tools that help them apply these guidelines to their APIs during the development process. This is where our design guide tooling came in.

Design guide tooling—which goes by names like style guide, standardization, governance, and linting—allows people to express guidelines in code and apply those guidelines to an OpenAPI document. Imagine a rule that makes sure every API endpoint requires authentication and returns a 403 when not provided. When a rule like this fails, the tools can prevent the teams from merging the code until they resolve the issue. It’s a big win for scaling an API strategy across many teams.

But there's always been a nagging feeling something is missing.

Once our APIs are deployed in a real environment like staging or production, traffic races in from all directions, interacting with the API in the ways we expect, and in ways we could never anticipate. These 'surprises' can teach us things and help us build better designed and more robust APIs.

Physicists smash particles into one another in giant particle accelerators looking for surprising patterns, and for new information that questions their assumptions about reality.

When we put our APIs in the path of real traffic, reality smashes into our API specifications, our well-landscaped-happy-paths, and our assumptions. We learn, revise, and improve our work -- hopefully.

There is so much to be learned from looking at our API's actual behavior and usage in the real-world. But are we really looking? Can we tell if our APIs are working as expected from the logs we collect today?

Users have long asked for options to run Optic in real environments to get more observability into their API's real behavior. We have even observed teams using the local-cli in their real environments without official tooling or support from the main project. Now you can join the beta and start monitoring your deployed APIs with Optic. You can also check out the docs to learn more about how the integrations work.

To everyone who has tried Optic, contributed code, told us your stories, written about the project, and helped us chart our course — we (the maintainers) want to say thank you! We know it’s been a crazy year and API tools are not the most important happening in the world right now. Still you’ve made the time for our open source project, and in doing so, helped keep us all employed. We can’t thank you enough for giving our team the opportunity to work on these problems. Thank you, sincerely.

Today I’m really excited to show off a new ‘primitive’ for API tooling that Optic has been working on. Everyone, meet spec coverage.

API spec coverage measures how much of your API spec was covered by traffic, and like everything else we do, it’s open source!

The Optic CLI now produces a coverage report for any of your test scripts, Postman collections, or even a bunch of manual CURLs to a server.

Optic helps teams write and maintain their first OpenAPI specifications. You don't need to get your team on-board to learn OpenAPI or worry about maintaining 10k line YAML files -- Optic takes care of all of that. Optic manages this process whether your development environment is running locally on your machine out in the Cloud, or somewhere in between. No matter where you develop and test, you have confidence that every change to your API is reviewed, approved, and documented before it's released. Optic can work at any interface in your project, even with microservices.

Optic helps teams write and maintain their first OpenAPI specifications. You don't need to get your team on-board to learn OpenAPI or worry about maintaining 10k line YAML files -- Optic takes care of all of that. Optic manages this process whether your development environment is running locally on your machine or out in the Cloud. No matter where you develop and test, you have confidence that every change to your API is reviewed, approved, and documented before it's released. Your documentation can provide details for your procedures, and assist in your governance goals.

Time after time I’ve encountered teams who understand the importance of API Change Management, but do not have the mechanisms in place to understand how their APIs change over time.

To me, this has always felt like it should in the bucket of ‘solved problems’. We have amazing change management tools for our code and our infrastructure, but not the APIs that are run by that code, on that infrastructure.

Now Available: The Optic GitBot#

Imagine a GitBot that made it easy for teams to understand how/when their APIs change. Enter the Optic GitBot, which automatically adds an API Changelog to Pull Requests within your project. With the bot installed, developers understand how each PR affects their API contract and can use these insights during code review.

When you open a PR, Optic diffs the specifications in your PR branch against the base branch to produce a semantic changelog, then comments on the PR. It's that easy.

This past summer, I worked with the Optic team as a software engineering intern. While it's the first time I've worked in an all-remote environment, I had an amazing experience and learned a lot from everyone here.

I've gotten to poke around the entire codebase, talk to all the amazing team members here, and work on everything from marketing to devops to even working on new products.

Optic is a Git-like version control system for API contracts. With Optic, maintaining an accurate API contract is as easy as reviewing diffs and approving suggested changes. This proves itself to be an incredibly valuable tool, and I spent a lot of this summer using Optic hands on.

Optic helps teams write and maintain their first OpenAPI specifications. You don't need to get your team on-board to learn OpenAPI or worry about maintaining 10k line YAML files -- Optic takes care of all of that. Optic manages this process whether your development environment is running locally on your machine or out in the Cloud. No matter where you develop and test, you have confidence that every change to your API is reviewed, approved, and documented before it's released.

Part 1 of our Changelog Specification Series#

We've had several other API tools engage us about using the Optic Changelog Spec in their products so we're going back to the basics with a repost from our July 8th, 2019 Problems with API Specifications. This is where Dev and I first shared Changelogs specs publicly :) Enjoy!

It's been over a year since Dev first came up with the idea to apply CQRS / Event Sourcing to modeling API behavior, and the idea has proven to be very robust. Since publishing the original form of this article, Optic has pivoted from a GUI for API contracts, to a code generator, then to an API spec generator, and finally to the "Git for APIs" tool that it is today. Don't worry -- you've shown us it's working, and we're sticking with it. No big pivots ahead.

Optic helps teams write and maintain their first OpenAPI specifications. You don't need to get your team on-board to learn OpenAPI or worry about maintaining 10k line YAML files -- Optic takes care of all of that. This is great, because you have your OpenAPI specification, you can tap into the thousands of community-built tools that make building, publishing and testing your APIs easier.

Optic Scripts#

We're proud to announce a new capability in Optic, Scripts, which makes it easy to use the OpenAPI spec Optic gives you with a variety of other tools. Scripts help you share the information Optic knows about your APIs with other systems. Script definitions live inside your optic.yml file and are invoked with the api scripts command. The Optic Scripts feature is included when you add Optic to your API.

This week I had the opportunity to talk about Optic on the Software Engineering Daily Podcast. Jeff and I covered a lot of ground -- everything from why API Change Management Matters to how we see APIs intersecting with the low-code movement.

Take a listen or browse the transcript! A big thanks to Jeff for giving our project airtime! If you don't already, please subscribe wherever you get your podcasts.

Episode: https://softwareengineeringdaily.com/2020/09/02/api-change-management-with-aidan-cunniffe/

Transcript: https://softwareengineeringdaily.com/wp-content/uploads/2020/08/SED1127-Optic-API-Change-Management.pdf

The following post is a conversation with Victor, the author and maintainer of wgcf

With the release of Optic 8.2 a few weeks ago, major improvements were made to both performance, and the reliability of the diff. Version 8.2 is noticeably faster, but the performance still breaks down when a large number of endpoints have been documented or when diffing large JSON Bodies (multiple MBs). We had hoped the improvements in 8.2 would cary us through the end of the year, but some large teams trying to adopt Optic are still blocked by poor performance -- often having to wait several seconds between critical actions.