Contract Testing

Why do it?

Contract-based unit tests tell us if API endpoint connections are active and working properly.

With good contract tests in place, we should be reasonably assured that our code doesn't break upstream/downstream dependencies because of other teams unexpectedly changing their APIs.

ex. a good reason to adopt contract tests would be if you had a service that served data to many different teams in your software organization. Say we had 5 different teams that consumed data from our interal API. It would be hard for us to know and keep track of which data each client expects. With contract tests, we can be kept aware of what each client needs, and know immediately if changes to our API will affect anyone who depends on it. This will save us in situations where say, we upgrade our API to v2 and break a few consumers (out of potentially dozens), because for instance the shape of the data changed.

A good contract test for a deployed service gives us confidence that:

The API under test understands the request made of it
The API under test can send the response expected of it
The endpoints are accessible
The provider and the consumer have a working connection (assuming we are testing against a deployed service)
An expected integration is working

The whole point of contract testing is to enable independent releases of components and continuous delivery (Private).

Coupling this way prevents this and scales poorly as you add more teams and components.

What is it?

Contract testing is an alternative to e2e integrated testing.

Contract testing is a methodology for ensuring that two separate systems (such as two microservices) are compatible and are able to communicate with one other

interactions exchanged between each service are captured, storing them in a contract, which can then be used to verify that both parties adhere to it.
- to engage in contract testing is to interrogate a deployed (or mocked) service endpoint to get information back and compare to the expected values.
contract testing requires both parties to come to a consensus on the allowed set of interactions, and allowing for evolution over time.

The difference between contract testing and other methods that aim to achieve the same thing is...

each system is able to be tested independently from the other
the contract is generated by the code itself, meaning the contract is always kept up to date with reality.

Contract testing is more relevant with a microservice architecture, given its distributed nature.

Contract tests exist to help with integration testing.

In a distributed system, integration testing is a process that helps us validate that the various moving parts that communicate remotely - things like microservices, web applications and mobile applications - all work together cohesively.

How does it work?

When writing contract tests, you have to ask yourself: am I the consumer or the provider?

if consumer, then you need to look at only the endpoints you consume
if provider, then you need to look at all your endpoints

The contract must be available to all parties, which serves as our API documentation.

The consumer (client) makes a request to a mock provider, and in turn receives a response. The request and response are both placed into a contract:
```
request: GET /users/123
response: 200 OK {"name": "Mary"}
```
A simulated consumer (provided by a tool like Pact) then replays each request against the real provider and compares the actual and expected responses. If they match, we've verified that simulated applications should behave as real applications.

Consumer/Provider

The consumer initiates the HTTP request.

The consumer side bargains to keep the mock provider aligned with the contract, while the producer does everything to follow the contract.

because of this, integration-like tests can be run without requiring the actual service provider to be available.

For applications that use...

HTTP
- the consumer is always the application that initiates the HTTP request (eg. the web front end), regardless of the direction of data flow.
- the provider is the application that returns the response.
Queues
- the consumer is the application that reads the message from the queue.
- the provider (also called producer) is the application that writes the messages to the queue.

The provider is often called the service

Implementation

Pact tests exist as unit tests in the source code of the consumer.

ex. if we are testing an OrderApiClient class, whose responsibility is to make an http request, then this would be the unit to apply the contract test to.

In the unit test, we set up a few things:

The mock provider service, specifying a port at which it will run.
- The mock service will respond to the client's queries over HTTP as if it were the real OrderApi.
The mock provider object, which allows us to set expectations.

Example provider:

// Setup Pact
const provider = new Pact({
  port: 1234,
  log: path.resolve(process.cwd(), "logs", "pact.log"),
  dir: path.resolve(process.cwd(), "pacts"),
  consumer: "OrderWeb",
  provider: "OrderApi"
});

// Start the mock service!
await provider.setup()

Value of contract testing

Contract tests generally have the opposite properties to E2E-integrated tests:

They run fast, because they don't need to talk to multiple systems.
They are are easier to maintain: you don't need to understand the entire ecosystem to write your tests.
They are easy to debug and fix, because the problem is only ever in the component you're testing - so you generally get a line number or a specific API endpoint that is failing.
They are repeatable
They scale. Because each component can be independently tested, build pipelines don't increase linearly / exponentially in time
They uncover bugs locally. Contract tests can and should run on developer machines prior to pushing code.
Contract testing keeps the API producers in sync with the consumers.

Mostly used for:

Detecting irregularities in a consumer workflow
Detecting any service configuration defects
Keeping the connections safe even when the producer changes any service configuration

In contract testing for message queues, for the purpose of verifying a contract, the underlying message queue is mostly irrelevant. As long as you can ensure that the message producer is generating a correctly formatted message, and the consumer is able to consume it, you don’t need an actual message queue.

Contract testing types

Consumer-driven

This means the consumer oversees contract creation.

Why would a consumer take charge of contract creation?

In most cases, consumers only care about a certain subset of data. They use a certain set of fields, they expect certain status codes etc. All of these details gets put into a contract that is then agreed to by the provider. Now, the provider has a good idea of what its consumers actually need, and now the provider can be free to develop its API without fear of breaking things for its consumer, since it now needs to make sure that it still satisfies the contracts laid out by the consumers.

Producer-driven

Rarely used.

Testing Broker

Once we have our contract tests in place, we need somewhere to put them. They don't do much good packaged alongside the repo whose code they are testing. Instead, we need it somewhere where it can be used to: