WHAT IS DOCKET BASED CHOREOGRAPHY?

DEFINITIONS

Docket Service - A service which has the responsibility of keeping track of a list of tasks or states. A docket service also has the task of publishing major state change events like "Application Complete" or "Info Needed". A docket is similar to a shopping list.
Engine Service - A service that contains logic and functionality and updates a docket with information.
Choreography - Asynchronous approach to workflow. Interested services respond to state changes in real time rather than having a central service controlling "what needs to happen next". Choreographed systems are built to be reactive and respond to events as they happen.

BENEFITS

Simple and extremely flexible
Separation of concerns - Services do not need to know what happens upstream or downstream.
Minimizes need for distributed transactions.
Cross-cutting concerns such as observability, monitoring, analytics, notifications can be plug and play.
Sagas and Compensating Events are not necessary.
PII and sensitive information is kept out of the messages, making it easier to manage regulated data and data at rest.
Highly parallel
Very easy to find the state of a particular process

CORE TENANTS

Small messages published when state of a docket changes - Messages contain an Id and timestamp. Usually nothing more.
Pub/Sub model where topics represent the state change. "Application Saved", "Payment Received" or "Information Needed"
One message type per topic/queue rather than one topic with multiple message types per.
Docket Services and Engines follow DDD constructs.
State changes are pushed by services and responded to by any service that "cares about" that change.
Dockets keep track of a check list of information that needs to happen before it goes to the next step. For example, an Credit Application can not be marked "complete" until a credit report, address validation, fraud check, ID validation have all been completed.
Synchronous APIs are exposed by Docket services to allow for updating of the "check list" as well as for fetching details.

EXAMPLE

SITUATION

Mythical company, We-B-Food is a competitor to UberEats, DoorDash and GrubHub. We-B-Food matches hungry people with restaurants and delivery drivers. The on-boarding of new drivers is a surprisingly complex process. We-B-Food requires that all drivers must have valid driver's licenses, insurance that meets specific standards, a vehicle that has met safety standards, identity verified, credit and background checks passed.

When an application to be a driver is submitted, 5 things need to happen before an approval decision can be made. The completion order of these tasks is not important at the moment so we can assume they can be done in parallel.

In this case, we design the following components:

The Driver Application Docket Service to keep track of the application data as well as the check list of "things to be done" before the application can change its state to "Application Complete"
An Insurance Engine which manages and provides insurance information for a driver
A Credit and Background check engine
Safety Service - Engine managing details around car safety inspection
DMV Service - Engine managing interaction with various DMV systems
ID Service - Engine that manages the verification of individuals

THE CHOREOGRAPHY

STEP ONE

Wanna be driver enters personal information into web page which inserts the information into the Driver Application Docket

STEP TWO

Application Docket validates that it has all the information it can expect from the Wanna Be Driver, but it doesn't have all of the boxes check in his docket.

STEP THREE

Application Docket publishes small message consisting of {id|timestamp} to the "App Info Needed Topic" because the Application Docket Service knows what is needed, but it doesn't know how / shouldn't be responsible for figuring out how to get it.

STEP FOUR

All 5 Engines (Services) who care about "App Info Needed" get the message, triggering them to each do work.

NEXT STEP(S) - REPEAT

Credit Service decides it needs more information about the Wanna be driver to complete its task to it invokes the Driver Application Docket API synchronously (and securely) to get SS# of the applicant.
DMV Service fetches DL info securely through the Driver Application APIs
All other services do the same thing. Effectively saying "do I have enough info to do my job? No? Ok, I'll fetch it since I have the ID of the Application"

FOLLOWING STEPS (RANDOM ORDER)

Insurance Engine gets its result first because we have an automated interface with Liberty Mutual. Insurance Engine stores details of the findings and the updates the Docket Service with a small message saying "I'm done! Got the results!"
DAD Service (Driver Application Docket) says: "Cool, Insurance came back. I'll check that box."
DAD Service looks to see if all boxes checked. Nope. Do nothing yet. Still waiting for others.
Credit Report Engine gets its result first because we have an automated interface with Equifax. Credit Engine stores details of the findings and the updates the Docket Service with a small message saying "I'm done! Got the results!"
DAD Service (Driver Application Docket) says: "Cool, Insurance came back. I'll check that box."
DAD Service looks to see if all boxes checked. Nope. Do nothing yet. Still waiting for others. ...
Finally the ID Engine gets info and updates Docket that results are in.
DAD Service says "Cool, ID came back! Box Checked!"
DAD Service looks to see if all boxes checked. YES they are all checked. I'm going to post a small message to the "Application Complete" topic!

At this point the DAD Service is done. All boxes checked. State of this particular Application is "Complete" which triggers something downstream. (Probably a Decisioning service, Notification Service, Analytics...) But the DAD Service really doesn't care.

FUTURE

In the future, we may decide to swap out Equifax for Experian. Easy change. Just a new consumer. DAD is not impacted. Or we decide to run all 3 credit bureaus. Still no biggie, Equifax, Trans Union and Experian all become subscribers to the same queue. Again no big deal.
Also, maybe today the ID service is manual. A notarized form is mailed in, which takes forever. It is still managed by the ID service, but we can add a new service that offers an instant biometric solution. Sweet, suddenly we have two services able to "check the id verification complete box on the docket" DAD doesn't care who gets done first or even how many services are working on it. We were able to add a new service and dramatically improve the capability of the system with out touching anything. That is powerful!