Webhook Implementation Best Practices

Prompt response to webhooks

When you receive our webhook events, please respond to the request with 200 OK as soon as possible. Your response time and response status will be kept monitoring. If your endpoints are deteriorating ( slow response, too many 500 Internal server error), we might turn off your endpoint automatically.

Missing Webhook & reconcilliation job

Generally speaking, webhook is a very reliable system. However, it is recommended that your app should not rely entirely on receiving data from webhooks, especially if your app and use cases require to maintaining 100% data integrity and always stay synchronized with SHOPLINE events. Because the delivery of webhooks is not always guaranteed due to various reasons. For example, network problems and upstream services downtime.

In this case, the app should implement resampling jobs to fetch data from us from time to time, so as to improve the stability of your app.

Our OpenAPI supports queries for a specific timeframe, like created_after, created_before, updated_after, updated_befored, and in another syntax created_at=lt:{timestamp} etc. We recommend you make use of it.

Understand why missing webhook may happen

Understanding the reasons is crucial for effectively managing and mitigating their impact. Below are some reasons and examples for reference.

General Considerations if reconciliation job needed

Below are some general rules of thumb to consider when deciding if a reconciliation job is needed

1. Data sensitivity & Impact evaluation
   1. Highly sensitive data, particularly data involving financial impacts like transactions (which could lead to financial loss), usually warrants more stringent reconciliation measures
   2. Evaluate the impact of potential missing data or desynchronization on your application's functionality, as well as its potential impact on the business and customer experience
2. Resource Allocation: Consider the resources needed for implementing and maintaining reconciliation jobs. For some use cases, the effort may outweigh the benefits

In summary, whether to implement reconciliation jobs for handling missing data should be determined based on the criticality of data accuracy and timeliness in your specific use case.

📘

Example on app use cases that require data 100% stay synchronised

Scenario: SHOPLINE stores selling shirts with an integrated external accounting application

(example for illustration the concept only)

In this usecase, App think it is crucial for keeping app's data 100% in sync with the store. The end-of-day balance computations must be accurate. Reconciliation jobs will be vital under this assumption.

How to do?

1. Schedule a reconciliation job to run regularly / shortly before your app need to compute balances
2. Using the App token to request corresponding APIs to fetch and validate all purchase information
3. Update the records in your accounting application if there is any missing pieces

Below are some more examples for demonstration the decision process. Please be aware that App should review and justify base on their own situations.

---

Understanding Webhook Sequence & Delays

Webhook Sequence

Sequence of triggering webhook events is not the same as the sequence of receiving the webhook events. For example, a set of webhook events are triggered in the following sequence: A -> B -> C, you might receive the webhook events in the following sequence: B -> A -> C.

Webhooks delayed

Sometimes you might find delays in receiving our webhook events. Yet, webhook events always reflects the latest attributes of the given resources. We recommend you to compare updated_at in the resource payload with the current time or ts in the event payload.

ts represents the time that the event is triggered, if the event is triggered in the following sequence A --> B --> C, then the timestamp for A would be the earliest, followed by B, then C (i.e., A's ts < B's ts < C's ts).

Implications for Sequence-Sensitive Applications

🚧

Example | Order webhook usecase 1

Webhook with Different Order Status Payloads may receive in different sequence occasionally

i. Sequence Based on Timestamp: A -> B = Order pending -> Order confirmed

ii. Sequence Based on Receiving Time: B -> A = Order confirmed -> Order pending

In this scenario, the sequence of events based on receiving time may misrepresent the actual order of events. If the application does not account for the timestamp, it will inevitably encounter logical errors.

🚧

Example | Order webhook usecase 2

Different Order Webhook Topics may receive in different sequence occasionally

i. Sequence Based on Timestamp: A -> B = Order created -> Order payment complete

ii. Sequence Based on Receiving Time: B -> A = Order payment complete -> Order created

In this scenario, the 'Order Payment Complete' webhook is received first, but lacks order details because the 'Order Created' event has not yet arrived, the application might encounter significant logic issues.

Solving Sequence Issues:

If your application's use case is sensitive to the sequence of events, it's essential to incorporate the timestamp from each webhook event into your application logic. This approach ensures that actions are processed in the correct chronological order, based on when events were actually triggered, rather than when they were received. Furthermore, applications may need to develop custom logic to effectively manage dependencies between different webhook topics.

For example, it might be necessary to cache the details of an order payment confirmation and then incorporate them only after the corresponding 'order create' event has been received. This method allows for the accurate assembly of information in the proper sequence, ensuring data integrity and consistency.

Webhook Idempotence Concept (may receive webhook more than once)

SHOPLINE aims to ensure webhook events to be delivered to endpoints a minimum of once, a feature known as "at least once" delivery. This also mean an endpoint may also receive same webhook event more than once. To recognize and manage these duplicate events, you can compare the 'trace id' of incoming events with those of previously received events.

Implementing Deduplication
To maintain the efficiency and accuracy of your application, it is crucial to incorporate a deduplication process (dedup) to avoid processing duplicate webhook events. This process should occur before the events are passed on to the application layer.

📘

Possible steps for Deduplication (for ref. only)

1. Trace ID Check: On receiving a webhook event, check its 'trace_id' against a list or database of 'trace_id(s)' from previously processed events.
2. Duplicate Identification: If the 'trace_id' is found in your records, identify the event as a duplicate and prevent it from proceeding further.
3. Processing Unique Events: Only allow events with unique 'trace_id' to move forward to the application layer for processing.
4. Record Keeping: Maintain a record of 'trace ids' from processed webhook events to ensure ongoing accuracy in duplication checks.