Ring API

ShipClojure Datom uses reitit and ring for handling API calls. All of the middleware and configuration is done through ring. Reitit is used only for the final routing part where needed.

I chose this architecture because ring & reitit have different approaches to middleware configuration, that builds confusion when trying to mix them together. You cannot have one without the other as ring is the underlying framework that transforms requests and responses to and from clojure maps, and reitit is just a router based on the content of the request map.

Why not just use simple Ring then?

In theory you can simply just use a handler like the one below and be done. Especially since we use CQRS and the command/query system chooses what handler to use based on the keyword.

(defn handler [{:keys [uri] :as req} respond raise]
  (try
    (cond
      (= "/" uri)
      (respond (response/resource-response "/index.html" {:root "public"}))

      (= "/query" uri)
      (respond
       {:status 200
        :headers {"content-type" "application/edn"}
        :body (pr-str (query req))})

      (= "/command" uri)
      (respond
       {:status 200
        :headers {"content-type" "application/edn"}
        :body (pr-str (handle-command req))})

      :else
      (respond
       {:status 404
        :headers {"content-type" "text/html"}
        :body "<h1>Page not found</h1>"}))
    (catch Exception e
      (raise e))))

Since ShipClojure also handles static page routes, oauth2 handshake routes and various webhooks for external services, the example above would become quite heavy to manage and this is where a router comes in:

(def routes
  (into
    [["/echo" {:tags #{"Debugging"}
               :trace {:handler (fn [req]
                                  {:status 200
                                   :body (select-keys req [:headers :params])})}}]
     ["/openapi/openapi.json"
      {:get {:no-doc true
             :openapi {:info {:title "SaaS API Reference"
                              :description "openapi3 docs with [malli](https://github.com/metosin/malli) and reitit-ring"
                              :version "0.0.1"}}
             :handler (openapi/create-openapi-handler)}}]
     ["/cqrs" {:tags #{"CQRS"}
               :middleware [[mw/wrap-jwt-auth]]}
      ["/command" {:post {:summary "Main entrypoint for commands"
                          :description "Commands are actions that generally mutate in a way or another"
                          :middleware [[cqrs/wrap-command-validation]]
                          :handler cqrs/handle-command}}]
      ["/query" {:post {:summary "Main entrypoint for queries"
                        :description "Queries just fetch information as opposed to commands"
                        :handler cqrs/handle-query
                        :middleware [[cqrs/wrap-query-validation]]}}]]
     ["/webhooks" {:tags #{"Webhooks for external services"}}
      ["/stripe" {:post {:summary "Stripe webhook for receiving events from stipe."
                         :description "Use this handler for all stripe related logic based on events like customer.subscription.created|customer.subscription.updated|customer.subscription.deleted etc."
                         :handler (constantly {:status 200 :body "TODO"})}}]]

     ;; For all the SPA routes, we just serve the html containing the app bundle to be rendered clientside
     ["" {:get #(layout/app-page %)
          :tags #{"Single Page App Routes"}}
      app-routes/routes]]

    ;; Oauth2 routes related to handshake and oauth2 login initialization There
    ;; are also commands related to oauth2 but they contain the logic of what
    ;; happens AFTER the oauth2 authentication is done
    ;; see :command/finalize-oauth2 and :command/oauth2-onboarding
    (mapcat (fn [[id profile]]
              (oauth2/reitit-routes-for-profile (assoc profile :id id)))
            (c/oauth2-providers-config))))

Most of the time, the only API calls you'll need to make are /cqrs/command and /cqrs/query so you don't have to think too much about this.

Middleware chain

I'm adding this section about middleware because I've found that it is difficult to fully grasp the execution order of the middleware chain.

The main middleware chain that affects all requests is in handler.clj :

(-> reitit-handler
    ;; ring nexus for FCIS support
    (ring-nexus/wrap-nexus nxs/nexus system)
    ;; Log and catch exceptions
    (exception/exception-middleware)
    ;; format the response based on accept header
    (muuntaja.middleware/wrap-format-response middleware.formats/instance)
    ;; Log request parameters
    (logger/wrap-log-request-params)
    ;; Merge :body-params parsed by Muuntaja into :params map
    (muuntaja.middleware/wrap-params)
    ;; Specific ring config (params, cookies, session, static files)
    (ring.middleware.defaults/wrap-defaults config)
    ;; format the request
    (muuntaja.middleware/wrap-format-request middleware.formats/instance)
    ;; Log and catch exceptions
    (exception/exception-middleware)
    ;; negotiate the request & response formats
    (muuntaja.middleware/wrap-format-negotiate middleware.formats/instance)
    ;; Log response and duration - this is lower in the chain so the
    ;; request duration accounts for most of the middleware execution too.
    (logger/wrap-log-response)
    ;; Start logging and add ::logger/start-ms to request
    (logger/wrap-log-request-start))

Contrary to first instinct, the execution is from the bottom to the top and then again to the bottom.

The order of execution is:

1. logger/wrap-log-request-start goes first - logs request start and adds :saas.logging/start-ms to the request so we know how long it took to fully respond to the request. Also sets a log context :req-id so all of the logs further in the chain will share it so you can filter all the logs for the same request. Calls next in the chain.

2. logger/wrap-log-response - In the first part does nothing, just takes the start-ms from the earlier middleware and calls next in the chain.

3. muuntaja/wrap-format-negotiate - adds to the request muuntaja/request-format and muuntaja/response-format based on Content-Type and Accept headers from the request. This information will be used to decode the request body and encode the response in subsequent muuntaja middleware. Calls the chain forward

4. exception/exception-middleware - Calls the next function in the middleware chain but wrapped in a try/catch for potential errors. See Api Exception handling for more details

5. muuntaja/wrap-format-request This middleware takes the muuntaja/request-format from the request map added on step 3 (order of middleware matters) and decodes the request body from the specified format into edn and adds the content to :body-params

6. wrap-defaults is a configurable middleware from /docs/datom/backend/system/ . It adds the default ring middleware like setting up cookies, session, query params, serving static assets from the public directory and more.

7. muuntaja/wrap-params takes the :body-params inserted at step 5 and merges them in the :params with the other potential :params added from the middleware at step 6.

8. wrap-log-request-params logs all of the request params from the :params key (if any). Calls next in the chain

9. muuntaja/wrap-format-response at this point does nothing, but waits for the final response map from further down the chain so it can encode it with the appropriate format. Example: If the request has an Accept header of application/json, this means the client expects JSON back so if the final handler will return a response of {:hello :world}, this handler will replace the response body with a stream containing the JSON equivalent.

10. Another exception-middleware that wrawps the next functions in the chain in a try-catch. Why? Because if the next handlers throw, the thrown error will bypass response formatting from step 9 and go straight to step 4 in the chain where the error is caught and response map is generated. Therefore we need another exception handler here, before the format response middleware, that generates the response map, which then comes back to step 9 so it is formatted to the expected client format (JSON, EDN, or transit - what the ShipClojure SPA uses)

11. ring-nexus/wrap-nexus - this is a middleware that enables us to use nexus on the backend so we have pure handlers.

12. The reitit handler takes the request and mapps it to the appropriate handler. Example: if the request has uri /cqrs/command and is a POST, it will pass it to cqrs/handle-command, which if you notice has a reitit middleware defined. Reitit middleware function the same as ring middleware. It is important to understand the order. Middleware defined closer to the final handler, execute latter in the evaluation chain, but still before the final handler:

    ["/command" {:post {;; This middleware validates the data for the command
                        :middleware [[cqrs/wrap-command-validation]]
                        :handler cqrs/handle-command}}]
    

13. Final handler, let's say here cqrs/handle-command executes and returns a nexus action response [[http/respond {:status 200 :body {:message "success"}}]]. Now we go backward in the chain

14. Reitit router pass through

15. ring-nexus/wrap-nexus gets the return value and transforms it into a ring response map {:status 200 :body {:message "success"}}

16. Exception middleware returns the value since nothing was thrown

17. muuntaja/wrap-format-response receives the response and converts it to JSON (if necessary)

18. wrap-log-params already logged the response so it sends the response further. Same for step 7.

19. Depending on config, wrap-defaults might set the content-type, charset and not-modified on the response.

20. Step 5, 4, 3 will do nothing since they have logic pertaining to the request, not the response so they just send the response further

21. wrap-log-response finally has the response and it can log the response status, content and duration of the entire chain

22. wrap-request-start doesn't have anything else to do so it just passes the result to ring which will hand the appropriate format to the underlying server (http-kit by default, but can be changed to use jetty)

That's it! That's the rather complex but necessary chain of events needed for us to handle a API request.

If you think there is

Api Exception handling

TODO