event conditional restarts

Author: Chris Zheng  (z@caudate.me)
Date: 27 November 2018
Repository: https://github.com/zcaudate/hara
Version: 3.0.2

1    Introduction

hara.event aims to provide more loosely coupled code through two mechanisms:

  • a signalling framework for normal operations
  • a conditional restart framework for abnormal operations

The two paradigms have been combined into a single library because they share a number of similarities in terms of both the communication of system information as well as the need for passive listener. However, abnormal operations are alot trickier to resolve and requires more attention to detail.

1.1    Installation

Add to project.clj dependencies:

[hara/base "3.0.2"]

All functions are in the hara.event namespace. 

 (use (quote hara.event))

2    Signal

clear-listeners ^

empties all event listeners

v 3.0
(defn clear-listeners
  []
  (reset! *signal-manager* (handler/manager)))
link
(clear-listeners)
;; all defined listeners will be cleared

deflistener ^

installs a global signal listener

v 3.0
(defmacro deflistener
  [name checker bindings & more]
  (let [sym    (str  (.getName *ns*) "/" name)
        cform  (util/checker-form checker)
        hform  (util/handler-form bindings more)]
    `(do (install-listener (symbol ~sym) ~cform ~hform)
         (def ~(symbol name) (symbol ~sym)))))
link
(def -counts- (atom {}))

(deflistener -count-listener- :log
  [msg]
  (swap! -counts- update-in [:counts] (fnil #(conj % (count msg)) [])))

(signal [:log {:msg "Hello World"}])

(signal [:log {:msg "How are you?"}])

@-counts-
=> {:counts [11 12]}

install-listener ^

adds an event listener, `deflistener` can also be used

v 3.0
(defn install-listener
  [id checker handler]
  (swap! *signal-manager*
         handler/add-handler checker {:id id
                                     :fn handler}))
link
(install-listener '-hello-
                  :msg
                  (fn [{:keys [msg]}]
                    (str "recieved " msg)))

(list-listeners)
=> (contains-in [{:id '-hello-
                  :checker :msg}])

list-listeners ^

shows all event listeners

v 3.0
(defn list-listeners
  ([]
   (handler/list-handlers @*signal-manager*))
  ([checker]
   (handler/list-handlers @*signal-manager* checker)))
link
(deflistener -hello-listener- :msg
  [msg]
  (str "recieved " msg))

(list-listeners)
=> (contains-in [{:id 'hara.event-test/-hello-listener-,
                  :checker :msg}])

signal ^

signals an event that is sent to, it does not do anything by itself

v 3.0
(defn signal
  ([data]
   (signal data @*signal-manager*))
  ([data manager]
   (let [ndata   (handler/expand-data data)]
     (doall (for [handler (handler/match-handlers manager ndata)]
              {:id (:id handler)
               :result ((:fn handler) ndata)})))))
link
(signal :anything) => ()
  
(deflistener -hello- _
  e
  e)
  
(signal :anything)
=> '({:id hara.event-test/-hello- :result {:anything true}})

uninstall-listener ^

uninstalls a global signal listener

v 3.0
(defn uninstall-listener
  [id]
  (do (swap! *signal-manager* handler/remove-handler id)
      (if-let [nsp (and (symbol? id)
                        (.getNamespace ^Symbol id)
                        (Namespace/find (symbol (.getNamespace ^Symbol id))))]
        (do (.unmap ^Namespace nsp (symbol (.getName ^Symbol id)))
            nsp)
        id)))
link
(uninstall-listener 'hara.event-test/-hello-)

with-temp-listener ^

used for isolating and testing signaling

v 3.0
(defmacro with-temp-listener
  [[checker handler] & body]
  `(binding [*signal-manager* (atom (handler/manager))]
     (install-listener :temp ~checker ~handler)
     ~@body))
link
(with-temp-listener [{:id string?}
                     (fn [e] "world")]
  (signal {:id "hello"}))
=> '({:result "world", :id :temp})

3    Restart

choose ^

used within a manage form to definitively fail the system

v 3.0
(defn choose
  [label & args]
  {:type :choose :label label :args args})
link
(manage (raise :error
               (option :specify [a] a))
        (on :error
            _
            (choose :specify 42)))
=> 42

continue ^

used within a manage form to continue on with a particular value

v 3.0
(defn continue
  [body]
  {:type :continue :value body})
link
(manage [1 2 (raise :error)]
        (on :error
            _
            (continue 3)))
=> [1 2 3]

default ^

used within either a raise or escalate form to specify the default option to take if no other options arise.

v 3.0
(defn default
  [& args]
  {:type :default :args args})
link
(raise :error
       (option :specify [a] a)
       (default :specify 3))
=> 3

(manage
 (raise :error
        (option :specify [a] a)
        (default :specify 3))
 (on :error []
     (escalate :error
               (default :specify 5))))
=> 5

escalate ^

used within a manage form to add further data on an issue

v 3.0
(defmacro escalate
  [data & forms]
  (let [[data forms]
        (if (util/is-special-form :raise data)
          [nil (cons data forms)]
          [data forms])]
    `{:type :escalate
      :data ~data
      :options  ~(util/parse-option-forms forms)
      :default  ~(util/parse-default-form forms)}))
link
(manage [1 2 (raise :error)]
        (on :error
            _
            (escalate :escalated)))
=> (throws-info {:error true
                 :escalated true})

fail ^

used within a manage form to definitively fail the system

v 3.0
(defn fail
  ([] {:type :fail :data {}})
  ([data]
   {:type :fail :data data}))
link
(manage (raise :error)
        (on :error
            _
            (fail :failed)))
=> (throws-info {:error true})

manage ^

manages a raised issue, like try but is continuable:

v 3.0
(defmacro manage
  [& forms]
  (let [sp-fn           (fn [form] (util/is-special-form :manage form #{'finally 'catch}))
        body-forms      (vec (filter (complement sp-fn) forms))
        sp-forms        (filter sp-fn forms)
        id              (handler/new-id)
        options         (util/parse-option-forms sp-forms)
        on-handlers     (util/parse-on-handler-forms sp-forms)
        on-any-handlers (util/parse-on-any-handler-forms sp-forms)
        try-forms       (util/parse-try-forms sp-forms)
        optmap          (zipmap (keys options) (repeat id))]
    `(let [manager# (handler/manager ~id
                                    ~(vec (concat on-handlers on-any-handlers))
                                    ~options)]
       (binding [*issue-managers* (cons manager# *issue-managers*)
                 *issue-optmap*   (merge ~optmap *issue-optmap*)]
         (try
           (try
             ~@body-forms
             (catch clojure.lang.ExceptionInfo ~'ex
               (manage/manage-condition manager# ~'ex)))
           ~@try-forms)))))
link
(manage [1 2 (raise :error)]
        (on :error
            _
            3))
=> 3

raise ^

raise an issue, like throw but can be conditionally managed as well as automatically resolved:

v 3.0
(defmacro raise
  [content & [msg & forms]]
  (let [[msg forms] (if (util/is-special-form :raise msg)
                      ["" (cons msg forms)]
                      [msg forms])
        options (util/parse-option-forms forms)
        default (util/parse-default-form forms)]
    `(let [issue# (data/issue ~content ~msg ~options ~default)]
       (signal (assoc (:data issue#) :issue (:msg issue#)))
       (raise/raise-loop issue# *issue-managers*
                         (merge (:optmap issue#) *issue-optmap*)))))
link
(raise  [:error {:msg "A problem."}])
=> (throws-info {:error true
                 :msg "A problem."})

(raise [:error {:msg "A resolvable problem"}]
       (option :something [] 42)
       (default :something))
=> 42

4    Walkthough

4.1    Signal

hara.event contains a flexible signaling and listener framework. This allows for decoupling of side-effecting functions. In normal program flow, there may be instances where an event in a system requires additional processing that is adjunct to the core:

  • logging
  • printing on screen
  • sending an email
  • creating audio/visual effects
  • writing to a queue, database or cache

The signalling framework allows for the specification of signals type. This enables loose coupling between the core and libraries providing functionality for side effects, enabling the most flexible implementation for signaling. As all information surrounding a signal is represented using data, listeners that provide actual resolution can be attached and detached without too much effort. In this way the core becomes lighter and is stripped of dependencies.

signal typically just informs its listeners with a given set of information. An example of this can be seen here:

(signal {:web true :log true :msg "hello"})
=> ()

It can also be written like this for shorthand:

(signal [:web :log {:msg "hello"}])
=> ()

A signal by itself does not do anything. It requires a listener to be defined in order to process the signal:

(deflistener log-print-listener :log
  e
  (println "LOG:" e))

(signal [:web :log {:msg "hello"}])
=> ({:result nil,
     :id documentation.hara.hara-event/log-print-listener})
;; LOG: {:web true, :log true, :msg hello}

A second listener will result in signal triggering two calls

(deflistener web-print-listener :web
  e
  (println "WEB:" e))

(signal [:web :log {:msg "hello"}])
=> ({:result nil, :id documentation.hara.hara-event/web-print-listener}
    {:result nil, :id documentation.hara.hara-event/log-print-listener})
;; LOG: {:web true, :log true, :msg hello}
;; WEB: {:web true, :log true, :msg hello}

Whereas another signal without an attached data listener will not trigger:

(signal [:db {:msg "hello"}])
=> ()

This can be resolved by adding another listener:

(deflistener db-print-listener :db
  e
  (println "DB:" e))
(signal [:db {:msg "hello"}])
=> ({:result nil, :id documentation.hara.hara-event/db-print-listener})
;; DB: {:db true, :msg hello}

hara.event also provides for a conditional framework. It also can be thought of as an issue resolution system or try++/catch++. There are many commonalities between the signalling framework as well as the conditional framework. Because the framework deals with abnormal program flow, there has to berichness in semantics in order to resolve the different types of issues that may occur. The diagram below shows how the two frameworks fit together.

4.2    Restart

In this demonstration, we look at how code bloat problems using throw/try/catch could be reduced using raise/manage/on. Two functions are defined:

  • value-check which takes a number as input, throwing a RuntimeException when it sees an input that it doe not like.
  • value-to-string which takes a number as input, and returns it's string
(defn value-check [n]
  (cond (= n 13)
        (raise {:type :unlucky
                :value n})

        (= n 7)
        (raise {:type :lucky
                :value n})

        (= n 666)
        (raise {:type :the-devil
                :value n})

        :else n))

(defn value-to-string [n]
  (str (value-check n)))

uses of value-to-string are as follows:

(value-to-string 1)
=> "1"

(value-to-string 666)
=> (throws-info {:value 666
                 :type :the-devil})

The advantage of using conditionals instead of the standard java throw/catch framework is that problems can be isolated to a particular scope without affecting other code that had already been built on top. When we map value-to-string to a range of values, if the inputs are small enough then there is no problem:

(mapv value-to-string (range 3))
=> ["0" "1" "2"]

However, if the inputs are wide enough to contain something out of the ordinary, then there is an exception.

(mapv value-to-string (range 10))
=> (throws-info {:value 7
                 :type :lucky})

4.3    Exceptions

manage is the top level form for handling exceptions to normal program flow. The usage of this form is:

(manage
 (mapv value-to-string (range 10))
 (on {:type :lucky} e
     "LUCKY-NUMBER-FOUND"))
=> "LUCKY-NUMBER-FOUND"

This is the same mechanism as try/catch, which manage replacing try and on replacing catch. The difference is that there is more richness in semantics. The key form being continue:

(manage
 (mapv value-to-string (range 10))
 (on {:type :lucky}
     []
     (continue "LUCKY-NUMBER-FOUND")))
=> ["0" "1" "2" "3" "4" "5" "6"
    "LUCKY-NUMBER-FOUND" "8" "9"]

continue is special because it operates at the scope where the exception was raised. This type of handling cannot be replicated using the standard try/catch mechanism. Generally, exceptions that occur at lower levels propagate to the upper levels. This usually results in a complete reset of the system. continue allows for lower-level exceptions to be handled with much more grace because the scope is pin-pointed to where raise was called.

Furthermore, there may be exceptions that require more attention and so continue can only be used when it is needed.

(defn values-to-string [inputs]
  (manage
   (mapv value-to-string inputs)
   (on :value
       [type value]
       (cond (= type :the-devil)
             "OH NO!"

             :else (continue type)))))
(values-to-string (range 6 14))
=> ["6" ":lucky" "8" "9" "10" "11" "12" ":unlucky"]

(values-to-string [1 2 666])
=> "OH NO!"

5    Strategies

This is a comprehensive (though non-exhaustive) list of program control strategies that can be used. It can be noted that the try/catch paradigm can implement sections and . Other clojure restart libraries such as errorkit, swell and conditions additionally implement sections , and .

hara.event supports novel (and more natural) program control mechanics through the escalate (), fail () and default () special forms as well as branching support in the on special form ().

5.1    Normal

5.1.1    No Raise

The most straightforward code is one where no issues raised:

(manage            ;; L2
 [1 2 (manage 3)]) ;; L1 and L0

=> [1 2 3]

fig.1  -  No Issues Flow

5.1.2    Issue Raised

If there is an issue raised with no handler, it will throw an exception.

(manage                  ;; L2
 [1 2 (manage            ;; L1
       (raise {:A true}))]) ;; L0

=> (throws-info {:A true})

fig.2  -  Unmanaged Issue Flow

5.2    Catch

Once an issue has been raised, it can be handled within a managed scope through the use of 'on'. 'manage/on' is the equivalent to 'try/catch' in the following two cases:

5.2.1    First Level Catch

(manage                          ;; L2
 [1 2 (manage                    ;; L1
       (raise :A)                ;; L0
       (on :A [] :A))]           ;; H1A
 (on :B [] :B))                  ;; H2B

=> [1 2 :A]

fig.3  -  Catch on :A Flow

5.2.2    Second Level Catch

(manage                          ;; L2
 [1 2 (manage                    ;; L1
       (raise :B)                ;; L0
       (on :A [] :A))]           ;; H1A
 (on :B [] :B))                  ;; H2B

=> :B

fig.4  -  Catch on :B Flow

5.3    Continue

The 'continue' form signals that the program should resume at the point that the issue was raised.

5.3.1    First Level Continue

In the first case, this gives the same result as try/catch.

(manage                          ;; L2
 [1 2 (manage                    ;; L1
       (raise :A)                ;; L0
       (on :A []                 ;; H1A
           (continue :3A)))]
 (on :B []                       ;; H2B
     (continue :3B)))
=> [1 2 :3A]

fig.5  -  Continue on :A Flow

5.3.2    Second Level Continue

However, it can be seen that when 'continue' is used on the outer manage blocks, it provides the 'manage/on' a way for top tier forms to affect the bottom tier forms without manipulating logic in the middle tier

(manage                          ;; L2
 [1 2 (manage                    ;; L1
       (raise :B)                ;; L0
       (on :A []                 ;; H1A
           (continue :3A)))]
 (on :B []                       ;; H2B
     (continue :3B)))
=> [1 2 :3B]

fig.6  -  Continue on :B Flow

5.4    Choose

choose and option work together within manage scopes. A raised issue can have options attached to it, just a worker might give their manager certain options to choose from when an unexpected issue arises. Options can be chosen that lie anywhere within the manage blocks.

5.4.1    Choose Lower-Level

(manage                        ;; L2
 [1 2 (manage                  ;; L1
       (raise :A               ;; L0
              (option :X [] :3X)) ;; X
       (on :A []                  ;; H1A
           (choose :X))
       (option :Y [] :3Y))] ;; Y
 (option :Z [] :3Z))        ;; Z

=> [1 2 :3X]

fig.7  -  Choose :X Flow

However in some cases, upper level options can be accessed as in this case. This can be used to set global strategies to deal with very issues that have serious consequences if it was to go ahead.

An example maybe a mine worker who finds a gas-leak. Because of previously conveyed instructions, he doesn't need to inform his manager and shuts down the plant immediately.

5.4.2    Choose Upper-Level

(manage                           ;; L2
 [1 2 (manage                     ;; L1
       (raise :A                  ;; L0
              (option :X [] :3X)) ;; X
       (on :A []                  ;; H1A
           (choose :Z))
       (option :Y [] :3Y))]       ;; Y
 (option :Z [] :3Z))              ;; Z
=> :3Z

fig.8  -  Choose :Z Flow

5.5    Choose - More Strategies

5.5.1    Overridding An Option

If there are two options with the same label, choose will take the option specified at the highest management level. This means that managers at higher levels can over-ride lower level strategies.

(manage                            ;; L2
 [1 2 (manage                      ;; L1
       (raise :A                   ;; L0
              (option :X [] :3X0)) ;; X0 - This is ignored
       (on :A []                   ;; H1A
           (choose :X))
       (option :X [] :3X1))]       ;; X1 - This is chosen
 (option :Z [] :3Z))               ;; Z

=> [1 2 :3X1]

fig.9  -  Choose :X1 Flow

5.5.2    Default Option

Specifying a 'default' option allows the raiser to have autonomous control of the situation if the issue remains unhandled.

(manage                            ;; L2
 [1 2 (manage                      ;; L1
       (raise :A                   ;; L0
              (default :X)         ;; D
              (option :X [] :3X))  ;; X
       (option :Y [] :3Y))]        ;; Y
 (option :Z [] :3Z))               ;; Z

=> [1 2 :3X]

fig.10  -  Choose Default Flow

5.5.3    Overriding Defaults

This is an example of higher-tier managers overriding options

(manage                            ;; L2
 [1 2 (manage                      ;; L1
       (raise :A                   ;; L0
              (default :X)         ;; D
              (option :X [] :3X0)) ;; X0
       (option :X [] :3X1))]       ;; X1
 (option :X [] :3X2))             ;; X2

=> :3X2

fig.11  -  Choose Default :X2 Flow

5.6    Escalate

5.6.1    Simple Escalation

When issues are escalated, more information can be added and this then is passed on to higher-tier managers

(manage                            ;; L2
 [1 2 (manage                      ;; L1
       (raise :A)                  ;; L0
       (on :A []                   ;; H1A
           (escalate :B)))]
 (on :B []                         ;; H2B
     (continue :3B)))
=> [1 2 :3B]

fig.12  -  Escalate :B Flow

5.6.2    Escalation with Options

More options can be added to escalate. When these options are chosen, it will continue at the point in which the issue was raised.

(manage                            ;; L2
 [1 2 (manage                      ;; L1
       (raise :A)                  ;; L0
       (on :A []                   ;; H1A
           (escalate
            :B
            (option :X [] :3X))))] ;; X
 (on :B []                        ;; H2B
     (choose :X)))
=> [1 2 :3X]

fig.13  -  Escalate :B, Choose :X Flow

5.7    Fail

Fail forces a failure. It is used where there is already a default option and the manager really needs it to fail.

(manage                          ;; L2
 [1 2 (manage                    ;; L1
       (raise :A                 ;; L0
              (option :X [] :X)
              (default :X))
       (on :A []                 ;; H1A
           (fail :B)))])
=> (throws-info {:A true :B true})

fig.14  -  Force Fail Flow

5.8    Default

Default short-circuits higher managers so that the issue is resolved internally.

(manage                          ;; L2
 [1 2 (manage                    ;; L1
       (raise :A                 ;; L0
              (option :X [] :X)
              (default :X))
       (on :A []                  ;; H1A
           (default)))]
 (on :A [] (continue 3)))
=> [1 2 :X]

5.8.1    Escalation with Defaults

This is default in combination with escalate to do some very complex jumping around.

(manage                            ;; L2
 [1 2 (manage                      ;; L1
       (raise :A                   ;; L0
              (option :X [] :X))        ;; X
       (on :A []                   ;; H1A
           (escalate
            :B
            (default :X))))]       ;; D1
 (on :B []                        ;; H2B
     (default)))
=> [1 2 :X]

fig.15  -  Escalate :B, Choose Default Flow

5.9    Branch Using On

Customized strategies can also be combined within the on handler. In the following example, it can be seen that the on :error handler supports both escalate and continue strategies.

(manage (manage
         (mapv (fn [n]
                 (raise [:error {:data n}]))
               [1 2 3 4 5 6 7 8])
         (on :error [data]
             (if (> data 5)
               (escalate :too-big)
               (continue data))))
        (on :too-big [data]
            (continue (- data))))
=> [1 2 3 4 5 -6 -7 -8]

Using branching strategies with on much more complex interactions can be constructed beyond the scope of this document.