week 13: strange models of computation — comments on Sloman's H-CogAff? — Liquid State Machines (Maass; Fernando) — enzymatic computation (Barrett)

Liquid State machines

Any-time computation

Time invariance

Fading memory

slide 2

Liquid State machines

A liquid-state machine with a fixed-dynamics circuit (top left) can be augmented by a memoryless feedback circuit (top right), in which case it becomes capable of implementing any dynamical system (bottom) of the appropriate order, as long as the function that defines it, G, is smooth enough.

slide 3

a real Liquid State Machine

Chrisantha Fernando
The Liquid Brain.
Talk given at ECAL 2003 Dortmund (best paper award).

slide 4

Fodor's Problem

slide 5

a solution: an "enzymatic" blackboard architecture

"When one device outputs information, there must be properties in that information — aspects of its representational format — that allow its 'meaning' to be properly interpreted by other devices."

"Fodor (2000) explicitly denies that processing of information in the pool could be handled by specialized devices. He resists the idea that central processes could be composed entirely of specialized computational devices in which all devices in the system have access to exactly the same information. Indeed, he argues that because modular systems accept only local inputs, a global processing system composed entirely of modular devices could not be instantiated by any kind of computational system of which we are currently aware."

slide 6

enzymatic computation

"Enzymes catalyze reactions: they systematically combine substrates (molecules) to form new products (other molecules). This can be regarded as a kind of computation (in fact, Magnasco, 1997, has shown that enzymes are Turing universal)."

slide 7

binding specificity

1. Enzymes accept information of a particular kind, generally in the form of chemical substrates with particular properties that meet the binding specificity criteria of the enzyme in a 'lock and key' fashion.
2. Enzymes perform specific operations on the information they admit, catalyzing reactions that produce reaction products with different properties than the input substrates. [...] In each case, the computation is specific and dependent on the properties of the substrates in interaction with the properties of the enzyme.
3. Enzymes output the resulting information in a format useable by other systems. In other words, the products of enzymatically catalyzed reactions can then participate in further chemical reactions. There can be enzymatic processing cascades, feedback loops, and so on.

"Clearly, cognitive modules are not enzymes; enzymes are simply a metaphor. In this metaphor, information processing is catalysis."

slide 8

cognitively relevant properties of enzymatic computation

• Access generality with processing specificity. One can have an enzymatic system in which all of the enzymes in the system have access to all of the substrates, and in which only the 'correct' reactions are catalyzed.
• Multidimensional input criteria and byproduct processing. The reason that binding procedures of enzymes have analog properties is that many individual chemical bonding events contribute to the recognition process. The sum of these determine recognition; there can be better or worse degrees of fit. In this sense (i.e. fuzzy input criteria) the input procedures of enzymes resemble neural networks more than classical symbol-manipulation systems.
• Class-level processing, carry-through, and tags. Enzymes can be designed to pick out not just one substrate, but to identify substrates of a particular class — for example, molecules that all have one set of subunits in common, even if subunits in other portions of the molecule vary.
• Horizontal and top-down control. Tags added to substrates can influence how they are processed by other devices they may later encounter. This allows for horizontal and 'top down' control, in which the outputs of devices can influence other devices at the same horizontal level, a kind of feedback.

slide 9

horizontal control

slide 10

an enzymatic switch for horizontal control

slide 11

catalysis of a tagging reaction

"A 'LION' tag, in effect, carries the information, 'Attention all procedures that can generate true inferences from information about lions: here's something for you'. A one-to-one input-output piping system is one, rather inelegant and inflexible way of solving the routing problem; such systems face the problem that mechanisms outputting lion information must 'know' where to send it. Semantic tags, however, bypass this problem."

---

How can this functionality be implemented with neurons?

slide 12

catalysis of a secondary tagging reaction

slide 13

enzymatic computation of social contract primitives

(A propos Fodor's difficulties with understanding how complex concepts such as "this is a social interaction" can be reliably detected by his "modules")

slide 14

the really tough problems, such as...

... abduction, global belief revision, and context effects on inference.

First, percolation (diffusion, and diffusion with catalysis) is possible: there is nothing to prevent the same representation from passing through many different computational procedures, and a given representation may be compared to many beliefs (i.e. other representations in the system) simultaneously.

Second, unlike Fodorean systems, enzymatic systems allow for the possibility of feedback, including both positive feedback (e.g. amplification) and negative feedback (e.g. inhibition).

Third, reprocessing means that tagging of a representation by one procedure can affect how it is then processed by other procedures.