Cognitive semiotics and neurodynamics. RT: Cognitive semiotics, Cognitive science 15th of September 2006

1
Cognitive semiotics and neurodynamics.RT
Cognitive semiotics, Cognitive science15th of
September 2006
Wolfgang Wildgen

• Early Fall School in Semiotics
• 11th to 18th of September 2006
• Sozopol, Bulgaria

2
Dynamic vs. cognitive semiotics

• Dynamic semiotics starts from biological
  morphogenesis and expands this thought to
  linguistics. In his holistic strategy René Thom
  (1923-2002) preferred a Gestaltist view on
  biology and not so much a mechanistic one, which
  takes the brain as the central (and finally) only
  organ responsible for thought, language and
  culture. This view is further developed in
  Wildgen (1994) introducing modes of cognitve
  self-organization in style of synergetics (cf.
  Haken and Stadler, 1991).
• The research line of cognitive semiotics was
  first developed in the philosophical atmosphere
  of logical empiricism (Quine) and formal syntax
  (Carnap). Whereas Chomsky created a compact
  mentalistic theory, Lakoff (since 1975) and with
  him Langacker and Talmy combined insights of
  gestalt-psychology with ideas stemming from
  issues of generative semantics. Cognitive
  semiotics have extended this methodology (cf.
  Brandt, 2002).

3
Ways of a neurodynamic expansion of theses models

• The current advances in neurodynamics due to the
  techniques of neuro-imaging and research in the
  neurodynamics of animals (mice, rats etc.) during
  perceptual and motor tasks give us a new
  understanding of what the brain does, when a
  subject (animal or human) has to solve specific
  problems.
• Dynamic semiotics (in Thoms tradition) must
  consider the neural machinery and specify the
  dynamics in the context of the brain.
• Cognitive semiotics must leave the theoretical
  isolation of structuralism, its primary concern
  with grammar and go beyond psychological
  analogies towards a foundation in the dynamics of
  the brain.

4
The representation of dynamics in cognitive
grammar (Langacker)
Langacker's analysis of the verb ENTER.
5
Description of a process in cognitive semantics
(Talmy)

• The ball kept rolling because of the wind blowing
  on it.
• intrinsic force tendency of the Agonist (right)
  towards rest (?),
• the Antagonist (left) is stronger (?),
• intrinsic force tendency of the Antagonist
  action (?),
• result of the force interaction action ( ).

Schematisation of force-dynamics by Talmy (1988).
6
The representation of dynamics in dynamic
semantics
Derivation of a process-schema (right) from a
path p in the vector-field of the cusp (A3)
capture A catches B.
7
Are neural representations a map of external
dynamics?

• Both the quasi-imaginistic and the dynamic model
  of meaning assume that a kind of abstract
  pictures of external events exist in the mind.
  Although neural representations must have an
  enactive capacity and be able to govern action in
  the external word (and social coordination in
  such actions), they have a very specific format
  and rather independent dynamics.
• At the local level, where specific observation
  and experimentation in a neuro-cognitive
  experiment can be achieved, one can not assume
  the appearance of such global mappings. This ma
  only appear at a higher level, where learned
  schemata are activated top-dawn. They must ,
  however, be able to use the more basic binding
  dynamics explained in the forgoing sections.

8
Shortcomings of both cognitive and dynamic
semiotics

• If images filling Langackers articles represent
  only vague and fluctuating after-effects observed
  in the analysis of linguistic structures, on may
  just dispense of them and return to classical
  models.
• All concepts taken from gestalt psychology have
  since undergone a redefinition in terms of
  neurodynamics. Why stick to the state of the art
  fifty years ago?
• The concepts taken from morphogenesis by Thom
  (cf. his discussion with Waddingston) have also
  been redefined using more specific higher order
  knowledge on neurochemical processes.
• The neurodynamics of perception, motor programs
  and memory are nearer to language than basic
  morphogenetic processes.

9
The dynamics of composition (Freges principle)

• In a simple case which avoids the complexity of
  verbal valence, and rather takes nominal syntax
  as a basic example, one may consider a noun
  related to the form of an object, say a square,
  an adjective of colour, say red, and a present
  participle of motion, say moving
• red moving square
• How does the brain compose a head-noun referring
  to form with two satellites referring to colour
  and motion?

10
Some elements of a model of neurolinguistic
binding

• For Andreas Engel (2004) the binding process is
  one of temporal synchronization of neural
  assemblies, which form wholes (gestalts) from
  parts and one of desynchronization which
  distinguishes figure and ground.
• This type of analysis concerns only the
  composition in perception, attentiveness and
  memory, but one may conjecture a parallel process
  for words (related to perceptual information) and
  their composition in syntactic constructions
  involving nouns and adjectives.

11
The basic idea of temporal binding
Parts or features of a visual whole are linked by
the synchronic firing of a set of neurons (an
assembly) during a short time interval. In the
example the parts and features of the cat and
those of the woman are bound together by the
internal synchrony of the assemblies 1 and 2 and
they are distinguished by the asynchrony of these
assemblies.
From Engel et alii, 1997 572
12
Ambiguity and Binding
Picture a is ambiguous. If it is seen as one face
(and a candle in front of it) the zones (1,2) and
(3,4) (see series d) are bound in the case two
faces looking at each other are seen, the zones
(1,3) und (2,4) are bound. The binding may be
recognized by the synchronic firing rates in the
series d versus e.
From Engel, Fries und Singer, 2001 707
13
Top down effects due to expectation and memory
The remembered object produces higher
synchronization at the ?-level (30 to 60 Hz)
At the left a Kanitza-triangle At the right a
non-Kanitza-triangle If the tested person is
instructed to recognize the non-Kanitza-triangle
, the synchronization is higher for this
configuration, although basic gestalt laws would
predict the contrary.
From Hermann, Munk und Engel, 2004349
14
Two different pathways starting from the visual
center V1 spezialize on the analysis of Where
was it seen? What was seen?
What and Where are the basic questions to
be answered in visual perception. Not only must
we recognize what we are looking at, but also we
need to know where it is in order to respond
appropriately. Gazzaniga u.a. (1998 165)
15
Restrictions on compositionality due to temporal
binding

• Teisman (1999 108) writes
• It the binding-by-synchrony hypothesis also
  provides a plausible reason for the attentional
  limit of around four objects that is widely
  observed in the perception of brief displays and
  in studies of visual working memory. The
  different firing rates that can be easily
  discriminated on a background of inherent noise
  and accidental synchronies may set a low limit to
  the number of objects that can be simultaneously
  bound.
• The restrictions of valence patterns and of
  embeddings (recursive operations) have been
  discredited by Chomsky as performance effects. To
  the contrary, they are hints to the nature of the
  compositional process in language and thus more
  interesting than the algebraic notion of
  recursive operations.

16
Some conclusions

• Cognitive semiotics must try to have a
  neurodynamic foundation of such basic concepts as
  the composition of meanings to a whole (Freges
  problem)
• Such a model should consider the topology of the
  brain, synchronization and desynchronization, and
  the coupling of subnetworks with
  self-organization (filtering, choice of dominant
  modes), self-reference and monitoring in
  consciousness, etc.
• As cognitive semiotics goes beyond cognitive
  semantics and grammar, it must also consider the
  composition of meaningfull entities in in the
  visual (art, media, architecture) and in the
  domain of audition (music) and motion (dance)

17
Some bibliographical indications

• Engel AK, Fries P, Singer W (2001) Dynamic
  predictions oscillations and synchrony in
  top-down processing. Nature Reviews Neuroscience
  2 704-716
• Engel AK, Roelfsema PR, Fries P, Brecht M, Singer
  W (1997) Role of the temporal domain for response
  selection and perceptual binding. Cerebral Cortex
  7 571-582
• Herrmann CS, Munk MHJ, Engel AK (2004) Cognitive
  functions of gamma-band activity memory match
  and utilization. Trends in Cognitive Sciences 8
  347-355
• Gazzaniga, Michael S., Richard B. Ivry und George
  R. Mangun, 1998. Cognitive Neuroscience. The
  Biology of the Mind, Norton, New York.
• Machery, Edouard, Markus Werning, Gerhard Schurz
  (Hg.), 2005. The Compositionality of Meaning,
  Bd. II Applications to Linguistics, Psychology
  and Neuroscience, Ontos, Frankfurt/Main.
• Riesenhuber, Maximilian und Tonaso Poggio, 1999.
  Are Cortical Models Really Bound by the Binding
  Problem?, in Neuron, 24 87-93.
• Singer, Wolf, 1999. Neuronal Synchrony A
  Versatile Code for the Definition of Relations?,
  in Neuron 24 49-65.
• Teisman, Anne, 1999. Solutions to the Binding
  Problem. Progress Through Controversy and
  Convergence, in Neuron, 1999 105-110.
• Werning, Markus, Eduard Machery, Gerhard Schurz
  (Hg.), 2005. The Compositionality of Meaning and
  Content, Bd. I Foundational Issues, Ontos,
  Frankfurt/Main.
• Wildgen, Wolfgang, 1994a. Process, Image, and
  Meaning. A Realistic Model of the Meanings of
  Sentences and Narrative Texts, Benjamins,
  Amsterdam.

18
My home page ishttp//www.fb10.uni-bremen.de/hom
epages/wildgen.htm

• You may find my presentation in Sozopol (when I
  have returned) together with other conference
  papers of the last two years in a list at the end
  of my homepage.