Title: Cognitive semiotics and neurodynamics. RT: Cognitive semiotics, Cognitive science 15th of September 2006
1Cognitive semiotics and neurodynamics.RT
Cognitive semiotics, Cognitive science15th of
September 2006
Wolfgang Wildgen
- Early Fall School in Semiotics
- 11th to 18th of September 2006
- Sozopol, Bulgaria
2Dynamic vs. cognitive semiotics
- Dynamic semiotics starts from biological
morphogenesis and expands this thought to
linguistics. In his holistic strategy René Thom
(1923-2002) preferred a Gestaltist view on
biology and not so much a mechanistic one, which
takes the brain as the central (and finally) only
organ responsible for thought, language and
culture. This view is further developed in
Wildgen (1994) introducing modes of cognitve
self-organization in style of synergetics (cf.
Haken and Stadler, 1991). - The research line of cognitive semiotics was
first developed in the philosophical atmosphere
of logical empiricism (Quine) and formal syntax
(Carnap). Whereas Chomsky created a compact
mentalistic theory, Lakoff (since 1975) and with
him Langacker and Talmy combined insights of
gestalt-psychology with ideas stemming from
issues of generative semantics. Cognitive
semiotics have extended this methodology (cf.
Brandt, 2002).
3Ways of a neurodynamic expansion of theses models
- The current advances in neurodynamics due to the
techniques of neuro-imaging and research in the
neurodynamics of animals (mice, rats etc.) during
perceptual and motor tasks give us a new
understanding of what the brain does, when a
subject (animal or human) has to solve specific
problems. - Dynamic semiotics (in Thoms tradition) must
consider the neural machinery and specify the
dynamics in the context of the brain. - Cognitive semiotics must leave the theoretical
isolation of structuralism, its primary concern
with grammar and go beyond psychological
analogies towards a foundation in the dynamics of
the brain.
4The representation of dynamics in cognitive
grammar (Langacker)
Langacker's analysis of the verb ENTER.
5Description of a process in cognitive semantics
(Talmy)
- The ball kept rolling because of the wind blowing
on it. - intrinsic force tendency of the Agonist (right)
towards rest (?), - the Antagonist (left) is stronger (?),
- intrinsic force tendency of the Antagonist
action (?), - result of the force interaction action ( ).
Schematisation of force-dynamics by Talmy (1988).
6The representation of dynamics in dynamic
semantics
Derivation of a process-schema (right) from a
path p in the vector-field of the cusp (A3)
capture A catches B.
7Are neural representations a map of external
dynamics?
- Both the quasi-imaginistic and the dynamic model
of meaning assume that a kind of abstract
pictures of external events exist in the mind.
Although neural representations must have an
enactive capacity and be able to govern action in
the external word (and social coordination in
such actions), they have a very specific format
and rather independent dynamics. - At the local level, where specific observation
and experimentation in a neuro-cognitive
experiment can be achieved, one can not assume
the appearance of such global mappings. This ma
only appear at a higher level, where learned
schemata are activated top-dawn. They must ,
however, be able to use the more basic binding
dynamics explained in the forgoing sections.
8Shortcomings of both cognitive and dynamic
semiotics
- If images filling Langackers articles represent
only vague and fluctuating after-effects observed
in the analysis of linguistic structures, on may
just dispense of them and return to classical
models. - All concepts taken from gestalt psychology have
since undergone a redefinition in terms of
neurodynamics. Why stick to the state of the art
fifty years ago? - The concepts taken from morphogenesis by Thom
(cf. his discussion with Waddingston) have also
been redefined using more specific higher order
knowledge on neurochemical processes. - The neurodynamics of perception, motor programs
and memory are nearer to language than basic
morphogenetic processes.
9The dynamics of composition (Freges principle)
- In a simple case which avoids the complexity of
verbal valence, and rather takes nominal syntax
as a basic example, one may consider a noun
related to the form of an object, say a square,
an adjective of colour, say red, and a present
participle of motion, say moving - red moving square
- How does the brain compose a head-noun referring
to form with two satellites referring to colour
and motion?
10Some elements of a model of neurolinguistic
binding
- For Andreas Engel (2004) the binding process is
one of temporal synchronization of neural
assemblies, which form wholes (gestalts) from
parts and one of desynchronization which
distinguishes figure and ground. - This type of analysis concerns only the
composition in perception, attentiveness and
memory, but one may conjecture a parallel process
for words (related to perceptual information) and
their composition in syntactic constructions
involving nouns and adjectives.
11The basic idea of temporal binding
Parts or features of a visual whole are linked by
the synchronic firing of a set of neurons (an
assembly) during a short time interval. In the
example the parts and features of the cat and
those of the woman are bound together by the
internal synchrony of the assemblies 1 and 2 and
they are distinguished by the asynchrony of these
assemblies.
From Engel et alii, 1997 572
12Ambiguity and Binding
Picture a is ambiguous. If it is seen as one face
(and a candle in front of it) the zones (1,2) and
(3,4) (see series d) are bound in the case two
faces looking at each other are seen, the zones
(1,3) und (2,4) are bound. The binding may be
recognized by the synchronic firing rates in the
series d versus e.
From Engel, Fries und Singer, 2001 707
13Top down effects due to expectation and memory
The remembered object produces higher
synchronization at the ?-level (30 to 60 Hz)
At the left a Kanitza-triangle At the right a
non-Kanitza-triangle If the tested person is
instructed to recognize the non-Kanitza-triangle
, the synchronization is higher for this
configuration, although basic gestalt laws would
predict the contrary.
From Hermann, Munk und Engel, 2004349
14Two different pathways starting from the visual
center V1 spezialize on the analysis of Where
was it seen? What was seen?
What and Where are the basic questions to
be answered in visual perception. Not only must
we recognize what we are looking at, but also we
need to know where it is in order to respond
appropriately. Gazzaniga u.a. (1998 165)
15Restrictions on compositionality due to temporal
binding
- Teisman (1999 108) writes
- It the binding-by-synchrony hypothesis also
provides a plausible reason for the attentional
limit of around four objects that is widely
observed in the perception of brief displays and
in studies of visual working memory. The
different firing rates that can be easily
discriminated on a background of inherent noise
and accidental synchronies may set a low limit to
the number of objects that can be simultaneously
bound. - The restrictions of valence patterns and of
embeddings (recursive operations) have been
discredited by Chomsky as performance effects. To
the contrary, they are hints to the nature of the
compositional process in language and thus more
interesting than the algebraic notion of
recursive operations.
16Some conclusions
- Cognitive semiotics must try to have a
neurodynamic foundation of such basic concepts as
the composition of meanings to a whole (Freges
problem) - Such a model should consider the topology of the
brain, synchronization and desynchronization, and
the coupling of subnetworks with
self-organization (filtering, choice of dominant
modes), self-reference and monitoring in
consciousness, etc. - As cognitive semiotics goes beyond cognitive
semantics and grammar, it must also consider the
composition of meaningfull entities in in the
visual (art, media, architecture) and in the
domain of audition (music) and motion (dance)
17Some bibliographical indications
- Engel AK, Fries P, Singer W (2001) Dynamic
predictions oscillations and synchrony in
top-down processing. Nature Reviews Neuroscience
2 704-716 - Engel AK, Roelfsema PR, Fries P, Brecht M, Singer
W (1997) Role of the temporal domain for response
selection and perceptual binding. Cerebral Cortex
7 571-582 - Herrmann CS, Munk MHJ, Engel AK (2004) Cognitive
functions of gamma-band activity memory match
and utilization. Trends in Cognitive Sciences 8
347-355 - Gazzaniga, Michael S., Richard B. Ivry und George
R. Mangun, 1998. Cognitive Neuroscience. The
Biology of the Mind, Norton, New York. - Machery, Edouard, Markus Werning, Gerhard Schurz
(Hg.), 2005. The Compositionality of Meaning,
Bd. II Applications to Linguistics, Psychology
and Neuroscience, Ontos, Frankfurt/Main. - Riesenhuber, Maximilian und Tonaso Poggio, 1999.
Are Cortical Models Really Bound by the Binding
Problem?, in Neuron, 24 87-93. - Singer, Wolf, 1999. Neuronal Synchrony A
Versatile Code for the Definition of Relations?,
in Neuron 24 49-65. - Teisman, Anne, 1999. Solutions to the Binding
Problem. Progress Through Controversy and
Convergence, in Neuron, 1999 105-110. - Werning, Markus, Eduard Machery, Gerhard Schurz
(Hg.), 2005. The Compositionality of Meaning and
Content, Bd. I Foundational Issues, Ontos,
Frankfurt/Main. - Wildgen, Wolfgang, 1994a. Process, Image, and
Meaning. A Realistic Model of the Meanings of
Sentences and Narrative Texts, Benjamins,
Amsterdam.
18My home page ishttp//www.fb10.uni-bremen.de/hom
epages/wildgen.htm
- You may find my presentation in Sozopol (when I
have returned) together with other conference
papers of the last two years in a list at the end
of my homepage.