Artifacts and Human Thought: Probing the Delicate Balance Between Words, Tools and Inner State - PowerPoint PPT Presentation

1 / 92
About This Presentation
Title:

Artifacts and Human Thought: Probing the Delicate Balance Between Words, Tools and Inner State

Description:

Just how different ARE these two paradigms really? ... STOP feeling the tickling on the back and START to report rough, quasi-visual ... – PowerPoint PPT presentation

Number of Views:89
Avg rating:3.0/5.0
Slides: 93
Provided by: terri8
Category:

less

Transcript and Presenter's Notes

Title: Artifacts and Human Thought: Probing the Delicate Balance Between Words, Tools and Inner State


1
From Mere Embodiment to the Cyborg Mind
Andy Clark Cognitive Science Program Indiana
University andy_at_indiana.edu
As of Fall 2004 Dept of Philosophy Edinburgh
University Scotland
2
(No Transcript)
3
disembodied symbolic abstract rule-following clunk
y chunky high-level slow brittle..etc
4
(No Transcript)
5
(No Transcript)
6
embodiment grounding dynamics perception-action
systems real-world real-time embedded enactiveet
c
7
  • Just how different ARE these two paradigms
    really? Just what is it about embodiment that
    (when taken seriously) really matters for the
    very shape of a science of the mind?
  • What is the RELATION between those two apparently
    very different sets of target activities (between
    abstraction - rich reflective reason and more
    basic capacities for embodied action)? If
    embodiment matters, does it still matter all the
    way up?

8
  • Two Questions v
  • 2.Embodiment What Really
  • Matters?
  • 3.Sensing
  • 4.Incorporation versus Use
  • 5.Mind and Reason
  • 6.Can the be a Science of Hybrids?

9
  • Two Questions v
  • 2.Embodiment What Really
  • Matters?
  • 3.Sensory Substitution
  • 4. Incorporation versus Use
  • 5.Mind and Reason
  • 6. Can there be a Science of Hybrids?

10
Mere Embodiment Modest Embodiment Profound
Embodiment
11
(No Transcript)
12
For Shakey, the body and the environment were
first and foremost problems to be solved. The
environment was the problem arena. The sensors
detected the lay-out in that arena. The
reasoning system planned a solution. The body
was just another problem, that then needed to be
micro-managed so as to put the solution into
practice.
13
Passive Dynamic Walkers (PDWs) (Andy Ruina
Lab, Cornell original work by Tad McGeer) No
actuation except gravity, and no control system,
except for a mechanical knee. Inner and outer
legs are paired to constrain it from falling
over sideways. Surprisingly, PDWs are capable
(when set on a gentle incline) of very stable,
human-looking walking.
14
(No Transcript)
15
Systematically pushing, damping and tweaking a
system in which Passive Dynamic effects play a
very major role a very simple example of
ecological control
16
Ecological Control An ecological control system
is one in which goals are not achieved by micro-
managing every detail of the desired action or
response, but by making the most of robust,
reliable sources of relevant order in the bodily
or worldly environment of the controller. (See
also Rob Wilson on exploitative control).
17
systems that are specifically designed so as to
constantly search for opportunities to make the
most of body and world, checking for what is
available, and then (at various time-scales and
with varying degrees of difficulty) integrating
it deeply, creating new functional wholes.
i.e. a dynamically adaptive form of ecological
control.
18
Any control system that is highly engineered so
as to be able to learn to make maximal
problem-simplifying use of an open-ended variety
of internal, bodily, or external sources of
order. dynamically adaptive ecological
control
19
  • Ecological control names an overall effect not
    a single mechanism.
  • That effect (the achievement of delicate adaptive
    balances between environmental, neural and
    bodily dynamics) comes in many degrees and
    flavors, all the way from more-or-less
    hard-wired ecological balances to
    learnt-on-the-fly ecological balances.

20

Stelarc
21
The third hand is controlled by EMG signals
detected by electrodes placed on four strategic
muscle sites on Stelarcs legs and abdomen. The
third hand is controlled by Stelarcs brain via
muscle commands to these sites that are then
relayed to the prosthesis. Since these
sites are not normally used for hand control, the
third hand can be moved independently of the
other two
22
Stelarc simply feels as if he wills the third
hand to move, just as he wills his biological
hands to move. In each case, the control is
fluent and intuitive. It seems to require no
special effort or conscious focus.
23
Normally, you dont feel as if you are (for
example) USING your hand to do the washing
up. Instead, you just feel as if YOU are
washing up. Your hand functions as what some
philosophers call TRANSPARENT EQUIPMENT
equipment through which you can act on the world
without first willing an act on anything
else. (see Heidegger (1927) on the
ready-to-hand)
24
The same is true in the biological case. The
human infant must learn, by trial and error and
practice, which neural commands bring about which
bodily effects,and must then practice until she
is skilled enough to issue those commands without
conscious effort (so the body becomes transparent
equipment)
25
A monkey, with implanted electrodes monitoring
brain activity, learns to control a joystick to
move a cursor to get rewards. The monitoring
computer learns what neural commands correspond
to what joystick motions. Next, the joystick is
disconnected. The monkey discovers, though, that
it can still use its own neural commands ( as
transmitted by the implanted electrodes and
decoded by the monitoring computer ) to directly
control the cursor. Finally, the commands are
diverted to control a distant robot arm, whose
motions are reflected in the on-screen cursor
movements, thus closing the loop.
26
Picture from New Scientist webpage
27
the dynamics of the robot arm (reflected by the
cursor movements) become incorporated into
multiple cortical representationswe propose that
the gradual increase in behavioral
performanceemerged as a consequence of a plastic
re-organization whose main outcome was the
assimilation of the dynamics of an artificial
actuator into the physiological properties of
frontoparietal neurons Carmena et al Public
Library of Science Biology Vol 1 22003 p.205
28
Whereas modest embodiment treats the body as a
fixed (though highly significant) resource,
profound embodiment is characterized by
constant learning and re-calibration. Biological
forms of embodiment, unlike a lot of current work
in robotics, all tend towards the profound end
of this spectrum, though we primates seem
especially plastic and well-engineered for
multiple embodiment and fluent tool -use.
29
Even the minds that, in the movie The Matrix,
populate the Matrix dream-world count as
profoundly embodied, since those minds display
the same adaptive ecological control abilities as
our own. For example, a Matrixer could learn to
fluently incorporate a Stelarc-style third-hand,
or to use a thought-controlled robot arm. It
is just that the physical dynamics of the new
components would be held in place by the
Machines computer simulation rather than worldly
physics.
30
These kinds of minds are promiscuously
body-and-world exploiting. They are forever
testing and exploring the possibilities for
incorporating new resources and structures deep
into their problem-solving regimes. They are
indeed the minds of Natural-Born Cyborgs
(shameless plug) systems continuously
re-negotiating their own limits, components, and
(as well next see) data-stores and interfaces.
31
  • Two Questions v
  • 2.Embodiment What Really
  • Matters?v
  • 3.Sensing
  • 4. Incorporation versus Use
  • 5.Mind and Reason
  • 6.A Science of Hybrids?

32
Tactile Visual Sensory Substitution
(TVSS) Work by Paul Bach y Rita and
colleagues
33
STOP feeling the tickling on the back and START
to report rough, quasi-visual experiences of
looming objects etc. After a while, a ball
thrown at the head causes instinctive and
appropriate ducking. The causal chain is
deviant it runs via the systematic input to
the back. But the nature of the information
carried, and the way it supports the control of
action, is distinctive of the visual modality.
34
TVSS systems have been sufficient to perform
complex perception and eye-hand co-ordination
tasks. These have included face recognition,
accurate judgment of speed and direction of a
rolling ball with over 95 accuracy in batting
the ball as it rolls over a table edge, and
complex inspection-assembly tasks. Bach-y-Rita
2001
35
The head-mounted camera was under the subjects
motor control. This meant that the brain could,
in effect experiment via the motor system, giving
commands that systematically varied the input, so
as to begin to form hypotheses about what
information the tactile signals might be
carrying. For example, you hear someone
approaching from the left, turn the camera that
way, and see what tactile pattern corresponds to
this event
36
The motor system operating the camera could be
changed, eg to a hand-held camera, with no loss
of acuity. The touch pad, too, could be moved
to new bodily sites. Also, there was no
confusion an itch scratched under the grid
caused no visual effects. (see Bach y Rita and
Kercel Sensory Substitution and the
Human-Machine Interface Trends in Cognitive
Sciences 7122003)
37
(No Transcript)
38
Leprosy patients who have lost feeling in their
hands. Fitted with a sensor-laden glove that
transmits signals to a forehead mounted tactile
disc-array, they report feeling sensations of
touch at the fingertips . This is because the
motor-control over the sensors runs via commands
to the hand, so the sensation is projected to
that site. Bach y Rita and Kercel op
cit. (notice that this has clear implications
for tele-presence based touch, etc).
39
Tactile Flight Suit (US Navy) Jacket delivers
small puffs of air controlled by complex sensors
that determine if a plane or helicopter is
tilting to the right or left or forward or
backward. The pilot feels a puff-induced
vibrating sensation on the side of the body
corresponding to the direction of tilt, and can
control the vehicles response by moving their
body so as to cancel the puff/vibration.
40
The suit is so good at transmitting and
delivering information in an intuitive way that
it allows even inexperienced helicopter pilots to
perform difficult tasks such as holding the
helicopter in a stationary hover, while military
fighter pilots can use it to fly blindfold. The
suit thus rapidly links the pilot to the aircraft
in the same kind of closed loop interaction that
linked Stelarc and the third hand, or the monkey
and the robot arm, or the blind person and the
TVSS system While wearing the suit, the
helicopter itself behaves very much like an
extended body/sensory sheath for the pilot.
41
What matters, in each case, is the provision of
closed-loop signaling so that motor commands
affect sensory input. What varies is the amount
of training (and hence the extent of deeper
neural changes) required to fully exploit the new
agent-world circuits thus created.
42
The specific details of the (old or new)
circuitry by which the world is engaged fall
transparent in use. The conscious agent is
aware of the oncoming ball, not of seeing the
ball, or (by the same token) of using a tactile
substitution channel to detect the ball. The
pilot becomes aware of the planes tilt and
slant, not of the puffs of air
43
Integrated but constantly negotiable platforms
of sensing, moving and (as well see later)
reasoning. Platforms able to fluidly incorporate
new bodily and sensory kit so as to engage (in
the service of goal-directed activity) a larger
and potentially hostile world. Profoundly
Embodied Agents
44
  • Two Questions v
  • 2.Embodiment What Really
  • Matters?v
  • 3.Adaptive Ecological Controlv
  • 4. Incorporation versus Use
  • 5.Mind and Reason
  • 6.Can there be a Science of Hybrids?

45
You are making quite a song and a dance out of
this, what with talk of incorporation of new
bodily and sensory kit and so on. But we all
know we can use tools and stuff, and learn to use
them better (more transparently if you
must). Why talk of extended bodies and
reconfigured users, rather than talk of the same
old embodied user just being in command of a new
tool?
46
Any control system that is highly engineered so
as to be able to learn to make maximal
problem-simplifying use of an open-ended variety
of internal, bodily, or external sources of
order. dynamically adaptive ecological
control
47
Pre-motor, parietal and putaminal neurons that
respond both to somatosensory information from a
given body region (ie the somatosensory Receptive
Field sRF) and to visual information from the
space (visual Receptive FieldvRF) adjacent to
it Maravita and Iriki Tools for the body
(schema) Trends in Cognitive Sciences vol
822004, p. 79 For example, some respond to
somatosensory stimuli (light touches) at the
hand AND to visually presented stimuli near the
hand, so as to yield an action-relevant coding
of visual space.
48
After 5 minutes of rake-use, the responses of
some bi-modal neurons whose original vRFs picked
out stimuli near the hand now expanded to
include the entire length of the tool, as if the
rake was part of the arm and forearm (op
cit). And other bi-modal neurons, that
previously responded to visual stimuli within the
space reachable by the arm, now had vRFs that
covered the space accessible by the arm-rake
combination.
49
Such vRF expansions may constitute the neural
substrate of use-dependent assimilation of the
tool into the body-schema, suggested by classical
neurology And note that any expansion of the
vRF only followed active, intentional use of the
tool not its mere grasping by the hand (op cit
80,81). (see also work on haptic and dynamic
touch- eg Turvey and Carello (1995))
50
In a patient whose neglect selectively affected
space close to (one side of) the body, use of a
stick extends the neglect to the whole area
reachable by the tool. See Berti and Frassinetti
When Far Becomes Near Re-mapping of space by
tool use Journal of Cognitive Neuroscience 12
(2000) 415-420
51
The brain makes a distinction between far
space (the space beyond reaching distance) and
near space (the space within reaching
distance) and that simply holding a stick
causes a remapping of far space to near space. In
effect the brain, at least for some purposes,
treats the stick as though it were a part of the
body Berti and Frassinetti (op cit)

52
The changes (for most tool uses) take time to
occur, and lag slightly behind fluency of
use. No doubt they contribute to fluency of use.
But they do so in a rather distinctive
fashion. Contrast Representing the shape and
dimensions of a hand-held tool and then INFERRING
that you can now reach such and such an object,
and altering the shape of these vRFs ( perhaps
other neural tweaks too) so that objects within
range are now automatically classified as
reachable.
53
What I dubbed adaptive ecological control is
actually achieved by the use of neural resources
that are multiply and delicately tuneable so as
to factor in the properties of a soft-assembled
system of components. This is not the same as
just representing those properties and features
and then engaging in micro-managed deployment of
resources. Rather, the effect is to create new
body-schemas pretty much on-the-fly
54
The genuine sense in which we are OF our world
and not just IN it. See Heidegger, Haugeland,
Varela.
55
  • Two Questions v
  • 2.Embodiment What Really
  • Matters?v
  • 3.Sensingv
  • 4. Incorporation versus Usev
  • 5.Mind and Reason
  • 6.Can there be a Science of Hybrids?

56
Could anything like the notion of incorporation
(rather than mere use) get a grip in the domain
of mind and cognition? Could human minds be
extended and augmented by technological tweaks,
or is it still just the same old mind with a
shiny new tool (I think this is often
evolutionary psychologys line)?
57
External information-processing resources are
also apt for temporary or long-term recruitment
and incorporation by processes quite analogous to
those of basic adaptive ecological control. To
the extent that such processes operate, we are
not just bodily and sensorily but also
cognitively permeable agents.
58
Dennett, Hutchins, Donald, Wilson, Vygotsky,
Varela, Thompson Rosch, Bruner, Norman,
Heidegger, Gregory, Gibson, Merleau-Ponty Bates
on just fill in your favorites.
59
Our cognitive permeability is related to the
ability of the brain to learn complex routines
that make implicit commitments to the robust
availability of certain bodies of information
while carrying out on-line problem solving. This
manifests as the delicate temporal tuning of
multiple automatic calls to interacting
sub-routines (including calls to bodily action
and motion) that simply factor in that
availability. (for two nice, and quite different
examples, see Kirsh and Maglios (1991) account
of expert skill at Tetris and Ballard et als
(2000) account of performance in a
block-copying task)
60
The experiential reflection of this kind of
systemic commitment is that once specific bodies
of information are thus factored in we feel as
if we already know them. This feeling is better
glossed as already being in command of the
information. This may be the informational
equivalent of equipment falling transparent in
use
61
Our ability to perform fast, frequent,
intelligent (Yarbus (1967)) saccades allows us to
make maximal use of any stable local environment
as a store of information (a kind of external
memory) See ORegan (1992), Noe and ORegan
(2000) In this way, some say, the stable local
environment becomes poised for real incorporation
into information-driven problem solving
routines. See eg work by Ballard and colleagues
(eg 2000), Churchland et al (1994).
62
Experimental evidence from so-called Change
Blindness (as well as magic tricks and undetected
film continuity errors) shows dramatic gaps in
our (on-the-spot, conscious) knowledge. (Big
lit. and complex issues, including substantial
non-conscious uptake) And yet we feel as if we
are constantly aware of a great deal of detail
concerning our current visual surroundings.
63
The way that a large body of information
concerning the rich and varied detail in the
current scene is indeed poised, in normal
circumstances, for active retrieval and use in
the service of our current goals. Most of the
information is left out in the world, but its
easy availability is what yields our (correct)
sense of knowing a rich and detailed
scene. (hence my paper rejecting the claims of a
grand illusion in Journal of Consciousness
Studies (2002)).
64
The feeling of seeing all that detail in the
scenes is really a reflection of something
implicit in the overall problem-solving
organization in which vision participates. That
organization assumes the (ecologically normal)
ability to retrieve more detailed info when
needed, so we feel (correctly, in an important
sense) that we are already in command of the
detail.
65
Compare Your feeling that you know what month
this is. Its not due to your constantly
rehearsing the answer in your conscious mind
(always saying, April, April, April). Rather,
it is due to your implicit knowing that this is
the kind of thing you know and can usually
retrieve (in this case from bio-memory) at will.
66
If confronted by a repeated series of this
-months- name involving problems, the brain
would simply build a set of interlocking routines
that took for granted the availability, at low
metabolic cost, of that information.
67
Just as the experienced brain need not (though it
sometimes can) represent the shape of the tool
and infer the available reach, so too it need not
(though it sometimes can) first represent the
availability (externally or internally) of the
information and then infer that it can find what
it needs by accessing a given resource. When no
such processes of resource representation and
inference intervene, the tool, whether physical
or informational, should be counted as
(temporarily) incorporated into the
problem-solving whole. (establishing this was
the main point of Clark and Chalmers (1998))
68
Gray and Fu (Cognitive Science, In Press) Task
Programming a VCR (on-screen simulation) Set-up
so that successful programming could be achieved
either by retrieval of knowledge from the world
or from biological memory. From-world retrieval
involved saccading to an on-screen box, which in
one condition (free access) was plainly visible
and in another (gray box) required uncovering by
mouse move and click Experimenters thus
systematically varied the effort required to
deploy each source of information, using
time-taken as the measure of effort
69
(No Transcript)
70
Results Milliseconds matter! The subjects
robustly settled on whatever strategy yielded the
least-cost (measured by time) information
retrieval. Whichever mix yielded the
least-effort solution (with effort measured by
time) was recruited and calls to that date store,
whether internal or external, became built-in
into the dominant strategy. (This was so even
when the least-effort-by-time solution was
non-optimal for accuracy)
71
the time spent retrieving something from memory
is weighed the same as time spent in
perceptual-motor activity and that it is
therefore a mistake to presume the privileged
status of any location or type of
operation Gray and Fu (Cognitive Science, in
press)
72
The cognitive control of interactive behavior
minimizes effort by using a least-effort
combination of all the mechanisms available to
it. All mechanisms or sub-systems are on the
table. There is no reason to think that one
mechanism or subsystem has a privileged status in
relation to another The embodiment level is
the right level of description for functional
cognition Gray and Fu (Cognitive Science, in
press)
73
Body parts, sensory channels, non-biological body
parts, sensory subsitution systems, internal
information storage and processing, and external
information storage and processing are all fair
game for adaptive ecological control. They are
just resources apt for dynamic recruitment into
the whirl of problem-solving activity and hence
for incorporation, not just use.
74
The integrative processes by which the brain
adapts to control interaction are relatively
agnostic concerning the source of the structure
participating in the process... Christensen
(In Press)
75
  • Two Questions v
  • 2.Embodiment What Really
  • Matters?v
  • 3.Sensingv
  • 4.Incorporation versus Usev
  • 5.Mind and Reasonv
  • 6. Can there be a Science of Hybrids?

76
Adams and Aizawa (2001) Transcranialism The
transcranialist (yuch) holds that cognitive
processes extend in the physical world beyond the
bounds of the brain and the body(43) (typical
offenders include Clark and Chalmers, Clark,
Dennett, Donald and Hutchins)
77
Science tries to carve nature at its joints
(51) But the causal arrangements whereby
external stuff contributes to reason and action
are very different to those whereby internal
stuff does. For example, , biological memory
systems display a number of what appear to be
law-like regularities, including primacy
effects, recency effects, chunking effects and
others (61) By contrast, our external props and
tools (notebooks, PCs, rolodexes) form a varied
bunch, with nothing much in common either with
each other or with the biological substrate itself
78
transcranial extended processes are not
likely to give rise to interesting scientific
regularities. There are no laws covering humans
and their tool-use over and above the laws of
intercranial inner human cognition and the
laws of the physical tools (61)
79
The work by Nicolelis, by Bach-y-Rita, and by
Gray and Fu are all examples (in very different
domains) of the systematic exploration of hybrid
(biological and non-biological) systems that
function (on various time-scales) as integrated
problem-solving wholes.
80
There are (at least) two perfectly legitimate
objects of study hereabouts the multiple and
exquisite forms of bio-side plasticity that
support this ability and the new functional
wholes themselves
81
Neural plasticity has long been treated as if it
were just a ploy designed to cope flexibly with
early insult or injury, and one whose prime
efficacy is limited to an early developmental
window. As if it were a reactive rather than
an active property of the immature system
(240) Joan Stiles Neural Plasticity and
Cognitive Development Developmental
Neuropsychology 18 2 2000 p. 237-272
82
The construct of plasticity is best defined by
complex dynamic processes in which the biological
system progressively adapts to contingencies of
input and the demands of the learning
environment. They are notancillary, optional or
reactive. Rather, plasticity is a fundamental and
essential propertynot unique to
development Mature form and function emerge as
stabilized plasticity (see Christensen, In
Press). Stiles, op cit p 252
83
First, the neural plasticity and adaptivity does
not itself constitute the knowledgeable agent so
much as a resource that underpins an ongoing and
never-ending search for participative components
and information stores. It is just the
biological bootstrap that allows our full
physical and cognitive agency to come into
being.
84
Second, the new functional wholes that are
discovered by the operation of the system are
often themselves the integrated systems that
participate in subsequent rounds of
resource-seeking and annexation. (A simple
example was Stelarc learning to use that pencil
with the third-hand)
85
ratchet effects whereby hybrid systems (level
1) come to incorporate still further layers of
structure, becoming hybrid systems (level 2) that
go on to seek out still further layers of
structure, etc. (for a nice exploration of
this in the cognitive realm, see Kim Sterelnys
new book Thought in a Hostile World)
86
A kind of basic science of the biological
plasticity and multiple specific adaptations that
originally poised us for the long journey into
hybrid space. and A science of the new
functional wholes that (repeatedly) result, and
of the ratchet effects in which they participate.
87
What are the basic mechanisms of adaptive
ecological control? What is the basic tool-kit
for understanding adaptive ecological control,
and does it require the use of more than the
standard information-processing notions of
representation and computation (e.g. dynamical
field theory)?
88
What extra tweaks enable we humans to be such
world-class experts at such control? What is the
basic tool-kit for understanding hybrid
functional wholes, and the complex
cultural-evolutionary scenarios in which they
participate?
89
What exactly happens when biological endowments
for plasticity and for adaptive ecological
control interact with material symbolic artifacts
such as spoken words and written
inscriptions? Given all that plasticity and
permeability, how should we best identify and
analyze minds, persons, agents, environments, and
perceptual and cognitive systems? What about the
cognitive/non-cognitive divide itself does it
still make sense when everything is just a
resource on a level playing field?
90
Agents for whom body, sensing, world and
technology are all just resources for soft
assembled processes of dynamically adaptive
ecological recruitment, yielding a permeable and
repeatedly reconfigurable agent/world boundary.
91
For the profoundly embodied agent, the world is
not something locked away behind the fixed veil
of a certain skin-bag, a reasoning engine and a
primary sensory sheath. Instead, it is a
resource apt for active recruitment and use, in
ways that bring new forms of embodied
intelligence into being.
92
Short bibliography Bach-y-Rita, P and Kercel, S
(2003) Sensory Substitution and the
Human-Machine Interface Trends in Cognitive
Sciences 712541-546 Carmena, J et al (2003)
learning to Control a Brain-Machine Interface
for Reaching and Grasping by Primates Public
Library of Science Biology 12
193-208 Available free in the archive at
http//www.plosbiology.org/plosonline/?requestin
dex-html Clark, A (2003) Natural-Born Cyborgs
Minds, Technologies and the Future of Human
Intelligence (Oxford University
PressNY) González, J and Bach-y-Rita, P (ms)
Perceptual Adaptive Recalibration Tactile
Sensory Substitution in Blind Subjects Gray, W
and Fu, W (In Press) Soft constraints in
interactive behavior the case of ignoring
perfect-knowledge in-the-world for imperfect
knowledge in-the-head Cognitive
Science Mussa-Ivaldi, F and Miller, L (2003)
Brain-machine interfaces computational demands
and clinical needs meet basic neuroscience
Trends in Cognitive Sciences 266329-334
Write a Comment
User Comments (0)
About PowerShow.com