Visual Programming Languages ICS 519 Part 1

1
Visual Programming LanguagesICS 519Part 1

• ICS Department
• KFUPM
• Sept. 15, 2002

2
1Introduction

• Learning programming is a difficult task.
• Writing and testing a program is a difficult and
  time consuming task.
• The majority of computer users are non
  programmers.
• Programming languages are designed for
  professional programmers and computer scientists.

3

• There are historical reasons for this lack of
  emphasis on human convenience in programming
  languages (Shu, 1988).
• The computer scientists, for the last fifty
  years, have concentrated on machine efficiency.
• Computing costs were high and human costs were
  relatively low.
• In the early years of computing, it was
  justifiable that efficiency took precedence over
  ease of programming.

4

• With the decline in computing costs it is
  realized that it is the human resource that must
  be optimized.
• The challenge is to bring computer capabilities
  to people without special computer training.
• To meet the challenge a number of facilities have
  been introduced to ease the pain of programming
  and remembering commands.
• These facilities include icons, pointing devices,
  and menus.

5

• Programming languages have evolved over the last
  fifty years from first generation programming
  languages to fourth generation programming
  languages.
• One characteristic of most of these languages is
  that they use one-dimensional and textual formats
  to represent programs.
• These formats are suitable for sequential
  machines but not for the multi-dimensional and
  visual human mind (Chang, 1986).

6

• Visual programming is an attempt to simplify
  programming and to make better use of the human
  mind's capabilities
• George Raeder states "... human mind is strongly
  visually oriented and that people acquire
  information at a significantly higher rate by
  discovering graphical relationships in complex
  pictures than by reading text. ... When we scan
  paragraph, headlines, and material in bold type
  to speed up text search, we are really acting in
  the pictorial mode to overcome this limitation"
  (Raeder, 1985).

7

• Visual programming represents a conceptually
  revolutionary departure from the traditional
  one-dimensional and textual programming
  languages.
• It is stimulated by the following premises
• 1) Pictures are more powerful than words as a
  means of communication. They can convey more
  meaning in a more concise unit of expression.
• 2) Pictures aid understanding and remembering.
• 3) Pictures may provide an incentives for
  learning program.

8

• 4) Pictures do not have language barriers. When
  properly designed, they are understood by people
  regardless of what language they speak.

9

• Visual programming has gained momentum in recent
  years.
• The falling costs of graphics-related hardware
  and software has made it feasible to use pictures
  as a means of communicating with the computers.
• Visual programming is a very young but growing
  field.
• There is no common agreement on the definition of
  visual programming.

10
2Definitions

• Visual programming languages is a new paradigm.
• There is no single formal definition for VPL. For
  this reason we will present a number of
  definitions that appear in the literature.

11

• 1) Chang, 1987
• Visual programming languages, usually deals with
  objects that do not have an inherent visual
  representation. This includes traditional data
  types such as arrays, stacks, and queues and
  application data types such as forms, documents,
  and databases. Presenting these objects visually
  is helpful to the user.
• For the same reason, the languages themselves
  should be represented visually.
• In other words, both programming constructs and
  the rules to combine these constructs should be
  presented visually.

12

• 2) Shu, 1988
• A visual programming language can be informally
  defined as a language which uses some visual
  representations ( in addition to or in place of
  words and numbers) to accomplish what would
  otherwise have to be written in a traditional
  one-dimensional programming language.
• Note that this definition imposes no restrictions
  on the type of data or information.
• It is immaterial whether the object being
  operated on or being displayed to a user by a
  visual language is textual, numeric, pictorial,
  or even audio.
• What is important is that the language itself
  must employ some meaningful visual expressions as
  a means of programming.

13

• 3) Tortora, 1990
• A visual language models an icon system. In the
  icon system, a program consists of a spatial
  arrangement of pictorial symbols ( elementary
  icons ).
• Note that, the spatial arrangement is the
  two-dimensional counterpart of the sequential
  statements in the traditional programming
  languages.
• In these languages, a program consists of a
  string in which terminals are concatenated.
• Concatenation is the only construction rule,
  therefore it doesnt appear in the language
  grammar.

14

• In the case of visual languages, three
  construction rules are used to spatially arrange
  icons
• Horizontal concatenation.
• Vertical concatenation.
• Spatial overlay.
• For this reason, there is a need to include both
  spatial operators and elementary icons in the
  vocabulary of terminal symbols of the visual
  language grammar.
• Elementary icons are partitioned into two
  categories
• Object icons which identify objects.
• Process icons which express computation.

15

• 4) Al-Mulhem, 1993
• Visual programming languages is a language which
  uses a combination of text and meaningful
  graphical symbols to write programs. These
  graphical/textual symbols represent the
  following
• a) Data types such as numbers, characters,
  pictures, and sound.
• b) Data structures such as arrays, records, and
  pointers.
• c) Programming constructs such as iteration,
  selection, and conditional looping.
• d) Rules to combine the programming constructs.

16

• 5) Burnett (1999)
• Visual programming is programming in which more
  than one dimension is used to convey semantics.
• Examples of such additional dimensions are the
  use of multi-dimensional objects, the use of
  spatial relationships, or the use of the time
  dimension to specify before-after semantic
  relationships.
• Each potentially-significant multi-dimensional
  object or relationship is a token (just as in
  traditional textual programming languages each
  word is a token) and the collection of one or
  more such tokens is a visual expression.

17

• Examples of visual expressions used in visual
  programming include diagrams, free-hand sketches,
  icons, or demonstrations of actions performed by
  graphical objects.
• When a programming languages (semantically-signif
  icant) syntax includes visual expressions, the
  programming language is a visual programming
  language (VPL).

18
VPL Research

• Directions of research in this area are
• Visual approaches to traditional programming
  languages (flowchart).
• Visual approaches that deviated from traditional
  programming (programming by demonstration).

19
How good are they?

• Good for toy programs.
• Bad for realistically-sized programs.

20
More approaches

• Incorporating visual techniques into programming
  environments
• To support textual specification of GUI layout
• To support electronic forms of software
  engineering diagrams.
• To create/visualize relationships among data
  structures.
• To visually combine textually-programmed units to
  build new programs.

21
Examples

• Visual Basic (for BASIC)
• VisualWork (for SmallTalk)
• CASE tools that support visual specificatio of
  relationships among program modules, automatic
  code generation, etc.

22
(No Transcript)
23
More approaches

• Domain-specific visual programming systems
• Examples
• LabVIEW programming laboratory data acquestion.
• AVS programming Scientific Visualization
• PhonePro programming telephone and voice-mail
  behavior.
• Cocoa programming graphical simulations and
  games.

24
What is next?

• VPLs for general-purpose programming.
• This can be reached by
• Improving the ways VP can be used.
• Improving domain-specific programming

25
Strategies used in VPLs

• Concrete specify some aspect of semantics on a
  specific object or value.
• Direct manipulate a specific object or value
  directly to specify semantics.
• Explicit describe dataflow relationships by
  drawing directed edges among related variables.
• Immediate visual feedback automatic display of
  effects of program edits.

26
Classification

1. Pure visual programming
2. Hybrid (textual / visual) programming
3. Programming-by-example
4. Constraint-based programming
5. Form-based programming

27
Pure visual programming

• Programmer manipulate icons to create a program.
• A program is debugged and executed in the same
  visual environment.
• A program is compiled directly from its visual
  representation.
• A program is never translated into intermediate
  text-based language.
• Examples Prograph, VIPR, and PICT.

28
Hybrid (textual / visual) programming

• Two forms
• 1. Systems in which programs are created
  visually and then translated into an underlying
  high-level textual languages.
• Example Rehearsal World the user trains the
  system to solve a particular problem by
  manipulating graphical actors (icons) and then
  the system generates a Smalltalk program to
  implement the solution.

29
Hybrid (continue)

• 2. Systems which involve the use of graphical
  elements in a textual language.
• Example Work of Erwig et. al. developing
  extensions to languages like C and C which
  allow programmers to mix their text code with
  diagrams.
• For instance, one can define a linked list data
  structure textually and then perform an operation
  like deletion of a node by drawing the steps in
  the process.

30
Programming by example

• Allowing the user to create and manipulate
  graphical objects with which to teach the system
  how to perform a particular task.
• Examples Rehearsal World and Pygmalion

31
Constraint-oriented programming

• A programmer models physical objects as objects
  in the visual environment which are subject to
  constraints designed to mimic the behavior of
  natural laws, like gravity.
• Popular for simulation design.
• Examples ThingLab, ARK

32
Form-based programming

• Borrowed their visualizations and programming
  metaphors from a spreadsheets.
• Represent programming as altering a group of
  interconnected cells over time and often allow
  the programmer to visualize the execution of a
  program as a sequence of different cell states
  which progress through time.
• Example Forms/3

33
VPL Orphans

• Algorithm animation systems
• Provides interactive graphical displays of
  executing programs
• Example BALSA
• user-interface design systems
• Provided with many modern compilers.
• Example Visual Basic, Visual C

34
Theory

• Based on work by Chang
• Definitions
• Icon (generalized icon) An object with the dual
  representation of
• logical part (the meaning)
• Physical part (the image)
• iconic system structured set of related icons
• iconic sentence (visual sentence) spatial
  arrangement of icons from iconic system

35
Theory (con.)

• visual language set of iconic sentences
  constructed with given syntax and semantics
• syntactic analysis (spatial parsing) analysis of
  an iconic sentence to determine the underlying
  structure
• semantic analysis (spatial interpretation)
  analysis of an iconic sentence to determine the
  underlying meaning

36
Theory (con.)

• A visual Language Models an Icon System.
• In the Icon System, a program (Visual Sentence)
  consists of a spatial arrangement of pictorial
  symbols (elementary icons).
• The spatial arrangement is the two-dimensional
  counterpart of the standard sequentialization in
  the construction of programs in the case of
  traditional programming languages.

37

• In traditional languages, a program is expressed
  by a string in which terminals are concatenated.
• As this operation (concatenation) is the only
  construction rule, it does not appear explicitly
  in the vocabulary of the language grammar.
• In the case of visual languages, three
  construction rules are used to spatially arrange
  icons
• Horizontal concatenation
• Vertical concatenation
• Spatial overlay

38

• For this reason, there is a need to include the
  spatial operators and elementary icons in the
  vocabulary of terminals of the visual language
  grammar.
• The elementary icons are partitioned into
• object icons
• process icons.
• Elementary object icons identify objects while
  process icons express computations.
• The meaning depends on the specific visual
  sentence in which it appears.

39
Example

• Consider the following primitive icons from the
  Heidelberg icon set.
• where
• the square denotes a character, and its
  interpretation is unique in the system.
• the arrow can be used to express insertion or
  moving operations in different visual sentences.

40

• A Visual Language (VL) is specified by the triple
  
• (ID, Go, B)
• where
• ID is the icon dictionary
• Go is a context-free grammar
• B is a domain-specific knowledge base.

41
The icon dictionary ID

• It is a set of triples of the form
• i (il , ip , type(i))
• Each triple i describes a primitive icon where
• il is the icon name (logical part or meaning)
• ip is the icon sketch (physical part)
• type(i) is the icon type (process or object)

42
The grammar Go

• It is a context-free grammar (N, T, S, P).
• Where
• N is the set of nonterminals
• T Tl U Tsp
• Tl il i ( il, ip, type(i)) ? ID
• Tsp , ,
• S is the start symbol of the grammar
• P is the set of productions of Go

43

• The grammar Go specifies how more complex icons
  can be constructed by spatially arranging
  elementary icons.
• Usually, nonterminals in Go the start symbol
  represent composite object icons.
• Composite object icons can be derived in one or
  more steps starting from a given nonterminals, N,
  which is different from S.
• In other words, composite object icons are
  obtained by spatial arrangement of elementary
  object icons only and are assumed to be of type
  object.

44

• Visual Sentences contain at least one process
  icon.
• Visual Sentences have no type.
• Here, object icons denote both elementary object
  icons composite object icons.
• It is easy to prove that all the icons involved
  in the icon system (elementary object, composite
  object, and visual sentences) are objects with
  dual representation of a logical and a physical
  part as follows

45

• Elementary object --- by definition in ID.
• Composite object and visual sentences ---
• The physical part is their image
• The logical part (the meaning) can be derived
  from the meaning of the elementary objects
  occurring in them.
• The meaning of a visual sentence is expressed in
  terms of a conceptual tree, which represents an
  intermediate code for later execution.

46
Conceptual Trees

• The meaning of a visual sentence is expressed in
  terms of a conceptual tree (CT), which represents
  an intermediate code for later execution.
• In CT notations a concept is denoted by ,
  and a relation is denoted by ( ).

47

• The concepts are expressed in the form
• OBJECT object-name
• EVENT event-name
• where
• object-name represents the logical part (meaning)
  of an object icon (elementary icon or composite
  object icon)
• event-name are event names, i.e., procedure
  names, which accomplish the task expressed by the
  visual sentence

48

• The following CTs are basic substructures
  occuring in all the CTs which represent the
  logical part of a visual sentence.
• EVENT event-name (rel1) OBJECT
  object-name1
• (rel2) OBJECT object-name2 (1)
• EVENT event-name (rel) OBJECT
  object-name (2)
• OBJECT object-name (rel) OBJECT
  object-name (3)
• The meanings of these CTs are as follows

49

• 1) A process icon has two object icons as
  arguments
• Example
• consider the following icon from the Heidelberg
  icon set

50

• It is composed of the object icons
• Selected-string
• String
• and the process icon Up-arrow

51

• Its meaning is expressed by the CT
• EVENT replace (object) OBJECT
  String
• (place) OBJECT Selected-string

52

• 2) A process icon requires only one argument.
• Example

53

• It is obtained by composing in spatial overlay
• the object icon Selected-string
• and the process icon cross

54

• Its meaning is described by the CT
• EVENT delete (object) OBJECT
  Selected-string

55

• 3) Here we have two combined object icons. This
  is the case in which one of the two objects
  involved in the visual sentence represents a
  qualitative or quantitative specification of the
  other object.
• Example
• The object "name identifies the object "file"

??????????? ??????????? ???????????
File name
56

• The meaning is
• OBJECT file (identifier)
  OBJECT name

57

• Complex structures can be obtained by composing
  the basic substructures as shown below
• 1 2 3
• This visual sentence could mean append file1 to
  file2 and copy the result into file3

??????????? ??????????? ???????????
??????????? ??????????? ???????????
??????????? ??????????? ???????????
58

• The sentence could be decomposed into two
  subsentences
• 1 2
• and
• 2 3

??????????? ??????????? ???????????
??????????? ??????????? ???????????
??????????? ??????????? ???????????
??????????? ??????????? ???????????
59

• These subsentences are described, respectively,
  by the following CTs
• EVENT append (object) OBJECT
  file1
• (place) OBJECT file2
• EVENT copy (source)
  OBJECT file2
• (destination) OBJECT file3

60

• The whole visual sentence is described by
• EVENT copy (destination) OBJECT
  file3
• (source) EVENT append
  
• (object) OBJECT
  file1
• (object) OBJECT file2

61
The Knowledge Base B

• The Knowledge Base B contains domain-specific
  information necessary for constructing the
  meaning of a given visual sentence.
• It contains information regarding
• Event names
• Conceptual relations
• Names of the resulting objects
• References to the resulting objects.

62

• It is structured in the following seven tables
• 1) EVENT-NAME proc-name, obj-name1, obj-name2,
  op1, op2 This table returns the name of the
  event (procedure) to be associated to the process
  icon proc-name when the objects obj-name1,
  obj-name2 are spatially arranged (via op1, op2)
  to proc-name.
• 2) LEFT-REL proc-name, obj-name1, obj-name2,
  op1, op2
• 3) RIGHT-REL proc-name, obj-name1, obj-name2,
  op1, op2
• These two tables hold the conceptual relations
  existing between EVENT-NAME Proc-name, obj-name,
  obj-name2, op1, op2 and obj-name1 or obj-name2
  respectively.

63

• 4) RESULT-NAME Proc-name, obj-name1, obj-name2,
  op1, op2
• This table returns the name of the object
  resulting from the execution of the event.
• EVENT-NAME Proc-name, obj-name1, obj-name2,
  op1, op2. With arguments obj-name1 and
  obj-name2.
• 5) RESULT-REF Proc-name, obj-name1, obj-name2,
  op1, op2.
• This table returns a reference to the object
  representing the result.

64

• The following two tables take into account cases
  in which object icons are composed (composite
  object icons).
• 6) RELATION obj-name1, obj-name2, op.
• This returns the conceptual relation existing
  between obj-name1 and obj-name2 when they are
  spatially combined by means of the operator op.
• 7) RESULT-OBJECT obj-name1, obj-name2, op.
• This returns the name of the object resulting by
  the spatial combination (via op) of obj-name1 and
  obj-name2.
• In the previous tables, obj-namei refer to the
  logical part of the icons.

65

• Besides domain-specific relations the tables can
  also contain three special relations
• ltnewgt This relation means that the combination
• obj-name1 op obj-name2
• gives rise to a new object, different from
  obj-name1 and obj-name2.
• ltnullgtThis relation means that the combination
• obj-name1 op obj-name2
• generates an object that has the same logical
  part of the obj-name1 or obj-name2.
• In other words, it points out that no
  additional information is derived from such a
  combination.

66

• ltl-objgt , ltr-objgt
• These two relations mean that the combination
• obj-name1 op1 proc-name op2 obj-name2
• generates an object that has the same reference
  of the icon whose name is obj-name1 or obj-name2
  respectively.

67
file3
??????????? ??????????? ???????????
??????????? ??????????? ??????????? file2
Obj-name1
Op1
Obj-name2
Op2
Proc-name
68

• Table 1 returns the name of the event copy
• Table 2, 3
• LEFT-REL
• EVENT copy (source)
  OBJECT file2
• (destination) OBJECT file3
• RIGHT-REL

69

• Table 4 Return file 3
• Table 5 Return a reference (pointer) file3
• Table 6
• OBJECT file (identifier)
  OBJECT file3
• returns this
• Table 7 Return the name of the resulting object
  in 6

70
The Attribute Grammar

• An attribute grammar consists of a semantics
  rules for synthesizing the meaning of a visual
  sentence
• AG consists of an underlying context-free
  grammar where
• i) each nonterminal in the context-free grammar
  has associated two attribute sets, namely,
  synthesized and inherited attributes.

71

• ii) each production has associated a set of
  semantics rules, used to determine the values of
  the attributes of the nonterminals depending on
  the values of the attributes of the remaining
  nonterminals appearing in the same production.
• The initial nonterminal has only synthesized
  attributes, and one of them is designated to hold
  the meaning of the derivation tree.

72

• The production rules in Go are of the following
  form
• a) X ?t t ? T
• b) X Y where type (X) type (Y)
• c) X M op L L op M M1 op M2
• d) X M1 op1 L op2 M2
• where
• type (L) PROCESS
• type (M) type (M1) type (M2) OBJECT.
• op, op1, op2 nonterminals for spatial operators
  ( , , )

73

• The type for a given nonterminal X is determined
  as follows
• Let
• X t be a derivation in Go and t ? T
• Then type (X) spatial operator if t ?? Tsp
• type (X) type (t) otherwise.
• The set of inherited attributes is empty for each
  nonterminal in Go.

74

• The set of synthesized attributes associated to
  each nonterminal X is defined as follows.
• For a given non-terminal X
• 1. if type (X) OBJECT
• NAME (X) is the name of the object icon
  represented by X.
• CG(X) is the CT associated to the subtree rooted
  in X.
• REF(X) is the reference number. The reference
  number is an index associated with each
  elementary object icon occurring in a visual
  sentence. It allows one to distinguish between
  different occurrences of the same elementary
  object icon in a visual sentence.

75

• 2. if type (X) PROCESS
• NAME(X) is the name of the process icon
  represented by X.
• 3. if type (X) "spatial operator"
• OP(X) is the spatial operator derived from X.

76

• The semantic rules associated with the four
  production rules are
• a) X t
• a1) If t icon-id
• where
• ( icon-id, icon-sk, type( i ) ) ? ID and
• type ( i ) OBJECT (i.e. t is a
  terminal for an elementary icon).
• Then
• NAME(X) icon-id
• REF (X) ref-set
• CG (X) OBJECT NAME(X), REF(X)

77

• a2) If t icon-id
• where
• ( icon-id, icon-sk, type( i ) ) ?? ID and
• type ( i ) PROCESS
• Then
• NAME(X) t
• a3) If t ??? , L ,
• Then
• OP(X) t

78

• b) X Y
• b1) If type (Y) OBJECT
• Then
• NAME(X) NAME (Y)
• REF (X) REF (Y)
• CG (X) CG (Y)
• b2) If type (Y) PROCESS
• Then
• NAME(X) NAME (Y)
• b3) If Y op and op ??? , L ,
  
• Then
• OP(X) OP(Y)

79

• c) X M op L
• c1) If type (M) OBJECT and type (L)
  PROCESS
• Then
• NAME(X) RESULT-NAME NAME(L), NAME (M),
  null-obj, op(op), null-op
• REF(X) SET-REF (L, M, null-obj, op,
  null-op)
• CG (X) SELECTIVE-MERGE (L, M, null-obj, op,
  null-op)

80

• c2) If type (M) PROCESS and type (L)
  OBJECT
• Then
• NAME (X) RESULT-NAME NAME(M), null-obj,
  NAME(L), null-op, OP(op)
• REF(X) SET-REF (M, null-obj, L, null-op ,
  op)
• CG (X) SELECTIVE-MERGE (M, null-obj, L,
  null-op, op)

81

• c3) If type (M) type (L) OBJECT
• Then
• NAME (X) RESULT-OBJECT NAME (M),
  NAME(L), OP(op)
• REF(X) REF (M) U REF (L)
• CG (X) LINK (X, M, L, op)

82

• d) X M1 op1 L op2 M2
• If type (L) PROCESS and
• type (M1) type (M2) OBJECT
• Then
• NAME (X) RESULT-NAME NAME (L), NAME(M1),
  NAME (M2), OP(op1), OP (op2)
• REF(X) SET-REF (L, M1, M2, op1, op2)
• CG (X) SELECTIVE-MERGE (L, M1, M2, op1, op2)

83

• Semantic rules c and d make use of the following
  procedures
• Procedure LINK (X, M1, M2, op)
• rel RELATION NAME(M1), NAME (M2), OP(op)
• case rel of
• ltnewgt if NAME (X) ?? ID then
• - add the new composite object NAME(X) to
  ID
• - return OBJECT NAME (X), REF(X)
• endif
• ltnullgt return CG(M1)
• else return APPEND (CG(M1), CG(M2), rel)
• end case
• end LINK

84

• Procedure SELECTIVE-MERGE (L, M1, M2, op1, op2)
• C EVENTEVENT-NAME NAME(L),
• NAME(M1), NAME(M2), OP(op1), OP(op2)
• if OP(op1) null-op then G1 C
• else rel1 LEFT-REL NAME(L), NAME(M1),
• NAME (M2), OP(op1), OP(op2)
• G1 APPEND (C, CG(M1), rel1)
• endif
• if OP(op2) null-op then G2 C
• else rel2 RIGHT-REL NAME(L), NAME(M1),
• NAME (M2), OP(op1), OP(op2)
• G2 APPEND (C, CG(M2), rel2)
• endif
• return HEADER-MERGE (G1, G2)
• end SELECTIVE-MERGE

85

• Procedure SET-REF (L, M1, M2, op1, op2)
• ref RESULT-REF NAME(L), NAME(M1), NAME
  (M2), OP(op1), OP(op2)
• case ref of
• ltl-objgt return REF(M1)
• ltr-objgt return REF(M2)
• ltnew-objgt return NEXT-REF( )
• end case
• end SET-REF

86
The Icon Interpreter

• The system diagram of the icon interpreter is as
  follows

G0
ID
B
Visual Sentencce S
Construction of the Logical part
Pattern Analysis
Syntax Analysis
Meaning of S CG
Pattern String of S
Parse Tree of S
87

• The module Pattern Analysis" transforms the
  visual sentence into pattern string .
• Example
• The visual sentence
• is represented by the pattern string
• (char char char) cross

88

• The icon interpreter is logically divided into
  two phases
• 1. The syntax analysis is accomplished by a
  parsing algorithm for context-free grammars.
• 2. Constructing the meaning of the given visual
  sentence by evaluating the attributes of the
  nonterminal on the root of the parse tree using
  the ATTRIBUTE_EVALUATOR procedure
• The attribute evaluation is accomplished by means
  of a postorder visit of the parse tree t of the
  visual sentence S

89

• Procedure ATTRIBUTE-EVALUATOR (X).
• if X is a nonterminal in Go
• then / let p be the production applied to the
  node X in t, and let 1, ..., np be the indices
  of nonterminals in the right-hand side of p/
• for j 1 to np do
• call ATTRIBUTE-EVALUATOR (jth
  nonterminal son of X in t)
• end for
• apply the semantic rules associated to p
  so as to evaluate the attributes of X
• endif
• end ATTRIBUTE-EVALUATOR

90
Example

• This example shows an iconic command language for
  simple operating system.
• The set of elementary icons ID and the grammar
  G0, are shown in the following slides.

91
(No Transcript)
92
(No Transcript)
93

• The knowledge base B is given next.
• Each entry in the tables, which constitute the
  knowledge base, contains five items, which
  correspond, from top to bottom, to the tables
• EVENT_NAME
• LEFT_REL
• RIGHT_REL
• RESULT_NAME
• RESULT_REF

94
(No Transcript)
95

• Consider the following visual sentence
• Whose corresponding pattern string is
• FILE,1 PLUS FILE,2 ARROWFILE,3
• The parse tree for this sentence is shown next.

??????????? ??????????? ???????????
??????????? ??????????? ???????????
??????????? ??????????? ???????????
96
(No Transcript)
97

• The construction of the conceptual tree is given
  in the next table. At level 1, the resulting
  conceptual tree is given by
• EVENT copy (destination) OBJECT
  file3
• (source) EVENT append
  
• (object) OBJECT
  file1
• (object) OBJECT file2

98
Examples of VPLs

• Prograph (1989) Dataflow programming
• DataLab (1990) Programming by example
• Form/3 (1994) Form-based programming
• Clarity (2001) Functional programming

99
Prograph

• The Prograph language is an iconic language.
• Program is drawings.
• Prograph interpreter and compiler execute those
  drawings.
• Prograph is object-oriented
• Prograph supports dataflow specification of
  program execution

100
Simple program

• A simple Prograph code which sorts, possibly in
  parallel, three database indices and updates the
  database.

101
Paragraphs icons

• Prograph uses a set of icons
• classes are represented by hexagons,
• data elements by triangles,
• pieces of code by rectangles with a small picture
  of data flow code inside, and
• inheritance by lines or downward pointing arrows.

102
(No Transcript)
103
More Icons

• The representations can then be combined and used
  in a variety of language elements.
• For example, initialization methods - which are
  associated with an individual class - are
  depicted as hexagonally shaped icons with a small
  picture of data flow code inside.

104
Icon (C0nt.)

• Triangles represent instance variables in a
  class Data Window to show that theyre data.
• Hexagons drawn with edges and interiors similar
  to the instance variable triangles represent
  class variables. This associates them with the
  class as a whole while also associating them with
  data.
• Next Figure shows three more complex
  representations initialization methods ,
  instance variables, and class variables.

105
Icon (Cont.)
106
Object-Oriented

• The Prograph language is object-oriented.
• Class-based,
• Single inheritance (a subclass can only inherit
  from one parent),
• Dynamic typing
• With garbage collection mechanism based on
  reference counting.

107
Cont.

• The programmer can design and implement new
  classes,
• He can visually specifying
• the new class inheritance,
• additional instance or class variables,
• modified default values for inherited instance or
  class variables,
• and new or overridden methods.

108
Cont.

• Polymorphism allows each object to respond to a
  method call with its own method appropriate to
  its class.
• Binding of a polymorphic operation to a method in
  a particular class happens at run-time, and
  depends upon the class of the object.
• The syntax for this message send is one of the
  more unusual aspects of Prographs syntax.
• The concept of message sending per se is not
  used in Prograph.

109
Annotation

• Rather, the idea of an annotation on the name of
  an operation is used. There are four cases
• No annotation means that the operation is not
  polymorphic.
• / denotes that the operation is polymorphic and
  is to be resolved by method lookup starting with
  the class of the object flowing into the
  operation on its first input

110
Annotation (Cont.)

• 3. // denotes that the operation is polymorphic
  and is to be resolved by method lookup starting
  with the class in which this method is defined.
• // plus super annotation means that the
  operation is polymorphic and resolves by method
  lookup starting with the superclass of the class
  in which this method is defined.
• In Prograph there is no SELF construct (or a
  this construct, to use the C terminology).

111

• Prograph has an almost completely iconic syntax,
  and this iconic syntax is the only representation
  of Prograph code.
• Next Figure shows a small code fragment typical
  of the Prograph language.
• There is no textual equivalent of a piece of a
  Prograph program - the Prograph interpreter and
  compiler translate directly from this graphical
  syntax into code.

112
Example
113
Icons

• Next Figure shows most of the lexicon of the
  language (icons).
• The inputs and outputs to an operation are
  specified by small circles at the top and bottom,
  respectively, of the icons.
• The number of inputs and outputs - the
  operations arity - is enforced only at run-time.
  
• In addition, there are a variety of annotations
  that can be applied to the inputs and outputs, or
  to the operation as a whole to implement looping
  or control flow.

114
(No Transcript)
115
Spaghetti Code

• Visual languages like Prograph sometimes suffer
  from the spaghetti code problem there are so
  many lines running all over that the code for any
  meaningful piece of work actually ends up looking
  like spaghetti.
• Prograph deals with this issue by enabling the
  programmer to iconify any portion of any piece
  of code at any time during the development
  process.

116
Local

• This iconified code is called a local.
• Effectively, locals are nested pieces of code.
• There is no limit to the level of nesting
  possible with locals, and there is no execution
  penalty to their use.
• In addition, locals can be named and this naming,
  if done well, can provide a useful documentation
  for the code.

117
Example

• Next Figure is a code to build a dialog box
  containing a scrolling list of the installed
  fonts without the use of Prograph locals to
  factor the code.

118
(No Transcript)
119
Same Example

• Next Figure is the same code with the use of
  Prograph locals to factor the code.

120
(No Transcript)
121
Local (Cont.)

• Note that long names, often containing
  punctuation or other special characters can be
  used for the names of locals.
• The proper use of locals can dramatically improve
  the comprehensibility of a piece of code.
• The one negative aspect of their use is the
  so-called rat hole phenomena - every piece of
  code in its own rat hole. This can sometimes make
  finding code difficult.

122
Syntax

• Three of the most interesting aspects of the
  Prograph syntax are
• list annotation,
• injects, and
• control annotation.

123
List Annotation

• Any elementary operation can be made to loop by
  list annotating any of its inputs.
• When this is done, the operation is invoked
  multiple times - one time for each element of the
  list that is passed on this input.
• Thus, list annotation on an input causes the
  compiler to construct a loop for the programmer.
  Since this annotation can be made on a local as
  well as a primitive operation, this looping
  construct is quite powerful.

124
List Annotation (Cont.)

• In addition, list annotation can be done on a
  output.
• On an output terminal, list annotation causes the
  system to put together all the outputs at this
  terminal for every iteration of the operation and
  to pass as the final output of the terminal
  this collection of results as an output list.

125
List Annotation (Cont.)

• List annotation, then, enables the programmer to
  easily make an operation into a loop that either
  breaks down a list into its component elements
  and runs that operation on the elements, or to
  builds up a list from the multiple executions of
  an operation.
• An individual operation can have any number of
  its inputs or outputs with list annotations.

126
List Annotation (Cont.)

• Next Figure shows several examples of list
  annotation, an efficient mechanism for iterating
  over a list.
• The operation will automatically be called
  repeatedly for each element of a list, or its
  outputs will be packaged together into a list.

127
(No Transcript)
128
(No Transcript)
129
(No Transcript)
130
Inject

• Inject lets you pass a name at run-time for an
  input which expects a function.
• This is similar to Smalltalks perform, procedure
  variables in Pascal or function pointers in C.
• Suppose, for example, that you want to implement
  a function FooMe that takes two arguments a list
  of objects, and a reference to a method to be
  applied to each of those objects.

131
Inject (Cont.)

• In Object Pascal pseudo-code, this function might
  look something like this
• Function FooMe(theList TList Procedure
  DoThis(object TObject))
• Iterates through the list of objects applying
  the DoThis procedure
• BEGIN
• theList.Each(DoThis)
• END

132

• Next Figure shows a Prograph implementation of
  FooMe.
• Note that just the name - as a string - of the
  method to be applied to each object is passed to
  FooMe.
• This string is turned into a method invocation by
  the inject.

133
(No Transcript)
134
Inject (Cont.)

• The Prograph representation of inject - a
  nameless operation with one terminal descending
  into the operations icon - is a particularly
  well-designed graphic representation for this
  programming concept.
• When properly used, inject can result in
  extremely powerful and compact Prograph
  implementations of complex functions.
• However, when used improperly, it can result in
  code that is very difficult to read or debug,
  like the computed GOTO of FORTRAN.

135
Control Flow

• Control flow is a combination of annotations on
  operations and a case structure.
• A Prograph method can consist of a number of
  mutually exclusive cases. These cases are
  somewhat similar to the cases in a Pascal CASE
  statement or a C switch expression.
• The main difference is that unlike Pascal or C,
  the code to determine which case will be executed
  is inside the case, not outside it.

136
Example

• Suppose that we want to implement a small
  function that has one integer input, and
• if that input is between 1 and 10, the value of
  the function is computed one way
• if it is between 11 and 100, another way, and
• if it is anything else, yet a third way.
• In Pascal-like pseudo-code, this function would
  look something like this

137

• Function Foo( i Integer) Integer
• Begin
• Case i of
• 1..10 Foo Calculation_A(i)
• 11..100 Foo Calculation_B(i)
• OtherwiseFoo Calculation_C(i)
• End Case
• End Foo

138
Control Flow (Cont.)

• The implementation of Foo in Prograph would also
  involve three cases, as shown in the next Figure.
  
• The control annotations are the small boxes on
  the right of the match primitives at the top of
  the first two cases. (Note that the window titles
  show the total number of cases in a method, and
  which case this window is, as in 23 the second
  of three cases.)

139
(No Transcript)
140
Control Flow (Cont.)

• In this simple example, the same annotation - a
  check mark in an otherwise empty rectangle - is
  used.
• The semantics of this annotation are If this
  operation succeeds, then go to the next case.
  The check mark is the iconic representation of
  success, and the empty rectangle represents the
  go to the next case notion. If the check mark
  were replaced by an x, then the semantics would
  have been If this operation fails, then go to
  the next case.

141
Control Flow (Cont.)

• In addition to the otherwise empty rectangle,
  there are four other possibilities, and the five
  different semantics of control annotations are
  shown together in the next Figure.
• It is a run-time error and a compile-time warning
  to have a next-case control annotation in a
  location where it cannot execute, for example, in
  the last case of a method.

142
(No Transcript)
143
(No Transcript)
144
(No Transcript)
145
(No Transcript)
146
(No Transcript)
147
(No Transcript)
148
(No Transcript)
149
(No Transcript)
150
(No Transcript)
151
Classification of Visual Programming

• Looking at the recent work reported in the
  literature, we see visual programming progressing
  in number of directions.
• 1) Visual Environments for different
  applications.
• 2) Visual Languages.
• 3) Theory of Visual Languages.

152
1) Visualization of Data

• Typically, the information or data is stored
  internally in traditional databases, but
  expressed in graphical form and presented to the
  user in a special framework.
• Users can traverse the graphical surface or zoom
  into it to obtain greater detail with a joystick
  or a pointing device.

153
Visualization of Data (Cont.)

• The user can use the mouse to answer many
  questions without the need of a keyboard.
• These systems are devoted primarily to using
  direct manipulation as a means of information
  retrieval, using a graphical view of a database
  for visualization of the information retrieved.

154
Example

• A visual environment can be built for the
  retrieval of information from a database of
  ships.
• Each ship will be represented by an icon.
• Icon shape, color and size represent type,
  status, and size of the ship.

155
2) Visualization of Programs

• The goal of this is to provide programmers and
  users with an understanding of
• What the program can do
• How they work
• Why they work
• What are the effects

156

• This is beneficial for
• beginners learning to program.
• professionals dealing with various aspects of
  programming tasks
• Designing,
• Debugging,
• Testing, and
• Modification.

157
BALSA

• BALSA ( Brown Algorithm Simulator and Animator )
  developed by Sedgewick and his colleages in Brown
  University. Sedgwick, 1983
• It enables users
• to control the speed of an animated algorithm
• to decide which view to look at.
• to specify the input data to be processed.

158

• BALSA include the following animated algorithms
• Mathematical algorithms such as
• Euclids GCD algorithm.
• Random numbers.
• Curve fitting.
• Sorting algorithms such as
• Insertion sort.
• Quick sort.
• Merge sort.
• External sort.

159

• Searching algorithms such as
• Sequential search.
• Balanced trees.
• Hashing.
• Radix searching.
• Selecting an icon from the iconic table with a
  mouse causes a 10 - 15 minute dynamic simulation
  of the selected topic to be run, with pauses at
  key images, after which the user can interact
  with the algorithms and images.

160
(No Transcript)
161
3) Visualization of Software Design

• It is very important throughout the software life
  cycle to understand
• specifications
• design
• system structure
• dependencies among data and components
• etc.
• This observation, together with the success of
  using graphical techniques has led to the
  development of a visual environment for software
  life cycle.

162
The PEGASYS

• The main purpose of PegaSys is to provide a
  visual environment for the development and
  explanation of large program designs (as opposed
  to detailed algorithms and data structures in a
  program)
• PegaSys supports the specification and analysis
  of data and control dependencies among the
  components of large programs.

163

• A program design is described in PegaSys by a
  hierarchy of interrelated pictures.
• Icons denote predefined or user-defined concepts
  about dependencies in programs.

164

• The predefined primitives denote
• objects (such as subprograms, modules, processes,
  and data structures),
• data dependencies (involving the declaration,
  manipulation, and sharing of data objects), and
• control dependencies (dealing with asynchronous
  and synchronous activities).
• A PegaSys user can define new concepts in terms
  of the primitives.

165
Example

• A network of 4 hosts can be represented in
  PegaSys as follows
• Each of the four hosts in the network is
  indicated by an ellipse,
• the communication line indicated by a dashed
  rectangle.
• a packet of data indicated by a label on arcs.

166

• Dependencies among hosts, packets, and the line
  are described by the write relation (denoted by
  the letter W on arrows) and the read relation
  (denoted by R)
• The system says that
• the broadcast network consists of four hosts that
  communicate by means of a line.
• More precisely, processes named Host1,... Host4
  write values of type pkt into a module called
  Line and read values of the same type from the
  Line module.

167
(No Transcript)
168
Visualization of Parallel Systems

• A number of display facilities have been
  associated with
• debuggers
• performance evaluation tools
• program visualization systems

169
Debugger displays

• Debugger displays attempt to provide some view of
  program state such as
• the pattern of access to shared memory
• the interprocessor communication of a particular
  distributed memory architecture
• the order in which subroutines are called

170
Displays for Performance Visualization Tools

• Performance visualization tools generally provide
  a graphical representation of standard metrics
  such as
• Processor utilization
• Communication load

171
Displays for Program Visualization Systems

• Program visualization systems provide highly
  application specific views of
• the program's data structures
• the operations which update these data structures
• or some more abstract representation of the
  computation and its progress.

172
Program Graphs

• Many systems display graphs in which
• the vertices represent program entities and
• the arcs represent call relations or temporal
  orderings, similar to the graph shown in Fig. 1.

173
Fig. 1. An animated call graph
MAIN
SELECT
PROCESS
EXPAND
PRUNE
UPDATE
COUNT
BROADCAST
174
Concurrency of the Computation

• A number of systems use a graph to represent the
  concurrency of the computation.
• 1) The concurrency map
• The concurrency map displays process histories as
  event streams on a time grid.

175

• Each column of the display represents the
  sequential stream of events for a single process.
  
• Each row of the grid represents an interval of
  time during which the events in different columns
  may occur concurrently.
• Every event in one row must occur before any
  event in the next row.

176
Communication Graphs

• A number of systems present displays that
  represent communication among processors through
  the use of boxes, arrows, and lines.
• Boxes represent processors,
• Lines represent connections. and
• Arrows represent communication events, as shown
  in Fig. 2.

177
Fig. 2 Communication Graph
P1
P0
P2
P3
P4
P5
P7
P6
178
VPL

• Here we will show some Visual Programming
  Languages.

179
BLOX

• The basic elements in BLOX system are icons
  similar in shape to jigsaw puzzle pieces.
• Programs are composed by
• 1) Juxtaposing icons in appropriate ways, so that
  neighbors interlock.
• 2) Encapsulating substructure

180
Pascal-BLOX

• The lock and key metaphor enforces proper
  syntax.

WHILE
181
Programs in Pascal - BLOX

• It uses top-down program design.
• The code segments could be encapsulated in the
  appropriate tiles. This include codes
  corresponding to
• the alternative branches
• loop bodies
• Boolean expressions associated with IF and WHILE
  statements.

182
Example

• Begin
• S1
• If not I1 Then
• Begin
• S2
• S3
• End
• Else
• While L1 Do
• Begin
• If I2 Then S4
• Else S5
• S6
• End
• While L2 Do S7
• S8
• End.

183
Pascal - BLOX Program
184
Advantages

• Top-down specification of programs
• Extension of a familiar metaphors from children's
  toys.
• Automatic elimination of the most source of
  possible syntax errors in user programs, by the
  use of the Lock and Key metaphor.
• The representation can be applied to diverse and
  varied activities in a uniform manner.

185
Our VPLs

• Our efforts to design and implement visual
  programming languages include
• 1) DataLab (with T. Lewis, and H. Kim, 1989)
• 2) VISO A Visual Languages for Concurrent
  Programming (with S. Ali, 1994)

186
VISO

• VISO is a visual programming language for
  concurrent programming.
• It has a graphical syntax based on the language
  OCCAM.
• It uses a modular approach of visual programming

187
(No Transcript)
188
(No Transcript)
189
DataLab

• DataLab is a general-purpose system for the
  specification and synthesis of abstract data
  types (ADTs).
• DataLab uses a combination of graphical and
  textual forms as a specification model for ADTs.
• An imperative semantics is used to automatically
  generate an encapsulated executable code (Pascal
  modules) from such a specification.

190
Contribution

• DataLab extends earlier work by
• 1) allowing data structures to be created and
  displayed in their classical notation,
• 2) generating encapsulated imperative code
  (Pascal), and
• 3) incorporating composition of simple structures
  into more complex structures.

191
Example - Linked List

• The first step is to define the interface part
• unit LINKED_LIST
• interface
• type
• LL_list LL_node
• LL_node record
• data string
• next LL_list
• end
• procedure LL_insert(var list LL_list item
  string)
• implementation
• end.

192

• Next, the procedure "LL_insert" is specified by
  two Conditional transformations

1
1
list
5
2list
node
3
6node
list
node
0
4
9next
10list
8
list
list
7
11
node.data item
193
list
4
6list
2
1
list
7
5
2list
3
8next
list
0
9
list.data item
194
What is next for VPLs?

• More effective use of color and sound.
• 3-D instead of 2-D graphics.
• Animated graphics.
• Languages for concurrent and parallel programming