From the discussion of Brad Myers' classic 1990 paper (originally published by the ACM CHI conference in 1986, then updated in 1990 in the Journal of Visual Languages & Computing), "Taxonomies of Visual Programming and Program Visualization" (where Brad dropped by to answer questions):
>Brad Myers' paper answers the age-old argument about whether or not spreadsheets are visual programming languages!
>Google sheets (and other google docs) can be programmed in "serverless" JavaScript that runs in the cloud somewhere. It's hellishly slow making sheets API calls, though. Feels like some kind of remote procedure call. (Slower than driving Excel via OLE Automation even, and that's saying something!) Then it times out on a wall clock (not cpu time) limit, and breaks if you take too long.
>A CS grad student friend of mine was in a programming language class, and the instructor was lecturing about visual programming languages, and claimed that there weren't any widely used visual programming languages. (This was in the late 80's, but some people are still under the same impression.)
>He raised his hand and pointed out that spreadsheets qualified as visual programming languages, and were pretty darn common.
>They're quite visual and popular because of their 2D spatial nature, relative and absolute 2D addressing modes, declarative functions and constraints, visual presentation of live directly manipulatable data, fonts, text attributes, background and foreground colors, lines, patterns, etc. Some even support procedural scripting languages whose statements are written in columns of cells.
>Maybe "real programmers" would have accepted spreadsheets more readily had Lotus named their product "Lotus 012"? (But then normal people would have hated it!)
Excerpt from "Taxonomies of Visual Programming and Program Visualization", by Brad A Myers, 1990/3/1, Journal of Visual Languages & Computing, Volume 1, Issue 1, pages 97-123:
Spreadsheets, such as those in VisiCalc or Lotus 1-2-3, were designed to help nonprogrammers manage finances. Spreadsheets incorporate programming features and can be made to do general purpose calculations [71] and therefore qualify as a very-high level Visual Programming Language. Some of the reasons that spreadsheets are so popular are (from [43] and [1]):
1. the graphics on the screen use familiar, concrete, and visible representation which directly maps to the user's natural model of the data,
2. they are nonmodal and interpretive and therefore provide immediate feedback,
3. they supply aggregate and high-level operations,
4. they avoid the notion of variables (all data is visible),
5. the inner world of computation is suppressed,
6. each cell typically has a single value throughout the computation,
7. they are nondeclarative and typeless,
8. consistency is automatically maintained, and
9. the order of evaluation (flow of control) is entirely derived from the declared cell dependencies.
The first point differentiates spreadsheets from many other Visual Programming Languages including flowcharts which are graphical representations derived from textual (linear) languages. With spreadsheets, the original representation in graphical and there is no natural textual language.
Action Graphics [41] uses ideas from spreadsheets to try to make it easier to program graphical animations. The 'Forms' system [43] uses a more conventional spreadsheet format, but adds sub-sheets (to provide procedural abstraction) which can have an unbounded size (to handle arbitrary parameters).
A different style of system is SIL-ICON [49], which allows the user to construct 'iconic sentences' consisting of graphics arranged in a meaningful two-dimensional fashion, as shown in Figure 5. The SIL-ICON interpreter then parses the picture to determine what it means. The interpreter itself is generated from a description of the legal pictures, in the same way that conventional compilers can be generated from BNF descriptions of the grammar.
10. Conclusions
Visual Programming and Program Visualization are interesting areas that show promise for improving the programming process, especially for non-programmers, but more work needs to be done. The success of spreadsheets demonstrates that if we find the appropriate paradigms, graphical techniques can revolutionize the way people interact with computers.
>By the way, something I always meant to ask you, Brad:
How does "C32" fit into your acronym theme of gemstones and rocks? Is it a teeny tiny 4x4x2 carbon atom block of diamond? How many carats would that be?
>Brad Myers wrote several articles in that book about his work on PERIDOT and GARNET, and he also developed C32:
>C32: CMU's Clever and Compelling Contribution to Computer Science in CommonLisp which is Customizable and Characterized by a Complete Coverage of Code and Contains a Cornucopia of Creative Constructs, because it Can Create Complex, Correct Constraints that are Constructed Clearly and Concretely, and Communicated using Columns of Cells, that are Constantly Calculated so they Change Continuously, and Cancel Confusion
>Spreadsheet-like system that allows constraints on objects to be specified by demonstration. Intelligent cut and paste. Implemented using Garnet.
> The dynamic spreadsheet is a good example of such a tissuelike superobject. It is a simulation kit, and it provides a remarkable degree of direct leverage. Spreadsheets at their best combine the genres established in the 1970's (objects, windows, what-you-see-is-what-you-get editing and goal-seeking retrieval) into a "better old thing" that is likely to be one of the "almost new things" for the mainstream designs of the next few years.
The "Visual" in "Visual Programming Language" is about the graphical, interactive method of creating and understanding programs, rather than merely the visibility of textual or graphical code.
Spreadsheets typically show the entire formula of the selected cell at the top of the window, at the full width of the window.
Visual programming languages based on outliners and notebooks, like UserLand Frontier, Jupyter, or Mathematica, let you hide the code by closing the outlines or code editor views.
Ivan Sutherland's pioneering PhD thesis "Sketchpad" didn't show code or formulas or constraints on the screen all the time either, focusing on the graphical content itself that you were creating and editing and programming with direct manipulation, demonstration, and constraints, instead of just the code.
>This video is a TV show made about the software Ivan Sutherland developed in his 1963 thesis at MIT's Lincoln Labs, "Sketchpad, A Man-Machine Graphical Communication System", described as one of the most influential computer programs ever written. This work was seminal in Human-Computer Interaction, Graphics and Graphical User Interfaces (GUIs), Computer Aided Design (CAD), and contraint/object-oriented programming. While watching this video, remember that the TX-2 computer (built circa 1958) on which the software ran was built from discrete transistors (not integrated circuits -it was room-sized) and contained just 64K of 36-bit words (~272k bytes).
Many visual programming languages don't necessarily display a connected graph of nodes, or explicit visual code beyond the data you're acting on.
What many people don't realize about visual programming languages is that there are so many of them that look and behave extremely differently than the few recent and popular ones they might have actually seen and used, like Unreal Blueprints.
People have been inventing wildly diverse VPLs for a long time, and there is no one standard visual design (like graphs with boxes and arrows, interlocking blocks, text, images, etc) or execution model (like data flow, control flow, cellular automata, constraints) or interface style (like keyboard, mouse, dialog panels, notebook, direct manipulation, demonstration or example) that defines the genre.
Your narrow definition of VPL excludes not only spreadsheets but also Visual Programming by Example (VPBE), Programming by Demonstration (PBD), and Visual Constraint Programming (VCP), topics that Brad Myers and others have studied, researched, and written code and papers about for decades.
It also excludes groundbreaking influential work like Ivan Sutherland's 1963 "Sketchpad" PhD thesis that pioneered Visual Constraint Programming and many other advanced interaction techniques.
VPBE and PBD enable users to teach the computer new behaviors by demonstrating actions on the interface, rather than by writing code explicitly. This approach is inherently visual and interactive, focusing on the outcomes of actions to infer the underlying logic or procedures.
Brad Myers' 1987 PhD thesis "Creating User Interfaces by Demonstration" is one of his early and influential works in the field of PBD. It discusses the design, implementation, and evaluation of Peridot, a system that allows users to create user interfaces by demonstrating actions instead of writing code.
>Peridot was a system for creating User Interfaces that was created between 1985 - 1987. This video was previously published as: Brad A. Myers. Creating User Interfaces by Demonstration: The Peridot UIMS. Technical Video Program of the SIGCHI'88 Conference, Washington, D.C., May 15-19, 1988. and IFIP Interact '87 Conference on Human-Computer Interaction. Stuttgart, West Germany. Sept. 1-4, 1987. SIGGRAPH Video Review, Issue 59, no. 2.
"Demonstrational Interfaces: Sometimes You Need a Little Intelligence, Sometimes You Need a Lot" (1998): In this article, Myers explores the concept of demonstrational interfaces, which are a subset of intelligent user interfaces that allow users to demonstrate actions to achieve goals. The paper discusses the spectrum of intelligence required in such systems, from minimal to substantial, depending on the task complexity.
"Garnet: Comprehensive Support for Graphical, Highly Interactive User Interfaces" (1990, IEEE Computer): Myers and his colleagues introduced Garnet, a toolkit for creating graphical, interactive user interfaces. Garnet supports PBD in the context of UI development, making it easier for developers to create and manipulate UI elements.
"Programming by Example: Intelligence in Demonstrational Interfaces" (1992, Communications of the ACM): This paper discusses the role of intelligence in demonstrational interfaces, emphasizing how such systems can infer user intentions from examples to automate tasks and create programs.
"Past, Present, and Future of User Interface Software Tools" (2000, ACM Transactions on Computer-Human Interaction): Although not solely focused on PBD, this paper, co-authored with Scott E. Hudson and Randy Pausch, provides an extensive overview of user interface software tools, including those that incorporate principles of programming by example and demonstration.
Myers has contributed to several books either as an author or editor, covering topics related to user interface design, software development, and PBD. One example is "Watch What I Do: Programming by Demonstration," which Myers edited. This book is a comprehensive overview of the field of programming by demonstration, featuring chapters by various authors on different PBD systems and theories.
Ivan Sutherland developed "Sketchpad" as his PhD thesis at MIT in 1963, which is often considered one of the earliest examples of visual programming. Sketchpad was revolutionary for its time, introducing concepts that laid the groundwork for interactive computer graphics, graphical user interfaces (GUIs), computer-aided design (CAD) systems, and indeed, visual programming itself.
>This video is a TV show made about the software Ivan Sutherland developed in his 1963 thesis at MIT's Lincoln Labs, "Sketchpad, A Man-Machine Graphical Communication System", described as one of the most influential computer programs ever written. This work was seminal in Human-Computer Interaction, Graphics and Graphical User Interfaces (GUIs), Computer Aided Design (CAD), and contraint/object-oriented programming. While watching this video, remember that the TX-2 computer (built circa 1958) on which the software ran was built from discrete transistors (not integrated circuits -it was room-sized) and contained just 64K of 36-bit words (~272k bytes).
Nicholas Jackiw's "The Geometer's Sketchpad" (GSP) is considered a Visual Programming Language (VPL) within the context of educational technology and mathematics due to its innovative approach to geometry, algebra, and calculus through direct manipulation and visualization. While not a VPL in the traditional sense of software development or general-purpose programming, GSP embodies key aspects of visual programming that make it a powerful tool for learning and exploration in mathematics.
Macro Recorders (i.e. Emacs keyboard macros, Photoshop actions, etc): Many applications allow users to record a series of actions (like formatting text or organizing data) as macros. You demonstrate the desired actions, and it records your actions as executable steps. The "code" behind these macros is often generated automatically and can be edited textually, but the initial programming is done visually through demonstration.
Interactive Data Analysis Tools (like Tableau, Microsoft Power BI, etc): Some data analysis tools let users manipulate data sets visually (e.g., through drag-and-drop interfaces for selecting data ranges or specifying graph types), and then automatically generate the scripts or queries that represent these actions. This approach allows users to "program" data transformations and visualizations without writing code directly.
Graphical User Interface Builders (like XCode, etc): GUI builders, often found in integrated development environments (IDEs), allow developers to design interfaces by arranging components visually. Properties and event handlers can be assigned through demonstrations or interactions with the interface, such as by selecting elements and setting their characteristics through property windows. The underlying code for the GUI is generated by the tool, based on the visual design.
End-User Development Tools: Platforms designed to creating custom applications or automations through demonstration. For instance, tools like IFTTT (If This Then That) or Zapier allow users to create "applets" or "zaps" by choosing triggers and actions from a list of services and configuring them without seeing any traditional programming code. The logic is entirely defined through the visual assembly of components.
AgentSheets is a visual programming environment that allows users, including those with little to no programming experience, to create their own simulations, games, and interactive stories. It is particularly known for its use in educational settings to teach computational thinking and programming concepts.
Programming by Example in AgentSheets: The environment supports a PBE approach through its "Visual AgenTalk" programming language. Users can specify the behavior of agents (the active components in their simulations or games) by defining rules in a visual manner. These rules are often created by demonstrating actions or setting up conditions and outcomes using a graphical interface, which the system then translates into executable logic.
David Ackley's "SPLAT" programming language for the Moveable Feast Machine, and Lu Wilson's "Sandpond" visual cellular automata programming language, let you define rules by example, which I recently wrote about on HN here:
https://news.ycombinator.com/item?id=26057530
https://www.cs.cmu.edu/~bam/papers/VLtax2-jvlc-1990.pdf
https://news.ycombinator.com/item?id=26061576
>Brad Myers' paper answers the age-old argument about whether or not spreadsheets are visual programming languages!
>Google sheets (and other google docs) can be programmed in "serverless" JavaScript that runs in the cloud somewhere. It's hellishly slow making sheets API calls, though. Feels like some kind of remote procedure call. (Slower than driving Excel via OLE Automation even, and that's saying something!) Then it times out on a wall clock (not cpu time) limit, and breaks if you take too long.
>A CS grad student friend of mine was in a programming language class, and the instructor was lecturing about visual programming languages, and claimed that there weren't any widely used visual programming languages. (This was in the late 80's, but some people are still under the same impression.)
>He raised his hand and pointed out that spreadsheets qualified as visual programming languages, and were pretty darn common.
>They're quite visual and popular because of their 2D spatial nature, relative and absolute 2D addressing modes, declarative functions and constraints, visual presentation of live directly manipulatable data, fonts, text attributes, background and foreground colors, lines, patterns, etc. Some even support procedural scripting languages whose statements are written in columns of cells.
>Maybe "real programmers" would have accepted spreadsheets more readily had Lotus named their product "Lotus 012"? (But then normal people would have hated it!)
>I Was Wrong About Spreadsheets And I'm Sorry:
https://www.reifyworks.com/writing/2017-01-25-i-was-wrong-ab...
Excerpt from "Taxonomies of Visual Programming and Program Visualization", by Brad A Myers, 1990/3/1, Journal of Visual Languages & Computing, Volume 1, Issue 1, pages 97-123:
Spreadsheets, such as those in VisiCalc or Lotus 1-2-3, were designed to help nonprogrammers manage finances. Spreadsheets incorporate programming features and can be made to do general purpose calculations [71] and therefore qualify as a very-high level Visual Programming Language. Some of the reasons that spreadsheets are so popular are (from [43] and [1]):
1. the graphics on the screen use familiar, concrete, and visible representation which directly maps to the user's natural model of the data,
2. they are nonmodal and interpretive and therefore provide immediate feedback,
3. they supply aggregate and high-level operations,
4. they avoid the notion of variables (all data is visible),
5. the inner world of computation is suppressed,
6. each cell typically has a single value throughout the computation,
7. they are nondeclarative and typeless,
8. consistency is automatically maintained, and
9. the order of evaluation (flow of control) is entirely derived from the declared cell dependencies.
The first point differentiates spreadsheets from many other Visual Programming Languages including flowcharts which are graphical representations derived from textual (linear) languages. With spreadsheets, the original representation in graphical and there is no natural textual language.
Action Graphics [41] uses ideas from spreadsheets to try to make it easier to program graphical animations. The 'Forms' system [43] uses a more conventional spreadsheet format, but adds sub-sheets (to provide procedural abstraction) which can have an unbounded size (to handle arbitrary parameters).
A different style of system is SIL-ICON [49], which allows the user to construct 'iconic sentences' consisting of graphics arranged in a meaningful two-dimensional fashion, as shown in Figure 5. The SIL-ICON interpreter then parses the picture to determine what it means. The interpreter itself is generated from a description of the legal pictures, in the same way that conventional compilers can be generated from BNF descriptions of the grammar.
10. Conclusions
Visual Programming and Program Visualization are interesting areas that show promise for improving the programming process, especially for non-programmers, but more work needs to be done. The success of spreadsheets demonstrates that if we find the appropriate paradigms, graphical techniques can revolutionize the way people interact with computers.
https://news.ycombinator.com/item?id=26061279
>By the way, something I always meant to ask you, Brad: How does "C32" fit into your acronym theme of gemstones and rocks? Is it a teeny tiny 4x4x2 carbon atom block of diamond? How many carats would that be?
>Brad Myers wrote several articles in that book about his work on PERIDOT and GARNET, and he also developed C32:
>C32: CMU's Clever and Compelling Contribution to Computer Science in CommonLisp which is Customizable and Characterized by a Complete Coverage of Code and Contains a Cornucopia of Creative Constructs, because it Can Create Complex, Correct Constraints that are Constructed Clearly and Concretely, and Communicated using Columns of Cells, that are Constantly Calculated so they Change Continuously, and Cancel Confusion
>Spreadsheet-like system that allows constraints on objects to be specified by demonstration. Intelligent cut and paste. Implemented using Garnet.
http://www.cs.cmu.edu/~bam/acronyms.html