As JourneyApps is nearing the launch of OXIDE, our next-generation IDE, we are exploring the “dreams versus reality” of low-code, and showing how the industry can chart a more enlightened course for software development. This is part 2 of the series. You can read part 1 here.
In part 1 of our series, Low-Code and the Road to Software Enlightenment, I wrote that “As the low-code Kool-Aid is wearing off, many business and IT folks alike are starting to realize something that software thought-leaders have known for decades: In many scenarios, writing code is by far the most efficient and easy solution to a software problem, and there are countless everyday use cases where a low-code/no-code development tool is simply not the right tool for the job.”
This may seem like a controversial statement to some. You may be thinking: “Perhaps the perfect low-code platform just hasn’t been invented yet. Perhaps we are just waiting for an innovative company to build a breakthrough new low-code platform that allows not only simple apps, but even highly sophisticated and complex apps to be built easily and quickly using exclusively visual point-and-click tools”.
I wish that were a possibility. But software researchers have been studying the art and science of software creation for decades, and around 40 years ago, many of the leading thinkers came to the conclusion that there are fundamental limitations to how much we can limit the complexity of software creation, or increase its productivity. They also found that visual software development tools inherently have serious drawbacks that cannot be wished away.
No Silver Bullet
Perhaps the most famous work comes from the computer scientist and Turing Award winner Fred Brooks. In an essay titled No Silver Bullet — Essence and Accident in Software Engineering, published in 1986, Brooks wrote:
“The familiar software project [is] capable of becoming a monster of missed schedules, blown budgets, and flawed products. So we hear desperate cries for a silver bullet, something to make software costs drop as rapidly as computer hardware costs do.
But, as we look to the horizon of a decade hence, we see no silver bullet. There is no single development, in either technology or management technique, which by itself promises even one order of magnitude improvement in productivity, in reliability, in simplicity.”
- Image: Michal Jurkowski/Shutterstock.com
Central to Brooks’ argument is that there are two types of complexity involved in software creation:
-
Essential complexity is complexity inherent to a problem. It is the entanglement/combination of concepts that are necessary for solving the problem at hand. Nothing can remove essential complexity. [1] Put differently, “essential complexity is how hard something is to do, regardless of how experienced you are, what tools you use or what … architecture pattern you use to solve the problem.” [2]
-
Accidental complexity is the complexity which is not strictly necessary for solving a problem. It is created by engineers who develop software or who create the tools with which software is developed. It’s what makes something more complex than it really needs to be at its essence. Accidental complexity can be reduced or eliminated.
Brooks argues that the productivity and simplicity of software development is fundamentally limited by the fact that there is essential complexity that cannot be removed from the problem of software creation:
“The essence of a software entity is a construct of interlocking concepts: data sets, relationships among data items, algorithms, and invocations of functions. This essence is abstract, in that the conceptual construct is the same under many different representations. […]
I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it. […] We still make syntax errors, to be sure; but they are fuzz compared to the conceptual errors in most systems.”
There have certainly been massive innovations which have dramatically increased the productivity of software creation in the past few decades — such as the invention of high-level languages. However, Brooks argues that those innovations are only capable of reducing or eliminating the accidental complexity of software creation.
The gains from continued innovations in software development therefore become increasingly marginal, since much of the existing innovations have already removed most of the accidental complexity. This is why order-of-magnitude leaps in software development productivity will remain elusive.
Hopes for the Silver
Brooks goes on to look at each of the major categories of technological advances that have been hawked as potential silver bullets in software creation, and asks whether they offer “revolutionary advances, or incremental ones”.
Brooks’ thoughts on Graphical Programming are especially relevant to the major low-code platforms of today:
“A favorite subject for PhD dissertations in software engineering is graphical, or visual, programming. […] Nothing even convincing, much less exciting, has yet emerged from such efforts. I am persuaded that nothing will.
In the first place … the flow chart is a very poor abstraction of software structure. In the pitiful, multipage, connection-boxed form to which the flow chart has today been elaborated, it has proved to be essentially useless as a design tool.
More fundamentally … software is very difficult to visualize. […] In spite of progress in restricting and simplifying the structures of software, they remain inherently unvisualizable, thus depriving the mind of some of its most powerful conceptual tools.”
- Image: Ryzhi/Shutterstock.com
On the subject of Artificial Intelligence (AI), Brooks argues that unless we one day invent artificial general intelligence, AI can only offer marginal gains to software development: AI can be used to determine specific heuristics or rules of thumb that humans use in solving problems, and then “suggest” or “advise” humans to apply those in specific scenarios. While this is a big benefit, it is inherently limited:
“The hard thing about building software is deciding what to say, not saying it. No facilitation of expression can give more than marginal gains.”
A Small Matter of Programming
Bonnie Nardi has also produced excellent work on the idea of “End-user computing” — the notion that everyday computer users should be able to create their own software programs. In her 1993 book, A Small Matter of Programming, she surveyed much of the academic literature on this subject.
- Image: The MIT Press
I credit the cofounder of Heroku, Adam Wiggins, with introducing me to Nardi’s work. In 2013, Wiggins wrote an excellent summary of her book.
On the subject of Visual Programming, Nardi writes:
“One of the main arguments for the use of visual programming notations is that they will be easier for end users to read and to understand. Several studies … cast doubt on this assertion. A number of problems have been identified in the interpretation of diagrams, and where benefits have been seen, they do not represent quantum leaps in improvement. […]
The available empirical research does not support the more general claims of proponents of visual programming as far as naturalness, comprehensibility, or the elimination of syntax. And Myers (1992) notes that “visual languages do not actually help end users with the difficult concepts of programming such as conditionals and iteration”.
She also specifically addresses Forms-Based Programming:
“Form-based systems lack generality, and performance degrades as the problem to be solved varies away from the prototypical solution embodied in the form (Jeffries and Rosenberg, 1987). A major limitation of forms-based systems is that they generally do not handle complex procedural tasks well. Forms-based systems seem most suited to relatively simple applications without complex procedural demands.
Wiggins does not pull any punches:
“Purely visual programming is a red herring, and history is littered with the corpses of products that have attempted to offer programming without code.”
Where Do We Go From Here? Where Are Visual Tools Useful?
Nardi argues that while visual notations are ineffective for generic programming, they can ease program understanding when they are designed for a limited application domain. For example, designing graphical interfaces is naturally something that lends itself well to visual notation.
This aligns with the vision that we had when we built JourneyApps: We made app development on our platform primarily code-centric, but we complement it with select visual tools — ultimately striving to eliminate as much as possible accidental complexity from the process of developing apps. This stands in stark contrast to most major low-code platform vendors, who are still stuck with trying to make generic visual programming work.
Besides JourneyApps, some other forward-thinking app platform vendors are finding ways to combine code-centric development with visual tools. Yet there are many different approaches to this problem, and some of these approaches have serious drawbacks too. We will explore these different approaches and their pros and cons in the next part of our series.
Read part 3 of our series here.
