29 December 2011

The Web Is Wrong

The Analogies Are Wrong

Originally, web pages were static documents, and web browsers were static document viewers; there was text, some formatting, and images—if you could pay for the bandwidth to serve them. Hyperlinks were the really big thing, because they were the main point of user interaction—but what a powerful thing they were, and still are.

Then along came CGI and Java, and the web was transformed: all of a sudden, a web browser became a way to serve interactive content to users. A web page was no longer just a document, but a portal to a living, breathing world. You yourself could make content that would appear on a page for others to see. That was wicked cool, and it’s still a huge part of what makes the web so compelling today.

The more interactive the web has become, the more it has been used to connect people and their interests—and the less the “document” analogy has applied. At this point, the vast majority of pages that I use on a daily basis are not documents in any real sense: they are applications.

Nobody refers to Twitter, Google+, Facebook, Stack Exchange, Blogger, or even forums and message boards as documents. It’s laughable! Not only has the analogy been stretched so far that it’s absurd, but it has been absurd since about fifteen years ago. But that’s not all.

Human-Readability Is Wrong

Why are we still using text-based document formats for all of our web interaction? HTML is fine for writing documents, but why do we actually transmit it as text? Why do we make software that just writes HTML for us, serving pages to the browser whose source is nigh indecipherable to a human user? What is to be gained from using text-based documents for stylesheets and scripts? Human-readability? Not at all.

Programmers may be surprised to note that the vast majority of users do not ever view the sources of a web page. They don’t know what makes things tick, they don’t care, and even if they did, they wouldn’t be able to decipher it all without experience, especially if the source had been minified.

If you’re not yet convinced that this whole situation is outlandishly, hilariously wrong, let’s talk about minification. Minification is taking a text-based document—where the sole reason for being text-based is human-readability—and compressing it to such an extent that it is no longer human-readable, but still text-based. Now for absolutely no reason whatsoever. Have I made anybody facepalm yet?

Furthermore, a minified document still has to be parsed according to the rules of the unminified language. This means that minification relies on details of the parser, which for permissive HTML and subtle JavaScript could be fragile and dangerous. In addition, the browser must perform all of the parsing and internal compilation it would for unminified source, causing a totally unnecessary performance hit.

The Obvious Solution

At the very least, we should stop serving HTML, CSS, and JavaScript to users. Let’s instead serve things in concise, binary format—compiled documents and stylesheets, and, by far the most important…

Compiled JavaScript that runs in a standard stack-based VM, which, let me stress, could easily be targeted by other languages. Because as if the situation weren’t mad enough, nowadays we write languages that compile to JavaScript, just so that they can run in a web browser.

By using binary formats, browsers will be exchanging less data with servers in order to convey the same information, improving performance. Web page performance is an important consideration: if your page takes too long to load, no one will bother waiting for it to load, and you’ll lose traffic.

Serving binary data is just as reliable as serving textual data. The direct source of a page won’t be human-readable, but so what? The source of the desktop applications you use isn’t typically human-readable unless the source is open and you seek it out. What’s wrong with using the same principle for Web pages? And the compiled version will contain the same data, so you will always be able to see decompiled HTML or CSS if you want it.

The Non-Obvious Solution

But there is, in my opinion, an even better solution than that. In order to make a standard-issue web application, you must deal with a minimum of four languages: HTML, CSS, JavaScript, and a server-side language such as Perl, or Haskell, or whatever you like, really, as long as you can install a toolchain on your server.

But even if you don’t write one or more of these directly, you must still contend with them, and that becomes problematic when the abstractions leak. This is too complicated. Beginners need consistency, and the majority of developers are always going to be beginners, so we have no choice but to help them.

The multitude of languages is good, in a way: each language ostensibly serves a single purpose. HTML is for structure and content; CSS, for presentation; JavaScript, frontend interaction; and the server-side language, backend interaction. And the modularity that this brings is, in a way, a good thing. However, you can attain modularity in many sane ways, without defining complete domain-specific languages.

Moreover, you can do it without necessitating that code in one language be composed and transmitted by another language as text, which is awkward and unsafe. Injection attacks are ubiquitous for a reason.

So what if, instead of serving content in three languages (plus images and videos and what have you), we were to use just one language for presentation, styling, and interaction alike? A page would be served as a compiled bundle, and the browser would just run it in a protected VM. Content that doesn’t need to be loaded asynchronously could just be embedded in the bundle.

You could then take it a step further: web applications could be treated as sources of interactive, structured information. They could be be queried in structured fashion, vastly simplifying content scrapers; or composed with one another and with desktop applications, using something analogous to pipes. We could create powerful distributed applications, or share data across our networks however we like.

Final Remarks

This is, I believe, an important problem, and one I’m actively trying to solve. If enough people demonstrate interest, maybe we can make this a reality, and give the web the overhaul it needs. Because, at the moment, the web is just plain wrong.

14 December 2011

There Are Only Types

I never liked object-oriented programming. I started programming half a lifetime ago, and I found object-oriented languages early on, because of their popularity. For the most part I was uncomfortable with them, but it took some time to learn enough about programming to really understand why.

So I became proficient in Java and C++ and what have you, and I acknowledge that these languages can be useful for some things. If a language has some library that lets you get your task done quicklier or at lower cost, then it’s probably safe to choose it over another language that might be better, in the usual sense of “good” = “expressive”. Quality isn’t the whole story when it comes to quality.

It wasn’t the many fanatical followers of what seemed more like a radical philosophy than an actual technology that ultimately turned me away from OOP. I realised at some point that the primary reason I disliked that world was the fact that I find it really difficult to decompose the world in terms of objects. Not because I found object-oriented programming difficult; just that I found it unnatural.

I’ll clarify. An object is an entity that responds to messages and has some internal information. But not all things conceptually talk to one another, nor have state; so I thought this was backward; why think about entities that manifest qualities, when you can just think about qualities themselves? Why would you declare the contracts (interfaces) between entities before you know if those contracts are true? And most importantly, why would you use two different languages to compute the values and types in one program?

This is how I—inadvertently—came to independently reinvent the concept of dependent type systems. Values depend on types, which can be verified and manipulated and so forth; but types, too, depend on values, and this reverse relationship is treated in exactly the same way. For a trivial example, in a dependent type system, you can encode the type “list of size N with elements of type T”, or “sorted list” or “balanced tree”. A dependent type system in an imperative language would obviate the need for a lot of explicit imperative implementations of contracts on data structures.

But I went a step further than that. Conceptually, types tend to be computed as a side-effect of computation on values. That is, almost any language you can name is interested in computing values, and only verifies types as something of an afterthought—assuming, of course, it verifies types at all. This is true even of languages with very rich type systems such as Haskell.

But what if, I thought, you were to turn that on its head? I became intrigued with this problem around +37 (2007), I think, and set about designing a language in which, essentially, all terms denote types and values are computed as a side-effect of reasoning about those types.

For example, the term “3” is not just the value 3, but a reference to the set of all underlying values that are equal to 3. By “underlying value”, I mean machine representation: every possible size and signedness of integer or floating-point number. But, since types were computed non-strictly, the singleton type of 3 could be represented with a regular old machine integer, which would be converted only as needed.

There are several advantages to dealing with singleton types rather than ordinary values. The first is that value equivalence becomes explicitly type- and context-sensitive, rather than implicitly so; type equivalence behaves similarly if you generalise to singleton kinds. And though it may not be immediately obvious, type checking within a type system based on singleton types is statically decidable, because singleton types are basically just equivalence classes, and subtypes form proper subsets of those classes. That means, essentially, that we can prove some kinds of extensional equality.

So singleton types have a few perks, and they coexist nicely with a dependent type system. Unfortunately, dependent types are not statically decidable, because you cannot, in general, prove that two functions are extensionally equal, which is what statically deciding dependent types would require.

But that’s actually not such a bad thing. In a system with singleton types, the types themselves contain enough information to provide more reasonable diagnostics in the event of a type error. And of course the compiler would not attempt to decide all types statically, but only those which are known to be decidable; the rest would be decided dynamically. Taking this view, static typing can be seen as a constant-folding optimisation, and likewise constant-folding can be seen as a form of static typing, for in this system types and values are one and the same.

I never did finish the language that would implement these ideas, but I fully intend to. And at the very least, exploring this field gave me some insight and informed my ability to write programs in existing languages. But to this day I have to shake my head at the languages I still use every day, from C++ to Haskell.

Because they don’t know that there are only types.

13 December 2011

Concatenative Rant

What follows is a rant disguised as a question, which I was going to post on Programmers.SE, but I ultimately thought better. Enjoy.

Most of our favourite programming languages get lumped into the categories of “imperative” or “functional”. While those paradigms don’t exactly present a complete picture of what languages are out there—or even how the languages that are out there actually work—they serve to characterise the majority of what we work with on a daily basis. And all is well with the world.

Except perhaps not. Conceptually speaking, in an imperative language, I order the computer to do what I want; in a functional language, I describe what I want the computer to do. However, in a functional language, I have to contend with more than functions, and the fundamental unit of computation is not the function, but rather function application (specifically, beta reduction).

I can still write useful programs in both kinds of languages, but I find it odd that functional languages get the reputation for purity when I can’t even write most expressions in point-free form.

Enter concatenative languages. When your programming language is stack-based, you can think of it imperatively: all terms denote imperative operations on stacks. But you can also think about it declaratively: all terms denote functions, the juxtaposition of terms denotes function composition, and a program is just a single large function, written entirely point-free. It’s turtles all the way down.

That, to me, seems really elegant. It’s the Holy Grail of paradigms: one which is easy to compile, efficient on existing machines, and yet also amenable to straightforward formal analysis. Cat, for instance, is statically typed, with a simple type inference algorithm based on Hindley-Milner.

And don’t get me started on the opportunities for metaprogramming. In some concatenative languages, I can straightforwardly implement comments, strings, or even numbers as syntactic extensions in the language itself.

So what gives? Why aren’t these languages everywhere? Is it simply that exceptional languages can’t be popular because most programmers can’t be exceptional?

06 December 2011

Redefining the Epoch

The Internet Calendar

In the Gregorian calendar, it is the year 2011 of the Common Era—or Anno Domini if you’re more of a traditionalist. In the Hebrew calendar, it’s 5772 Anno Mundi. In the Chinese calendar, it’s 4708 or 4648—depending on which of the two epochs you prefer. In the Islamic calendar, it’s 1433 Anno Hegirae. And, just for kicks, in the Discordian calendar it’s 3177 YOLD.

The epochs of all of these calendars have one thing in common: they refer to important historical events in the cultures that spawned them. But in an era of global communication, they are, for all intents and purposes, completely arbitrary. We can do better.

It’s fairly silly that the vast majority of us still cleave to the Gregorian epoch as though it were culturally neutral—because it is assuredly not. Now, there’s nothing wrong with a Christian epoch for a calendar, as least insofar as there’s nothing inherently wrong with any particular epoch. But certain dates are certainly more culturally relevant than others, and ours is not an exclusively Christian culture.

What, then, is the day that started our era?

My vote is for the day when the Internet—as we would come to know it—came into existence. That day, the 29th of October, 1969, defines a cultural turning point that has since changed the world in unfathomably many and deep ways.

The first year following the epoch is +0; the year preceding, −0. The best way to understand this is by thinking of a date as a fractional number of years since the epoch: January 1, 1970 (the epoch for Unix) is +0.1749. Lisp was created in −10. Keep in mind that I don’t want to change anything else about our calendar: not our system of months, nor the days of the week, nor the way we calculate leap years. Those all work just fine.

So, without further ado:

This is the year +42, and today is the day that I redefined the epoch.

Discuss.