29 December 2011

The Web Is Wrong

The Analogies Are Wrong

Originally, web pages were static documents, and web browsers were static document viewers; there was text, some formatting, and images—if you could pay for the bandwidth to serve them. Hyperlinks were the really big thing, because they were the main point of user interaction—but what a powerful thing they were, and still are.

Then along came CGI and Java, and the web was transformed: all of a sudden, a web browser became a way to serve interactive content to users. A web page was no longer just a document, but a portal to a living, breathing world. You yourself could make content that would appear on a page for others to see. That was wicked cool, and it’s still a huge part of what makes the web so compelling today.

The more interactive the web has become, the more it has been used to connect people and their interests—and the less the “document” analogy has applied. At this point, the vast majority of pages that I use on a daily basis are not documents in any real sense: they are applications.

Nobody refers to Twitter, Google+, Facebook, Stack Exchange, Blogger, or even forums and message boards as documents. It’s laughable! Not only has the analogy been stretched so far that it’s absurd, but it has been absurd since about fifteen years ago. But that’s not all.

Human-Readability Is Wrong

Why are we still using text-based document formats for all of our web interaction? HTML is fine for writing documents, but why do we actually transmit it as text? Why do we make software that just writes HTML for us, serving pages to the browser whose source is nigh indecipherable to a human user? What is to be gained from using text-based documents for stylesheets and scripts? Human-readability? Not at all.

Programmers may be surprised to note that the vast majority of users do not ever view the sources of a web page. They don’t know what makes things tick, they don’t care, and even if they did, they wouldn’t be able to decipher it all without experience, especially if the source had been minified.

If you’re not yet convinced that this whole situation is outlandishly, hilariously wrong, let’s talk about minification. Minification is taking a text-based document—where the sole reason for being text-based is human-readability—and compressing it to such an extent that it is no longer human-readable, but still text-based. Now for absolutely no reason whatsoever. Have I made anybody facepalm yet?

Furthermore, a minified document still has to be parsed according to the rules of the unminified language. This means that minification relies on details of the parser, which for permissive HTML and subtle JavaScript could be fragile and dangerous. In addition, the browser must perform all of the parsing and internal compilation it would for unminified source, causing a totally unnecessary performance hit.

The Obvious Solution

At the very least, we should stop serving HTML, CSS, and JavaScript to users. Let’s instead serve things in concise, binary format—compiled documents and stylesheets, and, by far the most important…

Compiled JavaScript that runs in a standard stack-based VM, which, let me stress, could easily be targeted by other languages. Because as if the situation weren’t mad enough, nowadays we write languages that compile to JavaScript, just so that they can run in a web browser.

By using binary formats, browsers will be exchanging less data with servers in order to convey the same information, improving performance. Web page performance is an important consideration: if your page takes too long to load, no one will bother waiting for it to load, and you’ll lose traffic.

Serving binary data is just as reliable as serving textual data. The direct source of a page won’t be human-readable, but so what? The source of the desktop applications you use isn’t typically human-readable unless the source is open and you seek it out. What’s wrong with using the same principle for Web pages? And the compiled version will contain the same data, so you will always be able to see decompiled HTML or CSS if you want it.

The Non-Obvious Solution

But there is, in my opinion, an even better solution than that. In order to make a standard-issue web application, you must deal with a minimum of four languages: HTML, CSS, JavaScript, and a server-side language such as Perl, or Haskell, or whatever you like, really, as long as you can install a toolchain on your server.

But even if you don’t write one or more of these directly, you must still contend with them, and that becomes problematic when the abstractions leak. This is too complicated. Beginners need consistency, and the majority of developers are always going to be beginners, so we have no choice but to help them.

The multitude of languages is good, in a way: each language ostensibly serves a single purpose. HTML is for structure and content; CSS, for presentation; JavaScript, frontend interaction; and the server-side language, backend interaction. And the modularity that this brings is, in a way, a good thing. However, you can attain modularity in many sane ways, without defining complete domain-specific languages.

Moreover, you can do it without necessitating that code in one language be composed and transmitted by another language as text, which is awkward and unsafe. Injection attacks are ubiquitous for a reason.

So what if, instead of serving content in three languages (plus images and videos and what have you), we were to use just one language for presentation, styling, and interaction alike? A page would be served as a compiled bundle, and the browser would just run it in a protected VM. Content that doesn’t need to be loaded asynchronously could just be embedded in the bundle.

You could then take it a step further: web applications could be treated as sources of interactive, structured information. They could be be queried in structured fashion, vastly simplifying content scrapers; or composed with one another and with desktop applications, using something analogous to pipes. We could create powerful distributed applications, or share data across our networks however we like.

Final Remarks

This is, I believe, an important problem, and one I’m actively trying to solve. If enough people demonstrate interest, maybe we can make this a reality, and give the web the overhaul it needs. Because, at the moment, the web is just plain wrong.

13 comments:

  1. I'm not so sure that text HTML is such a bad idea. Sure, most humans never view source, but it sure makes life a billion times easier when you actually want to program any kind of bot, whether it be a bot which consumes the web or a bot which serves the web.

    Moreover, consider this. If we'd used a binary language from the start, then JavaScript would never even have *come into existence*. Remember that it was originally only supported by Netscape, who had the brilliant idea of adding it despite it being totally nonstandard; they were able to do this *BECAUSE* of the non-compiled nature of HTML. If we scrapped that, what future innovations like JavaScript would we be snipping in the bud?

    As for issues of compactness, there are already standard protocols for zipping HTTP transmissions and having the client unzip them.

    ReplyDelete
  2. @Xamuel:

    Does it really make life easier to use HTML, though? When you’re scraping a page, you’re either using an HTML parsing library (for which a drop-in replacement could be made to support the binary format) or stripping things yourself with regular expressions. If you treat a web application as an actual application, then you can interact with it in a more meaningful way, or at the very least receive its output over a pipe, or interact with it over a socket.

    I don’t think having a standard VM model would discourage innovation; on the contrary, I think it would prompt a bevy of new languages competing to target the new architecture, just like what’s happening now to poor JavaScript. And JavaScript would remain popular under the new model. But there should be no difference from the language’s point of view whether a document is compiled or not, because the language should interact with the document semantically, not textually.

    The compactness issues aren’t quite as big a deal, but still, compiled code is going to be much more concise than source, especially since we’re not talking about arbitrary relocatable executables, here. The very reason that gzip works so well on our current transmission formats is that they’re highly redundant—necessary for human-readability, but my argument is that that oughtn’t to be a concern. Use a binary format, and compression often becomes unnecessary.

    ReplyDelete
  3. It's not every day that you hear someone complaining because everyone is using an technology stack that has clearly defined, fully modular parts, everyone is communicating in plaintext instead of with impenetrable binary blobs, and all the software they write is open-source. It's not clear to me why I should want to throw that away to gain back some fraction of the available bandwidth.

    ReplyDelete
  4. Node server -> ExtJS front-end using less.js for advanced styling? Done.

    ReplyDelete
  5. @Anonymous:

    Thanks for that. I added a bit in the last section to clarify why using multiple languages is not necessarily the best way to achieve that modularity. The open-source argument is a red herring, because deobfuscating and deminifying source gets you just as far as decompiling it. But if the document had the same structure, it could still be decompiled sanely into HTML, or any other presentation format you like.

    Also, there’s no reason why a binary format need be any more complex than a text-based format, and in fact there’s significant pressure for it to be less so. When a human needs to read it, a format needs considerably more redundancy and fault-tolerance, which can obscure its straightforward logical structure.

    ReplyDelete
  6. I believe we already have something thats compiled to bytecode and used in a VM model.

    Java.

    Flash also tried what you're referring to.

    HTML, CSS, JavaScript make it easy for anyone and I do mean ANYONE AT ALL to start learning how the web works. Open up text editor, write some HTML, save it, open in web browser. No compiling and its right in front of you. With the advancements of Web Inspectors we can now even edit the DOM live.

    Moving to a VM would be a huge step backwards. Languages are text-based because they're easy to write and easy to debug, and with the way HTML is now its easy to debug right in the browser. With a VM, you'd essentially have to decompile it right back to source-code for the same level of debugging we have right now - and that source will be text-based.

    The way HTML, CSS, and JavaScript work together to make editing web documents and running web applications simple and intuitive was probably completely accidental, but it *works* and it works *damn well*.

    ReplyDelete
  7. You are right about that, and I am willing to help. How you think this approach would affect search engines?

    ReplyDelete
  8. @Erico:

    Presumably search engines can only benefit. They won’t have to put nearly as much effort into crawling a site to deduce its structure, nor scraping a page in order to understand the nature of its content. They’ll just submit a standard query for interesting content, and process the results however they choose, without having to worry so much about unstructured information or content that would be loaded asynchronously for a user in a regular browser.

    ReplyDelete
  9. Navarr mentioned Java and Flash. There's also Google's Native Client (NaCl).

    ReplyDelete
  10. I think you're trying to come up with a solution to a non-problem. With modern web apps, the bottleneck is in the network - bandwidth and latency - not in the parsing or even execution of human-readable documents-as-code.

    In fact, you can kind of see the performance of Javascript when given hardware control: there are plenty of WebGL demos that run pretty fast, with no need for compilation. If it's compile-time checking you're after, tools like JSLint and Google's closure compiler do that for you too.

    I feel like the most challenging problems in technology aren't actually technological, but political; it's telling that while browsers and browser vendors have sped up the development proprietary features (like WebGL, CSS extensions, V8, DART, audio and video formats, etc.), HTML5 itself isn't set to be ratified until well beyond the next decade. It's remarkable that we have strong enough of a base now to develop good web apps, and that everyone has agreed to support HTML/CSS/JS, so it makes the most sense to build from this base as opposed to try to replace it.

    ReplyDelete
    Replies
    1. "the bottleneck is in the network - bandwidth and latency"

      Binary blobs can often be made a lot more compact than text

      "there are plenty of WebGL demos that run pretty fast, with no need for compilation"

      I disagree. If they run fast, its because the graphics card is doing most of the work, not because Javascript is fast.

      My desktop is 8 years old with a 3 year old graphics card and I can play Skyrim on medium settings at between 20 and 30 FPS (at 1280x960). I have yet to see a WebGL demo that is more than just a few cubes beat that, despite the fact that all of the WebGL demos I've seen (and I've seen quite a few) have had much much simpler scenes with rarely more than a handful of animated elements and rarely much other logic besides the graphics (Skyrim on the other hand has beautiful and complex graphics and still simulates a large world made up of AI, combat, audio and so on). One of the more complex WebGL demos I saw ran at about 10 FPS despite only using about 8% on my GPU monitor suggesting that the graphics aren't what was making it slow, but rather the Javascript is.

      Now compare this to Lua running on LuaJIT: it is claimed to often be as fast as or even outperform, in some cases, native programs. There are even some benchmarks proving this. While I'm sure its not that fast in the general case, remember that this is the work of one guy, while there are teams of people working on a number of Javascript VM's trying to make it as fast as possible.

      So no, I don't think Javascript is particularly fast and theres room for improvements.

      Also, Javascript isn't a particularly great language - there are entire websites dedicated to Javascripts many inconsistencies and gotchas.

      But thats all besides the point because I think theres many more benefits to replacing the web with something that was actually designed for the webs modern use cases besides potential performance boosts or replacing Javascript with something else.

      Delete
  11. Nice post!

    I never understood why web programming must be like it is. I am so glad that currently mobile applications are not delivered as web apps but as native applications. It's just so much better to develop a native Android app than a web app. It's true they cannot be crawled but they are not meant to be delivering plain content anyway. They offer functionality.

    And how is learning HTML, CSS, JavaScript and a backend language easier than learning Android(Java)? It's text yeah but it also sucks because it's overwhelming. For beginners it's easier to wrap their head around one or two frameworks. The next big thing stalling beginners are the differences between the browser implementations of JavaScript and CSS and probably HTML too.

    And for every single technology we use there must be a standard so all the vendors can implement them(if they want). This process is just sluggish. Why is the web still lacking fundamental features like client databases, camera support, gps.
    Why must some other technologies like Android and iOS come along to show the web what it is lacking????

    Sure HTML5 will soon offer some of their featuers but it's really obvious by now that the web platform is all but an innovative driver. The Internet is the hottest thing in our time. We need technologies to make use of this connection and not some frameworks which help us to hack around the fundamentally flawed architecture the web platform offers.

    Let's do it better!

    ReplyDelete
    Replies
    1. "I never understood why web programming must be like it is"

      Simple - it was never designed to do what we now use it to do. Its original use cases were much simpler and it has evolved to be the beast it now is.

      Delete