Upstart: Service Mangement modeled as a bi-parental network of stator phase flip-flops

One of the problems Upstart has encountered recently is ensuring service starting with the appropriate log-n phase delay when given a bi-quantum dependency network over a Von Nelson stator grid. Recently we proposed a radical new model for service management centred around a bi-parental network of stator-phase flip-flops, bijected directly over the transient monodirectional service network.



While this prevented degradation due to non-selmann adhesion, stator flux jitter continued to oscillate in a pseudo-sapient Le Chevalier sequence, resulting in bled q-functors due to a 3-7 paradox arising between the node-region offsets.

To mitigate this, side slip is prevented via the use of a k-factor blowoff quotient directly infabricated with the 4th arity q-functor. This applies a L'hospital retrodamping sequence effectively over the whole transform, causing the stator flux jitter to regress naturally into the transient monodirectional service network.



Under this new system, the event stream is fed directly to a high-in-fast-out jimmy loop which heats up the 3-phase quantum decay region of the CPU cache, The functor transferrence occurs in O(3.2 n^kx * frx(q)) via the use of a Deimann-Schleiz algorithm to apply the q-stator in constant eigenspace.

The arity of q can be trivially proven to be non-congruent with the 3-phase quantum decay's step region via induction over the set of polyhesive truth assertions within the reference frame.



Implementation for the infabricated k-factor blowoff quotient should land in libnih by the end of the quarter.

Teach me a story

A friend of mine linked me a semi-technical blogpost about a parsing technique (completely unrelated to my last post). Its basically a cute little fiction story told in first person in a heavily anime-trope-influenced universe, as a frame for an introductory discussion on PADS.

Certainly its a bit more fluff than substance in this example, but I always liked the idea of softening instruction with a little creative writing, particularly for new users or small children, where the reader is coming into a very alien environment.

I think what appeals to me about this teaching technique is it gets the reader accustomed to learning how things work, rather than just trying to demonstrate how to get a particular behavior out of them.

To back up for a moment, consider this synopsis of Star Wars: "There's an Empire and a Rebellion and they try to kill each other." Granted its short, but even forgiving that for a moment, you can give a much more interesting description of Star Wars in the same space. While factual, it omits everything we find interesting in story; from the characters and their feelings to the people, customs, and politics of that "galaxy far far away." You might not realize how much you really absorb from the scenery in a fictional work. Sure Star Wars nerds can tell you what company manufactured the Imperial star destroyers (Kuat Drive Yards, but I only know by proxy I swear), but just about anyone who's seen The Empire Strikes Back knows that the Jawas were a nomadic tribe of traders on Tatooine (even if to you its "those little druid guys" and "that desert place.") Heck, even if you slept through most of it you probably know that sand people always travel single file to hide their numbers.

Now lets pair that off with "when the computer is slow and that light on the side blinks you need to close some programs." We're talking about something technical now, its a different mode of writing. But is it still deficient for the same reason as our Star Wars tag line? Well, consider that opening text that scrolls by in Star Wars as a substitute for our one-line description. Now consider the equivalent for our technical statement: "Programs on your computer store themselves and their important information in RAM while they're running, and when you run out of RAM some of that information is moved to your hard drive. It takes a long time to move things to your hard drive and a long time to read things on your hard drive compared to doing the same in RAM, so if this happens a lot it can make your computer slower. Closing some programs will usually help." I made that about as friendly and gentle as I could. My mother would still be hopeless with it, and she can generally follow most feature films.

In technical writing, when the audience is an extremely non-technical case of "user," any sort of "backstory" on how the thing you are describing works is met very quickly with "whatever, get to the point." Fiction is different. Motivation is imperative for character development, and a believable universe is necessary to make the story relatable, and while any element of fiction can be used to bludgeon the audience to death, they're willing, at least passively, to learn about the larger mechanics of the world where the story takes place.

My theory is that by practising technical and creative writing in close proximity, you can disarm the user into learning how things work. If the inner whirring of cogs in the user's operating system is fundamental to the hero getting the girl, maybe the user will swallow it easier. The goal is to get them to stop looking at the computer as something that is in the way any time they are consciously aware of it. You don't have to think about the mechanics of leverage and cutting paper to use a pair of scissors, but a computer is a general purpose device, and applying it to a new purpose requires understanding its nature. Enjoying a story is similar in this way; if the story doesn't seem to take place in a universe that continues after the story is over, with characters that live their lives when we're not watching them, its not interesting.

Now its 2am and I need a cheeseburger and sleep. As a closing question, are we using this sort of technique anywhere in Fedora's documentation? Are any other projects out there doing this sort of thing? Should you be?

Why you should be using libnih allocation

After reading Stephen Gallagher's planet post on talloc it occurred to me that it might be worth pimping Upstart's own malloc wrapper, and the nifty little toolbox it comes with.

libnih started life as a sort of growth on the side of Upstart; not really a standalone project, not really part of Upstart itself. Recently its come into its own as a stable library, and its available in Fedora 12 and up.

Central to libnih is nih_alloc, which began life as a talloc-like API. Since then, it has grown one important distinction: multiple parentage.

#include <nih/alloc.h>
#include <nih/list.h>

NihList *queue_a;
NihList *queue_b;

int
send_message (char *msg_text,
              int   send_to_a,
              int   send_to_b)
{
        nih_local char *message = nih_strdup (NULL, msg_text);

        if (! message)
                return -1;

        if (send_to_a)
                send_to_queue (queue_a, message);

        if (send_to_b)
                send_to_queue (queue_b, message);

        return 0;
}

int
send_to_queue (NihList *queue, char *message)
{
        NihListEntry *entry = nih_list_entry_new (queue);

        if (! entry)
                return -1;

        entry->str = message;
        nih_ref (message, entry);
        nih_list_add (queue, entry);

        return 0;
}

Let's look at the libnih calls one by one:

• nih_strdup duplicates the given string into an libnih-allocated object. The first argument specifies an initial parent object for the allocation. All objects must have parents, and when an object has no more parents it is freed. NULL can be a parent, as it is here.


• nih_list_entry_new creates a new NihList object. The first argument, again, specifies a parent.


• nih_ref adds a parent to a libnih-allocated object.


• nih_list_add is part of libnih's NihList linked list API, and does what it sounds like.

We skipped the nih_local keyword, but I'll come back to it.

In our example, the new message string we create is created with one parent, NULL. Each time it gets passed to send_to_queue, it gets one additional parent, in the form of a new list entry in the appropriate queue. Assuming we disposed of the list entries as we processed the queue, the object would loose one parent each time a queue finished with it. Also, since the list entries have the queue they are in as their only parent, they would be freed if we freed an entire queue, and our message string would lose them as parents.

But what about that NULL parent? How does it go away allowing the object to be freed? That's where nih_local comes in. nih_local uses a glibc extension to specify a destructor for the variable it is used with. That destructor will call nih_discard on whatever is in message. nih_discard in turn is equivalent to nih_unref (.., NULL), i.e. it removes one NULL parent from an object. In other words the NULL parent goes away as soon as send_message returns. If we called with both of the send_to arguments as 0, then message will be freed immediately. Otherwise message will be freed when both queues have dispensed with it.

In addition to other allocation bells and whistles, like destructors, there's a lot more to libnih besides an allocator. It has everything from a hash table to a dbus API wrapper. It's also available for experimentation in F12 right now. Go check it out.

Parsing

I'm about as anal about source code as anyone can get. I loose tooth enamel over long functions and winding spaghetti code, and even still, I have to say Upstart's code quality is pretty good. Its mostly straightforward, pretty well partitioned, and its about 30-50% comment by volume in most places.

Then you get to configuration file parsing.

Upstart grew up alongside libnih, a general purpose "toolbox" library full of useful generic building blocks for low-level C code. Upstart's config file format is parsed by nih-config. This is where the issues begin. nih-config is an awkward beast. A bit more than a tokenizer, a bit less than a parser, its clumsy and tends to leave the user with a good bit of the parsing to do by themselves.

The Right Way to do anything in programming is, of course, to steal the work of people who already did it better. So I went on my way to find pre-existing parser generators and parsing libraries.

flex/bison was the name I'd heard most before. Its not perfect but it more or less does the job. Its failing, though, can be summarized in 3 words: start on start.

The problem with flex/bison is that they need to separate the string into larger tokens, and they need to do this without examining the progress of parsing so far, which means that we can't tell bison that the first start is a keyword and the second start is an identifier.

On the other hand, take a look at treetop. Treetop is based on Parsing Expression Grammars, which are a lot like regular expressions, but with added firepower that lets them handle context-free languages.

To explain that better for non-automata-theory-nerds, try to write a regex that looks at a string of < and > characters and determines if they're well nested (i.e. <<>><> matches but <>>< doesn't). It can't be done. Regexes don't have that power.

But now imagine that you could have many regexes assigned to variables, and that you could "use" one regex from inside another, so you could write "a followed by b followed by something that matches regex C". If we allow for recursion, we can solve our problem easily.

A = /^(<$A>|)$/

(Note, I'm assuming you're using a regex dialect where < and > aren't meaningful).

So Treetop gives us this power, but again, its no good. Treetop is for Ruby. Upstart is not in Ruby and never will be. We need something with Treetop's powers, that's written in, or at least can output, C code.

And that's exactly what I've been writing.

Moving to mutt

I've been a long time Thunderbird user for reasons I don't entirely understand.

T-bird replaced Evolution for me after I couldn't stand that any more, and it was certainly a step up, but its never been more than tolerable. It liked to freeze for 15 seconds while it was busy, and it never quite did threading the way I wanted. Message formatting could be a pain too, which was often embarrassing, and lead to my ultimately being coaxed into giving mutt another look. So, for the past 5 hours, I've been a mutt user.

It's pretty ok.

I like getting rid of the mouse, as always, but there's still some things in the UI I'm missing:

1. The list of folders I get with c? doesn't show me which ones have new messages.


2. Changing folders is agonizingly slow, even with a local header cache. I probably don't need a 140,000-message archive of lkml for the past few months, but why can't I have it?


3. No Thunderbird means no Lightning, so now I need calendar software. Not really mutt's fault, but something to consider. I'll also need a new RSS feed reader.

Its definitely lighter and faster, and email in vim is a huge win, but its going to take some time before it really flows for me.

"Right-click-Save-As" notices considered harmful

Look at this web page. Now ignore for a moment the lack of CSS and general web-0.1 feel, and you'll notice an offence against one of my biggest pet peeves: a request that we "right click and save as."

I've seen it on all manner of website, with all manner of professional graphic artistry surrounding it, and I'd like to set the record straight: this instruction is a completely unnecessary inconvenience to the user.

The UI complaints against the action are obvious: we're asking the user to interact in a completely unintuitive way to do something they expect to be straight forward. "Click link, receive file" is the user's instinct, and the extra, awkward step can only trip them up. This isn't news; nobody wants their site to work this way. The problem is nobody is communicating that there is a better alternative.

Suppose we have a site with a large number of movies/documents/text files/et cetera in a downloads folder that we'd like to make sure always get offered to the user as a download rather than content in the browser window. All we need is a little Apache configuration like so:

<Directory>
  Header set Content-Disposition attachment
</Directory>

Now anything served out of that folder will prompt the user for a download location and be saved to their hard drive by the browser. It works just about everywhere. Even IE mostly gets it right (one exception and more configuration examples here).

For web application writers, just add "Content-Disposition: attachment" to your headers by whatever means your language/framework of choice prefers when you want your generated page to be downloaded rather than displayed.

This has been here for over a decade and people still don't seem to have figured it out. Tell your friends! Stop abusing your users! Together, we can end poor understanding of HTTP features.

Japan Linux Symposium

I was saving this post for when I had the pictures from my trip online, but Flickr is fighting me on that front, so I'm just going to have to wing it.

JLS was my first time out of the US. Tokyo was everything I imagined, and I'm already trying to find a way to go back. Its a fantastic city.

There weren't quite as many familiar faces at JLS as there were at other conferences I've been to, which was a nice excuse to make friends. I did bump into Marcel (who seems to update his blog less than I do :) and Dougsland (who seems to have no linkable internet presence), but I found myself shaking a lot more strangers' hands. I did manage to chat a little with Matz (who I assume has a blog somewhere, but here's his wikipedia for now). I thanked him for showing me how bad Java was.

Most of the technical bits of the conference seemed to be about tracing tools. Ftrace came up in 3 separate talks. I got the sense this had to do with how Linux is used in Japan. Getting quicker resolutions to problems was a big deal.

I mention "technical bits" of the conference because there seemed to be a slightly larger than average number of non-technical talks; particularly open source evangelism. This was probably the strangest thing about JLS: the sense that the Open Source way hadn't yet been embraced by the audience. Japan's industry knows Linux is something they want, but open source is strange to most of them, and getting them comfortable with free software was a continual theme.

On the Upstart front: I've been hacking on everything from a new config parser (the old one was a bit hairy) to the beginnings of the increasingly stabilized 1.0 semantics, to a few niceties for 0.6 that should help to get Fedora moved over in the interim. I'll try to detail those 1.0 semantics in a coming post, and I'll have more about my travels in Tokyo when I get the pictures online.