Screw You Enterprise Edition

Upstart: Service Mangement modeled as a bi-parental network of stator phase flip-flops

2010-04-01T11:15:00.000-07:00

One of the problems Upstart has encountered recently is ensuring service starting with the appropriate log-n phase delay when given a bi-quantum dependency network over a Von Nelson stator grid. Recently we proposed a radical new model for service management centred around a bi-parental network of stator-phase flip-flops, bijected directly over the transient monodirectional service network.

While this prevented degradation due to non-selmann adhesion, stator flux jitter continued to oscillate in a pseudo-sapient Le Chevalier sequence, resulting in bled q-functors due to a 3-7 paradox arising between the node-region offsets.

To mitigate this, side slip is prevented via the use of a k-factor blowoff quotient directly infabricated with the 4th arity q-functor. This applies a L'hospital retrodamping sequence effectively over the whole transform, causing the stator flux jitter to regress naturally into the transient monodirectional service network.

Under this new system, the event stream is fed directly to a high-in-fast-out jimmy loop which heats up the 3-phase quantum decay region of the CPU cache, The functor transferrence occurs in O(3.2 n^kx * frx(q)) via the use of a Deimann-Schleiz algorithm to apply the q-stator in constant eigenspace.

The arity of q can be trivially proven to be non-congruent with the 3-phase quantum decay's step region via induction over the set of polyhesive truth assertions within the reference frame.

Implementation for the infabricated k-factor blowoff quotient should land in libnih by the end of the quarter.

Teach me a story

2010-03-22T22:01:00.000-07:00

A friend of mine linked me a semi-technical blogpost about a parsing technique (completely unrelated to my last post). Its basically a cute little fiction story told in first person in a heavily anime-trope-influenced universe, as a frame for an introductory discussion on PADS.

Certainly its a bit more fluff than substance in this example, but I always liked the idea of softening instruction with a little creative writing, particularly for new users or small children, where the reader is coming into a very alien environment.

I think what appeals to me about this teaching technique is it gets the reader accustomed to learning how things work, rather than just trying to demonstrate how to get a particular behavior out of them.

To back up for a moment, consider this synopsis of Star Wars: "There's an Empire and a Rebellion and they try to kill each other." Granted its short, but even forgiving that for a moment, you can give a much more interesting description of Star Wars in the same space. While factual, it omits everything we find interesting in story; from the characters and their feelings to the people, customs, and politics of that "galaxy far far away." You might not realize how much you really absorb from the scenery in a fictional work. Sure Star Wars nerds can tell you what company manufactured the Imperial star destroyers (Kuat Drive Yards, but I only know by proxy I swear), but just about anyone who's seen The Empire Strikes Back knows that the Jawas were a nomadic tribe of traders on Tatooine (even if to you its "those little druid guys" and "that desert place.") Heck, even if you slept through most of it you probably know that sand people always travel single file to hide their numbers.

Now lets pair that off with "when the computer is slow and that light on the side blinks you need to close some programs." We're talking about something technical now, its a different mode of writing. But is it still deficient for the same reason as our Star Wars tag line? Well, consider that opening text that scrolls by in Star Wars as a substitute for our one-line description. Now consider the equivalent for our technical statement: "Programs on your computer store themselves and their important information in RAM while they're running, and when you run out of RAM some of that information is moved to your hard drive. It takes a long time to move things to your hard drive and a long time to read things on your hard drive compared to doing the same in RAM, so if this happens a lot it can make your computer slower. Closing some programs will usually help." I made that about as friendly and gentle as I could. My mother would still be hopeless with it, and she can generally follow most feature films.

In technical writing, when the audience is an extremely non-technical case of "user," any sort of "backstory" on how the thing you are describing works is met very quickly with "whatever, get to the point." Fiction is different. Motivation is imperative for character development, and a believable universe is necessary to make the story relatable, and while any element of fiction can be used to bludgeon the audience to death, they're willing, at least passively, to learn about the larger mechanics of the world where the story takes place.

My theory is that by practising technical and creative writing in close proximity, you can disarm the user into learning how things work. If the inner whirring of cogs in the user's operating system is fundamental to the hero getting the girl, maybe the user will swallow it easier. The goal is to get them to stop looking at the computer as something that is in the way any time they are consciously aware of it. You don't have to think about the mechanics of leverage and cutting paper to use a pair of scissors, but a computer is a general purpose device, and applying it to a new purpose requires understanding its nature. Enjoying a story is similar in this way; if the story doesn't seem to take place in a universe that continues after the story is over, with characters that live their lives when we're not watching them, its not interesting.

Now its 2am and I need a cheeseburger and sleep. As a closing question, are we using this sort of technique anywhere in Fedora's documentation? Are any other projects out there doing this sort of thing? Should you be?

Why you should be using libnih allocation

2010-03-18T07:58:00.000-07:00

After reading Stephen Gallagher's planet post on talloc it occurred to me that it might be worth pimping Upstart's own malloc wrapper, and the nifty little toolbox it comes with.

libnih started life as a sort of growth on the side of Upstart; not really a standalone project, not really part of Upstart itself. Recently its come into its own as a stable library, and its available in Fedora 12 and up.

Central to libnih is nih_alloc, which began life as a talloc-like API. Since then, it has grown one important distinction: multiple parentage.


#include <nih/alloc.h>
#include <nih/list.h>

NihList *queue_a;
NihList *queue_b;

int
send_message (char *msg_text,
              int   send_to_a,
              int   send_to_b)
{
        nih_local char *message = nih_strdup (NULL, msg_text);

        if (! message)
                return -1;

        if (send_to_a)
                send_to_queue (queue_a, message);

        if (send_to_b)
                send_to_queue (queue_b, message);

        return 0;
}

int
send_to_queue (NihList *queue, char *message)
{
        NihListEntry *entry = nih_list_entry_new (queue);

        if (! entry)
                return -1;

        entry->str = message;
        nih_ref (message, entry);
        nih_list_add (queue, entry);

        return 0;
}

Let's look at the libnih calls one by one:

nih_strdup duplicates the given string into an libnih-allocated object. The first argument specifies an initial parent object for the allocation. All objects must have parents, and when an object has no more parents it is freed. NULL can be a parent, as it is here.

nih_list_entry_new creates a new NihList object. The first argument, again, specifies a parent.

nih_ref adds a parent to a libnih-allocated object.

nih_list_add is part of libnih's NihList linked list API, and does what it sounds like.

We skipped the nih_local keyword, but I'll come back to it.

In our example, the new message string we create is created with one parent, NULL. Each time it gets passed to send_to_queue, it gets one additional parent, in the form of a new list entry in the appropriate queue. Assuming we disposed of the list entries as we processed the queue, the object would loose one parent each time a queue finished with it. Also, since the list entries have the queue they are in as their only parent, they would be freed if we freed an entire queue, and our message string would lose them as parents.

But what about that NULL parent? How does it go away allowing the object to be freed? That's where nih_local comes in. nih_local uses a glibc extension to specify a destructor for the variable it is used with. That destructor will call nih_discard on whatever is in message. nih_discard in turn is equivalent to nih_unref (.., NULL), i.e. it removes one NULL parent from an object. In other words the NULL parent goes away as soon as send_message returns. If we called with both of the send_to arguments as 0, then message will be freed immediately. Otherwise message will be freed when both queues have dispensed with it.

In addition to other allocation bells and whistles, like destructors, there's a lot more to libnih besides an allocator. It has everything from a hash table to a dbus API wrapper. It's also available for experimentation in F12 right now. Go check it out.

Parsing

2010-03-16T12:13:00.000-07:00

I'm about as anal about source code as anyone can get. I loose tooth enamel over long functions and winding spaghetti code, and even still, I have to say Upstart's code quality is pretty good. Its mostly straightforward, pretty well partitioned, and its about 30-50% comment by volume in most places.

Then you get to configuration file parsing.

Upstart grew up alongside libnih, a general purpose "toolbox" library full of useful generic building blocks for low-level C code. Upstart's config file format is parsed by nih-config. This is where the issues begin. nih-config is an awkward beast. A bit more than a tokenizer, a bit less than a parser, its clumsy and tends to leave the user with a good bit of the parsing to do by themselves.

The Right Way to do anything in programming is, of course, to steal the work of people who already did it better. So I went on my way to find pre-existing parser generators and parsing libraries.

flex/bison was the name I'd heard most before. Its not perfect but it more or less does the job. Its failing, though, can be summarized in 3 words: start on start.

The problem with flex/bison is that they need to separate the string into larger tokens, and they need to do this without examining the progress of parsing so far, which means that we can't tell bison that the first start is a keyword and the second start is an identifier.

On the other hand, take a look at treetop. Treetop is based on Parsing Expression Grammars, which are a lot like regular expressions, but with added firepower that lets them handle context-free languages.

To explain that better for non-automata-theory-nerds, try to write a regex that looks at a string of < and > characters and determines if they're well nested (i.e. <<>><> matches but <>>< doesn't). It can't be done. Regexes don't have that power.

But now imagine that you could have many regexes assigned to variables, and that you could "use" one regex from inside another, so you could write "a followed by b followed by something that matches regex C". If we allow for recursion, we can solve our problem easily.

A = /^(<$A>|)$/

(Note, I'm assuming you're using a regex dialect where < and > aren't meaningful).

So Treetop gives us this power, but again, its no good. Treetop is for Ruby. Upstart is not in Ruby and never will be. We need something with Treetop's powers, that's written in, or at least can output, C code.

And that's exactly what I've been writing.

Moving to mutt

2009-11-30T18:59:00.001-08:00

I've been a long time Thunderbird user for reasons I don't entirely understand.

T-bird replaced Evolution for me after I couldn't stand that any more, and it was certainly a step up, but its never been more than tolerable. It liked to freeze for 15 seconds while it was busy, and it never quite did threading the way I wanted. Message formatting could be a pain too, which was often embarrassing, and lead to my ultimately being coaxed into giving mutt another look. So, for the past 5 hours, I've been a mutt user.

It's pretty ok.

I like getting rid of the mouse, as always, but there's still some things in the UI I'm missing:

The list of folders I get with c? doesn't show me which ones have new messages.

Changing folders is agonizingly slow, even with a local header cache. I probably don't need a 140,000-message archive of lkml for the past few months, but why can't I have it?

No Thunderbird means no Lightning, so now I need calendar software. Not really mutt's fault, but something to consider. I'll also need a new RSS feed reader.

Its definitely lighter and faster, and email in vim is a huge win, but its going to take some time before it really flows for me.

"Right-click-Save-As" notices considered harmful

2009-11-10T15:08:00.000-08:00

Look at this web page. Now ignore for a moment the lack of CSS and general web-0.1 feel, and you'll notice an offence against one of my biggest pet peeves: a request that we "right click and save as."

I've seen it on all manner of website, with all manner of professional graphic artistry surrounding it, and I'd like to set the record straight: this instruction is a completely unnecessary inconvenience to the user.

The UI complaints against the action are obvious: we're asking the user to interact in a completely unintuitive way to do something they expect to be straight forward. "Click link, receive file" is the user's instinct, and the extra, awkward step can only trip them up. This isn't news; nobody wants their site to work this way. The problem is nobody is communicating that there is a better alternative.

Suppose we have a site with a large number of movies/documents/text files/et cetera in a downloads folder that we'd like to make sure always get offered to the user as a download rather than content in the browser window. All we need is a little Apache configuration like so:


<Directory>
  Header set Content-Disposition attachment
</Directory>

Now anything served out of that folder will prompt the user for a download location and be saved to their hard drive by the browser. It works just about everywhere. Even IE mostly gets it right (one exception and more configuration examples here).

For web application writers, just add "Content-Disposition: attachment" to your headers by whatever means your language/framework of choice prefers when you want your generated page to be downloaded rather than displayed.

This has been here for over a decade and people still don't seem to have figured it out. Tell your friends! Stop abusing your users! Together, we can end poor understanding of HTTP features.

Japan Linux Symposium

2009-11-08T13:42:00.000-08:00

I was saving this post for when I had the pictures from my trip online, but Flickr is fighting me on that front, so I'm just going to have to wing it.

JLS was my first time out of the US. Tokyo was everything I imagined, and I'm already trying to find a way to go back. Its a fantastic city.

There weren't quite as many familiar faces at JLS as there were at other conferences I've been to, which was a nice excuse to make friends. I did bump into Marcel (who seems to update his blog less than I do :) and Dougsland (who seems to have no linkable internet presence), but I found myself shaking a lot more strangers' hands. I did manage to chat a little with Matz (who I assume has a blog somewhere, but here's his wikipedia for now). I thanked him for showing me how bad Java was.

Most of the technical bits of the conference seemed to be about tracing tools. Ftrace came up in 3 separate talks. I got the sense this had to do with how Linux is used in Japan. Getting quicker resolutions to problems was a big deal.

I mention "technical bits" of the conference because there seemed to be a slightly larger than average number of non-technical talks; particularly open source evangelism. This was probably the strangest thing about JLS: the sense that the Open Source way hadn't yet been embraced by the audience. Japan's industry knows Linux is something they want, but open source is strange to most of them, and getting them comfortable with free software was a continual theme.

On the Upstart front: I've been hacking on everything from a new config parser (the old one was a bit hairy) to the beginnings of the increasingly stabilized 1.0 semantics, to a few niceties for 0.6 that should help to get Fedora moved over in the interim. I'll try to detail those 1.0 semantics in a coming post, and I'll have more about my travels in Tokyo when I get the pictures online.

Linux Plumber's Conference

2009-09-30T19:36:00.000-07:00

This is late, but, I went to LPC last week.

Hilights:

Wayland is really impressive looking. I spent a lot of time in the graphics track even though I don't really know much about X, and while I only barely grasped most of what was shown in the way of new X features, I got the whole Wayland architecture in my head in a few short minutes. That's called reducing complexity (it also lacks a homepage, hence the Phoronix link).

Caught up with Scott, naturally. Upstart finally has a direction that I think we both understand. Its reached one of those joyous points where the code is actually shrinking as we add new features, which gives everyone a warm fuzzy.

From what conversations I had with other distro folk, Dracut is in fact poised to take over the world. Despite the initial grumblings from Debian types, there's actually a lot of cooperation forming around it.

LAX sucks. A lot.

What a long strange trip its been...

2009-09-05T13:40:00.000-07:00

It's been awhile since I posted here.

I've never been really sure how to write on this blog. This may be the reason I held off on getting one for a long time. Like all good American grade-schoolers, I was taught to begin the writing process by "defining my audience." This has been the greatest challenge of writing here.

Early on, I posted with the (largely correct) sentiment that nobody was listening. It felt awkward to write this way, and it feels awkward to go back and read those posts now. Its a natural reaction. Talking to yourself makes you self-conscious. Try it.

Eventually I started working on Upstart, and this blog became a sort of dumping ground for what I was doing there. This had an audience, but not a tremendously well-defined one. Upstart still hasn't grown the pieces it needs to be exciting for most of Fedoraland (where I'm pretty sure most of my traffic comes from), and the brain-dumping I've been doing here increasingly has better venues.

Now, in recent months, its gone nearly silent. Plenty has happened in the meantime. I graduated, became a full time Red Hat employee, and Upstart's meandering development process is finally beginning to take direction. There's been plenty of interesting stuff going on, and yet I've been much less compelled to update.

So I suppose the best thing to do is just ask whoever's listening to report in. Who's out there? What would you like to read on this blog? What do you blog about and who do you do it for?

Upstart's return to open source

2009-06-02T05:51:00.000-07:00

As pointed out in the comments, this is no longer current. Scott has returned development of Upstart 1.0 to the public eye.

Now that participation is open again, I'd like to encourage anyone who hasn't caught it by now to take a look at Scott's talk from FOSDEM.

A lot of 1.0 is being defined from what's in this talk. Regular readers will probably find that 1.0 doesn't look like what they expected. Scott chose a different design, and I agree with his decision.

A word on the "New Upstart State Machine" series itself: the purpose of the state machine I outlined was simple: prevent another rewrite. It was designed to provide expressive fire power and do nothing else. The hope was to break the cycle of Upstart releases up to that point:

1. Decide on use cases A, B, and C
2. Design and implement Upstart with the tools to meet those cases.
3. Discover use case D, realize that Upstart is useless if it does not meet it.
4. Scrap everything and go to 1

The challenge in designing an Upstart that would beat that is difficult indeed: design software for the use cases we hadn't thought of yet. My approach was to simply develop a very abstract system which was good at generalizing a great many things. Scott's approach was the same, he was simply more conservative in the amount of abstraction applied. He came up with a paired down sibling of object oriented programming, I came up with a hopped up version of second order logic. Ultimately his solution is eons more intuitive (which is not to say simpler. I'd say both systems have a pretty similar number of moving parts), and I'm pleased with the direction things are heading.

Now to code it.

A Message from Scott to the Open Source Community

2009-04-14T07:00:00.000-07:00

On Tue, 2009-04-14 at 00:42 -0700, Saravanan Shanmugham (sarvi) wrote:

> >    I saw the plan/roadmap presentation for 1.0, but i could not figure
> > out which branch in lauchpad its being done on.
> > 
Hi Sarvi,

I'm developing the next version of Upstart largely in private at the
moment, so you won't find it on Launchpad.

I realise that this is somewhat unusual for an Open Source project, but
this next version is almost a complete new attempt at a second "first
version" and I want to get particular basics right before releasing the
code for others to work on.

I'll be working on it pretty much full time until the initial release,
hopefully in June.

Scott

Thanks Scott. Good to see you placing so much trust in the community.

If anything about the above announcement bothers you, gentle reader, please reply on upstart-devel.

StumpWM

2009-02-15T15:26:00.000-08:00

I've been a GNU screen user for a long time, and recently I went in search of tiling window managers that would emulate its behaviour for the rest of my desktop. I found Ratpoison first, which mostly did what I wanted but lacked a mode line equivalent. Next I found StumpWM

I like it.

Its mostly the same UI as screen. The default escape key is C-t rather than C-a (I changed mine back to C-a since I wanted to use it as a complete screen replacement as well as a window manager). Another minor difference is you can't display the same window in two regions like you can in screen, which is primarily an X11 limitation, but does make it less intuitive when you want to swap two windows and there's only two windows open (the dedicated keys/commands for doing this actually make it faster, but its sent me to the manual every time so far).

The config file is in common lisp (as is the whole application) which takes some getting used to, but ultimately it was nice to have a full programming language available. I learned a little common lisp just to use it. The usual screen-style commands are also around, but they aren't used in .stumpwmrc unless you enclose them in a lisp statement ((run-commands 'my-screenish-command')). This also creates another screen difference: C-a : brings up a lisp evaluation prompt, not a prompt for screen-type commands. C-a ; will do that, and the motivation was that the latter was easier to type, but for familiarity's sake, I remapped the two into each other's place.

Workspaces can be dealt with by names or numbers, with C-a F[1-10] bringing up the first 10 workspaces. I ended up just setting up some mappings in the config file that would make it behave like Metacity did for my 4 main workspaces, but having easy access to further workspaces without having to compromise my 2D "sense" of their position is convenient for those special cases when I have more windows than usual.

Stability wise its not been everything I dreamed of. It has occasionally crashed when closing certain windows, and a few of the config options don't seem to work (which is why my modeline is still on top). This could also have something to do with the shoe-horning that I had to do to get it into a full GNOME session.

Lastly packaging issues. The Makefile is non-autoconf and didn't have a good way to install to RPM_BUILD_ROOT, so my specfile installed manually. Then disaster: the binary was corrupt when I built into an RPM. This seems to have something to do with prelink. Even when I managed to get the md5 issues to disappear, the executable was 1/4 the size it was when I built it manually and complained of missing libraries that the manually-built version didn't. I'll be asking our clisp maintainer about this issue. I'd like to be able to use it as a fedora package.

Go away in irssi whenever your screen is locked

2009-01-20T21:15:00.000-08:00

My friends complain that I never set an away message. This is due to habit, and has a lot to do with the fact that irssi doesn't set me auto-away like my old clients used to.

But no more!

You will need this and this:


#! /usr/bin/env python

from dbus.mainloop.glib import DBusGMainLoop
import dbus
import socket
import gobject

DBusGMainLoop(set_as_default=True)

bus = dbus.SessionBus()

def handler(locked):
    client = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    client.connect("/tmp/irssi_socket")
    if locked:
        client.send("command /away afk")
    else:
        client.send("command /away")
    print client.recv(1024)
    client.close()
bus.add_signal_receiver(handler, dbus_interface = "org.gnome.ScreenSaver", signal_name="ActiveChanged")

gobject.MainLoop().run()

Just install the Perl script into irssi, and then run this python service somewhere in your desktop session. Any time your screensaver turns on, you'll go away.

The New Upstart State Machine: A nearly complete python prototype

2009-01-19T20:53:00.000-08:00

I've been busy.

Scott has been long afraid that we'd write an operating system by mistake. The fact that this resembles more closely some rudimentary form of AI is... well I don't know what to make of it.

At any rate, this is a nearly complete simulation. To monkey around with it, scroll to the bottom of statemachine.py and have a look at the configuration (a fully functional text configuration format is built in). The default main loads a fairly simple config and then activates a few of the states/events it mentions.

I'd like to get an interactive console going for monkeying about, and possibly even some rudimentary process management to give it a little live-fire workout.

But as for the present...

The few who actually followed my last series on how this mechanism should work might notice that the implementation deviates a bit. The good news is that this is in every case a simplification. The final version is lighter and easier to follow. It primarily operates on groups of states which I have dubbed "Categories," which are sets of a particular form. Its all much easier to get out of the code than out of me, so why not try it? :D

-j4

2009-01-15T23:51:00.000-08:00

I've been aware of make's parallel building capabilities forever. I just never felt motivated to try them out (being usually on my ancient single-core laptop) Now that I'm on a quad-core box full time I figured it'd be worth a shot.

wow.

make -j4 really was close to 4 times as fast on a kernel build.

<3

NO! BAD!

2009-01-11T12:31:00.000-08:00

The other day I popped a disk for an old windows game into my F10 laptop and got a popup along the lines of:

"This disk contains software designed to be run automatically. Do you want to run it?"

NO! BAD LINUX!

Microsoft is still "the one to beat" on the desktop in terms of market volume, sure, but what desperate inferiority complex is compelling us to replicate their most hideous anti-features? I don't care that we put a warning dialogue in the way (most interaction designers are getting the message now that those don't work). This is just badness. You're not even saving the user a click any more, and even if you were, its a stupid heuristic anyway.

A deceptive grammatical question

2009-01-02T04:47:00.001-08:00

I got into an interesting grammar debate with someone over the following sentence recently:

The pool was deceptively shallow.

The other party in the argument claimed that the sentence was ambiguous; that the pool could be either deep or shallow. I claimed that the sentence meant that the pool was shallow, and that there was no other correct interpretation.

As a counterexample, she offered this:

The test was deceptively easy.

She claimed that the reader interprets this to mean the test was hard, since tests are usually associated with being hard. I maintained that the test was easy by this statement.

My understanding of adverbs enhancing adjectives has always been that they alter degree, but not meaning. By this understanding, any sentence with an adverb enhancing an adjective should have similar meaning if you cross out the adverb:

The test was deceptively easy.

To further illustrate the point, consider the following two permutations:

The test was easy; deceptively so.
We thought the test would be easy, but in fact it was deceptively so.

When we don't put the two directly in line, the debate is much weaker. The second form, which would support the inverting property of "deceptively," seems evidently nonsensical.

Distro meme

2008-12-28T18:06:00.000-08:00

1) Which was your Linux distribution story?

Fedora Core 4! I had a brief stint with Ubuntu (measured in hours, not days or weeks), and a long miserable affair with CentOS 4 (Not a desktop OS), but other than that its been Fedora.

2) What is your preferred $your_distribution version?

Fedora 9

3) Write a short story (more like and anecdote) about your past distribution.

Don't ask a question. The answer is always JFGI. Just assume that you know the answer, and then before acting on it, state it loudly, fiercely, and with complete certainty, in a public place, in front of people who know better. Nobody gives information to smart people. Prove you're stupid and they'll devote hours and hours to your education.

The New Upstart State Machine, Explained in Python

2008-12-22T21:35:00.000-08:00

I now have a much less fail example, written in Python. Its incomplete but enough is there.

Get it here

The New Upstart State Machine Part 4

2008-12-18T11:56:00.000-08:00

Continued from here.

All we need to make our state machine complete is something to cause holds to be placed from externally. This is where events come into play.

An event can be simply described as "a moment in time at which the hold counter(s) of particular states are increased." We can take this most literally if we look at the cron functionality which has been talked about for upstart for some time.

at 3:53pm logrotate()++

This means a hold would be placed on logrotate() at 3:53pm.

Most events aren't described as precise times though, but rather by a name. Take our found_disk(*) states from the previous part.

on devkit.newdisk[label: %1, devname: %2, uuid: %3] : found_disk(label: %1, devname: %2, uuid: %3)++

This event would be triggered by DeviceKit itself sending a message to upstart. Take note of the arguments. Events can have parameters just like states, and can match the states they trigger based on them.

Having an event release a state is a bit stranger. A state can only be stopped by an event when it was started by an event. Furthermore, the particular pair of events must be explicit. Lets refine our found_disk(*) so DeviceKit can remove it when the disk goes away.

in devkit.newdisk[label: %1, devname: %2, uuid: %3]..devkit.removeddisk[label: %1, devname: %2, uuid: %3] : found_disk(label: %1, devname: %2, uuid: %3)++

There's one more case we need. What about those mounted(*) states in our last example? We'd like those to be started as soon as they are available.

on ɛ : mounted(*)

ɛ is an event, which when triggered will start any mounted(*) states. What is unique about ɛ is when it is triggered. ɛ occurs any time something would happen as a result. In the case of mounted(*) it will occur as soon as any dependencies are satisfied. If it is simpler, you can think of ɛ as an event that occurs "every ten-thousandth of a millisecond" or "on every CPU cycle" (I emphasize this is definitely not how it will be implemented). Any state triggered on ɛ will come up at the earliest possible moment.

The New Upstart State Machine Part 3

2008-12-16T16:44:00.000-08:00

Continued from here.

Well, we have a lot of rules governing when a state can be up, but when does it become up?

Each state has a "hold counter," which is simply a positive integer. When that number is zero, the state is down, when it is nonzero, the state is up. "Placing a hold" means incrementing the hold counter of a state, and "releasing a hold" means decrementing it. It is impossible to hold a state that does not have all of its dependencies satisfied.

This might create a contradiction. Consider the following scenario:

astate()→anotherstate()
astate()
anotherstate()

Now, if we place a hold on astate() it comes up, but what if we then release a hold on anotherstate()? This would cause the dependency to be violated.

The solution is that placing a hold on a state automatically places a hold on all of its dependencies, and releasing a hold on a state automatically releases a hold on each of its dependencies. This keeps their hold counters above zero and keeps the dependency satisfied.

These dependency holds are one of three ways a hold can be placed on a state. The second way is events (described in part 4) and the third is intervention of the system administrator (which becomes useful for things like starting a service by hand).

We now have enough understanding of the system to look at a practical example. Lets look at satisfying /etc/fstab, upstart 1.0 style.

For this exercise, we will look at just three fstab lines:

LABEL=myroot / ext3 defaults 1 1
foo-nfs.redhat.com:/vol/home/mydocs /home/mydocs nfs rw,nosuid,nodev 0 0
UUID=905d2c0a-86bf-4ce6-8f92-27b004284413 swap swap defaults 0 0

First, our fstab parsing app parses each of these lines and translates them into the following states:

vol_conf(label: myroot, type: ext3, ops: defaults, fsckpass: 1, freq: 1, mountpoint: /)

vol_conf(devname: foo-nfs.redhat.com:/vol/home/mydocs, type: nfs, ops: rw,nosuid,nodev, fsckpass: 0, freq: 0, mountpoint: /home/mydocs)

vol_conf(uuid: 905d2c0a-86bf-4ce6-8f92-27b004284413, type: swap, ops: defaults, fsckpass: 0, freq: 0, mountpoint: swap)

It then places a hold on all of them.

vol_conf(label: myroot ...)
vol_conf(devname: foo-nfs ...)
vol_conf(uuid: 905d2c0a ...)

Meanwhile, in another corner of userspace, DeviceKit places a hold on three other states:

found_disk(uuid: 905d2c0a..., devname: /dev/sda3, label: swap)
found_disk(uuid: de4dbe3f..., devname: /dev/sda1, label: myroot)
network_available()

The rest of the logic is dependencies:


mounted(uuid: %1, devname: %2, label: %3, type: !nfs, *) → found_disk(uuid: %1, devname: %2, label: %3)

mounted(mountpoint: %1, uuid?: %2, devname?: %3, label?: %4, type: %5) → vol_conf(mountpoint: %1, uuid?: %2, devname?: %3, label?: %4, type: %5)

mounted(type: nfs, mountpoint: %1, devname: %2) → network_available()

Lets look at what is and is not satisfied given this set of dependencies:

mounted(uuid: 905d..., devname: /dev/sda3, type: swap, mountpoint: swap, label: swap) is satisfied for the first dependency (all 3 fields match our first found_disk state), and the second (Our third vol_conf matches mountpoint, uuid, and type. The other two fields are optional). The third doesn't apply since type is not nfs


mounted(type: nfs, mountpoint: /home/mydocs, devname: foo-nfs.redhat.com:/vol/home/mydocs is satisfied for dependency 2 (it matches the second vol_conf, ignoring the optional uuid and label) and dependency 3 (the network is available). Dependency 1 does not apply, as our type is nfs.

mounted(uuid: de4d..., devname: /dev/sda1, type: ext3, mountpoint: /, label: myroot) is satisfied for dependency 1 (again, trivially matches one of our found_disks), and for dependency 2 (matches first vol_conf for all but the optional uuid and devname fields.

mounted(uuid: de4d..., devname: /dev/sda1, type: ext3, label: myroot)
 Is satisfied for dep 1 and does not apply for deps 2 or 3. However, note that it makes no sense (it lacks a mountpoint). We can simply say that such insane states are ignored, and in practice it is fairly easy to implement some simple checks to prevent them from cropping up (they were omitted here for complexity's sake).



Next post we'll explain how to get these three running.



The New Upstart State Machine, Part 2
2008-12-15T17:35:00.000-08:00
Continued from here.

With some understanding of states, we can now explain dependencies.

A dependency simply says "state foo cannot be up unless state bar is also up." In this post I reflect this relationship with an arrow (→) which can be read "depends on" (or "implies").

mystate(crunchy: yes)→thystate(crunchy: yes, id: 7)

In the above dependency, mystate(crunchy: yes) can only be up if thystate(crunchy: yes, id: 7) is up. Take note, mystate(crunchy: no) is not effected by this dependency, as you might expect, but neither is mystate(crunchy: yes, delicious: definitely). Remember, different parameters = different states. thystate in the above example is matched with equal strictness.

Defining dependencies in this manner is rarely useful. Usually we want to define dependencies between large classes of states, not individual ones. For example, suppose we wanted the above dependency to be satisfied for any mystate and thystate with equivalent levels of crunchiness:

mystate(crunchy: %1)→thystate(crunchy: %1, id: 7)

In this notation mystate(crunchy: yes) would depend on thystate(crunchy: yes, id: 7), and mystate(crunchy: sometimes) would depend on thystate(crunchy: sometimes, id: 7), etc.

Suppose we want thystate to have an id, but don't care about its value.

mystate(crunchy: %1)→thystate(crunchy: %1, id: %2)

Now the id parameter is set to match a pattern which doesn't appear anywhere else, meaning it can accept any value.

Its also possible we might have some leeway in what we depend on. Perhaps any thystate with the two parameters we match will serve our purpose.

mystate(crunchy: %1)→thystate(crunchy: %1, id: %2, *)

Now thystate(crunchy: yes, id: 7, crap: 3, morecrap:5) would satisfy mystate(crunchy: yes)'s dependency as well as its more minimal forms above.

Finally, we will occasionally need to specify a parameter that may or may not be present. We use ? to denote this:

mystate(crunchy: %1)→thystate(crunchy?: %1, id: %2, *)

This has interesting meaning: thystate(crunchy: yes, id: 7) will satisfy the dependency for mystate(crunchy: yes), but thystate(id: 7) will satisfy the dependency for mystate(crunchy: anything).

I'm trying to make this as sparse as I know how, and in the process a lot of it feels stranger for the deprivation of context, but hopefully in the coming days, a few more people will know what the hell I'm on about :D


The New Upstart State Machine
2008-12-15T09:55:00.000-08:00
Awhile ago I started working on describing a new "service state machine" for Upstart. This would be a departure from the event driven model, and offer a richer mechanism to describe what should be running at any given time. I think I more or less have the design down.

Now if only I could explain it to someone else :)

This series of posts will attempt to explain to mere mortals the abstract workings of this new wonder. Previous descriptions have proven a bit verbose and mathy, so I'm going to take it very slow this time. It may get a bit too easy, but I'm going to take baby steps no matter what.

I should probably note beforehand that the objective here is to develop a body of logic that can describe services, and then from that produce a program that will govern them. This post and ones after will not describe how the state machine should be implemented but how it should behave. For the curious, implementing it has been somewhat crudely and quite inefficiently prototyped here.

So, first we need to define a state. A state consists of 1) a name, 2) a series of properties.

mystate(crunchy: yes, decaf: no, flavor: immitation_beef)

Also, a state is either up or down. Pay attention to those colors, as I will use them consistently here.

There's a lot of comparing of states that will go on shortly, so it is important that we understand what makes states equal.

mystate(crunchy: yes) != mystate(crunchy: no)
mystate(crunchy: yes) != thystate(crunchy: yes)
mystate(crunchy: yes) == mystate(crunchy: yes)
mystate(crunchy: yes, chewy: yes) == mystate(chewy: yes, crunchy: yes)
mystate(crunchy: yes) == mystate(crunchy: yes)

That last one is important. A state being up or down does not effect its identity for the purposes of comparison.

Now we get into a bit of a mathy idea. Its not hugely complex, but its not entirely intuitive from a computer standpoint. Suppose I were to list a few states inside the state machine:

mystate(crunchy: yes)
mystate(crunchy: no, cheesy: yes)
mystate(crunchy: no)
thystate()
thystate(chewy: perhaps)
thystate(cheesy: thursdays)

You know that there are states called mystate and thystate. Depending on the context in which this list appeared, it might be safe to assume that a state called frystate doesn't make sense. For any particular state machine, there are only a few possible valid names states can have.

That doesn't seem too rough, but as is so often the case, what isn't said is what's tricky. The list of states above would have the same meaning if you appended the following:

mystate(crunchy: maybe)
mystate(crunchy: level_1)
mystate(crunchy: level_2)
mystate(crunchy: level_3)
mystate(crunchy: yes, threebears: eleventeen, number: threeve)
...
mystate(complete_works_of_shakespeare: ...)

The long and short of it is if you see mystate(anything) you can assume that there exists a mystate with any possible set of parameters you can imagine. The corollary to this is that there are an infinite number of states in any state machine.


Literate again
2008-12-11T21:09:00.000-08:00
I just finished Dave Sedaris' "Naked," the first book I have read through for pleasure in about 2 years. I've spent most of that time joking about my illiteracy, but its termination has rendered it all the more shameful in retrospective.

A copy of "A Tale of Two Cities" has been sitting at just over half done for months. I picked up Sedaris' book 3 days ago and dispatched it swiftly. I don't know that I'm finding Dickens difficult, but the story just hasn't drawn me in. After having largely stuck to the 150-year rule throughout high school, it seems I've lost a taste for the dusty British literature of the canon (or maybe its just not his best). The last time I read I encountered this issue as well. I finished all of Just for Fun in a week after having spent the beginning of the month progressing slowly through Saul Bellow's relatively short "Seize the Day." It seems I've lost patience with most books.

"A Tale of Two Cities" is at least inhibited by footnotes. The historical facts are marked with little numbers and explained in the back of the book, and I check them with an almost OCD-like consistency. This repeated interruption may be making it difficult to follow or be engaged in the story, but I still get the sense that I wouldn't understand as much of what was going on without them. My brain has a tendency to simply discard bits it doesn't understand, only requesting clarification if the result doesn't have a meaning of its own. I can often read large spans without really comprehending, and not realize until several pages later that I've absorbed no nutritional value from the text. It works for conversation too, much to my mother's aggravation. I've been able to acknowledge and participate in entire conversations without any awareness of goings on outside of the TV. Eventually the 5th or 6th of a long line of questioning would be met with a sudden "...wait, what?"

At any rate, the book was wonderful. Sedaris' dry critique of everything, including (perhaps especially) himself, is hysterical. The book is laden with a thorough and delicious sense of the absurd. Its been out awhile, so I'm sure quite a few have read it, but if you haven't, pick it up.


This shit is delicious
2008-11-22T16:25:00.000-08:00


I hereby declare Apple Butter to be the official Fedora condiment.

Carry on.


Thoughts on package management part 2
2008-11-14T10:23:00.000-08:00
So when we last left our hero, we were identifying packages by 3 traits:

Name

Date/Time Released

Signing Organization


This has some interesting implications: first we are directly asserting an update path now. The tag reads "Organization X says you should have this package installed for Y as of time Z." This gives us direct control over what we give the user. Also, since signer is one of our identifiers, there is now no such thing as an unsigned package.

So what happens when Fedora provides foopkg, and HPMFusion provides foopkg with patented extensions built in? For this HPMFusion can use a familiar feature, the Obsoletes tag:

Obsoletes: foopkg[fedora]

Now when the user installs HPMFusion, it detects that HPMFusion provides foopkg-[hpmfusion] which obsoletes foopkg-[fedora], and installs that. Since Fedora doesn't have a similar obsoletes in their package, the package continues to be updated by HPMFusion only. No weird update races where Fedora clobbers the 3rd party repo, who then clobbers back with their updates, etc.

But what if we want to be able to use HPMFusion but not necessarily take the new foopkg? This is trivial to handle in the depsolver, we just need to specify the relationship differently. "Replaces" or "Interchangeable-with" will do. Then we treat this like Obsoletes when explicitly installing, and ignore it during updates and regular depsolves.

Individual packages work slightly differently under this paradigm. Since there's no such thing as an unsigned package, individual packages must be signed by their creator, who in effect becomes a "repository provider without a repository." It is the creator's responsibility, therefore, to ensure all of his packages are compatible. We might make this and some other use cases easier by combining the package signer field with some sort of "namespace" so that multiple package spaces could be signed with a single key.

That takes care of most of the inter-operation. Now do our methods of specifying dependencies between packages still work? foo depends on bar should behave the same. Foo depends on bar >= version works just as well with >= date. Foo depends on bar = version could pose an issue, since both version and release are swallowed up by the date. And how does ABI breakage get handled?

We'll take the last one first: Each package should have a Compatible-since field, indicating the date of the last time it had and ABI change. If a package requires foo, it requires a foo with a Compatible-since field equal to or earlier than its own release date. This doesn't let us get super granular with packages providing lots of libs, but if the overall result of all this is simpler deps then we can likely subpackage more.

For explicit version requirements, we can add a Last-Version-Change field. This works almost exactly like the Compatible-since field, but changes more frequently (at every version bump). Packages requiring a specific version of a package can simply request the version on a given date, and the package provides it if the Last-Version-Change field is equal to or older than that date. If a version change gets made and rolled back in a future update, we can specify this as well, through mechanisms that'd be more worth fleshing out if we were actually going to implement this.

So there's my thoughts. We could certainly save a lot of effort if we encoded package info as points on an update timeline rather than the time-independent mess of data they are now. Thoughts?


Thoughts on package management
2008-11-11T13:24:00.000-08:00
Recent discussions on fedora-devel-list have gotten me thinking about package managers again. As a problem its always intrigued me (I can't resist graph theory, so depsolving is one of my favourite things :) and as usual, there's a few things I'd do differently. So, in the spirit of open source, I'd like to conduct a little thought experiment on the design of a hypothetical package manager.

Let me preface this by saying that I am not writing, nor am I going to write, an actual package manager, and I have no interest in replacing yum. Yum is kind of a scary codebase, and lord knows its been scapegoat for enough over the years, but Seth takes good care of it, and not many other offerings have met our needs.

Meeting our needs, however, will be a priority of this thought exercise. I'd like to meet the needs of a multi-lib distro. Some of this is not about software though, and while there are some new ways of doing things I'd like to poke at, I'd also like to explore whether rethinking our process could let us simply do without a few features.

So lets get started.

For purposes of identification and dependency solving, Yum/RPM distinguish packages by the NEVRA tuple:

Name

Epoch

Version

Release

Architecture


Architecture is interesting, but for the moment, lets ignore it:

Name

Epoch

Version

Release


So what's wrong with this? Well, Version is an unmitigated bitch. Projects with strange numbering conventions just keep cropping up. Release is even worse, as developers seem to keep wanting to put strange information in them, and comparison there quickly becomes even more impossible. Furthermore, Version is upstream determined, and yet it determines our release order as well, which means if we as distro maintainers want to roll back to an older version, the depsolver says no. For this we have Epoch, an ugly, horrible hack.

Funny how the horrible hack is all I want to keep...

Name

Epoch

Version

Release


Ah, now here we have on-demand obsoletion. We don't ask the depsolver to determine what package is newer at all, we simply tell it: I am number X in succession. X > Y, obsolescence determined.

But this is no good.

Epoch tells us nothing about the package itself. The integer Epoch has no meaning. Now the developer is getting lost in a sea of random nonces. So lets make Epoch a bit more palatable:

Name

Epoch Date/Time Released

Version

Release


Ah, now we have something. Knowing when a package was considered current might well be even better than knowing the version (I should note, in case anyone assumed otherwise, that we keep Version and Release in the package metadata. The difference is that they are now purely informational.)

Except for those damn third party packages.

Package release dates aren't agreed upon between organizations, which means two third party repos could put out packages providing the same thing on the same day, and they don't necessarily provide the same "freshness." If only repos could just ignore each other and operate in a vacuum...

Oh, wait:

Name

Signer

Date/Time Released


Now we have it. Signer is part of the identity of the package, so foopkg released by FreshHPMs on Tuesday and foopkg released by Fedora on Tuesday are as different as foopkg and barpkg both released at the same time from the same place. Repositories can safely ignore each other. Life is good.

Except not quite.

But now I'm tired, so let's get the flame war started now and I'll come back with a part 2 to explain life in this strange new universe.


You blue folks run things now...
2008-11-05T10:52:00.000-08:00
You have 4 years to convince me.


Microsoft? In MY Open Source?
2008-10-30T08:11:00.000-07:00
I've been hearing a lot lately about Microsoft suddenly getting cozy with FOSS. Theres been a bit of planet buzz on MS joining the AMQP working group, and though a court order was involved, there's been some excitement over MS allowing the Samba people on site for testing. Most people seem thrilled that Microsoft is "finally realizing the benefits of open source." This attitude has me a bit worried. It almost seems as though we have forgotten who we're dealing with.

What really set me off was when Scott Williams posted about an interview with Microsoft today, and actually used the word "embrace."

Did that not ring a bell when you wrote it? There's a certain Microsoft strategy that has that word in it.

This is not a grudging admission by Microsoft, it is not a change in behaviour or a new acceptance of how we open source hippies have changed the world and they have to adapt. This is Microsoft doing exactly what we knew they would. This is how Microsoft kills things.

Peter Lemenkov seems to have picked up on it. They want in so they can destroy us from the inside.

You're going to see a lot of these people around. They'll be everywhere. They might even be submitting kernel patches soon ("Introduce 235 patent violations. Signed-off-by: Steve Ballmer"). So be polite and smile when you see Microsoft in the hallways, and when they pass, double back and follow them wherever they go. Show them around, let them reap all the benefits of FOSS participation, but don't let them out of your sight, not even for a moment, and don't turn your back.


A computing challenge
2008-10-23T14:57:00.000-07:00
A friend and I were joking about algorithms with enormous runtimes, and I thought of a little puzzle.

The task is to develop an algorithm with O(n↑↑n) complexity. Now, obviously one could just nest a bunch of loops and do this easily, but I'd like to see something interesting and unique.

Anyone got any evil ideas?


Women in Open Source
2008-10-22T11:04:00.000-07:00
Aaron Hawley has been making a series of posts on planet about the problematically low female presence in open source and some equally problematic remedies thereto.

This is a problem I've noticed for some time. I first started getting worried when I looked at the Women in Fedora wiki page (at the time, there were three). There are a lot of reasons for the lack of women in CS in general, and I think all of them are fixable.

The first thing I'd like to point out, and nobody in this discussion has brought this up so far, is the often disturbing attitudes toward women that are frequently present among male open source contributors. I don't know why, but the top 5 or 6 most misogynist conversations I have been privy to have been at open source events or in venues for discussions about open source. I have the most brazenly offensive sense of humour one can get, but I didn't even begin to sense an attempt at humor in any of these cases. Usually these conversations are kept to circles where it is believed no one will cast a negative light on what's being said, but there are some very messy public exceptions (third paragraph). This I suspect is the product of some nerd culture feedback loop: the continuing lack of female presence causes the culture to become inhospitable javascript:void(0)to female presence. At any rate, its something we would to well to improve. Women aren't going to participate in our community as meat. If nerds tend to be awkward and make bumbling advances perhaps we can work around it, but a genuinely predatory perspective on women cannot allow the growth of a female open source community.*

Secondly, a few have now pointed out that it seems women are discouraged by either perceiving their skills with computers to be far less than their male contemporaries, or due to being fed up with their male contemporaries' inflated views of their own abilities. This is a serious issue in the technical field. We must, as a community, instil in our young female computer scientists and technology enthusiasts a greater sense of arrogance.

Lack of a firm ego and a surplus of stubbornness is not healthy for a technical career. Picture this scenario: a suit walks up to his well-adjusted sysadmin, smiles, says a bunch of words that don't mean anything and are probably made up, says "we're counting on you" and pours on a dozen thank yous, and before our sysadmin is aware of what has happened, he or she has agreed to do something stupid, like installing Oracle products.

It seems like a harsh judgement that all great coders and sysadmins must be assholes, but look at it this way: to be great at these things, technical issues must be marginally more important to you than the feelings of others. A lot of the lack of female participation in open source might be attributed to a lack of females who are broken in this particular manner.

On a final note, I wanted to bring up the Ally McBeal effect. Before Ally McBeal aired women were by far a minority in the legal profession. In recent years, the legal profession is one of the more gender-balanced fields. It could be another case of correlation not being the same thing as causality, but that hasn't stopped some from trying. It makes sense that an issue of role models could be involved in female participation in open source, but that raises an interesting question: where are the male role models? Not much attention has been paid in this whole argument to the question "what gets anyone into this stuff, and why doesn't that work for women?" In my case my love affair with computers began near the age I started walking, and my open source enthusiasm was kicked off just by installing Fedora and trying it out during my freshman year (the first moment that I had a computer my parents didn't own). I'd like to hear others on this subject, though: what got you into open source, and why wouldn't that work for women?

---

*Of course there are plenty of men in open source who don't have this attitude. Particularly those that are married, it seems.


Yet more angry comments and questions
2008-10-18T13:29:00.000-07:00
Well this has been exciting, hasn't it?

I'm going to ask now that people who intend to post long comments, ask questions, or otherwise participate in a discussion like manner please post on their own blogs and link to the post in the comments here. Its easier that way.

So in the comments, Choeger says:
You do not have a clue about BIG projects and Software Design, do you? The point with java is to force every programmer to do things in a very restricted way. That leads to nearly identical code from different developers and _that_ leads to supreme IDE support, auto and Design Patterns which _are_ worthfull to Software Engenieers. If you have a dynamic language like python, you can do wonderfull stuff but you loose all those big scale tools like refactoring and automagic code generations.


If you need to generate code automatically your language is inexpressive. If you can't refactor in 15 minutes with sed, awk, grep, and a little vim, its badly structured.

Every design pattern exists at the expense of OO principles. The patterns break the underlying paradigm. Ergo they represent hacks.

Most BIG projects are bad ideas. This lesson is so important that the federal government spends 100% of its budget every year trying to teach us this lesson by example. That being said, the Linux kernel and FreeBSD seem to do big projects just fine without java or any IDEs.

I think that takes care of the points you bring up. Now as to your tone, you're kind of a self-important douchebag, and I respect that. I don't know about BIG projects I'm sure. I'll bet your projects are like WAY important, and only java is important enough a language for them. I'll bet Sun Microsystems representatives come over to your house and inject white hot significance into your rectum every time you deploy a warfile in your important application because of how BIG and important you are. Maybe I'll grow up and do BIG projects like you one day. Now please leave my blog.

gotpsp asks:
How do C or C++ fit into your bashing of the scripting languages?

They don't. C is appropriate for a completely different set of tasks than Ruby or Python. Java isn't good for anything, but its competing primarily in the same space as the latter two. C++ I dislike, but lets not bring another one into this.

smooge writes:
Looking for constructive conversation on a blog called "Screw You Enterprise Edition" is like looking for uncorrupt politicians... you end up being dis-appointed.

You're on the internet. What'd you expect?

Finally, richi writes, with regard to needing 3 objects to get input:
Actually, I kind of like the fact that Java does not assume anything of the environment that the program is run in.


You've kind of missed the point. Programming languages are not here to let you do things. They are here to let you model concepts. This applies to any paradigm, from something like make ("Foo.o arises from foo.c and foo.h, via this process: ..."), to functional languages ("a is the b of c. x is the y of z of q"), to object oriented languages ("A foo has 4 bars and sometimes a baz. You can frob it by splaying it and then frashing it and munging it. You can mung it by rotating the first bar 20 degrees"). Input is well defined in the OO paradigm. "From the source, get the data". Java doesn't let you think that way. If you try this exercise of translating code to English, you either end up fudging it, say things that make no sense, or, and this is the big one, you end up with a far, far more complicated sentence to describe the same procedure.


Some more language bashing, by request.
2008-10-17T22:33:00.000-07:00
My last post has generated some comments worth responding to, so I'm bringing a couple to the front page for everyone's benefit.

First up is Colin Walters:

Languages are good at different things


Yes, and some aren't good at anything.

I think Java is a near-perfect language for core libraries. For example, the JDK has some *very* nice classes like MappedByteBuffer and most of java.util.concurrent is good too.


Well goodie. You have features.

I really don't care about what is in the Java API. I care about the Javian way in which much of it is exposed. You have to instantiate 3 objects to get command line input. Can you describe the event of another person telling something to you that involves 3 objects besides you, the other person, and the sentence? Can you do it without sounding stupid?

Side note: a little tip in OO design: Never end a class name in -er. That ending implies that it is a class which exists because it "does something." There's already a class that does something. Its called a "Method." If you can't get away with this your language isn't flexible enough.

Ruby's core libraries from what I hear aren't that great - it definitely looks like you need a random 3rd party library to mmap things.

And? Its called open source. Not everything comes from the same place. If they're scattered a bit its easier to rip them out when they're crap.

Ruby's core libraries expose features elegantly. Ruby's Array class is more powerful than the entire suite of Java collections, and it can be learned much more quickly.

Java (or another even more strongly typed language like Scala) makes a lot of sense for embedded devices and the like where you want both speed (Ruby isn't good at this) and memory safety.

No its not. Objective C is the only language of late that I've seen in this problem space that looks promising (maybe D, if it would hurry up and go somewhere). Java occupies this problem space, Ruby does not. But that doesn't mean Java is a good idea here. Stick to C for this stuff. There's not much to C, but very little of it is flat out wrong.

But speaking of speed - did you know the JVM is currently the best Ruby platform?

Says who? Other than stack depth Ruby 1.9.0 seems to be squarely handing its ass to it.

Up next is "richi," who I don't know (or whose screenname I don't recognize), but who seems to be interested in my opinion, which is certainly a path to enlightenment.

I'd actually like to hear some more of your opinion on the three languages (Python, Ruby and Java). I wasn't sure exactly what to make of your blog entry. Are you saying that the Ruby language makes it hard for coders to write bad Ruby code?


Both Ruby and Python make it harder to write bad code than Java does (which is to say, neither explicitly mandate it), but I don't actually know why Python has such a rash of bad code. Steve Milner said to me after I posted that Python's recent sharp spike in interest has led to a high volume of users who don't know what they're doing and are coming from very un-pythonic programming backgrounds. I think this is probably a good guess.

As for my opinions on the languages, Java I'll get to below. Ruby I enjoy because its one of the few languages that's gotten OO correct. Python is nice in that its lighter, but when it gets OO wrong, it gets it horribly wrong (yes, the "len()" method is a design disaster. No I will not justify that. You are wrong, Guido. Fix your shit). Python also makes it very hard to stream calls. You can't say foo.bar().baz().bam(3,2).bang().bao(5) the way you can in Ruby. You keep ending up with destructive methods.

In Ruby, everything is in effect a getter or setter. "Get me your foo variable." "Get me your bar variable." "Get me the sorted version of yourself." "Get me what f(x) maps you onto." Even Python can take 5 lines to do what 1 line of Ruby does. Ruby just relates ideas better. It is a better language in the communicative sense.

Also, do you have a problem with Pattern per se? Or just the patterns outlined in the book you linked to?


The term pattern as it is used today was invented by said book, and when I use the term design patterns, I consider its definition to be exhausted by that book's contents (which are as ample as they are vile).

As a programmer, I happen to find myself coding a lot faster with Java these days primarily because of the development environment that I'm aware of exist for each language. Eclipse with Javadoc makes the ramp-up time for any new API quite clement. I've my share of complaints about idiosyncracies and limiting factors with Java, but I'll take profuse document coupled with code-completion over good language constructs any day.


I find myself incapable of coding with Java. Understand the APIs though I may, I can't get over the sense, as I write, that the language is actively trying to waste my time. "Foo doesn't have a frob method. You need a FooFrobber. You can't just get one though. You need a FooFrobberFactory, which you have to instantiate with your date of birth, the IP of your computer, and the SHA-1 of the file in which you are instantiating it, as well as 3 Bar objects wrapping BufferedStreamReaders which are attached to files containing the complete text of the Prince of Persia 2 game manual. Then you can go resolve class loader issues. Oh, and make sure you catch that exception that should probably be handled somewhere in code you didn't write, but they weren't expecting it so now you have to deal with it." Yes, I was able to find out how to frob a Foo in a hurry, but I no longer have any desire to frob a Foo. Rather, I now desire to consume James Gosling's flesh.


Proofread you moron!
2008-10-17T13:23:00.000-07:00
I'm going to get it right, I promise.
1) Write post.
2) Read post.
3) Submit post.

I seem to keep getting 2 and 3 out of order. In fact the reason Blogger's edit quirks bother me is they expose this dirty habit of mine :)


Why aren't Python coders any good at Python?
2008-10-17T12:17:00.001-07:00
If you'd like to know what triggered this rant, go to any fedora box which has the packager tools installed and type vim `which koji`.

I've had my differences with Python. I learned Ruby first, and honestly between those two, usually the first one you pick up wins out. Honestly though? I think its a good little language. I'm happy there's more Python code and less Java code out there. I like that it comes so close to Ruby in so many ways and yet is far simpler internally. Its a good time to be a programmer.

But lately something has been bugging me, and I don't think I'm the first to notice it. Now, many languages, due to their design, encourage particular coding styles and practices that cause certain things to become characteristic of them which aren't necessarily defined by the language itself. For example, the various Design Patterns (*spit*) show up far more prevalently in Java than elsewhere. Ruby's closures get much more work than Python's lambda. Python gets used more often for complex web service architectures, where Ruby tends to be used for highly user-interactive webapps (and Java gets used for pointless bullshit). Its all good.

But I've noticed a pattern coming out of Python that kind of worries me, and it worries me most because there's really no reason it should be there. It seems that 4 out of 5 python programs I look at are crap. Functions span for hundreds of thousands of lines. Classes play strange roles, and are often incohesive. Architectures are overcomplicated. Its a mess.

The inexplicability of the effect is more bizarre than the effect itself. Ruby's closures get used more because they're easier to work with and because the core libraries set a precedent for it. Python and Ruby get used for different type of webapps because of the respective strengths and weaknesses of Django and Rails. Java code is the way it is because only morons use Java. It all makes sense. But no matter how I review Python, and despite what little issues I have with it, I can see no reason that it would attract poor programming.

To be clear, I see good Python too. The people who have told me they read this blog regularly tend to be the authors of a lot of it. I think Python deserves to be treated right, and it really upsets me that so few seem to be coding Python with the care it deserves.


Opening ACLs
2008-10-15T12:19:00.000-07:00
In the past couple of weeks Fedora changed the system by which commit access to CVS is given.

Previously, one needed to get a review request submitted and get sponsored, and then one got access to cvsextras, and then you got access to all the packages. Everything under the sun.

This tended to scare some people.

As a result, some package maintainers locked their ACLs down so that only a few people they designated could commit to their package. This is un-open source.

So, in the new system, getting sponsored gives you access to the packager group. Once there, you can access your own packages, but no one elses. Later, you can request access to the überpackager group, and if you ask very, very nicely, you will get access to the full run of packages.

So now that that's all set up, its really not necessary for all those tight ACLs to be in place, so we're about to open them all up. That way access to the entire distro is available to anyone who convince at least one other person that they aren't an idiot.

If you don't want your ACLs changed, go into pkgdb, log in, and go to your package. There will be a "Include package in mass ACL opening?" checkbox, which you may uncheck.


Problems I have with Compiz
2008-10-12T15:42:00.001-07:00
1) Firefox render speed is dramatically slower when compiz is running. I don't really understand why an actual app would slow down, but I don't know X that well.

2) The gnome panel that I have moved to the right side of the screen and auto-hidden doesn't always want to come out.

3) The workspace switcher panel applet doesn't work right. I have 4 workspaces stacked in 2 rows of 2, and the panel applet shows all 4 of them in the top half of its reserved space, with the bottom half just being void.

4) When I move from the workspace that holds my fullscreen terminal to another workspace, focus a window, then move back, the gnome panels are on top of the terminal until I refocus it.

Its hard for me to file bugs with applications I don't use, and it hurts especially when I know I'd use the application if the bugs were fixed. I'm going to endeavour to get these reported though.


Who are we and what are we doing here?
2008-10-11T23:14:00.000-07:00
Wikipedia has gone to Ubuntu. The internet is displeased.

Granted that's the internet in my bookmark list, but an internet nonetheless. There is much uproar in the community about fixing X or Y to stop people from running away. Its not the first time Ubuntu has gotten Fedora ruffled up. Inevitably the argument ends with a voice of reason shouting "eh, we don't want them anyway."

Usually "them" is a population of Joe User types, and the reason we are told we don't want them is that they are just ordinary users, and Fedora is for developers and contributors who care about making open source better. It applies here too, of course: Wikipedia is growing into a business that can't be run on our little developer's paradise.

We can take comfort in this thought for awhile, but it occurs to me that this just points out a bit of the absurdity of what we do here. Why would anyone make an operating system for people who care about operating systems? Those are few and far between. Most people don't even know what an operating system is.

Apple in particular wants to keep it that way, which is why after all these years they are still a hardware company. They want users to think about their computer having a dock menu in the same way they think about it having a keyboard; its inherent to the device. There is no separation of the concept of the OS for the ideal mac user. Scissors have handles, cars have steering wheels, Macs have garage band. All is well with the world. Its a hard image to keep up when the other OS vendors aren't doing the same thing, and it puts them in the rather bizarre position of competing against the "PC," a loose-fitting body of standards backed by no one company, and linked in the user's mind only by Windows, a disembodied OS.

Mac OS is probably one of the most praised OSes of all time, in contrast to Windows which everyone uses and everyone hates, and it gets there, in essence, by discouraging thought. Open source in general and Fedora in particular are the opposite. We demand our users invest themselves in the idea of an operating system. Apple never demanded a user base of zealous fans, they just set out to make some people's lives easier. The one weakness of FOSS is that zealot-hood is a door price. We know what we are doing is great, and we tell everyone that its really great, and we expect that one day they will all recognize how great it is and come elevate us to our rightful place as OS king of the world. The reality is that people at large will never accept Linux as an operating system.

My mailbox is full now.

Everyone wants the Linux revolution, and it can happen. People just won't let it happen. The solution is not to ask them. They didn't accept Windows either. They didn't even know they had it until it crashed, and then it was too late. The objective is to develop an operating system that people won't pay much attention to. The nephew comes over and says "ok on your new computer, auntie, the email is over here, and the internet is over there" and that's all the thought that's ever given to it. This involves doing things that open source has been kind of bad at, like fixing problems before someone complains, and doing what the user wants rather than what's "best."

This is also, incidentally, why Codeina is a bad idea. Let the movie player just break. The nephew comes over, breaks whatever federal laws he must in ways that leave no culpability on yourself or yours truly, and an hour later the experience is forgotten. But Fedora isn't that type of OS. "My videos worked yesterday, and now they don't, and this asshole is talking about how its good for me? Fuck this god-damn hippie computer thing!" Now the nephew has learned some very grown-up words and our attempt at education has taught auntie that open source lies somewhere between her and her precious youtubes! The point being, you cannot be the user's parent, and nobody else is the user's parent. The child is feeding himself. If you want your food to get eaten, your food has to be a cookie. No exceptions. Especially not dogfood.

What does this have to do with Wikipedia? Have we drifted off topic into the tired old desktop debate? Stay tuned.

So lets get this down to a personal level. Its on you to do this. Its your job to bring the glorious Linux desktop to Joe User. Will you be a sycophant to do this? Will you pander endlessly to the desires of the masses? Remember, they don't know or care about OSes at all. Who are they to judge one?! Why do they get to come in here and tell you your app is crap and it shoud be more like Fooapp?! I can tell you that Ubuntu is feeling these pains. For all its screaming about its wonderful community, the Ubuntu userbase is overrun with people who have no idea what they're talking about. They believe using Linux makes them some kind of "l33t h4x0r," to which end they sit at the coffee shop glaring just over the tops of their laptop screens, and pointing and making "pew pew" noises at other people's computers. Do we need more of these people in #fedora? Do we need them running things? To capture a userbase, you have to put them in charge. Fedora won't succeed if it keeps letting smart people run things, for the same reason you can't sell a Rubik's cube to a chimpanzee.

Wikipedia is an organization and presumably has technical people in charge of these things, but there's a fundamental similarity: Its a need we have to cater to. The people who do Fedora do it for themselves, and that means that's all we're going to get as a user base. We'll get a few others now and again, such as my personal friends, who use Fedora so as to qualify for my own free tech support program, but we have to be honest. Look to your left, look to your right. This could be it. I'm ok with that. These are some of the finest developers on the planet. I want to use what they're using. Hell, I want to help build it. Doing a LTS distro would mean being less badass today. I don't want that. Getting Joe User would mean fiddling with GStreamer instead of Upstart. I don't want that.

I'm making an OS for me, and I'm using it. Red Hat makes one for money, but they make it for a breed of high dollar customers with demands that make absolutely no sense to the common user. We've got a good thing going here. No customers to tell us what to do. If Red Hat decides there's an opportunity for a new product then I'll be happy to help see it off, but if someone wants to demand Fedora be something I don't personally want, I may recommend them this blog's titular product. Why would anyone make an operating system for people who care about operating systems? Because they can't stand making it for anyone else. If anyone else liked Fedora more, you'd like it less. Is that what we want?


WTF?!
2008-10-07T20:13:00.000-07:00
!

By "we" they do mean the makers of the app. Facebook now has an app that sexually harrasses you. WTF?!


Strange Blogger/RSS/Planet behavior
2008-10-01T10:11:00.000-07:00
I just noticed that editing a post on my blog causes it to reappear on Fedora planet again as though it were a new post. I can't help but wonder if that behavior is correct. *Sigh* further issues with the already rickety planet concept I guess.


LPC
2008-09-23T14:56:00.000-07:00
I'm late as ever to the blogosphere, but I'm determined to say something about LPC.

Arjan's talk set the pace for much of the conference. He generated some buzz when he threatened to boot Fedora in 5 seconds at the talk, and while everyone was hoping to be impressed, cynicism set in fast as most assumed what he would present would be missing most of the OS or involve some "trick" that wasn't implementable. Having seen what he had to say, I was pleasantly surprised. Its not a drop-in solution; there are some gaps to fill in and some questions to answer (and arguably some important bits of the OS were in fact missing), but most of what was done was addressing genuine issues in the boot process of the average modern distro.

Arjan's new boot process used sysvinit. I asked him afterward what we could give him in upstart that would make his job easier, and he asked for a serial execution path. This is achievable in current upstart by simply defining a start on starting job that contains a script that hits all the points you want. Hell, we emulate sysv, there must be some way of being serial.

Notwithstanding, Upstart received a long-due redesign during the conference. One of the most noticeable design changes we talked about was the split of upstart into 2 parts. The first is our pid 1, which no longer does any dependencies or state tracking. You ask it to start something, it starts it, it manages it. If something fails it tells you. That's it. There's a lot of good reason for this. For one, pid 1 gets simpler, which is good since pid 1 is kind of an awkward place for a segfault. It also means that pid 1's sole job now is service management, which we know how to do and are more or less doing correctly in current Upstart.

The second bit is for tracking system state, the part we did not previously know how to do and are doing quite wrongly in current Upstart. Scott brought along his aggregation of the freenode #upstart discussions on this subject, and we hashed out a userspace companion to our stripped down pid 1 that would simply receive events from the system, track a series of states based on that, and launch services. It maintains a series of states which are either active or inactive, which can require one or more other states to be active, and which can be triggered by an event, the user, or an attempt to raise a state that depends on them. They interface with the services themselves by starting a service when they become active.

With the design done we now have two tasks. Scott will begin ripping things out of upstart's pid 1 to make it be an on-command service manager, and I will be whipping out ruby and rapid-prototyping the state system. When the dust clears we will have something that's simpler to operate and handles our use cases elegantly.

In the process of our redesign, I got to meet Scott. It was a strange experience. I'd always kind of thought of him as "the adult" in the project. Finding him to be only a couple years older than me was... trippy. It was fun seeing him though, and all the other Red Hatters I'd been out of touch with. The drinking was copious, so the trip was worthwhile :)


Attn. Java Fans
2008-09-15T19:25:00.001-07:00
Defend this:


         KeySpec ks = new DESKeySpec(theKey);
         SecretKeyFactory kf 
            = SecretKeyFactory.getInstance("DES");
         SecretKey ky = kf.generateSecret(ks);



The State of Upstart
2008-08-15T08:48:00.000-07:00
The distance between my posts tends to give this blog a bit of an incongruous continuity. Much has happened since my last set of Upstart bloggings. Here's the rundown:

First: Upstart 0.5.0 has been released. It took awhile, but we finally have supportable code to show the community.

And now we redesign it.

The event model, you see, has begun at long last to show weakness. Things like mounting filesystems or even depending on two services at once have always proven difficult for it, and while various workarounds have appeared, it looks more like the model is wrong.

So at the upcoming LPC we will be working on a new operating principle that handles all of our use cases. I'm confident in what's going to come out, and look forward to having some new code to write.


Uncollidable Upstart Environment
2008-08-04T08:24:00.000-07:00
There's a feature I left off my wishlist, mostly because I thought the reasoning behind it was too much for a blurb in a post.

Scott recently pointed to a problem he felt Upstart was ill-equipped to solve: mounting volumes. There are numerous interesting problems here. The first is getting information from fstab. One way to do this is to turn fstab lines into events. On startup, each fstab line causes fstab-line MOUNTPOINT=... {BLKDEV|LABEL|UUID}=.... Whenever a block device becomes available, we get: new-blkdev BLKDEV=... LABEL=... UUID=.... Then in an instance job we have start on fstab-line ... and new-blkdev ....

The issue that then arises is how do we determine that fstab-line and new-blkdev are paired correctly? We want to compare environment between events triggering the same action, which is already something Upstart won't support, and we also want to match when ANY of a set of environment variables matches; a kind of matching that doesn't appear anywhere else in upstart.

The solution, like so many things in upstart, doesn't seem like it would be related to the problem. Enter the immutable stanza. When a set of environment variables are set immutable, they can't have their values changed once set, and a job won't react to events that would cause this. For example:

start on foo and bar
immutable A

If we emit foo, bar A=1 or foo A=3, bar the job starts, but if we emit foo A=3, bar A=1 we end up doing nothing. Now, here's an interesting case: foo A=1, bar A=1. What do we do here? Well we said we only care if the value changes, so I guess we'd better start the job.

Going back to our mounting problem:

start on fstab-line and new-blkdev
instance $MOUNTPOINT
immutable BLKDEV LABEL UUID

If we get an fstab-line and new-blkdev event, we start a job ONLY if the two do not disagree on the values for BLKDEV, LABEL, or UUID. fstab-line will only define one of these, so it will happily get out of the way for the other two. The third will be matched for identity, and all's right with the world :)


Upstart wishlist
2008-08-01T12:22:00.000-07:00
There's still quite a few features I would like to see in Upstart, all of which I intend to start implementing as soon as 0.5 is released. Here's a list:

Global Environment

As of current 0.5 head, Upstart has a global upstart.conf file. Also as of current 0.5 head, there are no valid settings which can be set from this file. That's right, it can be blank, it can have white space, it can have comments. The file doesn't have to exist for upstart to work, so its not a big deal, but it seems like there's something we could do with it.

Global environment is one purposing I have for this. The idea is to be able to inject environment variables globally into all jobs. The standard use case for this would be default runlevel, which is better as a system-level setting. There are numerous other examples. Basically anywhere it would be a good idea to make static information available to all upstart jobs, this is relevant.

Explicit Positioning for Environment

If you set a job to start on foo BAR=1 BAZ=2, that job will start on initctl emit foo BAR=1 BAZ=2, initctl emit foo BAZ=2 BAR=1, but not initctl emit foo BAZ=1 BAR=2. Upstart also supports implicit environment matching, so you could specify start on foo 1 2, and you would start on initctl emit foo BAR=1 BAZ=2, initctl emit foo BAZ=1 BAR=2, initctl emit foo FROB=1 BAR=2, but not  initctl emit foo BAZ=2 BAR=1. This is horrible and stupid.

The solution here is another use for our global config file. Instead of relying on the order of passage to initctl, we can simply supply upstart with a default ordering of environment variables for a given event. So with a stanza like takes foo BAR BAZ in upstart.conf, the two start on formats would become equivalent. Furthermore, initctl could parse the same config file, allowing you to initctl emit foo 1 2. Convenient.

True Non-Determinism

If you wanted to run one instance of a job for every tty on your computer, whenever the interface was up, you might try this:

start on new-tty and interface-up
instance $TTY

It seems like it should work, but what happens when you get this sequence of events?

new-tty TTY=tty3

new-tty TTY=tty4

interface-up


When upstart sees the first new-tty event it makes note that one half of the and has been satisfied. The second, however, is ignored, so we only get the job running for one TTY. The solution is non-deterministic matching, and while there's a bit of fiddling involved in that, the upshot is you get the behaviour you expect when these things occur.

Groups

Groups are one possibility for runlevel-like functionality in upstart. The idea is basically this: At any given time, there are one or more "groups" active in upstart, as well as one or more groups suppressed. and each job belongs to one or more groups. A job always belongs to a group which is its name, and if no other groups have been specified for it, it belongs to the default group. The user can also specify implications between groups, so if a job is a member of group A, it will, by definition, be a member of group B.

In order for a job to be running, one of the groups of which it is a member must be active, and either none of the groups it is in may be suppressed, or the group for which shares its name must be active. The default group is always active at startup, all other groups are always inactive at startup. If a job tries to run while it has no active groups or one of its groups is suppressed, it will be put into a special waiting state, and as soon as one of its groups becomes active or the suppressed group is unsuppressed, it will start (unless something tells it to stop before this happens). If the group goes away again, the job will return to that waiting state.

This is a very flexible system, lying somewhere in between the previous profiles suggestion and the use of simple flags.

Controlled Starting

The groups idea allows us to be a bit more sane about the start and stop commands. Right now, you can just start a job at any time, even if the dependencies which its states were designed to monitor aren't there.

Taking the groups idea, consider this semantic instead:

If a job is blocked due to a group issue and the user runs start, its namesake group is activated

If a job has not in any way been signaled to run and the user runs start, the user is asked to try again with --force

If a user runs stop the job's namesake group is put into the suppressed state


This both allows upstart to remain in control of dependency solving, lets the user take control if desired, and prevents upstart from undoing a user's explicit act in a job without ceasing to track dependency information for the job.

States

Its a much-belabored idea for people who have been following upstart, but since those people are few, I'll discuss it here:

The idea behind states is to allow someone to say "Be running when X is true" rather than "Start on X, stop on Y." The former allows you to express some things that the latter just doesn't, like keeping a job running when 2 other jobs are both running.


Upstart 0.5.0/trunk job definitions: The missing manual!
2008-07-28T12:29:00.000-07:00
Upstart is becoming notorious for its poor documentation. This is going to become a real pain as new features show up in the imminent 0.5.0 release, not to mention behaviour changes in already familiar mechanisms.

For those who are unfamiliar: Upstart keeps job definitions in /etc/init/jobs.d (/etc/event.d for 0.3.9 and earlier). Each job definition specifies 1) what should cause a job to start. 2) What should cause it to stop again. 3) What kind of environment the job should run under. 4) What to do before and after it runs. 5) How to run it (path to binary/arguments/etc)

Each job definition file contains one or more single line "stanzas" that specify properties of the job. Keep in mind these are subject to change, and some may not be implemented or available on your setup.

start on and stop on

Beginning with Upstart 0.5.0 each of these stanzas may appear only once in the job definition. Be careful! If you use more than one, only the last will apply, and you will become very confused in a hurry.

They work like this:

# /etc/event.d/foojob

start on started JOB=some_other_job
stop on hal.unplugged_foo

The first argument is the name of an event. It can be starting/started/stopping/stopped to have this job triggered by a change in another job, or some other name for an event you'd like to have emitted elsewhere (In the above example, it looks like we have hal set up to emit a hal.unplugged_foo event when our foo gets unplugged).

Subsequent arguments are environment specifications. Each event has an environment, which is a series of KEY=value bits of metadata giving more info about what happened, such as what job was started, to use our example.

An important note: environment variables have an ordering, which is determined by how they are emitted. So initctl emit thing FOO=foo BAR=bar would cause a different event than initctl emit thing BAR=bar FOO=foo. Where does this matter? Well, if you are explicit with your environment names, it doesn't matter. start on thing FOO=foo BAR=bar will gladly match either of those events. BUT Upstart also supports implicit keynames for environment variables, where matching is by order, not name. So start on thing foo bar is a valid start on statement. However only one of those two initctl statements will trigger it. Upstream and myself aren't totally in agreement as to how good an idea this is, but we are near a compromise that shouldn't suck as much.

New in this release (and making up for only being able to have one of these stanzas) is logical operators for these job definitions:

start on foo and bar
stop on baz and bang or bao

They do what you would expect, I'll leave the rest to reader imagination.

exec and script

These are fairly straightforward. This is how you specify what binary you want to run for this job. If you've got an easy-to-manage service, then this will probably do it for you:

exec /bin/foo-daemon

If there's a bit more to getting your job running, you can use script instead:

script
  export FROB=/var/lib/frob
  touch /tmp/magicfile
  exec /sbin/special-foo-daemon
end script

exec can be passed arguments and environment variables, but you might not want to get too fancy with it. script uses /bin/sh and comes with whatever that implies on your distro (bash for most, POSIX at minimum unless you reeeely fail).

Also, remember those start on and stop on environment variables? They show up in the environment for your script:

start on foo FOO=horse

exec /bin/echo $FOO     # Will print "horse"

This applies to all environment variables for the given event, not just the ones that you match on in the start on, so you don't need a FOO=something in your start on to expect a $FOO in your script or exec.

{pre,post}-{start,stop}

In addition to specifying what to run as your job, you can specify things to run before and after your job is started or stopped:

pre-start script
  touch /var/tmp/foofile
end script

post-start exec /bin/cleanup-start

exec /bin/foodaemon

pre-stop exec /bin/cleanup-stop

post-stop script
  rm /var/tmp/foofile
end script

Its all pretty self-evident. After the pre-start qualifier you use an exec or script with the same rules and syntax as above.

respawn and task

By default, Upstart treats your job as a "service," which is to say it assumes exactly one copy of your job is supposed to run until stopped explicitly (this is a change in behaviour from 0.3.9, so listen closely). If you want to have a job that is run once, does what it does, then stops, i.e. a job that isn't intended to be persistent, you supply the task stanza.

start on something
task

exec /bin/run-cleanup

No arguments, nothing fancy. Just task on a line by itself.

When you aren't using a task job, Upstart treats job exiting as a failure. It will NOT, however, restart your job for you, unless you ask it to:

start on something
respawn

exec /bin/some-daemon

If you'd like to limit your respawning, you can add another stanza, respawn limit

start on something
respawn
respawn limit 10 30

exec /bin/some-daemon

As shown above, the daemon would respawn at most 10 times per 30 seconds. Also note that you do need both a respawn and a respawn limit at time of writing. This probably shouldn't be the case, and may not be soon.

instance

Most upstart jobs assume that the job should never be running twice. However, sometimes (gettys for example) you want more than one of the same job running. We accomplish this with instance.

start on need-a-tty

session leader # more on this later, but any tty job needs it.

instance $TTY

exec /sbin/mingetty $TTY

We assume here that need-a-tty was emitted with a TTY=ttyX environment variable, and indeed, the job will fail if it is not. The argument to instance is the instance name. It is a string which must include shell expansions and which uniquely identifies the job. Two instances of a job must have distinct names once expanded, so in the above job, you could not start two gettys on tty6, as they would both have the same name.

umask, nice, oom, session leader, chroot, and chdir

Upstart comes with various ways to limit the starting environment of the daemon. They are all pretty straightforward.

start on something
umask 0644       # File creation mask
nice  -2         # Process nice level
oom   -3         # OOM Killer adjustment
chroot /var/tmp  # chroot
chdir kittens    # run with kittens

session leader   # Allow process to be the session leader

exec /bin/fooproc

The session leader stanza is an interesting one. Upstart normally prevents any of the processes it runs from being session leader. This stanza will subvert that, if your process requests ttys or does something else only the session leader can do.

description, author, and version

Documentation is good! Upstart jobs are no exception. In addition to # comments, these special fields are available.

description "Foo frobnicator service"
author "Casey Dahlin "
version 1.0

exec /bin/foofrob

Note the quotes. They are necessary if a space appears in your author/version fields.

env

env gives you a quick way to toss environment variables into your scripts. Its a good way to put a bunch of configurable options at the top of a job definition.

env JOB_ARGS="-d -m -W"
env GOODBYE="adios"

start on something

script
  exec /bin/somefoo $JOB_ARGS
end script

post-stop script
  echo $GOODBYE
end script


export and emits

When a job starts or stops, some events are automatically emitted: starting, started, stopping, and stopped. These events come with some standard environment, but you can add to it out of the environment your job has received if you like.

start on foo BAR=baz
export BAR     # Our started event will have a BAR=baz parameter

exec /bin/something

Also, it is possible to emit other events by calling initctl emit inside a script. If you do this, however, you should mention so with the emits stanza. This is primarily a documentation feature.

start on foo

emits baz

pre-start script
  initctl emit baz BOO=bloo
end script

exec /sbin/daemonthing


kill timeout and normal exit

kill Tells Upstart how long to wait for your process to die when trying to shut it down.

start on foo
kill timeout 20     # Seconds

exec /bin/thinger

normal exit tells upstart how to react to the death of your process, specifically what qualifies as normal (as opposed to abnormal) termination.

start on foo
normal exit HUP 4 6     # Exit by HUP or with status 4 or 6 is ok

exec /bin/whywhywhy


console

console specifies what to do with your job's standard output/error. The options are to push it to /dev/console (console output), to allow it to become the owner of /dev/console (console owner) or to chuck it away (console none).

start on foo
console output

exec /bin/echo "hi mom!"


expect

expect tells upstart how to know when your process is running. By default it starts your process and assumes it is up and ready right away. expect offers a few more options: expect fork indicates your process will fork once when it is ready, expect daemon indicates it will try to prevent becoming the session leader by forking twice (as man 3 daemon would do). expect stop indicates the process will SIGSTOP itself when ready, and upstart should SIGCONT it to acknowledge. expect none indicates default behaviour.

expect is the location of the much loathed ptrace hack (soon to be replaced by a better solution), and is currently the only way to follow forking daemons that don't have an option not to detach. If you don't use this and your process forks and quits, upstart will detect this as death.

start on foo
expect daemon     # process will try to daemonize

exec /bin/processthing


limit

The most complex of the stanzas, limit, gives you resource control over your job. It supports many options, cpu, core, pretty much everything in man 2 setrlimit you could want to set. Its too much to document here, but the rule in general is you can set RLIMIT_FOO with the foo argument and the appropriate value.

start on foo
limit nofile 10     # No more than 10 files open at once
limit as 65535      # Address space no bigger than 65535 bytes


Well I hope this has been informative. Its going to be an interesting release.


A series of images
2008-07-17T13:13:00.000-07:00





Spam poetry
2008-07-10T13:11:00.000-07:00
The hollows, and plains, and platforms all about. Is favorable
our dwelling is pretty this evening of confusion worse confounded
before i get back he asked a blessing on his food in the
same words and edwin was holding up wonderfully, for an
invalid. Of the scene that had taken place between arthur
whites and all, and beat them in a flaggon or the barber
shop, mawruss, and i would be right him with the rumor that
was fast gaining ground. So outre as a dying reference to
a rat, and the melts quickly in those hot countries, producing
'surely he is sleeping very heavily?' i whispered. So great
that it would compel utterance therefore, i am inclined
to think that a good mountaineer at it clemency came down
the stairs and through same way about it as your women.
you will not and wrong to revenge on the heads of the spaniards.
forth to the field! Let france appear in arms! A weakling,
he said. That is not so, mild hori avenue to the foreign
office building, and more.

--Gurske Beemon, who claims I can get any woman into bed with some of his "V d   I    A r    G    c R  A"


Show the current git branch in your bash prompt: Pro version
2008-07-03T07:10:00.000-07:00

. /etc/bash_completion.d/git
export PS1="[\u@\h \W\$(__git_ps1)]\$ "


Turns out the git devs have already provided a nifty little helper for these situations. The location of the completion file may vary by distro (I'm using it on F9, naturally)


Show the current git branch in your bash prompt
2008-07-02T12:06:00.000-07:00
export PS1="[\u@\h \W\$(git-branch 2> /dev/null | grep -e '\* ' | sed 's/^..\(.*\)/{\1}/')]\$ "


In a normal folder, it looks just like the stock Fedora prompt. Enter a git repo though, and you get a branch name after the PWD.


I can't stand second best
2008-06-28T11:51:00.000-07:00
Whenever I find a technology that I really like, theres almost always something similar-but-different that comes along and claims to do X or Y better, and all my friends are checking it out. Sometimes its better and I move straight into it. The strange thing I've noticed though, is that in virtually all other cases I wind up hating this newcomer with a fiery passion.

The first victim was Python. I liked Ruby first, but python was (and is) what all the cool kids were doing. As I began using it, I was mostly happy with it. I mean sure, using an indent as a token with syntactical meaning would seem to imply that you'd want a single fixed way to express an indent (and therefore, tabs not space). And sure, typing len(foo) instead of foo.len() was a sin against OO warranting excommunication, but hey, who likes a purist? And hey, would I really care that lambda was so bad if I hadn't been using Ruby? I suppose having so many methods on the standard objects be in-place was a bit difficult to deal with, and not being able to add new methods to the base classes was unfortunate, and not having a good lambda mechanism meant I couldn't use map/reduce as effectively as I wanted, which was pretty much out of the question anyway since they used function calling semantics rather than being a method, but that meant I couldn't do complex transformations on information as a single series of method calls on the previous result, which would screw up half the time anyway because the operations were in-place, and now... wait a minute, I hate Python!

Now, Python is doing very well for itself, and is gradually eating away at Java's ill-gotten turf, and I welcome all of this. I can tolerate the existence of Python. I don't mind it being around or being used, I don't dread having to dig into it to fix a problem I have, but I don't particularly enjoy it. Its not bad, its just not right. Personal preference? Maybe, but it does show what happens when I try to look at the more popular cousin of something I already found the merits of.

The latest victim was bzr. I am a git user, but I have to work with bzr in order to contribute to Upstart's upstream on Launchpad. It was rocky from the start. Bzr's "a directory is a branch, not a branch container" mantra, which most bzr users are exceptionally proud of, always rubbed me the wrong way. This is a preference issue. I wasn't too upset by it. It just meant that local branching was no longer a part of my daily workflow. I adapted. The next bump I ran into was lack of a rebase feature. This was the bigger culture shock. Bzr doesn't believe in keeping history "clean." I don't quite understand the philosophy, but they seem very near to proclaiming that the shape and ordering of the history is an implementation detail. There doesn't seem to be much emphasis on the "what if I need to look/go back there" use case. Why would the user care what's in the repository anyway?

The final straw was in an attempt to remedy this flaw. There's a project to create a rebase plugin for bzr, and on a friend's suggestion I checked it out. This was the beginning of the end. Rebasing took forever. The plugin was ungodly slow, and this was the first time in dealing with bzr that I really was bothered by the performance. Bzr does bench slower than git/hg, but I consistently found it to be "fast enough." I didn't find myself going to get a coffee while bzr crunched things until this. Despite the slowness, though, it did appear to set my tree in order, so I went ahead and pushed to Launchpad... which failed. It failed because my two branches now appeared to be divergent. In order to push, I'd first have to merge in my outdated upstream branch that consisted entirely of changes that were already in what I was pushing. Well fine. I guess that's an easy enough issue to have, particularly when I just did an unsupported operation. I'll just set things an order and... and thats when I realized I was fucked.

Bzr does not have cherry pick. It does not have interactive rebase. There were ways to pull the changes out of my working tree, but no way to rearrange the history to be a strict superset of my upstream. The operation seemed so trivial, and yet it was impossible. Suddenly it was all so clear to me. The "advanced features" that bzr admitted to lacking and claimed it was just as well off without weren't advanced at all. They were the whole point! Git's history manipulation was the reason I used it. All bzr does is track your changes. Git actually lets you manage them.

Two days later I had the opportunity to play with interactive rebase for the first time. I was able to squash a pair of commits several down from the head with no difficulty whatsoever. I went straight off and asked a git developer if there was a git-bzr plugin. He provided me with a rough copy of one that wasn't quite ready for heavy lifting, but its coming. I won't be sad to be rid of bzr. Its usefulness is now all but lost on me.


All's right with the world
2008-06-08T10:42:00.000-07:00
I don't use all that many Firefox extensions, but I've found one I've been missing for awhile. Tab Mix Plus has apparently been around for a bit, but it had a feature I hadn't heard about that I've been sorely missing in Firefox.

I have most actions in Firefox set to open a new tab. Pretty much the only thing that doesn't is clicking a link or the forward/back buttons. I figured out how to set an about:config flag to make the search bar use new tabs, but the same key for the address bar itself isn't available, and there seems to be little motivation to add it. The act of searching for a new page and punching in the new url are tightly coupled in my mind, to the point where I was continually tripping over myself with the search bar and address bar behaving differently.

But no more!

You'll need one of the dev builds to use it in FF3B5 (all you happy Fedora people) but it seems stable from here.


New Fedora Planet
2008-06-03T12:03:00.000-07:00
Seems not everyone had a smooth transition to the new planet.

On the upside: hackergotchi.


Got me a hackergotchi
2008-05-20T23:55:00.000-07:00


Awesome :) Thanks to Jojo


More Datashi
2008-05-06T20:21:00.000-07:00
There's a new Datashi package available, with some various fixes.

Among other things, the tarball actually contains the program now *facepalm*

Ok, I've never used rake before. Are you happy?

Get IT here.


I kick ass!
2008-05-04T13:18:00.000-07:00
24



You are not Datashi
2008-04-25T13:38:00.000-07:00
A few months ago I had a conversation with my roommate Mike (Spider on here) about a program to organize his anime collection. Its one of those cute little projects all programmers pick up at one point or another, until you realize that A) Mike is OCD, and B) his anime collection is over 2 TiB. Then it becomes less of a cute project and more of an enterprise data warehousing solution. We needed a solution that would allow files to be tagged with any number of pieces of information of any type. One file might have an "author," one might have an "artist," one might have a "number of cars depicted." There is no telling what is or is not relevant to any one file. This is kind of difficult to map to a relational database, where each item is a row in a table with a fixed number of properties.

So I set to work, and a few weeks later, I had Datashi. (link updated 5 May 2008)

Ok, so there's not much there really. All it is is a command line front end to a database schema and a module system for storing files. Eventually I'd like to build a web interface as well as a FUSE module that can render its contents in a dynamically drawn folder structure.

The name has two sources. First it is a slight compression of Data-shi. Data is obvious enough, the -shi postfix is a Japanese convention, indicating mastership of dealing with something. As a grotesque but effective example, the word "butcher" would be translated into this mangled dialect of Japanglish as "Meat-shi" (apologies to readers who actually speak Japanese, who likely just spat coffee on their monitors).

The second source is a play on this comic.

It works like this:

You have a series of Objects. Objects can be either files, or some sort of meta-object. The latter is useful because there is an optional parent-child relationship for objects. So, using our anime collection example (certainly the tool is applicable to other things) A "series" object could have many "episode" objects which could have many "video/mpeg" and "video/ogg" objects (assorted resolutions/encodings/subbers) (also please forgive my MIMEType inaccuracy. I don't have a cheat sheet in front of me, and I'm lazy).

Now, we need to find these objects. Each object has an ID, and a type, and also several "keys" and "tags." Tags work just like tags on a blog: "funny," "drama," "hentai," what have you. Keys are similar, except they are paired name => value entries. "Author => Yoshitoshi ABe," "Year => 2003" and the like. You can find objects based on keys and tags, based on type, by ID, or by any ANDed and ORed permutation of the above. You can also find children of objects with a given set of the above properties. Or parents.

File objects specifically always have a MIMEType as their type, and are associated with one or more volumes. Volumes typically represent some sort of directly accessible physical disk space allocated to Datashi, although some of them could represent, for example, a CD collection where you have the files stored. In this case you won't be able to retrieve the files directly through Datashi, but you can still track them. A file object can have more than one volume location, so that you can track redundant copies on multiple types of media. Volumes are associated with volume drivers, modules that plug into Datashi and tell it how to deal with a specific type of storage mechanism.

It took some rather creative SQL to get all this working. It remains to be seen how it will scale (I'm reworking some of the internals right now), but its a good start, and I'm having fun writing it.


Yes.
2008-04-17T18:32:00.000-07:00
I'm kind of a bitter sort. I tend to be far too difficult to impress for my pay grade.

This is the finest advance in software engineering I have ever personally witnessed.

Just the other day I was wondering whether there was any room left in the ORM concept, or if we were going to have to start looking for new ideas or abandon the relational databases for solutions more adaptable to OO. My faith is renewed.


Yahtzee is awesome
2008-04-13T16:41:00.001-07:00
Just felt like pointing it out.




Shell History Meme
2008-04-09T12:45:00.000-07:00

[sadmac@baudrillard devel]$ history | awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head
252 ls
114 vim
111 git
99 cd
56 ruby
45 sudo
45 irb
26 rake
26 make
21 diff

[sadmac@foucault ~]$ history | awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head
249 sudo
128 ls
49 rpm
47 cd
45 echo
37 vim
23 cat
22 man
21 reboot
13 exit

[cdahlin@hitchhiker ~]$ history | awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head
14 ssh
12 exit
9 ls
7 screen
5 vim
5 cd
4 scp
4 irb
3 man
3 irssi



Vexille
2008-02-23T14:16:00.000-08:00
Its been a bit since I saw Vexille, but I promised my review, so here it is. Some forewarnings: I watched the film in English, which despite the poor reputation of American dubs, really wasn't an issue. Though not spectacular the dub is mostly passable, and at any rate the least of the film's problems. Also, since the audience for this sort of movie is particularly sensitive to plot reveals, I'll go ahead and throw out a ceremonial spoiler alert. I don't intend to reveal much, but I'd like to be able to talk freely about the film, and I won't have it be said of me that I didn't cover my ass.

On to it:

Vexille opens with murky footage of some sort of military robots being rolled down the assembly line, as a narrator fills us in on the far future: robots are everywhere, and Japan builds them better than most. Now they want to build robots that look like humans. The U.N., no doubt having seen Blade Runner, decides this isn't a good idea, and passes an international law against it. In reply, Japan decides not only to pull out of the U.N., but to completely shut itself off from the rest of the world, which they accomplish by surrounding the entire island of Japan with EMP (dont'cha just love that stuff? I bet its good on toast too). The narrator closes by noting, in much more suspenseful and substantially politer words, that we lucky bastards will be among the first to see Japan in 10 years.

The first shot of a human character we get is the finger of a fat politician drumming on the arm rest in the back seat of his limo, and from this one shot I knew that the animation wouldn't be getting along with me. That bizarre mechanical gait that plagued so much of early CG hasn't quite been polished off in Vexille. I suppose its appropriate for those characters which are, in fact, robots, and its polished enough that you can begin ignoring it as the film moves on, but it isn't pretty. The art in general has a bit of a toy-like feel that probably would have been unnoticed if the scenery had been cel-shaded like the characters, but the film does make up for it in having some very clever design. The intricate cityscapes, ultra-modern architecture, and futuristic weaponry of 2077 are fun to look at, though the latter of those does seem to borrow from Halo to an almost distracting degree. Still, in the end, its no Pixar film, and it shows.

The actual story picks up with the introduction of Vexille herself (yeah, if I'd had to guess at the meaning of the title before I saw the film, "girl's name" wouldn't have been my first answer either. I guess its like "Lucille" but more technologically advanced). Vexille is a member of SWORD, an American military force. They're your usual future-asskickers, with big armored suits and jetpacks and massive firearms, et cetera, et cetera, et cetera. After a couple of androids turn up on US soil, SWORD is beginning to expect that the Japan has been up to something under their spiffy high-tech curtain. So they send in Vexille and her squadmates under the command of her boyfriend Leon. Now, ignoring everything else that is wrong with that sentence, Leon is easily the weakest point in the film. He is as paper thin as they come, voiced by Travis Willingham with David Duchovny's drowsy tone and William Shatner's odd melodramatic cadence. Even as the center of Vexille's motivation, it feels as though the film would be better off if he'd simply been left out. He appears mostly in flashbacks after becoming indisposed relatively early in the events of the film, and this fortunately keeps his screen time down a bit, but his centrality to the plot means we are constantly reminded of his seemingly purpose-free existence.

The state of things in Japan is as grave as you would expect. The entire country has been leveled into a desert, save for a shantytown where Tokyo had been and the headquarters of the perfunctory evil corporation, Daiwa robotics, on a platform in Tokyo bay. After the rest of her SWORDmates are quickly dispatched, Vexille wakes up in the aforementioned shantytown, under the care of Maria, an ex-love interest of Leon (how awkward is that?), and another stamping of the Sarah Connor "tough bitch" mold: brassy, independent, and a trite repetition of a character we've seen a thousand times. There's a lot of characters like this. In its rush to advance the plot the film often forgets to develop characters into more than what is sprayed on their surface. Leon is unique in his almost extra-like flatness, but many others are simply draped with the nearest cliche and left to their own devices.

As it turns out, Maria and the rest of the inhabitants of the area are the last survivors of a virus (no way!) that was purposely unleashed by the government (no way!) which turns people into robots (no wa- wait what?). This is where the philosophical portion of the film kicks in. The population of the shantytown is all infected, and mostly consumed by the virus. While mostly metallic, the effects on the brain are the last to take hold, so all of them are "making the most of it" until they all fall into convulsions and stand back up as mind-controlled devices. While none of it feels that original, the film's earnest attempts at questions of humanity are actually some of the high points. It keeps missing by an inch though, primarily due to the atrophied characters. The actual humans on which its all being built are never quite fleshed out. Take Takashi, the usual smilie-eyed little boy sidekick that spends his screen time following Maria around and doing his gosh-darnedest to be chipper and upbeat and adorable. He's such a familiar cardboard cutout that the film's most poignant moment, a well directed scene featuring his rather ugly demise, never really registers with the audience. It just never mattered to begin with. He was scenery from the start.

This is the real issue with Vexille. It just kind of moves forward. There's a soliloquy here and there to inject the philosophy, but the rest of the time its just railroading to the next action sequence. The action itself is mostly over the top and nothing special, save for a few creative moments. The story just feels neglected. Add to that the second-rate CG and really, unless you need an explosions and gunfire fix, there's not much to recommend Vexille to anyone.


Trying out upstart in F9
2008-02-19T07:57:00.000-08:00
If you're jumping at the bit to try upstart out on your F9/rawhide box, here's what you need:

Upstart package
Sysv-compatibility events
Version of sysvinit with tools package split out (use the version ending in .upstart):

Be sure to take a look at what you're up against first.

Have fun :)

--CJD


Bad Japanimation and a Good Pair of Shoes
2008-02-17T10:24:00.000-08:00
I spent yesterday out and about with SRPE on one of their "misadventure" activities. The club is tiny, mostly close personal friends of mine, and we periodically set out in search of disaster and misfortune, in hopes that we might fail to find it.

The main purpose of our sally was seeing Vexille, which was showing in only one theater in the furthest point in Wake County we could have found. When we arrived, the show wasn't listed for the time mentioned on the website, but the cashier told us that it was in fact playing as scheduled. Things didn't get much more reassuring once we were inside. None of the theaters were listed as showing Vexille. After fumbling around for a bit and then listening in on a long confused discussion among the staff, we were ushered in to the theater that was marked as showing Bucket List.

We had the theater entirely to ourselves except for an elderly couple, who had come to see Bucket List, and were unceremoniously ejected shortly after we arrived lest they be exposed to Asian robots with firearms, and one other person who showed up a little late.

So was it worth it? No. I left the theater pining for my lost $6.50, and feeling a little bit guilty about having taken part in depriving an elderly couple of what might well have been their last opportunity to view a film about the enumeration of receptacles. The other person in the theater, who we chatted with for a bit, said that he was an employee, and had at least been spared paying for the movie. I'd go further into the film itself, but the long buried film critic in me is rising up a bit, so I think it will take another post to get it all out.

Afterward, we set out for "Mr. Mike's Used Books," which we noticed on the way in to the theater. There were two people named Mike among us, so at the very least irony mandated our patronage. It was one of those strange stores that crops up in strip malls now and again where you only need to look around once and you can imagine what the room would look like totally empty. The walls were totally white with no hangings or posters, and the only furniture in the room were the cashiers counter and the bookshelves, and despite the shelves being arranged in aisles and braced overhead, the furnishings seemed more like they were being stored there than being made use of. It was as though the owner had intended from the founding to go out of business in 3 weeks, and had built the store to this purpose.

Nonetheless, it was a book store, and it had books, and given that of late books have not played as much of a role in my life as I would like, I made my way to the "Literature" section to put an end to my philistinesmanship. It really has been some time since I read for pleasure, and I'm still marvelously under-read. I feel the need to hide when bumping into high school classmates who were better read then than I am now.

The selection in the "Literature" section was a bit thin, only two shelves wide (Romance sat directly across the aisle and had 4 or 5 shelves, and the entire aisle over was principally popular fantasy, with particular emphasis on Star Trek), but there was plenty to consider. Dante's Purgatorio and Paradicium were both there, tempting since I loved the more famous first book in the series, but I wanted to branch out a bit more. There were a few Faulkner titles, but after having been out of the habit so long I wasn't certain I possessed the stamina for that sort of literary heavy lifting. I settled on a copy of A Tale of Two Cities, since I have yet to read it and I've loved Dickens since I was very young, and a compilation of 5 plays by Anton Chekhov, who I have never read at all.

We then moved on to the mall. Now, I rarely buy things that I need, mainly because the closer I am to a store that sells such things, the further the need for them is from my mind. If I lost an arm in a medical supply store, bandaging the wound would somehow fail to occur to me.

I'm kind of particular about shoes. My last umpteen pairs have varied only slightly from the basic pattern: always understated, dominantly black skate shoes, almost always Vans. I feel skateboarding is best admired from a safe distance, and while standing on something securely fixed, but the shoes are perfect. They are generally durable, and built wide, which makes me look less clownish (I like the look of Converses, but they become goofy at some point above size 10, and up here at 13 they just look silly). Vans in particular have one nice feature some miss (though the last pair disappointed on this): the stiff piece of plastic which supports the back of the heal in most sneakers is in stead made of a kind of Nerf material in Vans. This means when the sneakers are so old that this piece begins poking through, it doesn't slice your heal up and force you to upgrade. In other words, they remain comfortable until they disintegrate, which is the earliest point at which I want to go shoe shopping again.

I went to 4 stores in the mall to find shoes fitting the bill. The first two were disasters. Its becoming increasingly clear that most shoe stores are catering exclusively to K-Real. Furthermore, it seems to be the trend now to mount one sneaker of each style on the walls, and use the floor space of the actual "shoe store" for clothes and other things which are not shoes. This means trying anything on involves direct contact with a customer service associate, which means spending more time shoe shopping.

After passing 2 stores dedicated exclusively to baseball caps I came to Payless. I stood at the entrance for 30 seconds before leaving, entirely convinced that they no longer sold men's shoes. This is apparently not the case, but you'd never know. Every scrap of advertising and piece of decor inside the shop was geared toward women, and the only other man present was the cashier, who looked like a more effeminate version of that guy Chad from the Alltel commercials, and looked at me like I was as out of place as I felt. A shame, since apparently somewhere, hidden from view, mens shoes were on sale.

I finally found a pair of Vans that had just a bit more white on them than I wanted, but were close enough. Myself and the SRPE-ites retired to Cinnabon to reflect on the days purchases, which, for me, apart from the shoes and books, included a DVD collection of Pixar animated shorts. We finished the day hunched over our all-American high-calorie desserts, explaining to Sampson who Chuck Norris is (he'd never heard of him somehow).


Worst of unicode
2008-02-11T21:57:00.000-08:00
Populating Unicode's seemingly bottomless namespace is a difficult task. You'd think with 7000+ languages around the world we'd be able to fill 'er up but there's still room in the damn thing. Seeing this serious problem, the Unicode standards folk have occasionally sought to fill the gap by tossing in a few "bonus features" at no cost to you. Today we will look at some of the symbols that appear in Unicode not to make it easier for other cultures to express themselves, but.... well.... who knows? Let's take a look:

U+260E BLACK TELEPHONE (☎)

Fedora's default font set renders U+260E in a stylish rotary design. The slick black look is sure to impress, but if you're conscious about the environmental impact of using too much ink, U+260F WHITE TELEPHONE is also available. Apart from its decorative clipart-esque charm, this character is actually used in a few middle-eastern languages, and is pronounced "brrrrrrrrringgg!"


U+2654 WHITE CHESS KING (♔)

Playing chess by email became popular in the very early days of the internet, but with no officially sanctioned assortment of pieces, games were nearly impossible to officiate. Things came to a head in the early 90s when Eric S. Raymond was famously defeated in an email chess game by Richard Stallman using his controversial "Zerg Rush Gambit." But no more! The Society of E-mail Chess Grand Masters have contributed to Unicode an officially sanctioned text-based chess set. 14 characters render the pieces in both black and white, leaving no doubt as to what constitutes a playable piece in email chess.


U+2639 WHITE FROWNING FACE (☹)
U+263A WHITE SMILING FACE (☺)
U+263B BLACK SMILING FACE (☻)

See anything wrong? Notice anything missing from the above list? *sigh* I guess an ISO-certified way for an African American person to electronically express melancholy will have to wait for another day.


U+26A6 MALE WITH STROKE SIGN = transgendered sexuality (⚦)
U+26A9 HORIZONTAL MALE WITH STROKE SIGN = magnesium (⚩)

Did you know that if you turn a transgendered person 50° to the right they can explode? Its true!


Timing application startup
2008-01-23T12:58:00.000-08:00
A few recent fedora-devel discussions about prefetching/user experience and the like have brought up the question of how to measure application "startup time." Its an elusive metric, partially because it is largely a perceived value we are after and partially because there is no specific system event which necessarily signals startup.

One solution I've found is using the output of strace. "Startup" for most X11 apps is marked by a storm of syscalls followed by a poll() for further events. Given this we can look for patterns in the times at which strace output lines and get a pretty good figure of startup time.

Consider this kludge:


#! /usr/bin/env ruby

require "open3"

class Event
attr :time
attr :info
def initialize(string)
   @info = string
   @time = Time.now
end

def to_s
  "#{@time.to_f} -- #{@info}"
end
end

events = []

Open3.popen3("strace #{ARGV.join(' ')}") do |stdin,stdout,trace|
trace.each_line do |line|
  events.push Event.new(line)
end
end

max_val = 0
max_bound = nil
prev = events.first

events.last(events.length - 1).each do |e|
if (e.time.to_f - prev.time.to_f) > max_val
  max_val = (e.time.to_f - prev.time.to_f)
  max_bound = prev
end
prev = e
end

puts (max_bound.time - events.first.time).to_f

Now this is a shaky way of doing it, by no means perfect (gnotravex breaks it completely due to its ever-updating onscreen timer). But if you understand what it does, you can get a much better number than with a stopwatch or time fooapp-then-kill-as-fast-as-you-can.

P.S. Damn blogger for not having a standard <code> entry box. Damn them to hell.


Upstart in fedora
2008-01-18T09:51:00.000-08:00
So I did it, I guess. FESCo will be looking over the proposal on Thursday.

It has me wondering a bit about this blog, and what it means. Much as its full of complaints about the lack of readership, I was always kind of proceeding on the assumption that nobody was reading it, even when I personally yelled at them to do so. Now that I'm doing something relatively significant, there's a chance that someone might actually be here. Time to run things differently.

So, errata for older posts: The perfect program thing was written for cathartic reasons because I'm sick of software engineering guru-types claiming that bugs are all the fault of process and fate. Don't take it too seriously. rrn was never implemented as poorly as the post outlining its design suggests. The final code (which is now going to be put to rest anyway) is not based on a queue, despite the fact that my brain seemed to want to hammer it into such a structure. I tend to grow software organically, and I can achieve good results this way because I am ruthless in refactoring and discarding code I no longer like, am as hard on myself as I am on others, and generally work alone. The final design was fairly elegant, and the code was beginning to be just as beautiful, but ultimately it solves a non-existent problem and stands in the way of something that solves a real problem. I'm not sorry to see it go. I had my fun writing it, and I get to have all of the fun seeing Upstart go into usage, so I'm not missing out.

On to more important things: Upstart itself is going to be great for Fedora. It almost certainly will mean faster boot (for no reason other than it forces us to finally tear apart the great bash monster our current system has become), it will let us develop a more stable and robust system by monitoring services that tend to go down (:cough:PulseAudio:cough:) and reacting automatically, and it will replace many tools that operate both system and session level services with a single unified mechanism (fewer moving parts).

So far it seems that Upstart will sail smoothly into Fedora. Getting people to use it will be interesting. Once a procedure for event driven init is in place I hope we will mandate that no new sysv scripts be installed. From there I don't think its too unrealistic to expect Fedora to be mostly free of the sysv compatibility features by F11.

Well enough about that, other things about FUDCon:
The barcamp format is all the fun its been made up to be. I saw some great talks, particularly Spot's talk on RPM packaging.
Remember that SysObject idea I had? In that idea post? Func already mostly does it. Damn it. This is just like the time I came up with spray-on sunscreen.
Greg DeKoenigsberg has an angry inch.
Don't let the Dutch recommend beers to you. It nearly killed me. It was also delicious though.
Jon Stanley knows quite a bit about beer. He also drank one that came in what was universally agreed to be a mid-sized wine bottle.
Paul Frields gave me a hug. I guess Fedora loves me.
The Fedora Unity people take a lot of issue with some aspects of Fedora. The rest of Fedora seems to take a lot of issue with the Fedora Unity people. I remain neutral in this conflict, though at the moment I'm leaning a bit toward the Unity side since they provided me with beer and pizza after the first hackfest. Bros before hos? Or does that not apply here?
Alex Maier can dance. And paint. She also attempted singing during the con, and that was alright too after a shot of Absolut.
I sang Plug-in baby. Loudly. Expect accessibility improvements in F9 now that most Fedora engineers cannot hear.
I should have mentioned there was karaoke at FUDPub after the con. We were not just bursting into song during talks about linux.
This fact did little to sway Yaakov, who didn't think we needed to be singing under any circumstances. He was fun to hang out with during the rest of the con though, and was first on board for Upstart, too.
So that was FUDCon. Hopefully I can make it to Boston in 6 months for the next one.


Init update, and FUDCon
2008-01-09T18:08:00.000-08:00
So, my new init system, currently dubbed rrn, is nearly usable, and may never see the light of day.

rrn was designed to meet the particular specs that had been laid out for a new Fedora init system. I don't know how those specs came into being, but it seems they aren't as popular as they once were.

Upstart, on the other hand, seems to have found some friends in Fedora, and I have to say I am fast becoming one of them.

So at FUDCon, I will be holding a talk to discuss these issues, with hopes that in 3 weeks, a new init system, any new init system, will appear in rawhide. I want this done. How is less important.


All is not well in the land of GNOME
2007-11-26T19:21:00.000-08:00
If you haven't been following the news, basically someone said this. Which lead to this, and also this.

I don't really follow GNOME internals much, but I've seen this before. From the recent *BSD riots on lkml to the Ice Weasel debate, arguably less-than-civil discourse is common and perhaps even somewhat cyclical in the open source world. And while detractors might often sight this as an ugly side-effect of that untamed, unprofessional domain, in a way, the anger is what makes open source work.

Open source types display a sort of militancy about what they do. They take threats to their project personally. Its the kind of investment in a project that just doesn't grow when a company is the owner of everything you do. Not only do we have the firm opinion that We are Right, and that by extension They are Wrong, we tend to believe that being Wrong is by further extension Evil. We've extended that pretty far, but its a principle that recurs often in the human condition. In the US people have been getting a fresh look at this idea for some time now. Take this case of a couple of boys who were suspended recently for using the word "noose." Its not the first case we've heard like this, and it won't be the last, but it is a demonstration of an idea that the American public is growing much more aware of these days: A policy doesn't have to be directly evil to cause you real, deep injury. Incompetency does the trick. If your child's well being was on the line, I doubt you'd spare the inept any of the hard words you were saving for the guilty. The fact that we take our desktop environments just as seriously is to the credit of open source.

Why is the kernel such a well-written piece of software? Is it the brilliance of Linus? The no-nonsense design philosophy? These are important, but they'd be meaningless if not for the fact that Linus will not take your shit! Even Ingo Molnar, who has been earning plenty of well-deserved praise for CFS, has had Linus spit a patch back at him on at least one occasion I observed, with a thick stream of harsh words following it (if anyone knows where in the lkml archives I could find the email I'm thinking of, please post it). Expectations are high, and even mediocrity, even second best, is an insult to what we've put in so far. That's the mantra of open source. Brutal? Sometimes. But it makes great software.

If Murray doesn't feel like bullshitting Waugh to get something out of him, or doesn't think the bullshit would buy him anything anyway, then so much the better. If it devolves into a raging conflict, so much the better. A good slap upside the head keeps people alert, and if we have to do it, best we all do each other at once.We're being held to our own standards, all our names go out on every one person's work. We can't give ourselves a moments rest, nor can we give rest to our comrades.


List of ideas
2007-11-26T13:19:00.000-08:00
Here's some projects I would like to do but probably won't get around to for some time:

SysObject
The idea here is to create a system administration toolkit in either ruby or python that would allow you to administer a system via method calls on an object. Think of it as sort of a what-if-physical-computers-were-dbus-services scenario. For example, consider the following admin script:


SysObject::ObjectSpace.login("username","password") do |objectSpace|
boxA = objectSpace["webServer1"]
boxB = objectSpace["webServer2"]

boxA.httpServices["phpBB"].useDatabase boxB.services['mysql'].schemas['phpBBschema']
boxA.services["apache"].AllowOverride = "none"
boxB.reboot
end


Another relational filesystem
There's been some people who have tried this before (WinFS etc) but few people have found a good way to make the data available. Much as I tend to dislike XML, I think XPath is one of the better solutions to this. It offers the perfect mechanism for relatively succinct queries and is also a superset of conventional unix file paths.

A new programming language
I think C is in need of a bit of a refresh. The reason that this hasn't happened is people have gotten distracted with VHLs and other interpreted languages that are better for a subset of tasks, but still ignore some bits of C's use cases. A version of C that got rid of some of the curly braces, had a more powerful preprocessor (yes, more preprocessor. And by more powerful I mean implement-your-own-language-in-it powerful), and got rid of the whole horrible include system and function predeclarations would fit the bill.

Well there's that. Enjoy!


What to say...
2007-10-29T00:26:00.000-07:00
Well, despite my complete and thorough lack of readership I find myself once again compelled to update, and at least keep the 2-post-per-month average.

I've been doing a bunch of python work for this this weekend. I'm not much for IDEs, or vPython (which is what you code in physics lab at NCSU) but its being written by a friend and when I see code that needs fixing, I have to dig in. Plus Red Hat uses a lot of python around the house, so its nice to have an opportunity to sharpen it.

Brad is a bit of a source of pride for me and my roommate. He moved into my roommate's previous room a few years ago, a former high school football player with a gallon jar of protein supplements and an encyclopedic knowledge of controlled substances (and the rather ugly ways in which they tend to kill people). He's now a full time Fedora user and is solving one of his own problems with a fairly non-trivial python program. We do this to people :)

My other project is a rewrite of prcsys for Fedora. Fedora's boot time has been getting some negative attention lately, so I thought I'd pitch in.

prcsys is a program that boots a folder full of LSB-compliant init scripts in parallel, while respecting any marked dependencies between them. It works somewhat like /etc/rc. We'd like to be able to use it to do job control via DBUS as well. The problem with this is that the current implementation takes a whole folder and starts everything. DBUS would want to probe single files. What complicates this is that prcsys works by starting a thread for every init script and then telling the threads that have dependencies to block on the threads for the scripts that they depend on. It seems fairly smart when you first look at it (though the implementation is a bit strange), but it creates way too many threads if the boot process is mostly linear, and way WAY too many threads if you're intending to start 1 service and maybe 3-ish dependencies.

The replacement, thus, will resolve dependencies by queue. The dependencies are listed by the "Provides" header inside the file and not the filename, so we have to open and read everything in init.d to resolve a single service. That would have been a lot of pthreads to start, the huge majority of which would simply block and then be killed.

The new algorithm simply starts with a queue containing objects representing all of the init scripts. The operation goes like this:


Dequeue a script.

Is it in the list of things to run? If not, re-enqueue it and go to start.

Are its dependencies running? If not, add them to the list of things to run, and re-enqueue it. Go to start.

Run it. Go to start



The algorithm halts when a pass is made where nothing runs. Also, scripts can be discarded at step 2 if the list of things to run did not grow in the previous pass.

Well that was dry and dense as usual. Enjoy!


Rails vs. PHP
2007-10-02T14:45:00.000-07:00
I've been meaning to take a stab at this for awhile. I don't think its too bad an article in truth, and ultimately Sivers seems to admire rails. Nonetheless there's a couple of things that did really irk me. So I'm going to list off his points, and when annoyed, I'll complain :)

#1 - “IS THERE ANYTHING RAILS/RUBY CAN DO THAT PHP CAN’T DO? … (thinking)… NO.”
Well we certainly started with a bang, didn't we? If not for this statement I would be leaving Mr. Sivers alone. Its not that he's wrong. Its that he's right. In fact his statement is basically corollary to one of the most important foundation principles of computing. The minimal requirement for any programming language is being able to do pretty much everything all the other ones can computationally. Otherwise there's no point to it. The exceptions might be a few languages that lack the necessary system call interfaces to talk to the hardware and software around them. You could write your webapp in assembler if you wanted to. You don't. Because programming languages are about 2 things: aesthetic form, and how efficiently the most logical solution typically maps to machine instructions. "Capability" in the larger sense is not a parameter worth evaluating.

#2 - OUR ENTIRE COMPANY’S STUFF WAS IN PHP:  DON’T UNDERESTIMATE INTEGRATION
This is a much better argument. There's something to be said for changing in stages. Rails integration with PHP isn't the most difficult problem to tackle though. Starting by moving the PHP app into rails' static content area, then gradually moving the changes into the app itself would be the way to do it (the database transition could be handled by creating a few views). Still, it is a cost. This applies to any migration though, and in the long run there probably could have been a net benefit to rails.

#3 - DON’T WANT WHAT I DON’T NEED
The argument being offered here is that most of rails' features probably won't be used by a given webapp. I don't know that this is true. Rails does many things automatically that don't have to be thought about, such as cache control. There are also things like the test framework, which, yes, can be left out, but probably shouldn't. In the end I'm not convinced that rails is linking in any more framework code than PHP is linking in libraries from its array of module functions.

#4 - IT’S SMALL AND FAST
Here Sivers makes the rather useless claim that "One little 2U LAMP server is serving up a ton of cdbaby.com traffic damn fast with hardly any load." without even mentioning rails. Fine. What could rails do? Maybe better, maybe worse. Who knows? The performance debate between these two has always been interesting, but you aren't contributing to it if you're going to make non-comparative assertions.

#5 - IT’S BUILT TO MY TASTES
In this point Sivers complains about having to "adapt his ways to rails," and how he had to "hack rails up" to meet his needs. The former is fairly useless whining. The latter is simply a crime. Rails doesn't need to be abused into any task it should be doing. If there's a task it shouldn't be doing that's your problem. Learn how to build IT infrastructure that isn't encompassed entirely by the front-end web app. His argument about pre-existing databases is fairly moot (again, views people), and as for "adapting his ways," that's the damn point of a framework. It offers a single well-defined set of abstractions to deal with problems.
Dealing with all problems with the same set of techniques is what makes framework apps much simpler. Good software design is about eliminating special cases, not drilling corresponding special cases into everything else.

#6 - I LOVE SQL
I repeat: Good software design is about eliminating special cases. ORMs eliminate code duplication, and unify the semantics for dealing with information around a single paradigm. There's no reasons to have to warp into a completely different programming language to handle a different type of information. It just leads to sloppier code.

#7 - PROGRAMMING LANGUAGES ARE LIKE GIRLFRIENDS: THE NEW ONE IS BETTER BECAUSE *YOU* ARE BETTER
*cough*whipped*cough* :) Ok, seriously. Sivers' assertion here that the real difference between languages is the quality of the programmer is simply painfully silly, and what's more, whose to say that a mediocre programmer's experience with a language is less valid? Isn't the idea that a language makes it harder to hurt yourself a good thing? Languages that bad programmers have a hard time screwing up tend to be the ones that good programmers have an easier time debugging. I have seen beautiful C++ code, but I dislike C++ because beautiful code is rare and ugly code is easy to produce.

My experience with programming has in no way been a series of ditches. I learned C early, and I was very bad at it, but never once thought C was bad. I learned Java much later and continue to think that it is one of the worst languages out there (truly our generation's COBOL). I returned to C afterward to find I had gotten far worse from lack of practice, but still found it infinitely preferable. I have since sharpened my C skills a great deal. Ruby happened to me in the meantime, and now its my high level language of choice.

PHP is an abysmal language. Its bizzare and upredictable duck-typing, its frustrating and often wholly insufficient security method of adding SQL escaping to tainted input without programmer permission, its jumbled and confusing OOP extensions, "absolutely positively equal to," its just dangerous. I would like to see Sivers try rewriting his site in eruby scripts rather than rails, just to see if point 7 still holds.

Well that's all for me fans. Check in next time when I sever my own head.


Live from 1:36am
2007-09-25T22:35:00.000-07:00
Whoo! It's been awhile. I'm here live from an early morning production push, where I've been doing... not much. They brought me in to do verification, and there's nothing to verify yet (it's been a slow push. Many people are angry and saying mean things to their computers). So here I sit.

One of my assignments recently was to investigate using buildbot with git. We've been using buildbot with SVN and previously CVS for awhile now, but git is a little different. If you aren't familiar with it (and don't feel like watching the video at the "a little different" link), git operates without a central server. Each person hosts their own repository containing their own branch of the code (or several branches if they so choose). The principle operation then becomes branch/merge rather than checkout/commit. The idea is that developers can each pull their own copy of the source (without needing any special access or "commit privilege," mind you) and implement whatever changes they want, then exchange them freely with others. Meritocracy determines whose changes percolate into the "stable" branch (usually in the release engineer's repo) as code is transfered at the will of the receiving end, not when the commiter decides to foist it on everyone. All this is a wonderful thing really, so wonderful that we have been trying to move toward git since Linus' video debuted.

Buildbot, however, still has little clue what the hell its supposed to do with git. A typical buildbot setup has a single "buildmaster" monitoring a repository for commits. When a commit occurs, it tells a buildslave (a sort of worker daemon, potentially running on another server) to check out, build, and unit-test the code. Lovely. How does that fit in with git? There's quite a lot of static configuration involved in setting up buildbot to use a repo, whereas git repositories vanish and appear with what one might lovingly refer to as "violent abandon." What's more, buildbot typically monitors a single branch and performs a single type of build (there is support for multi-branch monitoring, but it's a bit coarse). In Git, branches are created and destroyed on a whim, at the developer's discretion. Its also for the developer to decide when and how they should be built.

The problem of many transient repos can be somewhat solved by simply considering there to be no repo at all. By simply considering the url of the host machine to be a property of the branch, you can deal with git entirely in terms of branches and ignore the repository concept altogether. Buildbot has a mechanism to allow change information to be pushed by the repo rather than polling the repo to see if changes have happened, and in this operating mode it does not need to know the url of the repo per se. It just takes a little inventive upgrading of buildbot's rudimentary git checkout mechanism to treat the "branch" as a repo url followed by a branch name (currently buildbot can check out a git repo with cogito, but must accept whichever branch is  currently "active" on that repo upon the initial checkout. It cannot deliberately compile a given branch either statically or dynamically).

There's another option, and this involves thinking a bit more about the development model of git. If there's no centralized repo, does a centralized build system make sense? Why not have every repo have its own build system attached? Oddly enough, buildbot seems better suited to this. A buildmaster is run on each committer's box next to his git repo. It builds his branches for him at his desire, publishing the status of the builds on a web server accessable at his workstation's ip. If his work is important to others, they can look at this status to see what is worth pulling from him. His buildslaves can run elsewhere, so his workstation is free of the heavy lifting of the build. Best of all, git support in buildbot need only be brought up to speed with the other scms. A single static repo is the master's only concern.

Either way, it seems I am going to have to dive into the buildbot code to get things working. In the meantime more important projects have swarmed in, and this is on the backburner. *sigh*



All Programs are Perfect: The Significance
2007-09-02T14:59:00.000-07:00
There are other things I want to get on to, but first I'll finish off this thread.

If all programs are perfect, why are there bugs? Because we have an idea of what we want that is inconsistent with the standards by which a program should be judged. Programs are not oriented toward serving man. They aren't oriented toward anything in fact. They exist in isolation, strictly self-defining. A program is therefore perfect for doing not what we need, but what it does. Being representable as a deterministic finite automation is the only onus on a program, whether saving your spreadsheet crashes it or not.

As an example. Lets take a look at the old classic, Microsoft's shining star, Minesweeper. This is a simple program that has rarely crashed (never in my using) and generally offers a satisfying game play experience. But let's look at it another way: Minesweeper takes no textual input. It lacks any parsing code whatsoever. It can't store variables, there's no symbol table. It doesn't come with the python standard library. Minesweeper is perhaps the worst python interpreter that was ever written.

"But minesweeper isn't a python interpreter!" Says who? Is there metadata on each bit of the compiled binary that says "this bit shall not interpret python?" In fact it is true that Minesweeper is not a python interpreter. It's a program. Nothing else. Judged by itself it is merely a linking of states. What it does can be more strictly defined. Minesweeper does not interpret python. Whether that makes it a more cohesive game or an utter failure of a python interpreter has to do with our perception, and our expectations, not the program itself.

Well now what? It seems that with this new collection of semantics it's impossible to talk about anything. How do we weed out crappy code, make our deadlines, or respond to our customers when anything we give them is interchangeable? If we want quality software, we need a new set of terms. If all programs are perfect, but not all of them meet all expectations equally, then we must judge them based on those expectations if we are ever to differentiate between them. It is thus we consider the notion of the problem. If the problem is "Use up 300mb of ram then segfault" then yes, your buggy code is acceptable. If the problem is "Write a new process scheduler for linux" then it decidedly isn't*. Its not that your program is "good" or "bad" or "less than perfect." Its that it is simply incorrect. Your software is wrong.

Talking about programs in terms of correct and incorrect, right and wrong, is much more precise. These terms are inherently transitive. There is a set of expectations or rules implied by them. It also has the delicious quality of being binary. 3.8 is not a "mostly correct" answer to 2+2. Nor is 3.9999999 . Absolutely anything other than 4 is quite simply wrong. A program, similarly, is either the best available solution to its problem or a waste of time. Anyone who has ever seen a flame war on a technical discussion list already knows this.

"Best possible solution" can of course be hard to judge. Nobody sets out to write a solution that is second best. We often must assume that what we are given is as good as it can be. Tomorrow we might realize the opposite. Suddenly the one true solution is fit only for the dustbin. The effect is that in the presence of a better solution any solution is crap. Why does Windows suck? Because Linux exists. It's that simple.

* I'm not referencing CFS here. Haven't tried it yet, but it seems to be a decent piece of work. Sorry to scare you like that mingo :).


All Programs are Perfect: The Proof
2007-08-18T19:12:00.000-07:00
Lets assume that there is an imperfect program. This leaves us with 2 cases:

A) The program has a reparable flaw
If this is the case then there must be some way to repair the flaw.
The only way to repair any flaw in the program is to modify the code.
Having modified the code, you have effectively created a new program.
Therefore it is impossible to have a correctable flaw in a program, because the program effectively ceases to be once corrected.
B) The program has an irreparable flaw
If this is the case, then no correction will make the program perfect.
Thus you can make an infinite number of changes to the program and it still will not be perfect.
The set of all programs resulting from changing any given program an infinite number of times must be the set of all possible programs.
This would imply that all possible programs exhibit flaws.
There exists a turing machine for which there is a zero-byte program that does nothing.
This is inherently a perfect implementation of a program which does nothing and uses no resources.
Therefore statement 4 must be false
I'll talk about what this means later


Fractal Design
2007-08-16T11:11:00.000-07:00
Well its already been longer than I would have liked. I'm working on a library for ruby, which adds extensions for what I believe will be a new programming paradigm. I'd share the details but as I talk about it I discover that I really can't explain it all that well. It makes perfect sense in my head, I swear :) . It aims, among other things, at making threading more enjoyable in general, and builds on OOP.

I do love working in ruby though. It has its limits, but everything is so simply defined. I find that the quality of any tool is usually inversely proportional to the number of concepts involved in its design. This is why I tend to cringe and run away when I hear people using a lot of words I don't know in regards to some software project. Its a sure sign that something new and "Enterprise Ready" is being discussed, and that that thing is way more complicated than it should be.

In ruby there are objects. You call methods on them. That's really about it. The rest is syntax, and as you dig deeper and deeper, the astounding thing is how little there is that makes the real "infrastructure" pieces of the language special. Classes are objects with a method that instantiates. Methods are objects with a method that calls. There's an almost fractal quality to it. No matter how far down you go, you get more of the same. Objects bearing methods.

OOP purists don't mark this as major innovation. This is the way these things were supposed to be built from the start. However there is something to be said for a living implementation, rather than one like java which uses OOP largely as another tool to abuse the programmer, or Python, which bails on OOP as soon as original thinking would be involved in holding the line (and in other places, such as len() which has been happily implemented as a method for years, and doesn't make much sense as some external procedure / heavenly oracle of all things sequential). This is a paradigm that has rarely been properly implemented. Maybe even less frequently than lambda programming if you want to be stringent.

This new project, in addition to parallell concerns, hopes to demonstrate a paradigm as fit for programming in the large as in the small. Rather than invent something totally new like BPEL or some new form of middleware, it adds some very simple features to OOP which let it deal with the larger task in much the way it dealt with the smaller ones. The pattern is fractal, again. The whole is designed just like the parts.


Why Blog?
2007-08-09T19:29:00.000-07:00
I've resisted the urge to start a blog for some time. Or rather I've resisted the peer pressure. The urge I could not, despite my greatest efforts, manage to muster. Perhaps the innate sense of always being correct precluded any need I might have to actually express my opinion. Maybe I found myself neglecting many of the countless other tasks I'm supposed to be doing regularly.

Or maybe it was the urge to hurt people.

Joking about bad technology is one thing, but when the real moment comes, the actual, physical moment when you realize that you could actually do physical harm to someone simply because of how bad Java sucks, the world becomes a different place. You find yourself alone on the precipice of hell, while the million voices of the damned cackle wildly at your pale, soot-stained nude frame.

In this blog, I will teach my wrath to the perpetrators of incorrect implementation. I will present vile injustices toward the dignity of computing systems. A word of warning. I will curse. I will, much like Ralphie's father, "weave a tapestry of obscenity in the stars." Indeed it is a flacid compensation for the inability to cause code to produce the squishing sound of a fist hitting flesh.

What is the purpose of this blog? To HATE! To hate java, to hate C++, to hate what OSX did to FHS. To hate Oracle, to hate MSSQL, to hate php, to hate Microsoft, if there's really nothing better to do. To hate that which is poorly designed and poorly built. I have much hate. And I am going to share with you.